What is this and why do I care?
KoL's responsiveness - "lag", if you will - varies wildly, and it's not evenly spread; you can log in and have miserable lag, log out and back in again, and suffer much less lag. It is almost as if KoL's servers are not handling equal numbers of users: some are sharing their CPU with many more users than others and thereby are far less responsive.
You are not imagining this. According to gausie, at the moment, KoL has 5 servers, and 2 of them tend to operate at 100% CPU and 3 of them at 40% CPU.
Anecdotally (personally), it can be the difference in automating daily turns for a single character in 15 minutes vs. 2 hours. For at least a year, if I detect I am on a "slow" server, I log out and back in, until I get a "fast" server.
I decided to automate this process and make it available to the entire KoLmafia community.
Be aware that KoL also understands this issue and intends to change how they serve requests, such that every incoming request is sent to the currently "least loaded" server. This will also allow them to auto-scale at Crimbo, or other periods of extra demand. I approve of this.
They attempted this last week and KoL crashed and burned; their software infrastructure simply could not handle it. They've rolled back the change and will continue with their re-engineering effort on private dev servers, rather than the user-facing servers, but change IS coming.
In the mean-time, we likely have several months where there are "fast" vs. "slow" servers, and it is worth trying to get onto a "fast" one.
What is "ping testing"?
It is a term of art in Computer Networking for at least the last 40 years.
A computer sends a "ping" - a small request that induces a reply - to another computer on a network and measures the round-trip time. There are five aspects to the measured time:
Technical explanation. Feel free to skip on by!
1) processing time on the source computer to inject the message into the network
1a) your process to the operating system
1b) operating system to the network
2) network time to reach the destination computer
3) processing time on the destination computer to receive the message, handle it, and inject the response
3a) network to operating system
3b) operating system to server process
3c) server process
3d) server process to operating system
3e) operating system to network
4) network time to return to the source computer
5) processing time on the source computer to receive the message
5a) network to operating system
5b) operating system to your process
When measuring the speed of a network, ideally the transaction consists of a single packet in each direction.
We can use the same concept for testing server load, although there will be multiple messages exchanged to open connections.
That will be reflected in steps 1b-3a and 3e-5a.
The part that we care about is steps 3b-3d: delivering a request to a KoL process, handling it, and receiving a response from said process.
Individual "pings" can vary widely, but we expect that if your and KoL's computers aren't changing network connectivity, all the actual network stuff will average out and the prime variance will result from the load on the server - how quickly KoL can perform steps 3b-3d.
My eyes glazed over. How does KoLmafia use "ping testing"?
New feature.
If you opt in to "ping testing" (the default), KoLmafia will do a single ping test immediately after you log in. Unless you opt in to further processing, it will save the result in GLOBAL preferences, to record the maximum and minimum average ping time you've seen on the particular network your computer is connected to.
If you further opt-in - and configure - checking your current ping speed against historical times, KoLmafia will detect that you are on a "slow" server and will re-login to get assigned to a different server. This will run another ping test, and - as configured - KoLmafia will continue testing and re-logging in.
The expected result is that after comparatively few retries, you'll get a fast server and KoLmafia will finish logging in and refreshing your session. Keep in mind that all this ping testing and re-logging in is requesting nothing but login.php and whichever request you are using to do the ping.
You said "configure". Configure what?
You have to enable ping testing. That is on by default.
You choose which request to ping with. The ideal request is fast, can be issued at any time, once you are logged in, and does not vary widely based on user state, since we want to be able to compare ping times to historical pings when your character may have been in a very different state. The default is "api.php", although you have 5 options.
You choose how many pings to do in a test. As mentioned, ping time can vary widely, even to the same server, so more pings make anomalously slow or fast pings have less of an effect on the average. The default is 10 pings - which should be less than a second, for api.php.
You choose how to decide if a ping test is "good". There is no default for this.
You choose what to do if ping testing fails after that many tests.
Simplest options for people who do not want to experiment and configure:
Run ping test at login to measure connection lag - CHECKED
Kol Page to ping - api
How many times to ping that page - 10
Login ping check type - threshold
Allowed threshold above minimum historical lag - 0.2
Attempted ping checks before giving up - 5
Action after ping check failure - confirm
Where can I configure these options?
On the LoginFrame, go to the Connection tab
Once you are logged in go the Preferences/General/Connection Options
The Preferences panel has additional options to actually perform ping tests with various configurations - the aforementioned "experimentation" - and to "Time In" - i.e., try for a better connection.
You can also "experiment" in the gCLI with the "ping" command and the "timein" (or "relogin" or "relog") command, but you'll have to go to Preferences to change the configuration.
I guess the only thing that isn't obvious is the choice of page to ping. What are the options?
As used by the "page" argument to the ping function - and listed on the configuration panels:
api - api.php, no arguments. This essentially returns the "help" page for that request. It is fast, can be submitted anywhere, and does not require user data, so it is constant time to execute - modulo server load, which is exactly what we are trying to measure.
(event) - api.php?what=event&for=KoLmafia. This returns the current "events" for your user. It's a little slower, and it can be submitted everywhere, but it does have user data. Which is to say, if you have multiple characters, the times are not directly comparable. In fact, it can dynamically change for a given user - even while running tests - since events are externally generated.
(status) - api.php?what=status&for=KoLmafia. This returns the current "status" for your user. It is slower still, can be submitted anywhere, has user data - which will vary by user and CAN vary for the same user (active effects, for example) - but it will not change while you are running a test, at least.
council - council.php. Lets the council tell you whatever it wants. Even slower. Obviously, can vary between users and within the same user, but should be identical in aftercore, at least.
main - main.php. The main map and events. Even slower. Varies between user and dynamically changes as events arrive..
What if I'd rather just use my own script? Do I HAVE to use this?
Of course not. Do whatever sinks your boat!
You can use the "ping" function in either ASH or JS to do ping tests that qualify for inclusion in KoLmafia's historical ping data.
You can use the "timein"/"relogin"/"relog" command to try for a better server.
There are properties, documented elsewhere, for the various configuration options, and the latest (and historical) ping test.
Try "prefref ping".
KoL's responsiveness - "lag", if you will - varies wildly, and it's not evenly spread; you can log in and have miserable lag, log out and back in again, and suffer much less lag. It is almost as if KoL's servers are not handling equal numbers of users: some are sharing their CPU with many more users than others and thereby are far less responsive.
You are not imagining this. According to gausie, at the moment, KoL has 5 servers, and 2 of them tend to operate at 100% CPU and 3 of them at 40% CPU.
Anecdotally (personally), it can be the difference in automating daily turns for a single character in 15 minutes vs. 2 hours. For at least a year, if I detect I am on a "slow" server, I log out and back in, until I get a "fast" server.
I decided to automate this process and make it available to the entire KoLmafia community.
Be aware that KoL also understands this issue and intends to change how they serve requests, such that every incoming request is sent to the currently "least loaded" server. This will also allow them to auto-scale at Crimbo, or other periods of extra demand. I approve of this.
They attempted this last week and KoL crashed and burned; their software infrastructure simply could not handle it. They've rolled back the change and will continue with their re-engineering effort on private dev servers, rather than the user-facing servers, but change IS coming.
In the mean-time, we likely have several months where there are "fast" vs. "slow" servers, and it is worth trying to get onto a "fast" one.
What is "ping testing"?
It is a term of art in Computer Networking for at least the last 40 years.
A computer sends a "ping" - a small request that induces a reply - to another computer on a network and measures the round-trip time. There are five aspects to the measured time:
Technical explanation. Feel free to skip on by!
1) processing time on the source computer to inject the message into the network
1a) your process to the operating system
1b) operating system to the network
2) network time to reach the destination computer
3) processing time on the destination computer to receive the message, handle it, and inject the response
3a) network to operating system
3b) operating system to server process
3c) server process
3d) server process to operating system
3e) operating system to network
4) network time to return to the source computer
5) processing time on the source computer to receive the message
5a) network to operating system
5b) operating system to your process
When measuring the speed of a network, ideally the transaction consists of a single packet in each direction.
We can use the same concept for testing server load, although there will be multiple messages exchanged to open connections.
That will be reflected in steps 1b-3a and 3e-5a.
The part that we care about is steps 3b-3d: delivering a request to a KoL process, handling it, and receiving a response from said process.
Individual "pings" can vary widely, but we expect that if your and KoL's computers aren't changing network connectivity, all the actual network stuff will average out and the prime variance will result from the load on the server - how quickly KoL can perform steps 3b-3d.
My eyes glazed over. How does KoLmafia use "ping testing"?
New feature.
If you opt in to "ping testing" (the default), KoLmafia will do a single ping test immediately after you log in. Unless you opt in to further processing, it will save the result in GLOBAL preferences, to record the maximum and minimum average ping time you've seen on the particular network your computer is connected to.
If you further opt-in - and configure - checking your current ping speed against historical times, KoLmafia will detect that you are on a "slow" server and will re-login to get assigned to a different server. This will run another ping test, and - as configured - KoLmafia will continue testing and re-logging in.
The expected result is that after comparatively few retries, you'll get a fast server and KoLmafia will finish logging in and refreshing your session. Keep in mind that all this ping testing and re-logging in is requesting nothing but login.php and whichever request you are using to do the ping.
You said "configure". Configure what?
You have to enable ping testing. That is on by default.
You choose which request to ping with. The ideal request is fast, can be issued at any time, once you are logged in, and does not vary widely based on user state, since we want to be able to compare ping times to historical pings when your character may have been in a very different state. The default is "api.php", although you have 5 options.
You choose how many pings to do in a test. As mentioned, ping time can vary widely, even to the same server, so more pings make anomalously slow or fast pings have less of an effect on the average. The default is 10 pings - which should be less than a second, for api.php.
You choose how to decide if a ping test is "good". There is no default for this.
- If you are willing to do some experimentation - use "ping" commands on "good" or "bad" servers and see what average times are - you can set a "goal" to require such-and-such a time. Note that this is realistic only if your computer is immobile - always on the same network.
- If you don't want to bother experimenting and are willing to let KoLmafia attempt to get a connection which is "close" to the best historical ping it has observed, you can set a "threshold" - how much worse (percentage-wise) than the best historical ping is still acceptable.
- And if your computer is movable - you might take it to a coffee shop or whatever and log into a different network - you'll want to use a "threshhold" - and clear your historical ping times when you change networks, to let KoLmafia learn what the new normal is.
You choose what to do if ping testing fails after that many tests.
- You can "logout" and give up in despair.
- You can "login" - i.e. simply accept what the last test gave you. (Note that if you look at the gCLI, you will see the previous test results, and the last one might very well not be the best. You can "timein" or "relogin" to try again. It will run through the whole sequence of tests, as configured above. Of course, you can change your configuration before retrying.)
- You can "confirm" - get a dialog explaining what happened and what you're ending up with, and pointing you to configuration. If you have a "threshold", you will also have the option to immediately clear your historical data - as is appropriate, if you changed networks. You will end up logged in, but can "timein" or "relogin", as above.
Simplest options for people who do not want to experiment and configure:
Run ping test at login to measure connection lag - CHECKED
Kol Page to ping - api
How many times to ping that page - 10
Login ping check type - threshold
Allowed threshold above minimum historical lag - 0.2
Attempted ping checks before giving up - 5
Action after ping check failure - confirm
Where can I configure these options?
On the LoginFrame, go to the Connection tab
Once you are logged in go the Preferences/General/Connection Options
The Preferences panel has additional options to actually perform ping tests with various configurations - the aforementioned "experimentation" - and to "Time In" - i.e., try for a better connection.
You can also "experiment" in the gCLI with the "ping" command and the "timein" (or "relogin" or "relog") command, but you'll have to go to Preferences to change the configuration.
I guess the only thing that isn't obvious is the choice of page to ping. What are the options?
As used by the "page" argument to the ping function - and listed on the configuration panels:
api - api.php, no arguments. This essentially returns the "help" page for that request. It is fast, can be submitted anywhere, and does not require user data, so it is constant time to execute - modulo server load, which is exactly what we are trying to measure.
(event) - api.php?what=event&for=KoLmafia. This returns the current "events" for your user. It's a little slower, and it can be submitted everywhere, but it does have user data. Which is to say, if you have multiple characters, the times are not directly comparable. In fact, it can dynamically change for a given user - even while running tests - since events are externally generated.
(status) - api.php?what=status&for=KoLmafia. This returns the current "status" for your user. It is slower still, can be submitted anywhere, has user data - which will vary by user and CAN vary for the same user (active effects, for example) - but it will not change while you are running a test, at least.
council - council.php. Lets the council tell you whatever it wants. Even slower. Obviously, can vary between users and within the same user, but should be identical in aftercore, at least.
main - main.php. The main map and events. Even slower. Varies between user and dynamically changes as events arrive..
What if I'd rather just use my own script? Do I HAVE to use this?
Of course not. Do whatever sinks your boat!
You can use the "ping" function in either ASH or JS to do ping tests that qualify for inclusion in KoLmafia's historical ping data.
You can use the "timein"/"relogin"/"relog" command to try for a better server.
There are properties, documented elsewhere, for the various configuration options, and the latest (and historical) ping test.
Try "prefref ping".