Bug - Fixed Excessive calls to api.php & listener firing while refreshing session

[Veracity]

Two concepts:

- api.php is KoL's equivalent of charpane.php for use by programs, as opposed to humans. KoLmafia explicitly asks for it at the beginning of refreshing your session. Additionally, whenever KoLmafia makes a request of KoL that comes back with a request to refresh the charpane, KoLmafia instead calls api.php.
- The PreferenceListenerRegistry is a way for one part of KoLmafia to request notification when something changes. I believe it is usually used to update GUI elements when the part of the data model that they are hooked to changes. Alternatively, they can be associated with a particular Preference - hence the name - and will be called when that preference changes.

I just added a way to log when we fire listeners. "debug listeners on" and "debug listeners off" will log - the the debug log, which also has to be turned on - whenever we fire a named listener. (It is also possible to fire ALL listeners. I should soup up the debugging to show that, too.)

I started KoLmafia
I opened the gCLI
debug listeners on
debug on
I opened Preferences
I set Daily Deeds to open as a window
I logged in
I opened the gCLI
debug off

Here are all the requests and specific listener firings that were executed during my login:

Requesting: https://www.kingdomofloathing.com/login.php?password&secure=0&loginname=Veracity%2Fq&loggingin=Yup.
Requesting: http://www.kingdomofloathing.com/actionbar.php?action=fetch
Firing 0 listeners for "(character)"
Firing 0 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 0 listeners for "(character)"
Firing 0 listeners for "(location)"
Requesting: http://www.kingdomofloathing.com/topmenu.php
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/charsheet.php
Requesting: http://www.kingdomofloathing.com/desc_item.php?whichitem=809051828
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/api.php?what=inventory&for=KoLmafia
Firing 5 listeners for "(hats)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/closet.php?which=1
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/api.php?what=closet&for=KoLmafia
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/account_manageoutfits.php
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/questlog.php?which=3
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/questlog.php?which=2
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/questlog.php?which=1
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/familiar.php
Firing 10 listeners for "(character)"
Firing 0 listeners for "(familiarLock)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/campground.php?action=inspectdwelling
Requesting: http://www.kingdomofloathing.com/campground.php?action=bookshelf
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/campground.php?action=inspectkitchen
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/storage.php?which=5
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/api.php?what=storage&for=KoLmafia
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/desc_item.php?whichitem=239178788
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/clan_viplounge.php?action=hotdogstand
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/place.php?whichplace=forestvillage&action=fv_friar
Requesting: http://www.kingdomofloathing.com/choice.php?forceoption=0
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Requesting: http://www.kingdomofloathing.com/choice.php?whichchoice=720&option=4
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 10 listeners for "(character)"
Firing 0 listeners for "(location)"
Loading window: AdventureFrame
Firing 14 listeners for "(character)"
Firing 0 listeners for "(location)"
Firing 14 listeners for "(character)"
Firing 0 listeners for "(location)"
Loading window: CommandDisplayFrame
Loading window: MallSearchFrame
Requesting: http://www.kingdomofloathing.com/account_contactlist.php
Requesting: http://www.kingdomofloathing.com/api.php?what=status&for=KoLmafia
Firing 14 listeners for "(character)"
Firing 14 listeners for "(character)"
Firing 14 listeners for "(character)"
Firing 14 listeners for "(character)"
Firing 14 listeners for "(character)"
Firing 14 listeners for "(character)"
Loading window: SkillBuffFrame
Loading window: OptionsFrame
Firing 0 listeners for "scriptList"
Firing 0 listeners for "scriptList"
Loading window: CharSheetFrame

Observations:

- setting state from api.php fires the (character) listener at least 6 times.
- we ask for api.php 12 times - once to get the data from it and 11 more times because KoL specified a charpane refresh - even though none of the calls we made actually changed any character state.
- during login, we fired all of the (character) listeners - 10 to 14 active ones - 105 times

I think we could drastically reduce the processing here:

- create the concept of "deferring" listeners, just like we can "defer" concoction refreshing
deferListeners() - increments an integer
firePreferenceChanged() - if deferred, adds to a HashSet of deferred listeners
fireAllPreferencesChanged() - if deferred, adds all to the HashSet
undeferListeners() - fires all the listeners in the deferred HashSet and clears it
- api.php?what=status defers listeners at the start of processing and undefers at the end - we only need a single call to the (character) listeners, not 6 or more
- refresh session defers at the start and undefers at the end - there is no need to update GUI elements multiple times while the refresh is in progress; they can all be updated at the end
- refresh session explicitly calls api.php, but none of the requests it makes actually change anything that will be reflected in it. Therefore, when GenericRequest notices that a request asks for a charpane request, do NOT make it an api.php request if we are refreshing.

 

[roippi]

Interesting. That's definitely suboptimal. I do like the idea of enqueuing calls to fetch character state, so long as no operations depend on said information being up-to-date.

Do we perhaps want to explicitly tie the "undefer" step to RequestThread.closeRequestSequence? (or somewhere thereabouts)

 

[Veracity]

Revision 13337 (LEEET) does everything I suggested. Things seem to work just fine. I don't know if it is noticeably faster, even though there are 11 fewer intermediate calls to api.php.

I don't think a "request sequence" is relevant to this. From what I recall, that really only affects logging: you submit a series of requests one after the other and when you are done, it logs "Requests finished" or something.

 

[roippi]

It's also the block responsible for seting KoLMafia.continuationState to CONTINUE. That's all I had in mind when you said "defer", in retrospect it would be clunky.

I like the way you implemented this. I feel like this "defer things into a set" motif could be used in other places in mafia to save hits/speed things up.