Feature - Implemented Trendy Support

[GValko]

Down with Way of the Surprising Fist. New challenge run is out.

It's called Trendy and it's going to play havoc with the modifier maximizer. Basically, you can equip or use anything that was available for a limited time in the past year. (http://www.kingdomofloathing.com/typeii.php).

Yes it's going to rotate, and it's going to lock off old items as time progresses.

I'm not really too sure what would be the best way to keep a restriction list up to date.

 

[Fred Nefler]

There's a list in-game. Just look at the profile of anyone in the path. Comes from typeii.php

 

[Bale]

If you're logged into KoL through Mafia, copy/paste the following link into your browser: http://127.0.0.1:60080/typeii.php

KoL refuses it as a link, so copy/paste is necessary.

 

[slyz]

I think Mafia should scrape that page at the start of a session, if you are in a Trendy run, and create a list of all the expired items/skills/familiars/bookshelf skills/clan items/campground items.

Then those should be removed from the item/familiar/skill lists, and the appropriate requests should check the lists of expired items before submitting to KoL.

 

[GValko]

Right now a major problem is going to come in the familiar tracking. If you try to put a familiar that's out of style, you get the message "[Familiars is way too old. Suck it.]" (sans quotes) and the familiar won't be switched. Mafia doesn't know that it can't be switched...and then thinks it has been.

 

[slyz]

The same goes for equipment I guess.

 

[Veracity]

I expect that the table in type-ii.php is auto-generated. As in so many other places where KoL programatically generates HTML, it generates bad HTML and expects the Browser to sort it out.

I've extracted the key things to look at in order to parse that table:

<table>
  <tr>
    <th>Last Month</th>
    <th>Category</th>
    <th>Items</th>
  </tr>
  <tr class="expired">
    <td nowrap valign="top">2004-12</td>
    <td valign="top">Items</td>
    <td valign="top">Crimbo pressie, wrapping paper		</tr>
  <tr class="soon">
    <td nowrap valign="top">2011-11</td>
    <td valign="top">Campground</td>
    <td valign="top">Grumpy Bumpkin's Seed Catalog		</tr>
  <tr class="">
    <td nowrap valign="top">2011-12</td>
    <td valign="top">Clan Item</td>
    <td valign="top">Fax Machine		</tr>
</table>

The rows are tagged as being "expired", "soon", or "". If it is "expired", the item is currently unavailable.
The first datum in a row is the last month the item is available. For KoLmafia's purposes, we don't care; the row being "expired" tells us all we need to know.
The second datum can be "Items", "Campground", Bookshelf", "Familiars", "Skills", "Clan Item".
The third datum in a row is a comma separated list of items, with extra whitespace - and no closing tag for the datum.

I propose the following:

1) If we detect that your path is "Trendy", parse this table. Generate six boolean hash tables - Items, Campground, and so on - with string keys and value of "true" or "false", depending on whether the thing is currently available.
2) Define a function - boolean isTrendy( String type, String thing ) - which looks up thing in the appropriate table and returns true or false. If it is not in the table, return true. For that matter, if you are not Trendy, skip the lookup and return true
3) And, all over KoLmafia, filter items, skills, bookshelf summons, familiars, and so on, as needed.

I don't have a lot of time right now, but I think doing 1 & 2 would be easy enough and fun, and would allow others to do aspects of 3 at their leisure. I'll see what I can do, this afternoon, although I'm not actually going to ascend again until December.

 

[slyz]

It looks like implementation should be easier than what I expected:

- You can't pull items that or not trendy (so no need to filter those out, we simply need to recognize that the pull failed)
- Skills you don't have access to are not listed on your Character Sheet (so mafia doesn't even know you have them)
- We will have to wait until next month to see if campground items (ie, the pumpkin patch) simply doesn't appear either
- Clan items are not a problem, it is as if your clan didn't have them installed (you do get a "Huh? Unknown response." message which could be confusing)
- Mafia thinks you have Bookshelf skills, so we need to recognize they are too old when parsing that page, and not include them.

You have a Tome of Sticker Summoning.<p>Unfortunately, it's <i>far</i> too old to use.</p>

- Familiars still appear in your terrarium, but the untrendy ones can be skipped because they have:

<tr class="expired">

EDIT: it looks like we won't need to parse typeii.php

 

[Veracity]

- You can't pull items that or not trendy (so no need to filter those out, we simply need to recognize that the pull failed)

Yes, we should detect all "non-trendy" failures and not update our internal state incorrectly.

- Clan items are not a problem, it is as if your clan didn't have them installed (you do get a "Huh? Unknown response." message which could be confusing)

Like this.

- Mafia thinks you have Bookshelf skills, so we need to recognize they are too old when parsing that page, and not include them.

You have a Tome of Sticker Summoning.<p>Unfortunately, it's <i>far</i> too old to use.</p>

It is good to filter known skills.

- Familiars still appear in your terrarium, but the untrendy ones con be skipped because they have:

<tr class="expired">

It is good to filter available familiars.

What happens if you use a "familiar" command - or a visit_url - to try to use an unTrendy familiar? (Or, for that matter, a familiar you don't have?) What's the failure message?

EDIT: it looks like we won't need to parse typeii.php

If you can prevent KoLmafia from making un-Trendy server hits as part of its normal operation, rather than depending on detecting failure messages, I agree. We obviously still have to detect failures, since scripts can visit_url any old un-Trendy thing they want.

 

[GValko]

Yes, we should detect all "non-trendy" failures and not update our internal state incorrectly.
What happens if you use a "familiar" command - or a visit_url - to try to use an unTrendy familiar? (Or, for that matter, a familiar you don't have?) What's the failure message?

I had already posted this, but using the CLI command will make Mafia think you took it out...but it won't be taken out. You'll still have the prior familiar.

If you do it serverside, you get the message "[Familiars is way too old. Suck it.]" (sans quotes). No need to use visit_url, the radio buttons are still there for expired familiars.

 

[slyz]

I modified the REGISTER_PATTERN in FamiliarData.java, so lines with <tr class="expired"> are skipped. This means that Mafia doesn't know that you have untrendy familiars: they aren't listed in the Gear Changer, or when you click the familiar image in the GUI.

The "familiar" CLI command doesn't try to submit a request if Mafia thinks you don't have the familiar. Same thing when trying to take out an expired familiar from the relay browser: FamiliarRequest.registerRequest() simply returns if Mafia thinks you don't have that familiar.

 

[Veracity]

I modified the REGISTER_PATTERN in FamiliarData.java, so lines with <tr class="expired"> are skipped. This means that Mafia doesn't know that you have untrendy familiars: they aren't listed in the Gear Changer, or when you click the familiar image in the GUI.

This is the incorrect solution, for a variety of reasons.

- You still own the familiar. You can give it familiar equipment, for example. You just can't equip it.
- When you free the king, path restrictions are dropped. Do we re-read the Terrarium at that time? Nope.
- We filter out familiars in Beecore via FamiliarData.canEquip.

 

[slyz]

r10002 adds initial support for "Trendy". "expired" familiar and bookshelf skills are now disregarded when parsing familiar.php or campgroud.php.

I guess we should call KoLMafia.refreshSession() after breaking the prism, if we are on the "Trendy" path, so that mafia can update the list of skills and familiars.

- It looks like skills and familiars aren't a problem anymore.
- Mafia will still issue requests to KoL for Clan Items that are expired. The result will be the same as when the player is in a clan which doesn't have the Clan Item, so nothing much has changed here except this kind of pointless server hit might happen more frequently.
- I guess the pumpkin patch won't appear in campground.php, but we will have to wait until next month to do something about it.

The only remaining problem is untrendy items. I guess that pulling untrendy items won't fool Mafia, since we won't have the "item (1) moved from storage to inventory" message, but can players who break Ronin still empty Hagnks or buy untrendy items from the mall? If so, we will need to parse typeii.php, build a list of untrendy items, and prevent usage.

I guess we will need to parse typeii.php anyway, to avoid requesting pulls that we know are going to fail.

 

[Theraze]

Additionally, if you have another familiar hatchling for an untrendy familiar on a SC run, mafia will still make the server hit trying to grow the familiar that you already own...

 

[slyz]

This is the incorrect solution, for a variety of reasons.

Oh... I thought it was simple and efficient

- You still own the familiar. You can give it familiar equipment, for example. You just can't equip it.

The untrendy familiars don't appear on KoL's "Equip Thine Familiars Well" dropdown, so I'm not sure we can still equip items on them. I'll try to get a lead necklace to find out.

- When you free the king, path restrictions are dropped. Do we re-read the Terrarium at that time? Nope.

Same thing goes for the skills I guess. Since KoL already removes those from your Character Sheet, some kind of refreshing will be needed anyway.

Additionally, if you have another familiar hatchling for an untrendy familiar on a SC run, mafia will still make the server hit trying to grow the familiar that you already own...

Familiar hatchlings are themselves untrendy items, which apparently you shouldn't be able to pull. I haven't been able to speak with someone in SC yet though, maybe the rule doesn't apply to free pulls.

 

[Veracity]

Revision 10003 adds the TrendyRequest which handles typeii.php. It parses it into 6 internal HashMaps: one for each of "Items", "Campground", Bookshelf", "Familiars", "Skills", "Clan Item".

It provides a single public function to determine trendiness:

boolean isTrendy( final String type, final String key )

"type" must be one of the above categories.

Note that this does not tell you whether you can use the item or not; it does not check whether you are on a Trendy path. It just tells you whether the thing is allowed or not should you happen to be on such a path.

I did not use that in any existing parts of KoLmafia. The above FamiliarData.canEquip() function could use that, in conjunction with slyz's KoLCharacter.inTrendyCore() function, to have the gear Changer and side bar familiar images automatically update correctly.

I did add ASH functions:

boolean is_trendy( item)
boolean is_trendy( familiar )
boolean is_trendy( skill )
boolean is_trendy( string)

to interface with that function. The "skill" version will automatically convert Bookshelf skills to type "Bookshelf" rather than using type "Skill".

I think I'm done. I'll leave it to others to use this, as needed.

 

[slyz]

Thanks. I'll revert the changes in r10002.

 

[Veracity]

Don't revert ALL of them. Just the incorrect ones.

 

[slyz]

I'm trying to use TrendyRequest.isTrendy() in FamiliarData.canEquip(), but it generates a "runnable in event dispatch thread" error:

Runnable in event dispatch thread
class java.lang.Exception: Runnable in event dispatch thread
java.lang.Exception: Runnable in event dispatch thread
	at net.sourceforge.kolmafia.StaticEntity.printStackTrace(StaticEntity.java:339)
	at net.sourceforge.kolmafia.RequestThread.postRequest(RequestThread.java:83)
	at net.sourceforge.kolmafia.request.TrendyRequest.initialize(TrendyRequest.java:73)
	at net.sourceforge.kolmafia.request.TrendyRequest.isTrendy(TrendyRequest.java:90)
	at net.sourceforge.kolmafia.request.TrendyRequest.isTrendy(TrendyRequest.java:98)
	at net.sourceforge.kolmafia.FamiliarData.canEquip(FamiliarData.java:152)

Looking at the fix Hola used in r9981, I tried using RequestThread.runInParallel() instead of RequestThread.postRequest() in TrendyRequest.initialize(), but that didn't work (I just kept getting "Done. Are YOU a fashion plate?" messages in the gCLI).

Should TrendyRequest be initialized earlier on, when we notice the user is in Trendycore?

 

[Veracity]

That sounds safe. And reasonable, since you are almost certainly going to be using its database, if you are Trendy. It's done the way it is now - on demand - since you CAN call is_trendy() even if you are not on the path, but there's no harm in initializing upon login as soon as we detect you are on the path, and leaving the other calls to TrendyRequest.initialize() in place, since they are no-ops if it is already initialized.

Just don't call TrendyRequest.isTrendy() in FamiliarRequest.canEquip() unless you've already checked KoLCharacter.inTrendycore().

(Heh. in MY submit, I initially called it KoLCharacter.isTrendy(), but you submitted first, so I adapted to your name. Yes, we have Beecore and Fistcore as examples, but Trendycore sounds a bit off. No big deal. I wonder what the community will call end up calling it?)