Adaptive learning of items and effects

Veracity

Developer
Staff member
I've spent a lot of hours over the last five days adding a feature that I've had on my list for a long time: recognizing items and effects that are not in the internal database and figuring out what they do by parsing the description text. Obviously, full support for many items requires more than that: we can't tell what will grow from a familiar hatchling, many item enchantments have hidden formulas, rather than hardcoded constants, "usable" items might or might not be multi-usable (we assume unknown "potions" are multiusable and others are not), and so on. But, this should allow you to use previously unknown equipment in the Gear Changer, autosell new things, get an idea what modifiers they are giving you at the moment, and so on, without the devs having to scramble to enter new things in the database. In fact, all those KoLmafia users who stick with major releases rather than using daily builds will feel even less pressure to upgrade to the latest, possibly buggy, release.

In order to enter an item or effect into KoLmafia's internal database, we need two things: the item or effect id, and the corresponding description id.

There are many places where we are presented with both an item id and the corresponding description id:

- Finding a new item while adventuring
- Searching the mall
- Refreshing your closet
- Looking at inventory (with the exception of equipped items: we can see the description id, but not the item id for such items).
- Looking at storage
- Looking at your clan stash

For all of the above, upon seeing an unknown item, we will look at the description and parse it and enter the item into our internal tables. With the exception of the first - items found adventuring - we will then write override files: tradeitems, itemdescs, and possibly equipment and modifiers.

There are a few places where we can see item ids but no description ids:

- Managing your own store
- Managing your display case
- Perhaps, although I have not tested, inventory with inventory images turned off. I should test this; KoL might still let you click on the item name, say, to get the description.

With the exception of the last one, you will have to have inserted the item into the store or display case outside of KoLmafia for us to have not noticed it in inventory. So, big deal.

There are various places where you can click on items and look at them (using the description id) without having access to the item id. Other people's equipment and display cases, for example.

Effects are a little trickier. About the only place which shows you effect IDs is when you try to "use" a soft green echo eyedrop antidote and are presented with a list of effects it can remove. Unfortunately, although that has little effect pictures, they are not clickable to get the description. We have to look at the charpane to get description ids.

The other half of this change is, as I mentioned, automatically writing override files in your data directory. I went to a lot of effort to ensure that we can write full replacements for the built-in data files; that we can read the files into internal data structures and write files from the internal data structures that are complete functional equivalents to those that were read.

However, you don't want to be stuck using a stale override file if you upgrade to a new build which has more recent data. To cut down on the possibility of that happening, we do the following:

- When we load a data file from the data directory, log it in the gCLI
- If the file in the data directory is stale - older than the one built-in to KoLmafia - don't load it, and log that fact in the gCLI
- When we write an override file to the data directory, log it in the gCLI

This was a big project. It is possible that bugs remain (no!) - beyond the ones that you-all have reported within minutes of one of my submits going in. Please continue to report them.

Comments and suggestions are welcome as well. Thanks.
 

Bale

Minion
I've been watching the commit logs in amazement as you built this up.

I cannot wait to see it in action when we get some new content. Hopefully we'll soon see the nemesis quest, final Sea update or another traveling trader visit.

Mafia has needed this for soooo long. Thank you so much.
 

Veracity

Developer
Staff member
Yes.

I refetched all the item descriptions and looked at the format. Potions now have an effect by name and descid and a Duration built into the item description. It means that we could give "default actions" to effects - if we can get the effect id, somehow...
 

Spiny

Member
Somehow I missed the trivial update yesterday... in any event I checked it out just now and I would say if info is being gleaned from clicking on potions to use caution when considering items that have multiple effects. For example Marzipan Skull. Clicking on it now shows it offers 5 turns of Sugar Rush, but there are definitely other effects associated with the item: Hardly Poisoned at All and Hombre Muerto Caminando.
 

Bale

Minion
That's a good point. I find it troublesome that each item only shows one of its effects and not necessarily the most interesting effect. The marzipan skull is a good example of that. Good 'n' Slimy is another.
 

Veracity

Developer
Staff member
We can't learn effects from item description, yet, since they only have the effect desc id, not the effect id. I have submitted a feature request to KoL (ha ha ha ha ha ha ha) to have item/effect description text include the item/effect id in an HTML comment. If they do that, we will be able to learn about new effects from item descriptions.

The fact that only one effect is listed - and not necessarily the most interesting one - is interesting, but fairly irrelevant to KoLmafia's operation. For example, if the marzipan skull was a new item and Hombre Muerto Caminando was a new effect it granted, we could make the default action to acquire that effect be "use 1 marzipan skull". If the item description does not tell us about the effect, we obviously wouldn't think to do that. And if we discovered the effect elsewhere, we wouldn't necessarily know where it came from.

Given that, I don't understand why you suggest "caution". For the player, I assume, not KoLmafia. Can you elaborate, please?
 

Spiny

Member
Well the cautionary comment certainly would affect users and as far as the mafia aspect... I guess there is no caution ;) I clearly misunderstood what was occurring. I meant caution from the perspective of if a new item came out and mafia learned on it's own what the item did based on what the desc blip said (as the player views it) and only saw one effect (as in the example of the marzipan skull), rather than 3 effects... yadda yadda... some things could be overlooked that way, but since it currently cannot be learned from in that fashion, don't mind me. All of this is a bit over my head anyhow... I was just trying to be helpful in my own stick my foot in it sort of fashion ;) lol
 

Bale

Minion
Could mafia learn about what items cause effects if the item is used? Then check the charpane and SGEEA for details? That way when I got a new potion I could simply use it to let mafia figure out how to maintain the new effect.
 

Veracity

Developer
Staff member
I just looked at the html you get when "You acquire an effect". It includes a clickable image - hence, an effect desc id - and the effect name. The only thing it is missing is the effect id.

We could notice that we don't have a "default action" for gaining the effect and, if you got the effect via item use, set it accordingly.

I'm hesitant about looking at the sgeea, since you possibly won't have any. Definitely not, if you are less than level 10. That's why I'd like KoL itself to give me the effect id in the effect description. Alternatively, they could provide me a "rel string", as they do for items when "You acquire: <an item>", except that would require changing the actual code on the servers, rather than the data files, and that seems a bit much to ask.
 

Bale

Minion
It's great being able to search for a colorful plastic ball in the mall even though it hasn't been added to mafia yet, but I'm having so much of a ball that it seems like too much...
Code:
Verifying ingredients for shot of grapefruit schnapps (1)...
Purchasing fermenting powder (1 @ 70)...
You acquire an item: fermenting powder
You spent 70 Meat
[COLOR="Red"][B]Unknown effect found: Having a Ball![/B][/COLOR]
Purchases complete.
Purchasing grapefruit (1 @ 70)...
You acquire an item: grapefruit
You spent 70 Meat
[COLOR="#ff0000"][B]Unknown effect found: Having a Ball![/B][/COLOR]
Purchases complete.
Creating shot of grapefruit schnapps (1)...
You acquire an item: shot of grapefruit schnapps
Successfully created shot of grapefruit schnapps (1)
Drinking 1 shot of grapefruit schnapps...
You gain 3 Adventures
You gain 5 Enchantedness
You gain 1 Drunkenness
[COLOR="#ff0000"][B]Unknown effect found: Having a Ball![/B][/COLOR]
Finished drinking 1 shot of grapefruit schnapps.
I know I'm having a ball, but please stop telling me that!!
 

willock

New member
How about a way to keep track of which discoveries your character knows, and which ones Mafia has in its db?

Be useful to be able to use Mafia to fill the discoveries without overlap
 

willock

New member
Not bad, but it only reduces the checking part of looking at the wiki etc

If it was part of the creation in Mafia, marking them as red if no discovery it would show up missing ones easily. But I have no idea how hard that would be to implement
 
Top