Feature shop.php support

Intro:

I've been working on aspects of this project ever since Crimbo Town (Island?) 2024 emerged from the mists last mid-December.
I added support for modeling the three shops as coinmasters, rather than mixing methods, took a break to work on CyberRealm, but returned, with a vengeance, to this project.
I've made substantial progress, and things are looking good (to me), but there are still various aspects to be implemented.

For at least 10 years, KoL has been using "shop.php" for all the various NPC shops that players can shop in.
KoLmafia supports 139 of them. The user experience varies among them, but here are the common features, under the hood:

There is a shop name - visible, but not used internally.
There is an internal "shopid" - "whichshop=xxxxx"
Everything is a purchase - "action=buyitem"
Every item you can purchase is on a "row" - "whichrow=ROW"
Every item has from 1 - 5 "costs" - Meat or up to 5 items.
The yield can be more than 1 of an item.
You can purchase multiple of an item - "quantity=XXX"

KoLmafia models shops in three ways:

1) An npcstore - items have a single cost, which is Meat
2) A coinmaster - items have a single cost, which is a currency.
If the currency is a cost, this is a "buy"
If the currency is what you are buying, this is a "sell"
A coinmaster can have multiple currencies, but only one is used per item.
3) A "mixing method" - items can have multiple costs and, most likely, nothing looks like a "currency".

Examples:

1) The General Store sells everything for Meat.
This is an npcstore
2) The Applecalypse Store sells items for Chroner
This is a coinmaster.
3) The Dungeoneer Associate Vending Machine sells items for fat loot tokens.
This is a coinmaster
4) Cosmic Ray's Bazaar sells items for various items, but at most one kind per - including fat loot tokens.
This is a coinmaster
3) The Armory and Leggery sells everything for Meat - unless you have a pulverized Standard reward, in which case you can use it to buy a previous year's Standard reward.
This is an npcstore AND a coinmaster with multiple currencies.
4) The Black Market sells lots of stuff for Meat, except you can buy a Red Zeppelin Ticket for Meat OR a precious diamond - exactly once.
This is an npcstore AND a coinmaster.
5) The Star Chart trades stars, lines, and a star chart for items.
This is a mixing method

Observations:

Row numbers are unique across KoL - each row is associated with a particular shopid.
A shop can sell the same item for Meat OR an item - The Black Market
A shop can sell the same item for multiple different costs - the mini kiwitini from the Kiwi Kwiki Mart
Multiple shops can sell the same item - Vending Machine and Cosmic Ray's Bazaar both sell Legend keys for fat loot tokens.
Shops may require a path (Cosmic Ray), quest progress (Black Market), item (Star Chart), access (Dedigitizer),
Available items may depend on quest progress (Black Market), current inventory (Armory & Leggery), daily limit (Kiwi Kwiki Mart), permanent limit (Terrified Eagle Inn).

The "multiple recipes for the same item" has been an issue for years.
The latest example - the mini kiwitini - brought it to my attention again.
The Primordial Soup Kitchen is another example; there are 3 recipes for each advanced soup - one for each basic soup as an ingredient.

Ways to access shops in the KoL GUI (and scripts):

1) The Mall Search Frame will display user stores, NPC stores, and Coinmasters.
2) "acquire" will purchase from all three sources.
3) "create" will purchase from NPC stores and Coinmasters, and will invoke mixing methods.
4) The Usable panels in the Inventory Manager Frame - Food/Booze/Potions - will "create" for you - and display the required ingredients.
5) "coinmaster buy" and "coinmaster sell" will get items from the specific coinmaster you select.
6) The Coinmasters Frame will show you all available buy and/or sell rows - including ingredients - and let you select which recipe you want to use.

I've been interested in simplifying how KoLmafia implements shop.php stores for a long time.
This came to a head for the Crimbo 24 stores.
They sure looked like they could be coinmasters - with multiple currencies - but the Cafe and Bar each had a single offering which took all five currencies, and the Factory offered items that took either a single currency or two of the five types of currency.
Rather than make them into three different mixing methods, I decided to extend coinmasters to allow items to be purchased by more than one kind of currency.

And so this project began.

Progress so far:

1) There are now two kinds of coinmaster:

The classic model.
- coinmasters.txt lists every item as either a "buy" or a "sell".
- the currency is not specified. If there is only one, the buy/sell cost is for that specific token
- if the coinmaster has more than one currency - Mr. Store, for example - it has to provide methods to calculate which currency goes with which item.

The ShopRow model
- coinmasters.txt lists everything for sale in a format that doesn't refer to "buy" or "sell", but simply lists what (and how many) you get of an item when you trade in 1-5 items (and/or Meat?)

Support for this was implemented for the Crimbo25 shops.

2) All known shops are in shops.php, including how they are currently implemented.


3) All ShopRows are in shoprows.txt - regardless of how we implement them.


(Obviously you don't want to automate using those last two - which is why that are marked MANUAL in concoctions.txt)

4) If multiple coinmasters sell the same item, show all of them in the MallSearchFrame
If the coinmaster is not accessible, grey it out

5) Ditto for NPC stores, except show only stores which are accessible.
(This will show wad of dough available from Knoll Bakery and Hardware Store, Bugbear Bakery, and Madeline, assuming any of them are available to you, given zodiac sign, available outfit, or quest progress).

6) All request logging and response parsing for a shop.php request is now done by ShopRequest, rather than being scattered among numerous CoinMasterRequest subclasses, CreateItemRequest subclasses, and NPCPurchaseRequest.

7) Therefore, CoinMasterRequest subclasses no longer require NPCPurchaseRequest and RequestLogger to have numerous hardcoded checks for specific shopids.
Certain NPC stores still need shopid-specific code in NPCPurchaseRequest

8) I further simplified CoinMasterRequest subclasses. They no longer need to provide constructors, run(), or processResult() methods.
This is as simple as it's going to get. They can still provide class-specific code to check accessibility and handle properties and such.

Future tasks:

1) When we visit shop.php, we parse the available inventory.

If it is a new shop:
- Log the shop.txt entry
- Log all rows in shoprows.txt format
- If transactions use Meat, log rows in npcstores.txt format.
- If there seems to be a single "currency", log in coinmasters.txt "buy" and "sell" format.
- Otherwise, log in coinmaster"row" format

If it is a known shop:
- log new rows in shoprows.txt format
- log new rows in npcstores.txt, concoctions.txt, coinmasters.txt "buy" or "sell" format, or coinmasters.txt "row" format

Net result: exactly what to add to shops.txt, shoprows.txt, and concoctions.txt, or npcstores.txt and/or coinmasters.txt

2) Register Concoctions, PurchaseRequests, whatever, to be able to track transactions on this (new) shop and its (new) rows.
I.e, if you access this shop in the Relay Browser, even though we have no built-in support for it (which will be implemented by previous logging), any transactions you make in the Relay Browser will be logged appropriately in your session log, and Meat and items will be removed, if you make transactions that cause that.

3) What about concoctions / coinmasters that have multiple recipes to make the same item? MallSearchPanel and CoinmastersFrame, no problem. What do we do with Concoctions to allow this - and to allow all the options to appear in the "usable" panels of InventoryManager - Food/Booze/Potions?

4) Ditto for "acquire". Which recipe does it choose?

Comments & suggestions are welcome.
So are bug reports - if any existing Coinmaster/NPC store/concoction is not behaving as you expect, given these changes, report it here.
And as I implement various new features, per my afore-mentioned "future tasks", report bugs here.

Thanks!

This is really fantastic work from which everyone will benefit. Thank you Veracity. Is there an easily sliced area we can contribute codewise?

There are two shops I know about that we don't support.

1) When "using" a rose, white-tulip, red tulip, blue tulip:

The Central Loathing Floral Mercantile Exchange
whichshop=flowertradein

The issue was that the price of tulips changed every half hour.

I'm curious about the row#, but unless/until The Time-Twitching Tower returns, can't really do anything about it.
Wow. That's not actually true. I clicked on the "sell" link that KoL provides for a rose or tulip and was taken to the shop.


But it didn't parse the rows.
Probably because we expect shops to have 5 "costs", even if up to 4 of them are empty.
That's our bug.

I'll submit the responseText for use as a test fixture - test_shop_flowertradein.html

Similar, but different issue - The Crackpot Mystic has 3 (rather than 2) "td" tags per "cost" - test_shop_mystic.html

So, yeah. ShopRow.parseShop takes a responseText and returns a List<ShopRow>.
That could be made more robust.

2) The Gene-Sequencing Laboratory on level 6 of Your Fallout Shelter is a shop.php which trades rads for skills.
It has buttons labeled "Mutate" - and even says "Right-Click to Multi-Mutate". I suspect that doesn't work.

It also looks like "Fiddling With Your Genes" is a shop that trades rads for skills.
FalloutShelterRequest doesn't seem to support that one. I don't know the shopid.

That's a new concept. Looks like I did 13 Nuclear Autumn runs back in 2016, but I don't expect to go back any time soon.

Perhaps you could use your KoL dev powers and fetch a few response texts?

- visiting both of those shops.
- trading rads for a skill in each of them.

I'm not sure what to do about "skill" ShopRows, yet - but not only is there an existing path that needs them, it's obviously a shop.php feature that is bound to be used again.

So, if you want an "easily sliced" way to contribute:

- submit the responseTexts for the 2 "rads for skill" shops - both a visit and a skill purchase.
- take a look at ShopRow.parseShop and consider what it could do for "mystic", "flowertradein" - and the two "skill" shops.
- Do we need a SkillData which extends AdventureResult, much as MonsterData does?

Well here are those HTML files at least. I can take a look at the two problematic shops tomorrow

Thank you! I added those two test fixtures to my open PR which has the test fixture for the flower exchange.

Seems like a new kind of coinmaster - trade currency for skills.
I'll leave that alone, for now.

At the moment, more refactoring. I don't see why CoinmastersDatabase needs a map from row # to ShopRow, when that should be in ShopRowDatabase.

Been specifically having issues with this build and garbo throwing a debug log and not working at all. Doesn't present issues in 28346 or earlier builds.

I’m at Boskone until Sunday afternoon with no computer.

I looked at the DEBUG log and it seems to be in sell_price() in RuntimeLibrary.

I changed a getBuyPrice to itemBuyPrice - which seems like it should be correct, but apparently the latter can return a null?

Perhaps somebody else could look at that change. Sorry.

Hi Veracity. I got this error and debug log with the latest mafia builds.

@Ryo_Sangnoir submitted a PR to fix it (I think!) which has since been merged

I'm getting a minor issue on what I believe is latest main at the time of this writing, which can be trivially reproduced via this line in the gCLI:

ash $coinmaster[Crimbo24 Cafe].sell_price($item[one-day ticket to Dinseylandfill])

(the use case was a function that iterates through all coinmasters to find the one that sells a specific item)

Minor because there is a workaround atm in the form of checking sells_item first, in case anyone else is having this issue and it hasn't been fixed yet.

Checking sells_item first is correct - even for shops that DO sell the item you are looking for.
Look at the Crimbo24Cafe


What is the "sell_price" - an int - for a chocolate ostrich egg? 15, I guess. 15 what? Spirit of Easter.
candied beef and cabbage? Also 15 - but 15 Spirit of St. Patrick's day.
How about the holiday smorgasboard?

All the existing ASH support for coinmasters assumes that coinmasters only have a single currency.
Pre-Crimbo24 exceptions included Cosmic Ray's Bazaar and Mr. Store. And I guess Armory & Leggery, trading pulverize Standard rewards for older Standard rewards. But you are assumed/required to know which "currency" is being spent.

We really should have a "sell_costs" or something which returns item/count for 1 or more items. - however you'd express that in ASH.
In KoLmafia itself, we have AdventureResult, which is a name/id/count thingie.

(That would be a nice built-in record)

I'll fix sell_price so this error does not occur - you will get a zero - but if you want to check if a coinmaster sells an item, you really should call "sells_item".

The error that I ran into (and which I assume fewyn ran into) was in a script that essentially invoked sell_price($item[cyburger].seller, $item[cyburger]).

sells_item unfortunately does not help here, although I think you're spot on with your observation of coinmasters with multiple costs.

Well - that is the Dedigitizer - a coinmaster which deals in 0's, 1's, and multiple dedigitizer schematics.


What is the sell price? 32?

How about this one?


I have a PR open which will make both of those return 0 - since they have multiple costs and we don't have that yet.

Adding this here for completeness:

> ash sell_cost($coinmaster[none],$item[seal tooth])

()
Script execution aborted (java.lang.NullPointerException: Cannot invoke "net.sourceforge.kolmafia.CoinmasterData.getShopRows()" because "data" is null): ()
Returned: void

This needs a error case when coinmaster == none

Items have a .seller entry
As far as I can tell, only 3 items have multiple coinmaster sellers:
Boris's key / Jarlsberg's key / Sneaky Pete's key, which can be bought at both the [Vending Machine] and [Cosmic Ray's Bazaar]

Currently, $item[Boris's key].seller == $coinmaster[Cosmic Ray's Bazaar]

Given that there are only 3 items where this seems to be true, I think we can work around that as a special case, but it might be an issue for other things in the future.