price.ash: learn how much an item costs, courtesy items.ofloathing.org

dj_d

Member
After some discussion on the subject (http://kolmafia.us/index.php/topic,1922.0.html), here's a quick script to grab prices from items.ofloathing.org and the kolwiki. The cool parts are from Zarqon; the embarrassingly incompetent parts are from me.

In any case, this is a small piece of a bigger project but I thought I'd share. Feedback welcome!

<<updated with the latest - see thread below>>
 

Attachments

  • price.ash
    4.7 KB · Views: 152

zarqon

Well-known member
Nice! Didn't know that site existed. I may modify this and use it with my clan StashBot, perhaps as a supplement to my method.

To include a double quote in a string, just escape it with a backslash.

"Hi there!" = "\"Hi there!\""

I'm still a fan of mafia allowing us to access the mall minimum once per item per day. (Saying this again in case any devs hadn't seen the other thread and are now inspired... ha.)

Something like int value( item ) which returns the minimum mall price, once per day per item. This would still prevent mallbots, but would also eliminate the need for complex workaround scripts that hit remote servers or place meat in and out of the closet. These second-hand prices will never be as reliable as actually accessing the mall itself.
 

dj_d

Member
Cool! If you don't get to it first, I'm thinking about a method that grabs both, and compares them to the last recorded price produced by either source (in case they've gone down), plus the last price that the player paid for the item (if available), and makes the best guess based on all those data points. Basically, keep a local cache of expected prices and update it whenever new data is available.

And I agree - Mafia should have some sort of price estimation feature. Once per day lookup, time-delayed results, a central clearinghouse of prices updated periodically (kind of what we're dong here) - something.
 

zarqon

Well-known member
Okay, I took your idea and ran with it. Originally I was just going to clean up the string parsing, but I ended up adding in your "cache the page locally, with a 1-day expiration" idea. In the end, I think none of the code is the same, but it's basically what you were doing.

This scriptlet features a pretty nifty function which I wrote to be transportable:

string visit_daily_url(string url, string filename)

Which works just like visit_url() except it will first check to see if this data is stored in filename. If the cached data is less than 1 day old, it returns the cached data, otherwise it returns the visited url.

This pricing solution works nicely, except like I said before, the prices are still not as accurate as internal pricing would be. Some items are grossly mispriced, for example, Mr. A's (and just about every other Mr. xxxx item) are valued at 100 meat! I can only imagine the complaints I would get if someone donated a Mr. A to my StashBot and they were only given 100 credit for it!

Incidentally, the wiki method (value.ash) values this item at 4794988. Perhaps a script that uses both of them, then goes with whichever is higher would be the best solution at present. (Accessing a total of three other servers to find item values! Whee!)
 

Attachments

  • price.ash
    1.6 KB · Views: 75

dj_d

Member
Thanks for
1) the improvements
2) the suggestions
3) reminding me of how much I've forgotten since CS101. Good thing I don't do this for a living. That quote-mark code was just embarrassing. :)

The URL caching is brilliant. I think the stashbot has issues under load, so this is a great compromise. You can get all the values from items.ofloathing in one whack. Then, I added (your) code to check with stashbot if it finds a value of 100 or 0. The first post has that version now. Further improvements welcome.

This is all going to be very handy for a large chunk of code I've got sitting on my machine that currently spends like a drunken sailor. More on that as soon as I polish it enough for sharing.
 

zarqon

Well-known member
Warning: sometimes it's better to go with 100 rather than a higher value. For example, casino passes are always overvalued in the wiki because of them being sold for high prices in /games or as MMG debt. Their actual value should be 100 meat.

Also, let me clarify: the pricing workaround I shared before references the wiki. It is not StashBot, just a pricing method. It accesses both my own server and the wiki, and I wrote the method as part of my StashBot script.
 

dj_d

Member
Gotcha.

I did a quick survey of mall prices and of return values for both servers, and found that there were MANY more things mispriced at 100 than properly priced at 100, so I think the "not 100" heuristic is reasonable - but suggestions/changes (even just to correct the comments re: your stashbot point) are welcome. Otherwise, I'll try to get to it when I give it another overhaul.
 

Bale

Minion
[quote author=dj_d link=topic=2016.msg9843#msg9843 date=1227264478]
Gotcha.

I did a quick survey of mall prices and of return values for both servers, and found that there were MANY more things mispriced at 100 than properly priced at 100, so I think the "not 100" heuristic is reasonable - but suggestions/changes (even just to correct the comments re: your stashbot point) are welcome. Otherwise, I'll try to get to it when I give it another overhaul.
[/quote]

Here's a simple heuristic to deal with figuring if something is overpriced in the wiki due to reverse raffles or the like. If an item (eg, casino passes) can be bought in an NPC store and the wiki says that it sells for more than twice autosell, then the wiki price is the error. Make a list of items from NPC stores and that'll deal with 80% of the problems.
 

dj_d

Member
Good point Bale.

Zarqon, I realized that our uses for this may be very different - it doesn't particularly matter for my use if the price error is high or low, whereas your stashbot might be exploitable if it errs high. Feel free, of course, to modify to taste.
 

dj_d

Member
Another update, again inspired by Z. This writes out any lookups you do to a file, pricelist.txt, and if you look up the same item in the same day, grabs it from the text file. You might need to create a blank pricelist.txt with notepad in order for it to work, I'm not sure.
 

zarqon

Well-known member
@Bale: your heuristic isn't really very simple! It's a simple concept, but not simple to code. There is no mechanism in mafia for accessing the autosell value of an item.

@dj_d: Yes indeed, any pricing errors by StashBot (high or low) are exploitable. No one in my clan is the exploiting type, but I would rather not present anyone the opportunity. Also, cool idea caching the prices. Not sure if it helps though since the page data is already cached...?

I've incorporated a hybrid pricing system combining every pricing method we've come up with so far into StashBot (including support for Bale's idea of a list of NPC items) and he seems to be pricing things much more accurately now. I've attached a standalone version of the mashup I'm using. The great news is that it seldom needs to fall back on the "buy the item" method of pricing (that's the last resort). The bad news is it still gives grossly incorrect values for Mr. A's.
 

Attachments

  • value.ash
    2.8 KB · Views: 51

dj_d

Member
It's fun collaborating with you, Z. Question - I'm going to integrate more of your stuff in to the version at the top of the post. Can you use notify with two arguments ("notify zarqon dj_d") and if so would you like to be notified from that version? You've obviously put a ton of work in to this and I don't want to hog credit. :)

I'm using this pricing script in a place where it may be called for on hundreds of items, multiple times per day. Your HTML caching fixes the problem for the itemsofloathing lookup, but not for wiki lookups (which only returns one result at a time, as I understand). While I could cache each response from your page as a separate file using your function, that'd get ugly in the filesystem fast.

The second reason is that I'm moving to a more complex system where different lookup types have different lifetimes: amounts from the wiki/itemsofloathing, manually entered amounts, and amounts found through actual purchases in the mall. So if you bought a Mr A from the mall two days ago for 4.3mm, it wouldn't pay attention to itemsofloathing telling you it was worth 3000 meat today. Or you could manually enter prices for key items that were being misvalued.

The class of problem I'm working on, BTW, is shopping on a budget. To do that, you have to get a LOT of prices.
 

dj_d

Member
Just noticed from items.ofloathing:
"Nov 21 Wow! Prices haven't been recording since Nov 1, sorry guys! They should be fixed now."
 

Bale

Minion
[quote author=zarqon link=topic=2016.msg9863#msg9863 date=1227347817]
@Bale: your heuristic isn't really very simple! It's a simple concept, but not simple to code. There is no mechanism in mafia for accessing the autosell value of an item.
[/quote]

As I said, make a list. Just include the prices in the list.
 

asturia

Minion
Is there a way to input your complete itemlist? and write the results to a file?
I want to calculate how much my things are worth.
 

zarqon

Well-known member
@dj_d: Yes, what we've come up with here is far better than what I had before with the wiki solution. And don't worry about notification credit. I don't need to be notified for this particular script, since it is just a small part of one of my larger scripts. I don't think notify works for multiple people anyway. (ps - you might consider using my Registry script to keep track of who uses your scripts... [/plug])

@Bale: Making and especially maintaining an autosell-price list for all tradeable items would be much more of a chore than getting item prices manually, negating the whole point of writing the script. In my case, I need to be able to accurately appraise any tradeable item on the fly, including new items as soon as mafia supports them. It would be better to open mafia's itemlist and read the autosell from there -- but that would make the script huge and clunky and slow.

@asturia: I know for a fact there is a Greasemonkey script floating around that will do this. A little searching would probably turn it up. If you can't find it, I know someone who uses it and I could ask her where she got it. If you don't want to use Greasemonkey and want a mafia solution, you could try adding this function to the value.ash script, then calling it with get_inventory() as the parameter.

Code:
int stack_value(int[item] stack) {
  int result;
  foreach i in stack
   result = result + (stack[i]*get_value(i));
  return result;
}

Note that this will only return the sum total of all your tradeable items in inventory (not storage or closet).

A plea to the devs:

Please consider introducing these functions to ASH:

  • [li]int autosell_price( item ) - returns the autosell price of an item. Current methods for accessing this (creating an independent map of data, or loading mafia's entire item database only for one info point) are hugely inefficient.[/li]
    [li]int mall_price( item ) - returns the current lowest price for an item in the mall once per item per day, to prevent mallbots. This would allow item pricing without buying the item, but it would still be just as difficult as now to manipulate mall prices.[/li]

If there is some reason this cannot or should not be done, at least write us a note explaining why. Then I promise I won't whine about it again.
 
I just wanted to make some comments on this script.
I am Zecious Babaloni, the creator of Items of Loathing.
First off, I'd like to thank dj_d for creating this script, I'm glad to hear that people enjoy and use my site.
I have fixed some bugs the site was having, causing some items to be mispriced, and items with special characters (such as quotes) in their name to not be listed. These bugs should all be fixed now. Please kmail me in-game if they are not.
Also, if you plan to make a script similar to dj's that will be scraping my database, I would prefer that you gather the info from the following automatically updated files:
http://items.ofloathing.org/today.txt - This file lists the items and prices from today only
http://items.ofloathing.org/export.txt - This file shows the past fourteen days worth of data.

I wiped all data back to October 26th today, due to inaccuracies caused by bugs.
export.txt does not show the past 14 records of an item, it shows the past 14 days, meaning that if/when I do a wipe like this, it will not have the full 14 days worth of data in it. Sorry for the convenience.

Thank you, everybody for your support and interest. Please kmail me in-game if there is anything I can do to help you scrape data for your scripts.
 

asturia

Minion
Zargon, thanks for the help.
The greasemonkey script you're talking about is called UpUp.us Price Gun.
The problem is that it doesn't include the latest new items.

When I add your suggestion to the script, it gives me this error when I run it:
Code:
Undefined reference to function 'get_value' (value.ash, line 85)
 

dj_d

Member
Hm... attempts to fix the script aren't working. visit_url returns nothing when I pass it the today.txt URL. I'm stumped. Zarqon?
 

dj_d

Member
New version is ugly beyond belief, but it works. It's designed to scale a little better when multiday returns become available, though, as it can read them all in to a big map.
 
Top