EatDrink.ash: Optimize your daily diet (and see how your old diet stacks up).

xKiv

Active member
Ow. So you are already 5 times slower than I thought ... and you are doing that because sometimes increasing VoA makes you switch from good solution to worse solution? Maybe going for actually optimal solution would be cleaner then ... fewer unreliable workarounds, sometimes better results, and not even much slower ... (I bounded my solution to be at most 35 times slower than current ED in the worst case ... for 35 hunger, but then you are 5 times slower than I thought, and we are talking inly 15 hunger, so I am only 3 times slower now (probably even better than that, with all the super-pessimistic bounding I did on those sums)).

Except that I have no idea how to make mine work nicely with limited consumables.
You would have to slow it down by considering more than 1 type of food for each size (how many will depend on how many are limited and what the limits are - for example, if your hunger is 35 and you have max 5 of the best size 1 food, and 10 of the next best size food and the next best is unlimited, you consider only those 3 types ... but if you had 30 of the second best, you would only consider 2 types). That's what you would keep in the shortlists for each food size, adn then you couldn't have just one best pick for each size after that, and you would have to do some ugly checks to see if the stack under consideration is still within limits, and you would have to remember all possible combinations for each size anyway just to be sure that you will not overlook anything at bigger sizes ... yeah, no.
 

lostcalpolydude

Developer
Staff member
Eleron's site has all the results precomputed and simply retrieves the result when you fill in your parameters, I believe. I remember him saying that the calculations would be slow and can't be done right when someone enters in their values.
 

heeheehee

Developer
Staff member
While I was aware that Knapsack and IP were both NP complete which makes them in some sense algorithmically equivalent, I was not really thinking that the problems themselves were equivalent. If, indeed, I can convince myself that the constrained optimization problem I want to solve can be mapped into a knapsack problem (and I suspect it can and that I'm just not trying hard enough to do so) then we should reactivate our collaboration. I'm all in favor of something that can be described with mathematical rigor and knapsack plus widgets may very well be easier to build and maintain than a full blown IP. To be continued.

The whole point of NP-completeness is that all NP-complete problems are actually solving the same problem -- that is, search. It doesn't really matter if IP or knapsack is used.

(... can't ... resist ... biting ...)
Of course, if you pre-plan your diet with integer programming/knapsack, you might run into problems when enough of the foods are not available at the same price (which mafia won't let you know until after you acquire some of them), so you have to stop and recompute ...

Also, you might want to take a look at eleron's javascript at http://www.houeland.com/kol/diets, that thing is regarded as very good. And is very fast (in part because it has pre-cached item prices).

...

As for dynamic programming ...
[ snip ]
Of course this is wrong, because it doesn't consider many real-world factors, like limited availability (or availability of the same thing at different prices), maximum allowed cost of diet (it's hard to spend 30k on food when you only have 5k at hand), safety features for people who dislike spending too much on a single item, etc ...

I'm well aware of Eleron's diet calculator -- in fact, the first version of consumption.ash (my ED replacement) actually just scraped the old calculator, but then I realized that was probably not a good idea if I wanted to release this script into the wild, especially since I hadn't asked him for permission.

The currently available version of consumption.ash runs very quickly if it doesn't have to look up mall prices (i.e. just using historical_price) -- I think I've clocked it at a tenth of a second or so. It does use DP; and yes, I know that it's not quite optimal in real-world use, since that much mall information just isn't available to ASH.

I originally had planned for a third revision as a port of Eleron's javascript calculator, but this turned out to be more complicated than originally envisioned, and I just haven't had enough time / interest.

Also: the way DP would work with your example:
Code:
1: 1fc
2: 2fc
3: 3fc
4: yuck
5: fc + yuck
6: 2fc + yuck
7: nom
8: fc + nom
9: 2fc + nom
10: yum
11: fc+yum
12: 2fc+yum
13: 3fc+yum
At this point, 2nom is better than yum+yuck, so
Code:
14: 2nom
15: fc+2nom

Expected run-time for unbounded knapsack is simply O(S*N), where S is size of knapsack, and N is number of items. In fact, if you start considering each item at each price to be distinct, you can argue the same for bounded knapsack (once again, it's pseudo-polynomial runtime). Space used is fairly small (O(S^2), considering that we have a list of no more than S items for each fullness).

Also, you can improve runtimes by considering notions of dominance (where one item is simply better than another) and, say, taking the top MAX_FULL fullness worth of foods for each fullness tier (in sorted order). Since my implementation assumed unbounded knapsack, I could improve this further by simply taking the best for each tier and then removing larger foods that were completely dominated by smaller foods.

So the expected run-time here for fullness would be some constant multiplied by S * (S + S/2 + S/3 + S/4 + ... + 1), or let's just overestimate with roughly 3*S^2 (where S is bounded above by 39, if you count for distension and melange). If we also count all the precomputing, that'll be a single loop over all items (or consumables, doesn't matter) to categorize all relevant items, and then another 3 * S log S to sort all the lists.

Recomputing diet after every item... would increase runtime by maybe a factor of S. Even so, Eleron's javascript calculator certainly doesn't do this -- as you mentioned, prices are cached.

To be honest, the real bottleneck is finding updated prices in a timely manner, since Mafia does not expose this information. I imagine fifth-mall-price is a good enough approximation for most cases, although it will skip items where a few (<5) suppliers have a large quantity at reasonable prices, and the rest are very overpriced.

Eleron's site has all the results precomputed and simply retrieves the result when you fill in your parameters, I believe. I remember him saying that the calculations would be slow and can't be done right when someone enters in their values.

I thought that his new version of the calculator computed it in the browser. Sure, the old version simply retrieved the result, but the current version seems to actually contain logic.

edit: also, xKiv: that's why the greedy solution to the knapsack problem ensures at least 50% of the optimal result :)
Suppose you have 20 fullness. Item A gives 11 fullness for value of 5000, item B gives 10 fullness for value of 4999. Greedy algorithm picks locally best item at every step. It certainly runs in linear time but only gives a value of 5000 versus 9998. (yes, this is even more contrived :) )
 
Last edited:

Veracity

Developer
Staff member
To be honest, the real bottleneck is finding updated prices in a timely manner, since Mafia does not expose this information. I imagine fifth-mall-price is a good enough approximation for most cases, although it will skip items where a few (<5) suppliers have a large quantity at reasonable prices, and the rest are very overpriced.
Do we return the 5th-cheapest price or what you would pay for the 5th item?

Seems like either would prevent writing a mallbot from snatching up 1 - or even 4 - grossly underpriced items, but the second would let you have a realistic expectation of what you'd pay for (up to) 5 items and also take advantage of large numbers of items being offered "on sale", so to speak.
 

heeheehee

Developer
Staff member
I seem to remember it being fifth-cheapest price, for some reason. Will look at the source code.

edit: ooh, it is fifth item that you'd be able to purchase. That is definitely more useful!

(from roughly line 519 of net.sourceforge.kolmafia.session.StoreManager)
 
Last edited:

Theraze

Active member
Not actually 5* as slow to calculate out 5 stacks... the biggest delay is in checking the prices and initializing the entire stack of possible consumables. We cache that stack based on Veracity's suggestion, which cut it from ~30 seconds down to 8, if I remember properly. And that speed bonus was before we started the multiple stacks of possibilities... it was a wonderful day in the "wow, this takes forever" world. :)

Calculating out the 5 stacks should at worst double the execution time. I'll need to try to actually time that eventually though.

The main speed bonus from knapsacks would come if we didn't need to know item availability... which we do, since we don't live in a perfect world where unlimited items jump into our inventory when we need them. Barring backpack items, of course. :)
 

heeheehee

Developer
Staff member
The main speed bonus from knapsacks would come if we didn't need to know item availability... which we do, since we don't live in a perfect world where unlimited items jump into our inventory when we need them. Barring backpack items, of course. :)

Respectfully disagree with this comment -- we don't need unlimited items, but rather just enough to fill ourselves up. And since Mafia provides us with the fifth item's price, we can expect price to not rise too much. Furthermore, if we had more information available (i.e. all of the mall prices -- this won't happen, since that'd probably encourage the rise of many more mallbots), then as mentioned, we can just solve a basic variant of the bounded knapsack problem rather efficiently, as I outlined above. It'd still run orders of magnitude faster than whatever ED's doing while also being "optimal".

Since we don't have this much information at our fingertips, I don't see why we wouldn't use UKP, which is a far simpler problem. It's also the best we can do in terms of both efficiency and optimality as far as I can tell.

(note: the only reason why I've more or less discontinued work on consumption.ash is because at that point, all that needed to be done was account for weird edge cases like salty mouth and drunki-bears)
 

xKiv

Active member
Eleron's site has all the results precomputed and simply retrieves the result when you fill in your parameters, I believe. I remember him saying that the calculations would be slow and can't be done right when someone enters in their values.

That was the version before the current javascript thing (which, from reading the source, seems to iterate over all combinations of organ fulnesses (from empty to full), try to "fill up" with various best foods (of different sizes), and compare with previous results for the resulting fulness
i.e. ... we are at 4 fulness with profit X, what if we ate 1 of this size 5 food with profit Z? the previous best result for 9 fulness has profit X1, X+Z < X1, so don't eat that form here .. etc).


Also: the way DP would work with your example:
Code:
1: 1fc
2: 2fc
3: 3fc
4: yuck
5: fc + yuck
6: 2fc + yuck
7: nom
8: fc + nom
9: 2fc + nom
10: yum
11: fc+yum
12: 2fc+yum
13: 3fc+yum
At this point, 2nom is better than yum+yuck, so
Code:
14: 2nom
15: fc+2nom

Ow, I had 4-6 wrong already (from doing it in my head) ...

To be honest, the real bottleneck is finding updated prices in a timely manner, since Mafia does not expose this information. I imagine fifth-mall-price is a good enough approximation for most cases, although it will skip items where a few (<5) suppliers have a large quantity at reasonable prices, and the rest are very overpriced.

I don't know if eleron still does this, but he used to have files with scraped mall prices from all stores. Not sure how up-to-date they were, I think there were timestamp suggesting that the data wasn't very recent.


Not actually 5* as slow to calculate out 5 stacks... the biggest delay is in checking the prices and initializing the entire stack of possible consumables. We cache that stack based on Veracity's suggestion, which cut it from ~30 seconds down to 8, if I remember properly. And that speed bonus was before we started the multiple stacks of possibilities... it was a wonderful day in the "wow, this takes forever" world. :)

But that will happen the same in all implementations (and will slow them all by the same constant time, which will decrease the relative differences between them).


Respectfully disagree with this comment -- we don't need unlimited items

In-run, all items are limited. And the limits can be quite low.
 

Theraze

Active member
I disagree with hee[sup]3[/sup] exactly as xKiv says... in-run, we're building our own food and choices are limited. Just cause we can buy/make one boris pie doesn't mean we can make 3...
 

xKiv

Active member
And, in case it wasn't obvious, this is *not* optimal:

decide on a diet consisting of 3 boris pies (and something)
eat one boris pie
realize we can eat no more boris pies
recaluclate optimal diet for the remaining fulness.

(because a different combination of food sizes might have been optimal from the start)
 

heeheehee

Developer
Staff member
Why didn't anyone tell me we were also considering the case of in-run consumption? There, Eleron's script doesn't work either. :p And I don't see the point of having the same consumption script for both in-run and aftercore, unless you want to model value as a linear combination of weights, which change depending on the situation.

In-run is a very different case from out-of-run. There, I see no reason to just maximize adventures without any regard for the consequences, and so BKP still is a good representation of the problem -- as mentioned, just treat each food as separate. It's only a linear slowdown in that case -- instead of O(S^2), we get O(S^3). Preprocessing costs us O(N*S*log S) still, with N being the number of distinct items available (directly or other means, e.g. NPC stores, crafting), assuming that we maintain S sorted lists. In-run, this'll never get particularly large.

I suppose you're still constrained to a budget, namely however much meat you have on hand. Do you really ever find yourself optimizing in the early parts of a run, when you are low on meat, or do you just adopt a greedy style of consumption, eating whatever you can whenever you run out of turns? I've never done the first, but then again, I'm not really bleeding-edge.

(and yes, special handling is, of course, then needed for items that can be crafted but would require turns.)

Maximizing profit in-run, where you're probably not going to spend any turns farming (for the sake of profit), is rather pointless. Sure, you can always sell off what's left over after a run, or you could spend a day or two in aftercore and make more meat than you would from a few runs.

edit: and xKiv: that was one of the points I was trying to make a few posts ago, when I said that the greedy algorithm will only guarantee you 50% of the optimal solution. There are cases where it gets noticeably closer, e.g. in boriscore, where you have 30 fullness, and the most filling food you can rely on finding gives... 3-4 fullness, and so it'll guarantee about 90%, assuming that you don't run out of meat halfway through (at which point you have enough adventures to make more meat and continue consumption from where you left off).

So naturally, it's not really possible to be 100% optimal in-run, since if you finish consumption in one sitting, you might not be able to fill up on better foods that you find.
 
Last edited:

Theraze

Active member
Think I said it, but it's not in the last 50 or so posts, so I may not have... changing default stepMeat to -1 for new users, since that appears to frustrate more than it assists most people. Also changing the get_starter_items function to include both item_amount and equipped_amount, based on personal experience. Was getting duplicate items when I was wearing the mariachi hat... which isn't supposed to happen unless all 13 items are available. As such, this should reduce some wasted meat. :)

Edit: Another change for next version... will be tweaking the "successful consumption" detection to hopefully not list daily specials which you can create as "failed" anymore. :) Basically, instead of looking for adventure-count changes only if we MUST use the daily store, look for it anytime. Also look to see if our inventory count has changed. Between the two, they should take care of every iteration except for the PvP foods in the daily store... but EatDrink doesn't (currently) care about PvP in the slightest.

User-question time. Should it? It would not be hard to track fights before/after, but we'd need a FReq for mafia to make the proxy field for additive fites available, else we'd never know which items to pick. It's not hard to create a new zlib variable for VALUE_OF_FITES or something similar... could be ED only, could be global. Could even be a mafia preference, but...
 
Last edited:

Theraze

Active member
Another change for next version:
Since 26.0 is more than 24.5 we are using a speculative stack of 836.
We're now using floats instead of ints for estimating best adventure stack. :) This should get us some extra adventures for cheaper depending on luck. Also validated that daily specials appear to be consuming properly.

And since I'm guessing most people didn't see the edit above... Do we care about PvP?
 

fronobulax

Developer
Staff member
What's PvP?


;-)

Three characters. Three never broken hippy stones and proud of it, although that may change soon if the collection completionist actually "wins".
 

Veracity

Developer
Staff member
I care about PVP. It's something new to do that comes with new shinies.
On the other hand, I don't care about EatDrink.
That seems fair to me. :p
 

xKiv

Active member
Optimizing for fites *might* make sense if you are accounting for the goth kid (and then you have to know how much it will be used), but otherwise consuming for +pvp seems straightforward, so far. Pay through your nose, or pay just a lot.
 

Theraze

Active member
Sounds good. If we don't want to be able to maximize PvP consumables dynamically, no need to FReq for it to be added to consumables data files and the corresponding proxy fields added.

So... here's my latest EatDrink with the various changes I've talked about and can't remember currently. :)

Edit: Next version of EatDrink (not attached, whichever goes into the next attached-post) will have Tux-drinks actually display properly in the display.
Old display:
1: rockin' wagon lev:4 gain:4.0 adv:12.0 musc:0.0 myst:0.0 mox:35.0 meat:1840 own:1 value:21010
New display:
1: rockin' wagon lev:4 gain:4.0 adv:14.0 musc:0.0 myst:0.0 mox:35.0 meat:1840 own:1 value:21010
Note that it actually displays the proper 14 adventures now instead of just manually hacking it in the values section. :)

Edit2: And this is a rather large adventure-swing.
Since 71.5 is more than 59.5 we are using a speculative stack of 1112.
 

Attachments

  • EatDrink.ash
    99.5 KB · Views: 65
Last edited:

Theraze

Active member
Okay, here comes that version of ED promised back in July. This also has some VERY basic Zombie acceptance... basically, if it's food, and it doesn't have the word "brain" in its name, remove it from consideration.
 

Attachments

  • EatDrink.ash
    100.6 KB · Views: 43

fronobulax

Developer
Staff member
Okay, here comes that version of ED promised back in July. This also has some VERY basic Zombie acceptance... basically, if it's food, and it doesn't have the word "brain" in its name, remove it from consideration.

Enough experience with this (1539) to make it "official"?
 
Top