Bug Some maximizer combinations fail to respect +equip and min

[Xande1]

OK, I've hardly exhausted all the possibilities, but there are definitely still some maximizer strings that fail to respect an "+equip" directive and a "min" directive. With the right combination of modifiers, the maximizer makes suggestions that don't equip the requested item, and don't get enough of the requested min stat. These strings report FAIL, even though items capable of satisfying the request are available and equipable. Sometimes adding more terms to the maximizer string causes it to work properly, but sometimes adding terms still fails. This is in r20551

This appears to be related to my earlier bug report here (https://kolmafia.us/threads/maximizer-wonkiness.25633/), but since the exact maximizer string I was reporting there is fixed, and the strings that tickle the problem now are ones I haven't used regularly, so I can't say for sure if any of these new strings were ever working and when they broke.

minimal string I've found that causes the error: hot res, 15 min, hp regen max, +equip high-temperature mining drill
(Screenshot of Maximizer's suggestions attached; char has 8 hot resist from buffs and passives; suggested gear doesn't include mining drill, and only gets him to 14 hot res (+3 shark jumper, +3 hardened slime belt))

Anyway, here's a catalog of strings that I know are broken (they trip the bug and fail, even though they shouldn't), and similar strings that work fine. Maybe someone more familiar with maximizer internals than me will see a revealing pattern here.

Strings that are broken (FAIL, even though they shouldn't):
hot res, 15 max, 15 min, hp regen, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 max, 15 min, hp regen max, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen max, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen max, +equip high-temperature mining drill, hot damage

Simliar strings that work properly:
hot res, 15 max, 15 min, hp regen, +equip high-temperature mining drill, -familiar, -tie, 0.05 hp
hot res, 15 max, 15 min, hp regen min, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 max, hp regen, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen min, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen max, +equip high-temperature mining drill, myst

 

[fronobulax]

Thank you. Can you get similar failures with other equipment? I ask because high-temperature mining drill has a dash and spaces and that is an opportunity for parsing errors. Doesn't explain what did work but...

 

[Xande1]

Thank you. Can you get similar failures with other equipment? I ask because high-temperature mining drill has a dash and spaces and that is an opportunity for parsing errors. Doesn't explain what did work but...

Excellent point! I'm running up against trying to burn turns before RO now, so don't have time to explore fully, but on my main (who I'm currently playing), who's currently sitting at 22 hot res due to intrinsics and buffs, I got:
maximize hot res, 35 min, 35 max, hp regen, +equip chopsticks
to fail. It recommended the Pigsticker of Violence and card sleeve for main and off hands, and only got me to 30 hot res (I had a Pocket Square of Loathing available that it didn't suggest that would have taken me to 35 hot res, as well as other slots where it had 0 hot res equipped in favor of more hp regen). (30 min and max failed and didn't equip the chopsticks, but did get me to 30 hot res, but with the same gear suggestions it gave when I asked for 35 min, so the fact that it got enough hot res to satisfy the expression appears to have been essentially coincidental). So it doesn't appear to be a parsing error with the item name.

If I get spare time after RO, I'll try to trip the error with other attributes/gear pieces, but I just sort of stumbled into the initial failure, so I don't know how much luck I'll have.

 

[fronobulax]

At some point someone who is grasping at straws will ask you what your maximizer options are set to. That might not be a factor but it is what I would do if i were stuck or wanted to give the illusion of progress on racking down someone's bug.

 

[ckb]

This strangeness with min/max on maximizer strings sounds similar to some issues I have had: maximizer-inconsistency

FWIW, I tried your string "maximize hot res, 35 min, 35 max, hp regen, +equip chopsticks"

This fails to equip the chopsticks... but "maximize hot res, 15 min, 15 max, hp regen, +equip chopsticks" works ok (I have 13 res currently).
Increasing the min/max numbers shows me this fails to equip chopsticks when the res number hits 20.

 

[fronobulax]

This strangeness with min/max on maximizer strings sounds similar to some issues I have had: maximizer-inconsistency

To save me some digging, does that pre or postdate the change that added/changed filters? It is a different debugging path if the filter change just exposed an older bug rather than introduced one.

 

[ckb]

To save me some digging, does that pre or postdate the change that added/changed filters? It is a different debugging path if the filter change just exposed an older bug rather than introduced one.

It looks like the filter change was "20497: Optional radio-button style functionality to maximizer filter checkboxes" on 2020-11-11.
My issue was posted on 2020-06-18.
So this was before the filter change.

 

[MCroft]

minimal string I've found that causes the error: hot res, 15 min, hp regen max, +equip high-temperature mining drill

I pulled a high-temp drill and tried this.

max is only defined (per the maximizer help) as valid with a number in front of it. 0 max or 20 max causes me to use the drill. We should probably give a syntax error if we get an invalid command. That doesn't answer the rest of the

Strings that are broken (FAIL, even though they shouldn't):
hot res, 15 max, 15 min, hp regen, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 max, 15 min, hp regen max, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen max, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen max, +equip high-temperature mining drill, hot damage

I am. not sure how min 15/max 15 should work, but I'll look through the maximizer (probably debug it) later tonight.

I want to look at #1 and #3 on this list, which don't have the hp regen max command...

Simliar strings that work properly:
hot res, 15 max, 15 min, hp regen, +equip high-temperature mining drill, -familiar, -tie, 0.05 hp
hot res, 15 max, 15 min, hp regen min, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 max, hp regen, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen min, +equip high-temperature mining drill, -familiar, -tie
hot res, 15 min, hp regen max, +equip high-temperature mining drill, myst

I may well invalidate half of these, if I put in a check for "max with no value".

 

[MCroft]

Well, it may be that min 15/max 15 is failing for me because it thinks my starting state is 9.50 and it can get to 14.5 or 15.5, never 15...

 

[MCroft]

Well, min 15/max 15 is failing for me because it thinks my starting state is 9.50 and it can get to 14.5 or 15.5, never 15...

Not sure what correct behavior should be. Maybe mention that?

 

[AlbinoRhino]

How does the code interpret the 'max' in 'hp regen max' versus looking for its 'max' keyword? Seems a possible source for error.

 

[Xande1]

Replies hidden inside spoiler tags, because this post is a wall of text as is, and I'm trying to direct attention to what I consider the important bit. If you asked questions above, there's probably a reply hidden inside.

Spoiler: Replies to upthread comments and random musings before my edit

Reply roundup, since a lot happened while I was napping.

At some point someone who is grasping at straws will ask you what your maximizer options are set to. That might not be a factor but it is what I would do if i were stuck or wanted to give the illusion of progress on racking down someone's bug.

Fair question. I get this behavior doing maximize whatever from the GCLI (which, per another current thread ignores settings), as well as in the maximizer window with everything selected, and maximizer window with only "equip" selected. Other settings are my defaults: "Equipment: on hand" and "Max price: don't check". That covers the two extremes; I haven't done much exploring of the space of combinations of turning on and off other filters or changing equipment and purchase options.

I pulled a high-temp drill and tried this.

max is only defined (per the maximizer help) as valid with a number in front of it. 0 max or 20 max causes me to use the drill. We should probably give a syntax error if we get an invalid command. That doesn't answer the rest of the

I am. not sure how min 15/max 15 should work, but I'll look through the maximizer (probably debug it) later tonight.

I want to look at #1 and #3 on this list, which don't have the hp regen max command...


I may well invalidate half of these, if I put in a check for "max with no value".

Side note: switching to "+equip armgun" for testing here forward, since it's a 2-handed weapon with a one-word name and no hyphens or other special characters, so should have the same overall effect as the mining drill, but without the possibility of causing a parse error. ("Chopsticks" last night was caused by my being in a hurry and that being the first one-word weapon name I saw.)

Sorry if my "minimal example" created unnecessary confusion. Let me start off by saying that maximizing "hot res, 15 min, hp regen, +equip armgun" also triggers the bug (on this character), and gets nearly the same equipment suggestions on that character shown above (except it recommends the paperclip cape instead of misty cloak, because while they have the same average regen, they get there differently).

According to some old threads I read once upon a time the hp regen and mp regen keywords in the maximizer are a bit wonky, since those are derived stats based on the mafia internal modifiers "hp regen min" and "hp regen max" (etc.), and one suggestion when they were causing trouble was to use either "hp regen min" or "hp regen max" (which maximizes the bottom end hp regen, or top end end hp regen, respectively), or possibly explicitly replace them with "0.5 hp regen min, 0.5 hp regen max". Therefore, to me, showing that the error occured when using "hp regen max" was more minimal than showing it for "hp regen", but I do see how it could point toward a parsing error. However, "hp regen min" and "hp regen max" are valid numeric modifiers, and are supposed to be accepted by the maximizer ("The full name of any numeric modifier (as shown by the modref CLI command) is a valid keyword")

Also, what "min" number the failure happens at is dependent on the character running the maximize command's stats/passives/buffs. On the account I'm trying to do all testing on, 15 min triggers it, but 14 does not. (for reference, straight "maximize hot res" can get that character to 44 hot res, and maximize +equip armgun can get to 39). On my main currently (less buffed than when I posted yesterday, for reasons), 19 min doesn't trigger the bug, but 20 does. When I tried on my main yesterday when he was sporting full rollover buffs, 30 min didn't trigger the bug, but 35 did.

Well, it may be that min 15/max 15 is failing for me because it thinks my starting state is 9.50 and it can get to 14.5 or 15.5, never 15...

How does the maximizer think you have 9.50 hot res to start? Min/max are only (supposed to be) global at the very beginning of the maximizer string. When they come after a modifier, they only apply to that most recent modifier. "Max" doesn't mean 'don't allow this modifier to exceed this value', but "The weight specifies the largest useful value for the preceding modifier. Larger values will be ignored in the score calculation, allowing other specified modifiers to be boosted instead." There are use cases for a version of max that actually limits the modifier at X (drop testing with a perfectly fair coin, for example), but that's not how max currently (supposedly) works.

For the above minimal example ("hot res, 15 min, hp regen, +equip armgun"), I tested all 6 permutations of parameter order (treating "hot res, 15 min" as a unit, since separating them changes the sense of the expression). All 6 permutations failed in the same way.

Finally, here are some other things I've noticed while trying to get various combinations to fail:
The value before the min that trips the bug will be state-dependent, but should generally be less than your max achievable modifier for that stat.
If a particular min value fails, all larger min values will fail.
For example, on my test character, maximizing "familiar weight, +equip armgun" reaches a value of 103. Maximizing
"familiar weight, 103 min, +equip armgun" succeeds as expected
"familiar weight, 71 min, +equip armgun, ML" fails, and doesn't equip the armgun
"familiar weight, 71 min, +equip armgun, ML, myst" succeeds (I feel like this is important)
"familiar weight, 91 min, +equip armgun, -combat" fails, and doesn't equip the armgun

PS - All my tests were done starting from the same outfit, to be sure that equipment already in use wasn't affecting the results. In retrospect, I should probably start from naked; I'll do that if I have to do much more testing.

Edited to add:
OK, I think I've had a breakthrough, and I think we've got 2 separate interacting bugs. Take all of this with a grain of salt, because it's based on playing with the maximizer itself, and not looking at the internal code, which I won't have time for for at least a couple of days.
The following just happened:
"familiar weight, 76 min, ML" fails (note above that "familiar weight, 103 min" succeeds on this character)
"familiar weight, 76 min, ML, myst" fails
"familiar weight, 76 min, ML, DR" succeeds
Note that this one has no "equip". I had to jack up the value on min slightly from above, because without the equip statement, the maximizer can get slightly more familiar weight from the hands slots.

What I think is happening:
1) If a maximization fails, the maximizer isn't bothering to honor "equip", since failing to honor it is a failure, and it's already failed anyway. This should probably be changed, IMHO, and is probably easy enough to fix, relatively. (I'd even go so far as to say that except for maximizer's special case of "equip and outfit in the same command", the maximizer shouldn't even be considering combinations that don't have the "equip" target in at least one slot, which might cut down on the number of combinations for maximize strings with equip.)
2) If you're maximizing multiple stats, and one of them has a "min" attached (we'll say hot res and hp regen, for simplicity), the maximizer's pruning of possible equipment may remove top-end hot res gear, if it ranks too low in (hot res + hp regen), because it's contributing less to overall score. So then, the best hot res it can generate will be below the min for hot res, because it's not trying the top-end hot res gear (like Pocket Square of Loathing, because it has 0 hp regen, and there are so many hp regen accessories that it gets ranked too low to be on the maximizer's consider list after pruning) even though mafia can find plenty of hot res when not maximizing for hp regen as well. Adding a 3rd stat (like mysticality) brings some of that gear back into consideration (e.g., Pocket Square because maximizer has to consider partial Clothing of Loathing outfits due to the partial outfit stats bonus), thus causing the maximization not to fail when another stat is added, if it's the right stat to make it consider certain gear. (Above, I think DR succeeded in the familiar weight maximize because the belt of loathing, which is +10 fam wt, has DR on it.)

Following test based on that assumption: if I start with a straight "maximize hot res" and equip the suggestions, so I am wearing maximal hot res gear when I run the following commands:
"hot res, 15 min, hp regen, +equip armgun" fails
"hot res, 15 min, hp regen, +equip armgun, +current" succeeds!
The only difference between the first and second command is the +current directive (I'm in aftercore), which causes the maximizer to explicitly consider the gear I'm already wearing. In fact, everything up to "hot res, 39 min, hp regen, +equip armgun, +current" succeeds when starting from maximal hot res gear (as expected, since 39 is the max hot res that character can reach with an armgun equipped).

If I'm right, #2 is tricky to fix, without making the maximizer significantly slower than it already is. On the other hand, the slowdown will only affect maximizer strings that use "min", so it may not be that big a deal. I'm not sure what the right thing to do here is. If I have a brilliant idea on that front, I'll be sure to chime in, but I wanted to throw out my discovery.

Apologies for the wall of text.

 

[fronobulax]

Hmmm. When I was debugging and came up with https://kolmafia.us/threads/20458-p...n-maximization-under-some-circumstance.25533/ my recollection is that the maximizer got to a point where it hadn't found anything, thought it had and so accessed a null pointer. I have a vague recollection that there was something that I thought was the best so far and I chose to return it. I wonder if that was the wrong choice? I will add checking that to my ToDoASAP list...

 

[MCroft]

TIL that, despite similarities in names, min and max are very different. Min is “must reach value”, but max is “any value over this satisfies this rule”.

If we’re looking at optimizations, maybe we should pre-parse and sort the Maximizer string so we try the things with small combos first? Or maybe just equipment first? If you have two +equip shirts and just one torso, what do we want it to do? Fail fast would not even check beyond equipment.

maybe fail needs to tell us which constraints it failed to meet.

I was trying to wrap my head around Maximizer and I was thinking “this is like trying to write a db query optimizer”, but I woke up this morning and thought “no, it’s a constraint solver”. So after a quick rabbit-hole on techniques for optimizing constraint solvers, I am wondering if we can use existing libraries instead of home-rolled methods. That is an entirely different rabbit hole.

But what I think I want to do is document how the parts of Maximizer/speculation/evaluator/parse fit together for my own sanity and things like optimizations and libraries should fall out of that.

 

[fronobulax]

<veer>
At its core the maximizer is a constraint solver. But when you cannot formulate the problem easily as a constraint problem or can't afford the license fee for a third party solver, you cheat and use iteration knowing that you can get within a few percent of an optimal solution anyway. That is pretty much what the current code is trying.

The issues with an item based constraint formulation are things like outfits that give a bonus when worn, items that cannot be worn together, and items that change stats and thus allow or prohibit other items. It gets worse when you allow effects

I'd love to see a reformulated maximizer because there might be a better chance of folks actually understanding it. It also sets the ground for constraint based consumption, although I think @Veracity 's vcon treats it as an equivalent knapsack problem and finds an optimal solution for that.

To get slightly back on topic, when I was doing Numerical Analysis we were always cautioned to study the problem before writing or running code. Specifically we could often determine analytically whether a problem had no solution, a unique solution or multiple solutions. If we ran code for a no solution problem we got what we deserved. If we ran code for a multiple solution problem we needed to deal with possible non-convergence. So I do wonder if there is a mathematical framework in which some combinations of constraints just don't have a solution but demanding one anyway?

 

[fronobulax]

I reverted a previous commit that dealt with a NPE. It was comparing a new candidate with what it thought was the best so far. The best was null so I figured any candidate had to be better than null and set things accordingly. That might not be causing some of the issues but it removes one possible cause. I could not reproduce any of the wrong choices with my characters. I did get a lot of FAILs however so I expect the strings are unrealistic for my equipment etc. Given a fundamental laziness I am reluctant to embark on a search for something that fails for me.

As alluded to above it is also quite possible that these fall into a gray area where the algorithm, by unintentional design, is going to miss the optimal solution which means this is Bug - Won't Fix until we understand the existing code much better or replace it with a formulation that allows us to bring optimization theory to bear.