A Multi-Armed Bandit PVP Script

WARriorer

New member
For most casual PVP players out there, we might not know how our current minigames strength stacks up against the general rest of the field. This script seeks to automatically solve that problem (to squeeze out that little extra swagger) so you don't have to.

The implementation is an epsilon-decreasing multi-armed bandit strategy (from 0.9 to 0.1 linearly over 1000 pvp fights) which attempts to estimate your best minigame and plays that mini (1-epsilon)x100% of the time, and otherwise chooses to play the other minis uniformly at random. (Advanced users can probably code up a more complex strategy if so desired [e.g. Thompson sampling or UCB, although a big hurdle would be to get gaussian/beta distributions in the ASH script] - this was just a pretty simple implementation).

This script has a hardcoded dependency on UberPVPOptimizer, but may otherwise be run out of the box (it currently accepts no arguments and will simply use all of your remaining pvp fights). It should reset the stats every time the season changes (unfortunately, this means that even if a mini is repeated in the future, we do not consider how it has performed historically in the previous PVP seasons), so unless there's a drastic change to the peevpee.php?place=fight page, there's pretty much no upkeep to be done on the script.

Edit: It currently also does not track the difference between HC and non-HC stats, and simply assumes that they are one and the same thing.

Edit2: Updated to choose non-uniformly (or rather, uniformly across the [padded] win percentages so that minis with better win:loss ratios are more likely to be chosen - i.e. a mini with a 20% win percentage will be twice as likely to be chosen as compared to a mini with a 10% win percentage) when exploring (i.e. not playing the current statistically best minigame).

Edit3: Updated to carry stats over into Post-Season since the minis haven't changed yet.

Edit4: Fixed some errors in parsing pirate season minigame names. Arrrgh
 

Attachments

  • PVP_MAB.ash
    6.9 KB · Views: 1
Last edited:

SketchySolid

New member
Oh, wow. This sounds really neat. I just guess which pvp minigame will be the best for me each season, but this sounds so much better.
 

WARriorer

New member
With the season just ending (post-season just started) I thought I would post my stats from running this script since I started using it.

> Season 60 minigames statistics:
> - Barely Dressed: 29/67 (43.2%)
> - Basket Reaver: 47/96 (48.9%)
> - Polar Envy: 43/78 (55.1%)
> - Maul Power: 2533/4108 (61.6%)
> - Grave Robbery: 36/78 (46.1%)
> - Most Things Eaten: 50/95 (52.6%)
> - Visiting the Cousins: 35/73 (47.9%)
> - Northern Digestion: 21/51 (41.1%)
> - Hibernation Ready: 67/135 (49.6%)
> - What's in the Basket?: 46/94 (48.9%)
> - Bearly Legal: 38/74 (51.3%)
> - Beary Famous: 49/86 (56.9%)
> This season's win rate: 2994/5035 (59.4%)

This was achieved without any preparation for any specific mini, and it's also nice to see that my season win rate is (1) better than 50%, and (2) it's pretty close to the win rate of my best mini [as it should be]. Naturally these win rates don't apply to anyone else other than me, since each individual's win rate is a function of their character state (which is highly likely that it's different from mine).

Will be interesting to see how it fares in the next upcoming season.
 
Top