Performance drops

[Darzil]

I'm finding that Mafia will run fast for a while (2+ hours), and then slow to a crawl, for me. I'm running Windows 7, on a reasonable spec PC, running Java from a SSD, and Mafia from a fast USB memory stick. I mainly play in relay browser, where the most obvious symptom can be a 20+ second wait for the main pane to update. If I swap to the relay browser, the results display fast. Restarting internet browser doesn't help speed. Restarting (not just logging and in again) Mafia sorts it for another 2+ hours.

Is there anything reasonable I can do to help find the bottleneck on my system within Mafia ?

 

[Veracity]

I've noticed that when playing in the Relay Browser, it slows down as you say - and I notice that Safari's memory usage has grown considerably. I have to restart both KoLmafia AND my browser.

Given that, it's not obvious to me where the blame lies...

 

[roippi]

Assuming there is some memory leak in mafia (i.e. memory heap allocation stays near max after forced gc), this is the minimum that I would need to investigate such a thing:

(assumes you have a jdk)

• Open mafia and open chat
• Open jdkInstallationPath/bin/jvisualvm.exe
• choose mafia
• Go to Sampler, click Memory
• click "perform GC", wait 3-4 seconds
• click "heap dump", save the .hprof file in the upper left

then let chat run for 5+ hours, do another GC, then take another heap dump.

That's gonna be around 150MB of files to send me, I imagine one of the you-upload-it services out there will be necessary. You might not want to publicly post links to them as technically there's info in there that you wouldn't want public. (though it's way more effort to dig it out than it's worth)

Replace the chat operations in the above with whatever relay operations you need to do to recreate the problem.

If it's not a memory leak in mafia (and it doesn't necessarily sound like it is from the descriptions given), I have no idea where to start.

 

[Darzil]

Thanks, have mailed you links, Roippi, though wasn't the worst it's been. Should be in Aftercore tomorrow, will see if that takes a larger toll on performance.

 

[Theraze]

Don't know if you're on IE, Darzil, but if so, it doesn't actually properly close out when you close all its windows. Hey, it's MS!

To properly kill IE, you either need to go to the task manager and find every iexplore.exe and kill them, or run
taskkill /f /im iexplore.exe
in a command prompt window. I find that if I have over 10 IE windows open ever, my computer will bog down until I do a full restart. Or until I force-kill the background ghost applications.

 

[roippi]

Thanks, have mailed you links, Roippi, though wasn't the worst it's been. Should be in Aftercore tomorrow, will see if that takes a larger toll on performance.

One thing you haven't mentioned is your heap allocation/max heap size numbers. What are they when it "bogs down"? (upper right corner of mafia window)

 

[Winterbay]

Don't know if you're on IE, Darzil, but if so, it doesn't actually properly close out when you close all its windows. Hey, it's MS!

To properly kill IE, you either need to go to the task manager and find every iexplore.exe and kill them, or run
taskkill /f /im iexplore.exe
in a command prompt window. I find that if I have over 10 IE windows open ever, my computer will bog down until I do a full restart. Or until I force-kill the background ghost applications.

Well, neither does Chrome if my wife's netbook is anything to go by. Horrible browser that opens up 5-20 processes and then doesn't close all of them when the browser is closed...

 

[Darzil]

Don't know if you're on IE, Darzil, but if so, it doesn't actually properly close out when you close all its windows. Hey, it's MS!

To properly kill IE, you either need to go to the task manager and find every iexplore.exe and kill them, or run
taskkill /f /im iexplore.exe
in a command prompt window. I find that if I have over 10 IE windows open ever, my computer will bog down until I do a full restart. Or until I force-kill the background ghost applications.

You might want to fix your machine.

To be less flippant, I had many issues with IE10 (crashing, or pages forcing a refresh due to 'failure of the website's content') until I read a recommendation to delete all the cookies, files etc, and reset the settings. I guess the upgrade from IE9 to IE10 wasn't very transparent. Since then it has been steady as a rock. I've found iexplore.exe only hangs around after a crash or failure to exit correctly. I don't get this. I can shut down and it is gone.

Oh, and as you may notice from my OP, I don't need to restart IE to restore performance, only Mafia.

 

[Darzil]

One thing you haven't mentioned is your heap allocation/max heap size numbers. What are they when it "bogs down"? (upper right corner of mafia window)

I'll keep an eye on that. I recall the max heap was around 250 Mb this morning, and was around 1.1 Gb when I switched Mafia off this afternoon. Starting just now it's 1.4 Gb. I'll record stuff tomorrow and see what it says.

 

[roippi]

I'll keep an eye on that. I recall the max heap was around 250 Mb this morning, and was around 1.1 Gb when I switched Mafia off this afternoon. Starting just now it's 1.4 Gb. I'll record stuff tomorrow and see what it says.

Uh, max heap is the second number. And shouldn't change, unless you're installing more RAM in between mafia sessions.

 

[Darzil]

Uh, max heap is the second number. And shouldn't change, unless you're installing more RAM in between mafia sessions.

And that's why I shouldn't work from memory. Lets pretend I just said "I'll record stuff tomorrow and see what it says".

 

[roippi]

Cool. Be sure to report both values. And even a third report after you click the gc button right next to the heap number display.

 

[Darzil]

Well, that was strange, can't find the heap dumps from today. Though again, it didn't get particularly slow. I did take some interesting pictures of the memory using the monitor, though. I guess these are normal?

Heap over the first few minutes, it grows quickly then hits steady state:

When I broke prism and ran my after king script, it clearly shows the difference in memory increase rate between running many commands and just sitting in chat:

I switched on batbrain for a while, to see the impact. You can see how much more there is going on, sometimes several garbage collections a minute are needed, and it's hard to see the individual actions:

Ascending didn't change much, just a small flurry of activity, and no change in overall levels:

 

[Darzil]

Bizarrely, however, around 2 hours later, the current heap size reduces markedly. I have no idea why this happened after 8+ hours of pretty constant size !

I also have no idea why it didn't get anywhere near the max of 1.5 Gb.

 

[xKiv]

I also have no idea why it didn't get anywhere near the max of 1.5 Gb.

Once full GC becomes effective enough (frees up more than X% of current heap), the jvm probably prefers just doing GC to increasing heap size. It's tunable somewhere, and there are several different gc strategies, but ... from the first image, I guess your gc decided that clearing up roughly 70% of heap is enough?
Or not, because I have no idea how that heap is divided between generations. Also, I am not GC expert and anything I say on the topic shouldn't be taken too seriously.

 

[roippi]

Nice pictures. Thanks.

I see nothing there that looks like a memory leak, just normal operation. A memory leak would show up as the minima or "troughs" of the blue line steadily increasing.

The JVM will decrease your heap size over time if it decides you have a lot of free space. It should increase it again if it needs to.

 

[Darzil]

Oh, and out of interest, this is what another character looks like, not firing up chat or relay browser, or doing anything :

And yeah, no significant memory leak. Minima went from around 100k to 200k, but over many hours, and I'd expect it to increase just with per-session information.

Edit - And just for fun, this is what automated farming (relay browser open, but not in chat) looked like :

 

[roippi]

One other thing you could try is "greygui dump" when it slows down, maybe there are some extraneous threads running around. The results show up in a debug log.