Bug - Waiting for Info KOL Mafia memory leaks, is this why it becomes a CPU hog?

JLE

New member
After running KOLMafia for some time, its memory usage gets ever greater. Usually starting in the 70k range, but eventually rising to two or even three times that. Of late, this has been happening quicker and quicker.

In recent months, it's also been causing the amount of CPU usage to jump - often to 30%, 40% or even higher, from its normal 1-2% - and *something* is getting very hot inside the computer as well because it's causing the fan to spin up several notches. This is a phenomenon that is usually reserved for only the most graphical-intensive of modern games, which KOL is not.

In each case, the only cure is to completely stop and re-start mafia, whereupon it goes back to 1-2% CPU usage again.

Is this a KOLMafia problem, or a problem within Java itself? And how do I fix it, speaking as someone who basically understands zero about programming and software, but has already taken the cover off the computer, cleaned the dust out and re-applied thermal paste behind the heatsink (and I'm not sure I dare attempt any more hardware "maintenance" on my computer.) Have had a professional check it out for any further problems, and there is no malware or spyware on the system.

(System: Windows 10 Home Edition, desktop computer, quad-core A10 APU processor, bought over the counter at PC World. Running the latest instance of Java, 8/161. Usually using Firefox as the relay browser because I don't really like Google Chrome, and Microsoft Windows Edge is a pile of steaming fetid dingo's kidneys.)
 

xKiv

Active member
There are several numbers "memory usage" can refer to in java applications.

In order of size:

1) How much memory is actually still being used for something by the app. This number is not measurable.

2) How much memory is taken on the app's heap and some other java structures. This includes memory that is "dead", but will be reclaimed later. This is the number that mafia shows as used memory, and what jumps down when you trigger garbage collection.

3) How much memory java reserves for the app (heap size, ...). This can be a lot, and only changes slowly, if ever. It also includes memory that's still unused, but reserved for fast allocations.

4) How much memory the entire java process allocates from the operating system. This also includes structures not accessible to the application, java VM itself, memory mapped files and other delicatesies.

You are likely referring to #2, which you cannot do much about. Meanwhile, #4 (which is what you probably actually care about) is presumably still the same.

~~

If your mafia is causing a lot of CPU action, I guexpect that you (or one of the scripts running for you) are doing gear maximizes with ever-increasing range of options, which means (almost) ever-exponentially growing work for the maximizer.
 

JLE

New member
I'm not, as far as I know, using any scripts or maximizers. Just telling it to auto-adventure or auto-PVP (as in "select area, the tactic is LTS monsters until they are dead, adventure X adventures" and "pvp randomly for flowers with specified stance"), or using the relay browser for manual adventuring.

It still sometimes jumps from using 2% to 30-40% of CPU usage according to Task Manager, and one of the fans in my computer changes speed three times to go faster and louder, and the only way to stop this seems to be to stop it, close it completely and restart by clicking the icon in the folder.

I don't know what the "memory" thing in the top right corner means, apart from a number that goes significantly down when I hit the button by it which appears to be labelled "Collect Garbage". Sometimes this seems to stop the heating up and fan speeding up, sometimes it doesn't.
 

Veracity

Developer
Staff member
I've noticed that my Relay Browser experience can get slower and slower and slower. It varies with browser and version thereof; certain versions of Firefox became unusable within 10 minutes.

Last night I was having network issues - getting lots of IOExceptions - and my browser was sitting there "waiting for 127.0.0.1" for long stretches. I didn't think to see if KoLmafia was using more CPU time. I have had suspicions that Relay Browsing gets inherently slower over time, but have never investigated.
 

xKiv

Active member
I don't know what the "memory" thing in the top right corner means, apart from a number that goes significantly down when I hit the button by it which appears to be labelled "Collect Garbage". Sometimes this seems to stop the heating up and fan speeding up, sometimes it doesn't.

It should be almost irrelevant. When you hit that button, it's just doing something that it was going to do eventually anyway.

I've noticed that my Relay Browser experience can get slower and slower and slower. It varies with browser and version thereof; certain versions of Firefox became unusable within 10 minutes.

Last night I was having network issues - getting lots of IOExceptions - and my browser was sitting there "waiting for 127.0.0.1" for long stretches. I didn't think to see if KoLmafia was using more CPU time. I have had suspicions that Relay Browsing gets inherently slower over time, but have never investigated.

I have noticed that having addons like noscript, umatrix, adblock, requestpolicy, ... (basically anything that has to check every connection to see if should be denied) can cause extreme slowdowns. Especially on pages with many small images, and on pages that reload many times into the same frame (especially especially when it's done via javascript, like kol's chat). A new firefox profile with no addons tends to be comparably fast to chrome.

For some reason, this seems worse when going through mafia than when going directly to vanilla kol.
 

Crowther

Active member
certain versions of Firefox became unusable within 10 minutes.
Recent versions of Firefox have gotten really bad. Ever since they switched to a new rendering engine that was "faster". I used to restart it once or twice a week. Now I have to restart it once or twice a day, because the memory footprint becomes so huge my hard disk starts to thrash. I guess all the developers have lots of RAM and SSDs. I've also noticed mafia using more CPU, but not enough more for me to care.
 

JLE

New member
I seem to have already disabled adblock - some months ago - although actually I hate seeing ads (and in any case KOL relay browser doesn't even *have* ads, although the forums do.) I still technically have it but it's disabled.

Only plugins I have are the Cisco Systems OpenH263 Video Codec (automatically provided with Firefox), Google Widevine Content Decryption Module, and the latest Adobe Shockwave/Flash. I haven't a clue what the first two do, I certainly didn't install them personally, they must have come with Firefox. The third of them (Shockwave Flash) I only installed because some videos won't play without it, this was particularly true of some sports playback videos on the BBC website at the time I installed it.

Oh, and I did also have a plugin for the Facebook game MouseHunt, which I assume does absolutely nothing when not on that site. However it wasn't doing me any good even in that game so I got rid of that too...}

Firefox has definitely become a mighty old memory hog in its own right, but Task Manager attributes more CPU hoggery to KOLMafia (whether it's the specific program, or Java in general, I don't know), and it's a thing that seems to come on suddenly rather than gradually - like, whenever the top-right-corner "memory usage" stated in KOLMafia hits some kind of tipping point of, say, 120k or so (and this always creeps up gradually from the ~70k that it typically starts on: and, quite often, hitting the garbage-clear button only clears some of it, or for a short time, the usage always ends up reverting to the higher level within a few minutes.)
 
Last edited:

Ryo_Sangnoir

Developer
Staff member
Here's a case I just had involving mostly pure Mafia. This is to do with performance issues, not CPU / memory specifically, because this is what I notice.

I do a small amount of relay browsing, then a lot of automated turns. I run OCD inventory management and notice it's taking an awfully long time, so I abort, log out and log back in, without closing and reopening Mafia. It's opening Alice's Army packs.

I see the "Using 1 Pack of Alice's Army Cards..." line, then the five cards I got very quickly, but there's a pause of a few seconds before the "Finished using 1 Pack of Alice's Army Cards." appears. After I close Mafia completely and reopen, the Finished line appears almost immediately after the five cards result.

I also tend to see Relay Browser slowness whenever I use the Intergnat for an extended period.
 

fronobulax

Developer
Staff member
I am reminded of the wisdom that it is hard to fix what is not being measured.

My often wrong intuition suggest this is not a mafia problem as such. If mafia had a "memory problem" then I would expect either Out of Memory exceptions or some more obvious indication that Java was spending so much time garbage collecting that nothing else was being done.

I run Firefox at whatever the latest update pushed to me is and a couple of addons. I routinely find Firefox appearing to be "hung" or taking longer than I want it to. On occasion it will consume almost a GB of memory and 25% of the CPU (which is presume is one core running at 100%). I saw enough "waiting for..." messages that I ran the DNS Benchmark from https://www.grc.com/dns/benchmark.htm , tweaked my DNS servers and got a better response. Often find I get a better response from a site when I visit it in private mode.

All this suggests that the reported problems are due to the relay browser and not KoLmafia directly. The obvious data needed to prove me wrong or point the way to a solution would be to try different browsers as the relay browser and see what behaviour changes.

I have never seen significant CPU usage from Java running mafia except when there was some kind of automation running. Scripts that try and adventure somewhere that is unavailable (and don't check for success/failure) are a frequent culprit. Recovery and mana burning are sometimes culprits. So I would also consider disabling various automation features and see whether there is one that correlates with high usage.

I have some habits that are rooted in antiquity, in particular the days when "sleep" and "hibernate" were unreliable. Thus I completely shut down mafia (and any other program that I am not currently using) when I am not using it and I do a complete shutdown of the computer when it will be unused more than 15 minutes. While this costs me something in wall clock time, it does protect me from certain types of software error.

I also remember, although I think they have all been squashed by now, when mafia got confused if you did not shut it down between characters rather than just log out.

Just musing out loud as I take a break from figuring out what instrumentation I would want/need to actually track this down. If only end users routinely had a JDK instead of just a JRE :)
 

Ryo_Sangnoir

Developer
Staff member
Once you do that, it's entirely possible that kol just assigned you to a different (less loaded) webserver, and you know nothing.

The intended takeaway from the problem description was twofold:
* first, that logging out and logging in didn't fix the issue. Only completely closing Mafia fixed the issue. It's possible I'm just playing server roulette, but I can check this to a higher confidence level next time I have the issue by logging out and in repeatedly without closing Mafia.
* second, that the CLI updated quickly in displaying "using item" and displaying the results of using the item, but there was a substantial delay before it reported that it had finished. I can't find exactly where in the code the "You acquire an item:" printing is happening, but there doesn't seem to be anything substantial going on between that printing and the finished printing. I'd guess that the "You acquire an item:" printing couldn't happen until the response was returned, but I could be wrong.
 
Top