Bug Mafia Slowness

Over the last few weeks I've noticed Mafia getting slower and have had a few lockups. (Ubuntu 16.04, latest mafia version from Jenkins)

Today it is VERY bad, I ran `top` and I see java using 100% of the CPU and 9GB of virtual memory.

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10752 rugrat 20 0 9639444 1.063g 30724 S 100.0 6.8 21:05.65 java

Is there a memory leak or is this resource hog natural?
 
Last edited:
The only thing I have been updating is Mafia. This is my work laptop, updates are shut down it hasn't been updated for 3 months.

Correction, I updated libxerces-c28 on monday.
 
Last edited:

Veracity

Developer
Staff member
Literally every time something like this has been reported:

- It's been on a Linux system
- It's been an issue with the Java version that is installed.

What Java version do you have?
Is there anyone else - anyone - who has noticed an issue like this in the last month?
What else have you changed - regarding your use of KoLmafia? New scripts?

Looking at all the changes in the last month - easy to see if you look at the Version 19.7 announcement - nothing has changed that could cause an infinite loop or a memory leak, that I can see.
 

heeheehee

Developer
Staff member
9G of virtual memory is essentially meaningless, though... it's only actually using 1G of RAM (per the RES column from top).

Are you running any scripts?

For what it's worth, I have had no problems running Mafia on Linux, even with a relatively new version of Java --
$ java -version
openjdk version "10.0.2" 2018-07-17
OpenJDK Runtime Environment (build 10.0.2+13)
OpenJDK 64-Bit Server VM (build 10.0.2+13, mixed mode)

(I also have OpenJDK 12 installed, but I can't build for Java 6 with it.)
 

Veracity

Developer
Staff member
(Off topic, but I would like to move to Java 7, at least. How many people can’t run that?)
 

xKiv

Active member
(Off topic, but I would like to move to Java 7, at least. How many people can’t run that?)

(do java 7 releases even get current truststore? I think I had to update java recently because some certificates it trusts expired and I could not connect to zachbardon.com)
 

Veracity

Developer
Staff member
(do java 7 releases even get current truststore? I think I had to update java recently because some certificates it trusts expired and I could not connect to zachbardon.com)
When I say "move to Java 7", I mean "use an up-to-date Java but allow Java 7 syntax" rather than only allowing Java 6 syntax.

Code:
$ java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)
I can write and run Java 8 code, but KoLmafia is restricted to Java 6 features.
 
OOkay, conferred with my IT guys that my computer wasn't silly itself.

I explained to them the problem and they told me I should look into minecraft optimization for linux, since this problem is similar to what that community experiences.

So that being said I read, I tinkered, and I made a little launcher script over my lunch break, and have been testing it. It is showing a lot of promise. I probably went overkill on the upper limit of java's memory allocation, but screw it, go big or go home.

I'm going to continue testing over the next few weeks, if it all is good I'll have a script to help others.

Code:
#!/bin/bash
## This script optimizes the CPU and sets java's memory usage for Linux
## This script MUST be located in the same directory as the KoLmafia .jar file

## Set our variables
MyKoLM=`ls -t KoLmafia*.jar | head -n 1`
MyCPU=`which cpufreq-set`

## Fail if no KoLmafia found
if [[ -z $MyKoLM ]]; then
        echo "Could not find a valid KoLmafia .jar file in `pwd`"
        echo "This script MUST be located in the same file where the .jar is located"
        exit 1
fi

## Make sure we can run KoLmafia in the event of a fresh download
echo "Ensuring KoLmafia is executable"
chmod 777 "$MyKoLM"

## Get Free Memory Data
echo "Getting Memory Stats"
Mem=`free -mh | grep Mem | awk '{print $4}'`
Round=`echo $Mem | sed -e 's/[a-zA-Z]//g' | awk -F "." '{print $1}'`
HRA=`echo $Mem | sed -e 's/[0-9\.]//g'`

## Fail if Free memory is too small, otherwise optimize
echo "We have $Mem free memory"
if [[ "$HRA" != "G" ]] && [[ "$HRA" != "T" ]]; then
        echo "This script only allocates Gigabytes or Terabytes."
        exit 1
elif [[ "$Round" == "1" ]] && [[ "$HRA" == "G" ]]; then
        OptMem="512M"
elif [[ "$Round" == "2" ]] && [[ "$HRA" == "G" ]]; then
        OptMem="1024M"
elif [[ "$Round" -ge "3" ]] && [[ "$Round" -lt "8" ]]; then
        OptMem="2048M"
elif [[ "$Round" -ge "8" ]]; then
        OptMem="4096M"
fi
echo "Optimising to use $OptMem memory"

## Try to optimize the CPU if we have the required utility
if [[ -z $MyCPU ]]; then
        echo "Would prefer to maximize CPU consider installing optimizer."
        echo -e "\tRun: sudo apt-get install cpufrequtils"
        sleep 3
else
        echo "Setting CPU to Performance mode"
        sudo $MyCPU -g performance
fi

## Run java with our optimized memory (Xmx and Xms should be equal to prevent mid process resizing)
echo "Starting KoLMafia"
java -Dsun.java3d.opengl=true -Xmx$OptMem -Xms$OptMem -jar ./$MyKoLM
 
Last edited:
Oh, so you were running at 100% *underclocked* cpu? That makes some sense ...
I was running with a factory default CPU configuration, which I'd venture a lot of people use.



This is interesting. It should probably do absolutely nothing with kolmafia, though. Unless java now uses 3D when doing 2D operations.
Yeah, I hadn't started pulling parts out of it yet. I want to get to a certain optimization level and then scale back. My first step was getting the Minecraft stuff in there and working.
 

xKiv

Active member
I was running with a factory default CPU configuration, which I'd venture a lot of people use.

That's probably a no then. Unless your distro does some cpufreq stuff for you by default. Mine didn't, I had to set it up by hand.

Yeah, I hadn't started pulling parts out of it yet. I want to get to a certain optimization level and then scale back. My first step was getting the Minecraft stuff in there and working.

You probably don't need all of that anyway. I am running with 512M heap and it's fine. It's probably too much, even.
 
I'm also getting some mafia slowness/unresponsiveness, but on a windows machine (and a rather new one). How do I go about troubleshooting it?

Typical case would be running auto-adventure via GCLI. Mafia will be slow to respond, or unresponsive. (for example when trying to scroll up the GCLI, or switching between tabs in mafia. To be clear I'm not chaining commands- just looking at other things while it adventures)

Alternatively, I'll finish a combat in the relay browser (firefox), and when I switch back to mafia, the GCLI will take a few seconds to display the combat results, and freeze up during these seconds (the browser already displayed the results)


Not sure what other info I should note, so just a couple of things-
Win 10, amd 5 2600, 16GB RAM, SSD obviously, using a wired connection.
During these periods, my PC will running at its usual speed.
It gets worst the longer mafia is open. Restarting it makes things better, for a while.
I'm not running any high-demand programs in the background, other than firefox.
 
Last edited:
I'm also getting some mafia slowness/unresponsiveness, but on a windows machine (and a rather new one). How do I go about troubleshooting it?

Typical case would be running auto-adventure via GCLI. Mafia will be slow to respond, or unresponsive. (for example when trying to scroll up the GCLI, or switching between tabs in mafia. To be clear I'm not chaining commands- just looking at other things while it adventures)

Alternatively, I'll finish a combat in the relay browser (firefox), and when I switch back to mafia, the GCLI will take a few seconds to display the combat results, and freeze up during these seconds (the browser already displayed the results)


Not sure what other info I should note, so just a couple of things-
Win 10, amd 5 2600, 16GB RAM, SSD obviously, using a wired connection.
During these periods, my PC will running at its usual speed.
It gets worst the longer mafia is open. Restarting it makes things better, for a while.
I'm not running any high-demand programs in the background, other than firefox.

I don't know much about Windows, but I suggest looking at the minecraft community's suggestions for optimizing Java. The options they have (plus some experimenting of my own) have been what I have been using for my script.
 

Veracity

Developer
Staff member
Alternatively, I'll finish a combat in the relay browser (firefox), and when I switch back to mafia, the GCLI will take a few seconds to display the combat results, and freeze up during these seconds (the browser already displayed the results).
I have issues with Firefox as my relay browser. I’ll click something and Kolmafia will send it to KoL and the browser will be “Waiting for 127.0.0.1” for a looooong time. Eventually, KoLmafia will get the response - or Firefox will accept it - and the browser will unfreeze. If I close the browser window, the results will show immediately in the gCLI, so KoLmafia obviously has the response.

I’ve meaning to look into it. Or maybe ditch Firefox if it doesn’t behave like that in Safari or Chrome.
 

Magus_Prime

Well-known member
The issue Veracity describes is why I stopped using Firefox as the relay browser for KoLmafia more than a year ago. That and Firefox would, seemingly, crash for no discernible reason at semi-regular intervals outside of KoLmafia use. I could never figure out the problem and moving to Chrome for the relay browser fixed both issues for me.
 

fronobulax

Developer
Staff member
I suffer through similar issues with Firefox because I am not prepared to let Chrome into my environment. I have noticed that disabling addons, running in safe mode and/or whitelisting 127.0.0.1 sometimes helps.

I have also noticed that network conditions will bring KoLmafia to a halt or appear unresponsive. In my case, my ISP throttles upload speed and if something, Dropbox, for example, decides it is going to upload something that takes a couple minutes, mafia will be unusable.

An obvious question is how much memory does KoLmafia report using and how does that compare to the maximum available - both of which are to the left of the Collect Garbage button on the "buttons bar".

If mafia is using most of the available memory and/or garbage collecting a lot that could explain the lack of responsiveness. I recall in the distant past, a new feature was implemented in a way that created a lot of objects that persisted longer than they needed to and some folks with limited memory got the point where everything was in use and garbage collection freed very little. Perhaps a recent addition introduced something similar?
 
Just an FYI, while running java from the terminal, I notice this error right before the biggest slowdowns:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.tmatesoft.svn.core.internal.util.SVNSocketFactory (file:/home/rugrat/Desktop/GameStuff/KoLMafia/KoLmafia-19470.jar) to method sun.security.ssl.SSLSocketImpl.setHost(java.lang.String)
WARNING: Please consider reporting this to the maintainers of org.tmatesoft.svn.core.internal.util.SVNSocketFactory
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
 
An obvious question is how much memory does KoLmafia report using and how does that compare to the maximum available - both of which are to the left of the Collect Garbage button on the "buttons bar".


I saw mafia take up around between 700 to 1800 MB of ram (it builds up over time) according to taskmanager.
The numbers next to the garbage collect button are around 15% or so, no where near the max.
 
Last edited:
I’ve meaning to look into it. Or maybe ditch Firefox if it doesn’t behave like that in Safari or Chrome.

Huh, I'm surprised you aren't having issues with Chrome. Both Firefox and Chrome historically have issues w/ accessing localhost w/ IPv6 settings enabled.

Turning off IPv6 setting in both the browser and on the system usually helps.
 
Top