Bug InvalidClassException for Mafia r20602 and openjdk 14.0.2

djve

New member
This is more of an FYI should people be interested in this version of JDK. If it's not going to be fixed please mark it as such.

This started today with the new jar file. 20597 did not have this issue.


:~/KoL/Mafia$ java --version
openjdk 14.0.2 2020-07-14
OpenJDK Runtime Environment (build 14.0.2+12-Ubuntu-120.04)
OpenJDK 64-Bit Server VM (build 14.0.2+12-Ubuntu-120.04, mixed mode, sharing)

$ /usr/bin/java -Dsun.java2d.uiScale=2.6 -jar ./KoLmafia-20602.jar
KoLmafia v20.7 r20602
Released on July 30, 2020

Currently Running on Linux
Local Directory is /home/d/.kolmafia
Using Java 14.0.2

Code:
java.io.InvalidClassException: net.sourceforge.kolmafia.utilities.RollingLinkedList; local class incompatible: stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117
    at java.base/java.ibjectStreamClass.initNonProxy(ObjectStreamClass.java:715)
    at java.base/java.ibjectInputStream.readNonProxyDesc(ObjectInputStream.java:2021)
    at java.base/java.ibjectInputStream.readClassDesc(ObjectInputStream.java:1890)
    at java.base/java.ibjectInputStream.readOrdinaryObject(ObjectInputStream.java:2183)
    at java.base/java.ibjectInputStream.readObject0(ObjectInputStream.java:1707)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:517)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:475)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2578)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2518)
    at java.base/java.util.TreeMap.readObject(TreeMap.java:2465)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:564)
    at java.base/java.ibjectStreamClass.invokeReadObject(ObjectStreamClass.java:1216)
    at java.base/java.ibjectInputStream.readSerialData(ObjectInputStream.java:2381)
    at java.base/java.ibjectInputStream.readOrdinaryObject(ObjectInputStream.java:2215)
    at java.base/java.ibjectInputStream.readObject0(ObjectInputStream.java:1707)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:517)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:475)
    at java.base/java.util.ArrayList.readObject(ArrayList.java:899)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:564)
    at java.base/java.ibjectStreamClass.invokeReadObject(ObjectStreamClass.java:1216)
    at java.base/java.ibjectInputStream.readSerialData(ObjectInputStream.java:2381)
    at java.base/java.ibjectInputStream.readOrdinaryObject(ObjectInputStream.java:2215)
    at java.base/java.ibjectInputStream.readObject0(ObjectInputStream.java:1707)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:517)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:475)
    at net.sourceforge.kolmafia.persistence.AdventureQueueDatabase.deserialize(AdventureQueueDatabase.java:294)
    at net.sourceforge.kolmafia.session.LoginManager.initialize(LoginManager.java:225)
    at net.sourceforge.kolmafia.session.LoginManager.doLogin(LoginManager.java:137)
    at net.sourceforge.kolmafia.session.LoginManager.login(LoginManager.java:82)
    at net.sourceforge.kolmafia.request.LoginRequest.processLoginRequest(LoginRequest.java:405)
    at net.sourceforge.kolmafia.request.GenericRequest.handleServerRedirect(GenericRequest.java:2248)
    at net.sourceforge.kolmafia.request.GenericRequest.retrieveServerReply(GenericRequest.java:2124)
    at net.sourceforge.kolmafia.request.GenericRequest.externalExecute(GenericRequest.java:1672)
    at net.sourceforge.kolmafia.request.GenericRequest.execute(GenericRequest.java:1655)
    at net.sourceforge.kolmafia.request.GenericRequest.run(GenericRequest.java:1365)
    at net.sourceforge.kolmafia.request.LoginRequest.run(LoginRequest.java:266)
    at net.sourceforge.kolmafia.RequestThread.postRequest(RequestThread.java:300)
    at net.sourceforge.kolmafia.RequestThread.postRequest(RequestThread.java:250)
    at net.sourceforge.kolmafia.swingui.LoginFrame$LoginPanel.doLogin(LoginFrame.java:344)
    at net.sourceforge.kolmafia.swingui.LoginFrame$LoginPanel.actionConfirmed(LoginFrame.java:300)
    at net.sourceforge.kolmafia.swingui.panel.GenericPanel$ConfirmedListener.execute(GenericPanel.java:626)
    at net.sourceforge.kolmafia.swingui.listener.ThreadedListener.run(ThreadedListener.java:239)
    at net.sourceforge.kolmafia.RequestThread$SequencedRunnable.run(RequestThread.java:433)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)
 

gausie

D̰͕̝͚̤̥̙̐̇̑͗̒e͍͔͎͈͔ͥ̉̔̅́̈l̠̪̜͓̲ͧ̍̈́͛v̻̾ͤe͗̃ͥ̐̊ͬp̔͒ͪ
Don't worry about this, it should go away when you reset your turns_spent.ser file (i.e. ascend)

If anyone who is better with Java than me knows how to do this better please let me know!
 

djve

New member
I like the clean-up of not including unnecessary libraries, it reduces the technical debt burden.
 

djve

New member
I do, too. If it doesn't cause Invalid Class Exceptions.

Should it cause that? IDK, but it does.
I'm an old style sys admin (perl, python, haskell,...) so java isn't something I'm up on. But using UUID signatures to identify validity seems a little like trying to trim your nails by using a shot gun. It'll work but is the result what you're after.
 

MCroft

Developer
While I hate OO philosophically, I'll work with what I have to.
You're link makes the solution trivial, as the explanation also is concise and explicit!
Many thanks. Let's hope this is taken up buy the Mafia devs.
Well, I'm the greenest and most naive of Mafia devs, but here's the problem I see:

It looks like that's already implemented. And that number hasn't changed in 7 years. And it's not "all import statements cause serialization to be funky", because one was added 5 years ago to fix an issue. So hopefully a dev who knows why that throws that error can take a look.

Code:
public class AdventureQueueDatabase
       implements Serializable
{
       private static final long serialVersionUID = -180241952508113931L;

Note that AdventureTurnsDatabase does not set a serialVersionUID, but the stack trace above is for AdventureQueueDatabase.
 

fronobulax

Developer
If anyone cares to explain how removing an unused import caused this, I'd love to understand how and why. The failure mechanics I can come up with involve different versions of the same class in different jar files and that just doesn't seem likely.

The serialization issue is stirring some old memories from the days when REST APIs involved handwritten code and not libraries. I'll poke around but if anyone beats me, that's fine.
 

MCroft

Developer
If anyone cares to explain how removing an unused import caused this, I'd love to understand how and why. The failure mechanics I can come up with involve different versions of the same class in different jar files and that just doesn't seem likely.
Me, too. It'd be a learning experience.

What I see that causes me pause is that the error says
stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117


and AdventureQueueDatabase.java says private static final long serialVersionUID = -180241952508113931L;

So either
1: it never used the serialVersionUID that we set
AND changing that import changed one of the things used for auto-serialization
2: 4326148971984844054 == -180241952508113931L
3: Something else is wrong, like a bug in OpenJDK with one of those implementations that causes #1

These are not mutually exclusive.
The serialization issue is stirring some old memories from the days when REST APIs involved handwritten code and not libraries. I'll poke around but if anyone beats me, that's fine.
I've only checked it against OpenJDK 15, but I got the error without the import and not when I reverted that change. It's empirical engineering, based on "undo the last thing you did before you got the error".
 

fronobulax

Developer
What was the offending import?

We only serialize two classes, I added the serialVersionUID to the one that was lacking it. But I need some adventures to see if it behaves.
 

MCroft

Developer
What was the offending import?

We only serialize two classes, I added the serialVersionUID to the one that was lacking it. But I need some adventures to see if it behaves.
Java:
Michaels-MBP:persistence mcroft$ svn diff AdventureQueueDatabase.java
Index: AdventureQueueDatabase.java
===================================================================
--- AdventureQueueDatabase.java    (revision 20602)
+++ AdventureQueueDatabase.java    (working copy)
@@ -56,6 +56,8 @@
 import net.sourceforge.kolmafia.MonsterData;
 import net.sourceforge.kolmafia.RequestLogger;
+import net.sourceforge.kolmafia.combat.CombatActionManager;
+
 import net.sourceforge.kolmafia.preferences.Preferences;
 import net.sourceforge.kolmafia.request.FightRequest;
 

fronobulax

Developer
Java:
Michaels-MBP:persistence mcroft$ svn diff AdventureQueueDatabase.java
Index: AdventureQueueDatabase.java
===================================================================
--- AdventureQueueDatabase.java    (revision 20602)
+++ AdventureQueueDatabase.java    (working copy)
@@ -56,6 +56,8 @@
import net.sourceforge.kolmafia.MonsterData;
import net.sourceforge.kolmafia.RequestLogger;
+import net.sourceforge.kolmafia.combat.CombatActionManager;
+
import net.sourceforge.kolmafia.preferences.Preferences;
import net.sourceforge.kolmafia.request.FightRequest;

Duh. I should have been able to answer my own question. So we point a finger at CombatActionManager?
 

xKiv

Active member
Note that AdventureTurnsDatabase does not set a serialVersionUID, but the stack trace above is for AdventureQueueDatabase.
The error says RollingLikedList, though:
java.io.InvalidClassException: net.sourceforge.kolmafia.utilities.RollingLinkedList; local class incompatible: stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117
I've only checked it against OpenJDK 15, but I got the error without the import and not when I reverted that change. It's empirical engineering, based on "undo the last thing you did before you got the error".
Conjecture: the error went away because, after the first time, you exited kolmafia, which overwote the turns_spent.ser file.
Hypothesis: you will not see that error again no matter if you remove that import again.
 
Last edited:

MCroft

Developer
Duh. I should have been able to answer my own question.
literally 3 seconds in my sandbox. I was already in the directory.
So we point a finger at CombatActionManager?
So, Effective Java Chapter 11 says prevent default serialVersionUID collisions by setting serialVersionUID in your class that implements Serializable to avoid this.

Otherwise, it's gonna use classname and method signatures and private and protected instance fields in the class to auto-generate one.

But!! We didn't change any of those things and we did set a serialVersionUID.

So I don't know why it failed, and didn't fail in the past when we changed the imports (in 2015, per svn blame...)
 

MCroft

Developer
The error says RollingLikedList, though:
java.io.InvalidClassException: net.sourceforge.kolmafia.utilities.RollingLinkedList; local class incompatible: stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117
Yeah, but that class just extends java.util.LinkedList. No chance it caused that error. The RollingLinkedList is just the data type that the data is being deserialized into. That ever exists explicitly and only in the scope of deserialization with a UID mismatch.

In any case, RollingLinkedList didn't change...

List<TreeMap<String, RollingLinkedList<String>>> queues = (List<TreeMap<String, RollingLinkedList<String>>>) in.readObject();

The next error line that isn't java is
at net.sourceforge.kolmafia.persistence.AdventureQueueDatabase.deserialize(AdventureQueueDatabase.java:294), which does implement Serializable.
Conjecture: the error went away because, after the first time, you exited kolmafia, which overwote the turns_spent.ser file.
Hypothesis: you will not see that error again no matter if you remove that import again.
could be. I haven't experimented with it, but it seems like it shouldn't have been broken before.
 

fronobulax

Developer
I added serialVersionUID in r20604 so it is now in both the classes that implement Serializable.

The change didn't break serialization of files on my system but that could be tester error. I was prepared to see an error when files written with the computed value of serialVersionUID were read with the assigned value.

I commend to @djve and others Effective Java by Joshua Bloch, especially the chapter on serialization. That pretty much explains what problem serialization was designed to address, why it seemed like a good idea at the time and all the reasons why it should not be used in new systems and be replaced in legacy code. It then has numerous guidelines about how to serialize safely if that remains the choice. I'm trying to decide whether I want to confirm our implementation is safe, according to Bloch, replace it with some other persistence mechanic, of which JSON seems to be the most likely or just sit tight knowing there is another shiny object just around the corner to distract me.

If you are both curious and cheap then "free" PDF versions of the book are available with a little searching.
 

xKiv

Active member
Yeah, but that class just extends java.util.LinkedList. No chance it caused that error. The RollingLinkedList is just the data type that the data is being deserialized into. That ever exists explicitly and only in the scope of deserialization with a UID mismatch.

In any case, RollingLinkedList didn't change...
Yet, that's the class whose serialVersionUID changed. Possibly "since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations and can produce different serialVersionUID in different environments." (like a different JDK could be)
 

MCroft

Developer
I don't understand what you're suggesting. RollingLinkedList doesn't implement Serializable, does it? And I know I didn't change JDKs. Can you show me exactly what you think happened.
 
Top