Bug InvalidClassException for Mafia r20602 and openjdk 14.0.2

djve

djve

New member
This is more of an FYI should people be interested in this version of JDK. If it's not going to be fixed please mark it as such.

This started today with the new jar file. 20597 did not have this issue.


:~/KoL/Mafia$ java --version
openjdk 14.0.2 2020-07-14
OpenJDK Runtime Environment (build 14.0.2+12-Ubuntu-120.04)
OpenJDK 64-Bit Server VM (build 14.0.2+12-Ubuntu-120.04, mixed mode, sharing)

$ /usr/bin/java -Dsun.java2d.uiScale=2.6 -jar ./KoLmafia-20602.jar
KoLmafia v20.7 r20602
Released on July 30, 2020

Currently Running on Linux
Local Directory is /home/d/.kolmafia
Using Java 14.0.2

Code: 
java.io.InvalidClassException: net.sourceforge.kolmafia.utilities.RollingLinkedList; local class incompatible: stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117
    at java.base/java.ibjectStreamClass.initNonProxy(ObjectStreamClass.java:715)
    at java.base/java.ibjectInputStream.readNonProxyDesc(ObjectInputStream.java:2021)
    at java.base/java.ibjectInputStream.readClassDesc(ObjectInputStream.java:1890)
    at java.base/java.ibjectInputStream.readOrdinaryObject(ObjectInputStream.java:2183)
    at java.base/java.ibjectInputStream.readObject0(ObjectInputStream.java:1707)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:517)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:475)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2578)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2561)
    at java.base/java.util.TreeMap.buildFromSorted(TreeMap.java:2518)
    at java.base/java.util.TreeMap.readObject(TreeMap.java:2465)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:564)
    at java.base/java.ibjectStreamClass.invokeReadObject(ObjectStreamClass.java:1216)
    at java.base/java.ibjectInputStream.readSerialData(ObjectInputStream.java:2381)
    at java.base/java.ibjectInputStream.readOrdinaryObject(ObjectInputStream.java:2215)
    at java.base/java.ibjectInputStream.readObject0(ObjectInputStream.java:1707)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:517)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:475)
    at java.base/java.util.ArrayList.readObject(ArrayList.java:899)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:564)
    at java.base/java.ibjectStreamClass.invokeReadObject(ObjectStreamClass.java:1216)
    at java.base/java.ibjectInputStream.readSerialData(ObjectInputStream.java:2381)
    at java.base/java.ibjectInputStream.readOrdinaryObject(ObjectInputStream.java:2215)
    at java.base/java.ibjectInputStream.readObject0(ObjectInputStream.java:1707)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:517)
    at java.base/java.ibjectInputStream.readObject(ObjectInputStream.java:475)
    at net.sourceforge.kolmafia.persistence.AdventureQueueDatabase.deserialize(AdventureQueueDatabase.java:294)
    at net.sourceforge.kolmafia.session.LoginManager.initialize(LoginManager.java:225)
    at net.sourceforge.kolmafia.session.LoginManager.doLogin(LoginManager.java:137)
    at net.sourceforge.kolmafia.session.LoginManager.login(LoginManager.java:82)
    at net.sourceforge.kolmafia.request.LoginRequest.processLoginRequest(LoginRequest.java:405)
    at net.sourceforge.kolmafia.request.GenericRequest.handleServerRedirect(GenericRequest.java:2248)
    at net.sourceforge.kolmafia.request.GenericRequest.retrieveServerReply(GenericRequest.java:2124)
    at net.sourceforge.kolmafia.request.GenericRequest.externalExecute(GenericRequest.java:1672)
    at net.sourceforge.kolmafia.request.GenericRequest.execute(GenericRequest.java:1655)
    at net.sourceforge.kolmafia.request.GenericRequest.run(GenericRequest.java:1365)
    at net.sourceforge.kolmafia.request.LoginRequest.run(LoginRequest.java:266)
    at net.sourceforge.kolmafia.RequestThread.postRequest(RequestThread.java:300)
    at net.sourceforge.kolmafia.RequestThread.postRequest(RequestThread.java:250)
    at net.sourceforge.kolmafia.swingui.LoginFrame$LoginPanel.doLogin(LoginFrame.java:344)
    at net.sourceforge.kolmafia.swingui.LoginFrame$LoginPanel.actionConfirmed(LoginFrame.java:300)
    at net.sourceforge.kolmafia.swingui.panel.GenericPanel$ConfirmedListener.execute(GenericPanel.java:626)
    at net.sourceforge.kolmafia.swingui.listener.ThreadedListener.run(ThreadedListener.java:239)
    at net.sourceforge.kolmafia.RequestThread$SequencedRunnable.run(RequestThread.java:433)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)
 
gausie

gausie

D̰͕̝͚̤̥̙̐̇̑͗̒e͍͔͎͈͔ͥ̉̔̅́̈l̠̪̜͓̲ͧ̍̈́͛v̻̾ͤe͗̃ͥ̐̊ͬp̔͒ͪ
Staff member
Don't worry about this, it should go away when you reset your turns_spent.ser file (i.e. ascend)

If anyone who is better with Java than me knows how to do this better please let me know!
 
djve

djve

New member
I like the clean-up of not including unnecessary libraries, it reduces the technical debt burden.
 
MCroft

MCroft

Developer
Staff member
djve said:
I like the clean-up of not including unnecessary libraries, it reduces the technical debt burden.
Click to expand...
I do, too. If it doesn't cause Invalid Class Exceptions.

Should it cause that? IDK, but it does.
 
djve

djve

New member
MCroft said:
I do, too. If it doesn't cause Invalid Class Exceptions.

Should it cause that? IDK, but it does.
Click to expand...
I'm an old style sys admin (perl, python, haskell,...) so java isn't something I'm up on. But using UUID signatures to identify validity seems a little like trying to trim your nails by using a shot gun. It'll work but is the result what you're after.
 
MCroft

MCroft

Developer
Staff member
MCroft

MCroft

Developer
Staff member
djve said:
While I hate OO philosophically, I'll work with what I have to.
You're link makes the solution trivial, as the explanation also is concise and explicit!
Many thanks. Let's hope this is taken up buy the Mafia devs.
Click to expand...
Well, I'm the greenest and most naive of Mafia devs, but here's the problem I see:

It looks like that's already implemented. And that number hasn't changed in 7 years. And it's not "all import statements cause serialization to be funky", because one was added 5 years ago to fix an issue. So hopefully a dev who knows why that throws that error can take a look.

Code: 
public class AdventureQueueDatabase
       implements Serializable
{
       private static final long serialVersionUID = -180241952508113931L;

Note that AdventureTurnsDatabase does not set a serialVersionUID, but the stack trace above is for AdventureQueueDatabase.
 
fronobulax

fronobulax

Developer
Staff member
If anyone cares to explain how removing an unused import caused this, I'd love to understand how and why. The failure mechanics I can come up with involve different versions of the same class in different jar files and that just doesn't seem likely.

The serialization issue is stirring some old memories from the days when REST APIs involved handwritten code and not libraries. I'll poke around but if anyone beats me, that's fine.
 
MCroft

MCroft

Developer
Staff member
fronobulax said:
If anyone cares to explain how removing an unused import caused this, I'd love to understand how and why. The failure mechanics I can come up with involve different versions of the same class in different jar files and that just doesn't seem likely.
Click to expand...
Me, too. It'd be a learning experience.

What I see that causes me pause is that the error says stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117

and AdventureQueueDatabase.java says private static final long serialVersionUID = -180241952508113931L;

So either
1: it never used the serialVersionUID that we set
AND changing that import changed one of the things used for auto-serialization
2: 4326148971984844054 == -180241952508113931L
3: Something else is wrong, like a bug in OpenJDK with one of those implementations that causes #1

These are not mutually exclusive.
fronobulax said:
The serialization issue is stirring some old memories from the days when REST APIs involved handwritten code and not libraries. I'll poke around but if anyone beats me, that's fine.
Click to expand...
I've only checked it against OpenJDK 15, but I got the error without the import and not when I reverted that change. It's empirical engineering, based on "undo the last thing you did before you got the error".
 
fronobulax

fronobulax

Developer
Staff member
What was the offending import?

We only serialize two classes, I added the serialVersionUID to the one that was lacking it. But I need some adventures to see if it behaves.
 
MCroft

MCroft

Developer
Staff member
fronobulax said:
What was the offending import?

We only serialize two classes, I added the serialVersionUID to the one that was lacking it. But I need some adventures to see if it behaves.
Click to expand...
Java: 
Michaels-MBP:persistence mcroft$ svn diff AdventureQueueDatabase.java
Index: AdventureQueueDatabase.java
===================================================================
--- AdventureQueueDatabase.java    (revision 20602)
+++ AdventureQueueDatabase.java    (working copy)
@@ -56,6 +56,8 @@
 import net.sourceforge.kolmafia.MonsterData;
 import net.sourceforge.kolmafia.RequestLogger;
+import net.sourceforge.kolmafia.combat.CombatActionManager;
+
 import net.sourceforge.kolmafia.preferences.Preferences;
 import net.sourceforge.kolmafia.request.FightRequest;
 
fronobulax

fronobulax

Developer
Staff member
MCroft said:
Java: 
Michaels-MBP:persistence mcroft$ svn diff AdventureQueueDatabase.java
Index: AdventureQueueDatabase.java
===================================================================
--- AdventureQueueDatabase.java    (revision 20602)
+++ AdventureQueueDatabase.java    (working copy)
@@ -56,6 +56,8 @@
import net.sourceforge.kolmafia.MonsterData;
import net.sourceforge.kolmafia.RequestLogger;
+import net.sourceforge.kolmafia.combat.CombatActionManager;
+
import net.sourceforge.kolmafia.preferences.Preferences;
import net.sourceforge.kolmafia.request.FightRequest;
Click to expand...

Duh. I should have been able to answer my own question. So we point a finger at CombatActionManager?
 
xKiv

xKiv

Active member
MCroft said:
Note that AdventureTurnsDatabase does not set a serialVersionUID, but the stack trace above is for AdventureQueueDatabase.
Click to expand...
The error says RollingLikedList, though:
java.io.InvalidClassException: net.sourceforge.kolmafia.utilities.RollingLinkedList; local class incompatible: stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117
I've only checked it against OpenJDK 15, but I got the error without the import and not when I reverted that change. It's empirical engineering, based on "undo the last thing you did before you got the error".
Click to expand...
Conjecture: the error went away because, after the first time, you exited kolmafia, which overwote the turns_spent.ser file.
Hypothesis: you will not see that error again no matter if you remove that import again.
 
Last edited:
MCroft

MCroft

Developer
Staff member
fronobulax said:
Duh. I should have been able to answer my own question.
Click to expand...
literally 3 seconds in my sandbox. I was already in the directory.
fronobulax said:
So we point a finger at CombatActionManager?
Click to expand...
So, Effective Java Chapter 11 says prevent default serialVersionUID collisions by setting serialVersionUID in your class that implements Serializable to avoid this.

Otherwise, it's gonna use classname and method signatures and private and protected instance fields in the class to auto-generate one.

But!! We didn't change any of those things and we did set a serialVersionUID.

So I don't know why it failed, and didn't fail in the past when we changed the imports (in 2015, per svn blame...)
 
MCroft

MCroft

Developer
Staff member
xKiv said:
The error says RollingLikedList, though:
java.io.InvalidClassException: net.sourceforge.kolmafia.utilities.RollingLinkedList; local class incompatible: stream classdesc serialVersionUID = 4326148971984844054, local class serialVersionUID = 372957009574810117
Click to expand...
Yeah, but that class just extends java.util.LinkedList. No chance it caused that error. The RollingLinkedList is just the data type that the data is being deserialized into. That ever exists explicitly and only in the scope of deserialization with a UID mismatch.

In any case, RollingLinkedList didn't change...

List<TreeMap<String, RollingLinkedList<String>>> queues = (List<TreeMap<String, RollingLinkedList<String>>>) in.readObject();

The next error line that isn't java is
at net.sourceforge.kolmafia.persistence.AdventureQueueDatabase.deserialize(AdventureQueueDatabase.java:294), which does implement Serializable.
xKiv said:
Conjecture: the error went away because, after the first time, you exited kolmafia, which overwote the turns_spent.ser file.
Hypothesis: you will not see that error again no matter if you remove that import again.
Click to expand...
could be. I haven't experimented with it, but it seems like it shouldn't have been broken before.
 
fronobulax

fronobulax

Developer
Staff member
I added serialVersionUID in r20604 so it is now in both the classes that implement Serializable.

The change didn't break serialization of files on my system but that could be tester error. I was prepared to see an error when files written with the computed value of serialVersionUID were read with the assigned value.

I commend to @djve and others Effective Java by Joshua Bloch, especially the chapter on serialization. That pretty much explains what problem serialization was designed to address, why it seemed like a good idea at the time and all the reasons why it should not be used in new systems and be replaced in legacy code. It then has numerous guidelines about how to serialize safely if that remains the choice. I'm trying to decide whether I want to confirm our implementation is safe, according to Bloch, replace it with some other persistence mechanic, of which JSON seems to be the most likely or just sit tight knowing there is another shiny object just around the corner to distract me.

If you are both curious and cheap then "free" PDF versions of the book are available with a little searching.
 
xKiv

xKiv

Active member
MCroft said:
Yeah, but that class just extends java.util.LinkedList. No chance it caused that error. The RollingLinkedList is just the data type that the data is being deserialized into. That ever exists explicitly and only in the scope of deserialization with a UID mismatch.

In any case, RollingLinkedList didn't change...
Click to expand...
Yet, that's the class whose serialVersionUID changed. Possibly "since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations and can produce different serialVersionUID in different environments." (like a different JDK could be)
 
MCroft

MCroft

Developer
Staff member
I don't understand what you're suggesting. RollingLinkedList doesn't implement Serializable, does it? And I know I didn't change JDKs. Can you show me exactly what you think happened.
 
You must log in or register to reply here.
Top