Bug - Fixed nowToInt() vs. now_to_int()

MCroft

Developer
Staff member
The original symptom was that the javascript function nowToInt() was returning a negative number.

We determined that it was the 32 bit 2s complement representation of the 41 bit epoch time in milliseconds.

I tried a fix and nobody liked it, including me. I'd like input on the right way to approach this.

The current test cases look like this:
Rich (BB code):
> js nowToInt();

Returned: -1680943482

> ash now_to_int();

Returned: 1677653081709

Things I observed:
  1. The ash function is named now_to_int(), but it is returning a long and Ash allows it.
  2. Ash doesn't have a Long as a dataType (unless the wiki is out of date...): https://wiki.kolmafia.us/index.php/Data_Types#int
  3. now_to_int is returning milliseconds. While I don't know if it should return milliseconds, that's what's overflowing. If it was seconds, it would be safe until 2038. As it is, it can only return a long (or larger, but...).
  4. JavaScript has to go through ValueConverter.asJava() and ValueConverter.fromJava() to get the right value out.
  5. now_to_int() is defined as having DataType.INT_TYPE, returns a Value, and calls a Calendar function that returns a Long
  6. ValueConverter.asJava() returns the long value as expected, but it says it's a type int.
  7. ValueConverter.fromJava() converts it to an Integer and that becomes the 2s complement.
What I attempted at first was to write code that detected if the value in asJava() was greater than 2^31-1 and returned a BigInteger. This fixes the return value for most cases, but has problems:
  1. Nobody loves a polymorphic data type (even if it is predictable).
  2. It's not an int, so now_to_int() isn't really now_to_int(), it's now_to_long(). It's never been _to_int() if it returns time in milliseconds and int is a 2s complement data type.
  3. If something other than a time function had a 2.1B value, it would unexpectedly get a type that JavaScript can't coerce into a number. This will cause JS scripts that are currently getting a wrong value to get a data mismatch.
I'm currently thinking the return in ValueConverter.fromJava() is where we are getting the 32bit conversion.

Rich (BB code):
  public Value fromJava(Object object, Type typeHint) {
    if (object == null) return null;
    else if (object instanceof Boolean) {
      return DataTypes.makeBooleanValue((Boolean) object);
    } else if (object instanceof Float || object instanceof Double) {
      return DataTypes.makeFloatValue(((Number) object).floatValue());
    } else if (object instanceof Byte
        || object instanceof Short
        || object instanceof Integer
        || object instanceof Long) {
      return DataTypes.makeIntValue(((Number) object).intValue());
 
Last edited:

Veracity

Developer
Staff member
Ash doesn't have a Long as a dataType (unless the wiki is out of date...): https://wiki.kolmafia.us/index.php/Data_Types#int

The ASH "int" datatype is an "integer".
It started out as a Java "int", but, many years ago, we switched to using a Java "long".
It is still an "integer".

Similarly, the ASH "float" is a "floating point number".
It started out as a Java "float", but, many years ago, we switched to using a Java "double".
It is still a "floating point number".

Which is to say, ASH has two kinds of numbers - integers and floating point - each of which uses 64 bits of data.
We call them "int" and "float", since those are familiar keywords.

Unlike Java, ASH doesn't let you specify how much memory is consumed by a particular number.

The Wiki page says ASH numbers are 32 bits.
It is out of date. We should update it.
 

fronobulax

Developer
Staff member
The original symptom was that the javascript function nowToInt() was returning a negative number.

I'm missing the context that makes this a KoLmafia problem and not a JavaScript problem. There are places where KoLmafia does things to avoid known Rhino bugs so there is precedent for a KoLmafia "fix", but not fixing and making Rhino deal with it is also an option.

The resurgence of interest in now_to_int seems to be driven by scripting and a preference for particular ways of manipulating times. If scritpters are told this issue cannot or will not be addressed, is there something that could be done by or added to mafia that made a workaround acceptable?

A long time ago in a world where memory was scarce and precious "unsigned" was occasionally a defined data type. Would limited ash support for "unsigned" or "non-negative integer" be a step towards resolution?
 

MCroft

Developer
Staff member
Code:
return DataTypes.makeIntValue(((Number) object).intValue());
Changed to return DataTypes.makeIntValue(((Number) object).longValue());

Now it returns a Long to JS.
Code:
> ash now_to_int();

Returned: 1677680054630

> js nowToInt();

Returned: 1677680061644
 

fronobulax

Developer
Staff member
@fronobulax , my new understanding is that valueConverter.fromJava() (Java code on our side) is coercing all numbers to integers, instead of longs.

If that is the case then I interpret veracity's comments as suggesting that is an error because "The ASH "int" datatype is an "integer".
It started out as a Java "int", but, many years ago, we switched to using a Java "long".

What breaks if valueConverter.fromJava() coerces Java ints to longs?
 

MCroft

Developer
Staff member
I have two concerns with my change.
1: anyone who was relying on the wrong behavior.
2: longs greater the JavaScripts MAX_SAFE_INTEGER

Number one I don’t feel like should be a problem.

Number two could bite JS, but we’ve moved the problem from 32 bits+ to 53 bits+.
 

xKiv

Active member
I might argue that this is a Bug or feature request for Rhino.

Apparently, javascript now has an arbitrary precision decimal type?
Try in browser console the differences between these (the ones with n are "bigint" literals, according to typeof):

Code:
10000000000000000+1
10000000000000000n+1n

So if Rhino doesn't support that yet, that's probably a reasonable feature request thing.
But then the problem becomes that you might not be able to pass values back from javascript to mafia.
And also teaching people how to work with bigints in javascript, because they don't just mix with any other numbers.
 

MCroft

Developer
Staff member
well, I went to merge it, it required an update and invalidated the approval.

Also, it seems to be stuck waiting for tests that it says aren't being run.

Can I get some help getting this over the finish line?
 
This appears to have introduced a bug (or at least a meaningful behavior change) in retrievePrice and possibly other functions:

r27254 (just before the change)
> js retrievePrice(Item.get("the one mood ring"))

Returned: -1
r27255
> js retrievePrice(Item.get("the one mood ring"))

Returned: 9223372036854775807

This does appear to be more consistent with ash:
> ash retrieve_price($item[the one mood ring])

Returned: 9223372036854775807
but given that -1 is a less silly-looking number than the -1680943482 encountered in nowToInt (as well as the fact that -1 is more consistent with the behavior of mallPrice, which returns -1 for items like the hand turkey outline which are theoretically obtainable in the mall but are absent from it and 0 for straight-up untradeable items).

Actually, upon finishing writing this, I guess it would be wrong to call this a bug given that it is now more consistent with ash's behavior for retrieve_price. But this is still probably worth announcing, in case someone else runs into it.

One further comment I have is that the returned value is greater than the maximum safe integer in javascript, which feels less-than-ideal
 
Last edited:

fronobulax

Developer
Staff member
This appears to have introduced a bug (or at least a meaningful behavior change) in retrievePrice and possibly other functions:

r27254 (just before the change)

r27255


This does appear to be more consistent with ash:

but given that -1 is a less silly-looking number than the -1680943482 encountered in nowToInt (as well as the fact that -1 is more consistent with the behavior of mallPrice, which returns -1 for items like the hand turkey outline which are theoretically obtainable in the mall but are absent from it and 0 for straight-up untradeable items).

Actually, upon finishing writing this, I guess it would be wrong to call this a bug given that it is now more consistent with ash's behavior for retrieve_price. But this is still probably worth announcing, in case someone else runs into it.

One further comment I have is that the returned value is greater than the maximum safe integer in javascript, which feels less-than-ideal

I'm not sure whether you are telling us something new or merely highlighting information that was scattered over several posts.
 

MCroft

Developer
Staff member
I asked @phreddrickk to mention it in the forum even if he didn't think this was a bug so that others might find it in a search.

In ASH, that's returning 9223372036854775807 or 2^63-1. While I cannot read the mind of the long-ago developer of it, it is a reasonable value, because using MAXINT for the value of an unbuyable item allows simple comparisons to work, like if (CashOnHand < retrievePrice (expensiveThing) ) { fail }

It is worth considering what to do with that in JS, since it is larger than the safe JS Integer Max. Sadly, BigInt does not smoothly convert (we tried that for now_to_int() ), so that's not a simple solution.
 
Last edited:
Top