Postmortem: Attempt at source map support

This is a report of my failed foray into adding source map support for the JavaScript runtime. I'm putting this so that other people do not waste time like I did.

TL;DR: Due to limitations of Rhino, it is impossible to add source map support for JavaScript stack traces in KoLmafia.

Preamble

Modern JavaScript is rarely executed in the same form it is written. Although an interpreted language, JavaScript is frequently transformed--minified, transpiled, and bundled--so that programmers can use modern features while maintaining compatibility.

This becomes a problem with stack traces. Stack traces from transformed code is difficult to analyze, because the function names, line and column numbers from the executed JS code are different from the original code.

Enter source maps. These files contain information for reconstructing the original shape of JavaScript files. They do not affect normal program execution in any way. However, tools can use source maps to produce stack traces that point to the original code locations.

Rhino does not support source maps. Hence, I decided to implement support for them.

What I tried

All source map libraries take a code position (line and column numbers) from the generated script and produce a tuple of (file name, function name, line number, column number) for the original code. Thus, my goal was to:

1. Parse each stack trace message and extract the script path, line/column numbers
2. Use the script to determine the location of source map file
3. Load and parse the source map
4. Use the source map and the line/column numbers to determine the original code location
5. Append the original code location to the stack trace line

#1. JS Route

The first method uses JavaScript to parse source maps and decorate stack traces. In Node.js land, there is a JavaScript package named source-map-support which does this. It uses a non-standard feature called the Stack Trace API.

In Rhino, this feature is behind a flag. Enabling the flag is easy. However, the feature itself is incomplete: there is no built-in way to retrieve the stack trace generated by JavaScript from the Java side. I would have to retrieve the stack property on the error object.

Parsing the source map itself is tricky, because KoLmafia(Rhino) doesn't support several useful features available in Node.js--File I/O APIs, URLs, Buffers, etc. It would need lots of polyfills to work. It would also be slow, since most of the work is done by JavaScript.

With some encouragement from other devs, I turned to option #2.

#2. Java Route

I discovered a Java package named sourcemap made by Atlassian. Using it to parse source maps was easy. I even made a prototype to be integrated into KoLmafia.

The real problem arose when I tried to parse stack trace messages. It turns out that Rhino always uses 0 for the column number, regardless of where the exception was actually created! Since Rhino relies heavily on Java's built-in exception mechanism to generate its own stack traces, it only knows what Java knows. But Java doesn't know the column number of the location where an exception was thrown (see StackTraceElement). Thus, Rhino pretends to print a column number by always printing 0.

Without the column number, the source map is useless. This eliminates both route #1 and #2.

What now?

There are some ways to make JavaScript stack traces readable and debuggable:

• Avoid using transpilers and minifiers/obfuscators. In particular, don't mangle identifiers.
• Avoid bundling minified libraries. Try to bundle their source versions instead.
• If possible, prettify your code instead of minifying them.

Notes

All of the above applies only to JavaScript scripts directly executed by KoLmafia. If you're writing JavaScript that runs inside the web browser, it's perfectly fine to use source maps!

Interesting. Thank you.

Given the narrow scope of the problem - JavaScript executed by KoLmafia - and that, in turn, is generated by script authors, I might suggest a wiki page. The audience for the page would be people who have some basic familiarity with JavaScript and want to use it to automate KoLmafia. I imagine a section that starts with "If you want useful stack traces" and ends with a list of things to do and not to do.

My ignorance of the JavaScript ecosystem is vast but everything you mentioned sounds to me to be something that is done when packaging a script for distribution and execution in various environments. Code compactness, optimization, support for multiple browsers and protection against reverse engineering etc. don't really seem like worthwhile concerns for a KoLmafia script, at least when addressing the concern disables an otherwise useful feature.