Make your scripts automatically update their data files!

zarqon

zarqon

Well-known member
When I recently discovered that file_to_map() works with URLs, I had this idea:

"It would be wicked cool if scripts could automatically keep their data files updated by accessing some kind of central repository of map files!" (I think in italics.)
Click to expand...

Several hours and two scripts later, I am pleased to introduce the [size=+1]Mafia Map Manager[/size]!

Basically, the idea is a single location for map files to live, where they can be updated by script authors as well as contributors, and a method for scripts to access those files. When running a script, it checks to see if your data file is the current version. If not, it downloads the current version, replacing your old version.

Benefits:
  • The obvious one: the user has no need to update data files! This translates to a lot less clicking and happier users.
  • Fewer first-time downloads for the user. The user can just download the script and run it, without needing to download data libraries along with it.
  • Central location means improvements made by users are more likely to be shared.
  • Changes uploaded by one person will instantly propagate to all users. This has the benefit of providing instant bugfixes to everyone as soon as one user uploads a fixed data file. This would have come in handy when the location name for the Haunted Cellar changed from (Random) to (Automatic), and the bounty.txt file (used by bounty.ash) was causing the script to have errors.
  • As a result of the above, fewer errors encounted by users.

I hear what you're saying. You're saying "Enough already! I want to convert my script to do this!!" So here you go.

How To Make Your Script Automatically Update Its Data Files

  • First, upload the data file (xxxx.txt) used by your script here.
  • Next, import ZLib in your script as follows:

Code: 
import <zlib.ash>

This must be top-level (recommend putting it at the top of your script, since it also makes numerous other handy functions available).

  • Then, instead of calling file_to_map() in your script, call load_current_map(). The first parameter is the name of the file without the extension (e.g. "checklist" not "checklist.txt"). The function will return true if the map loaded successfully.
  • Remove the data file from this site. Users will no longer need to come here to download data file updates.

Enjoy this awesome new functionality!

~Z~
 
Last edited:
I just wanted to say that I'm giving this resource a thumbs-up.

It was awfully convenient to simply be able to update the map for my recovery script by just uploading the new version to the server as soon as mafia supported sugar shards. That isn't the big thing though. The really big thing is that the proper recovery range for sugar shards hadn't yet been fully spaded and that was not a reason to hold off on updating. I simply added the currently known data, even though it was doubtless incorrect. Then when the recovery range was correct, I just updated it again.

Without this service I would have refrained from adding the sugar shard until the data was fully spaded because it would have been troublesome to have two updates. Using this service changed the way I updated the map to be more responsive to my users so they could use the new recovery item immediately.

Thanks Zarqon.
 
Cheers Bale! Warm fuzzies all around! Glad you found it useful.

If only there were a way to write functions that take maps of any structure, this would definitely be in ZLib.
 
zarqon said:
If only there were a way to write functions that take maps of any structure, this would definitely be in ZLib.
Click to expand...
That's... ah. Yeah, there's no way to write the function without knowing how to define the map... If only you could pass the map's name as a string and then run map_to_file() on the map's name...

To do that, we'd either need a version of map_to_file() and file_to_map() that accepted a string as the name of the map or else be able to call a variable name like, map_to_file(call mapname, "fname.txt")

PS. for information on the obscure call command, see here. Obviously it doesn't work at all as we'd need it for this purpose.
 
Last edited:
I just uploaded a file (use_for_items.txt, designed to be a float [item] [item] map) of which a few lines have some quite small float values:

In the updated-from-Map-Manager file, they look like this:
tiny slimy cyst big bumboozer marble 4.8828125E-4
tiny slimy cyst black catseye marble 9.765625E-4

In the original file, they looked like this:
tiny slimy cyst big bumboozer marble 0.00048828125
tiny slimy cyst black catseye marble .0009765625

The values are preserved, just translated into E notation.

However, in using load_current_map() to load these, Mafia errors and prints a debug log (attached). The relevant portion seems to be:

class java.lang.NumberFormatException: For input string: "4.8828125-4"
java.lang.NumberFormatException: For input string: "4.8828125-4"
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
at java.lang.Float.parseFloat(Float.java:394)

Something in file_to_map() doesn't like to see that notation.

Is this a Mafia bug (file_to_map not accepting floats in scientific notation)? Or can I ask you to force the Map Manager not to change the representation of floats in files that it manages?
 

Attachments

This is mafia's fault. (Also recently came up in the SmartStasis thread.) I'd say it's worthy of a bug report, since for very small floats this actually prevents you from using floats.
 
Zarqon, I'm trying to use your script to get back a string[int][string][string] map I've made, but it just gives me the first entry and then an unexpected error (included here in case it makes it obvious to everyone).

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
KoLmafia r7859, Windows XP, Java 1.6.0_17
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Please note: do not post these logs in the KoLmafia thread. If
you would like us to look at the log, please instead email logs
to veracity@hambo.com using the subject "KoLmafia Debug Log"
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Timestamp: Mon Dec 21 21:46:08 GMT 2009
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=


Unexpected error, debug log printed.
class java.lang.ArrayIndexOutOfBoundsException: 3
java.lang.ArrayIndexOutOfBoundsException: 3
at net.sourceforge.kolmafia.textui.parsetree.CompositeValue.read(CompositeValue.java:133)
at net.sourceforge.kolmafia.textui.parsetree.CompositeValue.read(CompositeValue.java:147)
at net.sourceforge.kolmafia.textui.parsetree.CompositeValue.read(CompositeValue.java:147)
at net.sourceforge.kolmafia.textui.RuntimeLibrary.readMap(RuntimeLibrary.java:3543)
at net.sourceforge.kolmafia.textui.RuntimeLibrary.file_to_map(RuntimeLibrary.java:3515)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at net.sourceforge.kolmafia.textui.parsetree.LibraryFunction.execute(LibraryFunction.java:119)
at net.sourceforge.kolmafia.textui.parsetree.FunctionCall.execute(FunctionCall.java:156)
at net.sourceforge.kolmafia.textui.parsetree.BasicScope.execute(BasicScope.java:451)
at net.sourceforge.kolmafia.textui.parsetree.UserDefinedFunction.execute(UserDefinedFunction.java:127)
at net.sourceforge.kolmafia.textui.parsetree.FunctionCall.execute(FunctionCall.java:156)
at net.sourceforge.kolmafia.textui.parsetree.BasicScope.execute(BasicScope.java:451)
at net.sourceforge.kolmafia.textui.parsetree.UserDefinedFunction.execute(UserDefinedFunction.java:127)
at net.sourceforge.kolmafia.textui.Interpreter.executeScope(Interpreter.java:252)
at net.sourceforge.kolmafia.textui.Interpreter.execute(Interpreter.java:191)
at net.sourceforge.kolmafia.textui.Interpreter.execute(Interpreter.java:184)
at net.sourceforge.kolmafia.textui.command.CallScriptCommand.call(CallScriptCommand.java:193)
at net.sourceforge.kolmafia.textui.command.CallScriptCommand.run(CallScriptCommand.java:63)
at net.sourceforge.kolmafia.KoLmafiaCLI.executeCommand(KoLmafiaCLI.java:473)
at net.sourceforge.kolmafia.KoLmafiaCLI.executeLine(KoLmafiaCLI.java:366)
at net.sourceforge.kolmafia.swingui.CommandDisplayFrame$CommandQueueHandler.handleQueue(CommandDisplayFrame.java:191)
at net.sourceforge.kolmafia.swingui.CommandDisplayFrame$CommandQueueHandler.run(CommandDisplayFrame.java:172)
Click to expand...

I get this with the latest and the last stable (v7859). Also, is there a way to view the maps in our browsers, i.e. not through kolmafia, to remind us exactly what we've uploaded?
 
Stable in the sense that it was the last daily before the big changes to the code that caused a lot of the "Unexpected Errors" happened. It also happens on 13.7.exe.
 
bumcheekcity said:
Also, is there a way to view the maps in our browsers, i.e. not through kolmafia, to remind us exactly what we've uploaded?
Click to expand...

Another feature request: would it be possible to have some kind of diff display? It's great that anyone can update the maps, but it would be very helpful to know what, exactly, has been updated on "my" maps when I'm not the one who's updated them. Or what's been updated for any updated map, really.

(Sure, an intelligent and foresightful person would keep separate copies so the auto-overwritten version could be compared to older versions... but apparently I am not that person.)

Then again, this might be more storage and fancification than you want to put into this project, which would be understandable -- it's already very useful as-is.
 
@bumcheek: If you enter the URL used by mafia you can see your map:

http://zachbardon.com/mafiatools/autoupdate.php?f=turtles&act=getmap

Note that you would replace "turtles" with the name of your map, sans extension. When I get back home (I'm visiting my parents for the holidays) I'll turn the filenames on the website into the corresponding links.

@aqualectrix: Sorry, that's a lot of work that I don't want to do. I'm fine with information being free (to modify without notice). For what it's worth, I don't have backups of any of my data files either! (However, that's because I know the server's admin and he does daily backups so I could always revert any file on my server to whatever date I want.)
 
That's perfectly fine! (I'm also fine with it being modifiable by anyone, I just wondered what was being modified.) A lot of work you don't want to do -- that's work you shouldn't do, I'd say!
 
Ok, maps are viewable as text on the website now, without disclosing the actual file location. Yay.

I also have the function made that will go in ZLib to handle this! Next update will be a nice one!
 
The latest ZLib now contains load_current_map(), which accepts maps of any structure. This makes automatically updating data files in your scripts a ridiculously simple matter. w00t!
 
You must log in or register to reply here.
Back
Top