Automatically download the newest daily build

[Rinn]

I wrote a small console program that will automatically parse http://builds.kolmafia.us/ and download the latest daily build if it doesn't exist locally. You could set this as a scheduled task, but make sure you don't hammer the server looking for updates.

The program will also create (or modify) the file KoLMafiaUpdate.log so you can review previous attempts at updating. Some sample log output:


---------------
10/28/2009 11:56:41 PM
Checking http://builds.kolmafia.us/ for the latest version of KoLMafia...
KoLmafia-7772.jar is the latest version
Attempting to download http://builds.kolmafia.us/KoLmafia-7772.jar...
Successfully downloaded http://builds.kolmafia.us/KoLmafia-7772.jar
Copying KoLmafia-7772.jar to KoLmafia-latest.jar
---------------
---------------
10/29/2009 12:02:51 AM
Checking http://builds.kolmafia.us/ for the latest version of KoLMafia...
KoLmafia-7772.jar is the latest version
KoLmafia-7772.jar exists, you have the latest version
---------------


http://dl.getdropbox.com/u/220774/KoLMafiaUpdate.exe (http://dl.getdropbox.com/u/220774/KoLMafiaUpdate.exe)
http://dl.getdropbox.com/u/220774/KoLMafiaUpdate.cs

Version 1.8 - Fix for not being able to parse the revision number because the filename has an M at the end. Also fixed the -h typo.
Version 1.7 - Fix for the build links sorting by string instead of revision number.
Version 1.6 - Add some additional exception handling.
Version 1.5 - Only delete kolmafia jar files.
Version 1.4 - Delete old .jars after updating. Passing in -d as a command line argument will disable this functionality.
Version 1.3 - Open KoLmafia-latest.jar automatically. Passing in -o as a command line argument will disable this functionality.
Version 1.2 - Don't copy if the download fails
Version 1.1 - Automatically make a copy of the latest version as KoLmafia-latest.jar
Version 1.0 - Release

[ankorite]

Very nice! On my computer I have a cron job set up to run this script once a day. It uses ant to download the source and compile from svn, then copies the .jar to my working dir and runs it for me. The beauty of this is it only downloads the changes to kolmafia so server hits are always low. I use xterm to run the .jar because it is tough to get a cron script to start something graphical, plus it gives me a terminal where I can see any errors that mafia may spew.


#!/bin/bash
kol_dir=~/kol
mafia_src=$kol_dir/kolmafia

cd $mafia_src
ant daily

rm $kol_dir/daily.jar
mv $mafia_src/dist/KoLmafia* $kol_dir/daily.jar

xterm -display :0 -e "/usr/bin/java -jar $kol_dir/daily.jar"

[Rinn]

Yeah that's also a good solution, but a lot of people probably don't want to have to set up everything necessary to compile the source.

[Heffed]

I personally prefer to compile the .jar myself, but I can see an automated download process being very attractive to a lot of users.

[zarqon]

Cool!

Any chance you could make this also run the newly downloaded mafia? Or in the case that it already exists, run that? It would be cool to have a single click automatically run the latest daily.

[Rinn]

Yeah that's actually pretty easy to do. One sec.

Okay done, passing in -o as a command line argument will disable automatically starting the jar.

[zarqon]

Thanks, this is awesome, I just ran it and it worked like a charm. A few things you should probably mention in your first post: 1) it's for Windows, 2) you need .NET 2.0, otherwise you get a "could not initialize properly" error, and 3) it should go in your mafia directory.

EDIT: Just one other thing, and then I think it would be perfect. It would be nice not to have to regularly delete old builds. Could you add an option for deleting all other jar files in the directory other than the current daily and KoLmafia-latest? Either that, or an option for auto-deleting kolmafia-buildnum after copying it to kolmafia-latest... in which case instead of checking file.exists, you'd have to parse the log to see if the latest is really the latest. I think the first option is probably easier.

[Rinn]

Yeah I can do that. I'd rather keep the copy of the latest version downloaded so you can know at a quick glance what version the latest.jar is without opening up the log, but deleting other .jars should be simple enough.

[That FN Ninja]

This is great. Thanks Rinn!

[Rinn]

Okay it'll delete old jar files now.

[zarqon]

Truly Awesome. You have saved me lots of clicking, and probably kolmafia.us server bandwidth (not fetching images/CSS, also avoiding cases of re-downloading because you're not sure if it's the latest or not).

[Muhandes]

Very cool indeed. Thank you for this simple, yet very useful, idea.

[Bors]

I love the idea of deleting all the old programs, however it seems to do good of a job

Deleting old version AscensionLogVisualizer 2.1.0.jar

Is there anyway it can just del old versions of mafia and not all jar files?

[Heffed]

Is there anyway it can just del old versions of mafia and not all jar files?

I haven't used this application so I don't know exactly how it functions, but would another directory help?

The AutoIt script that I wrote to compile my .jar also backs up and timestamps the previous .jar and puts it in another directory. (with the extension renamed to .bak) It then goes through and deletes all the backups that are older than 7 days. This makes it very easy to rollback to previous builds if necessary.

Perhaps something like this would work for this application? It may be making things a bit more complicated, but it seems more forgiving if the application was pointed to a directory that you can be assured only the files you want to would be removed.

[lostcalpolydude]

The change for that should be easy.

FileInfo[] dirFiles = dir.GetFiles("*.jar");
should be

FileInfo[] dirFiles = dir.GetFiles("KoLmafia*.jar");
At least I think so, I can't exactly check on this computer, probably.

[Rinn]

Yeah that's exactly it, I didn't expect anyone to have other jar files but here we are.

Okay new version up.

[Bors]

Works great! Thanks for the fix.

[Muhandes]

Could it be that when there is no new jar file, it wont run the existing one, or am I just imagining?

[Rinn]

If KoLmafia-latest.jar exists it should run it regardless. If this is the first time you've run this program and you already have the latest daily build downloaded, it's not going to make a copy as KoLmafia-latest.jar until it actually downloads something.

[Muhandes]

It started working today. I think there were some permission problems, I played with them. Anyway, problem solved.

[NCSorcerer]

Thanks very much! This is quite useful.

[Idran]

It's a minor thing, but it looks like if it runs when KoLMafia happens to be already running (I was away from the computer and forgot about the scheduled task), then it crashes with a fatal exception. It's not a huge deal since I can just run it manually again after closing out Mafia, but I thought I'd bring it to your attention anyway. (I mean, obviously you wouldn't be able to delete the JAR while it's running still, but I just mean have it close nicely rather than passing the exception.)

[IceColdFever]

This is beyond awesome! Thanks so much.

[Rinn]

It's a minor thing, but it looks like if it runs when KoLMafia happens to be already running (I was away from the computer and forgot about the scheduled task), then it crashes with a fatal exception. It's not a huge deal since I can just run it manually again after closing out Mafia, but I thought I'd bring it to your attention anyway. (I mean, obviously you wouldn't be able to delete the JAR while it's running still, but I just mean have it close nicely rather than passing the exception.) Yeah I know about this, I just haven't had any time to add the catch for the exception.

[Rinn]

I've added the additional exception handling.

[bumcheekcity]

Hmm. I'm executing this, it says "Checking builds.kolmafia for the latest version", then "Kolmafia-7798 is the latest version, then just stops executing. I'd like it to automatically open up the latest version of Mafia every single time after it executes.

Also, it seems to download very slowly. Is there something I need to do?

Edit: Oh, no. It just downloaded v7799 properly and opened it. Perhaps it was just being weird because there'd been an update recently. Well, it's working now.

[lostcalpolydude]

Hmm. I'm executing this, it says "Checking builds.kolmafia for the latest version", then "Kolmafia-7798 is the latest version, then just stops executing. I'd like it to automatically open up the latest version of Mafia every single time after it executes.

Also, it seems to download very slowly. Is there something I need to do?

Edit: Oh, no. It just downloaded v7799 properly and opened it. Perhaps it was just being weird because there'd been an update recently. Well, it's working now.

http://kolmafia.us/showpost.php?p=19496&postcount=19

[EdFox]

Holy smokes that is awesome. :) Many thanks for saving me having to check the page, download, then move it to my mafia dir from where firefox stashes the file.

[Azrane]

I found a typo. The first "-o" should be "-d" right? :)

[redwulf25_ci]

Attempted to run this and got an application error

"The application failed to initialize properly (0xc0000135). Click on OK to terminate the application."

[zarqon]

You need to install .NET to run this program. (see post #7)

[redwulf25_ci]

You need to install .NET to run this program. (see post #7)

Ah. Thought I'd installed that to run something else, but maybe that was a different computer.

[meowtiger]

my version of this is to grab the latest hourly build, unix based and runs from where ever you hold your kolmafia file





if [[ -a index.html ]] ;then
rm index.html
fi
rm KoLmafia-*

wget http://builds.kolmafia.us/
latest=$(grep KoLmafia- index.html | cut -c11-53 )
wget ${latest}
filenm=$(ls | grep KoL)
chmod 700 $filenm

[bazo0ka]

I've been getting the following exception since 4/27.

An error occurred while downloading data from http://builds.kolmafia.us/
Exception Message: Unable to connect to the remote server

I am able to manually download builds fine, though.

[matt.chugg]

I've been getting the following exception since 4/27.

An error occurred while downloading data from http://builds.kolmafia.us/
Exception Message: Unable to connect to the remote server

I am able to manually download builds fine, though.

just downloaded and tested (compiled myself) works fine for me, are you still getting the error ?

[bazo0ka]

just downloaded and tested (compiled myself) works fine for me, are you still getting the error ?

Yep. I think it might be related to some malware, because I just discovered I can't connect to Windows Update. Will edit post if I figure out the problem.

Edit: For some reason my LAN options were set to use a proxy. Turned that off and everything works fine. Yay! I love this program. :)

[dai_vernon]

Hi, guys. I made my own utility that does this, not knowing that this version existed. My version is written in Python 2, so it's more cross-platform than the .Net version (dealing with Mono on a Mac is a real pain). It works unmodified on Windows, Linux, and Mac OS X. I've provided a windows executable (for convenience) too.

Instructions:

Download the .exe file (32-bit Windows (http://mafiaupdater.googlecode.com/files/muinstaller.exe) or 64-bit Windows (http://mafiaupdater.googlecode.com/files/muinstaller64.exe)) or the .py file (http://mafiaupdater.googlecode.com/files/mafiaupdater.py) (Windows, Mac or Linux)

.exe users: Run the installer, and when you want to run mafia, run mafiaupdater.exe instead
.py users: When you want to run mafia, run mafiaupdater.py instead

That's it!

The script downloads the newest version of mafia and runs it for you. This way, you don't need to worry about KoL content not being in mafia yet, because the version you're running is always the newest!

Windows users using the .py file will need to install Python 2.whatever. The script doesn't appear to work with Python 3. (http://www.python.org). Mac and Linux users have it installed already.

[Theraze]

Two potential tweaks. One, the typo that Azrane found is still there, namely this:
Command Line Arguments:
-o: Don't automatically delete old versions after updating
-o: Don't automatically open KoLmafia-latest.jar after updating
-h: Show this help message
Press any key to exit.
Most people don't make two different things to use the same argument... though I suppose it's possible that "-o" both stops deleting old files and tells it not to open. My guess though is that it actually wants -d: for the "don't delete" choice.

If possible, throwing -? for help as well would be nice. I only found -h after going through the compiled file with a hex editor. Well, not even that... I was lazy and used notepad, but the concept is still the same. :D For that matter, if possible, would be nice for / to work like - as well, so you could do /h or /? for your help, and /d or /o for the changed behaviour... but both of those freqs aren't really necessary, just there to make people used to old DOS command line flags less confused. :)

[Fryguy9]

I know this has been done to death in this thread, but I figured I'd give the script I wrote for myself if anyone prefers it...it uses ruby, so if you have that installed, you're good. Drop the script into the directory where you will run KoLmafia, and run it using:

ruby KoLmafia.rb

The script will check for the latest .jar file, download it, delete all old versions, and launch it.


Note: I couldn't upload the .rb file directly, so rename the .txt to .rb
4250

[Rinn]

I've been getting the following exception since 4/27.

An error occurred while downloading data from http://builds.kolmafia.us/
Exception Message: Unable to connect to the remote server

I am able to manually download builds fine, though. That exception is thrown whenever your computer is unable to connect to the mafia server. Currently if that error is encountered the application exits, while what it should do is at least continue on to run mafia (if it exists). I just haven't gotten around to updating it to do so since you can just run any preexisting jar by hand.

[DeathShade]

I finally got around to writing this today.

A powershell version of this script that as long as you have powershell downloaded you don't need any compilation or other programs.

Note this downloads and launches the file from the directory the .ps1 is stored in



#download latest
$loc=(Split-Path -parent $MyInvocation.MyCommand.Definition)
$webclient = New-Object System.Net.WebClient
$results=$webClient.DownloadString("http://builds.kolmafia.us")
$results=$results.substring($results.IndexOf('http ://builds.kolmafia.us/KoLmafia-'),80)
$latest=$results.substring(0,$results.indexof('.ja r')+4)
$filename=$latest.substring($latest.lastindexof('/')+1)
if (!(Test-Path $loc\$filename)) { $webClient.DownloadFile($latest, "$loc\$filename") }

#cleanup old, leave latest 2
get-childitem $loc\*.jar | sort -desc -prop LastWriteTime | select -skip 2 | del

#launch latest
cd $loc
& $((get-childitem $loc\*.jar | sort -prop LastWriteTime | select -last 1).ToString())



My desktop shortcut looks something like this:
C:\Windows\System32\WindowsPowerShell\v1.0\powersh ell.exe -file "C:\KoLmafia\Launch.ps1"

[Theraze]

Version 1.6 won't grab anything above 9999.

[matt.chugg]

Rinn's download works on the basis of the first build on the page being the latest, this will fix itself when fewyn fixes the daily build page.

[wrldwzrd89]

I've fixed my updater to work around this glitch. My updater made the exact same assumption in V8.0 and earlier. Now it determines what the latest KoLmafia version is by looking at the Last 30 Changes area, instead. ;)

[Weatherboy]

Hi, guys. I made my own utility that does this written in Python 2, so it's more cross-platform than the .Net version

Can someone fix this python version (http://kolmafia.us/showthread.php?2856-Automatically-download-the-newest-daily-build&p=30235&viewfull=1#post30235), please?

[matt.chugg]

Can someone fix this python version (http://kolmafia.us/showthread.php?2856-Automatically-download-the-newest-daily-build&p=30235&viewfull=1#post30235), please?

same problem as the others,



break #End the loop, since we found the first (ie most recent) daily build


removing the break would fix as it would be the last link on the page, however I don't know where 10001 is going to put itself on the page, so its a bit difficult fixing anything properly!

[Rinn]

I updated the updater.

[Theraze]

So... the typo in the help text from Feb 2010 (http://kolmafia.us/showthread.php?2856-Automatically-download-the-newest-daily-build&p=24183&viewfull=1#post24183) is still there. I guess I'll stop mentioning it now, but if anyone needs the right command line to skip deleting files, it's -d not -o, like the help text says. :)

PS: Thanks for making it grab the updated files again. :)

[fewyn]

Should work now, I needed to clean out all those old builds anyways.

[Bale]

The best solution to a problem is usually the easiest one.

[tgetgel]

I reloaded SW on a PC (XP Pro up through SP2). Now I get this application error: The application failed to initialize properly (0xc0000135). Click OK to terminate application.

The internet says to install the .NET framework. Does it make sense for that to solve the issue? Thanks in advance.

[wrldwzrd89]

Yes - the application is written in the .NET environment, and requires it to run.

[kitsunegami]

I reloaded SW on a PC (XP Pro up through SP2). Now I get this application error: The application failed to initialize properly (0xc0000135). Click OK to terminate application.

The Internet says to install the .NET framework. Does it make sense for that to solve the issue? Thanks in advance.My wife, for some unknown reason, despises .NET to the point where she refuses to install it no matter how many .NET apps she'd like to use. She actually prefers to manually download a new hourly build every couple of days. I, on the other hand, absolutely adore Rinn's immensely helpful bit of code and am willing to put up with .NET to use it. 8o)

If you, like my wife, would prefer to not use .NET, there is now an OS-agnostic automatic updater (http://kolmafia.us/showthread.php?6202-Universal-KoLmafia-Daily-Build-Updater) but my wife hasn't had time to try it out yet so I don't anything about it since I'm happy using Rinn's app.

[Raven434]

No problems ever, I just started getting these:

Unhandled Exception: System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
at KoLMafiaUpdate.Program.Main(String[] args)

[EdFox]

I can confirm Raven's report. It functioned fine on 10578 and this happened when 10608 was the latest version. That's a pretty broad range, unfortunately.

[Theraze]

Appears that the problem is probably KoLmafia-10600M.jar which has a letter, not just numbers...

[fronobulax]

KoLmafia-10600M.jar

My build tools append the M if the source is not 100% in synch with the latest from SVN. I have expended a fair amount of effort to use the project's build file so I would suggest that the KoLmafia-10600M.jar has no business being in a repository of files that are built from the latest source. Perhaps someone who understands and monitors that build process might check this out?

[fewyn]

Something probably didn't get cleaned out properly in the auto build process (ant daily) I have setup and it got created. I just manually deleted the file so it should be fine now.

[fronobulax]

Something probably didn't get cleaned out properly in the auto build process (ant daily) I have setup and it got created. I just manually deleted the file so it should be fine now.

Thanks. I have sometimes gotten unintended M versions when I updated SVN external to the build process and/or something got interrupted or restarted.

[Theraze]

Thanks for the delete. Verifying that the auto-update tools are working properly again. :)

[Rinn]

I fixed the updater exe yesterday morning before you reported the M link broke it since I had the same issue.

[Raven434]

Muchas Gracias!

[Theraze]

Yay! Thanks Rinn!

[Raven434]

More 'M' versioned files are breaking the auto-downloaders again:

KoLmafia-10623M.jar
KoLmafia-10618M.jar
KoLmafia-10617M.jar

Thanks.

[fewyn]

Buildfile: /home/kolmafia/svn/build.xml

clean:
[symlink] Removing symlink: dist/KoLmafia-15.1/Applications


init:


update:
[exec] U /home/kolmafia/svn/src/net/sourceforge/kolmafia/session/ChoiceManager.java
[exec] U /home/kolmafia/svn/src/net/sourceforge/kolmafia/RequestEditorKit.java
[exec] U /home/kolmafia/svn/src/net/sourceforge/kolmafia/request/GenericRequest.java
[exec] Updated to revision 10634.


svn-version:


gitsvn-version:


version:
[echo] Current revision: 10634M


daily:


init:


jikeshome:


javahome:
[exec] java version "1.6.0_26"
[exec] Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
[exec] Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode, sharing)


compile:
[echo] Using modern compiler
[javac] Compiling 70 source files to /home/kolmafia/svn/build
[copy] Copying 10 files to /home/kolmafia/svn/build
[copy] Copied 3 empty directories to 1 empty directory under /home/kolmafia/svn/build


svn-version:


gitsvn-version:


version:
[echo] Current revision: 10634M
[javac] Compiling 684 source files to /home/kolmafia/svn/build


svn-version:


gitsvn-version:


unversion:
[copy] Copying 137 files to /home/kolmafia/svn/build
[jar] Building jar: /home/kolmafia/svn/dist/KoLmafia-10634M.jar


BUILD SUCCESSFUL
Total time: 12 seconds

Not sure why it's not getting cleared out, will take a look into it.

[Veracity]

See my other note. Delete /home/kolmafia/svn/src/net/sourceforge/kolmafia/KoLConstants.java and try again. In fact, it would not hurt if your build script always did that before building.