Automatic Script Updating... Discuss!

[xKiv]

SVN would download and overwrite. There is no benefit to keeping old copies except in two cases. One is when access to the SVN repository is interrupted.

SVN keeps a local copy of the entire remote revision in separate files. AFAIK, if the access is interrupted, you might get those files corrupted, but your working copies should be unaffected (first download the current revision, then apply).

The other is when there are local changes to be preserved. In that case, it might be worth archiving what was there before a user requested update and merge was performed.

SVN should be doing that automatically - even CVS does. That is, if it detects changes that can't be easily merged just by applying (as a patch) the diff between last downloaded revision and current revision. And when that happens, you could issue a warning and open an editor to let the user finish the merge. Presuming that the changes come originally from the user.

Would embedding an SVN into Kolmafia be accessible to users with absolutely no knowledge of what that is?

I imagine the initial user interaction as copy-pasting something from a scripts.kolmafia.us page into an input box in kolmafia. Or even clicking a checkbox in kolmafia-provided list of possible scripts.

but may lead to short term vandalism if one is unlucky

Only if they don't use SVN for uploading. If you are developing properly, your local copy will know what revision it is and refuse to commit until you merge a newer current revision.

 

[Bale]

I've been silent thus far despite the fact that I am one of the people regarding whom this is most relevant. Be assured that I have been reading this thread most avidly regardless of that silence.

I have a question: If this is implemented will it be more work or less work for me to put out a new version of a script?

 

[fronobulax]

I posted a link to a Java package that AFAIK would allow SVN functionality to be embedded in KoLmafia in a user friendly/transparent way.

Reading the repository should be simple. Anyone can read. I haven't configured the SVN back end in a while but I think write access can be limited to specific individuals and specific portions of the SVN directory tree. Note that the KoLmafia.us SVN repository will not initially be a place where individuals can collaborate on a script. That may be possible but I'm going to call it Phase II. The SVN client and associated "protections" are much easier if it can be assumed that scripts in the repository work and have been vetted by the author.

I'll start looking at that kit.

 

[fronobulax]

I have a question: If this is implemented will it be more work or less work for me to put out a new version of a script?

If my vision works it will be easier. You won't have to worry about version numbers and checking for updates. Updating the script will be as simple as using a single SVN command, Commit, to load the file into the repository. Of course you will have to install and configure a SVN client to do that, but...

 

[Bale]

If it makes things simpler, then I like it.

 

[matt.chugg]

It sounds like an SVN server will do the job, and at worst if theres any functionality missing theres probably some PHP/SVN wrappers that would let me write a webservice.

If anyone is curious about svn clients, and would like to check one out, my client of choice for windows is Tortoise SVN: http://tortoisesvn.tigris.org/

For people who use Notepad++ theres at least one plugin to add svn functionality: http://www.switchonthecode.com/tech-news/notepadplusplus-subversion-plugin

I'll let someone with more mac experience suggest a client for OSx.


@fronobulax: do you have SVN server installed anywhere?

 

[matt.chugg]

If it makes things simpler, then I like it.

It will after the first time.

The first time you use the new system, you will need to install an SVN client, check in the current version of your script, then rcheck out a working copy. From that point on, releasing a new version is as simple as (if using tortoiseSVN) right clicking the file and clicking commit. (you then type in the "whats new" comments which get logged against the file commit).

 

[fronobulax]

@fronobulax: do you have SVN server installed anywhere?

I don't at the moment, but I can access a couple at work or SourceForge or just fire up a VM and install one. if the question was about what to test against, I'm good.

I think I will write a simple Java program that updates a single file from a repository. That will show me some of the config info we need, some of the pitfalls and in theory establish this as easy enough that I will do or Too Hard For Me.

 

[xKiv]

Wouldn't it also be simpler for users? (iirc you need to register to download attachments from this here forums; svn has anonymous access)

 

[Winterbay]

You definitely need to be logged in here to be able to download attachements so in that regard it would be one thing less. I guess the individual threads for discussing the different scripts will still stay despite this and so no functionality will be lost in that regard.

 

[fewyn]

If we need an SVN of some kind just ping me, I can always get one running on Kolmafia.us itself.

 

[matt.chugg]

If we need an SVN of some kind just ping me, I can always get one running on Kolmafia.us itself.

Thanks fewyn, it would probably work a lot better on this domain and usin your server and I have the system for rss if we need it. If svn works well for scripts, perhaps we could have a second rss bot and forum for script update?

If people want to play and test I have a vps with svn (and apache etc) doing nothing I'm happy to give root shell access to people I know on here for testing so we don't need the "production" server for testing, it's available for 30 days foc from the hosting guys I use in the uk and can be extended until we need to move to the live server pm me for access!

 

[bumcheekcity]

Would it be possible to feed this through to ASH, so that I can ask it what version of a particular script (if any) the user has installed)?

 

[fronobulax]

If it happens and if it remains SVN based then the only easily accessible identifier is SVN's revision number. I'm reasonably certain that could be exposed via ASH commands but I'm not sure any kind of reasonable dependency checking could be developed using it.

 

[xKiv]

You could start using tags to mark released "versions" of a script in a you-controlled manner, not sure if you can tell where a particular revision stands with respect to a given tag. I think revision numbers are enough, especially if various scripts can be isolated in their own modules with separate revision sequences.

Bout you will probably want tags anyway, so that you can publish a revision and still develop in the same repository. Something like (example from "the book":
svn copy http://svn.example.com/repos/calc/trunk \
http://svn.example.com/repos/calc/tags/release-1.0 \
-m "Tagging the 1.0 release of the 'calc' project."
) (yes, in svn, tagging is just a copy operation (which is almost free, because svn knows that there are no differences between this copy and that copy of the file))
Then the automagical kolmafia updater would just look for the newest tag (and not the newest revision) and could even store the installed version (tag name) for each automagically updated script.
Newest tag can be found for example by going through all tags and getting the one that has highest revision number.
Or you could have another file in the repository that lists your script's last tag and use that.

From command line
svn log URL/tags --limit 1

will print out the last tag (assuming they are in the "industry" standard directory /tags in the scripts repository).

This makes releasing a version slightly more complicated than just a commit, but it makes it easier to develop a script without breaking released versions (you don't have to copy files between branches or repositories or even between a non-versioned development environment and a repository).

This could also be used to switch between development version and released version (checkbox in the updater to switch between "last development revision" and "last released version"), so you could, as the script's author, edit the script in place (in your kolmafia scripts directory) and commit from there (preferably also though kolmafia's updater, so that it could instantly update its own .svn "administrative directory" (or metadata) and recognize that your local copy is already up-to-date when it checks next (otherwise you could get into a situation where SVN tells you that you have a conflict and diff doesn't show any difference). Then releasing a version could be as simple as clicking a button in mafia, entering the version string, and authenticating yourself to SVN (if it's anything like cvs, it will remember your access - on unix systems, cvs stores slightly encrypted passwords in a only-you-and-root-readable file in your home directory, no reason SVN wouldn't, if it isn't disabled in configuration).

 

[fronobulax]

@xKiv - One huge difference in our vision is that I do not expect the KoLmafia.us SVN repository to contain any development code. Nothing is checked in unless the developer/owner is happy with it and wants to release it to the world. Thus regardless of tags, numbers or any other way of labeling things, if SVN thinks what is in the repository is newer then it is an update and and safe to grab. As a script developer I can certainly make a case that this is suboptimal but there are a couple of things to consider. First, I'm lazy and unless someone else steps up and embeds a SVN client in KoLmafia before I do, there are some assumptions that make coding and configuration easier for me. Second, I'd like to keep things simple. At some level of complexity it might very well make more sense for script developers to migrate their scripts to SourceForge and then just have a utility function written in ASH, that checks for updates and so on. I'd like to remain below that level if that works for folks.

 

[fronobulax]

:-(

SVNKIT uses Java 1.5 features. It is not at all clear that I will be willing and able to code around that.

 

[matt.chugg]

:-(

SVNKIT uses Java 1.5 features. It is not at all clear that I will be willing and able to code around that.

Well, this is a free time project, and shoulnd't be hassle for anyone to work on, so lets just figure out the next step!

options as I see them:

--> Continue using svn (i'll bow to your knowledge as to whether this is workable) but code/find a php webservice to wrap the svn commands, feasibility?

--> Start looking at a fully bespoke build. (This will mean some decent discussion between the java coders and php/db developers (me! and anyone willing to help) with questions that I can initially think of being:

1: How to uniquely identify a script and its version (without using a file name preferably), some kind of header/footer in a file, possibly adding at download..
2: How should we keep things as easy as possibly for script writers?
3: What should the script repository registration process entail, svn would have been easier, but there may be a way to link to a kolmafia.us account saving code hours on a whole registration process.

I'm personally of the opion that this should be devloped "agile" style (I hate myself for even mentioning a development style!)
Lets work out the mimimum required feature set, and actually get something up and running then devlop and tweak as we go.

 

[StDoodle]

1: How to uniquely identify a script and its version (without using a file name preferably), some kind of header/footer in a file, possibly adding at download..

We already have the "script" identifier that must be included at / near the top of the script (after "imports" only, I believe off-hand) when used. We could probably add a "version" identifier that had to take a set format and follow immediately after, so that whatever system was in place could simply look through the first few lines of the file, skip "import" and blanks, and check the next two lines.

 

[matt.chugg]

We already have the "script" identifier that must be included at / near the top of the script (after "imports" only, I believe off-hand) when used. We could probably add a "version" identifier that had to take a set format and follow immediately after, so that whatever system was in place could simply look through the first few lines of the file, skip "import" and blanks, and check the next two lines.

Should work, it would need a version as well, then the mafia updater would only show "updatable" (scripts which have the correct headers) scripts in its gui for update and ignore one-off, test scripts etc.

I was actually thinking of a guid per version, so the version and the script would be tied to a single unique identifier, thus meaning it doesn't matter what filename is used and will be less likely to be any kind of clashes.

I suppose the repository should have search features and allow scripts to be searched, downloaded and installed at the click of a button (an apt-get of ash scripts)...