Mafia SVN for Scripts?

roippi

roippi

Developer
Okay, that little change makes me ~infinity percent happier -

Code: 
> svn checkout https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1/

Validating repo...
Repo validated.
C:\Users\Ben\workspace\kolmafia\svn\mafiasvntest-myvalidproject1
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1/scripts
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1/scripts/valid.txt
https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1
At revision 2

Successfully checked out working copy.

Haven't implemented svn update yet, but that's like 5 lines of code.

Here's the one thing that's still up in the air for me. To be clear on the whole process, we checkout and keep a Working Copy in svn/projectname/. We then copy the files and directories contained therein to the appropriate places in the mafia subfolders.

Left like that, this is going to result in some unintuitive behavior for lay users. Say they don't want a relay script any more, so they delete it from /relay... only to find that it keeps reappearing. It's not intuitive that you need to delete both the copy in /relay and the project folder in /svn. And, hey, maybe someone only wants half of a project, so they delete the other half of it. They don't want the stuff they deleted to keep reappearing.

Further complicating things is how the Working Copy model works. If you delete an individual file from the working copy, doing svn update will just restore it. And script maintainers can add new files to their project, which will similarly get pulled down with an svn update. We want those to be pushed to mafia subdirectories, but not any files that the user has deleted.

This is all possible to implement of course. SVN knows when it adds/updates a file (see the lines in the operation above that start with A), I just need to hand that information off to the method that pushes files to the mafia subdirectories. Also I think svn checkout should always push all files, whereas svn update should push new files and updated files iff the file currently exists in the destination subdirectory.

Whew. Lot to do still.
 
Winterbay

Winterbay

Active member
For the deletion issue could you possibly implement a "svn delete" command perhaps that would remove the script and its associated SVN-folders?
 
roippi

roippi

Developer
Yes. svn list (list checked-out projects) and svn delete are planned features. A GUI script manager eventually.
 
bumcheekcity

bumcheekcity

Active member
Would you recommend that we split our scripts so that they're in separate repositories.

For example I was just planning on having all my scripts under my single bumcheekascend sourceforge page, in separate subfolders as we noted in the previous page, but then it seems that it would be tricky for someone to have the snapshot script but NOT the ascend script (etc.) - am I understanding the suggestion correctly? Or would people be able to check out the bumcheekascend folder, snapshot folder, bumrats folder (etc.) separately, and update them separately?
 
Winterbay

Winterbay

Active member
I don't think you'd be able to update scripts separately, they would all update at once, everyone that is connected to a SVN. If I understood the presentation correctly,
 
lostcalpolydude

lostcalpolydude

Developer
Staff member
I set up my own SVN server to prepare for this feature being added. I can see using kolmafia.us instead of that if it gets set up, but I'm at least one person that's not planning to use sourceforge.

bumcheekcity said:
For example I was just planning on having all my scripts under my single bumcheekascend sourceforge page, in separate subfolders as we noted in the previous page
Click to expand...

That seems less user-friendly even if it will work. Your current setup seems strange to me even.
 
roippi

roippi

Developer
fronobulax said:
Thank you.
Click to expand...

You're welcome. I figure some people enjoy following along with/providing input to the development process, and if not it at least keeps me organized.

Current progress:

Code: 
> svn checkout https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1/

Validating repo...
Repo validated.
C:\Users\Ben\workspace\kolmafia\svn\mafiasvntest-myvalidproject1
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1/scripts
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1/scripts/valid.txt
https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject1
At revision 2

Successfully checked out working copy.

Pushing local updates...
valid.txt => C:\Users\Ben\workspace\kolmafia\scripts\valid.txt
Done.

Files are moving, yay. I had to homebrew a lot of the file handling stuff* so I want to run that code through the ringer before I give it the green light. Still have to do svn update, which I've already prepared most of the should-update-or-not logic for. Hopefully.

I do want to point out that while I'm trying to address as many security issues as possible here, there are some things that are probably going to be out of my control. Hooking mafia into a working copy system like this means that after installing a script, the script writer (or anyone with commit access to that repo) has the ability to push any new files that he wants to relay/, scripts/, images/, or data/ by putting them in the appropriate place in the repo and having you update. The nightmare scenario is someone maliciously committing a relay script that does nasty stuff, and it silently appearing in your relay/ directory when your login script does svn update.

I think the best I can do here is always pop up a confirmation dialog whenever svn update tries to add a new file. Deleting files is a lesser issue, I don't know if we should prompt about that. Only files that were installed in the first place can be deleted.

*Java's file system handling (pre-1.7) is kinda basic. It lacks a copyFile function (which we already had in FileUtilities thankfully) and a find-relative-path-between-two-files function.
 
bumcheekcity

bumcheekcity

Active member
The reason the various projects are hosted in an admittedly entirely-ridiculous way is because the vast majority of people would'nt update via SVN so we could use the SVN to update them, and then spin a new release via a copy-paste into the forum thread of a separate version.

It wasn't a great idea, but it wasn't a terrible idea, and I didn't have a better one.

Also the above sentence basically sums up my life.
 
roippi

roippi

Developer
Whoa more posts I didn't notice.

bumcheekcity said:
Would you recommend that we split our scripts so that they're in separate repositories.

For example I was just planning on having all my scripts under my single bumcheekascend sourceforge page, in separate subfolders as we noted in the previous page, but then it seems that it would be tricky for someone to have the snapshot script but NOT the ascend script (etc.) - am I understanding the suggestion correctly? Or would people be able to check out the bumcheekascend folder, snapshot folder, bumrats folder (etc.) separately, and update them separately?
Click to expand...

You can have them all in individual project folders on one repo, yes. Users will check them out separately. I'd planned for svn update to just loop through all of your checked out scripts and update them all, but if there's a desire for individual update control that's not difficult.

lostcalpolydude said:
I set up my own SVN server to prepare for this feature being added.
Click to expand...

FYI the regex that I'm using to turn sourceforge URLs into folder names is

Code: 
	private static Pattern SOURCEFORGE_PATTERN = Pattern.compile( "/p/(\\w+)/code(.*)", Pattern.DOTALL );

so if you want to conform to that you should get nicely-formatted folder names.
 
Last edited:
lostcalpolydude

lostcalpolydude

Developer
Staff member
What will happen if two repositories have files with the same name (and in the same folder, such as /scripts)? What about one of the file names matching a file that you already have, when the pre-existing file isn't part of a repository?
 
roippi

roippi

Developer
lostcalpolydude said:
What will happen if two repositories have files with the same name (and in the same folder, such as /scripts)?
Click to expand...

They are kept separately in the svn working copies. When they are then copied to the mafia subfolders, the one that is checked out/updated second will overwrite the first. That process will repeat itself every svn update. Don't necessarily know what we should do there.

What about one of the file names matching a file that you already have, when the pre-existing file isn't part of a repository?
Click to expand...

Right now it will overwrite it.

(also see my edit to my previous post if you didn't see it)
 
bumcheekcity

bumcheekcity

Active member
To be honest, I don't think having the same script names will be a massive problem. Presumably just scripters will have to make sure not to use existing names.
 
Theraze

Theraze

Active member
They do, but players get told when they're trying to overwrite an existing script currently with the manual system. With an automatic SVN-update system, you just never know which fight.ash is currently being used.
 
roippi

roippi

Developer
I'll probably add some cli output that lets people know there are conflicting files, but won't change behavior any more than that.

I've implemented svn update now. It adds/updates/deletes files with the logic that I think is appropriate. It has a nifty little prompt for making sure it's ok to add new files from an update. I won't try to get everything 100% polished since I want this to be in beta testing for a little bit before we ask Joe Sixpack to use it.
 
roippi

roippi

Developer
I have this implemented locally now in a form I'm ready to commit. Sort of.

Unfortunately in order to get svnkit to work, I've pulled in some required external libraries in the form of .jars; I've until now gotten building to work under eclipse by modifying .classpath, but I basically have no idea what I'm doing with build.xml. I've spent a few hours on this and discovered primarily that buildfiles are stupid.

If I can't make some headway on this, I'll probably just extract all the .jars and leave them as .class files. If anyone has any tips on how to add a .jar to the classpath in build.xml, let me know.
 
fronobulax

fronobulax

Developer
Staff member
I think the reason there is a lib branch with non-KoLmafia source is because no one really figured out how to build our jar "with dependencies". Maven will do that but I'm not sure how to bridge the gap between Maven and the associated infrastructure I use at work and KoLmafia.
 
Veracity

Veracity

Developer
Staff member
I tried to figure out how to use an embedded .jar file when I imported HTMLCleaner. I never got it to work, so I just pulled in the source. Good luck!
 
You must log in or register to reply here.
Top