Mafia SVN for Scripts?
- Thread starter bumcheekcity
- Start date
bumcheekcity
Active member
I have to say that the SVN update would be a massive help to everyone, and could help people with finding new scripts that they may not have known existed. I don't think it would be a trivial update or one which would only help us coders. I think the 2MB would be "well spent" as it were (particularly if fewyn says we're not close to bandwidth limits).
roippi
Developer
It's closer to 3.5 MB, which is a bump in .jar size by like 60%. Still, if we have that much bandwidth overhead I won't worry too much about it - though this is how feature bloat starts. I'm still philosophically grumpy about it and will see if I can pare it down by trimming unnecessary features.
roippi
Developer
Kinda cool.
Checked out all that stuff into /scripts, creates a little .svn folder and all that jazz. I realize now that this is going to take a great deal of work to implement right, but long term goals and all that.
Code:
trying to check out /p/bumcheekascend/code
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE REPOSITORY ( id INTEGER PRIMARY KEY AUTOINCREMENT, root TEXT UNIQUE NOT NULL, uuid TEXT NOT NULL );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE INDEX I_UUID ON REPOSITORY (uuid);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE INDEX I_ROOT ON REPOSITORY (root);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE WCROOT ( id INTEGER PRIMARY KEY AUTOINCREMENT, local_abspath TEXT UNIQUE );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE UNIQUE INDEX I_LOCAL_ABSPATH ON WCROOT (local_abspath);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE PRISTINE ( checksum TEXT NOT NULL PRIMARY KEY, compression INTEGER, size INTEGER NOT NULL, refcount INTEGER NOT NULL, md5_checksum TEXT NOT NULL );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE ACTUAL_NODE ( wc_id INTEGER NOT NULL REFERENCES WCROOT (id), local_relpath TEXT NOT NULL, parent_relpath TEXT, properties BLOB, conflict_old TEXT, conflict_new TEXT, conflict_working TEXT, prop_reject TEXT, changelist TEXT, text_mod TEXT, tree_conflict_data TEXT, conflict_data BLOB, older_checksum TEXT, left_checksum TEXT, right_checksum TEXT, PRIMARY KEY (wc_id, local_relpath) );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE INDEX I_ACTUAL_PARENT ON ACTUAL_NODE (wc_id, parent_relpath);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE INDEX I_ACTUAL_CHANGELIST ON ACTUAL_NODE (changelist);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE LOCK ( repos_id INTEGER NOT NULL REFERENCES REPOSITORY (id), repos_relpath TEXT NOT NULL, lock_token TEXT NOT NULL, lock_owner TEXT, lock_comment TEXT, lock_date INTEGER, PRIMARY KEY (repos_id, repos_relpath) );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE WORK_QUEUE ( id INTEGER PRIMARY KEY AUTOINCREMENT, work BLOB NOT NULL );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE WC_LOCK ( wc_id INTEGER NOT NULL REFERENCES WCROOT (id), local_dir_relpath TEXT NOT NULL, locked_levels INTEGER NOT NULL DEFAULT -1, PRIMARY KEY (wc_id, local_dir_relpath) );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE NODES ( wc_id INTEGER NOT NULL REFERENCES WCROOT (id), local_relpath TEXT NOT NULL, op_depth INTEGER NOT NULL, parent_relpath TEXT, repos_id INTEGER REFERENCES REPOSITORY (id), repos_path TEXT, revision INTEGER, presence TEXT NOT NULL, moved_here INTEGER, moved_to TEXT, kind TEXT NOT NULL, properties BLOB, depth TEXT, checksum TEXT, symlink_target TEXT, changed_revision INTEGER, changed_date INTEGER, changed_author TEXT, translated_size INTEGER, last_mod_time INTEGER, dav_cache BLOB, file_external TEXT, PRIMARY KEY (wc_id, local_relpath, op_depth) );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE INDEX I_NODES_PARENT ON NODES (wc_id, parent_relpath, op_depth);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TABLE EXTERNALS ( wc_id INTEGER NOT NULL REFERENCES WCROOT (id), local_relpath TEXT NOT NULL, parent_relpath TEXT NOT NULL, repos_id INTEGER NOT NULL REFERENCES REPOSITORY (id), presence TEXT NOT NULL, kind TEXT NOT NULL, def_local_relpath TEXT NOT NULL, def_repos_relpath TEXT NOT NULL, def_operational_revision TEXT, def_revision TEXT, PRIMARY KEY (wc_id, local_relpath) );
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE INDEX I_EXTERNALS_PARENT ON EXTERNALS (wc_id, parent_relpath);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE UNIQUE INDEX I_EXTERNALS_DEFINED ON EXTERNALS (wc_id, def_local_relpath, local_relpath);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE VIEW NODES_BASE AS SELECT * FROM nodes WHERE op_depth = 0;
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE VIEW NODES_CURRENT AS SELECT * FROM nodes AS n WHERE op_depth = (SELECT MAX(op_depth) FROM nodes AS n2 WHERE n2.wc_id = n.wc_id AND n2.local_relpath = n.local_relpath);
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TRIGGER nodes_insert_trigger AFTER INSERT ON nodes WHEN NEW.checksum IS NOT NULL BEGIN UPDATE pristine SET refcount = refcount + 1 WHERE checksum = NEW.checksum; END;
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TRIGGER nodes_delete_trigger AFTER DELETE ON nodes WHEN OLD.checksum IS NOT NULL BEGIN UPDATE pristine SET refcount = refcount - 1 WHERE checksum = OLD.checksum; END;
May 23, 2013 8:40:20 PM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
INFO: CLI: executing statement: CREATE TRIGGER nodes_update_checksum_trigger AFTER UPDATE OF checksum ON nodes WHEN NEW.checksum IS NOT OLD.checksum BEGIN UPDATE pristine SET refcount = refcount + 1 WHERE checksum = NEW.checksum; UPDATE pristine SET refcount = refcount - 1 WHERE checksum = OLD.checksum; END;
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts
URL: svn://svn.code.sf.net/p/bumcheekascend/code
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: dir
Schedule: normal
Last Changed Author: maymne
Last Changed Revision: 270
Last Changed Date: Thu May 16 01:45:43 EDT 2013
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\relay_bumSeMan0.2.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/relay_bumSeMan0.2.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 15
Last Changed Date: Sun Nov 27 12:19:13 EST 2011
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: b01fcd64c8fc3f4ae5aa596cf9168ab89e3cab06
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.9.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.9.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: icecoldfever
Last Changed Revision: 71
Last Changed Date: Thu Feb 23 15:28:51 EST 2012
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 8678e4c802dae7cc5a8f1b8c4212cc9f547a33c1
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv1.02.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv1.02.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: foxmoriarty
Last Changed Revision: 254
Last Changed Date: Thu May 02 23:24:20 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 0592c6d740f1af7fb7b2413defc648fd2fdc7faa
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bumrats_items.txt
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bumrats_items.txt
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: foxmoriarty
Last Changed Revision: 244
Last Changed Date: Thu Apr 11 23:18:30 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: a2349e15c9f1031a05efc1ed477f910fdc3b020d
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.12.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.12.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: icecoldfever
Last Changed Revision: 183
Last Changed Date: Mon Oct 01 10:59:11 EDT 2012
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 581ea25edbb83e9b9e45babf834b0fc84cf45585
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.14.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.14.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 250
Last Changed Date: Tue Apr 23 12:59:37 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 4476617c8fd6ca48d753ae3418510cdf1b0cd4cb
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv1.03.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv1.03.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: foxmoriarty
Last Changed Revision: 255
Last Changed Date: Sat May 04 01:05:11 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 5cbb14714af6c284af2e451ca57a9bed34068430
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.8.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.8.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 28
Last Changed Date: Thu Dec 29 17:29:18 EST 2011
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: aaab1b08ffd505c34e8ecfdd749e5b3412fad6c9
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\relay_bumSeMan0.3.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/relay_bumSeMan0.3.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 93
Last Changed Date: Mon Mar 19 02:47:51 EDT 2012
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 0ca56d578b2e01eb7a4146ec307081a4ece8fbe0
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\snapshot.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/snapshot.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 259
Last Changed Date: Tue May 07 01:51:51 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 04664b73088cbc34b8f31f6772ef05ff0f5ceabc
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.13.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.13.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 249
Last Changed Date: Tue Apr 23 12:50:43 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: b4a28e7f5d0cc1c4d3d664230b9716143dca1e4e
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.10.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.10.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: icecoldfever
Last Changed Revision: 83
Last Changed Date: Fri Mar 09 11:33:49 EST 2012
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 338f030ce9aeacaa45e30f20c7602dd28abba212
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv1.04.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv1.04.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: foxmoriarty
Last Changed Revision: 258
Last Changed Date: Mon May 06 21:11:21 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: c0f44abc311054fe812e376b1bce0a65c4634f86
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\hccheck.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/hccheck.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 172
Last Changed Date: Sun Sep 09 03:25:37 EDT 2012
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 89033f3f7b5936bcfe7ebdd3a673cdb9f18d83df
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\rollover.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/rollover.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 169
Last Changed Date: Sat Sep 08 01:46:33 EDT 2012
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 932cde790f25641cadb894d1ea2ffb95e93ee3ec
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv1.01.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv1.01.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: foxmoriarty
Last Changed Revision: 254
Last Changed Date: Thu May 02 23:24:20 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: cd102f1be2eb84c3e49f1c5cdd1f78f162a9d598
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.7.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.7.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 14
Last Changed Date: Sun Nov 27 12:17:11 EST 2011
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: b7b3a7e49730eafdf6b1ec938d0af94f3ccf92bc
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bumcheekascend.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bumcheekascend.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: maymne
Last Changed Revision: 270
Last Changed Date: Thu May 16 01:45:43 EDT 2013
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: deba0e902a977aee003c0e5605906aa4ecc75e6a
-----------------INFO-----------------
Local Path: C:\Users\Ben\workspace\kolmafia\scripts\bUMRATSv0.11.ash
URL: svn://svn.code.sf.net/p/bumcheekascend/code/bUMRATSv0.11.ash
Repository UUID: 1f9aa2c4-875c-429a-b064-a779cfda5ba2
Revision: 270
Node Kind: file
Schedule: normal
Last Changed Author: bumcheekcity
Last Changed Revision: 87
Last Changed Date: Sun Mar 11 14:04:15 EDT 2012
Text Last Updated: Thu May 23 20:40:21 EDT 2013
Checksum: 94790e6c8bfa676e13656f45501c515b7fba7a3f
doneChecked out all that stuff into /scripts, creates a little .svn folder and all that jazz. I realize now that this is going to take a great deal of work to implement right, but long term goals and all that.
roippi
Developer
I'm just gonna keep posting in here because I like talking to myself to keep my thoughts straight. Also, we should have strikethrough tags.
Anyhoo. New plan I think. This whole privileged line at the top of the script thing was unsatisfying to me. How about we only support installation of scripts via svn? This simplifies things significantly on both ends.
In mafia, do "svn checkout svn://url.to/script". Mafia first makes sure that the script at the svn address conforms to some guidelines then checks it out. The files go automagically into the right place, be it scripts/, relay/, or images/. Then a simple "svn update" (which can even be done externally with an svn tool of your choice if you'd like) updates all your stuff everywhere. Easy peasy.
Anyhoo. New plan I think. This whole privileged line at the top of the script thing was unsatisfying to me. How about we only support installation of scripts via svn? This simplifies things significantly on both ends.
In mafia, do "svn checkout svn://url.to/script". Mafia first makes sure that the script at the svn address conforms to some guidelines then checks it out. The files go automagically into the right place, be it scripts/, relay/, or images/. Then a simple "svn update" (which can even be done externally with an svn tool of your choice if you'd like) updates all your stuff everywhere. Easy peasy.
bumcheekcity
Active member
That sounds like a pretty good idea, although probably best that people are allowed to have scripts installable outside of the SVN.
roippi
Developer
Yes. That wouldn't change.
What would really happen under-the-hood is that there is a new base-level directory called svn/. when you checkout a repo, it creates a unique folder in svn/. In that directory goes a working copy. Once the working copy is synced, we copy all of the appropriate files into the scripts/, relay/, and images/ directories as appropriate. When updates are asked for ("svn update"), we go into all of the working copies, do updates, then push those to the mafia subdirectories.
Somewhat counterintuitively, users who want to maintain local changes to scripts would have to do those changes in the svn/UUID/ version of the script rather than the copy found in scripts/ or relay/. The latter version will be overwritten any time an update happens to the working copy version. This is a bit of a bummer but is a limitation of working with SVN. I could instead go with Git, but... that's a whole new can of worms. Also the JGit library (currently the best option available) is low-level and not at all fun to work with. Though, it is smaller than svnkit, so that's something.
What would really happen under-the-hood is that there is a new base-level directory called svn/. when you checkout a repo, it creates a unique folder in svn/. In that directory goes a working copy. Once the working copy is synced, we copy all of the appropriate files into the scripts/, relay/, and images/ directories as appropriate. When updates are asked for ("svn update"), we go into all of the working copies, do updates, then push those to the mafia subdirectories.
Somewhat counterintuitively, users who want to maintain local changes to scripts would have to do those changes in the svn/UUID/ version of the script rather than the copy found in scripts/ or relay/. The latter version will be overwritten any time an update happens to the working copy version. This is a bit of a bummer but is a limitation of working with SVN. I could instead go with Git, but... that's a whole new can of worms. Also the JGit library (currently the best option available) is low-level and not at all fun to work with. Though, it is smaller than svnkit, so that's something.
bumcheekcity
Active member
Would we then set up a separate mafia scripts SVN and contribute to that, or though the main mafia SVN?
roippi
Developer
Individual scripters should set up their own SVN repos. It is up to the person maintaining the repo to provide the right granularity to the repo so that users can check out the scripts that they want. SVN does not allow finer control than "give me this directory", so implementers should be mindful of that.
Say I maintain two individual scripts, scriptA and scriptB. I can do several things:
1) Maintain both scripts in the same directory: myrepo/scripts/
Further note to implementers: a valid project must have one or more of scripts/, relay/, data/, and/or images/ subfolders. Other folder names are invalid and will cause validation to fail and mafia will not check out your project. Having any files in the base directory other than those four folders will also fail validation. Think of it like unzipping an archive in your mafia base directory; the files will go into their respective subfolders when we unzip it. You're not allowed to put things in, say, settings/, nor are you allowed to put things in the top-level where DEBUG logs and the like go. Each file's extension will also be scrutiny to validation; having .exes or whatever will also cause validation to fail.
I don't think the above is being too restrictive, security concerns are a thing I need to think about.
---
If people want to have a centralized repo of scripts, note that that is still valid (it's just entry 3 above). I have concerns about that type of setup (permissions being a huge concern) but it is technically valid.
Say I maintain two individual scripts, scriptA and scriptB. I can do several things:
1) Maintain both scripts in the same directory: myrepo/scripts/
-This is valid but undesirable, since checking out myrepo/ will by needs give me both scriptA and scriptB.
2) Maintain the scripts in individual repos: myrepo1/scripts/ and myrepo2/scripts/-Valid, as long as you are okay with creating a new repo every time you make a new script
3) Maintain multiple projects in the same repo: myrepo/myscriptA/scripts and myrepo/myscriptB/scripts-Also valid. Users can individually check out the myscriptA/ project if that's all they want.
Further note to implementers: a valid project must have one or more of scripts/, relay/, data/, and/or images/ subfolders. Other folder names are invalid and will cause validation to fail and mafia will not check out your project. Having any files in the base directory other than those four folders will also fail validation. Think of it like unzipping an archive in your mafia base directory; the files will go into their respective subfolders when we unzip it. You're not allowed to put things in, say, settings/, nor are you allowed to put things in the top-level where DEBUG logs and the like go. Each file's extension will also be scrutiny to validation; having .exes or whatever will also cause validation to fail.
I don't think the above is being too restrictive, security concerns are a thing I need to think about.
---
If people want to have a centralized repo of scripts, note that that is still valid (it's just entry 3 above). I have concerns about that type of setup (permissions being a huge concern) but it is technically valid.
Theraze
Active member
Not entirely true. I don't get ALL the BCA scripts, but I do get some. The full folder has 19 files. My folder has 4 (bumcheekascend, hccheck, rollover, and snapshot). You can opt into a specific set of scripts to sync.
roippi
Developer
Perhaps I should say "the SVN working copy model does not allow finer control..." SVN checkout very explicitly does not allow for file-specific checkouts, because SVN working copies do not work like that.
You are probably doing SVN export or some operation along those lines that does not preserve .svn metadata. Which we need.
Anyway, the point is that this is how mafia SVN integration will work. If you want users to be able to individually checkout your scripts, you need to separate them into projects. A setup like BCC's current all-files-in-one-directory won't work unless you actually intend to pull down the whole directory.
edit: I've set up a dummy repo to do some testing here. It should pretty clearly show the project/directory setup I was talking about and whatnot.
Last edited:
bumcheekcity
Active member
Is the intention that script authors tell you what their repos are and then you'll hardcode a list of authorised repos into the script, or that they come from some sort of data file?
Sounding good. Keep thinking out loud. Also note that Fenwyn has graciously offered to host a repository and what works for him, with several potential script writers using it, might steer us towards one set of conventions over another.
roippi
Developer
Nothing actually goes into the script. Maybe I should explain the whole use cycle better.
Someone posts a new thread with their new WinTheGameScript. It both wins the game and conveniently overrides all relay pages with images of Pokemon. In their thread, they explain:
That's it. When a user wants to update all scripts that they've installed with SVN, they do svn update in the gCLI. This could be an on-login option, or whatever.
roippi
Developer
Checkout operations are mostly implemented -
So here's what I like so far. It works and it's pretty immune to repositories being poorly set up. It displays the information like I'd want an SVN client to display it.
Here's what I don't like: 84892fd9-0b54-40dc-9589-a4ebc549249b.
Right now, when the user puts in "svn checkout https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/", we need to reproducibly turn that into a unique identifier for that checkout. That awful string is the repository's UUID, a unique string that every SVN repo gets which can be queried. That is what I'm using right now for the folder name to store the working copy. So after checking out, I now have in my file system:
kolmafia\svn\84892fd9-0b54-40dc-9589-a4ebc549249b\<files>
I want a better method that results in that folder being named something human-readable. I'm brainstorming ideas. Maybe parse the URL, split by "/", and use a mashup of the project name (mafiasvntest) and the least-significant-element (myvalidproject2)? Not every svn repo formats their urls like sourceforge, so that's obviously a glaring issue. I dunno.
Code:
> svn checkout https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/
Validating repo...
Repo validated.
C:\Users\Ben\workspace\kolmafia\svn\84892fd9-0b54-40dc-9589-a4ebc549249b
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/images
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/images/valid
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/images/valid/nesting%20is%20okay.txt
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/scripts
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/scripts/valid
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/scripts/valid/nesting%20is%20okay.txt
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/data
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/data/valid
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/data/valid/though%20maybe%20we%20shouldnt%20allow%20subfolders%20in%20data.txt
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/relay
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/relay/valid
A https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/relay/valid/nesting%20is%20okay.txt
https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2
At revision 2
Successfully checked out working copy.
> svn checkout https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/
Validating repo...
Repo validated.
C:\Users\Ben\workspace\kolmafia\svn\84892fd9-0b54-40dc-9589-a4ebc549249b
At revision 2
Successfully checked out working copy.
> svn checkout https://svn.code.sf.net/p/mafiasvntest/code/myINVALIDproject3/
Validating repo...
The requested repo failed validation. Complain to the script's author.
repo at /p/mafiasvntest/code/myINVALIDproject3 did not pass validation. Aborting Checkout.So here's what I like so far. It works and it's pretty immune to repositories being poorly set up. It displays the information like I'd want an SVN client to display it.
Here's what I don't like: 84892fd9-0b54-40dc-9589-a4ebc549249b.
Right now, when the user puts in "svn checkout https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/", we need to reproducibly turn that into a unique identifier for that checkout. That awful string is the repository's UUID, a unique string that every SVN repo gets which can be queried. That is what I'm using right now for the folder name to store the working copy. So after checking out, I now have in my file system:
kolmafia\svn\84892fd9-0b54-40dc-9589-a4ebc549249b\<files>
I want a better method that results in that folder being named something human-readable. I'm brainstorming ideas. Maybe parse the URL, split by "/", and use a mashup of the project name (mafiasvntest) and the least-significant-element (myvalidproject2)? Not every svn repo formats their urls like sourceforge, so that's obviously a glaring issue. I dunno.
bumcheekcity
Active member
If we have the two facts that inevitably, most of us will use sourceforge, and that different SVNs will have different naming structures for the https://svn.code.sf.net/p/mafiasvntest/code/myvalidproject2/ bit, then we might in all honesty have no problems with just saying "use sourceforge". It would mean that we don't have to manually program in different naming structures for different hosts, although now I come to think of it, in reality this would likely just be a small amount of regex and probably easy to implement.
Then we could parse and do soemthing like the following to get the regex for our domains:
if (url.contains("svn.code.sf")) {
return "sf.net/p/([^/]+)/code/([^/]+)";
} else if url.contains("fakesvndomain.com") {
return "fakesvndomain.com/repos/([^/]+)/([^/]+)"
}
Or whatever.
Then we could parse and do soemthing like the following to get the regex for our domains:
if (url.contains("svn.code.sf")) {
return "sf.net/p/([^/]+)/code/([^/]+)";
} else if url.contains("fakesvndomain.com") {
return "fakesvndomain.com/repos/([^/]+)/([^/]+)"
}
Or whatever.
roippi
Developer
Pretty reasonable. I can probably at worst come up with a nicely-formatted solution for sourceforge URLs and fall back to the ugly solution for others. Obviously adding support for any repo that fewyn hosts.
The other thought I had was to require a top-level scriptinfo.txt file to be included. It can define the project name and whatever other metadata we want to define. The bonus of this approach is that it parlays nicely into other future features, such as a built-in script browser/installer. The downside being additional script maintainer burden, which is lame. I'm thinking that the first approach is preferable, as trivial of a thing as making a small text file is. One more hurdle is one more thing to keep people from using the feature.
The other thought I had was to require a top-level scriptinfo.txt file to be included. It can define the project name and whatever other metadata we want to define. The bonus of this approach is that it parlays nicely into other future features, such as a built-in script browser/installer. The downside being additional script maintainer burden, which is lame. I'm thinking that the first approach is preferable, as trivial of a thing as making a small text file is. One more hurdle is one more thing to keep people from using the feature.
Just my opinion, but I do not think it is inevitable that most script writers will move to SourceForge.
If there is enough interest in a repository that Fenwyn is willing to host and maintain one then I would much prefer that, as a writer and user, to SourceForge.
bumcheekcity
Active member
Ah, I didn't see that there. Either way, I don't think it's an issue based on the above posts.