Page 1 of 2 1 2 LastLast
Results 1 to 10 of 20

Thread: quick SVN primer (for scripters)

  1. #1
    Developer roippi's Avatar
    Join Date
    Aug 2010
    Posts
    2,663

    Default quick SVN primer (for scripters)

    With mafia now supporting an internal svn client, some scripters are undoubtedly wondering what "svn" is, how it works, and what it can do.

    What it is

    Subversion ("svn") is a centralized version control system. A repository ("repo") exists at a url - this repo contains your project. It not only contains the files for your project, but every previous version of those files that's ever existed. Think of old versions like hitting ctrl-z on a Word document - you can revert changes all the way back to the start, or anywhere in between.

    It is more than just a place to save files though. There is something called a Working Copy, which is a local version of a remote repository, plus any local changes you may have made. The process of creating this Working Copy is referred to as "checking out" the repo, or "checkout." You have probably also heard the term "commit", which is simply taking some/all local changes in a working copy and syncing them with the repo. Then other users sync their working copies with the repo, and voila!

    How it works

    A working copy contains a hidden folder, .svn, that contains all of the metadata needed to know how your local version differs from the repository's version. This is created for you when you checkout the project. You don't really need to know any of these details, really, for the most part using svn does not require knowing how it works under the hood. What you may be interested in is how to create your own repo.

    I'll go over the easiest method under Windows - using TortoiseSVN. Under *nix you should probably just be using the command line "svn" client, though I imagine there are a million different graphical alternatives. There are alternatives in windows as well, I just prefer TortoiseSVN for its file system integration.

    First, get a sourceforge account. Make a new project there. Download TortoiseSVN and install it. Make a new folder somewhere on your computer. Right-click, and choose SVN Checkout... from the context menu. You will want to give it the address that you see on your new sourceforge project code page - for now just grab the http:// address, it's easier. Ignore the commands in front of the address and after it, those are for command-line svn tools.

    The rest of the default options are okay. Click OK and you should get a new working copy of your repo! Cool. Open it up and let's add some stuff. Add a new directory (call it whatever), within that directory add a folder called scripts/, and within scripts/ put a new text file. Now navigate back up to the root of your working copy, right click on the folder, and choose SVN commit... Tick all the files/folders (tick the "show unversioned files" box if it's not ticked), give it a quick commit comment up top. You should have to enter your sourceforge user information, save it if you want. If you've done everything correctly, your new project should have been committed.

    Optional: to really play around with this, checkout a second working copy somewhere in another folder on your computer. Try committing changes from one working copy to the repo, then doing right-click SVN Update on the other working copy. Notice how TortoiseSVN helpfully changes the little file/folder icons for you when you make modifications to things. Hopefully you're starting to see how this whole thing works.

    What it can do

    svn is a very mature tool by now, so there's an absolute ton of stuff that I won't cover. You may for example notice that mafia itself has a "Latest SVN Changes" forum, which has .diffs of all the changes to files within mafia. There's also creating patches, merging repos, yada yada... Won't talk about it.

    I'm more interested in talking about how mafia svn integration works. Doing svn checkout <repo> creates a working copy of the repo in your svn/ folder. Then, mafia "pushes" the contents of that working copy to the appropriate subfolders. For example, if a repo has a scripts/dostuff.ash file, that file gets copied to your scripts/ folder.

    svn update goes through all your working copies and performs an update. There are other operations that can happen here. With a checkout, you can only be adding ("A") new files to a working copy. With an update you can update ("U"), merge ("G"), delete ("D") or also add ("A") files. Here, we warn the user whenever a file is added, due to security concerns. Note that "U"pdating only happens if the "pushed" local file still exists in the user's directory. I am bolding this because I'm sure it will confuse some people. This feature is a safeguard against svn re-installing scripts over and over that the user just wants to have deleted.

    If you installed TortoiseSVN above and browse to the svn/ folder, you may notice something: the working copies checked out with svn checkout are fully-featured working copies, just like you checked out using TortoiseSVN. This means that you can make changes directly to this folder and commit them to the repo using TortoiseSVN - if you have repo commit permissions, of course.

    And yeah. I meant this as a "quick" primer and it turned into a "moderately long" one, but them's the breaks.
    Last edited by roippi; 06-19-2013 at 04:51 PM.

  2. #2
    Developer
    Join Date
    Nov 2008
    Location
    Paris, France
    Posts
    4,511

    Default Installment 1.5

    I migrated a script to SVN, and I hit a couple of bumps I wanted to mention here:

    Click OK and you should get a new working copy of your repo! Cool. Open it up and let's add some stuff. Add a new directory (call it whatever), within that directory add a folder called scripts/, and within scripts/ put a new text file.
    Originally Posted by roippi View Post
    Apparently the scripts/ folder has to be at the root. Putting it inside another directory led to the error "The requested repo failed validation. Complain to the script's author."

    I think you should mention how dependencies.txt works too. It has to be at the root too (next to the scripts/ folder), and it should only contain the checkout URLs of the dependency projects, like this:
    Code:
    https://svn.code.sf.net/p/zlib/code
    https://svn.code.sf.net/p/slyz-nemesis/code/
    One last (very small) inconvenience was that I discovered after installing TortoiseSVN 1.8 that SVN 1.8 isn't backwards compatible. Mafia uses SVN 1.7, so you can't use TortoiseSVN 1.8 on Mafia's checked out copy of your script (the one in svn/).

    The workaround is either (a) install TortoiseSVN 1.7 or (b) check out a separate copy of your project that you use for commits.
    Last edited by Bale; 11-11-2013 at 10:07 PM.

  3. #3
    Developer roippi's Avatar
    Join Date
    Aug 2010
    Posts
    2,663

    Default

    Installment 2: Branches

    This is a condensed explanation of information found here. It's well written, I'd recommend reading it if you have time.

    -----

    Before We Begin

    There are some things that are necessary to facilitate a good branch/tag setup. Setting up your repo in the right way is going to save you a lot of headache down the line.

    Conventions dictate that in the top level of your repo, you want to create the following directories:

    • trunk/
    • branches/
    • tags/

    I won't go over what the difference between tags/ and branches/ is, they're basically the same thing. You can think of "tags" being like mafia's "point releases," if you're familiar with those. Not important, you don't really need to create tags.

    The reason you want to have these folders in the root of your repo is that if you have users check out the root of your repo, there's no room to create separate branches - and with mafia's validation, even trying to do so will break things. So if you plan on using branches, at the very minimum make trunk/ and have the majority of your users checkout that, so you can later create branches/ if you want.

    Some services like googlecode will even create these default folders for you when making a new repo. Like I said, it's just convention - you aren't forced to use these folders, but it's a good idea if you ever plan on making branches.

    High Level Overview

    Using branches simplifies your workflow when you want to work on a new feature. You can even have permanent development branches that periodically get merged into trunk. Also, there are space- and performance-related improvements when you create a branch. Physically, you are creating no new files on the repo server - you're just making symbolic links. If this is confusing or you want to read more, see the link at the top.

    Sometimes you just want to develop a single feature in a separate branch, then when you're done merge it in and delete the feature branch. Note that with svn, data is never lost permanently - you can always resurrect deleted files and directories if you want.

    Commands and TortoiseSVN

    If you're using a CLI svn tool, you want to first use svn copy to make a new branch that is a copy of trunk/.

    If you're using tortoiseSVN, have a checkout of trunk/ somewhere, then right click on it and do Branch/tag... The dialog is pretty straightforward. Up top, use the little repo browser to navigate to branches/ and give it a new folder name to create your branch. Give it a commit message and tell it you want to make the branch off of HEAD. If you want, tick the box in the lower left corner "switch working copy..." Like it says, this will turn your working copy into the new branch, instead of leaving it as trunk. Click ok and tortoiseSVN should make a new branch for you. Feel free to make as many random branches as you want - like I said above, there are no physical files being moved around, only symbolic links.

    Now, you do all your development as usual in the new branch. When it's ready to go back into trunk, you right click on your trunk branch and choose Merge... There's a couple options of what you mean by "merge," and TortoiseSVN helpfully explains what they are, briefly. For your purposes, you are interested in the top two choices. If you want to keep the development branch alive, you choose the top option, "merge a range of revisions." If you're done with the development branch - say, you created it just to work on a single feature - you can choose the "reintegrate" option. Note that this won't actually delete the branch, it just will cause complications in the future if you continue to try to use the branch. Honestly the benefits of using the "reintegrate" option are pretty limited, and the downsides pretty extensive; for our purposes you can basically always use the top "merge a range of revisions" option.

    Now, on the next screen, tell it that you want to merge your development branch into the working copy (trunk). SVN will go through and merge the differences from your branch into your working copy. Then if you're happy with the results, go ahead and commit the changes to trunk. If not, feel free to revert your local changes and try again. You can then go ahead and delete your dev branch if you want - or not.

    Do you want this?

    Some projects don't need anything other than a trunk branch. They can feel free to have users checkout the root of their repo and just not worry about it.

    Others may want the flexibility to create a "beta" branch where advanced users can help them troubleshoot features before they go out to the masses. Or maybe you're just working on a particularly complex feature that deserves its own temporary branch - keeping your version history separate for that one feature can be useful when reviewing your code. Up to you!

  4. #4
    Senior Member Winterbay's Avatar
    Join Date
    Feb 2010
    Posts
    5,033

    Default

    A couple of questions:
    1) Say you have already set up your repo, but not in accordance with these guidelines, and want to poke around with branches. Is there any way to do that without completely screwing it up for everyone who has already checked the script out?
    2) If you have a branch and someone wants to help you test it, how do they switch between the different copies? I assume that you can check out the branch as well, but will that then get a new name in MAfia? And if not how does Mafia handle having two scripts with the same name from different branches of a repo?
    My scripts:
    Best sushi for Profit
    Winterbay's Helpful Automatic Monsterbasher (WHAM)

    Scripts maintained by me:
    Autobasement
    Smashlib

    A small programmer joke:
    Wife: Honey, please go to the market and buy 1 bottle of milk. If they have eggs, bring 6.
    The man came back with 6 bottles of milk.
    Wife: Why did you buy 6 bottles of milk.
    Man: Because they had eggs...

    The 6 Stages of Debugging: 1 )That can't happen. 2) That shouldn't happen. 3) Hmmm, weird. 4) Why does that happen? 5) Oh, I see. 6 )How did that ever work?

    All programs can be optimized, and all programs have bugs; therefore all programs can be optimized to one line that doesn’t work. /Bob Fitch, Blizzard

  5. #5
    Developer roippi's Avatar
    Join Date
    Aug 2010
    Posts
    2,663

    Default

    A couple of questions:
    1) Say you have already set up your repo, but not in accordance with these guidelines, and want to poke around with branches. Is there any way to do that without completely screwing it up for everyone who has already checked the script out?
    Originally Posted by Winterbay View Post
    If you're having users check out the root of your repo, not really. If they're checking out a folder, any folder, you can just pretend that is trunk/, but if they're checking out the root of the repo, not much you can do. Easiest way is to just make another repo and tell people to migrate to that one. Maybe have an informational line pop up in your script or something.

    This is somewhat my fault that I didn't really educate people about the conventions ahead of time. The information is out there, so I don't feel too bad, but who reads the manual nowadays...

    2) If you have a branch and someone wants to help you test it, how do they switch between the different copies? I assume that you can check out the branch as well, but will that then get a new name in MAfia? And if not how does Mafia handle having two scripts with the same name from different branches of a repo?
    There is literally a command, "svn switch", that switches a working copy to a different location. If they're using an external svn tool, they can use that to just seamlessly move over to the dev branch. It's really straightforward to use.

    Or they can just delete their project and checkout the new branch. There won't be any errors from checking out both at once, you just will get some.. goofy behavior. Since both trunk/ and your branch have the same files in them, whenever updates to either of them fire, those updates will update the local file.

    example: you simultaneously checked out

    svn/script1/trunk/scripts/myscript.ash
    and
    svn/script1/branches/dev/scripts/myscript.ash

    Well, when updates to the dev/ version of the script fire, the dev/ version of myscript.ash gets pushed to your local copy. But if there are any updates to the trunk/ version, suddenly the trunk version of myscript.ash gets pushed to your local copy. Confusing. I would avoid this situation altogether and just make sure you only have one branch checked out at once.
    Last edited by roippi; 06-18-2013 at 04:32 PM.

  6. #6
    Senior Member Theraze's Avatar
    Join Date
    Mar 2010
    Posts
    8,741

    Default

    Sounds like if you want to simultaneously have two versions available, let's call them WHAM and WHAM beta for sake of example, you still do really need to have two repos set up and manually merge changes amongst the two.

  7. #7
    Developer roippi's Avatar
    Join Date
    Aug 2010
    Posts
    2,663

    Default

    Sounds like if you want to simultaneously have two versions available, let's call them WHAM and WHAM beta for sake of example, you still do really need to have two repos set up and manually merge changes amongst the two.
    Originally Posted by Theraze View Post
    I.. what? No?

  8. #8
    Senior Member Theraze's Avatar
    Join Date
    Mar 2010
    Posts
    8,741

    Default

    In your example, you have two versions of the same script with the same name. You can have one or the other as the currently available one, but not both in the active folder currently. To keep two versions both available, for example so that you can do:
    [default]
    consult WHAM beta.ash
    consult WHAM.ash
    You'd need to have two different scripts. Or maybe it's possible, but your example would have WHAM.ash in either beta or release format, not both available simultaneously...

  9. #9
    Developer roippi's Avatar
    Join Date
    Aug 2010
    Posts
    2,663

    Default

    (can people get away from using spaces in directory/file names? It's technically not breaking anything here but it's really bad form.)

    (and can we all use the same nomenclature? "active folder" means nothing to me. Do you mean working copy? Local copy?)

    Your issue has nothing to do with keeping things in two repos. If you kept them in two repos and left them as the same name, you would still get the same issue. If you made a dev branch in the same repo, you can rename the file to whatever you want. However...

    You can rename files when you make a dev branch; it's just not a good idea if you ever want to use merge, it creates a lot of issues. This is a limitation of svn - when you rename a file, it is the equivalent of deleting the old file and making a new one. So when you merge, some issues arise. I've not done it myself but I know that you're going to have to do some work to make it happen. Merging back into trunk would probably cause your file to be renamed. So I'd just.. not rename things. That is trying to do version control yourself by making multiple copies, you should be doing the VC through svn.

    Perhaps I should have included "svn switch" in my explanation. It is sometimes useful to switch back and forth between a dev branch and trunk, and that is the right way to do it, not doing it with renamed files. If mafia needs the facilities to handle pushing the switched working copy, we can talk about adding those.

  10. #10
    Developer roippi's Avatar
    Join Date
    Aug 2010
    Posts
    2,663

    Default

    Bale - I stuck your thread and changed the title of this one. I didn't really have a target audience in mind for this thread, but it's definitely more for scripters.

Similar Threads

  1. A request and possible challenge to the master scripters
    By Tom Sawyer in forum Turn-Burning Scripts
    Replies: 21
    Last Post: 12-18-2014, 04:27 PM
  2. xpath primer
    By roippi in forum Scripting Discussion
    Replies: 5
    Last Post: 10-03-2014, 07:39 PM
  3. Quick ? need a little help.
    By Cereal Killer 024 in forum Community Support
    Replies: 13
    Last Post: 11-06-2013, 07:12 PM
  4. Replies: 8
    Last Post: 08-16-2010, 12:18 AM
  5. Quick Q:
    By icon315 in forum Community Support
    Replies: 2
    Last Post: 02-05-2010, 02:39 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •