Beta testers

Winterbay

Active member
Yeah you pretty much want to use features in git. The main issue here is a lot of people are expecting the daily builds to be safe, but they've never been guaranteed to be. I don't like the idea of an additional experimental build, the daily builds are already experimental.

The problem here is that the standard answer to "this doesn't work, what should I do" currently is "go download a daily build", without any mention that these are experimental. Also the builds.kolmafia.us page does not state anything about them being experimental. In other words nowhere is it said that those builds are experimental.In fact the only thing that indicates this is the existence of the point releases on the Sourceforge page and to a person not at home with program development this may not be clear at all.
If we want to go with a status quo when it comes to builds I would then suggest to at least add a comment on builds. that the builds there are experimental that also direct people to the "How to do a BR/FR" thread in case they have problems.
 

Raijinili

Member
How about a simple pop-up with a changelog in Mafia (looks up a webpage and starts reading from the right version number)? Or even a button on the login screen to view an online changelog? It can be auto-generated, right?

Having a link on the changelog page for each version to the SVN thread may also encourage participation on the forums, which would, as I believe, bring more people in and make it harder for them to have an impersonal relationship with the coders.
 

heeheehee

Developer
Staff member
Just got around to reading this thread -- don't quite remember everything that was discussed.

As mentioned by several folks, git is very nice for various things, especially branch management. Also, you get a local repository in which you can sandbox everything before pushing anything. That in itself is an excellent reason to switch away from SVN. There is a bit of a learning curve, though.

I would imagine that a division of Mafia into (at the very least) stable and unstable branches would cut down on complaints at least to some extent; that is in itself a good idea. If we really do want to switch to git and keep SVN around as our stable branch, guess what? There's a git command for that. (git ?= Emacs of version control)

Granted, looking back on the thread, I see that some folks are uneasy creating a new branch for experimental builds. I can certainly see this. The issue is that Daily Builds aren't really viewed as "experimental: watch out, there might be bugs", they're seen as "if you want Mafia to keep up to KoL, use these". Part of the issue, yes, is that stable builds (i.e. point releases) happen very infrequently (in code/development terms). We could modify these point releases to have a third field that is pushed much more frequently (i.e. whenever there's a KoL change that makes a new release necessary, or a critical bugfix in Mafia). (also, updating the major version number should probably mean something other than "oh, the minor version was at 9, and we rolled out a new release". Heck, my version of Chrome tells me it's version "14.0.825.0", and I'm pretty sure 825 > 9.) The main issue, in that case, is getting people to stop using the dailies.

I probably have more to say, but I realize that the longer I drag this post out, the more likely everyone is to just skim its content. I'll just add more comments as the thread moves along.

(I should get around to submitting patches at some point... if I have the time...)
 

slyz

Developer
If we really do want to switch to git and keep SVN around as our stable branch, guess what? There's a git command for that.
Does this mean that a dev who wants to release an "unstable" version could use git to do it, and would also be able to merge it back to the stable branch over at SourceForge? This would be great.

We could then look at how to integrate those experimental builds in http://builds.kolmafia.us/, since that's probably where the vast majority of users download their builds.

It would give a little more work for the devs who want to do that, but if enough users choose to download those builds and give feedback (I was hoping more than 5-10 dedicated beta testers), it might be worth it in some cases.
 

heeheehee

Developer
Staff member
Does this mean that a dev who wants to release an "unstable" version could use git to do it, and would also be able to merge it back to the stable branch over at SourceForge? This would be great.

We could then look at how to integrate those experimental builds in http://builds.kolmafia.us/, since that's probably where the vast majority of users download their builds.

It would give a little more work for the devs who want to do that, but if enough users choose to download those builds and give feedback (I was hoping more than 5-10 dedicated beta testers), it might be worth it in some cases.

I haven't really looked much at that git svn command, since I've mostly just switched to using purely git, but I think you should be able to cherrypick whatever changes you want and just commit that as a single SVN commit.

For what it's worth, I'd be willing to be one of these beta testers. I mean, right now I just update from SVN and build manually; it's really not any more effort. And I'd probably get around to contributing code.
 

fronobulax

Developer
Staff member
There are many issues being discussed and confused. Perhaps I use different meanings for "stable" and "experimental" than most folks? I assume most people are using them as synonyms even though the "evaluate and discard" aspects of "experimental" have never been a part of mafia development. By the time it gets checked in all code (whether new content, new feature or fix) is presumed to be part of the mainline. No discarding. The only exceptions have been when one of the devs becomes convinced it was the wrong thing to do. Note that the people who determine what features get implemented are the devs and the user community's opinion only matters if it results in a dev making a change. Luckily none of the devs has been on the kind of ego trip where the community is totally ignored but that is because veracity, hola and jason not only established a standard to follow but are in a position to enforce those standards.

I have seen almost no complaints because KoLmafia was unstable in the sense I understand it. I have seen complaints about how KoLmafia does not support current features of KoL and complaints about newly implemented KoLmafia features. The latter are never going to go away. Beta testing, sandbox builds and forks are going to do very little to address that problem. At best, when a feature is released into the mainline release the whiners can be countered with "Well 98% of the beta testers liked it. Perhaps you would like to be a beta tester for the next round?"

I fully agree that having a smaller community evaluate implemented features is an improvement over just releasing those features to the larger community but I'm not sure I've seen a workable suggestion (mine included, now that I reflect on it) yet.

As for git let me just observe that any solution involving git either needs to be transparent to the folks with commit access at SourceForge or there needs to be a compelling reason to use git in place of or in addition to svn. I'm not saying there isn't one but I haven't heard it yet. This may be a sore point with me just because I have seen projects crash and burn because of tool selections. Generally the version control system supports the configuration and release manager roles. Perhaps agreeing on a release process might be clarify the need for a different tool?
 

Grotfang

Developer
Sorry for not weighing in before now. I've been away for quite some time with very limited internet access.

I have a few comments going through the thread, so bear with me!

Testers would need to be able to compile their own code off of sourceforge, and know how to create and apply patches.

I disagree. I understand why you would like this, but code that doesn't compile is very rare and should be resolved before submission. Competent testers do not need to be competent at writing code, and any tester should be able to test whether a release does what it is meant to do.

If I was given KoLmafia to manage and wished to correct the situation with the lowest impact I would do two things. I would create and manage an "approved request" list. I would then require that all code committed contain a reference to one or more items on the approved request list. This being the case, devs who only wished to deal with code and not the user base would only have to scan the approved request list for something to do. Devs who had a good idea would not commit it without some community discussion or awareness (i.e "approval").

I think, again, this is a lovely idea, and if this were a more commercially minded project would be a great way of managing it. Unfortunately, it misses the fact that the software is often being developed by a dev to shape it the way they want it. While this needs to be controlled (to make sure it remains useful and accessible to everyone else), it shouldn't be abandoned, as that risks making development of the project less appealing. I think the idea of a job list and an obligation to stick to it runs the risk of feeling as though you are working pro-bono for other people.

Get more people to test changes before they are experienced by all the regular users.

Restrict access to daily builds to beta testers, as big or as small a group as required, (and to people who are happy to build it themselves) and spin a more regular rtw, maybe every couple of weeks

If a case is made for more frequent releases then we need to make sure that someone besides veracity can spin them and post them to SourceForge.

I'm going to make that case :)

Daily builds currently give anyone who wants it access to the latest code developments. SVN gives a changelog. Our repository is always going to be available to anyone, and therefore there is no limit to who is able to access the most recent version of mafia. Instead of calling them daily builds, why not call them Beta Builds, or just make them accessible in a beta testing (or, testing) area of the forums?

Feature requests and bug reports are exactly that and are for the main project -- the official releases. Have a separate part of the site for feedback for the beta program. Every two weeks release a new version on sourceforge. This kills many birds with one stone. Firstly, the sourceforge site becomes more up to date, with users being prompted to update to versions that actually work. Secondly, we get the feedback for new ideas from users with enough time to spare to change things before general release. This also means we can be more experimental with development, since we can let people compare new ideas with their older counterparts in a more constructive environment before developers like Roippi, who work hard to do amazing things while still being very new to development, are exposed to the more vicious masses. (I still love what you did to daily deeds, Roippi!)

As for whether git, subversion, whatever is the best way to handle it... meh. We'll work something out, right?
 

lostcalpolydude

Developer
Staff member
I agree that more frequent version updates on sourceforge would be a good solution. Currently the updates happen about every 2 months, which means people have no choice but to use experimental builds if they want to keep remotely up-to-date on new content.
 

roippi

Developer
I disagree. I understand why you would like this, but code that doesn't compile is very rare and should be resolved before submission. Competent testers do not need to be competent at writing code, and any tester should be able to test whether a release does what it is meant to do.

I think you may have misunderstood what I was saying? I was saying that a beta tester should be able to build from source. Admittedly I phrased this poorly.

I agree that more frequent version updates on sourceforge would be a good solution. Currently the updates happen about every 2 months, which means people have no choice but to use experimental builds if they want to keep remotely up-to-date on new content.

After having seen all the steps it takes to spin a point release, I'm a little squeamish about agreeing. It's a rather time-consuming process. And very daunting. As always, just because something is difficult doesn't make it wrong, just pointing out the speedbumps. Though I would make that argument if we were considering making a weekly point release. :)
 
Last edited:

Grotfang

Developer
I was saying that a beta tester should be able to build from source.

I'm saying that they already can (as can everyone else), but it shouldn't be a requirement.

After having seen all the steps it takes to spin a point release, I'm a little squeamish about agreeing.

I wonder how much of it could be automated. You're right though in that this could be a barrier if the answer to my wondering is "not enough".
 
Strictly out of curiosity, but what are the extra steps needed to spin a full release?

Not talking in terms of uploading the files to the appropriate places once done, but the steps to build it.

What comes to mind are updating the data files (mallprices is the only one that seems like it'd take a significant amount of time), changing the version number in the build properties file, and then compiling it (with maybe a different parameter to make it a release version instead of just another revision). I'm sure there's stuff I'm missing, but I always compile from the source, and I don't notice any significant change when a new version rolls other than the properties file changing when normally it doesn't.

More on topic:
I really don't see how making a separate branch for special users (or rather users that -want- to, would you really prevent anyone?) to test out things would work. The current model is pretty set as is: most changes are tested on the dev's machine (or patchmaker's) before being uploaded, and if a crucial bug is introduced it gets fixed pretty quickly.

Trying to avoid non-useful feedback (ie "annoying") doesn't seem like good motivation for making a huge change to the dev process, because you will -always- have people that don't like everything.
Personally, I think the current model is superb; not without it's faults, but it's very elastic and faster than any other software I use at deploying fixes and implementing new features.

If anything, the userbase should be made aware that KoLmafia is -always- in an experimental state, as the system it's designed to augment is constantly evolving and out of the hands of the creators of mafia.

Making a new "version" of mafia for just the beta testers would either a) leave out the normal by-revision/daily users from new content or b) only be used for bigger projects and changes to mafia, like roippi's excellent Daily Deeds overhaul.
I'm not a fan of A, but B sounds nice, just that it'd be underused and potentially not worth the effort to set up a formal system to do so.
 

fronobulax

Developer
Staff member
In a clean room environment all a beta tester would get would be a jar file. So I am in violent agreement that a KoLmafia Beta tester should NOT have to know how to build from source. In fact if a so called beta tester can and does build from source how does anyone know what was actually tested by everyone? Letting a beta tester build is A Bad Idea. IMO, YMMV.

There are two pieces to making a release that I can figure out. One is building everything. I assume because "everything" includes a dmg file for a Mac, that the process has to occur on a Mac. It does seem to be automated via the build.xml file but it failed on my Windows system, which could be a configuration error but I kind of believe otherwise ;-) Point being that it is automated. Second piece is getting the files to SourceForge. At present only hola, veracity and Jason have the necessary permissions to upload the release files. One of them would have to build (or otherwise obtain) the release files and upload them OR delegate the job to someone else and assign appropriate SourceForge permissions.

There are also some forum updates associated with a release. I think veracity edited the page at GD and I know she posts release notes here. I do not know whether either is automated but someone else could certainly post the release notes here.

Edit:
@bordemstirs
Concurrent posts. The build.xml file automates everything you mentioned except updating mallprices which I seem to recall was a script someone ran. On an appropriately configured machine I believe the builder edits (and commits) build.properties and then runs ant with a target of dist. Of course, I could be wrong since I cannot actually make it work on a Windows box...
 
Last edited:

roippi

Developer
Various steps to spinning a release:

update data files
update mallprices.txt
update KoLConstants.java
generate release notes
create KoLmafia.app, KoLmafia.exe, KoLmafia.jar, and KoLmafia.tar.gz
upload to sourceforge, move various files around to be in the appropriate folders
make a post on the forums

A good deal of that can be automated, but creating One Automation to Rule Them All seems rather ambitious. There's two steps in ASH, one in a text editor, two in ant, and two done through a web browser.
 

holatuwol

Developer
I could combine the two steps in ASH, the one in the text editor and the two in Ant into a single Ant target if someone gave me the information on how those data files and mall prices are updated.
 

Veracity

Developer
Staff member
I just updated the "How to Spin a Release" sticky thread on the dev forum. Some of the steps were obsolete because of changes to the build process and changes to Sourceforge.

Updating the data files is manual. It basically consists of running various internal debug commands - checkitems and checkeffects, for example - to find obvious errors in our data vs. what the item or effect descriptions say.
 

matt.chugg

Moderator
I just updated the "How to Spin a Release" sticky thread on the dev forum. Some of the steps were obsolete because of changes to the build process and changes to Sourceforge.

Updating the data files is manual. It basically consists of running various internal debug commands - checkitems and checkeffects, for example - to find obvious errors in our data vs. what the item or effect descriptions say.

Is there any chance this could be put somewhere I could read it? I don't know how you (veracity) generate the release notes, but i'm sure a large amount of that could be automated using the database that the rss feed uses.
 

fronobulax

Developer
Staff member
Veracity says...
URL removed because Veracity seemed to not want it posted here for anyone to randomly stumble across. -lost

1) Make sure that the data files are up-to-date and accurate. We tend to enter new items into the databases continuously. That's easier, now that we can learn them by simply having them in inventory or seeing them in the mall. But, even assuming we have all the items, the data we have may be inaccurate.

The "checkitems" command fetches the description of every item, parses it, and compares what it learns to what we have in our internal data. That can be a lengthy process, so it caches them in data/itemhtml.txt and only hits the server for descriptions it does not have cached. It generates data/itemdata.txt. This contains tradeitems, itemdescs, equipment, and modifiers for all items, as generated right from the descriptions. Any discrepancy with our built-in data will be marked with a ***.

This is less useful than it used to be, now that we have expressions for modifiers - and "Familiar Effect" modifiers for hats, for example. But incorrect equipment power, autosell price, level requirements for food, and so on, are caught.

Also, the caching means that if KoL changes an item description, we will not notice, if we have the description cached. Caching is useful for repeatedly rerunning the "checkitems" command, after incrementally trying to fix errors, but you should probably start with an empty itemdata.html.

2) Update and check in a new mallprices.txt. I use this script to get the mallprices for every item. This takes a long time:

Code:
void main( int first, int last )
{
    foreach it in $items[]
    {
        if ( to_int( it ) < first )
            continue;
        if ( to_int( it ) > last )
        {
            print( "Done!" );
            break;
        }
        if ( !is_tradeable( it ) )
            continue;
        int price = mall_price( it );
        if ( price < 0 )
        {
            print( "No " + it + " in mall!" );
            continue;
        }
        print( to_string( it ) + " costs " + price );
    }
}


When it finishes, copy data/mallprices.txt to your svn tree and check it in.

3) Edit KoLConstants.java and set VERSION_NAME and VERSION_DATA appropriately.

(And if REVISION is not null, make it null. ant edits this file to insert the current revision before a non-dist build and restores it afterwards - unless you ^C the build. I've done that. It's a mistake.)

Edit build.properties and set version, version.date, and svn.revision.base appropriately (although that last one doesn't seem to work as expected/hoped).

Check in KoLConstants.txt and build.properties with comment "Bump version to 14.8", for example

4) Generate the release notes, such as they are. build.xml has a "notes" target which runs src/net/sourceforge/kolmafia/utilities/ReleaseNotes.java. This reads "history.txt" from the ROOT_LOCATION and writes "release.txt" in the same place.

history.txt is the output of "svn log"
release.txt is the "abbreviated revision history" complete with bbcode links to the sourceforge revision.

% svn update
% svn log > ~/Library/Application\ Support/KoLmafia/history.txt
% ant notes

- edit generated release notes (~/Library/Application\ Support/KoLmafia/release.txt) to have only the new revisions in the release
- Prepend holatuwol's release announcement (util/boilerplate.txt)

Voila! You have the text of the announcement to be posted on kolmafia.us, once the release is available.

5) Generate the four files we will upload to souceforge.

% ant clean
% ant dist

The dist directory now has exactly the files you need: KoLmafia-14.0.dmg, KoLmafia-14.0.exe, KoLmafia-14.0.jar, and KoLmafia-14.0.tar.gz.

6) Upload those files to sourceforge.

Every single time I try this, sourceforge has changed something. The following are current as of August 2011.

- log into sourceforge
- on the kolmafia page, goto Files

You will see multiple folders, one for each release,.

- Click on "Add Folder" and name it "14.0" (or whatever).
- Go into the new folder folder.
- Click on "Add File" and upload your four files.
For each uploaded file, the "i" lets you change "info" about the file.
- Make the .dmg file the default for OS X
- Make the .exe file the default for Windows
- Make the .jar file the default for everything else.

7) Make an announcement on KoLmafia.us. Title it "Version 14.0" and include the edited release.txt you made earlier.

You are done!
 
Last edited by a moderator:

holatuwol

Developer
As mentioned by several folks, git is very nice for various things, especially branch management. Also, you get a local repository in which you can sandbox everything before pushing anything. That in itself is an excellent reason to switch away from SVN. There is a bit of a learning curve, though.
From my perspective, a formalized branching approach would be a layer of bureaucracy on top of a project where you cannot do unlimited testing -- you test once a day on the character that you play on (assuming you play at all), and your use may never even see the new feature that was added so even a team of people maintaining the "master" branch would probably take weeks/months before being able to validate a change, let alone merge it in.

If you instead merge stuff in based on gut instinct, well ... there's no real advantage over the centralized SVN model other than being able to say, "Hey hola, you just completely broke chat again..." if the change broke a frequently used feature. Not that this has happened before or anything. >_>

With all that being said, github theoretically gives you a read-only SVN interface to the same github repository. Now, I have no idea how it works with branches (I'm guessing it's a read-only view of master?) so maybe it's possible for KoLmafia to switch to git and just use the normal SVN-like commands (just replace "svn update" with "git fetch upstream; git rebase upstream/master" and replace "svn commit" with "git commit; git push origin") and see how it works and make use of branches if/when they become applicable.

If you want to experiment with it, you could try forking git://github.com/holatuwol/kolmafia.git (which I created just to mirror all the code currently in SVN, with no promises to keep it updated) and see how you would interface with it through github's SVN, and make some small minor text file changes in your own branch and see how that all plays out.
 
Last edited:

heeheehee

Developer
Staff member
If you instead merge stuff in based on gut instinct, well ... there's no real advantage over the centralized SVN model other than being able to say, "Hey hola, you just completely broke chat again..." if the change broke a frequently used feature. Not that this has happened before or anything. >_>.

Aww, hola, we still love you. :D

I still maintain that having a local repository would benefit at least the small handful of people who like to stop after writing every handful of lines of code, check that what they've written compiles, commit what they've done so far, and continue mucking around with Mafia's internals.

(also, instead of "git fetch upstream; git rebase upstream/master", "git pull"?)
 

holatuwol

Developer
(also, instead of "git fetch upstream; git rebase upstream/master", "git pull"?)
I'd read that 'git merge' (which is implied with 'git pull') creates a merge commit to mark when you merged everything. Assuming that's true, then for purposes of making things like SVN, rebase matches up better.
 
Top