Multitool - An implementation and feature discussion.

fronobulax

fronobulax

Developer
Staff member
In the discussion of a nag feature (here) I speculated about Multitool, a new thing that users could download that would check for various mafia preconditions. My original vision was a shell script that only used native OS commands and there would be one each for *nix, Mac and Windows. A suggestion was made to build it in Java and target Multitool for a very old version (VOV) of Java. A user would download Multitool. If it didn't run then the user either had no Java installed or something older then VOV. If it ran then it would check and report some conditions. Then someone suggested Multitool could be a launcher . A user would download Multitool, it would check Java and KolMafia versions and if they were compatible then launch mafia.

Multitool needs to be in the same directory as KoLmafia (or in a directory that would have KoLmafia downloaded into it). Multitool would assume naming conventions that allowed a version number to be extracted from a file name for Multitool and KoLmafia. Version numbers would be assumed to be monotonically increasing with time. The lower version number corresponded to the older version. It is hoped that the environment Multitool runs in is identical to the environment the user expects mafia to run in and that parameters derived from Multitool execution apply to mafia.

For the first phase, Multitool would access a remote data source (GitHub, kolmafia.us?) and determine the latest version of Multitool, KolMafia and the lowest Java version needed to run mafia. It would then tell the user whether those were acceptable and make the user address the issues manually.

The next phase would add an option to download latest Multitool and KoLmafia versions and confirm that the current user could write to the current directory.

After that the next feature added would be to delete all but the latest versions of Multitool and KoLmafia.

The final phase would for Multitool to offer the option to launch the latest version of mafia and exit.

When the above work is done then there are some obvious enhancements such allowing multiple versions of mafia and some kind of preference settings so that launching Multitool would get the latest versions, purge, (anyone else remember VAX/VMS?) and launch mafia.

I have some simple code working, with lots of things hardwired that allows Multitool to launch KoLmafia and exit.

So, is Multitool a separate project and download or do we try and include it in KoLmafia?

If the former then I need help in setting up a GitHub project (licensing, workflows, testing, users etc.) and a build environment that uses a very old version.

If the latter then I need help making it so one Java file is compiled to it will run on a very old version. I need advice on whether Multitool is always present or whether a user launching KolMafia directly will get the "old" behavior.

Thoughts? Comments? Suggestions? Offers of help?
 
fronobulax said:
For the first phase, Multitool would access a remote data source (GitHub, kolmafia.us?) and determine the latest version of Multitool, KolMafia and the lowest Java version needed to run mafia.
Click to expand...

fronobulax said:
So, is Multitool a separate project and download or do we try and include it in KoLmafia?
Click to expand...
Given the first phase requirement, I feel like it might be easier to have Multitool (or launcher, or whatever) be in a separate repository under the KoLmafia umbrella project so we can simply check whether the git revision at HEAD matches. That also has the benefit of not polluting the main repository every time you commit a new change to the multitool.

If we later decide we want some sort of hybrid approach, we can always hook these up via git submodules.

Also, I'd imagine that the final phase would involve offering to download an appropriate version of Java, if necessary.

fronobulax said:
If the former then I need help in setting up a GitHub project
Click to expand...
It shouldn't be too hard to create something to play around with -- since you're a member of the organization, the "new" button on the KoLmafia project page will take you to https://github.com/organizations/kolmafia/repositories/new, where you can create a private repo to play around with.

fronobulax said:
(licensing,
Click to expand...
LICENSE.md currently uses 3-clause BSD, so if that's good enough for you, then that should be fine.

fronobulax said:
workflows,
Click to expand...
For workflows, you can start by copying some bits in https://github.com/kolmafia/kolmafia/tree/main/.github/workflows into a similarly-named subdirectory. You won't need npm.yml, and you'll need to make some small changes (e.g. java version 21 -> 8 or whatever).

fronobulax said:
testing, users etc.) and a build environment that uses a very old version.
Click to expand...
For build environment / testing, you can start with pared-down versions of the various files in the root directory that contain "gradle" in their names (build.gradle, settings,gradle, gradle.properties, gradlew, etc). We currently control the Java version via two entries in gradle.properties; while current JDKs should be capable of cross-compiling against older targets, the java { toolchain { ... } } stanza should be in charge of automatically downloading an appropriate toolchain for you.
 
I've slept on it and clarified some of my ideas and will be slowly working on them.

I think the cleanest thing is separate repository and jar. People who want to use it just opt in by downloading and running Multitool jar. People who don't want to use it do what they do now to acquire and run KoLmafia.

I didn't really want to do a lot of copy/prune for infrastructure from KoLmafia but I certainly can and will do so.

I'm fearful of making a wrong choice and having to live with it but I should also remember that this is a small enough effort that refactoring various things would approach being trivial, in contrast to similar tasks in KoLmafia.

I deferred actually downloading Java for two reasons. Once upon a time Oracle's version was mandated for my work and other implementations were untrustworthy. So I cringe at adoptium even though there is no rational reason to do so and in some circles Oracle is actually the "wrong" choice. I also felt if Multitool downloaded Java then it should at least offer support for installing it and I didn't want to open that can of worms. But yes, Multitool should offer some support for upgrading Java so what I am planning defers that for a while.

So I am working on a repository and build system and will either share or be back with questions. Thank you.
 
I "installed" gradle by copying files from KoLmafia. I am getting several "./gradlew: Permission denied" failures. What is the correct way to install and configure gradle for use in GitHub workflows.
 
(I don't have access to fork the repo or to push to a new branch, so no PRs from me for the time being. I did confirm that the "Permission denied" issue is a simple file permissions issue, though.)
 
Thank you. The good news is I am actually engaged in this project. What I should have made clear was that the permission error occurred on GitHub when trying to run workflows on my currently private repository.

I will look at the YML files and see if I can discern something I should have copied or kept and didn't. When building locally I think I see different gradle "deprecating features" messages so it may also be that the new project has an issue with which version of gradle.
 
fronobulax said:
What I should have made clear was that the permission error occurred on GitHub when trying to run workflows on my currently private repository.
Click to expand...
Yes, I saw that. As I mentioned, it's a file permissions error -- the Gradle wrapper doesn't have the executable bit set.
 
heeheehee said:
Yes, I saw that. As I mentioned, it's a file permissions error -- the Gradle wrapper doesn't have the executable bit set.
Click to expand...
OK. Then how do I set it on GitHub? I Googled and got a git command that included chmod but I couldn't figure out how to execute it for the GitHub environment. I tried the command locally and GitHub desktop told me there was a changed file but when I tried to push it to GitHub it said nothing had changed.

When I get tired of writing code I'll focus on GitHub and make the repository public so other folks can look at what I am not doing.

Thanks.
 
I am building releases on GitHub so I think my workflow issues have been resolved. Thanks again.

I have a laundry list of things I would like to do before 'going public". Working on that until I get stuck again.
 
When I go to https://api.github.com/repos/kolmafia/kolmafia/releases/latest I get JSON with a lot of info about the state of KoLmafia releases. When I try https://api.github.com/repos/kolmafia/multitool/releases/latest I get JSON that says "Not Found". Both of these checks are in a browser.

I can go to https://github.com/kolmafia/kolmafia/releases/tag/r28323 and https://github.com/kolmafia/multitool/releases/tag/r28 in the browser and see very similar things.

Is the "failure" for multitool because the repository is private or is there something I have to do to enable the API or is there something else?

Thanks.
 
fronobulax said:
Is the "failure" for multitool because the repository is private
Click to expand...
Exactly -- that's an authentication issue. Often, for security reasons it's preferable to return Not Found instead of, say, Unauthorized so you don't leak side channel information (in this case: whether a specifically-named repository exists).

docs.github.com

Authenticating to the REST API - GitHub Docs

You can authenticate to the REST API to access more endpoints and have a higher rate limit.
docs.github.com docs.github.com
and for completeness: https://docs.github.com/en/rest/releases/releases?apiVersion=2022-11-28#get-the-latest-release
 
I can build locally using the compiler and build and bundle classes that are allegedly compatible with Java 8. I cannot do that with the gradle build. The build is copied from mafia and when I change the source target to 8 it wants the toolchain to use Java 8 and then fails to download Java 8. Any obvious suggestions?

I'm fairly close to exposing my project so if the question is better answered by inspection then I am fine with waiting until there are files to look at.
 
I can see the entire private repository because I am part of the KoLmafia organization. I cannot push to the repository, or fork it.

It is not publicly viewable.
 
OK. The cat is out of the bag.

See https://github.com/kolmafia/multitool

In its current iteration, download the latest jar file from GitHub to the directory containing KoLmafia or a clean directory that will be the home for Kolmafia. Open a command line in that directory. Launch multitool with "java -jar multitool-rXXX. jar" and it will run, display some information and stop. "java -jar multitool-rXXX. jar run" will launch KoLmafia from the current directory before it stops. When it runs it will download the latest version of multitool and KoLmafia if there are newer version available at GitHub.

As a GitHub project I would like to to be as close to KoLmafia as makes sense. It needs the PR review and approval enabled and the KoLmafia staff should have the same privileges on multitool that they have on KoLmafia. I'll gladly accept help.

In theory the jar file should run on Java 8 or newer. But I have not tested that.

It is building other artifacts besides the jar. I'm not sure that is needed. Should those be removed?

Gradle builds "shadowjar" which is a way of bundling dependencies. multitool doesn't have any (yet). Should the build file be simplified?

As a philosophy discussion, should I make a GUI? Do I need to support headless? There are only a few preference equivalents - are they in a persistent file or do I force the user to use command line arguments or just make them click something every time?

The code needs a lot of help. I need to actually get the preferred Java version for KoLmafia from somewhere. I need a purge option that is smart enough not try and delete a running file. Lots of things work but are not robust. The code organization is suspect or non-existent and some implementation is Java 8 compliant even if it doesn't need to be. My efforts to avoid regex are pretty obvious. I need to think about a log file instead of or in addition to stdout.

I am having trouble refreshing my local main after I push a commit. I don't know if that is the way I set things up on GitHub, from using GitHub Desktop or whether IntelliJ isn't properly setup. I might have questions.

I think it should have tests but have been waiting to figure out code organization first.

I might experiment with issues in addition to or in place of a KoLmafia.us thread.
 
Hey all, I guess I am not as vanished as I thought.

It would definitely be interesting to see what the CI/CD pipeline in GitHub would look like with this integrated into KoLmafia.

I don't have a lot of time (job/spouse/opensource, pick any two), but building and installing for MacOS were pretty much my best contributions to KoLmafia, so I can help with that again.

General thoughts:
Do we need to set JAVA_HOME ($JAVA_HOME or %JAVA_HOME%)?

Do we expect the jar to be clickable by users and work as expected? What about when users have different settings to run vs. click.

I think we should be able to store the data you need as a .json file in Github as part of the project and retrieve it.
So along the same lines as https://github.com/kolmafia/multitool/blob/main/README.md, you'd have https://github.com/kolmafia/multitool/blob/main/current_versions.json That's sort of how garbo and other loathers projects do it, basically (they check the checksum of the package, but similar idea).

How will users debug multi-tool errors when they happen?

New features to consider, with no consideration of effort.
  • There are a number of things that can be checked here that might be helpful.
    • Is KoLmafia currently running?
    • Can we reach the internet/our cloud drive where we store our data files, etc.?
    • Is it maintenance, or almost?
    • Have my preferences been zeroed out?
      This would be a better place for the code I tried to add to help with prefs getting deleted.
    • backup/restore prefs
  • I'd like to see the release notes since the last version I downloaded.
  • I'd like a link to the install instructions in the wiki and the forums to ask for support. Maybe also to loathers for scripting support.
  • Working Directory questions
    • Is more than one user in the working directory?
    • Can we change the working directory?
    • Can we list the characters in the working directory and let the player choose one to start?
    • MacOS and Debian have specific and different working directories that separate data from code. Windows supports that in a way that wasn't reasonable back in the Java 5 days. It sure would be great if we used the multi-tool to at least allow that as an easy option for Windows users, who already have enough trouble in their lives... :)
  • Are there reasonable things to edit in prefs or globals to make things easier?
  • Can we sit in the background and watch for KoLmafia launching and crashing?
  • As a dev, I'd like all those features, but pointed at the version I built from source. A use local jar option would be nice, moreso when there are features from the backlog that would be useful with a local build.
  • I'd like to be able to put command line parameters into a text box and have them passed to the app.
 
Last edited:
You must log in or register to reply here.
Back
Top