Feature Persistent command history / control history size

[Irrat]

kolmafia/src/net/sourceforge/kolmafia/swingui/panel/CommandDisplayPanel.java at main · kolmafia/kolmafia

KoLmafia is a cross-platform desktop tool which interfaces with the online adventure game Kingdom of Loathing 🗡️🍸 - kolmafia/kolmafia

github.com

Command history has a size of 20, which means you can execute 20 commands before it'll start forgetting stuff.

Two thoughts

So I had two thoughts about this, the first is a preference that lets you control how many entries are saved.
The second is a preference to control if the command history persists across sessions/accounts. Another setting to share history between accounts?

Controlling command history size

So the first suggestion of controlling the size, this is mostly because sometimes when I'm doing dumb stuff in the CLI I'll want to go back to some complicated command. Mostly when I'm testing javascript though, then making lots of tiny changes to the string I'm testing which all counts as a new history entry.

So that's an easy feature, although it may not be a general user consideration as 20 should be plenty for the vast majority.
Then there's a consideration of memory constraints if some really silly shenanigans occur where you raise it to an absurd number and have commands spammed.
Not sure if its possible to determine how "big" the list is in terms of memory usage. Basic counting of string bytes may be enough.

Persistent command history

The second would be storing of command history, between sessions, between accounts. And when I say between sessions, I generally mean restarting the program. Though logging out and in is a thing too I suppose.

My main two thoughts here is that if we share it between sessions, it should be stored in global. Otherwise in user prefs. Checking which pref we read can be dependent on a new preference which controls if all users share it or not.

The logging of the pref changing should be suppressed/silent as we do other prefs, no need to echo that undoubtedly spammy pref where the user is reading.

 

[fronobulax]

I don't remember much but in a career that predates widespread GUIs I have no recollection of ever needing a command line recall that needed more than 20 entries. I would just recall the previous command and edit. If your use case is just that you cannot remember things perhaps writing the command to the session log would suffice. On those few occasions where you needed to go back more than 20 you could just check the log.

If you can't be convinced that the problem is your workflow I have some comments.

Since preference files are human readable and editable as text files I would want the length of the value string limited. A preference value of 120 character or more kind of violates the user friendliness of having a text file. Even with the current length of 20 if the 120 has meaning to anyone else that is 5 character for the command and one for a delimiter, on average. So if you persist the command buffer to a preference I see a very long string in the future.

I'm asking the question and it may be there is no problem, but what are the delimiter candidates that are not tab, space or a valid character in a command line? Maybe #?

I find no personal utility in sharing across characters. When I am testing or debugging something that would be used by all characters I tend to write a script and thus the frequently repeated command would be the same for each character.

We still have unresolved file corruption or deadlock issues. I am leery of adding more data to GLOBAL while those remain unsolved although I suppose it might help us debug and resolve things faster.

If the command buffer is a preference then it will be persisted at no extra cost.

I'm not a big fan but then I don't debug JavaScript with KoLmafia either.

I could live with:

• A preference, that defaults to 20, and controls the size of the linked list that holds the command buffer.
• Reluctantly, a size preference, a persist preference and a data holding preference. Changes to the data holding preference would not be logged by default. The command buffer will need to be initialized at least once from the preference. Changing the buffer should change the preference so persistence comes free.

I don't think the Most Recently Used Script experience applies directly but maintaining the MRU list finally had such a performance hit that I no longer use it. The MRU had more file system activity than I expect the command buffer would have.

 

[ckb]

Since preference files are human readable and editable as text files I would want the length of the value string limited. A preference value of 120 character or more kind of violates the user friendliness of having a text file.

FWIW, my "mpAutoRecoveryItems" preference is way longer than 120 chars, so there is some precedence for a long string. "maximizerMRUList" also has the potential to be a very long string too.

I don't think I would use this feature very often, but there have been rare times when I have been testing a script or a command, I restarted Mafia, and I wished that I did not have to type out that same list of commands again.

 

[Veracity]

I don't think I would use this feature very often, but there have been rare times when I have been testing a script or a command, I restarted Mafia, and I wished that I did not have to type out that same list of commands again.

I also have been in that situation. Sometimes when I am debugging a script, and sometimes when I am debugging new ASH syntax.
In the latter case, after manually re-typing test "ash" commands for a while, I wise up and actually write a script.

 

[MCroft]

tbh, I would be most interested in duplicating the "history" command in bash.

"What did I do?" It might be nice if we had the ability to dump the history with an error so we knew what the user did.

I might also want to filter it, but not unless it was stored to a separate history file (like bash's history).

It might even be useful to dump it to a file as a "do what I did" script.

But I'd support more history as useful-to-me.

 

[Magus_Prime]

tbh, I would be most interested in duplicating the "history" command in bash.

This has occurred to me as well but, to be useful, to me, we would then enter the realm of having to pipe the history command through something like grep.

 

[MCroft]

This has occurred to me as well but, to be useful, to me, we would then enter the realm of having to pipe the history command through something like grep.

That's what I was thinking with the filter option, similar to inv item. To do that, I'd want a specific separate history file (eg Mad_Carew.history.txt, like .zsh_history)

 

[fronobulax]

I note scope creep.
I infer that there is some interest in persistence.
If no one beats me to it I am willing to do the following, hopefully in the next few days.

• Make the length of the existing command buffer a preference that defaults to 20.
• Make a new preference that contains the current command buffer as an ordered list, delimited by a sharp because that's what musicians call # and I am pretty sure it is not a valid character in anything that could be considered a command.
• When loading preferences from a file incorporate the preference contents into the command buffer.
• Update the preference when the command buffer is updated.

When loading the buffer from a file I'm not sure whether the buffer should be replaced or the preference entries should be added. I will think about this because it might make a difference when the command buffer or preference list are not full and if, for some reason, there is data in the command buffer when preferences are loaded.

Preference persistence will be managed by the existing mechanism.

Not sure yet whether this should be user or global. My probably non-existent use case demands user but this doesn't have to be all about me.

I need to understand the effect of the new preference on preference change logging and possible exclude it from that as a default.

I am not sure how to test yet and not sure I want to change PreferencesTest while https://github.com/kolmafia/kolmafia/pull/1889 is still in limbo.

I am posting this now because changing requirements and understanding are easier than changing code

 

[Irrat]

• Make a new preference that contains the current command buffer as an ordered list, delimited by a sharp because that's what musicians call # and I am pretty sure it is not a valid character in anything that could be considered a command.

I'm assuming you are including parameters in this? Perhaps just use \n newline character? Doesn't sound like a newline should be something that we can expect to see in a command. An escaped newline, sure.

 

[fronobulax]

I'm assuming you are including parameters in this? Perhaps just use \n newline character? Doesn't sound like a newline should be something that we can expect to see in a command. An escaped newline, sure.

My intent is to store whatever KoLmafia had already stored in the command buffer. If the buffer includes parameters than so will this. If the buffer does not already include parameters then that is a separate feature request.

I have not looked at NL's in the command buffer but I do know that I will not use one as a delimiter in a text file. Too hard to read/edit.

If there are already NLs in the command buffer then I probably will have to escape/unescape them when going to and from the preference,

Thanks.

 

[fronobulax]

After almost two months I made the time to do this. See https://github.com/kolmafia/kolmafia/pull/2047

I chose not to make the buffer size user configurable. Doing so would require rebuilding and replacing GUI elements or shutting mafia down and restarting. I didn't want to do or require either.

 

[MCroft]

• Make a new preference that contains the current command buffer as an ordered list, delimited by a sharp because that's what musicians call # and I am pretty sure it is not a valid character in anything that could be considered a command.

Technically, the character is called an octothorp. A sharp is more like an italic octothorp.

There's a bar in Houston called "Numbers" which has used "#'s" as their logo for decades. People of my generation die inside when our friend's kids refer to it "Hashtag's".

 

[fronobulax]

Technically, the character is called an octothorp. A sharp is more like an italic octothorp.

There's a bar in Houston called "Numbers" which has used "#'s" as their logo for decades. People of my generation die inside when our friend's kids refer to it "Hashtag's".

Never heard of an octoharp. Never heard it called anything other than a sharp in traditional music notation. Thanks/

 

[MCroft]

They're technically different symbols. The sharp has two vertical lines and two angled horizontal lines. The # has two horizontal lines an two angled vertical lines. Hopefully you're not getting ♯ as a separator.

“♯” U+266F Music Sharp
“#” U+0023 Octothorp

But we digress. Thank you for working on the persistent command buffer. It seems like a good idea!

​

 

[fronobulax]

The question of delimiter was raised at https://github.com/kolmafia/kolmafia/pull/2047

My criteria was basically convenience when the preference file is viewed or edited in a text editor that "prefers" ASCII.

 

[Magus_Prime]

@fronobulax - The new feature is working thank you. One thing I note is that the saved command history is global to all characters and not per character. I see that was the intent based on the note in the commit. Would making it a per character setting be looked upon as a reasonable request?

 

[fronobulax]

@fronobulax - The new feature is working thank you. One thing is note is that the saved command history is global to all characters and not per character. I see that was the intent based on the note in the commit. Would making it a per character setting be looked upon as a reasonable request?

I understood the OP had a preference for a Global preference but I wasn't sure why. I'd be glad to make it User if that is what (more) people would prefer. My concern is doing so in such a way that an existing Global preference is migrated and then no longer used. I know mafia code has done that before, I'm just not recalling a specific example. Let's see whether other people have an opinion.

 

[ckb]

There is some advantage to having this as a Global pref, in that if you are testing a command or debugging something, it is nice to be able to log out of Mafia, switch characters, and test some more. A Global pref preserves this.

 

[Magus_Prime]

My main two thoughts here is that if we share it between sessions, it should be stored in global. Otherwise in user prefs. Checking which pref we read can be dependent on a new preference which controls if all users share it or not.

FWIW - The option to have it be global or per character, via a preference option, was in the original request.

 

[fronobulax]

FWIW - The option to have it be global or per character, via a preference option, was in the original request.

Ahhh. I did not understand things that way.

To be honest I am not sure how interested I am in doing something as fiddly as making a new preference that controls whether another preference is Global or User. The entrance to the Rabbit Hole begins with the question of whether this new preference should be global or user.

What can a user do, within KoLmafia with a User preference that they cannot do with a Global one? I say "within mafia" because I can imagine several text parsing projects that would require per user, but I have no clue whether anyone is doing anything like that. So my answer is command line recall, and possibly editing, with a global preference might require more scrollback and could lose one character's commands if the other characters are verbose compared to the size of the buffer. If the latter is the issue is making the buffer size an acceptable alternative?