So someone brought up how big their sessions files were, which had me check my sessions folder. 5.5gb
Granted, this is after a few years and with something like 3 accounts.
Still an impressive size.
Then I ran it with compression format of 7z and compression level of ultra which is not a real world scenario here.
185mb.
Took too long so I'm reluctant to run again with gzip, but I fully expect a similar compression ratio.
So it makes me think that maybe mafia should have some support for compressing our session logs.
When I say compressed, I mean archived into a compressed format of some kind. zip, rar, etc.
We could also expand this feature to introduce new functions "compress" and "uncompress" for scripts that may be bulky in their data storage/logging, but the main focus is session logs, which is the biggest waste of space that naturally occurs.
A bullet point list of my thoughts here are:
* This will be an opt in feature, no one will find their workflow breaking out of the blue
* A new global property introduced into mafia, which effectively translates into "archive sessions older than X days" where "X" is user defined.
* Archived sessions could either be stored in the same folder, or in a new folder "sessions_archived" to avoid issues with say grep
* KoLMafia has some refactoring so that it'll allow scripts to read achieved sessions as if nothing changed via "get_session".
* Perhaps a new function session_archived(string file) : boolean
* Each archive contains just one session file, no mega-archive containing all the sessions.
* Only after a session is archived and some verification has been done to that fact, is the session file deleted. We never want to lose the data.
* Only sessions older than 2-3 days can be archived. And only by the running user with a lock.file
* gzip seems like the compression algorithim to use, naturally supported in most environments and although it only supports storing one file: That still fits our use case.
Granted, this is after a few years and with something like 3 accounts.
Still an impressive size.
Then I ran it with compression format of 7z and compression level of ultra which is not a real world scenario here.
185mb.
Took too long so I'm reluctant to run again with gzip, but I fully expect a similar compression ratio.
So it makes me think that maybe mafia should have some support for compressing our session logs.
When I say compressed, I mean archived into a compressed format of some kind. zip, rar, etc.
We could also expand this feature to introduce new functions "compress" and "uncompress" for scripts that may be bulky in their data storage/logging, but the main focus is session logs, which is the biggest waste of space that naturally occurs.
A bullet point list of my thoughts here are:
* This will be an opt in feature, no one will find their workflow breaking out of the blue
* A new global property introduced into mafia, which effectively translates into "archive sessions older than X days" where "X" is user defined.
* Archived sessions could either be stored in the same folder, or in a new folder "sessions_archived" to avoid issues with say grep
* KoLMafia has some refactoring so that it'll allow scripts to read achieved sessions as if nothing changed via "get_session".
* Perhaps a new function session_archived(string file) : boolean
* Each archive contains just one session file, no mega-archive containing all the sessions.
* Only after a session is archived and some verification has been done to that fact, is the session file deleted. We never want to lose the data.
* Only sessions older than 2-3 days can be archived. And only by the running user with a lock.file
* gzip seems like the compression algorithim to use, naturally supported in most environments and although it only supports storing one file: That still fits our use case.
I just timed it and it took less than 5 seconds to archive an entire years worth of logs for one character.