Bug - Not A Bug Blue screen causes reset of many selections

Not sure if this would be considered a bug. Had a blue screen while mafia was open and logged in to KOL. After restarting and getting back in to mafia, logged in, and all daily deed completion had been forgotten, all choice adventure selections that I have looked at so far have been reset, hp / mp maintenance selections reset, current mood selection (but mood definitions are ok), custom combat, automation scripts, saved mod-max strings, misc. quest progress, skill usage, rave combos, etc, etc. Not a huge deal, but if I've been relying on non-default settings somewhere for the last - year? - I may learn the hard way ;-)

Is this a known behavior? Is there a known way to recover? Is there a way to prevent this somehow? (yeah, don't run windows ) Are there other effects I should look out for?

or, you know, maybe this is just good for a laugh...

Do you have "save options to disk whenever they change" checked? (under General in Preferences)

I believe that with this unchecked changes should only save when you exit, which can be bad if there is a crash.

Well, it's checked now! I think it was before. Thing is, it lost stuff that had been set for months. Is there something written somewhere on exit that is checked on restart, and if not seen, assumed to have a near blank slate?

Well, if it found no settings file, or a blank one, it would have populated with defaults, so maybe the file was damaged in the crash?

I've not actually looked at that bit of the code, though.

That's what I'm thinking. In any case, long gone for me, thought I'd mention it in case there was something besides corrupted files to be looked at.

A lot of resets will synch up when I ascend next, but I wonder what sort of multi-ascension progress has been lost. If any - there are some areas where nothing was lost - custom daily deeds is one. I bet there's a pattern here, I'll keep my eyes open.

My guess is that character-specific settings were lost, but settings that apply to all characters were not.

Given that mafia tracks settings by saving them in .txt files, there is absolutely no way to account for the OS crashing without saving the file. If these settings are important enough to you, then making regular backups just in case is a good idea. (That applies to all files on your computer, not just mafia files.)

settings\[charactername]_prefs.txt ? 300kb ? The top 2/3 of the file is repeated "\u0000" ?

And - as is frequently the case with non-tested backups, my backup settings were adequate when I set them up, but have been missing more recent file locations. Fixed now.

Looking through that file, yep, lots of lost stuff. Demon names - reset these manually, were in BCC profile. What else is hard to recover? Forgotten recipes...?

Suggestion: if the file is held open in such a way that a crash may corrupt it, it makes sense to make a copy at the start of each session (or when exiting a session), and then check at the next start for current file corruption, and restore from the copy if so.

The file isn't held open. Mafia is constantly telling your computer to save a new copy of the file, whenever any value in it changes (if you have that setting enabled). If the computer is saving the update when it crashes, bad things happen.

lostcalpolydude said:

The file isn't held open. Mafia is constantly telling your computer to save a new copy of the file, whenever any value in it changes (if you have that setting enabled). If the computer is saving the update when it crashes, bad things happen.

Click to expand...

The file is held open whenever you write into it.
"Copy (or rename) the original with a .bak or ~ extension, write new version, then delete the backup" is a time-honored strategy to prevent these "bad things" from completely wiping your data (you will always have a previous consistent version of the file, that you can get back to). And it can also serve as a "another instance is already trying to write this file at this moment" file lock (if you can't exclusive-create the backup file, then it already existed, indicating that either your writes would be corrupting the file, or that your process was forcefully terminated before it could finish writing the file last time (which would mean that you want to restore from the backup)) .

If someone wanted to do something like that (it won't be me), I would suggest creating the backup file at the start of the session and then ignoring it after that. That would remove the risk of losing everything while not having a meaningful impact on disk activity, and losing the stuff just from that day in the case of a major issue seems like an acceptable risk.

Given the nature of the interactions and the storage scheme, a temporary backup every write might be excessive. That would be an interesting experiment, for shits & giggles. Yes, losing a session would be acceptable, but not potentially a character's entire history. It would be the simplest option.

Or, y'know, testing one's own, um, "entertainment continuity plan" ;-)

But this does bring up another question - text file storage vs something else? I should look at the code, but I'll start by asking: how are reads/writes done? Given that the preferences are stored in alpha order (and what's the bulk of stuff at the beginning?), is some sort of indexed access used? Is there a data-access layer separately factored out? You know where I'm heading with this: would moving to simple database usage be useful? Lots of pros & cons there, I'm sure. Not that this would necessarily help the corruption-on-crash issue under discussion. It could conceivably even make it worse.

Anyway, I appreciate the feedback.

Alternatively, if you're using Dropbox to sync your files between computers, it might have version control and allow you to restore the version of the file before it was damaged. Or if you have any other sort of version control system set up...

Only one computer, so no synching need. Using a net drive for backup storage. In combination, that means no cloud storage. Win 7 backup / restore does have versioning protection, but only if the files are covered by its backup process - which would have been in this case, if I hadn't needed to customize included folders (to exclude a drive with a bunch of VMs). I assumed that having selected backup of "user's folders" meant any additional ones created there later. That apparently only works if custom selections weren't made, which makes sense. So, of course, I exposed myself to this risk. And I was just thinking of going to the store to buy another laptop today - the computer gods must have been watching.

fractalnavel said:

But this does bring up another question - text file storage vs something else? I should look at the code, but I'll start by asking: how are reads/writes done? Given that the preferences are stored in alpha order (and what's the bulk of stuff at the beginning?), is some sort of indexed access used? Is there a data-access layer separately factored out? You know where I'm heading with this: would moving to simple database usage be useful?

Click to expand...

Preferences are small enough that we just store them all in-memory in a big hashmap. Occasionally we serialize it to the preferences file - how often we do that depends on your settings. We should probably do that write operation in a background thread but it doesn't look like we do - I don't have the saveSettingsOnSet setting on, so I can't say if the performance hit is actually noticeable. I'd have to profile it to see how often writes actually happen and how long they take.

I think we have a sqlite3 client already somewhere in the source tree - I pulled it in when I pulled in the svn client. Preferences could be ported over to use a database behind the scenes without changing the public-facing API noticeably. However we just don't need most features that a database provides - everything fits in memory, we don't need relational queries, security is not an issue, etc. We don't even need the I/O benefits since we could just manage our writes to the pref file a better using threading, if that becomes an issue. So all that porting to a DB would give us is transactional locks to prevent corruption like you experienced. I don't think that's worth it.

Thanks for the explanation. I can always do a settings back-up on launch to time-stamped files and clean them up periodically. Doesn't have to be pretty.

Since the file is written all at once, could do as xKiv was saying, writing to a temp file and then deleting the previous file and renaming the new one on each write may not be much more of a performance hit anyway (and controllable by the existing save setting), and would pretty much eliminate this particular issue.

It's interesting seeing how much Mafia has actually been tracking for me, when it's no longer there.

I would actually second Theraze's recommendation about Dropbox even if you aren't syncing between two computers. Or at least some sort of versioning system/backup. The versioning system in Dropbox saved me from a similar crash before. I don't remember the details of what happened (power failure, maybe) but my prefs file was hosed and . Luckily, just had to log into dropbox and restore the previous version and I was mostly good to go. The main caveat with Dropbox is every once in a while it has a file conflict issue even though I only have one computer with Dropbox installed. Just my $0.02