map data corrupt somewhere

[bordemstirs]

I have a decent sized map that I load, change a few things (well, not really, but the potential is there) and then save back.

For some reason, and repeatedly after fixing, the first item in the map is put at the very BOTTOM of the data file, and (when viewed in notepad of all things) the first key is a different font than the rest of the text (and here I thought notepad insisted columns line up)...

I wouldn't really mind the slight disorganization, but even though the faulty data is -resaved- after further load/save cycles, the script seems to be unable to use it.

 

[bordemstirs]

Further Information:
If I delete the bad info -without- restoring it, the next item in the list gets the same treatment. But if I leave the bad line, nothing further goes wrong.

Any ideas what's going on or how to fix it?

 

[Grotfang]

Could you post the code? I am currently unable to replicate this.

My first thought is that the size of the map is unlikely to be the issue. I wrote a script for someone in Senam's a while back to mimic the functionality of the clan manager's leaderboard generator (it requires too much memory to work for that clan in normal operations) and that produces maps with over 17,000 entries with quite involved records very nicely. Maybe yours is bigger, but I can't see a reason why it would be.

So, excluding size, something else would seem to be at work. Without replicating, it's hard to fiddle and diagnose. Sorry.

 

[bordemstirs]

It doesn't happen immediately, and I'm also pretty sure it isn't happening during the files initial creation.
Here are the scripts, and, just so you can see, my corrupt data file. (Look at the bottom two lines)

There are multiple places in the script that modify the map, but only a few instances where it gets saved.

It seems to me that the bad save happens when the page in invoked without any form fields being set, which means the only function that saves any changes would be (now that I'm saying it, it seems backwards) the handle_forms function.

Script is incomplete, so don't judge its layout's choppiness just yet.

 

[jasonharper]

The garbage characters are a Unicode "Byte Order Mark", in UTF-8 format. In other words, somebody set up you the BOM!

I do not see any possible way that a BOM could have gotten inserted into the file just via loading/saving the map in mafia. However, if you at any point had edited the file with an editor that inserts a BOM at the start of the file (Windows Notepad apparently does this under certain circumstances), the first key in the map would have gotten corrupted, and would then get sorted to the end of the file after the next load/save cycle. Your prototype planner.txt file is basically pre-corrupted in this manner: the first line is actually being considered as a valid map entry instead of a comment (with "<BOM>#Type" as the first key), but this isn't actually causing any harm in your case because you never iterate over the entire map, and therefore never encounter that entry.

Using a text editor that doesn't uselessly add a BOM to files that didn't already have one would be the simplest solution.
Java apparently stripped any BOM from UTF-8 files at one point, but this change was reverted due to problems it caused in some programs (despite rather vociferous protests that the proper solution would have been to fix those programs).
Code is available for a Java file reader that auto-strips the BOM, but implementing that in mafia strikes me as a very low-priority project. After all, safely editing a map file already places various restrictions on the editors that are suitable for the job, adding "non-braindead" to the list doesn't limit your choices by that much.

 

[bordemstirs]

You are ridiculously knowledgeable about things. And thank you -so- much. I thought at one point it was due to using the file while mafia was writing to it, but I wasn't saving the file when I closed it in those cases, so I threw that out.

(despite rather vociferous protests that the proper solution would have been to fix those programs).

Lol. Just lol.

Now to convert my default text editor to my code editor. I'll miss you notepad!