Best way to load questslog.txt

zarqon

Well-known member
I would like to access all the quest steps in questslog.txt for a script I'm working on. I tried:

Code:
      string[string] qs;
      file_to_map("questslog.txt",qs);

Thinking that would give me the property name as the index, and all the other fields (complete with tabs) as the remaining string. I seem to recall a time when that used to work. Then I could just do a split_string(qs[<property>],"\t") to get all my quest steps in a row, no matter how many there are.

Unfortunately, this only gives me the first string (in this case, the quest title) -- nothing after the first tab.

There are some quests with upwards of 30 steps. What sort of map can I use to read the questslog.txt file without losing steps?
 
Last edited:

ckb

Minion
Staff member
There are some quests with upwards of 30 steps. What sort of map can I use to read the questslog.txt file without losing steps?

Based on my experience with importing mafia datafiles with file_to_map(), I believe you would need

Code:
string[string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string] qs;
file_to_map("questslog.txt",qs);

Ugly, but functional. Then you can parse 'qs' into other smaller maps for each quest.
 

zarqon

Well-known member
Thanks ckb. Was afraid I'd have to resort to something like that. The other option that comes to mind (which retains a single map key) is using a record with dozens of string fields, but that's even harder to use because the fields can't be foreached.
 

zarqon

Well-known member
I wish to list all the steps for an informational script. The data file is rather difficult to parse as a human, so I'd like to present it as a table with "unstarted", "step2", etc. on the left and the associated text on the right.
 

ckb

Minion
Staff member
I wish to list all the steps for an informational script.

This makes me think of loading it as part of a relay script, where you parse and format the text with regex rather than use file_to_map(). You would probably need something (javascript?) to load questslog.txt, but if anyone can do that, its probably zarqon.
 

zarqon

Well-known member
It is for a relay script. Unfortunately since questslog.txt is actually in the mafia .jar rather than /data, I lack the kung fu to access it that way. I also don't want to rely on hitting Sourceforge to access the source file there, given its relative sluggishness and propensity for taking vacation.

It's looking like string[string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string] is the only way to go. *sigh* ... *plays Factorio*
 

Darzil

Developer
Also bear in mind some of those entries aren't full quest text, as smaller sections were used to match different classes text where that differs.
 

zarqon

Well-known member
Ended up going with the record:

Code:
record queststeps {      // need this for reading questslog.txt
   string title;
   string started;
   string step1;   string step2;   string step3;   string step4;   string step5;
   string step6;   string step7;   string step8;   string step9;   string step10;
   string step11;  string step12;  string step13;  string step14;  string step15;
   string step16;  string step17;  string step18;  string step19;  string step20;
   string step21;  string step22;  string step23;  string step24;  string step25;
   string step26;  string step27;  string step28;  string step29;  string step30;
   string step31;  string step32;  string step33;  string step34;  string step35;    // hopefully there will never be more than 35 steps (presently Nemesis has 30)
};

Due to difficulties in parsing a many-dimensional map into 2D. Wrote a nifty recursive function to do it, but ASH got flummoxed by my attempt to foreach an "aggregate".

Now that I can see the questlog info, I note evidence of some things being hardcoded behind the scenes, i.e. places where the steps probably don't all seem to map directly to the usual quest step progression.

For example, I assume that the last THREE steps of questL12War, rather than mapping to step2, step3, and finished, respectively, instead all map to finished. Are there any other oddities that anyone knows about?
 

Darzil

Developer
Looks to me like the War may be the main place that happens (though some steps cannot be inferred from quest log, which is another issue), doesn't actually look like the information is used, though, just additionally checks the text and sets to FINISHED, rather than looking at the identified quest step and setting to FINISHED.

The bits of mafia to look at though are QuestDatabase, QuestLogRequest and QuestManager.
 

Veracity

Developer
Staff member
Due to difficulties in parsing a many-dimensional map into 2D. Wrote a nifty recursive function to do it, but ASH got flummoxed by my attempt to foreach an "aggregate".
By "got flummoxed" I assume you mean "did not have syntax to represent what I wanted to do."

Show me your function and lets talk about it. We had a discussion years about about letting functions deal with generic aggregates, much as map_to_file and file_to_map do. Perhaps this is more of the same. Perhaps it is time to revisit it and decide if there is anything we can do to the syntax that makes sense that would make your life easier.

Perhaps that should go into its own "scripting discussion" thread.
 

heeheehee

Developer
Staff member
I'm wondering if this problem could be solved by adding a pair of functions that operate similarly to map_to_file/file_to_map.

Namely, two possibilities readily come to mind:

1. Treat the entire file as tab-separated values with no additional structure -- works specifically with type[int, int].

2. Treat the file as a map, but in each line, interpret fields from 2 onward as an array -- so, type_v[type_k, int].
 

zarqon

Well-known member
@Darzil: Thanks!

By "got flummoxed" I assume you mean "did not have syntax to represent what I wanted to do."

Quite so. I hadn't expected it to work, honestly, with ASH being typed and all, but since I also recalled that discussion long ago which resulted in allowing passing a generic "aggregate" to a function, I figured it was worth a shot. I've since deleted it, but I can try to reconstruct it here. It was something like

Code:
void onerow(aggregate a, int depth) {
   if (depth > 100) return;
   foreach key, value in a {       // this is where the error was thrown
      add_table_row(depth, key);   // adds another <tr> to an HTML table being built in a buffer
      if (some test of value to determine it is an aggregate rather than a string) 
         onerow(value, depth + 1);
   }
}
 

Veracity

Developer
Staff member
2. Treat the file as a map, but in each line, interpret fields from 2 onward as an array -- so, type_v[type_k, int].
I'm thinking we almost have something like this already, given that I added arrays of unspecified length.

Code:
typedef string[] type_v;
typedef string type_k;
type_v [type_k] quests;
file_to_map( "questlogs.txt", quests );
That almost certainly doesn't work, currently.

It's also probably not quite right, because file_to_map insists on using an existing map, rather than crafting and returning a new one. It does that because the passed in map defines the type of map.

Huh. Actually, it might be right; the "value" type is a length-0 array, which is defined to be "an array which will be initialized to the actual number of elements given to it, but which is immutable afterwords."

To make that work CompositeValue.read() would have to recognize that if the datatype is a zero-length array, it should simply consume all the remaining values on the line and initialize an array of the correct length as the value. Perhaps by defining ArrayValue.read() to understand this case.
 

heeheehee

Developer
Staff member
To make that work CompositeValue.read() would have to recognize that if the datatype is a zero-length array, it should simply consume all the remaining values on the line and initialize an array of the correct length as the value. Perhaps by defining ArrayValue.read() to understand this case.

This would be interesting, but I'm somewhat concerned that this might be backwards-incompatible. That said, I feel like anyone who's using new features like 0-length arrays should expect some degree of churn (and keep an eye out for this sort of thing).
 

Veracity

Developer
Staff member
With a bunch of changes to KoLmafia, this script:

Code:
typedef string[] type_v;
typedef string type_k;
type_v [type_k] quests;
file_to_map( "questslog.txt", quests );

print( "There are " + count( quests ) + " quests." );

int longest_quest = 0;
string key = "";
foreach quest, steps in quests {
    if ( count( steps ) > longest_quest ) {
	longest_quest = count( steps );
	key = quest;
    }
}

print( "Quest '" + key + "' has " + longest_quest + " steps." );

type_v steps = quests[ key ];
print( steps );

foreach i, step in steps {
    print( i + ": " + step );
}
yields this:

Code:
[color=green]> Test/questlog.ash[/color]

There are 96 quests.
Quest 'questG04Nemesis' has 32 steps.
aggregate string [32]
0: Me and My Nemesis
1: Search the Misspelled Cemetary on the Nearby Plains for the Tomb of the Unknown
2: in the Misspelled Cemetary wants to fight you to test your worthiness.
3: Talk to the Ghost of the Unknown
4: Retrieve the Epic Weapon!
5: You've reclaimed the Epic Weapon! Time to go show it off at your guild.
6: recover the missing piece of the Legendary Epic Weapon.
7: Inform your guild that Beelzebozo has been dispatched.
8: Meatsmith the two parts of the Legendary Epic Weapon back together -- you'll need a tenderizing hammer from the Meatsmith.
9: Take the Legendary Epic Weapon back to your guild.
10: (no quest entry at all!)
11: The hunt for your Nemesis is on! Better check out that cave they sent you to.
12: Figure out how to get into your Nemesis' cave. If you're stumped, maybe your guild can help?
13: The cavern is full of weird mushrooms, but where's your Nemesis?
14: more fizzing spore pods to blow up the blockade in your Nemesis' cave.
15: Take those fizzing spore pods to the rubble!
16: Boom! Time to bring the fight to your stinking Nemesis in that stinking cave!
17: Heck yeah, you beat your Nemesis and got a sweet hat. Better take it back to your guild.
18: You're waiting for your guild's scouts to find out where your Nemesis went.
19: It appears as though some nefarious ne'er-do-well has put a contract on your head!<p>Gee, I wonder who it could be...
20: You handily dispatched some thugs who were trying to collect on your bounty, but something tells you they won't be the last ones to try.
21: Whoever put this hit out on you (like you haven't guessed already) has sent Mob Penguins to do their dirty work. Do you know any polar bears you could hire as bodyguards? No? Looks like you're on your own, then.
22: So much for those mob penguins that were after your head! If whoever put this hit out on you wants you killed (which, presumably, they do) they'll have to find some much more competent thugs.
23: Your suspicious[sic] have been confirmed: your Nemesis has put the order out for you to be hunted down and killed, and now they're sending their own guys instead of contracting out. Good luck!
24: Bam! So much for your Nemesis' assassins! If that's the best they've got, you have nothing at all to worry about.<p>You sure hope that's the best they've got.
25: You had a run-in with some crazy mercenary or assassin or... thing that your Nemesis sent to do you in once and for all. A run-in followed by a run-out, evidently, but you're going to have to deal with this sooner or later.
26: Now that you've dealt with your Nemesis' assassins and found a map to the secret tropical island volcano lair, it's time to take the fight to your foe. Booyah.
27: You've arrived at the secret tropical island volcano lair, and it's time to finally put a stop to this Nemesis nonsense once and for all. As soon as you can find where they're hiding. Maybe you can find someone to ask.
28: got away. Again.
29: Congratulations on solving the lava maze, which is probably the biggest pain-in-the-ass puzzle in the entire game! Hooray! (Unless you cheated, in which case: Boo!)
30: have some sort of crazy powerful and hideous final form? I was, but then I wrote all of this, so, y'know.
31: Okay, now you've defeated your Nemesis once and for all! (Well, probably. Almost definitely.) Nevermore will the members of whichever class you're playing at the moment be hassled by that terrible entity! Way to go!
Issues I found with aggregate literals:

1) They did not work in foreach since they didn't have iterators
2) Given iterators, you couldn't fetch values, since they didn't have aref
[3) They didn't have aset either. Since they are immutable, they should, but should generate a runtime error]
4) count() always returned 0

I fixed all of the above and also modified file_to_map as proposed: if you read into a 0-length array, you get all the remaining fields on the line read as the appropriate data type.

This is backwards-incompatible since if you tried it before, we threw an exception, caught it, and reported a "bad line" in your input file. Which is to say, it never did anything reasonable before and now it does.

As coded, it only works if your type_v is a simple type - one field per value. If you want the remaining fields to be <n> records, each with three fields, thus consuming fields three at a time, no go. I'd have to see an actual use case for this before I'd be inclined to make that work.
 

Veracity

Developer
Staff member
Hm. I did it via an ArrayLiteral since the code already existed for reading a variable number of elements into an array[]. I wonder if it might be better to duplicate that code and get an ArrayValue, not an ArrayLiteral?

I will experiment.
 

Veracity

Developer
Staff member
Yes, that works nicely.

Further experimentation showed that we couldn't use map_file to write out the map in the same format, however. That is because arrays do not have a "compact" representation in data files, unlike records.

I have an implementation of "compact" file format for arrays which works nicely, but which is non-backwards compatible without a small code change in scripts: if you have a map whose values are arrays - not maps! - and did map_to_file, it wrote it out in non-compact form - even though the default for map_to_file is compact. The solution is to change your map_to_file and file_to_map calls to specify "false" for the "is this compact" format.

I'll create a new thread to discuss this.
 

Bale

Minion
Just thought I'd let you know that I'm now making use of this feature in OCD. It helps when reading concoctions.txt since there are a variable number of ingredients for a given concoction. The old implementation worked, just as long as no new super long concoctions exceeded my dataspace. I worried that one day it would fail when KOL implemented something new.

zarqon isn't the only one making use of your work.

Thanks!
 
Top