JSON

[fronobulax]

@gausie has implemented an alternative to the Display Case Database, hosted at Coldfront. https://museum.loathers.net/

I like to know where my characters are on various Display Case leaderboards and I currently do this by one query per item to Coldfront for each character. The Museum allows me to make one query per character and then process the results and end up with the same information.

I can webscrape the Museum but the results are also available in JSON so I could ask for JSON and process that instead.

I can't find much ash support getting from JSON to a map. Maybe it is there and is so obvious that I am just clueless. But if it is there I would like some pointers, please.

I'm told JSON is not a problem for JavaScript so I could learn enough JS to fetch the JSON and write a map and then I can do then other things in ash by reading the file. I learn best by example so is there a JS snippet that deals with JSON and maps I could look at?

Thanks.

 

[MCroft]

I started writing something up and googled for a piece I needed and found this fine guide:

How to Convert JSON to/From a Map in JavaScript

Learn how to quickly convert a JSON string to a Map in JavaScript.

codingbeautydev.com

here's the meat of it:

const json =  '{"user1":"John","user2":"Kate","user3":"Peter"}';
const map = new Map(Object.entries(JSON.parse(json)));
// Map(3) { 'user1' => 'John', 'user2' => 'Kate', 'user3' => 'Peter' }
console.log(map);

 

[MCroft]

It would be possible to write a JSON_to_map function for ASH, but it's not a perfect fit.

1: JSON doesn't require each object to have the same format, so parsing needs to be careful.
2: JSON doesn't enforce any structure on the keys, and map will overwrite a key-value pair if the key is a match.

So there would have to be some guardrails, but it might be simple enough for your use case.

 

[heeheehee]

map will overwrite a key-value pair if the key is a match.

To be fair, JSON.parse will clobber any duplicate keys before you get to that point.

You shouldn't need to go through a Map -- JS Objects operate similarly to Maps (that's what we did 10 years ago, before ES2015 brought Map to the language).

As MCroft mentions, I'd suggest doing some preprocessing of the underlying data first to make sure your outputs are uniformly structured, since otherwise you'll probably run into some sharp edges of JS <> Java <> ASH conversions.. Something like...

let kolmafia = require("kolmafia");

let results = {};

for (let playerId of [1, 354981]) {
  let url = `https://museum.loathers.net/player/${playerId}?_data=routes%2Fplayer.%24id`;
  let obj = JSON.parse(kolmafia.visitUrl(url, false, true));
  let collection = obj.player.collection.map(x => { return {item: x.item.id, quantity: x.quantity, rank: x.rank}; });
  results[playerId] = collection;
}

kolmafia.mapToFile(results, "player-collection-ranks.txt");

 

[fronobulax]

Thank you very much. I run and get an error about an empty JSON string.

I tried via ash using

string url = "https://museum.loathers.net/player/215957?_data=routes/player.$id";
buffer out = visit_url(url, false, true);

and the return is of length zero.

The URL works in a browser.

Brute force being a tool of choice I tried all four combinations of the booleans and visit_url returns length zero in all four cases.

My next debugging step is a breakpoint in an IDE but if anyone has some other suggestions I'll try them too.

 

[heeheehee]

The above only works for me if I'm logged in, for whatever* reason. Otherwise it works just fine.

* The actual reason is at the start of GenericRequest.run().

  public void run() {
    if (GenericRequest.sessionId == null

 

[fronobulax]

That was it. Thank you.

Tangentially GenericRequest.sessionId == null and visit_url being a generic request means some scripts will need to be logged in to run. Not sure whether that is a bug or a feature but...

 

[fronobulax]

I've got something working so thank you. Not ready for prime time yet but it will eventually replace a portion of DCQuest. Thank you.

 

[fronobulax]

See https://kolmafia.us/threads/dcquest.3640/#post-25745