r28087 - What's Changed [*]Create JSON API for relay scripts. by ]@phulin] in
- Thread starter Github Bot
- Start date
@fronobulax raised some points in the pull request that I feel warrant more discussion / visibility.
@ikzann @Ryo_Sangnoir
The obvious place where faster JSON might be relevant is in parsing api.php, but it's also not clear to me that JSON handling is a bottleneck at this time. It's also not clear why we'd use this over, say, simdjson-java which benchmarks favorably against fastjson2. (Okay, one reason is that simdjson-java requires Java 18. Although... we've been talking about upgrading Mafia to require Java 21, which is the latest LTS.)
@ikzann @Ryo_Sangnoir
phulin said:
fronobulax said:
The obvious place where faster JSON might be relevant is in parsing api.php, but it's also not clear to me that JSON handling is a bottleneck at this time. It's also not clear why we'd use this over, say, simdjson-java which benchmarks favorably against fastjson2. (Okay, one reason is that simdjson-java requires Java 18. Although... we've been talking about upgrading Mafia to require Java 21, which is the latest LTS.)
fastjson2 has me worried because it's hundreds of thousands of lines of Java across thousands of files over thousands of commits, and it has a bunch of features we don't need (read: broad attack surface). Based on the "projects using us" list, I also don't see much adoption by other large organizations (there's a handful of Apache projects, mostly opensourced by AliBaba itself, although I saw one or two that came from XiaoMi).
If there really is a compelling use case, then I'm open to discussing it. But I feel like this just piggybacked on another change, and I'd rather we didn't add new external dependencies lightly.
If there really is a compelling use case, then I'm open to discussing it. But I feel like this just piggybacked on another change, and I'd rather we didn't add new external dependencies lightly.
Thank you. I never seem to know where to start discussions and am often not aware of socialization type discussions that occur "elsewhere".
I chose not to mention it on GitHub but when I was developing and deploying code for selected US Government agencies I would not have been allowed to use fastjson2. Basically a known vulnerability in an earlier version combined with unverified allegations that a somewhat antagonistic, non-US entity was heavily involved in the development would have resulted in a decision that is was safer to forbid the use of fastjson2 than to spend the effort to accredit it and "prove" it was safe. I understand that there are no national security implications of KoLmafia but the previous vulnerability did allow a malicious user to run an arbitrary operating system command. So I expect we KoLmafia devs would have taken steps to address that vulnerability and would do so in the future if fastjson2 introduces something similar.
Among the many rules of thumb that guided my development decisions was the one that said performance optimization for its own sake needs to be supported by benchmarks. I still would like to understand who is going to see better performance and under what circumstances. I also acknowledge that just because "I have a bad feeling about this" does not mean my "feeling" will come true.
I chose not to mention it on GitHub but when I was developing and deploying code for selected US Government agencies I would not have been allowed to use fastjson2. Basically a known vulnerability in an earlier version combined with unverified allegations that a somewhat antagonistic, non-US entity was heavily involved in the development would have resulted in a decision that is was safer to forbid the use of fastjson2 than to spend the effort to accredit it and "prove" it was safe. I understand that there are no national security implications of KoLmafia but the previous vulnerability did allow a malicious user to run an arbitrary operating system command. So I expect we KoLmafia devs would have taken steps to address that vulnerability and would do so in the future if fastjson2 introduces something similar.
Among the many rules of thumb that guided my development decisions was the one that said performance optimization for its own sake needs to be supported by benchmarks. I still would like to understand who is going to see better performance and under what circumstances. I also acknowledge that just because "I have a bad feeling about this" does not mean my "feeling" will come true.
I was not going to mention the China angle, but since you mention it:
I am not saying by any means that the CCP has backdoored this project (or has intentions to do so), but they certainly would have the means if they were so inclined.
I am also not saying that the only security vulnerabilities are from state-sponsored actors, or that our threat model should take them into consideration. But taking up unnecessary dependencies on feature-heavy projects is how we get problems like Log4Shell (which, let's be clear, was disclosed by a member of Alibaba's security org!).
I don't think there's any verification needed to state that Alibaba is a Chinese-owned company headquartered in China, and the vast majority of their employees are also based in China.
I am not saying by any means that the CCP has backdoored this project (or has intentions to do so), but they certainly would have the means if they were so inclined.
I am also not saying that the only security vulnerabilities are from state-sponsored actors, or that our threat model should take them into consideration. But taking up unnecessary dependencies on feature-heavy projects is how we get problems like Log4Shell (which, let's be clear, was disclosed by a member of Alibaba's security org!).
Still waiting for a response.
I see that textui/Datatypes.java has been modified.
Can you provide some documentation about how scripters will be able to use this?
Core changes to KolMafia need to be announced - and documented - here.
Whether or not they were discussed on another forum is irrelevant.
This is the “official” forum for announcing KoLmafia changes.
Make an announcement, please.
Thanks!
I see that textui/Datatypes.java has been modified.
Can you provide some documentation about how scripters will be able to use this?
Core changes to KolMafia need to be announced - and documented - here.
Whether or not they were discussed on another forum is irrelevant.
This is the “official” forum for announcing KoLmafia changes.
Make an announcement, please.
Thanks!