Mafia no longer identifies itself to servers -- is this intentional?

[zarqon]

Since the user-agent change r6962, Mafia is no longer identifying itself to servers when using visit_url(). Previously in the headers it identified itself, and this was meant to aid the admins of servers that KoL hits (primarily KoL itself).

I have used this feature for several of my scripts that hit my own server, to make php files that are only accessible via mafia.

Now, mafia is identifying itself as whatever you have for your relay browser.

My question: is this intentional behavior? Is mafia identifying itself to servers no longer necessary?

I wasn't sure if this was a bug or not so I wrote it up here rather than submit a report. But I was rather flummoxed when all of my ASH scripts that fetch data from my server started dumping HTML pages into the CLI. (Because they thought it was Opera rather than mafia.)

 

[Bale]

Well, it is listed in the commit log for 6962 (as I'm sure you noticed) so I suspect it was intentional. The question is why it was intended. Pros and Cons. I think Veracity just figured it would help ID the browser that processes the code for the relay browser.

Perhaps this would be good for the relay browser, but bad for visit_url()

 

[Veracity]

Interesting. It was always possible, via an (unconfigurable) option. It never occurred to me that user websites (like yours) would want to behave differently for KoLmafia than for a regular browser - and it always seemed undesirable for KoL itself to behave differently. Especially so, now that CDM has his new ajax code in there, since, as I understand it, that chooses to do different things, depending on what kind of browser you have.

My reasoning was that if you connect via the Relay Browser, we should forward whatever browser you are to KoL and let it send things based on that bit of info. After all, KoLmafia is acting as a "proxy" (according to the definition in the HTTP spec) in that case.

The problem is, KoLmafia logs in by itself - using its own version number as the user agent - and then when you open the Relay Browser, it cannot switch user agent. Doing so makes KoL log you out. My solution, for now, was to remember what your browser says when you opened the Relay Browser and use that the NEXT time KoLmafia connected. That's imperfect, since you could have switched browsers between sessions (as Bale mentioned, in another thread).

My worries about the Ajax code could be unfounded; perhaps the default KoLmafia user agent will give you all the features that your browser of choice can handle. But perhaps not. So, perhaps I should save the last browser's agent - as now - but give you an option or something to use that upon connection. (For that matter, we should be able to change at logout, without having to restart the program, which might be how it is now.)

I'm open to comments and suggestions.

 

[zarqon]

Veracity, thanks for your swift response as always. I think your current solution is good -- but only for the relay browser. And, most people have a preferred browser that they primarily use so I don't see the one-use delay as being a big issue.

For everything else (especially scripts), it would be nice if mafia would continue identifying itself. If that's possible, I would find (indeed, have found) it very useful. My KoL-related PHP scripts provide very basic text for mafia visit_url() requests, and styled HTML for browser requests. I have also limited some scripts to be only accessible through mafia.

So I am going to agree with Bale that only requests actually generated by the relay browser should use the browser as the agent; all the rest should remain identified as mafia. It makes sense if you think about where the HTML requests come from. Relay browser: your browser generated the request and will display the outcome, mafia is just the filter. Auto-adventuring or visit_url(): mafia itself is handling that transaction with the possibility of AJAX interaction being small to none. I think it would be best if agents were specified accordingly.

Right now, the auto-bounty script (and every other script that uses the automatic data file updating feature) and every single one of my daily routine scripts is broken. I don't want to rework everything unnecessarily, so if mafia could use itself as the agent for visit_url() requests again that would save me a lot of work. Otherwise, I'd like to know fairly quickly that it's not going to happen so I can rewrite all my scripts to be non-agent-specific.

Thanks!

 

[Bale]

If I correctly understand what Veracity just said, KoL itself will log you out if you switch user agent, so... if it logs in as Opera in order to get any Opera specific code for the relay browser, then if you were to send a visit_url() with mafia as the user agent, KoL will swiftly boot your ass out of there.

Sadly that really sounds like you would have to rewrite your scripts as non-agent-specific.

 

[zarqon]

Ugh. I believe you are right. Double Ugh.

Never before has a single revision caused so much script rewriting.

I seriously think this broke every single one of my scripts. OCW, Checklist, RandBot, StashBot, the bounty script, all my routine scripts, Script Registry... wow.

 

[Camber]

I have been having a "i think this is related" problem lately; When i try to login, after i hit the login button, the results displays "Requests complete." and just sits there.

When i have it show the gCLI, this is all it says:
Validating login server (www3.kingdomofloathing.com)...
Sending login request...
Requests complete.

I thought something in the settings was messed up, and after comparing an older copy of GLOBAL_prefs, this is the only difference:
userAgent=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
vs
userAgent

When i edit the GLOBAL_prefs.txt file, and remove the value of useragent, then i can login just fine. But after i logout, the line is rewritten, and i can't login again until i remove it.

It seems to be related to what this thread is about, so i thought to mention it here. If i was wrong, sorry about the hijack...

Thanks for all everyone does...

 

[Bale]

Ugh. I believe you are right. Double Ugh.

Never before has a single revision caused so much script rewriting.

I seriously think this broke every single one of my scripts. OCW, Checklist, RandBot, StashBot, the bounty script, all my routine scripts, Script Registry... wow.

Maybe we can petition jick to have CDM change that so KoL won't kick you out just for changing user agents?

 

[Veracity]

Bale understood me correctly: if we switch user agents mid-connection - like after you open the Relay Browser and it submits a request, which tells you what browser you are actually using - KoL kicks you out.

I submitted code that saves your last user agent (in "lastUserAgent", surprisingly ) but doesn't actually use it. We now (again) always use "Kolmafia-12.9", or whatever the checked-in version number is. The "userAgent" setting is no longer used at all. Revision 6982.

Don't rewrite your scripts. Yet.

Edit: I would like to be able to use the browser's agent when KoLmafia is simply acting as a proxy - for requests submitted by the Relay Browser - and KoLmafia's own version number when you are using KoLmafia's GUI - for automation and such. That way, if Jick is really interested in how much people use KoLmafia, and what for, meaningful info would actually be available. As it is now, I spend essentially my whole session in the Relay Browser - but KoL thinks my manual clicks were generated by KoLmafia.

 

[zarqon]

Thank you thank you thank you thank you.

 

[Bale]

Edit: I would like to be able to use the browser's agent when KoLmafia is simply acting as a proxy - for requests submitted by the Relay Browser - and KoLmafia's own version number when you are using KoLmafia's GUI - for automation and such. That way, if Jick is really interested in how much people use KoLmafia, and what for, meaningful info would actually be available. As it is now, I spend essentially my whole session in the Relay Browser - but KoL thinks my manual clicks were generated by KoLmafia.

Someone ought to bring that up to the devs to get it to Jick's attention 'cause it would be better.