HTTP 1.1 0 byte header

fronobulax

Developer
Staff member
I am seeing this a lot accessing Coldfront. Are there things I can do, solely on my end to reduce the frequency?

Most of the time I am iterating over items and making a query. It is certainly possible that my requests are being sent more quickly than Coldfront is equipped to respond. Would introducing "wait and retry" in the script be of benefit? Wait between queries?

There was some conversation about KoLmafia reusing connections. Should I investigate and see whether that is a factor that could be addressed in KoLmafia.

Ideas?

Thanks.
 

Ryo_Sangnoir

Developer
Staff member
Could you post a script that show this problem?

My naive idea without seeing anything is to add a "prefer HTTP 1.1" preference that does exactly that, and then your script can set it before it makes a bunch of calls to Coldfront (which is non-http2) and reset it after.
 

fronobulax

Developer
Staff member
Thanks.

The script in question is DCQuest and is published. If necessary I can pare it down to something that focuses on the error.

I was accessing

string domain = "http://dcdb.coldfront.net/collections/index.cgi?query_type=item&";

When I changed it to

string domain = "https://dcdb.coldfront.net/collections/index.cgi?query_type=item&";

I got

IOException retrieving server reply (https://dcdb.coldfront.net/collecti...st_query_type=player&last_query_value=2115655) -- PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

and visiting the url manually confirms its insecurity.

I suspect the best fix is to have dcdb.coldfront.net join the rest of the world in supporting https:

Failing that it does seem as if exploring some kind of fallback is appropriate. My reaction is that KoLmafia should detect when a fallback is appropriate and try it rather than a preference but if the probable solution is in mafia, I can poke around some.

Thanks.
 

Ryo_Sangnoir

Developer
Staff member
Does using the HTTP url not work? You can't use the HTTPS url because it serves a different page (and has a self-signed cert, but being a different website is more vital).

I think Mafia is doing okay here: you're telling it to go to a URL which doesn't serve a valid page, so it returns an error as expected.
 

fronobulax

Developer
Staff member
I have a script level workaround. If the response is 0 bytes then I submit it again and only treat it as an error if it happens twice.

I looked at the code and HttpClient and didn't really find a place where I thought it was safe to set or toggle the protocol version. The default is HTTP/2 and the docs say

If set to HTTP/2, then each request will attempt to upgrade to HTTP/2. If the upgrade succeeds, then the response to this request will use HTTP/2 and all subsequent requests and responses to the same origin server will use HTTP/2. If the upgrade fails, then the response will be handled using HTTP/1.1

and that makes me wonder about the benefit of forcing HTTP/1. If 2 fails then it already falls back to 1 and I'd like to believe the results are the same whether it starts at 1 or falls back to 1. But since this feels like it is time related to me maybe there is a server hit to determine failure of 2 that slows down the response to 1?

In any event I have a workaround and it is not totally clear how I would address this in mafia so I'm unlikely to try :)
 
Top