Not a bug - not a script problem - sourceforge issues

clubmasterflex

New member
Not really sure where to post this. I don't think this is an issue with kolmafia so I'm not posting in the bug reports forum. Is anyone else having trouble running svn update in kolmafia? I'm getting errors like this:

svn: E175002: OPTIONS request failed on '/p/guyymafia/code/spacetripper'
rlbond86-mafia-scripts-bounty_hunter_helper-trunk not checked - exception: org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the server
svn: E175002: OPTIONS request failed on '/p/rlbond86-mafia-scripts/code/bounty_hunter_helper/trunk'
guyymafia-tricktreat not checked - exception: org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the server
svn: E175002: OPTIONS request failed on '/p/guyymafia/code/tricktreat'
relaywtf-skillswtf not checked - exception: org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the server
svn: E175002: OPTIONS request failed on '/p/relaywtf/code/skillswtf'
mafiachit not checked - exception: org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the server
svn: E175002: OPTIONS request failed on '/p/mafiachit/code'
veracity0-vprops not checked - exception: org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the server
svn: E175002: OPTIONS request failed on '/p/veracity0/code/vprops'
slyz-nemesis not checked - exception: org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the server
svn: E175002: OPTIONS request failed on '/p/slyz-nemesis/code'
 

clubmasterflex

New member
I'm wondering if I'm getting put in a timeout/rate controlled by sourceforge. Before I run svn update, I'm able to hit sourceforge.net. After running svn update, I can't hit the web page until I wait for at least 30-60 minutes.
 

fronobulax

Developer
Staff member
Mafia just failed with similar messages.

Using TortoiseSVN I could not update a local copy of a script I am author of, nor could I browse the repository. The update failed with similar messages to what KoLmafia was reporting.

Attempting to access the repo via a web browser failed with a time out.

Attempting to log in to SF timed out.

Attempting to update mafia and build failed with cannot connect to server.

I'm blaming SourceForge although they don't seem to be telling users there is a problem.
 

fronobulax

Developer
Staff member
Is there any way to tune how many connections svn update makes at once/slow it down?

Not that I know of, without changing the code.

I'm not going to spend any time looking at that since every test in my post above failed this morning except "update mafia source code" which remains pretty strong evidence that it is a SF problem and not a mafia, or my system, timing problem.
 

xKiv

Active member
If I repeat svn update a few times, the first 14 scripts check version without timeout, and the remaining 38 time out. ... Actually, it looks like something gets stuck 9 scripts too early ... oh, they all timed out an hour later too. And presumably I won't be able to connect to sf.net for any purpose for another hour.
 

fronobulax

Developer
Staff member
If I repeat svn update a few times, the first 14 scripts check version without timeout, and the remaining 38 time out. ... Actually, it looks like something gets stuck 9 scripts too early ... oh, they all timed out an hour later too. And presumably I won't be able to connect to sf.net for any purpose for another hour.

How many of the ones that worked are actually hosted at SF and not GitHub? The only update checks that worked for me were ones that were hosted at GitHub.

That said, some of my TorotiseSVN checks just worked so...
 

lostcalpolydude

Developer
Staff member
A while back, svn updates were changed to run in parallel instead of 1 at a time. Does this (apparent) change on sourceforge's end mean that needs to be undone, at least for any sourceforge updates?

I don't have the setting to update all scripts every day enabled, so this won't affect me regardless...
 

fronobulax

Developer
Staff member
A while back, svn updates were changed to run in parallel instead of 1 at a time. Does this (apparent) change on sourceforge's end mean that needs to be undone, at least for any sourceforge updates?

I don't have the setting to update all scripts every day enabled, so this won't affect me regardless...

I do. Almost every time KoLmafia has had update problems for me, TortoiseSVN has had problems and both reported the same errors from SF. However, if all the reports are to be believed, a case could be made that SF has changed in a way that throttles certain requests from the same IP, in which case there are things KoLmafia should do.

My recollection behind the "go parallel" requests was that when an update failed because of a time out it took the entire sequence a very long period of time to finish. I am thinking a robust solution might stagger the requests in time and pay attention to the responses, possibly retrying.
 

xKiv

Active member
My recollection behind the "go parallel" requests was that when an update failed because of a time out it took the entire sequence a very long period of time to finish

It took a very long time whether it failed or succeeded, unless you only had a couple scripts. Because every script took several seconds.

..

So, for a test I added an increasing sleep at the start of CheckStatusRunnable.run (the N-th started thread waits N*100ms), and it still wasn't enough. I now got through 27 scripts before I started getting timeouts:
Code:
Checking all SVN projects...
Ezandora-Asdon-Martin-GUI-branches-Release is at HEAD (r10)
mafiachit is at HEAD (r579)
ccascend-cc_ascend is at HEAD (r1360)
Ezandora-Helix-Fossil-branches-Release is at HEAD (r21)
winterbay-mafia-wham is at HEAD (r47)
stewbeef-clanfortune-branches-release is at HEAD (r32)
balefull-raidlog-parser-branches-master is at HEAD (r3)
mafiarecovery is at HEAD (r36)
guyymafia-tricktreat is at HEAD (r274)
ccascend-snapshot is at HEAD (r1360)
bale-new-life is at HEAD (r70)
relaywtf-skillswtf is at HEAD (r42)
Ezandora-Choice-Override-branches-Release is at HEAD (r23)
batman-re is at HEAD (r52)
bale-counterchecker is at HEAD (r23)
Ezandora-Detective-Solver-branches-Release is at HEAD (r19)
clilinks is at HEAD (r6)
kolmafiascripts-cfstat is at HEAD (r2)
Ezandora-Genie-branches-Release is at HEAD (r44)
therazekolmafia-canadv is at HEAD (r96)
AllenTuring-KoL-MineVolcano-trunk is at HEAD (r61)
mikebryant-kolmafia-lar-forecasting-branches-master-src is at HEAD (r54)
Malurth-Auto-2-day-HCCS-branches-Release is at HEAD (r163)
Ezandora-Sweet-Synthesis-branches-Release is at HEAD (r20)
Ezandora-Briefcase-branches-Release is at HEAD (r80)
Ezandora-Far-Future-branches-Release is at HEAD (r39)
Ezandora-Guide-branches-Release is at HEAD (r516)
bale-snojo not checked - exception: org.tmatesoft.svn.core.SVNException: svn: E175002: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
svn: E175002: OPTIONS request failed on '/p/bale/snojo/code'

I think everything by Ezandora is on github?
 
I haven't been able to get to SF at all from home for the last week. I must be on their permanent anti-DDOS list or something. Works fine from work.

Odd that they would start throttling or something without notifying the user base so that they could adjust. Also odd to begin this behavior just when they are getting a lot of migration traffic from GitHub. Some sort of causality is probably involved, but I'm not sure what can be inferred if so.

In any case, it's good to see this being discussed, thought I was the only one for a while. After verifying that DNS looked ok I started thinking maybe it was a local router / firewall problem.
 

fronobulax

Developer
Staff member
I ran mafia this morning and it failed.

I saw the post above and thought I would try accessing the main page (SourceForge.net) via a web browser. I got a time out.

I then turned on a VPN for traffic coming from the browser and connected within seconds.

It sure smells like something SF is doing that is IP based. I wish someone at SF would tell people what it is.
 
I am having the same issue this morning needing an update to Veracity's script and getting the same errors trying to access. I did upgrade to the latest Java 10 would that be causing this?
 

xKiv

Active member
I'm pretty sure we're somehow triggering their DDOS protection?

But single-IP-based limits don't block *distributed* attacks ... at all ...

Also, a IP-based throttle *this* brutal can quickly block legitimate traffic from large groups behind NATs. Just imagine team of 30 people syncing the same project at the same time when they start working ...
 
Last edited:
Top