503 errors on Hackage

Seeing 503 errors when opening pages from Hoogle or looking up packages on hackage:

Error 503 error:0A000126:SSL routines::unexpected eof while reading

3 Likes

Yeah, it happens to me pretty much every midnight UTC. Goes away on its own in 20 minutes or so.

1 Like

Hmm, although not quite. I’ve seen 503 errors from Varnish cache server before, but not this one.

Seems to be good now :person_shrugging:

We started to get these small outages when we switched hackage boxes and haven’t been able to trace down the cause – it appears to be an error with too many file handles opened periodically, but haven’t sourced down where that comes from – although I suspect, if anyone wants to roll up their sleeves, that it could have to do with inefficient storage/calculation of download counts.

1 Like

Could it be Out of file descriptors! · Issue #100 · haskell/tar · GitHub / cabal haddock failure: out of FDs due to tar · Issue #11131 · haskell/cabal · GitHub? ?

hackage-server still uses tar-0.6: hackage-server/hackage-server.cabal at 350fd8aac61c76481fcccfeee499ab6675171320 · haskell/hackage-server · GitHub

2 Likes

that’s a great suggestion and could well be the culprit. thanks for raising that!

1 Like

@sclv even after the recent upgrade to tar-0.7, Hackage still gets extremely slow right after midnight UTC, for 10-15 minutes or so. Is there a background backup process or anything similar?..

1 Like

Ah drat. Yes, I think that’s when our mirroring runs. Maybe we should modify our mirroring to throttle it, as another possible improvement.

2 Likes