Good service is only noticed when its gone. So I am grateful for all the years where the haskell.org infrastructure worked like a charm.
Recently, the experience is mixed. Maybe these are still glitches after the migration from FP complete. Or do we have to accept this as the new status quo?
I’ve tried your commands a bit but I also use hoogle a lot as a quick entrypoint into a package/module documentation, in which case I want something with haddock, so more commonly I run a
Hi. The main issue here is that the web is plagued by a sheer absurd amount of AI crawlers that are unbelievably disrespectful, hitting expensive URLs on the server that runs the hoogle.haskell.org.
There are certainly measures that would improve this but they’re usually quite drastic. We could e.g. do tarpitting, do aggressive rate limiting etc. I think the general consensus across the haskell org infrastructure team has been that these more aggressive strategies are not acceptable as they would impair fair use in the same way as the abusers.
However, given that the user experience using haskell services has been drastically degraded since the onset of the AI crawler epidemic, I think it might be worth to reevaluate that policy. I will make sure to ask about this topic when the infrastructure team meets next.
In the meantime, I also host a hoogle under hoogle dot mangoiv dot com which employs some of the more aggressive strategies to stay fast. You are free to use it.
@andreasabel sidestepping the main question, some time ago I stopped using hoogle.haskell.org in favor of the command-line interface. Is there anything lacking from the command-line interface of Hoogle in your experience?