Upcoming maintenance on Stackage services, 2 Jan 2024

Announcements
#1

Which services: stackage.org and casa.stackage.org
When: Tuesday, 2 January 2024 08:00 UTC 11:00 UTC
Expected impact: None

What: The services will be served by new processes running on Haskell Foundation infrastructure.
How: DNS proxies will be updated to point to the new servers, which should be an instant change.

The new services have already been running for a couple weeks, although they haven’t been getting any usage. If you see any trouble, contact me on this thread or at one of the locations listed below. If anything goes terribly wrong, I can just roll back to using the original FP Complete servers.

Where to contact in case of trouble (pick the first one that’s easiest for you):

  1. This thread
  2. Haskell room on Matrix.org
  3. Haskell Foundation Slack
5 Likes
#2

I guess this isn’t related, since it’s not 2nd Jan yet, but all of Stackage Hoogle seems to be unavailable. Is this expected?

The given Hoogle database is not available.

For example

#3

Nope, that’s not expected. I’m looking into it now.

1 Like
#4

The same problem affects the new service (which is expected, since they should be using the same data source). So I’ll postpone the migration and fix this first.

2 Likes
#5

@sibi and I managed to track down and resolve the issue. Looks like it’s working now!

  1. The messages are a bit misleading: Stackage only uses the hoogle database from the latest snapshot.
  2. Stackage fetches hoogle databases from a certain data source (an S3 bucket).
  3. A process called stackage-server-cron generates the databases for each snapshot and loads them into that data source.
  4. The stackage-server-cron process had been hitting some resource limits and was not successfully generating the databases.
  5. Modifying the resource limits allowed stackage-server-cron to run successfully.
3 Likes
#6

Now that the Hoogle issue is resolved, I will continue with the migration at 11:00 UTC, right after I get some lunch.

2 Likes
#7

The new services are now serving casa.stackage.org (used internally by Stack, in case you were wondering) as well as stackage.org. Let me know if you see any issues!

3 Likes
#8

Feels snappy so far :+1: Thanks for the effort in all of the migrating :slight_smile:

Few things:

  • it doesn’t seem like you can view any of the haddock documentation since nightly-2023-12-27 (2023-12-26 still seems to have haddock documentation for all packages)
  • When clicking on password-3.0.2.1 in any LTS, a bunch of the modules aren’t listed in the module tree:
Data
  |
  `- Data.Password
       |
       `- (Data.Password.Argon2 missing)
       `- (Data.Password.BCrypt missing)
       `- (Data.Password.PBKDF2 missing)
       `- (Data.Password.Scrypt missing)
       `- Data.Password.Validate
#9

Thanks for the report!

Indeed. This is surprising. I’ll see what I can figure out.

Not sure about this one. I do see that all those modules are guarded behind flags. The flags default to True, but maybe something is interfering.

#10

Aha, I figured it out. One background service that I thought wasn’t needed yet is, in fact, needed. It’s running now and should finish populating the Haddocks soon.

2 Likes
#11

I peeked at the stackage-curation logs, and I see some messages that look related:

Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Argon2.hi
         does not exist:

Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Argon2.hi
         does not exist:

Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Argon2.hi
         does not exist:

Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Validate.hi
         does not exist:

So this looks like a stackage curation issue. @juhp, perhaps you have some insight?

Edit: those messages may be red herrings, as I see quite a few of them for many different packages, including many modules that do have Haddocks. E.g. Data.Password.Validate itself!

Edit2: @Vlix the same problem with password exists on the old site, which you can still visit via

curl -L --fail-with-body --connect-to ::a4b9a09e2a7d64063bed132d86a3a006-ca5e820cb2bff767.elb.us-east-1.amazonaws.com https://www.stackage.org/lts-21.25/package/password-3.0.2.1 | w3m -T text/html .

I agree it’s a problem but I think it’s unrelated to the migration. I suggest opening a ticket on Issues · commercialhaskell/stackage · GitHub

1 Like