# Upcoming maintenance on Stackage services, 2 Jan 2024

**URL:** <https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400>\
**Category:** Announcements\
**Created:** [December 27, 2023, 10:05am UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400 "2023-12-27T10:05:21Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [December 27, 2023, 10:05am UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/1 "2023-12-27T10:05:21Z")

</div>

**Which services** : [stackage.org](http://stackage.org) and [casa.stackage.org](http://casa.stackage.org)  
**When** : Tuesday, 2 January 2024 ~~08:00 UTC~~ 11:00 UTC  
**Expected impact** : None

**What** : The services will be served by new processes running on Haskell Foundation infrastructure.  
**How** : DNS proxies will be updated to point to the new servers, which should be an instant change.

The new services have already been running for a couple weeks, although they haven’t been getting any usage. If you see any trouble, contact me on this thread or at one of the locations listed below. If anything goes terribly wrong, I can just roll back to using the original FP Complete servers.

Where to contact in case of trouble (pick the first one that’s easiest for you):

1. This thread
2. [Haskell room on Matrix.org](https://matrix.to/#/#haskell:matrix.org)
3. [Haskell Foundation Slack](https://join.slack.com/t/haskell-foundation/shared_invite/zt-z45o9x38-8L55P27r12YO0YeEufcO2w)

---

<div class="post-metadata">

**Author:** ![tomjaguarpaw](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/tomjaguarpaw/32/1230_2.png) [@tomjaguarpaw](https://discourse.haskell.org/u/tomjaguarpaw)\
**Post date:** [January 1, 2024, 1:57pm UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/2 "2024-01-01T13:57:41Z")

</div>

I guess this isn’t related, since it’s not 2nd Jan yet, but all of Stackage Hoogle seems to be unavailable. Is this expected?

> The given Hoogle database is not available.

For example

- [Hoogle Search :: Stackage Server](https://www.stackage.org/nightly-2024-01-01/hoogle?q=%3E%3E%3D)
- [Hoogle Search :: Stackage Server](https://www.stackage.org/lts-22.4/hoogle?q=%3E%3E%3D)
- [Hoogle Search :: Stackage Server](https://www.stackage.org/lts-21.25/hoogle?q=%3E%3E%3D)
- [Hoogle Search :: Stackage Server](https://www.stackage.org/lts-21.21/hoogle?q=%3E%3E%3D)
- [Hoogle Search :: Stackage Server](https://www.stackage.org/lts-21.11/hoogle?q=%3E%3E%3D)
- [Hoogle Search :: Stackage Server](https://www.stackage.org/lts-21.7/hoogle?q=%3E%3E%3D)
- [Hoogle Search :: Stackage Server](https://www.stackage.org/lts-20.26/hoogle?q=%3E%3E%3D)
- [Hoogle Search :: Stackage Server](https://www.stackage.org/lts-20.24/hoogle?q=%3E%3E%3D)

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 7:49am UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/3 "2024-01-02T07:49:46Z")

</div>

Nope, that’s not expected. I’m looking into it now.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 7:53am UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/4 "2024-01-02T07:53:03Z")

</div>

The same problem affects the new service (which _is_ expected, since they should be using the same data source). So I’ll postpone the migration and fix this first.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 9:32am UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/5 "2024-01-02T09:32:30Z")

</div>

@sibi and I managed to track down and resolve the issue. Looks like it’s working now!

1. The messages are a bit misleading: Stackage only uses the hoogle database from the latest snapshot.
2. Stackage fetches hoogle databases from a certain data source (an S3 bucket).
3. A process called `stackage-server-cron` generates the databases for each snapshot and loads them into that data source.
4. The `stackage-server-cron` process had been hitting some resource limits and was not successfully generating the databases.
5. Modifying the resource limits allowed `stackage-server-cron` to run successfully.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 9:51am UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/6 "2024-01-02T09:51:51Z")

</div>

Now that the Hoogle issue is resolved, I will continue with the migration at 11:00 UTC, right after I get some lunch.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 11:22am UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/7 "2024-01-02T11:22:18Z")

</div>

The new services are now serving [casa.stackage.org](http://casa.stackage.org) (used internally by Stack, in case you were wondering) as well as [stackage.org](http://stackage.org). Let me know if you see any issues!

---

<div class="post-metadata">

**Author:** ![Vlix](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/vlix/32/1551_2.png) [@Vlix](https://discourse.haskell.org/u/Vlix)\
**Post date:** [January 2, 2024, 1:54pm UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/8 "2024-01-02T13:54:40Z")

</div>

Feels snappy so far 👍 Thanks for the effort in all of the migrating 🙂

Few things:

- it doesn’t seem like you can view any of the haddock documentation since [`nightly-2023-12-27`](https://www.stackage.org/nightly-2023-12-27) ([2023-12-26](https://www.stackage.org/nightly-2023-12-26) still seems to have haddock documentation for all packages)
- When clicking on [`password-3.0.2.1`](https://www.stackage.org/lts-21.25/package/password-3.0.2.1) in any LTS, a bunch of the modules aren’t listed in the module tree:

```nohighlight
Data
  |
  `- Data.Password
       |
       `- (Data.Password.Argon2 missing)
       `- (Data.Password.BCrypt missing)
       `- (Data.Password.PBKDF2 missing)
       `- (Data.Password.Scrypt missing)
       `- Data.Password.Validate

```

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 2:04pm UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/9 "2024-01-02T14:04:54Z")

</div>

Thanks for the report!

> [@Vlix](#):
>
> it doesn’t seem like you can view any of the haddock documentation since [`nightly-2023-12-27`](https://www.stackage.org/nightly-2023-12-27) ([2023-12-26](https://www.stackage.org/nightly-2023-12-26) still seems to have haddock documentation for all packages)

Indeed. This is surprising. I’ll see what I can figure out.

> [@Vlix](#):
>
> When clicking on [`password-3.0.2.1`](https://www.stackage.org/lts-21.25/package/password-3.0.2.1) in any LTS, a bunch of the modules aren’t listed in the module tree:

Not sure about this one. I do see that all those modules are guarded behind flags. The flags default to True, but maybe something is interfering.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 2:23pm UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/10 "2024-01-02T14:23:43Z")

</div>

> [@chreekat](#):
>
> > [@Vlix](#):
> >
> > it doesn’t seem like you can view any of the haddock documentation since [`nightly-2023-12-27`](https://www.stackage.org/nightly-2023-12-27) ([2023-12-26](https://www.stackage.org/nightly-2023-12-26) still seems to have haddock documentation for all packages)
> 
> Indeed. This is surprising. I’ll see what I can figure out.

Aha, I figured it out. One background service that I thought wasn’t needed yet is, in fact, needed. It’s running now and should finish populating the Haddocks soon.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [January 2, 2024, 2:39pm UTC](https://discourse.haskell.org/t/upcoming-maintenance-on-stackage-services-2-jan-2024/8400/11 "2024-01-02T14:39:09Z")

</div>

> [@chreekat](#):
>
> > [@Vlix](#):
> >
> > When clicking on [`password-3.0.2.1`](https://www.stackage.org/lts-21.25/package/password-3.0.2.1) in any LTS, a bunch of the modules aren’t listed in the module tree:
> 
> Not sure about this one. I do see that all those modules are guarded behind flags. The flags default to True, but maybe something is interfering.

I peeked at the stackage-curation logs, and I see some messages that look related:

```haskell
Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Argon2.hi
         does not exist:

Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Argon2.hi
         does not exist:

Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Argon2.hi
         does not exist:

Warning: Dependent file listed in:
         /home/curators/work/unpack-dir/unpacked/password-3.0.2.1-5c98dec407679828886891c1bd0efa11094028dd2bc49ba0f3074d3fbe005da7/.stack-work/dist/x86_64-linux-tinfo6/ghc-9.8.1/build/password-tasty/password-tasty-tmp/Data/Password/Validate.hi
         does not exist:

```

So this looks like a stackage curation issue. @juhp, perhaps you have some insight?

Edit: those messages may be red herrings, as I see quite a few of them for many different packages, including many modules that _do_ have Haddocks. E.g. Data.Password.Validate itself!

Edit2: @Vlix the same problem with `password` exists on the old site, which you can still visit via

`curl -L --fail-with-body --connect-to ::a4b9a09e2a7d64063bed132d86a3a006-ca5e820cb2bff767.elb.us-east-1.amazonaws.com https://www.stackage.org/lts-21.25/package/password-3.0.2.1 | w3m -T text/html `.

I agree it’s a problem but I think it’s unrelated to the migration. I suggest opening a ticket on [Issues · commercialhaskell/stackage · GitHub](https://github.com/commercialhaskell/stackage/issues)