Upcoming maintenance on Haskell services

Announcements
#1

Hello folks,

On Thursday 24 November at 6 AM UTC, I will start a 4-hour maintenance window for the following Haskell services:

  1. downloads.haskell.org
  2. gitlab.haskell.org

In particular, I expect the following outages:

  1. downloads.haskell.org will be read-only for the duration of the window.
    • No uploads
  2. gitlab.haskell.org will be read-only for the duration of the window.
    • No CI pipelines(!)
    • No git pushes
    • No commenting on issues or merge requests
  3. gitlab.haskell.org will furthermore be entirely inaccessible very briefly.
    • There will be 1-2 outages.
    • Each outage will last a few seconds.

The reason for the maintenance is that our hosting sponsor, Equinix Metal, is performing a facilities migration. One of our servers, gitlab-storage, is slated for retirement. All data and services that run on that server must be moved to a new server.

I’m sending this first notification to a wide audience as an extra precaution in case something goes wrong. :slight_smile: In reality, I expect this maintenance to only affect GHC developers and a small group of ecosystem maintainers. In particular, web access of https://downloads.haskell.org should not be affected by this maintenance.

I will send further updates to a smaller audience.

Note: The actual migration will be quick. Most of the maintenance window will be spent waiting for DNS propagation.

-Bryan
Haskell Foundation DevOps

7 Likes
Haskell Foundation DevOps Weekly Update, 2022-11-23
#2

I have finished the migration.

As it turns out, I did break https://downloads.haskell.org for a short while. :frowning:

Furthermore, CI on gitlab.haskell.org is unexpectedly down, and will remain so until later today.

I will write up an after-action report soon!

5 Likes
#3

Thanks for your work :heart:

And thank you Equinix Metal :grin:

#4

(…there’s probably a corollary for system upgrades ;-)

Always remember: things could have been worse…

#5

I think the fact that I ran into accidents in spite of all the prep I did is a good rationale for doing a bunch of prep. :slight_smile: I had plans for when things went wrong, and I was pretty familiar with the system from all the analysis, which allowed me to diagnose things faster.

#6

I think the fact that I ran into accidents in spite of all the prep I did is a good rationale for doing a bunch of prep. […]

…if what I wrote suggested the opposite, that wasn’t my intention. I interpreted:

As it turns out, I did break [some things …]

as an expression of disappointment, with my previous message being the result: apparently I was mistaken…

1 Like
#7

Ah, no, my mistake! I did interpret your message as a positive one, and I was agreeing and providing more thoughts about it. :slight_smile: I should have prefaced it with some sort of agreement noises. Text-based communication strikes again!

1 Like