DevOps Log, 2023-11-08

Surprise! It’s another weekly log.

This week, I worked on GHC CI runners, the Stackage migration, and a little Cabal CI.

The Darwin runners recently upgraded themselves, bringing new versions of MacOS libraries that have new behavior and disrupting the overall CI configuration. The machines needed some manual care, which is ongoing. And even when they work well, GHC still experiences failures because of the library changes:

For the Stackage migration, we have powered down the old snapshot builder—the most recent nightlies have been built on stackage-builder.haskell.org! [edit: sorry, that’s not a real url. There’s no website there :joy:]

It’s really easy to find unexpected data dependencies during a migration, so I have taken the time to document the architecture as I go. Snapshot curation, the process that was just migrated, still relies on some non-migrated components. Keeping track of this stuff is making the project feel like it’s taking me a long time, but it should keep getting easier as I dive into each of the pieces. Next up I will be migrating a few other processes: Hackage mirroring, Casa curation, and the Hackage metadata refresh. These also have data dependencies on non-migrated components, but they can be migrated without disturbing any user-facing services and they’re some of the more costly processes. Doing them early is win-win. Ideally nobody will ever notice any change during this migration, but if something does go wrong, I want there to be as few moving parts as possible.

The final tiny thing is that I started bumping the Docker images used in Cabal’s Release Pipeline. This should allow Cabal to be distributed for more platforms for its next release.

On with the show!

11 Likes

You are doing a fantastic job @chreekat! May you never finish it and keep doing it forever :joy:

2 Likes