The LAST DevOps weekly log, 2024-06-27

Hello, and welcome to the last weekly log I will write. From now on, I will write monthly updates instead. (Sorry for the clickbait title.)

The reason for the change is that the HF DevOps role is being reduced to a 20%-time contract. Nobody wanted this change to happen, but the economics of the software industry simply don’t support a full-time (or even 80%-time), sponsorship-funded, purely-technical role for Haskell right now.

To wrap up these weekly logs, I want to write on a few different topics:

  • Retrospective: what went well with the role, what could have been done differently
  • Plans for the role going forward: how will I spend these ~8 hours per week?
  • Personal plans: what will I be doing with the rest of my free time?

Things I did well

I think my main achievement has been being a consistent, active, communicative participant in the ecosystem. I was responsive to issues on GHC CI, Matrix moderation, and GitLab account requests. I proactively found people stuck getting access or understanding Haskell infrastructure and got them unstuck. I played nice with everybody and fostered collaboration as a result. My weekly logs increased the palpable tempo of Haskell progress. All of this was due to a deliberate strategy: foster community activity by being active. I am proud of this strategy and its execution.

That’s not to say I didn’t do good technical work, as well. I took the lead on two complex operational maneuvers: migrating downloads.haskell.org with its terabytes of critical assets, and migrating the entirety of the Stackage.org infrastructure with its multiple interconnected parts. These actions required investigation and planning and extreme attention to detail. At the end of the day, hardly anybody noticed the changes, which is (unfortunately) a great measure of success.

One result people did notice was my improvements to the GHC CI experience. My service, Spuriobot, is identifying and recovering from 28 different kinds of spurious failures that used to plague CI with illegitimate results. I also helped characterize failures so that GHC experts could identify and fix root causes. My only informal measure of success here was “GHC developer frustration levels”, so when people come up to me at ZuriHac and tell me that I’ve made a critical difference in their experience, I think that’s a win.

Things I could have done better

While I did a decent job of identifying the most important tasks and working on them steadily, I wish I could have done more to describe expected results and document my progress. What does ‘done’ look like for my projects? I also should have used more deadlines as a focusing tool. I think these actions would have improved my efficiency.

Strategically, I think I should have spent more time “selling”. As employee #2 of a non-profit foundation, securing more funding is just unavoidable work. I’ve been around this field long enough to know the realities. I could have been trying to find funding for the Foundation, or for myself personally to offset the HF’s spend. Since I didn’t do it earlier, I have to do it now on my own time. Oops.

Plan for the role

In the one day a week I can use on this role, I will keep making incremental technical improvements to the Haskell ecosystem.

My priorities for the moment are:

  1. Ensure long-term Stackage.org reliability
  2. Improve GHC CI runner availability and reliability
  3. Continue mentoring the GSoC project that is extending Spuriobot

I will stay on the contract for at least two months, regardless of my success at finding complementary income. I will reassess the situation at the beginning of September.

Random thoughts

  1. It’s really hard to work in isolation. I wonder how it would have gone if the HF had created two half-time positions, so there could at least be a two-person DevOps team.

  2. Going into the role, I knew it would be a lot of hard development work, which includes creating brand new things and even figuring out what needs to be done in the first place. Anybody with DevOps experience knows this. But I still feel there is a spectre of confusion about this. Why didn’t I fix GHC CI yet? It’s just bug fixing, right? Maybe I’m just being defensive and everybody knows how long it would actually take to “fix” GHC CI. (Hint: years. Many years. And each year that passes moves the goal posts even further away…) But to the extent that the role could be presented for what it truly is, or could be, or should be, it would be nice to have that monkey off my back. :slight_smile:

  3. Haskell.org and Haskell Foundation are linked financially, legally, and technically, which has created some confusion for me. I was given admin access to many, but not all systems – I am still not on admin@haskell.org, not in the Haskell org on GitHub, and not inducted into anything to do with Hackage. That’s probably a good thing! TBH I’m not sure I need all the access I have. But it’s unclear to everybody who is actually responsible for what. Developing a system for onboarding ecosystem admins would really help inclusiveness and transparency. There isn’t a problem per se, but there is a better future that is possible.

Personal

So what’s next for Bryan?

Reducing my workload will give me some time to focus on a long-standing personal goal, which is to pass the language proficiency exam and apply for Finnish citizenship.

But I’ll also start looking for other work. Ideally I can find work that complements the HF contract. I would like to remain an entrepreneur, either as a freelancer or in a startup. Besides the technical skills I’ve showcased in this DevOps role, I also have experience in engineering management, and it’s no less interesting to me. I’m sure I’ll eventually find something good.

Wrap-up

So although this is the last weekly log, it’s not the end of the DevOps role nor my Haskell participation. I like Haskell because it’s fun, and I like being in the community because it’s made a huge positive impact on my life. I look at Haskell as a big community art project, and don’t let anybody tell you that art isn’t practical. So, enjoy! And see you around.

21 Likes

Bryan, I just want to say a Huge Thank You! Your steady support and calm attention to detail has made a real difference to GHC’s CI process (the bit I am closest to) and that improves my life every day.

It’s quite hard to work in a role where the absence of drama is the goal, and where the lack of problems is a measure of success. (As you say, it is hard for outsiders to really appreciate just how difficult it is to get all the moving pieces of devops to fit together and “just work”.) Let me say now that, while devops is not my world, I am so grateful to you and others for making it all work for me.

I’m glad you are still going to be around to help, and offer your expertise.

with thanks and warm good wishes

Simon

12 Likes

Hmm:

…did I miss one? If not, this would seem to qualify as “The First DevOps Monthly Log” !

Thank you for helping the rest of us go grey more slowly, and may success be yours wherever you go!

1 Like

Seconding Simon’s thanks, We’ve been lucky to have your weekly updates, informing us all of the hard work that goes on behind the scenes keeping GHC development (in particular) and other Haskell services ticking over. I’m really glad you’re staying on to give us your support. Good luck with the Suomi exam!

5 Likes