What is the ideal dependency update workflow?

Learn
1

Summary

I recently needed to update a library to support building with Cabal 3.14.0.0. This was more difficult than I hoped. I think listing the exact (breaking) API changes in a single changelog document would have made it easier for me to do this. Should all major libraries adopt such a policy?

Ideal workflow

When a package I depend on makes a breaking change my ideal workflow would be:

  1. read the changelogs to see if there are any changes that could silently break code
  2. bump the version bounds try to build
  3. if there are errors, search changelogs for key words from the error
  4. read about the change
  5. follow recommended migration strategies if possible or figure out something by myself

This workflow relies critically on being able to find the changelog entry that corresponds to any breaking API change.

Do you think this is a reasonable workflow?

A case study of the status quo

This issue is motivated by my attempt at updating the uuagc package to support newer versions of the Cabal package.

The first step I took was relax the upper bound on Cabal from < 3.11 to < 3.15 and built with --constraint="Cabal == 3.14.*" which resulted in these errors:

src/Distribution/Simple/UUAGC/UUAGC.hs:60:50: error: [GHC-61689]
    Module ‘Distribution.Utils.Path’ does not export ‘PackageDir’.
   |
60 | import Distribution.Utils.Path (getSymbolicPath, PackageDir, SourceDir, SymbolicPath)
   |                                                  ^^^^^^^^^^

src/Distribution/Simple/UUAGC/UUAGC.hs:60:62: error: [GHC-61689]
    Module ‘Distribution.Utils.Path’ does not export ‘SourceDir’.
   |
60 | import Distribution.Utils.Path (getSymbolicPath, PackageDir, SourceDir, SymbolicPath)
   |                                                              ^^^^^^^^^

So PackageDir and SourceDir got removed in 3.12 or 3.14. I looked in the changelog to see what changed, but immediately ran into the fact that Cabal’s changelog just links to separate markdown documents for each version, that’s a bit inconvenient but I could to open both and search for PackageDir. Sadly that results in no hits.

I had to fall back on opening both versions on Hackage to find out 3.12 did still have PackageDir and it was dropped in 3.14. At this point I also noticed that Pkg and Source were introduced at the same time and they did (almost) fit in the same place where PackageDir and SourceDir used to be. Finally, the buildDir function also changed slightly and had to search around for something reasonable that would fit the types. In the end, I’m not 100% sure I did the right thing.

Further compilation revealed two new errors related to a different change:

src/Distribution/Simple/UUAGC/UUAGC.hs:109:52: error: [8;;https://errors.haskell.org/messages/GHC-83865GHC-838658;;]
    • Couldn't match type ‘[Char]’ with ‘Suffix’
      Expected: Suffix
        Actual: String
    • In the expression: "ag"
      In the first argument of ‘(:)’, namely ‘("ag", ag)’
      In the expression: ("ag", ag) : ("lag", ag) : knownSuffixHandlers
    |
109 |   hooks = simpleUserHooks { hookedPreProcessors = ("ag", ag):("lag",ag):knownSuffixHandlers
    |                                                    ^^^^

So some new Suffix type has been introduced somewhere in 3.12 or 3.14. Again the changelogs do not mention Suffix at all.

The only other way was to open both versions on Hackage and search for Suffix in their API directly. That showed me that Distribution.Simple.PreProcess.Suffix was introduced in 3.12.

Now to figure out what actually changed I looked over the changelog for 3.12 again, now with the added knowledge of it being part of the preprocessing mechanisms, but I still cannot find it.

At this point I gave up trying to figure out what had changed and just looked at the Haddock which said Suffix is just a newtype over String and that it had an IsString instance, so I just slapped a OverloadedStrings on the Simple.UUAGC.UUAGC module and that fixed the error. I want to stress that this is me just blindly applying a change until the typechecker accepts it, which is bad practice!

In fact, I just discovered that someone has written a description of this change even mentioning this is a breaking change for some users, but I think the only way to find it is to view the git blame of the line in Distribution.Simple.PreProcess.

8 Likes
2

I think the CLC could very well publish a recommendation document for core libraries and their maintenance practices.

But it should be clear that those are recommendations. We don’t want to overstep or dictate. It’s hard enough to find engaged maintainers.

4 Likes
3

I originally wanted to post this to the CLC issue tracker. But I thought there are people much more qualified to draw up such guidelines than me. So I would like to hear from you if you have any suggestions. Perhaps we can draft the guidelines together.

1 Like
4

Also, one of the advantages of Cabal’s approach of linking to changelogs on github is that the changelogs can be revised without uploading a new package version. I think a better way to solve that is to allow changelog changes on Hackage via revisions or some similar mechanism that does not require a full release.

Edit: it seems there’s already an open issue about this: Reupload changelogs / readmes from Hackage · Issue #771 · haskell/hackage-server · GitHub

1 Like
5

I think that is a reasonable workflow. In practice, I tend to start at Step 2 and skip Step 1.

My own interactions with the modern Cabal project suggest that it tries hard to encourage (and have) a comprehensive and useful change log, but things can slip through.

It might be useful if the CONTRIBUTING.md had a short section on what somebody should do if they spot a gap in the change log. (That was a gap in Stack’s equivalent - now filled.)

1 Like
6

I remember reading a thread/post that talked about tools for automated changelogs based on the library’s api, much like Elm. It would be nice if those tools were more prevalent in the ecosystem. Maybe even include their output in Changelog.md, or a new file.

3 Likes
7

Yeah, I think you mean this thread:

2 Likes
8

print-api has been lucky to receive more attention (and thus more tickets) recently. The most important of them require work from people who understand the GHC API in some capacity, or are willing to learn just what’s required. I need help in that area, otherwise the developer experience will suffer.

6 Likes