Maintain a golden test of your package's API with `diff-package-api` and `print-api`

[Kleidukos]

The machine never stops.

Joining the publication of get-tested, I have the pleasure to welcome diff-package-api to the suite of CI tools that power my continuous integration setups.

diff-package-api is a GitHub Action workflow that runs against your Haskell package and compares the API it exposes against a file tracked in your repository that contains the expected API exposed by your package.

It is powered by print-api, which is adapted from the dump-decls tool created by @bgamari for the benefit of the GHC release engineering team.

It’s still fresh from the oven, so you are highly encouraged to try it out for your projects and give usability feedback. Some aspects of its usage are a bit annoying, like the need to have one golden file per GHC version, but this is due to the very precise level of details regarding the provenance of the identifiers. For instance, between GHC 9.8.2 and GHC 9.10.1, the switch to ghc-internal has changed the provenance of TypeError:

 module Data.Text.Display.Generic where
-  type Assert :: GHC.Types.Bool -> GHC.TypeError.ErrorMessage -> Constraint
+  type Assert :: GHC.Types.Bool -> GHC.**Internal**.TypeError.ErrorMessage -> Constraint

(emphasis mine)

While I’m opening a bottle to celebrate, I will also produce an architecture document that ought to answer a lot of questions about the precise details of how it all works and when it stops working.

This is not a PVP compliance checker (although perhaps it could?). For the moment it forces you to stay honest on the changes of your packages’ API.

---

Here a few screenshots from a PR on text-display. You can go there to see (or hear) by yourself if the screenshots are not convenient.

The updated API interface file in a pull request

When diff-package-api complains

Me when I can keep trace of potentially breaking API changes

dope.

---

My warmest thanks to @bgamari, @TeofilC and @mpickering for their help, both direct and indirect.

[Vlix]

Love hearing about developments regarding stability and backwards compatibility

[Kleidukos]

Cheers, I have to admit that the zeitgeist of the Haskell community encouraged me, with the importance given to Stability and its working group.

[arianvp]

This is cool! I wonder if we can go a bit further and implement something like gorelease command - golang.org/x/exp/cmd/gorelease - Go Packages

which automatically suggests semver bumps based on the API diff. Always dreamt of something like that

[jaror]

There is a prototype: policeman :: Kowainik, but it is not maintained. Maybe it would be easy to revive using diff-package-api.

[andreabedini]

Wow this is fantastic @Kleidukos!! I had played with the same idea and I’m very happy to see you deliver it!

We struggled with noticing API changes in cabal, so let’s start using it there right away (or you opened a PR already? )

[harendra]

We are using a similar tool in our CIs which diffs API based on the haddock generated hoogle file. We also generate a API changelog using this tool. I updated the following in another thread:

We use packdiff (GitHub - composewell/packdiff) to find the complete API diff from a previous release/commit. In fact we have it integrated in our CI to report API changes in each PR, the output looks like this: Multiple issue resolution targeting the 0.11 release · composewell/streamly@ed72f22 · GitHub (check out the “run packdiff on streamly-core” section).

[Kleidukos]

Not yet, I am taking some vacations right now but it would be grand to have it indeed.

[Kleidukos]

Nice, that’s perhaps a less involved solution I’d be interested in providing it as a backend for the diff-package-api Workflow

[harendra]

Sure, we will be happy to help if it is useful.

[geekosaur]

It’s on my list. I’ve been somewhat busy with medical things of late, so we’ll see when I get to it. (@Kleidukos: Is the action you show available in general?)

[Kleidukos]

It’s available for all GitHub Actions workflows to re-use, yes!