For what it’s worth, @simonmar wrote that they at Facebook might be better off without warnings:
- Generating compiler warnings for things that will change in the future lets us choose whether to fix it now or later, but it doesn’t save us any time. In fact from our perspective it would be better not to have the warning stage, it’s just extra churn. We’ll fix it when it breaks.
I would certainly consider Facebook as a statistical outlier in many, many domains, including the workforce they can deploy to make such migrations happen.
I’m confused by this? head.hackage is a set of patches as a hackage overlay. It patches hackage packages to make them compatible with the compiler. Thus masking any breakage?
I’d much rather see the compiler be tested in CI against a set of vanilla hackage packages (maybe with --allow-newer base). That would show the real world breakage that the compiler has. head.hackage effectively says: if you apply all these patches to packages on hackage, the compiler can compile them. If you don’t apply them, the compiler will reject and not be able to compile them. (whether this is CLC, or SC breakage, …).
Edit: It could also validate that the diagnostic messages the produced are helpful towards migrating the code. Take the sample code from above, and try to compile it with 9.2, looking at the error messages it gives, and see if one can come up with appropriate migration. It’s a worthwhile exercise.
Yes I agree that an empty head.hackage would be best of all. But if for whatever reason (e.g. a well-warned-about change is actually taking place) breakage is non-zero, we don’t don’t want one package that breaks to disable the entire CI for the large package set. That would make the entire CI infrastructure bittle to single points of failure, would it not?
If the warning was there fore the last 2 versions (one year), why has this package, at the core of so many dependencies not been patched and a new release made it onto hackage for the whole year?
GHC N-2 would have been warning about this in CI for at least a year by then. Someone would have picked this up, and patched such an important package that is a dependency of a large package set or am I missing something?
Thank you very much for this post-mortem analysis, it’s really useful for people like me who are less acquainted with the intricacies of breaking changes during the development of GHC. 
Impact assessment has become a bigger subject in last couple of years (in great part thanks to the current incarnation of the CLC), and embracing it at all levels would certainly send the right message that the institutions of Haskell care about their users.
I for one am excited to see how the development process will transform.
I don’t know why not. I’m just saying that it would be better not to depend in such a brittle fashion on so many package authors whose actions we do not control. We need to be resilient, not brittle!
FWIW I’ve definitely seen instances of well signposted warnings/deprecations not being reacted to. The example that comes to mind is that in bytestring-0.11.0.0 some modules were removed that had been deprecated for multiple years, and I had to make some MRs to fix some of my dependencies.
I think there’s reasonable reasons why things like this can happen:
In any case, warnings definitely help, but even with warnings sometimes dependencies won’t be updated and it’s quite understandable
Oh! I fully agree! And thank you for writing this line it made me realize an important point I’ll come to in a bit. My first reaction to this statement was: but GHC is very brittle, this is precisely what I try to push back against.
Then I realized that we actually perceive the exactly same thing. Your “product” is GHC. Mine is another Haskell code base. You feel exactly the same pain I feel. But your solution to this is to produce a hackage overlay to patch all packages that need patching to make developing your product (GHC) more resilient.
The difference is: you are in full control of your product. There is a reason GHC tries to limit its dependencies a lot. How much more brittle would you perceive your product if GHC dependent on 10x as many packages from hackage?
So what you are effectively suggesting is, that I use head.hackage, and extend this for all the other dependencies that I have in my dependency tree, to make my Haskell codebase compatible. I am not in full control; the compiler is not part of my product.
If I am a consumer of a dependency that starts throwing warnings during compilation, I have two options:
Without warning for a given period, none of this is possible.
Indeed warnings can help a lot. My point is that they are not a silver bullet. Someone still needs to be a good citizen and put in the work to update things, and we shouldn’t be surprised if other people don’t do this work.
Not quite! Here is my thought process
This is not theoretical. We have experienced extensive frustration with slow package updates. I am not blaming the authors for that – there can be legitimate reasons. I just want a resiliency mechanism, that allows us to do extensive testing without relying on 100.0% timely responses from package authors: that’s head.hackage.
So it very much isn’t just the packages that GHC itself depends on.
Moreover, this same mechanism would be useful to you too. You kindly offered
I wouldn’t mind testing nightly releases against large codebases and reporting bugs, regressions, …
But again, you can only do so if the nightly build comes with packages that actually compile with it.
Does that make sense?
I hope it’s clear that I am not arguing that breakage should be regarded as unimportant, or swept under the carpet. I just want our testing and deployment infrastructure to be resilient to the churn that (however well advertised) I think we are all committed to dealing with as the price of innovation.
I feel like at the core of this entire issue is that building the software commons that we’d like to see requires a lot of work. Sometimes it’s highly specialized work like creating a refactoring tool that can help adapt to breaking changes. Sometimes it’s communicating planned changes, or figuring out if a change can cause breakage. Sometimes it’s less specialized work like submitting one-line MRs to fix a changed import.
But at the end of the day it’s all work and often it’s work that people are doing for free.
I think when thinking about these things these considerations should be brought to the fore.
I think we should be asking ourselves things like:
I feel like too often we focus on technical things and erase the human component to our detriment.
I think it’s good to indeed clarify that we have all various vantage points. It’s very easy for a web application (which is my domain of expertise) to directly depend on eighty packages (and I’m not exaggerating) from Hackage, and three hundred indirectly.
I count 35 dependencies in the libraries directory of GHC, and so I understand a bit more that there can be a disconnect between the GHC team and downstream users of the Haskell ecosystem on the (sometimes) disproportionate effect that dependencies have on projects.
Edit: Our main product at work as 531 indirect dependencies, for a bit less than 700 modules.
I think there’s an interesting question about how warnings help. Having warnings is no good unless you
Often this is not true for maintainers of packages. But Moritz points out that sometimes it is true for downstream. So that raises a question: could we find a way to encourage downstream consumers to help update packages that they depend on to fix compatibility issues? That might look something like: build some decent subset of Hackage with a recent GHC and -Werror=compat or some other way of sucking out the compat errors, make a big list, and make it available for people to pick up and submit patches for.
In some ways this is similar to what we might have hoped for with GHC.X.Hackage, except that the idea there was to have a focussed burst of fixing between an alpha release of a GHC version and its full release. Doing the same process based on compat warnings gives you much more time to handle things, but could still be amenable to community effort.
Let’s assume for a moment GHC 9.6 would accept code that 9.2 accepted warnings-free. We could put 9.6 alphas into CI today. And see which packages are a liability, and we’d have a serious incentive either (helping to) fix those packages. Or if that is not feasible, find alternatives instead.
We could also very proactively report regressions (it’s the same code between two compilers, if the new one exhibits bugs the old one did, or regresses, we could inform about this). As it is today, I can maybe tell you if 9.2 regresses against (a patched version) of 8.10. But I can’t tell this for the battle tested code against 8.10, as the migration to 9.2 required significant changes to the dependency tree. So I may be able to say if the new dependencies regressed against the old ones. And I may be able to say if 9.2 regresses against a patched 9.2; but what do I do with this information? 9.2 is pretty much end of line already. We have 9.4, 9.6 and master now.
As it is today, I have very little faith that putting 9.6 into CI today would yield anything that even remotely compiles. I’d love to be surprised, and I may perform this experiment next week if time permits.
I want to stress that downstream consumers of GHC (and the haskell ecosystem) have a vested interest in keeping their code working. This can either be by freezing it for all eternity, or by proactively helping along. But a compiler that doesn’t compile existing code, and requires significant surgery just to make existing (warning free!) code compatible, is not very useful.
I think @TeofilC’s observation is perhaps the most important observation in the whole thread.
@TeofilC I want to highlight these two points from your comment:
We rely on a lot of volunteer work. Especially a lot in the ecosystem of maintainers diligently looking after packages. Often in their spare time. And even if you are being paid to work on Haskell, package maintenance might not be on your job description.
In my opinion, being cautious and mindful of the downstream impact changes have goes a long way towards this. Getting a warning or deprecation notices that informs the user of the necessary changes reduces the burden to figure out what
Often times the breakage is not obvious. Taking the example I posted, the error messages are barely helpful.
Error 1:
Broken.hs:21:21: error: Operator applied to too few arguments: !
|
21 | data X a = X { s :: ! a }
| ^
what arguments are missing? I guess I could try to just remove the space? But the error doesn’t say: “did you intent to use a strictness annotation !a?”. After all BangPatterns is enabled.
Error 2:
Broken.hs:31:10: error:
• Non type-variable argument in the constraint: MonadError D m
(Use FlexibleContexts to permit this)
• In the instance declaration for ‘C m T’
|
31 | instance MonadError D m => C m T where
| ^^^^^^^^^^^^^^^^^^^^^^^
this one is actually quite good. It tells you that you need FlexibleContexts here. It doesn’t say that this used to be implied by UndecidableInstances though, which would have been helpful. (Fun fact: this error will not happen if you don’t have Haskell2010 enabled, something you might just have in your codebase because the stock cabal file contained the language stanza).
Error 3:
Broken.hs:38:10: error:
Variable not in scope: (@) :: (String -> a0) -> t0 -> t1
|
38 | h = read @ Int "5"
| ^
Broken.hs:38:12: error:
• Illegal term-level use of the type constructor ‘Int’
imported from ‘Prelude’ at Broken.hs:14:8-13
(and originally defined in ‘GHC.Types’)
• In the second argument of ‘(@)’, namely ‘Int "5"’
In the expression: read @ Int "5"
In an equation for ‘h’: h = read @ Int "5"
|
38 | h = read @ Int "5"
|
Pretty much as unhelpful as (1).
Error 4:
Broken.hs:36:5: error:
• No instance for (Data.Bifunctor.Bifunctor Y)
arising from a use of ‘bimap’
• In the expression: bimap id id Y
In an equation for ‘g’: g = bimap id id Y
|
36 | g = bimap id id Y
| ^^^^^
This one is so-so. It claims no instances was found, though the code to derive the instance is clearly there. It does not give any hint that there are now new staging/ordering restrictions.
The point I’m trying to highlight here (to exemplify @TeofilC’s first point) is that on a codebase with lots of code, you might not have written and are not intimately familiar with (someone else wrote that code, and maybe yet someone else reviewed it, and now you are spending your time looking at errors …), it is increasingly hard to decipher what’s wrong, and what the appropriate fix is. Especially if the file is multiple hundreds of lines.
While this thread might look like I’m complaining that GHC keeps breaking a production codebase, I don’t even think that’s the worst. There are people who are paid to look at this. What about all those poor folks who maintain packages in their spare time. Is their time best spent on dealing with continuous churn? I’d love to help out! If you are paid to work on a haskell codebase, and can trivially test that codebase against a recent compiler, instead of waiting until everyone else has done the work for you, you could proactively help migrating packages you rely on. This all however can only work if you can just swap out the compiler trivially.
As I’ve alluded to before, experience over the last years has been that there is little point in trying a compiler for any serious project before we haven’t hit at least .4, too much is simply broken hard, and simply won’t compile. Without using the compiler, no one will know about all the migrations necessary.
An idea floated around before is to gate a new release of ghc until haskell-language-server has caught up and ready to be used with it. Even without making this a hard rule, I think it would be useful to the GHC team to take haskell-language-server as a benchmark, it is becoming a central piece of the ecosystem and it has enough depedencies to give an assessment of the changes required to support a new GHC (yes, intentionally, including the GHC API itself).
Thank you for clarifying this @simonpj!
Why does this lead to “let’s patch the packages” instead of, “can we make the compiler more resilient to accept existing code”?
No, I could do that if the compiler accepted existing code (potentially with warnings), and not abruptly stops accepting existing code.
To come back to an earlier idea around a -DEXPERIMENTAL compiler, which could break arbitrarily and without any notice. Having a experimental.hackage overlay for this seems sensible to me.