The ^>= operator in .cabal files

#53

My PR makes this a recommendation rather than a requirement. I find that more reasonable spec-wise.

2 Likes
#54

I don’t think that makes a difference. PVP compliance itself is not required, but instead specified on hackage with the RFC-meaning of should. Since we only have recommended/encouraged adherence to the PVP as a whole, there’s no reason to weaken the PVP directly as well.

1 Like
#55

The spec indicates very clearly that this may change:

At some point in the future, Hackage may refuse to accept packages that do not follow this convention. The aim is that before this happens, we will put in place tool support that makes it easier to follow the convention and less painful when dependencies are updated.

1 Like
#56

No worries, as

so it won’t happen anytime soon :wink:

1 Like
#57

Yeah… and I think all this speak is not very spec-like and belongs into faq or some other soft document. Not in the spec.

1 Like
#58

After seeing and helping with the fallout from the (entirely correct and PVP-compliant) release of aeson-2.2.0.0, I’ll add another vote for “please don’t”. Cleaning up the mess involved (for several packages) downloading every release they ever made, grepping through for uses of import Data.Aeson.Parser (which moved to attoparsec-aeson; thank god there was an easy thing to test for), and then revising many Hackage releases so the old versions don’t get pulled into strange build plans.

3 Likes
#59

I think there are some misunderstandings:

  1. hackage doesn’t enforce PVP at the moment anyway: it’s questionable whether the proposed change would have any real world impact, except for tooling that uses the PVP spec. It’s still recommending to use upper bounds.
  2. Actually, I find it highly questionable that the spec talks about ecosystem issues at all… and not only that, it also talks about a specific ecosystem. That makes it a poor spec. It’s a version spec and shouldn’t talk about how to handle your dependencies at all other than giving you expressive syntax to describe them
  3. hackage or any other ecosystem can easily enforce a superset of the spec. That’s entirely reasonable.
  4. Why would I use upper bounds on bytestring if all I’m importing is pack? What is the reason to have the maintainer play whack the mole on upper bounds in this case?
#60

This is the first I have heard of such a thing!! Starting in Haskell in 2010, the upper bound requirement always felt to me like an idea based on theory rather than practice, and that’s what made all of its shortcomings seen even more regrettable. All I’ve ever known are experience reports with the upper bound requirement, which are universally negative.

If there are painful experience reports of a time before(?) the upper bound requirement, I would love to hear them. It might go a long way towards making it more palatable.

1 Like
#61

If there are painful experience reports of a time before(?) the upper bound requirement, I would love to hear them. It might go a long way towards making it more palatable.

The article motivating the pvp was published in 2005, right as cabal and hackage were really coming into being. The first pvp page on the haskell wiki reached a recognizable form in 2007: Package versioning policy - HaskellWiki

Here are all 15 packages that were on hackage in 2006:
https://web.archive.org/web/20060303165044/http://hackage.haskell.org/ModHackage/Hackage.hs?action=view

What I will note is that we do have experience with upper-bounds and not-upper-bounds that is extensive nonetheless, because there have been plenty of packages on hackage that do not follow the pvp with regards to upper bounds, and we have experienced the difficulties in keeping things working with them.

As jackdk describes, if a package has many releases, and no upper bounds, then a new release of a dependency that causes breaks necessitates adding upper bounds revisions to all prior versions as well, lest the solver fall back to them. In the converse case, it requires relaxing upper bounds on at most the few most recent releases.

Most end-users do not see the pain of this, and only the maintainers who do not put upper bounds do, as well as the trustees who have to go fix the ecosystem when problems occur. We never did create a database of all the reported issues to run quantitative analysis on, but the old reddit threads arguing about this stuff all had a fair accounting of stories as I recall (and also covered most all of the discussion here, repeatedly).

A sampling:

Skimming btw, I see a number of proposals to improve things have been already implemented (including something like the carat operator, as well as the --allow-newer flag [which can be very granular in project files]).

Ideas for how to collect more quantitative data on the effects of different bounds policies are very welcome – and indeed one of the threads has some pretty neat analysis derived from the 01-index.tgz hackage tarball and analyzing metadata revisions.

2 Likes
#62

Here is an alternative angle that would allow the haskell community or hackage to create their own policies and take away the burden from PVP maintainers to decide on such far reaching consequences: [RFC] Make PVP ecosystem/language agnostic · Issue #54 · haskell/pvp · GitHub

#63

@jackdk’s comment is an example of painful experience due to missing upper bounds, isn’t it? Or did I misunderstand your question?

1 Like
#64

You’re right, it is.

So am I correct in understanding that the trade-off is, at the extremes,

  • An author releases a new major version of a package for reasons other than a breaking change. No downstream user can use it until any/all intermediate packages are revised to mark compatibility.

Versus

  • An author releases a breaking change in a new major version. Any current or old version of an intermediate package will satisfy the build plan, but will fail to produce correct results.

(Of course, most situations are not at either extreme, i.e. releases are made for a mix of reasons and breakage only affects some percentage of consumers. I think a full analysis would need to account for the factors to be accurate, but I just want to be sure I understand the fundamental trade-off first.)

2 Likes
#65

@chreekat a cost-benefit analysis of upper bounds from the viewpoint of Hackage Trustees is available at https://github.com/haskell-infra/hackage-trustees/blob/master/cookbook.md#best-practice-for-managing-meta-data.

5 Likes
#66

(cross-posting from the GitHub issue linked above)

IMHO the essence of upper bounds is this: being able to compile an old unmaintained package should be a no-op.

Of course using a newer GHC might not be possible, but with upper bounds one is guaranteed to have at least one build plan that works, for some version of GHC.

As a test, someone tell me what dependency versions I need to make wai-middleware-authwork. It’s not a trick, I did spend a couple of days trying to figure it out and gave up :blush:

7 Likes
Compiling aeson-1.5 projects on ghc-9.2
#67

Another angle: package authors report that bumping upper bounds is time-consuming but this effort saves a ton of time to package consumers! Without upper bounds they will have no idea how to make your package work.

Also, let’s be honest:

  • CIs and automations will eventually disappear (burnout is real)
  • Authors and maintainers will disappear too, moving to other jobs, or other lives.
  • If you don’t put upper bounds in a package, your package will bit rot quickly, and I will have lost something valuable :slightly_smiling_face:
5 Likes
#68

Maybe…

What you actually would want to know is: for every dependency (of the old package) that doesn’t have an upper bound, what is the latest dependency version available at the time the old package was added?

This is similar to a freeze file, but not quite the same. You basically fall back to the index of the time the package was added, but only for its dependencies lacking upper bounds.

I believe this isn’t too hard to achieve maybe, although I am not too knowledgeable about the hackage index format.

2 Likes
#69

:thinking: I am not sure whether this works or not but it is an idea worth exploring. Should we care if the package does not work with those versions?

#70

I’m not sure upper bounds are sufficient though, since the package might not work with older versions of GHC. IMO if you care about this use case, use Stack snapshots

#71

Yes, this is another feature that’s necessary to implement: asking cabal “what is the latest version of GHC you can find a build plan for?”

That can’t be too hard: instead of pinning base (and boot library) versions, pass it to the solver without bounds, then extract the base version it comes up with, then map that to a GHC version.

Also see 'ghcup satisfy' command · Issue #109 · haskell/ghcup-hs · GitHub

1 Like
#72

I know you are repying to Julian, from my pov using an older GHC is not a problem.