So let’s all now get out of @Ericson2314’s way and see what he can do with @nomeata’s previous work - unless I’ve missed something (again!), we’re not being asked to join in. @Ericson2314 may be able to apply fresh insights to the problem, which @nomeata wasn’t aware of in 2013.
Thanks for the explanation, I think I get it now. The instance tree only is forced to only branch near the root, so most of the tree are just linear lists.
I do think that is rather limiting. For example you wouldn’t be able to write the popular data types a la carte subtyping class:
data (f :+: g) e = Inl (f e) | Inr (g e)
class (Functor sub, Functor sup) => (:<:) sub sup where
inj :: sub a -> sup a
instance Functor f => (:<:) f f where
inj = id
instance (Functor f, Functor g) => (:<:) f (f :+: g) where
inj = Inl
instance (Functor f, Functor g, Functor h, (:<:) f h) => (:<:) f (g :+: h) where
inj = Inr . inj
But that should probably be a closed type class anyway, so perhaps there are other ways to work around that limitation. I would love to see an analysis of what instance trees look like in Hackage packages.
Sure you could. It needs trickery to make it appear instances are in a strict substitution ordering. So you end up with difficult-to-follow code (untested):
instance {-# OVERLAPPING #-} Functor f => (:<:) f f where
inj = id
instance {-# OVERLAPPABLE #-} (Functor f, Functor g', SubT' f g') => (:<:) f g' where
inj = inj'
class (Functor sub, Functor sup) => SubT' sub sup where
inj' :: sub a -> sup a
instance {-# OVERLAPPING #-} (Functor f, Functor g) => SubT' f (f :+: g) where
inj = Inl
instance {-# OVERLAPPABLE #-} (Functor f, Functor g, Functor h, (:<:) f h) => SubT' f (g :+: h) where
inj' = Inr . inj
(Needs UndecidableInstances.)
One of the limitations with Swiestra’s approach is that the tree is right-biased. (The instances don’t try to recurse to the left of (:+:) like (f :+: f') :+: (g :+: g'). [2008 JFP paper, middle of 2nd page of Section 4 “may fail to find an injection”.]) This technique can be expanded to cope with that. (You need two auxiliary SubT classes; and you need to choose which direction to prefer first. The instances get pretty ugly.)
You mean that feature that we do not have? And do not need?
Yes. In particular are there any ‘partially overlapping’ instances?:
instance C a Bool ...
instance C Int b ...
Allowing that really hampers GHC sharpening up its act on overlaps. And I suspect nobody actually wants it. (Or if they do, the above technique can handle it, at cost of forcing a choice.)
1 Like
(It was your comment that drew me in.) @Ericson2314 doesn’t have to use up his time responding to me. His investigations are going to come back with the costs side, in terms of effort to reorganise base and breakages liable in client code.
The benefits side will still be unevaluated. So all this discussion will happen again. (Which seems to be the pattern with CLC.)
I thought that as I’m disinterested and largely uninterested, I could prod people to articulate the why – which would both build the evaluation for benefits and motivate where to put the split(s) in base.
I admit I’m in favor of the split, but I don’t understand how juicy, interesting, breaking changes to base are a benefit that the split would provide for us. Split or no split, everybody uses base, so breaking changes are just as hard to swallow either way.
Here, on the other hand, are concrete benefits to the split:
- Decreased cognitive load.
- Project maintenance gets more tractable. Building, releasing, packaging, etc.
- The blast radius for changes of any sort is decreased — both damage done by breaking changes, as well as reasoning about the breakiness of changes in the first place.
- Improve backward compatibility.
basecurrently includes a bunch of GHC internals. Since they change with each version of GHC, they trigger a breaking change for all ofbase. If those changes were constrained to a different library altogether, GHC could stay backward compatible with olderbases.
5 Likes
Can those out-of-place “GHC internal” definitions in base be replaced using e.g. Haskell 2010 FFI declarations and some extra C modules?
Can they? I thought they were mostly Haskell code in the first place. The GHC.* module hierarchy in base.
1 Like
Yeah some of the more tricky stuff is when the RTS calls a Haskell function in base! We would arguably need cyclic backpack to really do this sort of thing right.
…yeah, why bother with abstraction when there’s only one mainstream implementation of Haskell?
As for potential solutions:
-
if it’s a RTS primitive which is calling a
baseentity, there may be two options:- if the primitive is rarely used, you could try passing in the
baseentity as an extra argument; - if it’s widely used, another option is to replace the RTS primitive with a new
basedeclaration via the FFI - it may still be easier to add that extra argument, and ignore the complaints…
- if the primitive is rarely used, you could try passing in the
-
if, for some reason, it’s some part of the fundamental workings of the RTS that make the call to
base…let’s just say “tricky” is being diplomatic:- If the offending part can be easily isolated, you may be able to replace it as well using the FFI;
- Otherwise, being lazy is probably the simplest option - log an issue report with the tracker for GHC, and make a note of the miscreant in your progress report.
…progress report? Yes: for a good example of their use, see Philip Heron’s series of reports about Rust-GCC - most other requests for updates can then be silently deleted (ideally automatically!). You can then more easily focus on this task, which will require plenty of your attention.
Those progress reports will be invaluable e.g. if this topic is still being debated in, let’s say, 2027 people can just be directed to read them to find out what happened…
@AntC2 you are most welcome to elaborate on these statements. You seem to be under an impression that CLC has an articulated opinion on the matter, but I have no clue what makes you think so.
3 Likes
I said: I thought that …, I could prod people to articulate the why
Because the impression I’m under is that the CLC does not have an articulated opinion on the matter, so is failing to give guidance where leadership is clearly needed. I have no clue what makes you think I’m seeing articulated opinions.
Those quotes from me you snipped are pretty close to links and observations. But in case you missed it …
And I see @Ericson2314 got there before me: “we get these massive threads”. And that was barely a third of the way through that thread. (Addit: And after plenty of discussion on the mailing list.) At the half way point the Committee decided/closed the issue. Then there were requests to re-open and more rounds of discussion – which came to no conclusion I can see.
So somebody thought it a good idea to have Principles that would guide future discussions. (yes please!) A thread of 63 comments so far. No conclusion I can see.
Guiding Principles are the sort of thing a Committee should be there for. The CLC’s Charter says there will be Controversial Decisions. (d’uh, well yes) And then … ?
Specifically where I came in on Decoupling base from GHC wired-in, there’s 42 replies on the original thread (which went off track) and 108 on the newly-spawned attempt to corral it. I see no criteria for how the Committee might evaluate any proposal. I’d expect the criteria to derive from the general Principles guiding assessing impacts of breaking changes vs expected benefits.
I get it that Discourse isn’t under the aegis of CLC; and that the bulk of comments on those GitHub Issues are not from Committee members. If the CLC had some guiding Principles, they could shape the discussion to tone down apocalyptic language and sweeping unquantifiable claims (like @chreekat’s “blast radius for changes of any sort is decreased” – of any sort?).
There’s been mutterings about the design of base all the time I’ve been following Haskell. I’ve seen (versions of) this wishlist plenty of times. Nearly every attempt at it has faltered – with the conspicuous exception of the bruising ‘FTP changes’.
@Ericson2314 is about to do what? Split out the GHC wired-in parts and their dependencies? Revive nomeata’s work from 2013? “Rip base in two” along its natural fracture points?
I get the impression @Ericson2314 is not doing (whatever it is) for his own entertainment. It sounds like a lot of work. Whatever results, is the Committee just going refuse point blank – because it’s too controversial or causes too much breakage? If there is to be a benefits vs cost assessment, what are the metrics for benefits? As opposed to chreekat’s ineffable claims.
Your interventions in these threads have generally been at the ultra-conservative (small-c) end:
It’s difficult to imagine slower than the pace of failing to make a decision about removing (/=) from Eq. For the sake of managing potential contributors’ expectations, what does “slower” mean? And is (whatever that means) the collective view of the Committee?
1 Like
I am rather bemused by the whole idea a set of sourcecode (the libraries) could be insulated from the very compiler they rely on. If GHC (the compiler) changes – even with a tiny bugfix release – all the object code for those libraries must get regenerated.
Of course GHC developers are meticulous to avoid side-effects of changes; and to document everything that’s changed. Nevertheless, prudent software release practice is to trust nothing and re-run the whole test suite against the re-compiled application and libraries.
Reorganising base to minimise dependencies won’t remove the chief dependency: on the compiler. So how could changes to libraries be more conservative (small-c) than changes to the compiler?
Upgrading a compilation environment is different to upgrading an application environment, in which the asset to protect is its database content and continuing business operations that query and maintain it.
I don’t agree. Changes in the compiler should only affect a tiny fraction of the language and I think that has historically been the case (at least recently), so it isn’t necessary to consider adjacent compiler versions to be completely different. Of course you should still re-run test-suites and otherwise check that the changes don’t break things, but I don’t see how that makes it impossible to split base.
4 Likes
@antc2 The splitting of base is, strictly speaking, a mere implementation detail — it doesn’t even require CLC input if we were were to go with scary sounding ghc-internal-* libraries.
I do, in fact, want the split base libraries to be intended for public consumption, so I will in fact engage the CLC on what portions of the interfaces look nice stabilized separately, but I don’t anticipate this being very controversial.
In particular, even if I somehow get the split all wrong, it is very easy to turn those libraries into ones that reexport stuff defined elsewhere. Ergo, I see very little costs in terms of stability and tying our hands in the future with this stuff.
I am rather bemused by the whole idea a set of sourcecode (the libraries) could be insulated from the very compiler they rely on.
This is funny coming from someone who likes HUGS and misses the era of a diversity of implementations and perspectives!
Code is strongly linked to the interface of a compiler, but not its implementation. What we call base–GHC coupling is still just each observing the “unabridged, private” interface of the other.
Interfaces usually have some sort of subtyping / partial order structure. The goal is to fact things nicely to rely on as little private stuff as possible.
I would encourage you to check out Rust’s core vs std split, where std is far bigger but also far more normal a library, with very few wiring-in points in either direction. Nothing I am hoping for with the split isn’t pioneered “in production” over there already.
6 Likes
Precisely because I value a diversity of features, and am tolerant of churn at the bottom of the tech stack (the compiler), I expect everything built over it will get wobbly from time to time. What I don’t expect is you could mix-and-match different versions of base with different releases of GHC. As if you could pull-push blocks in-out of the bottom of the Jenga tower, and keep the upper storeys somehow suspended from a sky-hook.
As I thought I made clear, I reverted to HUGS not because I’m an antedeluvian, not because GHC is moving too fast or is too unstable, but because GHC is moving sideways relative to where I want to go. And because I found I could understand and tweak the innards of HUGS.
1 Like
What I don’t expect is you could mix-and-match different versions of
basewith different releases of GHC.
Why not? Perhaps we need an example. Let’s pick something boring from base, hmmm, Bifoldable. Suppose we began our base split by putting just Bifoldable in base and everything else in ghc-base (being the library shipped with GHC).
Now, could we build this base with different versions of GHC? I don’t see why not. It’s just a typeclass, why wouldn’t it compile with various versions? Maybe changes in GHC would make it not compile, and we solve that the way we do for any other library: ugly CPP. But it would work.
Similarly, we could change Bifoldable (add another instance, say), release a new version of base, and expect that to work with old GHCs too. Even GHCs from before we did the release! Radical!
Now: lots of stuff in base is in the same position as Bifoldable. Some stuff isn’t, for sure, and that’s why we’re going to be stuck with some kind of ghc-base thing. But that doesn’t mean it’s somehow impossible to remove anything from base and put it in a more “normal” library.
9 Likes
Maybe one challenge in this conversation is the name? That is, we want a library of standard helper functions / data structures that everyone will want access to. I’ll call it standard-goodies. The key aspect of standard-goodies is that it is just a library – no particular cozy relationship with GHC. Of course, standard-goodies could be upgraded independently of GHC. The conversation in this thread is about taking stuff out of a library whose existence is tied to GHC and putting more into standard-goodies.
6 Likes
We already have plenty of libraries of standard helpers such as relude and other alternative preludes. They indeed can be upgraded independently of GHC and strive to provide a stable interface across several GHC versions. This is a solved problem.
2 Likes
Keep in mind the GHC base dependencies go both ways. Trivially, we do know how to write Haskell code that doesn’t depend on GHC secret interfaces – that is most Haskell! There is also the case of GHC depending on regular Haskell code, like Maybe, List, and Monad, definitions that it needs to know about for various reasons.
When we don’t hard code GHC to look up these wired-in definitions in base, but instead do something more flexible, untangling the knot gets easier.
C.f Rust’s “lang items”, some of which could be defined any library (that uses unstable features).
Thank you Richard, yes that’s slowly becoming clear. I named this thread by smurshing together names from two threads, both of which seemed to lack direction or specifics. In my head, “GHC” means the compiler and language extensions it supports. So a thread ‘Evolution of GHC’ I expect to be about language and extensions. Turns out that wasn’t what was meant.
base or Prelude are (in my head) independent of any particular compiler. The Prelude is specified in the Language Report, which is compiler-agnostic. Once upon a time we could talk of " A substantial collection of library modules that are supplied with Hugs, GHC and Nhc98."
And the criteria for base was " the Prelude and its support libraries, and a large collection of useful libraries ranging from data structures to parsing combinators and debugging utilities." IOW stuff closely aligned to compiling and supporting developers running a compiler.
Thank you to @michaelpj for (at last!) providing a concrete example. So my thrashing around in these threads can boil down to one question: why on earth is Bifoldable in base? It’s not required to be from the Language Report. (For example it’s not in Hugs’ Hierarchical Libraries.) It wasn’t in GHC base until about 5 years ago. It doesn’t provide support to compiling or debugging tasks that I can see. It seems eminently suited to be in standard-goodies.
Just move Bifoldable and its ilk out of base (after the usual three-release deprecation cycle of course). Put it on Hackage. Job done. By all means create standard-goodies as an empty shell that just imports stuff that’s been moved out of base.