Towards a prevalent alternative prelude?

[bgamari]

editorial note: This was originally a post in the HF Technical Agenda discussion.

The matter of the amending or adding to the base library has come up a fair amount in the HF Technical Agenda discussion. While such changes are of course possible, I do wonder whether this is the best solution to the problems that users feel. Afterall:

• changing base comes at a high cost to existing users
• adding to base comes at a high cost with respect to maintenance and entrenches design decisions (e.g. merging containers or vector into base would introduce significant technical challenges)
• base is coupled to a particular GHC version, meaning that users need to upgrade both the compiler and codebase simultaneously, resulting in unnecessarily painful upgrades
• users’ needs vary: base can never and will never be the right library for all users.

This makes me wonder whether an HF-blessed alternative prelude isn’t a better approach. That is, develop and promote a base alternative which:

• closely follows the structure of base (where appropriate) and is relatively conservative with introducing its own abstractions
• does away with partial functions and known laziness foot-guns (e.g. foldl)
• re-exports the libraries that nearly every Haskell program uses (e.g. containers, text, bytestring, deepseq, etc.)

Of course, I’m almost positive that such a library is available on Hackage today. However, currently alternative prelude usage is pretty rare. I would like to understand why this is the case. A number of possible reasons come to mind:

• reluctance on the part of users to deviate from what comes with the compiler
• concerns about continued maintenance (or lack thereof) of alternative preludes on Hackage
• limited benefit due to the restrictions on what a prelude can change (e.g. the typeclasses of base are essentially set in stone, unless the alternative prelude is willing to provide many instances of its own)

My sense is that with proper support from the HF and careful design such a library could supplant base as the natural starting-point for new Haskell projects Moreover, such a library could be versioned and changed independently from GHC, reducing the burden of core library evolution on the rest of the ecosystem. Of course, similar ideas have been floated many times in the past so I would very much like to hear real users’ reasons on why they might shy away from such alternative preludes.

Have you considered using an alternative prelude in the past? Did you end up doing so? If not, why?

[hgolden]

There’s a tension between “batteries included” (e.g., Python) and official Haskell (e.g., Haskell 98, Haskell 2010). My opinion tends more toward making an updated, streamlined prelude and leaving out all of the GHC-specific parts of the current base. We should aim for a stable prelude instead of one that changes as quickly as base has changed. In general, it is time to admit that Haskell is no longer just a research project, where the language developers can do whatever they want without concern for existing code. Instead, keep the leading edge stuff in optional libraries that can be versioned if necessary. Then, make an effort to ease the transition to newer versions to allow for organic growth and change.

[emilypi]

Personally I am very hyped on this idea in almost every respect. There are still a few areas that need to be fleshed out in order to make this proposal concrete:

1. Who maintains this new base-pure package? Will it be a continuation (and slight expansion) of the Core Libraries Committee work? If so, we’ll need to coordinate with folks like @chessai in order to pin down what needs to happen on the CLC side to incorporate such a package into its charter.

2. What kind of stability guarantees are we planning for such a base? I would assume that this would have the same long-term guarantee as base, but we should be explicit.

3. At what point do we intend this new safe base to be the real base that we offer to users? What constitutes “critical mass”, or, is this even an issue? We’ll have to figure out a safe time, perhaps with a major GHC release, to do the switch so that this is done with minimal breakage to the Hackage and Stacakage ecosystems.

[hgolden]

Suggestion: freeze “base” and call the new prelude by a new name. Eventually, just let “base” wither away.

[Kleidukos]

(disclaimer: I talked about this in private with Ben but it deemed necessary to post that publicly)

(TL;DR: We need to avoid pleasing every possible group of users at once, and as such the better we can figure out our target group, the better the end result will be, that is why we need to split some of the changes we would all like to see between base and a new prelude)

At the moment, what the Prelude is trying to achieve is unclear beyond the simple “default exports”. It does not particularly help neither library writers, beginners, compiler engineers, nor application developers.
As a consequence, we have seen many alternative preludes trying to address the concerns of those categories. We have the preludes that pride themselves as being an application development framework, with effect tracking, logging, database, etc), those who provide better data-structures and re-exports other packages, and so on.

As such, I think it is important to have this discussion about a prevalent alternative prelude under the prism of “Which category do we aim to serve?”. The library writers certainly may not want to have mtl and transformers re-exported! But they want head :: NonEmpty a, which is also something that would benefit the beginners! The application developers may, on the other hand, need to have a more off-the-shelf access to ReaderT and Text-first environments, as well as having Vector and HashMap coming in handy.

Now, the reader may think “Well, isn’t that what the Haskell Platform was about?”, to which I will reply that whilst the Haskell Platform did in fact provide a set of blessed libraries, they were also pre-compiled and efforts were invested to ease the burden of development on the tooling/package handling side of things.

---

To directly reply to some of Ben’s points:

[…] away with partial functions and known laziness foot-guns (e.g. foldl)

I think we can have this problem solved in base

re-exports the libraries that nearly every Haskell program uses (e.g. containers, text, bytestring, deepseq, etc.)

I think we should solve this problem outside of base.

---

Now, to answer the question Ben asked:

Have you considered using an alternative prelude in the past? Did you end up doing so? If not, why?

I do at the moment use relude by Kowainik for application development and the occasional library that is so tightly coupled to me that I include it anyway (but would rather not, because of the deps footprint).

I initially did not use one because the ones that existed before were either abandonned or did not resonate with me, or wanted to much to tell me how to do things.

So, relude did check some of the boxes that had repelled me from alternative preludes at first, and that is why I am using it this day.

[bgamari]

What kind of stability guarantees are we planning for such a base? I would assume that this would have the same long-term guarantee as base, but we should be explicit.

Yes, I would expect this library would have similar stability characteristics to base. If we had a better story for preserving compatibility while evolving typeclass hierarchies then it may be possible for such a library to have even stronger guarantees than base since we could in principle maintain could retain compatibility with a wide range of GHC releases.

[bgamari]

@hgolden

Suggestion: freeze “base” and call the new prelude by a new name. Eventually, just let “base” wither away.

The tricky part here is that base (unfortunately) currently serves a few, largely orthogonal roles:

1. it serves as the home of GHC’s IO subsystem (the event manager) which is closely tied to GHC’s runtime system (e.g. GHC.Event, GHC.Conc)

• it serves as the home of several core typeclasses which are closely tied to the compiler (e.g. GHC.Base, GHC.Functor, GHC.Enum, GHC.Generics)
• it serves as the home for wrappers to GHC primops which could in principle be implemented elsewhere (e.g. Data.IORef)
• it serves as a “dumping ground” for things that are “foundational” enough that they should be shipped with the compiler, but which otherwise have no particular relation to GHC (e.g. Data.Fixed, Data.Foldable, Data.Tuple, Text.Printf)

In my mind, base really ought to be several different packages. This is even something that has been attempted in the past. However, implementation is sadly non-trivial as due to, among other things, module cycles and orphan instance issues.

[emilypi]

Feature freezing base would be a way forward here, agreed. We’d have to figure out what such a freeze would look like. One thing I can imagine we’d want to get in before freezing is the Profunctor work and a few changes to Bifunctor. As far as big ticket items, we don’t have any that I’m aware of.

[gilmi]

I really agree with Hecate here. Honestly I don’t want much out of Prelude - I want a fast String type and not having to use OverloadedStrings (omg my phone just auto completed this??), I want partial functions to not be exported by default, I want common functions such as for exported, and i want functions like foldl fixed.

Basically - I want the defaults for beginners and hobbists be safe to use.

Large projects will inevitably develop their own bigger Prelude which is relevant for their applications. This is not a problem imo.

[rebeccaskinner]

I have two concerns with alternative preludes and alternatives to base, and they are both I think more social problems than technical ones, and solvable, but something that I think needs to be addressed mindfully.

The first problem is that, when prelude and base are the default, bringing in alternatives can make things much harder for anyone new to a codebase, and especially new to haskell itself. When every project is using the same set of common names for things that are defined with perhaps subtle differences across many different preludes, there becomes a lot of unlearning and relearning as you jump from one codebase to another. Even having a single blessed alternative still means that for every project you have to orient yourself depending on whether you are using base or base’, prelude or prelude’.

The second problem is possible dependency bloat. If there’s a single pretty widely used base’ and prelude’, and it largely just collects dependencies everyone is using already, then it might not be such a huge deal, but as you start to have competing bases and preludes gain popularity then you need to pull them, and all of their transitive dependencies in, as your program grows larger, and for modestly sized programs there is a good chance you’ll end up having all of them as dependencies.

None of these points are to say that I think this isn’t a good idea, but they are a big part of my choice to not use alternative preludes in my own personal projects, and they’re worth considering I think as we have further discussion.

[bgamari]

When every project is using the same set of common names for things that are defined with perhaps subtle differences across many different preludes, there becomes a lot of unlearning and relearning as you jump from one codebase to another.

This is to some extent true; however, it isn’t necessarily the case that names need to change under a base'. In principle modules could be re-exported or shadowed in the new package to match those historically used by base. Of course, the price one pays here is potentially confusing error messages (since, e.g., Data.List.head would not exist).

However, if the goal is to mostly reexport existing libraries whole-sale and perhaps hide a dubious few functions then this may be an acceptable tradeoff.

The second problem is possible dependency bloat. …

Quite right. This is why I think it would be best if alternative preludes nearly exclusively consisted of re-exports of things provided by other packages. This way the user is never forced to import such a package; it’s merely a convenient collection of things defined elsewhere.

[edmundnoble]

I’d love a blessed alternative prelude which re-exports the things which are not included with Prelude that we import all the time and have a low risk of causing name conflicts. I have similar reasoning to Rebecca Skinner for not currently using an alternative prelude. Being able to move code (and myself) around safely between modules and projects without drastic context shifts is extremely important to me.

I tried to use papa before for this and it was great except that it didn’t include undefined and error, which breaks my workflow (insert XKCD comic here)

papa doesn’t look like it’s maintained now though, unfortunately.

---

In particular, I’d like access to:

• Data.Foldable
• Data.Maybe
• Data.Monoid
• Data.Semigroup
• Data.Void
• Control.Applicative
• Control.DeepSeq
• Control.Exception
• Control.Monad

Just to name a few.

In particular, Control.DeepSeq and Control.Exception.evaluate are so important for using nonstrict evaluation, perhaps making them more available would help people in this area. Maybe it would also be nice to have Data.Function.on and Data.Tuple.swap.

Not sure about partial functions. Partial functions are included with lots of libraries, from vector to lens, and I think on balance they usually make our code better.

foldl I’m similarly unsure about. Perhaps we can skirt around this by ensuring foldl' is accessible from this hypothetical prelude?

[and_pete]

not having to use OverloadedStrings (omg my phone just auto completed this??)

Just wait until you’re typing “you shouldn’t un…” and it autocompletes with unsafePerformIO.

Suggestion: freeze “base” and call the new prelude by a new name. Eventually, just let “base” wither away.

On-topic: I see that chessai has the package name std reserved and seemingly unused on Hackage (see here).

I think rather than basing off of base the name of any alternative Prelude (e.g. base-pure or something) that necessitates new users being aware of the original base, that a clean start with something like std (and import Std) could be had with less baggage and then progressively adopted over time.

My main concern here is a “blessed” alternate prelude won’t go far enough in addressing historical baggage and so even with this blessing it will just become an example of the “There are now 15 competing standards” xkcd comic (with base/Prelude still being what people default to far into the future).

I’d like for a future where if I pull in one of the current application development libraries & prelude alternatives, that I’d really now just be pulling in that particular framework’s opinionated app dev-oriented types and functions (e.g. a custom monad, logging, and re-exports of process context handling, probably unliftio, etc.). As most of the rest would be already provided by the HF’s new blessed prelude. And that “rest” is the type of stuff that tends to overlap among the excellent common prelude alternatives now like (but not limited to) relude, protolude and rio.

Anyway, thanks to everyone on the HF/HFWG for suggesting a change in this area

[JonathanLorimer]

I like the idea. I personally use Relude for this.

[sekun]

Not sure about having yet-another-alternate-prelude, instead of just replacing it with safer, and more performant standard library.

Since:

1. Setting up alternate prelude introduces more friction on getting started. Be it a project or just experimenting Haskell. This is especially bad for people who want to get their feet wet with Haskell.
2. Beginners have to deal with Cabal/Stack too early even if their main focus should be familiarizing themselves with Haskell. Getting started with Haskell is already daunting enough, and forcing them to deal with build tools this early might deter people away from Haskell. Makes it harder to suggest them using an alt prelude with less gotchas.
3. More overhead because now people have to be familiar with two or more preludes. Especially for those that just want a prelude with the aforementioned traits.

Some may argue that it is simpler, like with String. But I’m not sure how I feel about that if it’s at the expense for more headache in the near future. Like oh actually String is a linked list so common string operations aren’t performant at all.

Just some opinions from my limited experience.

[evanborden]

As with any common infrastructure, opinions are abound about what is wrong with it. base has plenty of historic baggage, yet many issues have been addressed. Applicative Monad, FTP, Semigroup Monoid, Monad of No Return. I’ve shipped hundreds of thousands of lines of code to production through many transitions and GHC’s type system has made what would be harrowing in another language actually quite pleasant. The community is forgiving of change if the direction is an improvement. There are clear and well established issues with base (String, partial functions, etc). The problems are known. The community is encouraging of fixing them. Why create another standard when we can major version bump and rip off the band-aid?

[mb64]

I don’t use an alternative prelude, but if there were an “officially blessed” more comprehensive prelude, I would not hesitate to use it.

The single most important thing for me in a new prelude is to make import lists useful when reading a module. To me, that largely means they should summarize what important datatypes are used in a module. For instance, I would love to look at an import list and see, “this module uses the State monad, and IntMaps, and these project local things.” Maybe the code also uses mapMaybe and asum and second, but that’s not the most important thing to know about it: the most useful thing to see is that it works with State and IntMaps.

That’s what I like about Edmund’s list of modules – for the most part, they provide functions rather than datastructures, and give little to no info about the big picture contents of the module. On the other hand, I would not want all the monad transformers in Prelude, because they give important clues about how the functions in the module will operate.

Of course, that’s just what I think would make the import section most useful to me, so I’d be interested to hear what others think a useful import section looks like.

[Elvecent]

I think Haskell has to have a bloated Prelude, not necessarily abandoning the existing one. Both beginners and experienced haskellers should benefit from not having to deal with package lists and a ton of imports every time they want to do something basic. One technical concern I have is that Stack to this day doesn’t really support mixins (so as to have an implicit alternative Prelude module).

Incidentally, I really like the way IHP has multiple preludes for different parts of the project.. Maybe having a lot of preludes is the right way in general?

[gilmi]

I noticed that my previous 1am comment didn’t explicitly answer the question:

I mostly use base. I’ve tried using relude a few times and while I think it has better defaults, I often found myself wanting more/other things when workingon different projects.

As my project grow I realized that my process for creating a new module is to copy the head of an existing module and change the module name. This is because often there is a set of common functions that I import and use throughout a project which are tedious to add by hand, and they may change from project to project.

For example, in some project I’d really want to have throwError everywhere, for another project view and modify from lens might be really common. For one project I might want a special trace function that uses pretty-simple, for another project maybe I want one that uses Pretty.

Different projects have different defaults, so I often have a local Utils.hs that grows organically and which caters for that project. It makes more sense to me to be able to control what a project needs by default, and this has a few advantages in terms of ergonomics:

1. Smaller import lists: makes it possible to focus on what’s actually unique for this module
2. GHC emits a warning for unused imports, and it’s tedious to remove and add imports for stuff I use frequently
3. Toolings aren’t as good for cross module stuff atm, but jump to definition locally works for me
4. Everything is in one place

So this is why I believe this approach is better for non trivial projects. We can’t satisfy everyone, I myself even have different needs for different projects! What we can do is prevent people from shooting themselves in the foot and crash their systems, leak memory and ultimately share their bad experience with Haskell due to legacy defaults that most agree are problematic.

Just my two cents.

[winterland1989]

As an industry user who rolls our own IO solution based on libuv, we’d like to see a minimal base. The Handle stuff in base feels like 90’s comparing to rust or golang’s buffered IO with utf8 default codec. The overall package structure change should make pushing progress easier rather than more difficult.