Pre-CLC Proposal: Integration of `deepseq` into `base` in order to provide Strict variants of mutable types

Hi everyone, I would like to gather some opinion about incorporating deepseq into base or ghc-internals in order to provide strict variants of mutable data types that usually trip people writing production servers.

For context, this is a follow-up to [ANN][RFC] strict-mutable-base - strict variants of mutable data types from base : Production woes due to space leaks with MVar.

The main motivator is to provide such strict variants to library authors, whose choices in terms of impact application developers in a transitive manner that is not easily fixable by “just” swapping implementations.

As such, that’s why strict-mutable-base is the foundation for effectful. The idea is not to provide shiny new things for end-users, but more reliable and more predictable foundations in terms of run-time characteristics.

If we can have Deepseq into the foundational compiler libraries, we can provide strict IORef, MVar and Chan by default, which lowers the barrier of adoption significantly. I don’t expect a massive usage throughout end-users, but effect systems and other libraries providing concurrent processing will undoubtedly benefit from them being first-class amongst core libraries.

What do you think?

3 Likes

By the way, the link was garbled. It should be [ANN][RFC] strict-mutable-base - strict variants of mutable data types from base


deepseq is an anti-pattern, an extremely expensive and blunt hammer, generally costing O(n) time to do what should take O(1) time. It is as much as a footgun as unexpected laziness.

I have explained the problem of and solution to space leaks due to unexpected laziness in more detail in these articles:

My library strict-wrapper paired with @TeofilC’s th-deepstrict should be all that is required to design algebraic data types without hidden laziness and therefore without space leaks.

Beyond algebraic data types, the suggestion to add versions of IORef, MVar etc. that don’t hide laziness is completely in line with that strategy, but that does not require deepseq.

9 Likes

Yes, in fact I could go with a CLC proposal to upstream the strict mutable types without having to worry about integrating Deepseq into base. Thanks for the encouragement!

4 Likes

I’m strongly in support of that!

3 Likes

In tune with @tomjaguarpaw, deepseq is a cure for a disease that a well-written application should not have. It’s good where it is now, every datatype should implement it, I should never need to use it.

Not in tune with him, I think the only way to properly fix strictness would be to figure out a way to put it on the type-level and everything short of that is unsatisfying. That is:

  • “strict wrapper variants” Ă  la containers I find to be both redundant at implementation level (since most functions don’t insert data) and confusing to end users (who likely have no idea what “strict” even means in that context);

  • “strict” Maybes, Eithers, and tuples are about as confusing, but also need extra scaffolding to be converted to/from their standard counterparts, which makes them even more of a hassle to use.

For things as they are right now I find the easiest way to think about laziness is to assume that a in every Datatype a is always lazy, so it’s my responsibility to evaluate it if I’m storing it for a long time. Everything else can be declared on the spot when I need it.

1 Like

I have strong opinions with respect to strictness as not just a type but a special kind of type. My own toy experiments involve understanding different evaluation strategies (strict, copy, lazy) as such, being a subset of what I call ‘displacement’ types related to the notion of linguistic displacement. There are qualities that we normally ignore as being way-above / way-below (as in infimum and supremum) due to either being compiled away as language and types or to as machine implementation. Long story to follow up on.

1 Like

strict-wrapper

Cool! I love all the examples and documentation too. I think though you want to specify that it needs a fairly recent base to work (> 4.9), due to TypeError etc https://hackage.haskell.org/package/ghc-internal-9.1201.0/docs/src/GHC.Internal.TypeError.html#TypeError

1 Like

That’s from GHC 8.0.1 (May 2016): Base package - HaskellWiki

1 Like