The Haskell Unfolder Episode 31: nothunks

[TeofilC]

Indeed as @tomjaguarpaw says my th-deepstrict library already implements this.

@edsko

You’d have to think about type parameters

Currently th-deepstrict only works with fully concrete types (Maybe Int rather than Maybe a), but I’m planning on eventually implementing something like what you sketch. It’s a bit more work to implement, and in practice I haven’t found myself needing it, so it might be a while before I get to it.

@edsko

It’s also not entirely clear to me how it would find out (in your example) that UTCTime is not deep strict;

We can access this information through TemplateHaskell’s reify interface. It can show you the definition of any datatype in your dependency tree, even if the constructors weren’t exported.
We don’t have to specify the strictness properties of any types at all (though some datatype might need manual overrides because Array# etc are not deep strict), and things will just be inferred. Eg, even builtin things like Int don’t need to be specified to be deep strict, we can see that it’s deep strict because it has one field with an unlifted type.

I’m happy to answer any questions about it

[edsko]

Sounds like I should take a look Perhaps even worth an episode in its own right

[tomjaguarpaw]

@edsko

Perhaps even worth an episode in its own right

I would say so. Very useful work from @TeofilC!

[atravers]

Is there also an explanation of the advantages of nothunks over options like -XStrict?

[kosmikus]

I think static guarantees are nice, and if they’re correct by construction, that’s even nicer.

I have a number of reasons though why I think that both th-deepstrict and nothunks have their place, rather than th-deepstrict being strictly (sorry) superior:

1. It seems th-deepstrict is only useful in the case that I really want a value being completely strict. Personally, I think that’s unrealistic in larger settings. I like about nothunks that I can choose exactly what kind of invariant I want to apply. I see no reason that th-deepstrict couldn’t allow this too, and perhaps it’s actually possible to do so by using assertDeepStrictWith, but I’m not sure.

2. There’s still a difference between a type being (deeply) strict and a value having no thunks. The latter can be true even if the former isn’t. And in order to plausibly establish the former (without reintroducing a need for testing via something like nothunks), the only plausible way may be to apply a potentially very costly deepseq in various places, especially if we do not control the data types in question because they come from other packages.

Regarding the requested comparison with -XStrict:
The Strict and StrictData language extensions are primarily switching defaults. I find Strict too invasive and would never use it. StrictData is in principle fine, but I don’t think it really solves the problem as it only switches the syntactic default. You have to think about the exact invariants you want to hold for your datatypes, and at some point you might want to use additional tools in order to independently check you got them right.

[TeofilC]

Yes I definitely agree these approaches are complimentary.

@kosmikus

It seems th-deepstrict is only useful in the case that I really want a value being completely strict. Personally, I think that’s unrealistic in larger settings.

You can use th-deepstrict in a scenario where you want some lazy parts to your datatype. What you can do is instead of using assertDeepStrict, you can dump the output from isDeepStrict to a file and set up a golden/snapshot type test. Then you can keep whatever laziness you desire, but get a CI failure if you accidentally add an unintendedly lazy field (eg, by using regular lazy Maybe). This is also helpful when you want to make a large datatype less lazy over time.

You can also use the *With functions to override the inferred strictness of some datatypes.

[darkxero]

So, if I shouldn’t use nothunks in production, what should I use?
How should I monitor memory?

If a memory leak crashes the server, should I pepper everywhere with nothunks and CPP flags, deploy it in prod and hope I get a hint?

If the map from the video was actually fully evaluated to dump it and query it, then the thunk chasing wouldn’t matter.
How do I isolate the thunks that matter from the ones that don’t, for the same type, in different functions, during runtime in production?
Can I annotate functions with debug symbols so that they remain even when inlined so I can go make strict just the things I want?

While it’s a good thing to make invalid laziness unrepresentable, only experience lets you foresee which function will benefit from strict types. You can’t go around deepseqing every type just in case the previous laziness becomes invalid as you reuse the type for a new function. And even then you can deepseq an infinite stream and explode anyways.

[tomjaguarpaw]

Regarding dealing with space leaks, my advice is the following:

1. Make invalid laziness unrepresentable. That is, design your types to be free of space leaks in the first place. In the same way you simply wouldn’t use strings "TRUE" and "FALSE" to represent booleans, don’t use data MyPair = Pair Int Int to represent a pair of fixed-precision integers. When evaluated it’s not a pair of evaluated fixed-precision integers! It’s a pair of (either a fixed-precision integer or thunk (potential space leak)). Instead, use data MyPair = Pair !Int !Int.

   Similarly, don’t use data MyPair2 = Pair !Int !(Maybe Int). There’s a thunk (potential space leak) hiding in that Maybe. Instead use data MyPair2 = Pair !Int !(Strict (Maybe Int)). (See the strict-wrapper library.)

2. Use th-deepstrict to confirm that the data types that you are defining don’t hide space leaks.

3. Only use the space-leak-free versions of various library functions. This is a bit more awkward, because you have to know which to avoid. For example, you should only ever use foldl' not foldl, Data.IORef.modifyIORef' not Data.IORef.modifyIORef, and Control.Monad.Trans.State.modify' not Control.Monad.Trans.State.modify.

   (Maybe one day this knowledge will be encoded into stan or some other static analyser, so everyone doesn’t have to just remember it.)

4. If you come across a space leak nonethless, use GHC’s heap profiler with retainer profiling. That should give you a good idea of which data type the space leak occurs in. Then, if it’s your data type, you can go back to 1 to fix it, perhaps using nothunks to help diagnose. Once fixed use th-deepstrict to ensure that the data type doesn’t regress. On the other hand, if the space leak is in a library you’re using then it’s more tricky. I guess file a bug report upstream, for example my patch to megaparsec.

@darkxero

While it’s a good thing to make invalid laziness unrepresentable, only experience lets you foresee which function will benefit from strict types. You can’t go around deepseqing every type just in case the previous laziness becomes invalid as you reuse the type for a new function. And even then you can deepseq an infinite stream and explode anyways.

I don’t think I really follow this. It’s not a question of “functions benefitting from correct types”. It’s a question of enforcing invariants on your data types (as @kosmikus explains in the linked video). If there’s no need for laziness in your data type then enforce its absence by making invalid laziness unrepresentable and it will be space leak free! The point of making invalid laziness unrepresentable is that deepseq becomes simple the same as seq. There is no longer and deep laziness to seq! deepseq is a massive anti-pattern. If you find yourself using it then something has likely gone terribly wrong. (For a discussion around the boundary between legitimate deepseq use and anti-pattern use, see Deepseq versus "make invalid laziness unrepresentable").

[darkxero]

@tomjaguarpaw

I don’t think I really follow this. It’s not a question of “functions benefitting from correct types”. It’s a question of enforcing invariants on your data types (as @kosmikus explains in the linked video). If there’s no need for laziness in your data type then enforce its absence by making invalid laziness unrepresentable and it will be space leak free!

What I’m saying is that invariants may change.
Imagine I have:

data Foo = Foo Int Int

cond x (Foo a b) = if x then a else b

f x = cond x $ Foo someThing veryExpensiveComputation

Now imagine I forget this amid a sea of functions.
I keep working, and in a few months, I introduce the problem in the video, with Map.
There’s a Map of Foos somewhere.
If I make Foo strict to fix the thunks in the Map problem, I’m degrading the system somewhere else. I introduced a regression.
It’s the same as enabling -XStrict to just fix the leaks: someplace somewhere really needed laziness.
You could say that I could assert laziness and seqness everywhere. But the cases where you use both, and seq on demand, means you either store surprises for the future or have to duplicate your business types into Strict and Lazy. With all the duplication and conversion between one or the other that the situation entails.

@tomjaguarpaw

If you come across a space leak nonethless, use GHC’s heap profiler with retainer profiling. That should give you a good idea of which data type the space leak occurs in. Then, if it’s your data type, you can go back to 1 to fix it, perhaps using nothunks to help diagnose. Once fixed use th-deepstrict to ensure that the data type doesn’t regress. On the other hand, if the space leak is in a library you’re using then it’s more tricky. I guess file a bug report upstream, for example my patch to megaparsec.

Rather than duplicating types and making and maintaining strict logic and lazy logic, I figure laziness as a default, and using profiling on production environments to seek out what places to seq and assert can go a long way. If you have a good profiler and tools to debug it.

[tomjaguarpaw]

@darkxero

What I’m saying is that invariants may change.

Oh yes, absolutely! If you’re using laziness in an essential way in your data type then you can indeed not change that and expect everything to work fine. I strongly suggest to not use laziness like that though. It’s a cute trick that can end up doing more harm than good.

@darkxero

I figure laziness as a default, and using profiling on production environments to seek out what places to seq and assert can go a long way.

I figure it can’t. I guess only time and experience will bear out which prediction is correct.

[MangoIV]

I’ve had something strange happening to me when using Seq. the code goes something like this:
\x -> x `deepSeq` maybe () (error . show) (unsafeNoThunks x) `seq` ….

And this failed telling me that x (a Seq) was a thunk. (So no other context, just the Seq itself)

Edsko said there was some problem with how Seq works internally, does anybody know more?

Thanks in advance.

[tomjaguarpaw]

When you say “Seq” do you mean Data.Sequence.Seq? It’s a bit unclear when you’re also talking about Prelude.seq!

[MangoIV]

Yes, I do! (I was actually pretty close to putting a note that the difference in capitalisation was no accident '^^

[tomjaguarpaw]

Can you share some code that exhibits this behaviour? I couldn’t replicate it with a quick experiment.

[MangoIV]

Probably not easily, I might have gotten rid of the change again ^^’

[MangoIV]

Maybe it was just a fluke but it kept me thinking so I thought I might just ask.

[tbidne]

For what it’s worth, Seq's performance depends crucially on laziness. From the paper:

Although the structure makes essential use of laziness, it is also suitable for strict languages that provide a lazy evaluation primitive.

FingerTree.pdf

Though I’ve never dug into Seq's internals, and I’ve also never noticed problems mixing Seq with other strict code .

[edsko]

@darkxero

So, if I shouldn’t use nothunks in production, what should I use?
How should I monitor memory?

If a memory leak crashes the server, should I pepper everywhere with nothunks and CPP flags, deploy it in prod and hope I get a hint?

For monitoring memory, there are tools such as ekg: Remote monitoring of processes that can help. As for “deploy in production and hope you get a hint”: if all else fails, then I guess that’s an option, but obviously not ideal. But this is no different to any other kind of bug. You don’t add assertions or print statements to your production code and hope to get a hint of where the bug is; instead, you have a test suite for this purpose. This is the main way in which I recommend the use of nothunks: use (state-based) property based testing so that you get minimal counter-examples of (a series of) API commands that introduce the unwanted thunks.

@darkxero

If the map from the video was actually fully evaluated to dump it and query it, then the thunk chasing wouldn’t matter.
How do I isolate the thunks that matter from the ones that don’t, for the same type, in different functions, during runtime in production?

I absolutely agree with you that “making everything strict everywhere” is not the way to go, but it’s a tricky thing to balance, no matter whether you use a dynamic approach with nothunks or a static approach with th-deepstrict. For fields for which you have made a conscious decision that they really should be strict, you can exclude them from checks with either approach, but if you haven’t, and there is a field somewhere that has an unexpected thunk, I think with either approach you will need to start annotating, potentially quite a bit.

[atravers]

@edsko

I absolutely agree with you that “making everything strict everywhere” is not the way to go, but it’s a tricky thing to balance, no matter whether you use a dynamic approach with nothunks or a static approach with th-deepstrict.

A simple fact:

• if a function has N (lifted) parameters, there are 2^N possible combinations of strictness annotations for those parameters;

(and in Haskell, data constructors are also functions e.g. Just :: a -> Maybe a). So merely “banging” constructor fields isn’t enough; arguments of functions may also have to be “banged” too. Now if this “bang-a-thon” is tiring for Haskell professionals to the point of needing bespoke support libraries, spare a thought for Haskell beginners…

---

…meanwhile, back in 2003:

(page 10 of 12)

Our goal is to allow a programmer to write programs in a
lazy language without having to use strictness annotations in order to obtain good performance.

Optimistic evaluation: an adaptive evaluation strategy for non-strict programs..

What a great idea - we build these nifty things called computers to do the boring stuff, so we’ll just have them deal with this boring stuff as well! Then Haskell beginners and professionals alike can attend to their actual tasks with less distractions: nice. There’s already a seven-year transition plan for Agda to be remade into Haskell’ dependent types to be implanted into Haskell, so how about a similar transition arrangement for the (Glasgow) Haskell implementation?

[darkxero]

@edsko

For monitoring memory, there are tools such as ekg: Remote monitoring of processes that can help.

Very cool.

@edsko

But this is no different to any other kind of bug. You don’t add assertions or print statements to your production code and hope to get a hint of where the bug is

O-of course, I… I would never!

@edsko

This is the main way in which I recommend the use of nothunks: use (state-based) property based testing so that you get minimal counter-examples of (a series of) API commands that introduce the unwanted thunks.

Yes! So the plan is to monitor possible hot spots with ekg and then inspect the offending procedure in dev.

(Before caving in and redeploying with debug in prod because we obviously quickchecked the wrong thing.)