# State monad - memory exhausted

**URL:** <https://discourse.haskell.org/t/state-monad-memory-exhausted/8602>\
**Category:** Uncategorized\
**Created:** [January 19, 2024, 8:46pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602 "2024-01-19T20:46:37Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![ajbarber](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ajbarber/32/2648_2.png) [@ajbarber](https://discourse.haskell.org/u/ajbarber)\
**Post date:** [January 19, 2024, 8:46pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/1 "2024-01-19T20:46:37Z")

</div>

Doing Advent of Code this year I experienced memory exhaustion on my machine running some code where the accumulated output of a Writer monad appears out of scope, yet does not appear to be freed. [_edit - this hypothesis is incorrect, read replies below_]

A simple reproduction of the issue is as follows, bearing in mind this code does not make sense other than to demonstrate the memory usage issue I experienced:

```haskell
import Control.Monad.State.Strict
import Control.Monad.Trans.Writer.CPS
...

-- Enumerates attempts at brute forcing the answer. 
-- The output of the Writer is not used here. 
usesUpMemory :: State String Int
usesUpMemory = loopM (\i -> do
                 (a,_) <- runWriterT test
                 pure $ if a then Right i
                        else Left (i+1)) 1

test :: WriterT [Bool] (State String) Bool
test = do
  -- real version does some logic here and returns a True when conditions
  -- are correct, otherwise False. It also accumulates some characteristics of the solution attempt into the Writer monad.
  tell (replicate 1000 True)
  pure False

```

Question is, why does `usesUpMemory` behave in this way, shouldn’t it just continue forever using a constant amount of memory, or have I missed something here? Compiled with `ghc -O3`

---

<div class="post-metadata">

**Author:** ![tomjaguarpaw](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/tomjaguarpaw/32/1230_2.png) [@tomjaguarpaw](https://discourse.haskell.org/u/tomjaguarpaw)\
**Post date:** [January 19, 2024, 8:55pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/2 "2024-01-19T20:55:24Z")

</div>

What happens if you change it to `i `seq` Left (i+1)`?

---

<div class="post-metadata">

**Author:** ![ajbarber](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ajbarber/32/2648_2.png) [@ajbarber](https://discourse.haskell.org/u/ajbarber)\
**Post date:** [January 19, 2024, 9:23pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/3 "2024-01-19T21:23:07Z")

</div>

Thanks, the thunks of unevaluated `Left` building up I’m presuming, I did read about this once. Are there some ground rules for avoiding this situtation? Perhaps hobbyists like me ought to just turn off laziness entirely with that extension…

---

<div class="post-metadata">

**Author:** ![Bodigrim](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bodigrim/32/1457_2.png) [@Bodigrim](https://discourse.haskell.org/u/Bodigrim)\
**Post date:** [January 19, 2024, 9:28pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/4 "2024-01-19T21:28:29Z")

</div>

I largely recommend avoiding any flavours of `Writer` altogether. Use `Control.Monad.State.Strict` (as you do) with a record with strict fields as a state.

---

<div class="post-metadata">

**Author:** ![lorenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/lorenzo/32/1294_2.png) [@lorenzo](https://discourse.haskell.org/u/lorenzo)\
**Post date:** [January 19, 2024, 9:29pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/5 "2024-01-19T21:29:47Z")

</div>

A good rule of thumb is that, whenever you are accumulating something, use your own data type in combination with the `StrictData` extension. I’ve been yet to experience a space leak after following this simple rule.

---

<div class="post-metadata">

**Author:** ![tomjaguarpaw](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/tomjaguarpaw/32/1230_2.png) [@tomjaguarpaw](https://discourse.haskell.org/u/tomjaguarpaw)\
**Post date:** [January 19, 2024, 9:36pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/6 "2024-01-19T21:36:50Z")

</div>

> [@ajbarber](#):
>
> Thanks, the thunks of unevaluated `Left` building up I’m presuming

Specifically, the `1 + 1 + 1 + ... + 1` thunk that you return inside the `Left`.

> Are there some ground rules for avoiding this situtation?

I’ve written an article about this: [Make Invalid Laziness Unrepresentable](http://h2.jaguarpaw.co.uk/posts/make-invalid-laziness-unrepresentable/). Its advice is also in line with [what @lorenzo is suggesting](http://discourse.haskell.org/t/state-monad-memory-exhausted/8602/5). However, that won’t help you here, because `loopM` is just a risky function. You just have to know how to use it correctly, by always forcing the payload to the `Left` (or the argument). That is, it has a dangerous sharp edge, just like `foldl` (i.e. the lazy version, to which `foldl'` is the strict analogue) and `Control.Monad.Trans.State.Strict.modify` (as opposed the strict version, `modify'`).

There should probably be a strict version `loopM'` which forces the state every iteration.

> Perhaps hobbyists like me ought to just turn off laziness entirely with that extension…

`Strict` would have worked here, but that is a very blunt hammer …

---

<div class="post-metadata">

**Author:** ![jaror](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/jaror/32/3271_2.png) [@jaror](https://discourse.haskell.org/u/jaror)\
**Post date:** [January 19, 2024, 10:43pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/7 "2024-01-19T22:43:18Z")

</div>

> [@ajbarber](#):
>
> `ghc -O3`

FYI, GHC -O2 is the maximum optimization level. All higher numbers are currently treated the same as -O2.

> [@tomjaguarpaw](#):
>
> `i `seq` Left (i+1)`

I’d write that as `Left $ i + 1`.

* * *

Also, I’d write:

```haskell
  pure $! if a then Right i else Left $! i + 1

```

That makes the Core look better (Edit: only for -O1. -O2 does get the good core without that extra `$!`), but I don’t know if that was actually causing a leak.

---

<div class="post-metadata">

**Author:** ![Ambrose](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ambrose/32/5672_2.png) [@Ambrose](https://discourse.haskell.org/u/Ambrose)\
**Post date:** [January 19, 2024, 10:57pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/8 "2024-01-19T22:57:26Z")

</div>

> [@Bodigrim](#):
>
> I largely recommend avoiding any flavours of `Writer` altogether.

I thought CPS `Writer` basically _was_ `State` under the hood? 🤔

---

<div class="post-metadata">

**Author:** ![jaror](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/jaror/32/3271_2.png) [@jaror](https://discourse.haskell.org/u/jaror)\
**Post date:** [January 19, 2024, 11:12pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/9 "2024-01-19T23:12:58Z")

</div>

If `loopM` would be defined like this:

```haskell
loopM :: Monad m => (t -> m (Either t b)) -> t -> m b
loopM act = go where
  go x = do
    res <- act x
    case res of
        Left x -> go x
        Right v -> pure v

```

Then you’d need no extra strictness annotations at all. And this improves the performance further by allowing the `Int` to be unboxed, so no allocations happen during the loop (except for the allocations in `test`). And even crazier, if you don’t add a `NOINLINE test` pragma then GHC sees that there is an infinite loop and removes the branching altogether.

---

<div class="post-metadata">

**Author:** ![atravers](https://avatars.discourse-cdn.com/v4/letter/a/45deac/32.png) [@atravers](https://discourse.haskell.org/u/atravers)\
**Post date:** [January 20, 2024, 2:34am UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/10 "2024-01-20T02:34:30Z")

</div>

You mean there’s a substantial difference between using a local definition (`go`) and how `loopM` is currently defined:

> [@](#):
>
> ```haskell
> loopM act x = do
> res <- act x
> case res of
> Left x -> loopM act x
> Right v -> pure v    
> 
> ```
> 
> [https://hackage.haskell.org/package/extra-1.7.14/docs/src/Control.Monad.Extra.html#loopM](https://hackage.haskell.org/package/extra-1.7.14/docs/src/Control.Monad.Extra.html#loopM)

…or are you referring to some other definition?

---

<div class="post-metadata">

**Author:** ![tomjaguarpaw](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/tomjaguarpaw/32/1230_2.png) [@tomjaguarpaw](https://discourse.haskell.org/u/tomjaguarpaw)\
**Post date:** [January 20, 2024, 8:00am UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/11 "2024-01-20T08:00:05Z")

</div>

> [@jaror](#):
>
> I’d write that as `Left $ i + 1`.

Ah yes, that would be better.

> Also, I’d write:
> 
> ```haskell
> pure $! if a then Right i else Left $! i + 1
> 
> ```

Indeed, as you found, that will make no difference. `loopM` always forces the `Either` to know whether it should go for another iteration, so there’s no benefit in forcing it before `return`.

> If `loopM` would be defined like this …Then you’d need no extra strictness annotations at all.

What forces the argument to `act`? The following doesn’t crash under either version.

```haskell
loopM (\_ -> pure (Left undefined)) ()

```

Do you mean that from strictness analysis (and knowledge of the definition of `+`) GHC can infer that the argument to `Left` would always be better off evaluated (and can never crash when evaluated), so it specialises it for that case?

Tangentially, I never understood why GHC can’t rewrite functions to `go` form by itself.

---

<div class="post-metadata">

**Author:** ![ajbarber](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ajbarber/32/2648_2.png) [@ajbarber](https://discourse.haskell.org/u/ajbarber)\
**Post date:** [January 20, 2024, 10:23am UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/12 "2024-01-20T10:23:06Z")

</div>

Is it statically known that thunks _can_ build up infinitely in a given block? A compiler warning would have a huge impact would it not, you just find it and fix it, by `!` forcing or using a strict record field etc. as suggested here.

---

<div class="post-metadata">

**Author:** ![jaror](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/jaror/32/3271_2.png) [@jaror](https://discourse.haskell.org/u/jaror)\
**Post date:** [January 20, 2024, 11:17am UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/13 "2024-01-20T11:17:31Z")

</div>

> [@tomjaguarpaw](#):
>
> Indeed, as you found, that will make no difference. `loopM` always forces the `Either` to know whether it should go for another iteration, so there’s no benefit in forcing it before `return`.

It doesn’t make a difference for the leak, but it does make a difference in terms of total allocations. If you use `$!` in both places then you get this STG (this is all with -O1):

```haskell
usesUpMemory2
  :: Int -> [Char] -> Identity (Either Int Int, [Char]) =
    \r [i eta]
        case usesUpMemory_m1 eta of { -- m1 is essentially 'test' in the source
        (,) a1 s' ->
        case a1 of {
        (,) a _ ->
        case a of {
          False ->
              case i of {
              I# x ->
              case +# [x 1#] of sat {
              __DEFAULT ->
              let { sat :: Int = I#! [sat]; } in
              let { sat :: Either Int Int = Left! [sat]; } in (,) [sat s'];
              };
              };
          True ->
              let { sat :: Either Int Int = Right! [i]; } in (,) [sat s'];
        };};};

```

This is nicely flat and only has two allocations for the `I#` and `Left` constructors. But if you leave out that outermost `$!` then you get:

```haskell
usesUpMemory2
  :: Int -> [Char] -> Identity (Either Int Int, [Char]) =
    \r [i eta]
        case usesUpMemory_m1 eta of {
        (,) a1 s' ->
        case a1 of {
        (,) a _ ->
        let {
          sat :: Either Int Int =
              \u []
                  case a of {
                    False ->
                        case i of {
                        I# x ->
                        case +# [x 1#] of sat {
                        __DEFAULT -> let { sat :: Int = I#! [sat]; } in Left [sat];
                        };
                        };
                    True -> Right [i];
                  };
        } in (,) [sat s'];
        };};

```

Which creates an updateable closure (the `\u []`) a.k.a. a thunk. That thunk is put into the `(,)` that is used by the `Writer`. Of course this thunk will be forced immediately in the next generation, but I can’t imagine that’s good for performance.

> [@tomjaguarpaw](#):
>
> Do you mean that from strictness analysis (and knowledge of the definition of `+`) GHC can infer that the argument to `Left` would always be better off evaluated (and can never crash when evaluated), so it specialises it for that case?

Yes

> [@tomjaguarpaw](#):
>
> Tangentially, I never understood why GHC can’t rewrite functions to `go` form by itself.

You can enable it manually with [`-fstatic-argument-transformation`](https://downloads.haskell.org/ghc/9.8.1/docs/users_guide/using-optimisation.html#ghc-flag--fstatic-argument-transformation) (although for some reason it doesn’t seem to work in this case), but it is not always an optimisation. If the function is not inlined then it will a have to allocate a closure for that `go` function. @sgraf is working on improving it though. I believe the latest idea was to only apply this transformation if it makes it possible to inline the function.

> [@atravers](#):
>
> You mean there’s a substantial difference between using a local definition (`go`) and how `loopM` is currently defined

Yes. Compare the above listings with the result of using the better `foldM` that uses the recursive helper function `go` (to get this result you have to either use one of the two `$!`'s, it doesn’t matter which, or you have to compile with `-O2`):

```haskell
usesUpMemory_$s$wgo :: Int# -> [Char] -> (# Int, [Char] #) =
    \r [sc eta]
        case m1 eta of {
        (,) a1 s' ->
        case a1 of {
        (,) a _ ->
        case a of {
          False ->
              case +# [sc 1#] of sat {
              __DEFAULT -> usesUpMemory_$s$wgo sat s';
              };
          True -> let { sat :: Int = I#! [sc]; } in (#,#) [sat s'];
        };
        };
        };

```

Now GHC can see the loop instead of only a single iteration and it can remove the `Either` completely and unbox the `Int`. Now there is only one allocation left, but that is in the base case so that only happens once.

> [@ajbarber](#):
>
> Is it statically known that thunks _can_ build up infinitely in a given block? A compiler warning would have a huge impact would it not, you just find it and fix it, by `!` forcing or using a strict record field etc. as suggested here.

Kind of, but I think in most cases the thunks are forced rather quickly and no leak occurs. So you’d get a lot of false positives. Edsko de Vries from Well-Typed has written the `nothunks` library which can give warnings if there are thunks in your code: [Being lazy without getting bloated - Well-Typed: The Haskell Consultants](https://well-typed.com/blog/2020/09/nothunks/).

And all thunks are really potential memory leaks, because if GHC could predict that a thunk is always forced quickly then it wouldn’t have to create the thunk in the first place.

Personally, I’m also excited by [`ghc-debug`](https://gitlab.haskell.org/ghc/ghc-debug) (also from Well-Typed) which is a run time memory analysis tool. That tool can be used to find actual memory leaks at run time without all the false positives.

---

<div class="post-metadata">

**Author:** ![atravers](https://avatars.discourse-cdn.com/v4/letter/a/45deac/32.png) [@atravers](https://discourse.haskell.org/u/atravers)\
**Post date:** [January 20, 2024, 12:21pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/14 "2024-01-20T12:21:00Z")

</div>

> You mean there’s a substantial difference between using a local definition (`go`) and how `loopM` is currently defined?

> > Yes.

Then it’s a problem for GHC and its implementors - **no-one else** should have to `go` and rewrite all their existing code in some _“special style”_ just because the implementation will produce the best result. Haskell is meant to be a _declarative_ language…

* * *

> And all thunks are really potential memory leaks, because if GHC could predict that a thunk is always forced quickly then it wouldn’t have to create the thunk in the first place.

…and all evaluations are really potential _“time leaks”_, because if GHC could reliably predict that an evaluation wasn’t needed then it wouldn’t need thunks **at all**.

* * *

> Personally, I’m also excited by `ghc-debug` […]. That tool can be used to find actual memory leaks at run time without all the false positives.

Personally, I would be a **lot more** interested in the appearance of a heapless (or perhaps just GC-less) call-by-need implementation for GHC:

- [On Inter-deriving Small-step and Big-step Semantics: A Case Study for Storeless Call-by-need Evaluation](http://www.zerny.dk/danvy-al-tcs12.pdf) (2011).

At least then (Glasgow) Haskell would be much closer to parity with Rust…

---

<div class="post-metadata">

**Author:** ![ajbarber](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ajbarber/32/2648_2.png) [@ajbarber](https://discourse.haskell.org/u/ajbarber)\
**Post date:** [January 20, 2024, 12:49pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/15 "2024-01-20T12:49:43Z")

</div>

> [@jaror](#):
>
> Edsko de Vries from Well-Typed has written the `nothunks` library

I don’t know, it just seems too important to be relegated to some third party tool.

---

<div class="post-metadata">

**Author:** ![BurningWitness](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/burningwitness/32/3665_2.png) [@BurningWitness](https://discourse.haskell.org/u/BurningWitness)\
**Post date:** [January 20, 2024, 2:06pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/16 "2024-01-20T14:06:35Z")

</div>

> [@ajbarber](#):
>
> Are there some ground rules for avoiding this situtation? Perhaps hobbyists like me ought to just turn off laziness entirely with that extension…

The fundamental rule is that a part of your programming process is reasoning about laziness. Every time you apply a pure operation to a piece of data, it is applied lazily and will only matter once that piece of data is demanded. You don’t have to be painstacking about it, unless you want the maximum bang for your buck, but if you do no work in this department you’re destined for situations like the one you described.

On a broader scale, try to think about the problems you solve through a functional lens. Instead of writing a sequence of operations that perform side effects in place, figure out what are the inputs in your system, what is the state, and what you need the output. You do not have to use monad cakes to solve problems, I would even argue they’re harmful because they promote imperative programming in a place that does not require it (and everything that does could instead use `IO` directly).

---

<div class="post-metadata">

**Author:** ![atravers](https://avatars.discourse-cdn.com/v4/letter/a/45deac/32.png) [@atravers](https://discourse.haskell.org/u/atravers)\
**Post date:** [January 20, 2024, 11:06pm UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/17 "2024-01-20T23:06:48Z")

</div>

> On a broader scale, try to think about the problems you solve ~~_through a functional [`lens`](https://hackage.haskell.org/package/lens)_~~ from a (mathematically) functional point of view.

- by breaking the original problem down into sub-problems,

- then repeating that process on those sub-problems,

… [Top Down Design](https://userweb.cs.txstate.edu/~js236/201712/cs1428/topdowndesign.pdf) (2017)

* * *

> You do not have to use monad [layer] cakes to solve problems […]

…unless:

- someone else gives you a serving of monadic _“layer-cake”_ as a dependency,

- and there’s no way to contain it (in the way `runST` can for monadic `ST s a` terms).

Then you can expect much of your code to also be passing along serves of monadic _“layer-cake”_. If you find that annoying… **you are not alone:**

- [`Monad`, I Love You; Now **Get Out** Of My Type System](https://www.youtube.com/watch?v=2PxsyWqZ5dI) (2022)

* * *

> I would even argue they’re harmful because they promote imperative programming in a place that does not require it (and everything that does could instead use `IO` directly).

…which most tend to do anyway, having no great interest in the _“slicing”_ most effects-systems offer: ultimately such systems also serve monadic _“layer-cakes”_ (with 2_N_ combinations, for _N_ individual _“layers”_ ).

---

<div class="post-metadata">

**Author:** ![ajbarber](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ajbarber/32/2648_2.png) [@ajbarber](https://discourse.haskell.org/u/ajbarber)\
**Post date:** [January 21, 2024, 8:02am UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/18 "2024-01-21T08:02:49Z")

</div>

Thank you, I see your point that it is a mindset, rather than a rule.

But consider this. Taking heavy inspiration from the Well Typed blog link above, say I am in an organisational setting. And say a PR lands on my desk with a small change where, in reviewing this change, I am guilty of not doing a thorough top down analysis of the problem, and approve the seemingly innocuous change, which passes CI, but which as you’ve no doubt guessed by now, leads to a horrible server crashing memory leak.

Do you think it is acceptable, that ghc provides no warning (albeit noisy), of this situation occuring?

We’re all about type safety, but I wonder have we paid enough attention to memory safety in ghc.

---

<div class="post-metadata">

**Author:** ![BurningWitness](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/burningwitness/32/3665_2.png) [@BurningWitness](https://discourse.haskell.org/u/BurningWitness)\
**Post date:** [January 21, 2024, 10:00am UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/19 "2024-01-21T10:00:09Z")

</div>

> [@ajbarber](#):
>
> Do you think it is acceptable, that ghc provides no warning (albeit noisy), of this situation occuring?

Relative to how often this issue arises, I would say the status quo is fine. I would prefer seeing this problem solved from the ground up (being able to just say a certain piece of data is always strict), instead of bringing in a whole host of speculative tools.

> [@ajbarber](#):
>
> organisational setting

I do not advocate for production-grade Haskell, unless the company knows what it’s doing. My answer to “what if someone makes a stupid change and it sets the production environment ablaze” is “just don’t use Haskell in production”. There has to be a better way to structure code that doesn’t require sledgehammer solutions.

And, of course, it’s not just a production problem, the entire ecosystem is like this. Libraries choose to hide laziness from the end user, so the only time you come across it is when you find out your little program takes 8Gb of memory to run, at which point your forum question yields a bunch of “that’s why I use this cool library to solve the language for me” answers.

---

<div class="post-metadata">

**Author:** ![tomjaguarpaw](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/tomjaguarpaw/32/1230_2.png) [@tomjaguarpaw](https://discourse.haskell.org/u/tomjaguarpaw)\
**Post date:** [January 21, 2024, 11:36am UTC](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602/20 "2024-01-21T11:36:57Z")

</div>

> [@Ambrose](#):
>
> I thought CPS `Writer` basically _was_ `State` under the hood? 🤔

Yes, that’s right.

> [@jaror](#):
>
> Edsko de Vries from Well-Typed has written the `nothunks` library

`nothunks` was the inspiration for my article [Make Invalid Laziness Unrepresentable](http://h2.jaguarpaw.co.uk/posts/make-invalid-laziness-unrepresentable/). After I learned about that library I thought “after we’ve checked a value for thunks it would be nice to indicate that in the type, so that we know we don’t have to ever check it again” (I was imagining introducing a `ThunkFree` newtype wrapper). Then I realised that the simpler approach was to forbid thunks from occurring in the first place, through a better definition of the type in question!

> [@ajbarber](#):
>
> it just seems too important to be relegated to some third party tool.

Yes, that’s why you should make invalid laziness unrepresentable. Then you don’t have to rely on any tool. The impossibility of thunks just becomes part of your program.

> [@ajbarber](#):
>
> say a PR lands on my desk with a small change where, in reviewing this change, I am guilty of not doing a thorough top down analysis of the problem, and approve the seemingly innocuous change, which passes CI, but … leads to a horrible server crashing memory leak.
> 
> Do you think it is acceptable, that ghc provides no warning (albeit noisy), of this situation occuring?
> 
> We’re all about type safety, but I wonder have we paid enough attention to memory safety in ghc.

I don’t think this situation is acceptable, but it’s not GHC’s fault. The author of `loopM` chose to allow thunks to build up in the state that is passed between iterations (similar to `foldl`). Why should GHC object to that? Maybe that’s necessary for some applications. If the author wanted different behaviour he should have implemented it differently. If the user wanted different behaviour she shouldn’t have used `loopM`. It’s not GHC’s job to decide! (This is why I have opened the discussion [Why shouldn't I make my monads "value strict"? - #2 by sgraf](http://discourse.haskell.org/t/why-shouldnt-i-make-my-monads-value-strict/8609/2))

So what exactly do I find unacceptable? Our ecosystem has 100 laziness footguns, `foldl`, `modifyIORef`, `Control.Monad.Trans.State.Strict.modify`, all of `Control.Monad.Trans.State` (i.e. not `.Strict`), all of `Data.Map.Lazy`, … . It is so easy to create catastrophic space leaks using them that they should only be used by experts in very specific circumstances (generally to eke out performance). But we don’t do a very good job of educating users about this in general, nor of finding way of discouraging use of the footguns through other means.

[Next page](https://discourse.haskell.org/t/state-monad-memory-exhausted/8602.md?page=2)
