You don't need an effect system

[BurningWitness]

You don't need an effect system

[Ambrose]

You don’t need Haskell either, and yet here I am

[olf]

In a sense, you seem to demonstrate that IO is the final effect system: Anything any effect system can do, you can do in IO.
Perhaps the same can be said about continuation-passing style?

[janus]

you seem to demonstrate that IO is the final effect system: Anything any effect system can do, you can do in IO.

But the article doesn’t say anything about all effects system everywhere. It explicitly says:

plain IO is incapable of these things, and that’s something I could neither confirm nor deny.

For example, if you consider memory usage an effect, then it seems like IO is not equipped to reason about that at all? That seems like a useful effect to reason about, though. That’s why there are unlifted datatypes, right?

I suppose I am really asking: Who gets to define what an effect is? And if something is doable in IO, but unsafe, does it really qualify as “doable”? You’d have to sacrifice something.

[jaror]

@janus

Who gets to define what an effect is?

The person who designs the effect system, which is also constrained by the language design if it is an embedded DSL.

You can imagine a language that is like Haskell, but requires all allocation to happen inside IO. That would be quite a different language. All constructors, lambdas, and let expressions would return a value wrapped in IO. (Unlifted types have little to do with the allocation itself, more with the memory layout and when evaluation happens).

[Ambrose]

This article’s points are why I really like just using ImplicitParams of records on top of IO [1]. It’s been very nice in games. The records can be abstract and of functions or just of stuff. I’m not too picky.

But it feels to me that “just pass stuff around” would be well-served by the “passing stuff around for you” language extension

[1] Actually, apecs System

[BurningWitness]

Video games (or, more generally, graphical applications) exist in a league of their own, as pointed out at the very end of On the purported benefits of effect systems. There isn’t any space for a “we failed partway, here’s why” behavior, the application either moves on or crashes hard. It’s very similar to how I didn’t list web API calls as an effect type in the post: a bad response is a reason to branch and do something different, not to throw exceptions.

In my game I ended up with an actor model to support multithreading, so I pass actor references as arguments. My code still uses note logger, but that doesn’t log directly, it shoots a message.

---

As for extensions, the approach shown in the article doesn’t use any. So the question I’d ask is the same as for effect systems: sure, you can mitigate the inconvenience of passing arguments to an extent (though you’ll still have to pass them somewhere), but does this one benefit outweigh all other problems you’re causing along the way?

[wolverian]

I’m confused about this article. To me, the purpose of MTL/effects is more type safety. That’s it. This article seems to argue that you should just put everything in IO instead. I also don’t understand this argument:

• No, tracking effects does not preclude anyone from adding bad IO to an effect implementation, from importing unsafePerformIO, or from finding a sum of an infinite list.¹

I don’t even know what to say to this straw man.

[tomjaguarpaw]

I broadly agree! This is basically RIO/ReaderT _ IO minus the (reader) monad bit (i.e. just passing stuff around, nice) plus the attempt at scoped exceptions (nice). I think most Haskell users would be better served by that style than by traditional (“synthetic”) effect systems. However, I think “analytic” effect systems (first effectful, later Bluefin) which are wrappers around IO (in Bluefin’s case, a very simple wrapper around IO) are a compelling alternative.

There are some caveats with the article, one serious. The serious one, a deal breaker for me, and probably a dealbreaker for all pratical use cases, is that Early e aliases. If I have two Early es floating around, with the same e, they interfere with each other. Consider this:

-- > aliasExample
-- "inner caught: tried to leave via outer"
aliasExample :: IO String
aliasExample = do
  result <- runEarly $ \outer -> do
    let act = leave outer "tried to leave via outer"
    innerResult <- runEarly $ \_inner -> do
      act
    case innerResult of
      Left err -> pure ("inner caught: " ++ err)
      Right impossible -> absurd impossible
  pure $ case result of
    Left err -> "outer caught: " ++ err
    Right message -> message

act can be arbitrarily complicated and defined arbitrarily far away from its use within the inner runEarly, but there’s nothing the caller can do to ensure that its exception gets picked up by the intended handler (i.e. the outer runEarly).

The particularly dangerous case is where you have a function f uses runEarly at type e and that accepts a g :: ... -> IO r as an argument. If you call f with a g that has another Early e in its closure then a call of leave in g will not return to the point you expect.

That’s already solved though, by Bluefin.Internal.Exception.Scoped (based on an ideas from Andrzej Rybczak and @Leary). It’s a simple drop-in replacement for Early defined in the article.

A second caveat, less serious and not a dealbreaker, is that Earlys can escape their scope:

-- > escapeExample
-- *** Exception: Early return exception
escapeExample :: IO ()
escapeExample = do
  innerResult <- runEarly pure
  inner <- case innerResult of
    Left () -> error "Impossible"
    Right early -> pure early

  escapedEarly <- runEarly $ \outer ->
    leave outer inner
  case escapedEarly of
    Left inner ->
      leave inner ()
    Right impossible -> absurd impossible

It might be nice to prevent that from happening. How could we do that? Well, one could use scope tracking a la Bluefin. That comes with some, but I would claim not much, additional complexity. I won’t hold it against anyone who doesn’t want to pay that additional complexity.

---

Beyond that, there were a number of claims in the article that don’t make sense to me, and some other things I thought worth commenting on:

Does it make for a safer codebase?

No, tracking effects does not preclude anyone from adding bad IO to an effect implementation, from importing unsafePerformIO, or from finding a sum of an infinite list

Perhaps not, but then by the same logic Haskell doesn’t make for a safer codebase than Python.

Am I using it outside of IO?

No, for pure functions that do mutation the ST monad works just fine.

For pure functions that do mutation and throw exceptions the ST monad doesn’t work fine. For that you need Bluefin.

random number generation is commonplace, but the steps necessary to generate a random value are different every time. The only shared behavior is the use of generator state, and even then it’s unclear if passing it around has any benefits over using the global generator

Passing generators around has a performance benefit over the global generator, if one is in a concurrent situation: “AtomicGenM is the slowest of the monadic adapters due to the overhead of its atomic operations”.

Have I made use of higher-order effects?

No, none of the effects I’m using need to overlap or nest

There seems to be a misconception here. The effects from this article are perfectly consistent with higher order uses, and here is a higher-order effect from later in the article:

runStack
  :: Logger -> Metrics -> DatabasePool -> MessagingEnv
  -> (Early () -> Database -> Messaging -> IO ())
  -> IO ()

The presence of something returning IO in argument position makes runStack a higher order effect.

(I’m not sure what the article means by “nest”. The effects in this article clearly nest! And higher order effects are not really about overlapping or nesting.)

This may well be the most embarrassing problem in all of Haskell. Over and over again people waltz in with the exact same basic question: “How do I return from a function early?”.

I agree it’s embarrassing. To dispute nomenclature, runEarly is not really an “early return”, it’s more like a “scoped try”. “Early return” would be something like Typeable e => (Early e -> IO e) -> IO e.

(For the record, the GHC proposal linked as a footnote was one of the things that led me to develop Bluefin, since I did consider it embarrassing that early return was considered difficult enough to require a syntax extension.)

And just like that I ran out of reasons to use an effect system.

IO doesn’t have a “thread-safe, locally modifiable state” so if one wants that one has to use an effect system (although there is a proposal to add thread-safe, locally modifiable state to IO, in the scoped thread-locals proposal). (Perhaps @BurningWitness really does not need that, though).

There is only one unique feature effect systems—or, more generally, custom monads designed to supercede IO—provide: they can do anything in between the lines [of code].

Hmm, no, not really. Bluefin doesn’t do anything “between the lines” (with the arguable exception of passing around a Vault to support the Ask capability, but if IOScopedRef is added to GHC then it won’t even need to do that). Bluefin effectful operations accept value-level capabilities as arguments.

Compared to the style presented in the article, the benefits are Bluefin are

• No leaked effects (such as the leaked exceptions above), no leaked resource handles (no “Database” is in scope after the connection is closed)
• You can run observationally-pure code to a pure value, i.e. you have runPureEff :: (forall e. Eff es r) -> r

The relative costs are:

• You write e1 <: es => Throw e e1 -> Eff es r rather than Early e -> IO r
• You sometimes have to manually munge these e1/es thinks with useImpl and friends.

(I’ll let the authors of other effect systems speak for themselves if they object to that characterization, but the others are all significantly more complex than Bluefin.)

Effects are tracked as function arguments; this is quite different from effect systems, which prefer constraints

No, not in the case of Bluefin. It also tracks effects as function arguments.

Here are the disadvantages:

• Some functions will be verbose.

I hope this knowledge is used for things.

Strong agreement with the author in this point of style. After using arguments-as-effects in Bluefin for 2.5 years I have never once thought “passing arguments is tedious; I wish it would happen automatically”.

I’ll use the database effect as an example. Here’s a rough outline of the implementation function:

runDatabase connInfo f =
  mask $ \unmask -> do
  ...

I don’t understand why this isn’t just using bracket. Isn’t that one of the advantages of this style, that you can use bracket as it is, without any wrapping? Have I missed something? I also don’t understand why the implemenation throws a naked DatabaseException:

case fromException ex of
  Just dbEx -> pure $ Left (dbEx :: DatabaseException)
  Nothing   -> _ -- (4) Failure above.

Shouldn’t it take advantage of Early DatabaseException?

[BurningWitness]

@wolverian

To me, the purpose of MTL/effects is more type safety. That’s it. This article seems to argue that you should just put everything in IO instead.

And I have no idea what you think it is that supposedly makes MTL/effects more type safe over the approach I outlined.

I could replace every user-facing mention of IO in the article with

newtype Eff a = Eff { runIO :: IO a }
                deriving (Functor, Applicative, Monad)

liftIO :: IO a -> Eff a
liftIO = Eff

This would force users to add an additional liftIO in front of every IO action. Will it make the code safer? Does it do anything beyond creating friction?

[tomjaguarpaw]

@BurningWitness

I could replace every user-facing mention of IO in the article with

newtype Eff a = Eff { runIO :: IO a }
                deriving (Functor, Applicative, Monad)

liftIO :: IO a -> Eff a
liftIO = Eff

That is roughly what effectful and Bluefin do. Would you like me to elaborate on why that is a more type safe approach?

[Ambrose]

@BurningWitness

And I have no idea what you think it is that supposedly makes MTL/effects more type safe over the approach I outlined.

You can constrain what IO a function can do, and track it in the type system. Get fancy enough and you can statically analyze and lint effects.

[BurningWitness]

@tomjaguarpaw

The serious [caveat], [..] is that Early e aliases. If I have two Early es floating around, with the same e, they interfere with each other.

There’s no need to ever use more than one Early effect at a time, so I don’t consider this an issue. The very liberal typing of Early is precisely due to this, otherwise I’d make a separate effect for servant errors.

The solution you have is a global counter for all exceptions, and while it’s not unreasonable, I think it can be avoided by having users solve conflicts, see the “Duplicate effects” subsection of the post.

@tomjaguarpaw

A second caveat, less serious and not a dealbreaker, is that Earlys can escape their scope [..] It might be nice to prevent that from happening.

Or we could leave it as a warning sticker, same as every other use of bracket in the language. All of my effects are handled in stacks, so I don’t think this representative of common usage.

I also carry a CallStack when returning in my codebase, so even if this error somehow occurred, it wouldn’t take long to solve.

@tomjaguarpaw

For pure functions that do mutation and throw exceptions the ST monad doesn’t work fine.

Is there a reason why I can’t make an EarlyST s e by converting ST to IO and back? The only caveat I know of is about memory locations and I don’t think of exceptions like that.

@tomjaguarpaw

I’m not sure what the article means by “nest”.

As in using effects as foundations for other effects. It’s the reason why I wouldn’t write Early DatabaseException, both effects throwing exceptions are their respective implementation details, not something to share.

Admittedly I do pass a Logger to a Database effect later on, but if I were abstracting
the code to a library I’d swap it out for a (Builder -> IO ()).

@tomjaguarpaw

I don’t understand why this isn’t just using bracket. [..] I also don’t understand why the implemenation throws a naked DatabaseException.

The only case that throws is (4), more specifically it rethrows. Cases (2) and (3) both return Left. (I should have been more specific here, I’ll fix it)

bracket is unrolled because sometimes case (3) needs special handling, think running a connection pool where errors nearly always taint the connection. It also looks better than catch over bracket, in my opinion.

@tomjaguarpaw

Would you like me to elaborate on why that is a more type sate approach?

You’re free to if the explanation is not “it stops me from using IO indiscriminately”. In my mind if I want to prevent all uses of print in the database, I should simply never import it (so I blame the kitchen sink design of Prelude and move on).

[tomjaguarpaw]

@BurningWitness

There’s no need to ever use more than one Early effect at a time

It’s worse than that though. Suppose someone writes myReplicateM_ :: Int -> IO a -> IO a using forever, an IORef as a counter and Early () to break out. If anyone calls myReplicateM_ with second argument a closure which itself has an Early () in scope things will go terribly wrong if you ever leave via that Early ().

Unless you know the instantiations of Early used in a higher order function do not conflict with yours, you can never call it safely.

@BurningWitness

Or we could leave it as a warning sticker, same as every other use of bracket in the language.

I don’t follow that. I’m not aware of any warning sticker that comes with bracket. What are you thinking of?

@BurningWitness

@tomjaguarpaw

For pure functions that do mutation and throw exceptions the ST monad doesn’t work fine.

Is there a reason why I can’t make an EarlyST s e by converting ST to IO and back?

Not sure exactly what you’re suggesting, but it sounds like if you pursue that idea and make it compositional you’ll end up with Bluefin (Bluefin is basically “ST but with exceptions and IO too”).

@BurningWitness

As in using effects as foundations for other effects. It’s the reason why I wouldn’t write Early DatabaseException, both effects throwing exceptions are their respective implementation details, not something to share.

Oh right, so the design presented in this article is not supposed to be compositional, in the sense that you do not expect to be able to package up Earlys with other types (as types) and pass them around together?

@BurningWitness

The only case that throws is (4), more specifically it rethrows. Cases (2) and (3) both return Left. (I should have been more specific here, I’ll fix it)

Still don’t quite grasp it. Maybe you want an analogue of catch rather than the analogue of try you currently have? Anyway, I’ll look again when you’ve fixed it.

@BurningWitness

You’re free to if the explanation is not “it stops me from using IO indiscriminately”.

Well, that’s more or less the explanation, so I guess I won’t give it I don’t think anyone who wants that sort of type safety will be convinced by your approach though.

@BurningWitness

In my mind if I want to prevent all uses of print in the database, I should simply never import it (so I blame the kitchen sink design of Prelude and move on).

What if you call a function that used print and you didn’t know?

[arybczak]

Well, of course you don’t need an effect system. You also don’t need Haskell to write programs.

The issue is that you argue for no need of an effect library, then go on and build a half-baked implementation of one.

The main gain from using effectful (or bluefin) comes from the fact that you get a unified interface for dealing with effects and you can reuse pieces across multiple libraries/applications. You’re also using a framework a lot of thought went into to eliminate common bugs and pitfalls you might not even know exist.

If you build your own bespoke flavor of effect handling in application A, someone else builds one in application B etc., then whenever these two meet in some way, there’s gonna be friction.

[BurningWitness]

@tomjaguarpaw

I’m not aware of any warning sticker that comes with bracket.

“[value] passed to [function] must not be used after this” in Foreign.Marshal.Utils.with and similar. No such warning with file handles because they close automatically, caveat for that.

@tomjaguarpaw

Unless you know the instantiations of Early used in a higher order function do not conflict with yours, you can never call it safely.

Now that sounds like something that could happen, sure. Though it feels weird because I should be the one driving the control flow, not libraries.

The best current solution being a global counter is quite annoying, if we had thread-local state I think putting a counter there would look really nice.

@tomjaguarpaw

if you pursue that idea and make it compositional you’ll end up with Bluefin.

No, I would end up with a global counter that should be shared among all exception-throwing effects I use. The entire point of the post is that I want to be able to pick and choose features I need, not to be forced to subscribe to a magical box of features and subsequently get mocked by half the community when I challenge dogma.

The only box of features I should need to write Haskell is Haskell itself.

@tomjaguarpaw

[..] you do not expect to be able to package up Earlys with other types (as types) and pass them around together?

I don’t do anything with effects beyond what I outlined in the post, which is specifically passing them around as arguments to functions that use them.

@tomjaguarpaw

What if you call a function that used print and you didn’t know?

If it’s a function in a library and that’s not its documented behavior, it’s not behaving correctly. If it’s my function, I screwed up.

[tomjaguarpaw]

@BurningWitness

“[value] passed to [function] must not be used after this” in Foreign.Marshal.Utils.with and similar. No such warning with file handles because they close automatically, caveat for that.

Oh I see, a sticker on things created using bracket, or more generally bracket-like things, rather than bracket itself (which doesn’t allocate resources).

@BurningWitness

Now that sounds like something that could happen, sure. Though it feels weird because I should be the one driving the control flow, not libraries.

Do you not want libraries to use Early?

@BurningWitness

The best current solution being a global counter is quite annoying, if we had thread-local state I think putting a counter there would look really nice.

I hadn’t thought of that but I agree. It makes a lot of sense!

@BurningWitness

No, I would end up with a global counter that should be shared among all exception-throwing effects I use

Well then, please share your “exceptions inside ST” implementation since it sounds like it’s different to what I originally imagined.

@BurningWitness

The entire point of the post is that I want to be able to pick and choose features I need, not to be forced to subscribe to a magical box of features and subsequently get mocked by half the community when I challenge dogma.

Don’t follow that, sorry. And on that topic, what do you consider “magical” about Bluefin?

@BurningWitness

I don’t do anything with effects beyond what I outlined in the post, which is specifically passing them around as arguments to functions that use them.

I see, but you said “You don’t need an effect system”, not “I don’t need an effect system”. Many Haskell users do want to make compound effects. You’re saying your approach doesn’t support that. Fine, but it probably won’t satisfy many people’s use cases.

@BurningWitness

If it’s a function in a library and that’s not its documented behavior, it’s not behaving correctly. If it’s my function, I screwed up.

I see, so do you prefer to rely on documentation when there are alternative approaches to rely on types? Do you prefer to have the chance of screwing up when there are types that can prevent you from screwing up?

[BurningWitness]

@tomjaguarpaw

Do you not want libraries to use Early?

I just never encountered a scenario like this

runStack $ \early -> do
  ei <- some_action_that_leaves_early_by_itself $ \good ->
          unless good $
            leave early ()

  case ei of
    Left () -> _
    Right a -> _

I can’t come up with a compelling case for it on the spot. I can see servant passing me an Early ServerError, then I just wouldn’t make one myself.

@tomjaguarpaw

Please share your “exceptions inside ST” implementation since it sounds like it’s different to what I originally imagined.

ST can definitely use the no-counter version of the effect, I really can’t imagine why someone would need to interleave Early there. Alternatively thread-local state, as always.

@tomjaguarpaw

do you prefer to rely on documentation when there are alternative approaches to rely on types?

As far as I understand, all of the code in the codebase I rewrote is exactly as type safe as if I had used an effect system. The effects do come with a few caveats that would need to be ironed out if they were to be generalized to libraries, but nothing particularly out of the ordinary.

I would definitely prefer it if I could do more type level things in Haskell, but that would call for a different Haskell. The fact that the common convenience operator for listing multiple effects both runs like hot garbage and makes it impossible for GHC to point at unused effects is the very issue that spurred me to try to express effects as arguments (which take up less space than constraint effects listed one by one).

[BurningWitness]

It took me years to figure out what my issues were, months to come up with solutions and weeks to write a blog post. I don’t think it took you (and most of the Reddit community) more than a dozen minutes to skim the article and write comments that don’t say anything I don’t already know.

The title is “You don’t need an effect system” so that you can answer “No, I do need an effect system, your reasoning fails critically here, here and here, consider for example these cases I encountered through my years. I do agree with blank and blank though, nice catch.”. Instead all I get in response is indignation, and the disappointment makes my gut wrench.

[mchav]

No opinions on this thread but the framing here is a little disingenuous. “Do you really need an effect system?” Would be better. “You don’t need an effect system” sounds like a declaration. saying that one could answer “no, I don’t” to a statement sounds like even at its best invites relatively combative responses. @BurningWitness nit trying to antagonize you in any way but hope that feedback is useful when thinking about how to start and facilitate productive community discussions.