Why use an effect system?

Learn
115

I’m not sure this discussion is leading anywhere. But bear in mind that the term “effect system” is IMO overloaded. There are at least 3 features an effect system (could) offer:

  1. Effect tracking: The framework helps tracking what kind of (most-often IO) capabilities a piece of code needs. This is a generalisation of the ReaderT IO/handle pattern with some additional type foo (ST-like use of higher-rank types) so that the types actually ensure that a use of a database connection cannot escape the lexical scope ot its handle.

  2. Effects as interfaces/(Dynamically-dispatched) effect handlers/probably some term I don’t know of that is established in literature: By building on an effect-tracking framework, one may ask

    • Does it really matter that reading from the database needs to be implemented in IO?
    • Isn’t that rather an implementation detail of the database implementation used in production?

    For example, it would be conceivable to handle database requests with a mocking implementation instead. This implementation would not need to run in IO at all.
    “Algebraic effects and handlers” refers to the technique were the particular implementation of a capability can be swapped out dynamically. The effect itself merely becomes an interface, given meaning by at least one concrete implementation, its handler.

  3. (Statically-dispatched) Handlers for common effects: Likewise by building on an effect-tracking framework, one could provide a toolbox of known-useful effects+handler implementations such as for exceptions, state and concurrency. It doesn’t make much sense for these to be implemented as dynamically-dispatched effects, because their implementation is morally pure anyway (no need to mock local state; just use the implementation). Hence these kinds of capabilities are statically dispatched, which means reliable performance.

Effect systems like effectful, bluefin enable all three of these features. In particular, even though their implementation is in terms of IO, the semantics of state, exception etc. are still pure, so it’s OK for their handler implementations to use unsafePerformIO under the hood to enable efficient implementations.

For users of one of these libraries, there are advantages to each of the 3 features. The first 2 features improve programming “in the large”, while the last feature is useful “in the small”.

  1. Effect-tracking (1) is very useful for the same reasons that a type system that tracks IO is generally useful. For example it can aid debugging in big applications: When a function’s type says it can access the database but not launch new processes, that is useful information when the problem you are investigating is related to spawning zombie processes, because the function’s implementation must be irrelevant.

    How granular you need to structure your effect domain depends on the use case and your taste. I certainly wouldn’t reach for effect-tracking if my entire program can be defined as main = interact foo (interact). However, I would certainly welcome if GHC’s code base were to be structured using more granular effects, because I have often wondered whether some particular IO-based function in a call stack of 10 is relevant to the problem I’m debugging. Saying “I don’t see the appeal in effect-tracking” likely means that your IO-layer isn’t complex to warrant effect-tracking, and that’s fine. It does not mean that effect systems are useless, IMO.

  2. Effect handlers (2) are useful for mocking production implementations in unit tests. Yes, there are other means to achieve mocking. But effect handlers make it particualrly convenient to define and implement new effects, while retaining acceptable performance. Effect handlers are basically the evolution of dialogue-based APIs that @atravers keeps bringing up, as evidenced by @jaror in Dialogues vs continuations (and algebraic effects) to implement I/O. Whether you pick continuations (the mother of all monads) or implement a monad with continuations to sequence effects is IMO a bit beside the point.

    As others such as @osa1 and @danidiaz have implied, there is a deep connection to dependency injection (DI). The job of a DI framework such as autofac is to construct instances of a class (I’ll stay with C# lingo here because that’s where I know it from). Every class specifies interfaces such as IDatabase it needs access to (commonly via constructor parameters), and the job of the DI framework is to resolve these interfaces to their proper implementation (such as MySqlDatabase) upon constructing the class via reflection. Which interface gets resolved to what implementation is specified at one central location: The registry (hence I suppose the name for the Haskell DI library).
    Defining a handler can be seen as registering an implementation in a DI container, which subsequently resolves every use of that effect with said handler. I think one fundamental difference to DI containers is that handlers have a builtin notion of scope, whereas that counts as an advanced use of DI frameworks. Yet, I would guess that you could probably implement a useful DI framework on top of bluefin or effectful.

  3. An effect toolbox (3) is pretty handy, locally, in the small. For example, it is useful to have efficient exceptions. Additionally, I have the guarantee that the effects in the toolbox work well together; the interaction of state, exceptions and concurrency is just as one would expect. Furthermore, these effects can all eliminated in pure code! Thus, use of the toolbox does not even need to show in any exported type signature.

    Some anecdotal evidence wrt. efficiency and exceptions:

    I recently have reviewed a function that was basically applying escapes to a string, escape :: String -> Either Error String. This function should return Left err when there was an error during escape resolution. It would be prohibitively expensive to implement this function in terms of Either, though, because it means that every call of the recursive worker function would need to implement a Right upon returning. It would be far more efficient to implement this function using unsafePerformIO and throwIO+catch, allocating Right/Left only upon exiting the function. The resulting function was still pure, but I felt bad for the use of unsafePerformIO. Using bluefin in such a scenario would yield just the same implementation, but without needing to justify unsafePerformIO in my code.

Summary

To summarise my understanding, I would pick an effect library such as effectful or bluefin if

  1. I want to implement a function using exceptions, early return and/or state, because such formulations can be clearer than applicative encodings (see also ‘do’ unchained for examples). Then the effect toolbox (3) is useful.
  2. I have (or expect to be having) an IO-heavy code base where I have difficulty to debug the source of faulty behavior of a pervasively used side-effect such as interacting with a database. Then the effect-tracking feature is useful.
  3. I want to unit-test such a complex IO layer, so I need to be able to mock out application domains. One way to achieve this is by decoupling application domains into effects with dynamically specified handlers. In particular, if I use bluefin or effectful for effect-tracking anyway, it should be a pretty cheap transition to use it for mocking and dependency injection as well.
20 Likes