Another potential benefit is that effect systems provide flexibility when testing effectful code.
…provided the number of independent effects is kept relatively small, because the number of combinations of effects will be exponential: n effects → 2n potential combinations to test with.
But since effect systems these days are trying as much as possible to decrease the performance issues when using big effect stacks, doesn’t that mean that these combinations should be fine? Unless you mean that it makes testing difficult, but I believe you just change the handler of the effect to another handler that does what you need for testing, right?
[…] I believe you just change the handler of the effect to another handler that does what you need for testing, right?
That only helps if [the use of] each effect is tested in isolation. If effects are going to be combined, then those combinations will also have to be tested. Now imagine [potentially] needing a separate test for each combination of effects…
[…] since effect systems these days are trying as much as possible to decrease the performance issues when using big effect stacks, doesn’t that mean that these combinations should be fine?
If you have 32 independent effects, that’s 232 potential combinations of those effects - over 4 billion of them to test.
I think testing effect implementations themselves and testing effectful functions are different. If you aim to test whether an effect you developed is well behaved with other effects, then what you described is a potential problem (and maybe a reason why we should ensure the properties in the effect system formally instead of test them).
However for effectful functions, effects enable a form of mock testing - we swap out the implementation for certain effects that the function requires for mock ones, like an in-memory database and FS for the respective DB and FS effects. This has nothing to do with testing exponential numbers of effect combinations.
I’ll try to sum up the Haskell effect system lore that I have had some participation in, which is from a reasonably early point, i.e. from circa 2020 up to now. Since you can’t watch Alexis’ videos, this hopefully serves as a rudimentary substitute as well.
There had been free(r) monads-based implementations for quite some years as of 2020, most popular (read: most usable) ones are freer-simple and polysemy. They are slow because they essentially construct a program tree at runtime, and then the handlers traverse the tree to slowly materialize a program. freer-simple is actually often respectably fast though, but this brings us to another topic of our story: higher-order effects.
Higher-order effects means you can create effects with operations that take effectful computations as arguments. That probably sounds confusing; essentially, it allows operations like WithFile :: FilePath -> (Handle -> Eff es a) -> Eff es a. The second argument is an effectful computation because it returns an Eff. If your effect library supports higher-order effects, you can write different handlers for this operation: for example, one reading files from a web server while the other reading from your local hard disk. Many people think this is a good thing to have; freer-simple doesn’t have it however, but polysemy does (in an awkward way, but it does the job most of the times).
Then there are fused-effects which everyone believes is fast because it is supposed to “fuse away” intermediate structures… well, it has the same problem as mtl: if the code doesn’t specialize, compiler has no way to fuse them away! For your code to specialize reliably, the only way is to add {-# SPECIALIZE #-} all over your functions, which is very not ideal; not only is it tedious to write, this also slows down compilation.
So the situation is not exactly good. Alexis King and Ningning Xie almost simultaneously had an idea circa 2020 (I believe; I don’t know if they had communicated about the idea before), which is to use evidence-passing and delimited continuations to build an effect system in Haskell that would be both fast and expressive enough; specifically, in Alexis’ vision, it will support fast higher-order effects.
What is evidence-passing? It basically means to pass the effect handlers around in the Eff monad, so instead of handlers traversing a big program tree like in free monads, we directly call the handlers in place. This is a much more performant approach, and it’s pretty shocking that nobody thought of this before. Nowadays, eff, eveff, mpeff, speff, cleff, and effectful all use evidence passing.
Delimited continuations are nothing new actually, and has been used to develop other languages with effects like Koka since at least 2015. Probably nobody thought of doing this in Haskell before because Haskell didn’t support it natively, and they didn’t think it would be efficient. Ningning came up with a monad that can do delimited continuations fairly fast, which is used in eveff, mpeff and speff. On the other hand Alexis decides to just add native delimited continuations to GHC and went down that way with eff.
OK, then what did cleff and effectful use instead of delimited continuations? The answer is nothing: they had different tradeoffs in expressiveness compared to the other libraries mentioned - they do not support nondeterminism, so that they can support MonadUnliftIO. This consequently eliminates the need for delimited continuations. Why we can’t support both nondeterminism and MonadUnliftIO? It is perhaps less well known that many IO functions accessible via MonadUnliftIO, mainly those that use fork and bracket, only work when the control flow is deterministic. This has not been a problem since the beginning since Haskell was a deterministic language - but not now! Future effect system users will probably face a choice between nondeterminism-capable libraries and MonadUnliftIO-capable libraries and they will need to choose based on their specific needs.
Hope that has fulfilled your curiosity. About your questions on effect system implementations in Haskell:
In reality, implementations could be way faster than they are right now (eff, polysemy, eveff, speff, mpeff)
As you can see, evidence-passing based implementations are already very fast, and increasing user adoption, instead of squeezing out more performance, is our primary goal now.
Eff uses delimited continuations but is waiting for some primops to be merged to gain speedups
It’s merged and to be shipped in GHC 9.6!
eveff/speff/mpeff seem to use a new paper to implement effects that seems really weird
One thing I keep saying is that users of effect systems should not need to know about the internals. If an effect system forces users to do so, it has failed. But on the flip side, “implementation technique sounds weird” should not scare you away from using a library - you only need to interact with the API after all! However I must warn you that these 3 libraries are all proofs of concepts, and should not be used in production.
In some cases, they’re more complex than required.
Polysemy’s API is… not ideal. But as you mentioned, new libraries’ APIs are getting intuitive and easy-to-use, so you can try out those libraries more instead. We can’t control what others think, really; and the notoriety of the last wave of effect libraries cannot be simply erased. But I’d continually encourage Haskellers I meet to give the new generation of effect libraries a chance.
16 Likes
…so if the action being tested uses n effects, that would mean 2n potential combinations of mock and regular implementations are required. The main difference would be that the average action would only use a few of the set of available effects, so testing is more plausible.
Yes, but what you’re describing is still testing the effect implementations, not the effectful computations. I believe both the original comment by @tcard and I are referring to the advantage of effect systems concerning testing effectful computations.
About testing effect implementations, I really think we should prove the properties formally instead. Or maybe I’m wrong and we actually should test them, in which case what you described is an unsolved problem.
1 Like
I believe both the original comment by @tcard and I are referring to the advantage of effect systems concerning testing effectful computations.
In what way? Since effects can be represented with monadic types (as shown by Philip Wadler), I’m having difficulty trying to understand what benefits an effects system have over them - links to pertinent URLs would be helpful…
What I meant is that “by allowing the user to provide multiple implementations for one interface, we make mock testing possible.” Any system that has this property will be able to do mock testing, and common Haskell effect libraries happen to have this property. Does this answer your question?
What are effect system really for?
To me, there are two interesting properties:
- Effects are statically checked and I can constrain them to certain functions etc
- Effects are abstract and one can switch out the interpreter and run the same code (e.g. your data in memory instead of in postgres or a fake filesystem etc.)
Well, 1. doesn’t necessarily need to live in the type system (or in a single type system). An effects system as part of the language could be implemented in a number of different ways.
Point 2. is wildly exciting, but also wildly chaotic. We already have massive issues with type classes for this reason and try to come up with laws and properties that hold for all instances, so that we can at least do some reasoning. With effects and interpreters of arbitrary complexity and depth, this will easily make reasoning about effect code impossible, unless you have a specific interpreter in mind. Then the question is: why use an effects system at all?
Overall, I see their use cases, but I’m not sure we’re advanced enough yet to know when and how to use them. So I’m very skeptical of introducing them in production settings.
The times I’ve dealt with effect system code professionally, it made it really hard to figure out what the code is actually doing and not in an abstract sense. Tracking down the real implementation bits took a long time, jumping through loads of indirections.
4 Likes
“see what effects a block of code does not have”
There’s no way to do the above in anything that uses
IOas a base type
Working with records-of-functions, we can restrict effects by parameterizing the records by a monad and being polymorphic over the monad: the effects must then come only from the dependencies. Like in the above-mentioned constructor:
makeRepository :: (Has Logger m deps, Has SomeOtherDep m deps)
=> deps -> Repository m
The “methods” in Repository can’t do IO on their own. Of course, when wiring together the application, we might then commit to IO.
…but according to you, mock testing only tests the implementation of effects - by the responses of yourself and others, I thought there were other benefits I didn’t know about.
Again, if there’s a monadic type for each effect, mocking is still possible - what other benefits does an effects system provide?
I think you have misread. It tests the effectful computations, not the implementation of effects.
FWIW I personally don’t find number 2 very exciting. The number of times I’ve wanted to switch out interpreters is very limited (perhaps zero). One might argue I’ve never done it because it hasn’t been easy before now, but I am skeptical.
For me 1 is the key property: I really, really want to know what effects parts of my programs can do and, probably more importantly, can’t do. That’s most of the reason I use Haskell.
2 Likes
Can you show me an equivalent example, in the records-of-functions setting, to the one I showed you? That is, throwing an exception (or exception-like thing), catching it, and having the fact that an exception was thrown internally be invisible externally (i.e. it’s not present in the type)?
That is, can you write something like:
bar :: Has (Exception String) m deps => deps -> m Int
bar = if True then throw deps "Hello" else pure 5
foo :: deps -> m (Either String Int)
foo _? = handleExceptionSomehow bar
I’m not aware of any record-of-functions approach that can do this currently.
(Personally I like the record-of-functions approach and I’m working on a version that can remove effects like this. If there’s already an effect system that can do that, so much the better, I’ll just use that! But I don’t believe there is.)
1 Like
One other advantage of effect systems is that they are like modular monads. Instead of having to write one monolithic monad from scratch for every application, you can simply compose it from existing effects.
Yes, mtl can also do things like that, especially if you use deriving-trans, but that approach is not really any easier than using effect systems in my opinion.
2 Likes
Hrm:
If you want to test the action, why would you want to use a mock implementation of the effect? Surely you would want to know that the action works with the real implementation…
Effect systems allow you to both study the a mock implementation and a real implementation. The mock implementation can simulate very rare events that might never occur when testing the real implementation (e.g. disk failure). Also, the mock implementation can be much faster and thus allow you to run much more tests than the real implementation (e.g. an in-memory database).
The question you’re asking is answered right above in the quote. Idk what you’re trying to prove here. (If you’re trolling then well done)
2 Likes