Why use an effect system?

Surprisingly, the most obvious option – providing a pure `IO’ interface – is missing. This would be the easiest for library users to integrate into whatever framework they’re using.

1 Like
  • An effect implementation doesn’t necessarily directly interface with IO at all
  • Limiting the surface area of where IO is used (e.g., only in the final unwrapping of an effect system) has its own benefits which are basically the arguments in favour of using pure functions where possible

Yes, everything must eventually be mashed together into an I/O action. But that’s all Haskell code - it must eventually get to main. That doesn’t stop us from writing pure functions.

1 Like

An effect implementation doesn’t necessarily directly interface with IO at all

That’s why I’ve been taking care to specify the effects as being “external” (as in externally-visible) - as mentioned elsewhere, “internal” effects can be confined runST-style.


Limiting the surface area of where IO is used (e.g., only in the final unwrapping of an effect system) has its own benefits which are basically the arguments in favour of using pure functions where possible.

No.

For a time it was proclaimed that the solution to [chemical] pollution was dilution, which is false. Now for Haskell, a similar proclamation is being made - "the solution to the pollution of effects is the dilution of IO a into individually-typed effects". But one effect seems to always be ignored - the effect on code.

Be it:

  • Eff [... {- "external" effects -} ...] a
  • or regular IO a

…if a change to some obscure definition deep in the program means that definition then relies on an “external” effect, then everything that directly or indirectly relies on that formerly-obscure definition (i.e. its reverse dependencies) that was ordinary effect-free Haskell code must also be changed. That only using the maximum “dilution” - one “external” effect - or potentially all of them is irrelevant: as (correctly) noted in Kleidukos’s presentation, avoiding nondeterminism means all effects must be used in a sequential context which can only be accessed in full via the monadic interface.

So using individual “external” effects is definitely not the same as just using ordinary effect-free Haskell definitions.

1 Like

The unifying concept is indeed the existence of IO. You can choose how you want to deal with it. Maybe you feel more at ease with a “no missile launches here” tag effect reminded to you by the type system. Maybe not. Thus, each library with its opinion.

Also, from the paper:

The representation was error prone.

It certainly was! So when a way was found to do it, dialogue-based I/O (which kept the management of “external” effects outside Haskell) was replaced with functor applicative arrow comonad monad-based I/O in the form of the abstract type IO a (which brought the management of “external” effects inside Haskell).

But now, IO a has been deemed as being semantically error-prone, in need of dilution into separate “external” effects. However there’s a problem here too - as noted in Kleidukos’s presentation, side effects are arbitrary (with “external” effects being externally-visible side effects). So how IO a should be diluted is also arbitrary - there are no “unit effects” in the same way there are chemical elements…and now there are more effect systems for Haskell than the 92 naturally-occurring elements of chemistry.

To summarise:

“External” effects are managed… Problem/s
outside Haskell (“wrapper” ) Error-prone (finnicky!)
inside Haskell (IO a) Error-prone (semantically!)
inside Haskell (effect system) Error-prone (wrong choice!)

…are there any other alternatives that can work for Haskell?

In my view the only true benefit to effect systems is that it’s the only sane way of getting dynamic dispatch in Haskell, so for anything production-grade you *could* have a mirror test system that behaves exactly the same, but calls test versions of all real-world things it links to. It’s still a remarkably hard thing to achieve however: you have to know how to structure your code and you have to avoid any type shenanigans because recursive type families are exponentially slow.

I sympathize with the idea that it would be nice to keep track of which effects are used in any given function, it’s a strongly typed language after all, but having to choose one of five effect libraries and then getting “rewarded” with both a bulkier codebase and performance overheads squarely puts this in the “only use at work on the high level” territory. If GHC had seamless native support for this and some way to precompile functions that only use one implementation at runtime, it would be a no-brainer.

I don’t see a natural benefit of using Readers over plain argument passing. Passing arguments indeed looks bulkier at the first glance, but it allows me to divide the context to the smallest necessary bits at every point. Reader on the other hand necessitates using lens (or more recently field selectors), blurs the line between which arguments are actually needed in a given function, and does not look any prettier.

I don’t see a need in checked exceptions, I lean on the side of “if I expect something to fail without terminating the process, then it’s not an exception”. It’s a natural extension of trying to decouple everything pure from effects, as I can simply use datatypes to convey that an undesired (but not critical) condition has occurred instead of breaking the control flow. It’s also faster.

Regarding lack of modularity in effects libraries, that’s pretty much how all of Haskell’s ecosystem works: instead of providing the minimal tools and letting users stitch things together, it’s instead expected that you use one of fifteen convenient runner functions and for anything beyond that you have to dive into non-PVP-compliant internals. This won’t change unless the community agrees it’s undesirable, good luck with that.

4 Likes

Regardless of whether you reflect the change (throwing a new type of exception) in the type signature or not, throwing a new type of exception is a breaking change (i.e. can break clients), and the users should be aware of it. It should cause a major version bump in your library even without checked exceptions.

Clients then need to consider what to do about the exception, and revisit the call sites.

My observation is that a lot of commentary on checked exceptions from 2000s and earlier (mostly in the context of C++ and Java) do not apply to today’s type systems, scale, and language features. Java did it poorly, and left people traumatized.

There will always be a use case for unchecked (and asynchronous) exceptions, like StackOverflow, HeapOverflow, and ThreadKilled. It probably makes sense to give the users the ability to throw unchecked exceptions as well, and let them decide when it makes sense to have an exception checked vs. unchecked.

Checked exceptions can be modeled as effects, and I suspect with polymorphic variants they can be conveniently composed (I have some notes on this here). No one proved that they can’t be made convenient to use.

1 Like

My observation is that a lot of commentary on checked exceptions from 2000s and earlier […] do not apply to today’s type systems, scale, and language features.

Then here’s the challenge for you and everyone else who thinks IO a is now bunglesome and needs diluting:

type IO a = Eff All a

…use your preferred system of effects to provide a Haskell declaration for All .

I see a lot of speculation here and people speaking past each-other. Why don’t you all go and build something with an effect system, and come back with actual production insights? The conversation would certainly be more productive.

12 Likes

Why don’t you all go and build something with an effect system, and come back with actual production insights?

I think I’ve heard this before…yes, that’s what it was:

…can we just wait until the GHC developers choose one, so it can be deemed as recommended workable?

The second one is tightly coupled to the effect system. It’s impossible to convert to a pure IO action and pass to a non-Eff subsystem … It’s only reusable within Eff framework and unusable outside. Doesn’t look too modular.

This is not true. There is no coupling to the effect system (assuming we’re talking about Bluefin). To see this, note that this function is safe:

tag :: Untagged.Connection -> Tagged.Connection e

so you can define the former write in terms of the latter:

untaggedWrite :: Connection -> Foo -> IO ()
untaggedWrite conn foo = runEff $ \io ->
  taggedWrite (tag conn) foo

Agreed with this. It would be good to establish one.

I disagree with this. Effect systems as a whole are an extremely well understood area of the Haskell world. They’re so well known, in fact, that their significant weaknesses, and potential approaches to ameliorate them, is extensively covered and recovered ground.

However, until the progression from ReaderT IO to effectful that I covered in my talk there had been no approach developed that resolved all the significant weaknesses. effectful does address all the issues of existing effect systems (with one exception: it doesn’t directly support multi-shot continuations – that’s probably fine!) and we know it doesn’t have any additional weaknesses of its own relative to IO because it is just IO. Bluefin inherits these properties.

I suppose one could argue: “but the additional interface that effectful and Bluefin put on IO is too complex”. But firstly, IOE :> es => Eff es a is (almost) just IO a, so you’re never far from the lowest common denominator, and secondly, I don’t believe there is a simpler way to carve out effects from IO. Can you think of one? If not then that’s evidence that carving out effects requires a certain level of additional complexity. If you don’t want that complexity then so be it, but that’s not the same as a proof that effectful and Bluefin are experimental or flawed.

I don’t recall anyone saying effect systems were a pre-requisite for writing basic programs. Can you point out such a claim?

Regarding “this is the future of Haskell”, perhaps you’re referring to my slide “Bluefin is the future of Haskell!” at timestamp 40s of my talk. To be clear, that is not advertising. That is simply my belief. The point of the talk was to justify that belief based on properties of Bluefin. To summarise why that is my belief:

To justify effect systems per se: I believe it is useful in practical programs to “make invalid operations unrepresentable”, i.e. use types to tightly delimit what externally-visible effects a function can perform. This must include, at minimum, state, exceptions and I/O, and it must do so in a composable manner.

To justify IO-based effect systems: it is essential for practical programming that an effect system provide resource safety and easy reasoning about behaviour. I don’t believe this is possible outside IO-based effect systems.

To justify Bluefin (i.e. value level effect arguments) versus effectful (i.e. type level effect arguments), I think it’s simpler and more approachable. (I wouldn’t really be surprised or disappointed if effectful won out over Bluefin. It’s a matter of taste and I’m happy to let the market decide. But the Haskell ecosystem really does IO-based effect systems to displace all others, and I think that’s inevitable.)

EDIT:

Do you also feel the same way about ST, which has the same “polymorphic” property?

8 Likes

That’s good news for effectful and Bluefin then, because you can use them in internal components without effecting external APIs, which can either be pure (if the internal components didn’t use IO) or use IO (if they did).

I agree. Bluefin and effectful can do this, no problem.

This is completely false for Bluefin and effectful because they are just IO wrappers: they are minimal and you can always just unwrap to IO.

8 Likes

I agree, but not everything has to be reusable outside the context it is important in. I mentioned codebase, but that is not good enough. I’d draw a distinction between application/library code. Then I’d agree that for library code (API to be precise), Eff is not it. But for application code, you don’t care for unusable outside.

Noisy types, leaks details, hides details, unmodular, Java factory, infects code. I think those could arguably work. I don’t know if it is worth discussing, though. Just wanted to point out that Bluefin is really close to a record of functions, which is what I like about it.

1 Like

Because I also can’t remember everything:

…to avoid further gratuitous repetition here.

8 posts were split to a new topic: Dialogues vs continuations (and algebraic effects) to implement I/O

As someone who doesn’t have a strong opinion on this topic I’d say ultimately it depends on the kind of applications that you write and on personal preference.

However I’d like to point out another aspect, that I don’t see mentioned in this discussion, namely that Haskell is not a “batteries-included” language. If you consider a language like PHP, which has built-in support for HTTP, database clients and whatnot, Haskell is on the other end of the spectrum. It’s probably the only (almost) mainstream language without even built-in booleans!

This is both the strength and the curse of Haskell. It’s a strength because it makes it flexible and general-purpose. It helped the language to remain relevant for more than 30 years despite not being super-popular. It’s a curse because more functionality is delegated to the libraries, so for each feature that other languages would provide as built-in, we have a myriad of competing solutions and libraries. The whole ecosystem is more distributed compared to other programming language communities.

So, just like the lack of proper built-in record manipulation encouraged the development of optics (which ultimately became much more than just record selectors), we can say that effect libraries were also born from the lack of certain built-in features (and they probably provide more than what built-in features could have provided, at the cost of additional complexity/boilerplate).

Part of the discussion here seems to be just the latest reiteration of this same issue. The tension between vanilla Haskell being “too simple” and libraries being “too powerful”. Finding the sweet spot is hard. It may even be a wrong idea, and maybe the decision should be taken on a case-by-case basis.

7 Likes

There are many ways to achieve mocking without effect systems:

  • use original services but with test data
  • mock services
  • use functions: mockConnection :: IO Connection where Connection has functions inside (Connection {write :: ...} or Connection :: Service a => a -> Connection which is less convenient to use)
  • built-in mocking: data Connection = Real Socket | Mock (IORef (Input, Output))

Mocking worked long before effect systems. Ironically, it’s especially easy to mock with first-class functions.

Hm, if I look at base, text, async, network, unix, warp or http-client packages, I usually see pure functions or IO. What parts or ecosystem have these “convenient” runners?

I worked on a moderate sized production project (~150k lines) with effect system, so I do have some experience which I’m sharing here. It wasn’t effectful, but switching to effectful will still keep a lot of problems (though indeed it will improve performance and simplify the implementation of new effects).

It will work if you have Untagged.Connection. What if you only have Tagged.Connection e? And that’s what usually happen (at least in my experience).

That’s why write :: ... IO () is modular – you can use it standalone, or embed into an effect system, and write :: Eff ... is not (unless it uses a modular IO underneath).

I think you confirmed my “most of them are not production ready or have flaws”. Yes, effectful is a pretty good implementation. But there’s already a Bluefin that tries to replace it. A lot of other systems to come.

I think that Bluefin is still in its early stages. I was mostly talking about myriad of other systems.

That’s my general feeling. I see far too many discussions about effect systems, I see newbies asking which system to use. It’s becoming a matter of course to use an effect system, whereas it’s an advanced and experimental thing with no consensus on which one to use.

Yes, I was referring to this. Maybe if it was “the future of Haskell effect systems” I wouldn’t start all this discussion :slight_smile:

Yes, for performance considerations and to be able to use exceptions, state and concurrency properly, IO-based system looks inevitable.

Though there are many other use cases (like verification) where interpreting a program might be more convenient.

In a sense, yes. But ST is local and has a very strong guarantee – function that uses runST is still pure, I can easily use its pure results everywhere.

Effect systems are usually for managing IO, and there can be a need to connect systems that use Eff and systems that use IO. Here I can have issues connecting the two.

Hm, much like runST. It may work, but it will lead to a bizzare mixed codebase (one part is IO, another is Eff), and may complicate extracting parts of Eff system to a separate IO libraries.

Many libraries are initially part of the application and are extracted later. It’s a sign of good modularity if the application code can be extracted into a library. So it’s worth aiming for IO. Otherwise the Eff code, which could be a future library, would remain coupled to the application.

Hm, it’s still an abstract Connection data type, that is not noisy and doesn’t leak much details. Though yes, it might hide details as it’s not clean which functions are used.

And it’s great. It makes Haskell a vehicle for innovation, and sometimes we get amazing results.

This is one of the things I don’t like about effect systems – they are often proposed as the one true way to write Haskell.

4 Likes
  • Production services with test data is QA territory;

  • Built-in mocking means your declarations and implementations for both the production and the test versions have to live in the same module, whereas they most probably belong to two different executables.

  • Mocking downstream services is something I’d much rather have the service do itself via a sandbox, so that’s QA again;

  • Passing records of functions makes you responsible for juggling all that as extra arguments instead of constraints. It’ll definitely work, the question is just how much inline visual clutter the team you’re working in is willing to endure.

    Though it’s hard to say just how more convenient effect systems are because there definitely is a problem of sharing overlapping effects between components and I don’t know if anyone has solved it yet.

That part of my reply vaguely gestures at the mentality that produces duplications like this bunch and package after package with no refined constituent pieces, like most web-servers, but as it relates to this discussion specifically I think it’s a non-sequitur. (unless “I think everyone should agree on one effect system” is a point worth discussing)

1 Like

I think I finally understood this concern. Effect systems (those named in this topic at least) are not about breaking IO into pieces, they are not even about IO at all. They are about built-in/custom effects and restricting functions to specific effects. Of course, there are performance and ergonomics issues too. One of the ergonomic considerations is providing a way to make effects out of the existing code. Which happens to be Haskell code, and a lot of it lives in IO. So those libraries provide an easy way to make effect use IO, to use IO functions.

Just like effectful provides an easy way to integrate mtl code to make effects. Nothing profound about that at all.

1 Like

Unfortunately effects aren’t always so easily restricted:

Why does this matter?


The latest entry to Philip Wadler’s list of notable quotes:

Now another era is being foisted upon Haskell, an era where someone might have to generate an I/O action out of a library using fused-effects and provide it to a library using speff. Can anyone seriously argue that this is an improvement?