Looking for a library like `concurrency` but with file IO operations

#1

For a demo I’m looking for a library like concurrency but the typeclass should include methods for opening files, and file descriptor operations (for reading, writing, seeking, flushing buffers etc.).

Does something like this exist? I’ve been searching in hackage and hoogle but no luck…

Thanks.

#2

Are you strictly looking for a typeclass-based approach, or are you fine with using an effect system? effectful comes batteries-included with a bunch of effects, including filesystem-related ones.

1 Like
#3

Effect systems are also fine, but I was hoping to use dejafu for testing my code written with concurrency. Does something like dejafu already exist for effectful?

#4

Unfortunately, I don’t know anything of such for the effectful ecosystem. Though, now that I understand your use case a little bit more, maybe I can suggest taking a look into the io-sim package (and related set of libraries including io-classes). The package description states (emphasis mine):

io-sim together with io-classes is a drop-in replacement for the IO monad (with some ramifications). It was designed to write easily testable Haskell code (including simulating socket programming or disk IO)

1 Like
#5

fs-sim: Simulated file systems might fit your needs

1 Like
#6

Thanks for the suggestions. io-classes doesn’t have file IO. fs-api seems to be the file IO equivalent of it, except it doesn’t use a monad typeclass, instead uses a data type… (Why is the inconsistency? They seem to be developed by the same company.)

effectful seems to have it all and in one package. Perhaps that’s the way to go.

Anyway, thanks again!

#7

If I could understand your use case then I’d be happy to add support to Bluefin. If you’re inclined to explain to me what you need then please do.

#8

Probably a design choice. HasFS could indeed be a type class but Type Class <-> Record type, its not that much of a difference when it comes down to it. Passing a HasFS to your functions or requiring HasFS constraint (if it were a type class) is just a matter of preference.

#9

A bit unrelated, but concurrency (and probably any other such library that does not follow upstream Control.Concurrent faithfully) is a questionable library to use in production.

Yes, you can check that some events are handled correctly, but that means you’re forever tied to a simplified copy of Control.Concurrent that hasn’t been updated since 2020.

This means that:

  • concurrency doesn’t have 5 years (and counting) worth of bugfixes
  • concurrency has a different behaviour from a stock concurrency libraries.

This last point is extremely dangerous.

For example, the stock concurrently from async uses throwTo ... AsyncCancelled, while the bare-bones copy of concurrently in concurrency uses killThread. It’s a slight but very significant change in a behavior that breaks code written for stock libraries.

I worked on a code “proven correct” by concurrently, but when I added a simple no-brainer code that worked perfectly with Control.Concurrent I was unpleasantly surprised to find that it was broken with concurrently as it has an outdated and broken behavior.

Think twice before using concurrently and dejafu. The good old engineering approach of writing the simplest possible concurrent code that is obviously correct (can be proven in your head without any model checking) may be a much better option for both correctness and maintainability.

#10

killThread is just throwTo ... ThreadKilled, and it’s very worrying if changing AsyncCancelled to ThreadKilled can change behaviour. Neither of those exceptions should be caught by or interacted with by a thread in any way. They should just bubble up to the top and kill the thread. In what way did you see a difference in behaviour?

#11

The stock link handles AsyncCancelled differently:

link ignores AsyncCancelled exceptions thrown in the other thread, so that it’s safe to cancel a thread you’re linked to.

The behaviour was changed in Ability to cancel a linked thread without killing self · Issue #25 · simonmar/async · GitHub after the concurrently development stopped.

I had something like this:

withAsync (threadDelay ... >> error "timeout") $ \ timeout -> do
  link timeout -- will kill the current thread on a timeout
  connectToSomeService
  ... 
  cancel timeout -- we're fine there, the timeout is no longer needed
  ...

But cancel killed the current thread (and I suspect exiting from withAsync may throw an unexpected exception as well).

IIRC, this whole withAsync ... link was created because of concurrently didn’t reimplement System.Timeout.timeout.

So it’s a combo: I can’t use the standard functions, and when I try to imitate them, I run into unexpected 5-year-old behaviour.

#12

Very interesting! Thanks for explaining.

There are two possible consequences to this, and I’m not sure which you’re objecting to. (Maybe both.)

Firstly, concurrency's link doesn’t have the desirable behaviour. It’s nothing to do with the choice of ThreadKilled versus AsyncException, it’s just to do with whether it suppresses whatever exception it uses from being rethrown to the parent.

Secondly, if you killThread (via asyncThreadId) an async-linked thread, the ThreadKilled will be thrown to you when the linked thread terminates. That’s bad!

It can happen because Async is an abstract wrapper around a ThreadId (and result/exception). Having a ThreadId is a strong level of control over a thread! What would happen if we didn’t have

asyncThreadId :: Async a -> ThreadId

That sounds like it would be better encapsulation.

So, I’m not sure exactly what you’re objecting to, whether it’s firstly that the design of concurrency itself is lacking, or whether, secondly, it composes poorly with other libraries.

Having said that, perhaps “significant change in a behavior that breaks code written for stock libraries” means that code written against the async API won’t work when ported directly to the concurrency API.

Thanks!

#13

Firstly, concurrency's link doesn’t have the desirable behaviour. It’s nothing to do with the choice of ThreadKilled versus AsyncException

It has very much to do with it. How else would you implement a proper link that propagates exceptions to the parent thread yet allows the linked Async to be cancelled.

My mistake was to link to concurrently instead of cancel which made my ThreadKilled point less clear (more on why I originally linked to concurrencly below).

Secondly, if you killThread (via asyncThreadId) an async-linked thread, the ThreadKilled will be thrown to you when the linked thread terminates. That’s bad!

No, that the whole point of link – propagate exceptions to the main thread.

Even without asyncThreadId there was a design gap (I would rather call it a bug) when it was impossible to cancel the linked thread.

The stock library, actively used and maintained, got a bug fix. Stale concurrency didn’t.

If you compare both concurrently versions I linked above you could find that not only they throwTo different exceptions, but that the stock version has much more code which handles corner cases not handled in the stale concurrency.

So my objections:

  • The verification library does not have bugfixes which defeats its purpose.
  • It also has a different behavior which breaks programs written for stock libraries. The very programs it supposed to verify.
  • Due to the above, experience with stock Haskell libraries doesn’t transfer to the world of concurrency leading to bugs and frustration for new developers.
  • Stale MonadConc primitives do not match stock implementation primitives leading to even subtler differences.
  • Mixing concurrency and a monad transformer can lead to extremely convoluted and unexpected execution traces.

In general, concurrency is definitely a good topic for PhD, might be good to temporarily run some code with it and check for possible bugs. But I would be very wary of running a production system on it.

If it was actively maintained and faithfully updated to match the standard library then it might be ok. But it’s stale, so I suggest to use standard libraries and keep concurrenct code simple enough to be model checked in one’s head (helps with maintenance too).

#14

Well, perhaps I’m misunderstanding what you’re saying.

To me it seems that ThreadKilled and AsyncException are two arbitrary exceptions that are thrown to threads to indicate that they should terminate, by two different libraries. The choice of particular exception is completely irrelevant. They could be called Foo and Bar as long as they’re used consistently. I believe they are used consistently by their respective libraries, so the only inconsistency can arise if someone attempts to use the libraries together.

But perhaps your point is that concurrency should stay up to date with async, the library it is trying to generalize. That makes sense to me.