Haskell's missing mutable reference type

Links
1

Whilst looking at the Scoped thread-locals GHC proposal I realised that Haskell is missing a mutable reference type that some other languges have (and people have tried to encode in Haskell), so I wrote it up:

5 Likes
Fork-fragile reader-like operations in Haskell
A reference implementation of IOScopedRef
2

So, I’m guessing the use case here is that you have a computation that already somehow captures the IOScopedRef, as in

do
    ref <- newIOScopedRef 0
    let printCurrent = do
              current <- readIOScopedRef ref
              print current
    modifyScopedIORef ref (+1) do
         ...
         printCurrent -- implicitly uses the captured 'ref'

(otherwise I’m not sure how this would be different from just using a regular variable and shadowing it)

Do you have an example where this is useful to have?

1 Like
3

Right.

“A logging library” from the article is supposed to provide such an example. Is there something I could do to make that clearer? (Also feel free to skip forward two articles in the series to “Fork-fragile reader-like operations in Haskell” to see more.)

1 Like
4

Unless I’m totally misreading something, the logging example doesn’t capture the logger. Everything that accesses it also takes it as a parameter. So it’s really equivalent to

loggerExample :: IO ()
loggerExample = do
  let logger = newLogger 0
  logMsg logger 1 "Getting user"
  user <- getUser
  logMsg logger 1 ("Is VIP: " <> show (isVip user))
  let modification = if isVip user then (+ 10) else id
  d <- do
    -- Instead of overriding it with a continuation, we can just shadow the variable
    -- (the (<-) and pure are necessary since regular lets are recursive and don't allow shadowing)
    logger <- pure (modifySeverity logger modification)
    logMsg logger 0 "Getting data"
    getData user
  writeData d
  logMsg logger 0 "Done"

and then I don’t see the advantage of passing this additional reference around if all you’re doing with it is modifying it in a lexically scoped way like you already can with regular variables

1 Like
5

Well, the definition of Logger does not use IOScopedRef directly. Logger could be defined in a module that doesn’t even know IOScopedRef exists. The IOScopedRef is captured in the closures that are stored in the Logger value, isn’t it? If not, what do you mean by “capture”?

Sure, if you inline and use shadowing you can obtain similar behaviour. But what about when you can’t inline everything, because you’re trying to combine code from two independent libraries?

I don’t see the advantage of passing this additional reference around if all you’re doing with it is modifying it in a lexically scoped way like you already can with regular variables

OK, then tell me how I could obtain the same behaviour using a lexically-scoped variable but without IOScopedRef.

6

Oops, i did misread the example slightly sorry ^^ You’re using the IOScopedRef to implement modifySeverity not to modify the Logger value directly.

My point still stands though. Everything that (indirectly) uses this IOScopedRef needs to go through the logger variable anyway so you might as well use it to carry the state in a lexically scoped way.

OK, then tell me how I could obtain the same behaviour using a lexically-scoped variable but without IOScopedRef.

data Logger = Logger {
    logImpl :: Severity -> String -> IO (),
    baseSeverity :: Severity
}

logMsg :: Logger -> Severity -> String -> IO ()
logMsg logger severity msg = logger.logImpl (logger.baseSeverity + severity) msg

newLogger :: Severity -> Logger
newLogger baseSeverity = Logger { baseSeverity, logImpl = \severity message -> putStrLn ... }

modifySeverity :: Logger -> (Severity -> Severity) -> Logger
modifySeverity logger f = logger { baseSeverity = f logger.baseSeverity }

loggerExample :: IO ()
loggerExample = do
  let logger = newLogger 0
  logMsg logger 1 "Getting user"
  user <- getUser
  logMsg logger 1 ("Is VIP: " <> show (isVip user))
  let modification = if isVip user then (+ 10) else id
  d <- do
    logger <- pure (modifySeverity logger modification)
    logMsg logger 0 "Getting data"
    getData user
  writeData d
  logMsg logger 0 "Done"
2 Likes
7

OK, I think I probably understand what you mean. You mean that if you’re passing the logger around everywhere you may as well replace all instances of

modifySeverity logger f $ do
  ...

with

logger <- modifySeverity logger f
...

and so you’re asking for an example where you can’t do that transformation. Sure. First, notice that you can only do that transformation if you can see the call to modifySeverity. So the answer is that you can’t do that transformation anywhere that modifySeverity is being used in a way that the caller can’t see. For example, suppose I have a function from another library with this fype

libFun ::
  (String -> IO ()) ->
  (forall r. Int -> IO r -> IO r) ->
  IO T

Then if I have a logger :: Logger implemented with an IOScopedRef I can call libFun like

libFun
  (\s -> logMsg logger 0 s)
  (\i body -> modifySeverity (const i) body)

There’s no way I can do that with lexical scoping. libFun doesn’t know that modifySeverity might be used within it! I’m guessing that’s what you meant by “having a computation that already somehow captures the IOScopedRef”?

Yeah, I think didn’t explain well what I meant. There are two subtly different questions:

  1. Can you get something with the same API as Logger but without using IOScopedRef? (i.e. modifySeverity takes an explicit body)

    I think the answer is “no” and I suspect you don’t disagree with me, but if not then please do say because that would be very interesting!

  2. Can you get something with a different API to Logger that does the same job in some cases? (i.e. modifySeverity doesn’t take a body, rather the modification is observable in the continuation)

    The answer is “yes”, as you demonstrate above.

The difference is important because only something with the same API (i.e. modifySeverity takes a body) can be used with libFun.

Your original question was

Do you have an example where this is useful to have?

My claim is that Logger is an API that is unachievable without IOScopedRef. That is, my loggerExample cannot be written as written without IOScopedRef. Your original point was: but it can be written in a different way without IOScopedRef. That’s a fair point. I can say “the API itself is useful” but you could always come back and ask why it’s more useful than the lexical scoping way. In which case, the response would be that the lexical scoping way is not sufficient to integrate with libFun. (Then someone might say "well libFun should be different, which may be so, but I’d like to know how.) I’ll see if I can think of a way of making this clear in the article.

EDIT: Oh, now I’ve remembered what I originally meant, but I forgot the exact code in my article, so what I said didn’t quite make sense. What I meant was that this is an example of the sort you’re looking for, I think:

loggerExample :: IO ()
loggerExample = withStdoutLogger 0 useLogger

useLogger :: Logger -> IO ()
useLogger = \logger -> do
  logMsg logger 1 "Getting user"
  user <- getUser
  logMsg logger 1 ("Is VIP: " <> show (isVip user))
  let modification = if isVip user then (+ 10) else id
  d <- modifySeverity logger modification $ do
    logMsg logger 0 "Getting data"
    getData user
  writeData d
  logMsg logger 0 "Done"

Specifically, suppose the definition of Logger and useLogger are given to me; I can’t change them. How can I instantiate Logger so that it has the behaviour I want? I don’t think I can unless I have IOScopedRef.

EDIT: OK, I split off writeUserData from the original example, so at least that is a function I can’t call the way I want without IOScopedRef.

1 Like
8

Okay, I had to spend 40 minutes figuring out what this is, and I’m still not sure.

Am I correct to understand that the problem you’re looking to solve is that of a user-definable implicit global context, which can be adjusted locally, e.g.

overriding :: Contextual a -> (a -> a) -> (forall b. IO b -> IO b) -- deeply magical

contextual x :: Contextual Int -- still magical
contextual x = 5

foo = do
  a <- getContextual x
  print a

--- Library user doesn't get to implement anything above this line

-- Prints number 7
main = overriding x (+2) foo

I assumed if a feature like this were to exist it’d be on the compiler to collect all the “contextual” references upfront. This wouldn’t be mutable from user perspective, so I wouldn’t bring up IORef at all when talking about it.

2 Likes
9

Thanks for the feedback!

Maybe the reference implementation in the next article (linked at the top of the current one) will help?

Yes.

I’m not sure what that means. Can you elaborate?

FWIW, this is not allowed in my API. I suppose an API that allows it may be reasonable nonetheless.

Why is it not “mutable from the user perspective”? The value of x “mutates” here, doesn’t it?

a <- getContextual x
print a
overriding x (+2) $ do
  b <- getContextual x
  print b
10

It’s mutating if you look at the flow imperatively, but that would be an implementation detail. I instead think of x as a stack of changes over the initial value; no matter what I do, upon reaching the end of an overriding function x will be exactly the same as it was immediately before entering it.

In this regard it’s no more a mutation than let y = (+2) 5.

My naive assumption is that a feature like this would be implemented by having every worker thread carry around a table of these kinds of references, and the size of the table would be determined at compilation time.

Every contextual declaration would then be a reservation for a seat at the table.

1 Like
11

That’s a fine point of view. I don’t have a strong view on what “mutable” should mean, but I think it’s interesting to know that the fact that IORef is implemented by actually mutating the value stored in a piece of memory is also an implementation detail. According to The Key Monad it also has a completely pure implementation.

Ah right, the size doesn’t have to be determined up front. It can grow dynamically.

12

Yes, but that would require a completely different implementation. No matter how you slice it these references will have to live in CPU cache, and a static table is the most compact solution to that.

Much like with [RFC] Mutable records as a GHC extension, I view a simple solution that fits the existing model as infinitely more preferable to both the status quo and the “dependent types are just a week away” line of approach.

1 Like
13

I’m not sure exactly what implementation you’re thinking of, but a table which grows dynamically is essential to implement withIOScopedRef, which allows references to be allocated dynamically. However, that variable-size table can be implemented as a Vault inside a fixed-size table: A reference implementation of IOScopedRef

14

And that’s a feature you’ll have to argue for.

Per my earlier example the piece of code we’re looking to allow for is

main = overriding x (+2) foo

such that overriding x modifies the behavior of foo without passing x to foo.

Your implementation uses ReaderT Vault, which, as others have pointed out, is isomorphic to passing Vault to every single function, i.e.

main vault = overriding vault x (+2) $ foo vault
1 Like
15

I’m still not quite sure what you mean. Yes, withIOScopedRef in particular, and IOScopedRef in general, is a feature I am arguing for! Part of its essential behaviour is to allow references to be allocated dynamically, just like an essential feature of IORef is that they can be allocated dynamically.

But suppose I had your contextual. I could define

contextual vault :: Contextual Vault
contextual vault = Vault.empty

to reconcile my interface with your implementation, couldn’t I?

(This is essentially what A reference implementation of IOScopedRef suggests: one global contextual variable, which is a vault.)

16

If the value is dynamically allocated, then how do you use it inside a function without passing it? And if you’re passing a value, why not determine the wanted behavior based on passed value instead? The following is obviously not valid Haskell:

module Foo (foo) where

withIOScopedRef 5 $ \x ->
  let foo = _ x

The point here is that x is not a value, it’s a place. foo expects a value to be there when it executes, user knows they can augment it.

1 Like
17

I think I see: you are making the claim that dynamically allocating such references is not useful?

18

Slightly stronger, I don’t see how you can implement that without breaking some fundamental part of the language (I think referential transparency, but I don’t have the background to concisely describe it).

1 Like
19

Well, I explained how I can get my API from yours.

And another potential implementation is explained here: A reference implementation of IOScopedRef

20

Yes, and I explained that I have no idea what “reading from the vault” would look like. I’m essentially getting

contextual vault :: Contextual Vault

foo :: IO ()
foo = do
  a <- _ vault
  print a

---

main =
  withScopedIORef vault 5 $ \x ->
    overriding x (+2) $
      foo

with the questions:

  • How is foo expected to access x without breaking type safety? Passing a key is not an answer, that’s still data.

  • What happens if main = foo?

I guess you could solve both of these if IO is an effect that carries x at the type level, but that would require proper type-level programming implemented first.

1 Like