# Adventures assembling records of capabilities

**URL:** <https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623>\
**Category:** Show and Tell\
**Created:** [April 25, 2019, 10:12pm UTC](https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623 "2019-04-25T22:12:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![danidiaz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/danidiaz/32/92_2.png) [@danidiaz](https://discourse.haskell.org/u/danidiaz)\
**Post date:** [April 25, 2019, 10:12pm UTC](https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623/1 "2019-04-25T22:12:25Z")

</div>

When using the [`ReaderT`](https://www.fpcomplete.com/blog/2017/06/readert-design-pattern) pattern, or a monad like [`RIO`](http://hackage.haskell.org/package/rio), it is common to store capabilities in a record, which is then made available to the program logic as a [`MonadReader`](http://hackage.haskell.org/package/mtl-2.2.2/docs/Control-Monad-Reader.html) environment.

Suppose we have an environment like the following:

```haskell
data Env = Env { 
                    simple :: SimpleCapability,
                    complex :: ComplexCapability
               } deriving GHC.Generics.Generic
instance FromRecord Env -- From "red-black-record", will be useful later
instance ToRecord Env -- Ditto.

```

And capabilities like

```haskell
data SimpleCapability = SimpleCapability Int

data ComplexCapability = ComplexCapability SimpleCapability

```

They are trivial and we don’t care here about what methods they enable. But they do have configuration parameters, and the complex capability depends on the simple one. Both of them are available to this dumb bit of program logic:

```haskell
dummyLogic :: ReaderT Env IO ()
dummyLogic = liftIO $ putStrLn "running the logic!"

```

Let’s assemble an environment:

```haskell
env :: Env
env =
    let simple' = SimpleCapability 7
     in Env { simple = simple', complex = ComplexCapability simple' }

```

Simple enough! But there are two problems with this way of building the environment:

- Our main program logic will read its dependencies from the environment record. However, our complex capabilities won’t read their _own_ dependencies in the same way. Instead, these dependencies are assembled outside the record, in a `let` clause. This lack of uniformity between different levels is somewhat unpleasant, and it doesn’t happen in object-oriented dependency inversion frameworks.

- More importantly, it doesn’t let us easily replace beans (sorry, I meant “capabilities”). If we have a value of `Env` lying around, and we overwrite the `SimpleCapability` field (say, with a mock version for testing purposes) then our main program logic will see the new capability, but our `ComplexCapability` will still see the original version ☹

How to solve this? It’s as if each field of the `Env` needed to have available the fully-constructed final value of the record… as the field itself is being constructed! But this is clearly impossible, so let’s give up. Thus finishes the post.

Just kidding. We can do that, with a bit of judicious [knot-tying](https://wiki.haskell.org/Tying_the_Knot). First we must find a way to give the full `Env` record to each field. We can define an auxiliary `OpenEnv` data type and wrap each field with a `Reader`:

```haskell
data OpenEnv = OpenEnv { 
                         simple' :: Reader Env SimpleCapability, 
                         complex' :: Reader Env ComplexCapability 
                       }

```

And then write a function with type

```haskell
fixEnv :: OpenEnv -> Env

```

that we can call once we are done modifying the fields of `OpenEnv`. The function will tie the knot and return the fully constructed `Env`.

The pattern of wrapping all the fields of a record in a type constructor (`Reader Env` in our case) is sometimes called [Higher-Kinded Data](https://reasonablypolymorphic.com/blog/higher-kinded-data/). We will take the approach of using the [red-black-record](http://hackage.haskell.org/package/red-black-record) and [sop-core](http://hackage.haskell.org/package/sop-core) libraries to auto-derive these generalized representations from our vanilla data type.

So, instead of defining `OpenEnv` manually like we did above, we will define it like this:

```haskell
openEnv :: Record (Reader Env) (RecordCode Env)
openEnv =
      insert @"simple" (pure $ SimpleCapability 7)
    . insert @"complex" (makeComplex simple)
    $ unit

```

Using functions from [red-black-record](http://hackage.haskell.org/package/red-black-record). The `makeComplex` constructor is defined like this:

```haskell
makeComplex :: (r -> SimpleCapability) -> Reader r ComplexCapability
makeComplex getter = 
    do simple' <- asks getter
       pure $ ComplexCapability simple'

```

The constructor receives a getter for simplicity, in a real application it could obtain its dependencies using something like classy lenses or [generic-lens](http://hackage.haskell.org/package/generic-lens).

We also need auxiliary functions that allows us to “tie the knot” and get an `Env` as the result:

```haskell
fixRecord 
    :: forall r flat. (FromRecord r, Productlike '[] (RecordCode r) flat, 
                       All Top flat)
    => Record (Reader r) (RecordCode r)
    -> r
fixRecord = unI . fixHelper I

fixHelper 
    :: forall r flat f g. (FromRecord r, Productlike '[] (RecordCode r) flat,
                           All Top flat,
                           Functor g)
    => (NP f flat -> g (NP (Reader r) flat))
    -> Record f (RecordCode r)
    -> g r 
fixHelper adapt r = do
    let moveFunctionOutside np = runReader . sequence_NP $ np
        record2record np = fromRecord . fromNP <$> moveFunctionOutside np
    fix . record2record <$> adapt (toNP r)

```

These functions depend on other functions from [sop-core](http://hackage.haskell.org/package/sop-core), basically versions of [sequence](http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Monad.html#v:sequence) and other `Applicative` operations that have been generalized to work over [n-ary products](http://hackage.haskell.org/package/sop-core-0.4.0.0/docs/Data-SOP.html#t:NP).

If we overwrite the `simple` field of `openEnv` with a mock capability, and pass the modified record to `fixRecord`, it will do the right thing and propagate the changes to the `ComplexCapability`, which will use the mock just as the main program logic will.

```haskell
do let closedEnv = fixRecord openEnv 
   runReaderT dummyLogic closedEnv

```

That was nice. Now suppose—as it is often the case—that capabilities must allocate some resource that will be used for the duration of the computation. Perhaps a file handle for logging, or even some [background thread](http://hackage.haskell.org/package/async-2.2.1/docs/Control-Concurrent-Async.html#v:async).

How to incorporate this? We could wrap the construction of the environment in [`bracket`](http://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception.html#v:bracket)-like operations. But this is unsatisfactory:

- It disperses the code related to each capability: it moves the allocation away from where the capability is inserted in the record.

- Even worse: it makes the allocation _precede_ the construction of the record. We won’t have these nice, configurable “open environment” values directly at hand anymore, they will be hidden behind the allocation actions.

We can try a different strategy: putting the allocation code of each capability directly in the corresponding field of the open environment. How? By turning to the [managed](http://hackage.haskell.org/package/managed) library and the magic of applicative functor composition:

```haskell
managedOpenEnv :: Record (Managed :.: Reader Env) (RecordCode Env)
managedOpenEnv =
      insert @"simple" (Comp $ pure $ pure $ SimpleCapability 7)
    . insert @"complex" (makeManagedComplex simple)
    $ unit

-- A capability constructor that performs an allocation
makeManagedComplex 
    :: (r -> SimpleCapability) -> (Managed :.: Reader r) ComplexCapability
makeManagedComplex getter = 
    Comp $ managed $ \cnt -> bracket_ (putStrLn "activating")
                                      (putStrLn "deactivating")
                                      (cnt $ makeComplex getter)

fixManagedRecord
    :: forall r flat. (FromRecord r, Productlike '[] (RecordCode r) flat, 
                       All Top flat)
    => Record (Managed :.: Reader r) (RecordCode r)
    -> Managed r
fixManagedRecord = fixHelper sequence'_NP

```

[`:.:`](http://hackage.haskell.org/package/sop-core-0.4.0.0/docs/Data-SOP.html#t::.:) is a version of functor composition defined in [sop-core](http://hackage.haskell.org/package/sop-core). [`sequence'_NP`](http://hackage.haskell.org/package/sop-core-0.4.0.0/docs/Data-SOP-NP.html#v:sequence-39-_NP) is also from sop-core, it “pulls outward” one layer of the composition.

When we “fix” the `managedOpenEnv` using `fixManagedRecord`, we get a `Managed` action that we can run. If we modify a field before the “fix”, the allocations specified by the old field value won’t be performed. Configuration precedes allocation, as it should be!

```haskell
with (fixManagedRecord managedOpenEnv) (runReaderT dummyLogic)

```

Notice the following limitation though: the capabilities specify their allocations actions _before_ reading their dependencies from the environment. They can’t inspect their dependencies to decide what allocations to perform.

Anyway, now that we have started to compose applicatives, the sky is the limit! We could, for example, make each capability constructor carry around a parser for its own configuration. These parsers would be then assembled in a parser for a global configuration object.

* * *

What are the disadvantages of the approach described in this post?

Besides the complex types, there’s the usual bane of extensible record libraries: [long compile times](https://github.com/danidiaz/red-black-record/issues/12).

Also, this form of dependency injection is purely name-based: we wire capabilities according to their field names in the environment record. If instead we wanted smarted auto-wiring which used type information ("there’s only one available capability with type `Foo`, and this other capability needs a `Foo`") a library like [registry](http://hackage.haskell.org/package/registry) would better.

The gist with the full code is available [here](https://gist.github.com/danidiaz/40e1b380ce842d7423acc7e38ff341e4).

---

<div class="post-metadata">

**Author:** ![blamario](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/blamario/32/860_2.png) [@blamario](https://discourse.haskell.org/u/blamario)\
**Post date:** [April 26, 2019, 1:15pm UTC](https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623/2 "2019-04-26T13:15:04Z")

</div>

It’s nice to see more of the higher-kinded data knot-tying technique in practice. If you want to see another application, the [fixGrammar](http://hackage.haskell.org/package/grammatical-parsers-0.3.2/docs/Text-Grampa.html#v:fixGrammar) function from [grammatical-parsers](http://hackage.haskell.org/package/grammatical-parsers) is the equivalent of the `fixRecord` above except for rank-2 records of parsers / grammar productions. It makes extensible left-recursive grammars wonderfully easy to construct.

---

<div class="post-metadata">

**Author:** ![stevenxl](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/stevenxl/32/301_2.png) [@stevenxl](https://discourse.haskell.org/u/stevenxl)\
**Post date:** [May 2, 2019, 11:07pm UTC](https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623/3 "2019-05-02T23:07:15Z")

</div>

This is an interesting exercise, but if I was having the problem in this post, I would take a page from the [ReaderT Design Pattern](https://www.fpcomplete.com/blog/2017/06/readert-design-pattern) and abstract the way that the `ComplexCapability` is constructed:

```haskell
module Misc.Complex where

import Control.Monad.Reader (MonadReader, ask)
import Control.Monad.IO.Class (MonadIO, liftIO)

data SimpleCapability = SimpleCapability Int
data ComplexCapability = ComplexCapability SimpleCapability
data Env = Env {simple :: SimpleCapability }

class HasComplex a where
  getComplex :: a -> ComplexCapability

instance HasComplex Env where
  getComplex env = ComplexCapability (simple env)

dummyLogic :: (MonadReader env m, HasComplex env, MonadIO m) => m ()
dummyLogic = do
  env <- ask
  let complex = getComplex env
  liftIO $ putStrLn "running the logic!"

```

Maybe this simply doesn’t scale?

---

<div class="post-metadata">

**Author:** ![danidiaz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/danidiaz/32/92_2.png) [@danidiaz](https://discourse.haskell.org/u/danidiaz)\
**Post date:** [May 3, 2019, 5:50pm UTC](https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623/4 "2019-05-03T17:50:53Z")

</div>

That couples the definition of your `Env` type with how its fields are constructed. Sometimes that’s not what you want. For example, suppose `ComplexCapability` were defined like this:

```haskell
data ComplexCapability = ComplexCapability { doComplexStuff :: Int -> IO () }

```

The internals are now hidden behind a function. This `ComplexCapability` could be built in multiple ways. Some might require `SimpleCapability`, some others (some mock implementation for testing perhaps) won’t. If you want to reuse the `Env` type with all of them, you can’t bake the `ComplexCapability` constructor in the typeclass.

---

<div class="post-metadata">

**Author:** ![mpickering](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/mpickering/32/4585_2.png) [@mpickering](https://discourse.haskell.org/u/mpickering)\
**Post date:** [May 4, 2019, 5:53pm UTC](https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623/5 "2019-05-04T17:53:14Z")

</div>

I understand the problem and think it’s an interesting one to tackle but I couldn’t follow the post very easily after `openEnv` was introduced. (For example, how is `simple` brought into scope and what is its definition).

[This post](https://www.well-typed.com/blog/2018/03/oop-in-haskell/) by Edsko explores a similar problem but without the complicated machinery.

---

<div class="post-metadata">

**Author:** ![danidiaz](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/danidiaz/32/92_2.png) [@danidiaz](https://discourse.haskell.org/u/danidiaz)\
**Post date:** [May 5, 2019, 6:55am UTC](https://discourse.haskell.org/t/adventures-assembling-records-of-capabilities/623/6 "2019-05-05T06:55:11Z")

</div>

`simple` is a field accessor from the `Env` type. The idea is that each field of the `openEnv` has access to the final `Env` record that we intend to construct, and from it extracts its own dependencies.

[`Record`](http://hackage.haskell.org/package/red-black-record-2.0.2.2/docs/Data-RBR.html#t:Record) has two type parameters: one is a type constructor (usually some `Applicative`) with which we want to wrap each one of the fields. In the case of `openEnv` the `Applicative` is `Reader Env`, meaning that each field is actually a function from `Env` to the type associated to the field.

The second type parameter is a [type-level map](http://hackage.haskell.org/package/red-black-record-2.0.2.2/docs/Data-RBR.html#t:Map) from field [`Symbol`](http://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-TypeLits.html#t:Symbol)s to [`Type`](http://hackage.haskell.org/package/base-4.12.0.0/docs/Data-Kind.html#t:Type)s, that says which fields are in the `Record`. We want it to mimic the structure of `Env`, so we extract the latter’s type-level map using the [`RecordCode`](http://hackage.haskell.org/package/red-black-record-2.0.2.2/docs/Data-RBR.html#t:RecordCode) type family.