Flexible binds for combining musical patterns

yaxu · August 25, 2023, 8:45am

I’m working on the TidalCycles project for making music with Haskell, and trying to rejig things so they’re more flexible.

Tidal is all about representing musical patterns that can be combined in a variety of ways. There are two instances of a Pattern class, broadly Signal for representing patterns as functions of time (in a FRP manner), and Sequence for representing them as lists of events with durations.

github.com

tidalcycles/Tidal/blob/05274e53c667c5403a1c7c8fd9517bd3d391c6f6/tidal-core/src/Sound/Tidal/Types.hs#L252-L258

    
      
          -- A pattern that's a function from a timespan (along with other
          -- state) to events active during that timespan. A continuous signal,
          -- that can nonetheless contain discrete events.
          data Signal a = Signal {sigMetadata :: SignalMetadata,
                                  query       :: State -> [Event a]
                                 }
            deriving (Functor, Generic)

github.com

tidalcycles/Tidal/blob/05274e53c667c5403a1c7c8fd9517bd3d391c6f6/tidal-core/src/Sound/Tidal/Types.hs#L262-L279

    
      
          -- A pattern as a discrete, contiguous, finite sequence, that's
          -- structured to support polyphonic stacks, and embedded subsequences
          data Sequence a = Atom {atomMetadata :: Metadata,
                                  atomDuration :: Time,
                                  -- If the atom is a fragment, the 'original'
                                  -- duration that it's a fragment of is
                                  -- duration + inset + outset.
                                  atomInset    :: Time,
                                  atomOutset   :: Time,
                                  -- A 'gap' or silence is an atom without a value
                                  atomValue    :: Maybe a
                                 }
                          | Cat [Sequence a]
                          | Stack [Sequence a]
                          | SeqMetadata {seqBindAlignment :: SeqBindAlignment,
                                         seqData          :: Sequence a
                                        }
                          deriving (Eq, Ord, Generic)

There are many ways of combining patterns, particularly with sequences. You can align two sequences in a variety of ways (currently: justifyleft, justifyright, justifyboth, expand, truncateleft, truncateright, truncaterepeat, rep, centre, squeezein, squeezeout), and after that combine them with different strategies that preserve the structure from either the left- or righthand sequence.

However, Haskell seems to work against the idea that there can be different ways of combining things, where a monad only has one bind. If you want different behaviour you have to define a whole different type (as with ZipList). This approach seems too unwieldy for a code interface for making music.

I’m trying to work around this by adding extra fields to the datatypes that say how a pattern should be combined with another. The >>= bind can then use this information for deciding how to combine two sequences (or signals), and the <*> can use it for deciding how sequences should be aligned before being combined.

github.com

tidalcycles/Tidal/blob/05274e53c667c5403a1c7c8fd9517bd3d391c6f6/tidal-core/src/Sound/Tidal/Sequence.hs#L20-L29

    
      
          instance Monad Sequence where
            return = pure
            -- seqv >>= f = seqJoin $ fmap f seqv
            (>>=) a b = (patBind a) a b
          
          
instance Applicative Sequence where
            pure = step 1
            -- pf <*> px = pf >>= \f -> px >>= \x -> pure $ f x
            pf <*> px = pf' >>= (<$> px')
              where (pf', px') = patAlign pf px

With this I can e.g. do the equivalent of (rep [1,2,3]) + [10 20] to repeat both sides to the lowest common multiple [11, 22,13, 21,12, 23], or e.g. (expand [1,2,3]) + [10 20] to expand the shortest sequence to the duration of the longest, which results in fragments of events (which is a bit hard to visualise in text).

Hopefully this isn’t breaking too many monad laws…

Anyway I am a self-taught Haskell programmer and would really appreciate some feedback on this approach. I think this should be a common problem, are there common solutions?

atravers · August 25, 2023, 9:28am

o_- …I 've seen this before:

…if that’s (primarily) the reason why you’re now looking at implementing the monadic interface: I was using Monad as an analogy; I didn’t intend that to be a suggestion or advocacy - apologies for any confusion about that.

yaxu · August 25, 2023, 11:41am

Ah yes this is a continuation of that work. Your advice in that first post was extremely helpful and I’ve run with that. I’ve renamed the types, so that the class is now called Pattern, and the instances are called Signal and Sequence. This all seems to be going very well.

Your second post isn’t the reason why I’m working with monads. Tidal’s pattern representation always has leaned heavily on applicative and monad instances and I’m extending that work.

One motivation for all this is that in TidalCycles, just about function argument can be a pattern. For example the internal function _fast :: Pattern p => Time -> p a -> p a ‘speeds up’ a given pattern by a given time factor. A monadic bind used to lift it into fast :: Pattern p => p Time -> p a -> p a, where the time factor is also ‘patterned’.

github.com

tidalcycles/Tidal/blob/05274e53c667c5403a1c7c8fd9517bd3d391c6f6/tidal-core/src/Sound/Tidal/Pattern.hs#L82

    
      
          overlay :: Pattern p => p a -> p a -> p a
          overlay a b =  cat [a,b]
          
          
_fast, _slow, _late :: Pattern p => Time -> p a -> p a
          _fast t = withTime (/ t) (* t)
          _slow t = withTime (* t) (/ t)
          _late = _early . (0-)
          
          
-- patternify parameters
          fast, density, slow, sparsity, early, late, (<~), (~>) :: Pattern p => p Time -> p a -> p a
          fast  = patternify_P_n _fast
          density = fast
          slow  = patternify_P_n _slow
          sparsity = slow
          early = patternify_P_n _early
          late  = patternify_P_n _late
          -- Infix operator for `early`
          (<~) = early
          -- Infix operator for `late`
          (~>) = late

Which bind is used is decided by looking at the lifted pattern:

github.com

tidalcycles/Tidal/blob/05274e53c667c5403a1c7c8fd9517bd3d391c6f6/tidal-core/src/Sound/Tidal/Pattern.hs#L25-L27

    
      
          patternify_P_n :: Pattern p => (a -> b -> p c) -> (p a -> b -> p c)
          patternify_P_n f apat b = apat `bind` \a -> f a b
            where bind = patBind apat

This results in a really flexible system where you can not only give a pattern of factors to speed up another pattern, but also specify how these two patterns are aligned and combined in order to come up with a result.

I hope this makes some sense!

atravers · August 25, 2023, 1:36pm

Hrm:

…question: is that part of TidalCycles acting like a runtime system for the rest of the project?

yaxu · August 25, 2023, 2:05pm

Hmm I’m not sure what you mean by runtime system in this context.

This part is for making functions for an end-user programmer (live coder) to make music with in this sort of situation:

So maybe the concept of runtime doesn’t really apply, or at least the programmer is part of the runtime.

atravers · August 25, 2023, 2:37pm

Alright, I’ll consider my initial reaction a “red herring” arising from my ol’ implementor experience :-) …back to the current topic!

The lack of flexibility regarding (>>=) has lead to other abstract interfaces e.g. arrows. But not all combinator libraries need this style of interface:

since you mentioned FRP: Functional Reactive Animation (1997)
Composing contracts: an adventure in financial engineering (2000)

Of course, it could also be that no single approach is sufficient - you may need to combine them to achieve your goals.