I am convinced that Backpack addresses an important problem. I think the primary reason it has remained “niche and unused” is that Backpack lacks a champion: a person (or preferably several) who is excited to get into the details, fix the UX, write blogs and documentation, and generally knock down the barriers to entry.
Maybe someone reading this thread feels motivated to step up to that challenge?
Simon
I bet that Backpack error messages are way harder to understand
I was curious, so I took this example and introduced some errors to see what they looked like.
Leaving the Str module signature unfilled here:
mixins:
...
lesson2-signatures (Lesson2 as Lesson2.Text)
The error:
Non-library component has unfilled requirements: Str
In the stanza 'executable lesson2'
In the inplace package 'lesson2-signatures-1.0.0.0'
Giving an implementation type here a kind different to what the module signature specifies:
type Str = Maybe
splitOn :: Char -> Maybe () -> Maybe ()
splitOn c = undefined
The error:
• Type constructor ‘Str’ has conflicting definitions in the module
and its hsig file
Main module: type Str :: * -> *
type Str = GHC.Maybe.Maybe :: * -> *
Hsig file: type Str :: *
data Str
The types have different kinds
• while checking that lesson2-signatures-1.0.0.0:impl-text:Str.Text implements signature Str in lesson2-signatures-1.0.0.0[Str=lesson2-signatures-1.0.0.0:impl-text:Str.Text]
Removing the function splitOn from the implementation module here so that it can’t correctly instantiate the module signature.
The error:
lib/Str.hsig:1:1: error:
• ‘splitOn’ is exported by the hsig file, but not exported by the implementing module ‘lesson2-signatures-1.0.0.0:impl-text:Str.Text’
• while checking that lesson2-signatures-1.0.0.0:impl-text:Str.Text implements signature Str in lesson2-signatures-1.0.0.0[Str=lesson2-signatures-1.0.0.0:impl-text:Str.Text]
|
1 | signature Str where
Those first two are pretty good error messages. The last one could use some improvement. Edit: I didn’t read correctly that you actually deleted the whole function. In that case the last message also seems to be accurate.
I think it would be interesting if someone actually were to rewrite that CPP code of filepath/bytestring etc. with backpack and demonstrate that it’s nicer and easier to maintain.
The other argument that comes up frequently is a common API for containers without the overhead of typeclasses.
I think @ChShersh has blogged about it here: Picnic: put containers into a backpack :: Kowainik
I still find most of the examples not too convincing. Maybe I Iack imagination?
If we’re spitballing alternative methods of achieving backpack…
You could possibly also use convenience internal libraries, make them share the same source module that you want to have multiple implementations, but each import their parameter (types, implementations) from a module in isolated source directories. To instantiate them you’d use PackageImports. No CPP required (although could be used).
That would serve the local project case, but wouldn’t serve the open world case that backpack does.
At my previous job on a smaller codebase with a performance-constrained simulator, we wanted to use backpack.
The reason we didn’t is because HLS wouldn’t work well with backpack, and we relied on HLS to be productive.
Beyond that we were also concerned about sharp edges that might bite us. In the end we chose dependency injection via passing records (which has a runtime overhead ofc).
Arguably GHC could have specialised polymorphic types to reap benefits of unpacked strict fields automatically.
Have you seen any comprehensive benchmarks for Backpack-ed containers? Tree-like structures eat a ton of extra memory anyways, so memory savings from unboxing keys are unlikely to be significant. The real benefit could come from instance Ord key inlined and operating on unboxed Int#, but you get the same with IntMap (if you keys are small enough) or HashMap (if your keys are larger).
What about GC pause gains? It seems to me it would cut the amount of scanning the GC does for live containers. For most workloads and use-cases, it’s probably not interesting. But I suspect it can make a difference for large containers that are dynamic (cannot use compact) + have actual tight latency budgets (e.g. not your average Haskell production web app).
I remember this message by @mpickering about Backpack:
In my understanding, if a feature is barely used but causes major pain for GHC developers, it should be ripped off.
Haskell doesn’t need a kitchen-sink of all possible features. Haskell needs a good Product Manager.
There’s a long list of features that I’d like removed from GHC, but strangely backpack isn’t one of them. I think it desperately needs someone to advocate for it and to help improve the existing tooling (and perhaps that might even be me).
Backpack adds something to Haskell that I think is of value - it allows me to write a library where the user of the library can supply a typeclass.
For example I can write a library of utilities for higher kinded data types, and leave it up to the user whether my functions require BTraversable from barbies or FTraversable from hkd, or some other equivalent typeclass I didn’t even know existed.
I just need (in bkp’s single file format without anything instantiating it and completely untested)
unit my-hkd-base where
module My.HKD.Base where
-- I supply my own typeclass
class MyTraversableF t where
myTraverseF :: Applicative f => (forall a . h a -> f (g a)) -> t h -> f (t g)
-- and a newtype to wrap my things so they can be lifted into your typeclass
newtype MyTraversable t f = MyTraversable (t f)
unit my-hkd-sig where
dependency my-hkd-base
signature HKDSig where
import My.HKD.Base
-- You give me your typeclass, as well as the function(s) I need from it
class F (t :: (* -> *) -> *)
traverseF :: (F t, Applicative f) => (forall a . h a -> f (g a)) -> t h -> f (t g)
-- And supply a instance to lift my typeclass into yours (probably requires UndecidableInstances)
instance (MyTraversableF f) => F (MyTraversable f)
module My.HKD.Stuff where
import Data.Functor.Identity
import HKDSig
import My.HKD.Base
-- and the functions I provide use your specified typeclass
runIO :: F t => t IO -> IO (t Identity)
runIO = traverseF (fmap Identity)
Yes! Maybe it could be you! That would be so great.
I have a use-case for backpack at work, but it was discussed that the tooling experience would make everyone unhappy, so it probably won’t be used.
One potential use of Backpack that I haven’t seen in the wild is to manage the size of compilation graphs in development workflows.
If the code you’re working on explicitly describes what it expects from upstream dependencies, you no longer have to compile those dependencies in order to typecheck your own code! This is incredibly effective for establishing fast and ergonomic development workflows in a language like Haskell, where dependencies can take massive amounts of time to compile and impose a drag on ergonomics.
I think Backpack (or an evolution of it) has a critical role to play in a future where Haskell code on the web can be examined and traversed semantically (i.e. with features like type on hover or go to definition).
I also agree with several other commenters here that the problem isn’t with the basic idea of having “a Backpack” in Haskell, but with various ancillary engineering details that affect how easy it is to put it to good use. For example one concrete bit of tooling that would be very useful to have in a Backpack user’s arsenal would be a signature inference tool: i.e. a tool that can take an existing package or compilation unit and infer a signature describing what it produces, as well as signatures describing what it demands from its dependencies.
Another way to avoid compilation and only typecheck it to use -fno-code, like --ghc-options="-fforce-recomp -fno-code"
Good point. But if I understand correctly that would still require you to typecheck your dependencies, even if no code is generated… To me it feels like a killer feature of signatures is that working on our own project doesn’t entail even knowing how to obtain the source code for a dependency, provided there’s a trustworthy signature that abstracts it.
Originally part of the motivation of Backpack was that it would be possible to do truly separate compilation. Then you wouldn’t have to compile your dependencies at all (until you actually want to run the program). But I believe that idea has been dropped to to support optimisations that look through the interfaces.
@jaror Hmm. Maybe I am misunderstanding something. But I took it that in theory you can take pretty much any package and divorce it from its dependencies (for the purposes of static analysis, not execution) by adding enough signatures (in signature packages or otherwise). Yes, you do need enough of the kind and type signatures, and “type level implementation”, but no term level definitions are required. Of course there are edge cases like TH consumed downstream, but wouldn’t this still work for most typical packages?
Ah sorry, nevermind my previous comment. I understand what you mean now, I assume you mean it would be possible to produce a “partial” executable or shared library (which could be linked with the other bits) to produce an executable.
Yes, It works for type checking but not for optimization and code generation.
That makes sense. I’m probably in the minority (and pretty naive about the implementation challenges of Backpack), but I actually find some benefits to how much more limited Backpack’s module language is than, say, ML. It seems to place the focus on distributable software artifacts as the elements of composition in a way that more powerful “intra package” module systems don’t.
E.g. the ability to have separate, cached compilation at the expense of runtime performance seems like a good tradeoff in the context of a development workflow.