Buck2 build system for Cabal projects

Announcements
21

and while were at it, let’s clean up that CLI too :stuck_out_tongue:

22

I’d like to give a different perspective:

People using Haskell in their (small / normal-sized) companies:

  • Are not looking to build their own custom build system with Bazel or Buck or whatever.
  • Are not looking for 20% improvements.
  • Instead we need a 10x build performance improvement, quite urgently!
    The compile times are bad.
    They feel like they are much worse than they should be (“this code isn’t doing anything fancy, it’s just a lot of simple web server logic, why does it take 10 minutes to build 50k lines?”).
    They feel worse than in other languages (“with Go or Typescript or even Rust I wouldn’t have to sit around twiddling my thumbs so long”).
  • Need this for all Haskell projects, without lots of manual effort.
    Not everybody has a neobank’s money to throw at build systems.
  • See that GHC has a history of improving its own build system (Make → Hadrian → ?) but all other Haskell users benefit none of that.
    We need efforts to make the compiler fast, not for the compiler to be compiled fast!

Technically:

  • ghc --make seems like a very good idea to me.
    It seems to be necessary to achieve 10x, because compiler internals / language understanding seem necessary for that.
    If any build tool (cabal/buck/whatever) invokes ghc as it is today, the build probably won’t be fast!
    As a reminder, ghc --make has all inputs in memory and processes them once.
    Invoking multiple ghc processes, will have to re-read the same inputs (e.g. imports) repeatedly, like C++ with #includes. The C++ world for 30 years has found no way to make this approach fast, and is moving more into the direction of what ghc --make does (“unity builds” that concat all source code into one compilation module, “ThinLTO” which does only very superficial per-module compilation and instead moves all optimisation and codegen into the final link step where the linker calls the compiler as a library).
    Multi-process independent-compilation-unit build systems (Blaze/Bazel, Buck, Make etc) work well for languages where “includes are cheap and the compilation unit is independent and expensive”, such as C. They don’t work so well for languages with sophisticated imported generic code, such as C++ or Haskell.
    As a reminder, in C++, if you have 1000 .cpp files, each of which uses vector<Mytype> somewhere, all the code for vector<Mytype> gets compiled and codegenned 1000 times independently, and then thrown away at the link step. C++ programmers suffer from this codegen amplification since forever and unity builds (single compiler binary invocations) are the only available solution.
    They provide large integer speedups, but often use 1 core and lose incremental builds.
  • It seems good move more smarts into ghc --make:
    One of the most promising proposals I’ve seen is the Improve build parallelism issue #14095 (orignally called Improve parallelism in --make mode, and granularity in one-shot mode).
    It splits the parse/rename/typecheck/desugar/godegen phases and makes them fine-granular and overlapping across modules. So that for B imports A, B’s compilation can happen largely in parallel to A’s comilation, where as with current GHC they are sequential.
    This would make ghc scale better to multi-core, without creating duplicate work (which separate oneshot compilation creates).
    Doing that is only really possible inside the compiler because it depends on the exact semantics of the language, its constructs, how codegen works, etc.
  • (Of course any state of pipelining in the compiler could also be “externalised” to multi-process with a sufficiently good caching system; but caching is difficult so it seems in-process is far easier to get the results. It seems that should be done first before making the small steps available as separate process invocations so external tools like Make/Bazel/Buck can use them.)
  • GHC has various pathological performance degradations (deriving being slow, Generics being slow, large records or sum types being slow). If one of those is present in your codebase, no amount of moving-compilation-duty around between processes will fix that.

I’d find it a bit of a shame if we went this route towards more external compilation and end up in the unfixable state that C++ is in now.

Probably worth noting that TypeScript’s tsc mirrors ghc --make’s way of working, and it does pretty well with that.

14 Likes
23

Indeed, I’d like to add to that having a “bad” module/package graph that has a long critical path.

I feel like there’s a lot of performance left on the table by GHC not making it clear when these sorts of issues occur. Users merely perceive their builds as being slow without any reason. With better observability, users could more easily fix these issues themselves.

I’m excited by improvements like using buck2 or improving --make mode, but at the end of the day, the easiest way to improve performance is to do less

6 Likes
24

Realistically we won’t see that sort of improvement. Much more likely is small improvement here and there. And 20% is nothing to sneeze at.

Why indeed! I agree with @TeofilC we don’t have good tool for build time profiling. So we don’t really know where exactly all that build time is spent. So neither GHC maintainers nor library authors nor application authors understand what they could do to improve compilation speed.

2 Likes
25

I had a good experience with ghci for interactive loop, loading .o files. I think almost no one else does this, because .o loading tends to get fatal regressions that no one else notices. I now use ghci -fobject-code and recompiling is slower, but it works around the latest bug. But ghc’s ability to compile fast with bytecode is quite a nice feature. I gather the latest versions are able to write bytecode to files, I’m not sure if anyone will use that seeing as no one else uses .o files, but in theory you have access to quite a wide range of compile fast ↔ run fast.

My theory about why no one but me uses that is that people use cabal repl so whatever cabal doesn’t do may as well not exist. If the external bytecode stuff is used by cabal by default that would be great, but unlike .o files you can’t reuse them for your final artifact.

There was an old idea of a ghc server which is like ghc --make except it stays in memory and you throw source at it. I didn’t pursue it much because a persistent ghci (or ghcid) is effectively that same experience for me, and as far as end-to-end batch compile throughput, I’m actually content.

--make isn’t entirely incompatible with external build, just partially. E.g. you can build at the package level and use --make inside. It’s inefficient for single file packages since ghc adds a bunch of overhead making a package db. So you want many-file packages, but that increases your granularity, have to wait for all of GiantPkg to complete before starting on its dependent. What would it take to recover file level granularity? E.g. the build system is going to have a fast way to say “these source files changed”, and will naturally want to say “bring up to date with -c, next step”, but with --make it would have to go back up the graph the nearest transition out of haskell to figure which executables or libraries are involved, then ask ghc --make to build those, then ghc does the change detection and graph construction all over again itself. Sounds complicated and inefficient.

One path is the external build remains the driver, that I suppose means an interactive conversation where ghc has only rebuild responsibilities, not graph and (rough) change detection. That could be like -c but stays in memory. But it might be hard to extend that to cross module parallelism if that exists some day, because a “one node at a time” system maybe doesn’t give you a way to say “feed the entire haskell subset of the graph above the changed files”, that’s getting back to what it would have to do for --make.

The other path is ghc is the driver, which it sort of does in that it kind of knows how to build C, and knows what to do with .hsc (or maybe it doesn’t, cabal processes all those in advance?), and it could a shake-like DSL to describe what to do with needs that are out of date or not present but it’s just never going to catch up with a dedicated system.

Meanwhile the rest of the world (rust, python, go, …? I’m not familiar though) seems to have settled on per-language systems which are all unique, and then you have a separate meta-build-system at the CI level which is some bash or yaml thing, which may not be efficient but gets the job done and keeps people employed and lets all these systems evolve independently.

Anyway, this topic branches out forever so sorry to diverge! I’ll try cabal-buck2 out if I ever get time for it…

26

buck2 actually does make builds a lot faster due to its caching…

…assuming your dependency graph is under control, but even without buck2, a bad dependency graph is often the cause of slower and slower builds. it helps to regularly poke at it and make improvements.

if your dependency graph is chill, then even in a gigantic codebase, you can iterate quickly with a lazy loading ghci.

for CI, buck2 is pretty much unbeatable. module-level granularity for caching and rebuilds across package boundaries ensures CI does the least work necessary. it only has to build a commit’s worth of changed code + its dep closure at a time.

1 Like
27

I absolutely agree, but I don’t think we get there by optimising GHC. A build system with caching and remote execution could give you 10x though. I’d say it’s reasonable to hope that we could get 10x faster CI for GHC using Buck2 + caching / remote execution. What about regular users who can’t set up a build farm? You can still use a build cache - think about all the people building the same Cabal packages with the same compiler, it’s almost certainly faster to download the artifacts.

I’ll make one other observation: whether to use --make is orthogonal to the build system choice. My Buck2 prototype uses --make currently, and that’s what they’ve been using at Meta for ever (and still enjoying 10x faster builds due to caching and remote execution). You can move to single-module compilation which improves parallelism at the expense of raw compile time, which may or may not be a win depending on how much parallelism you have, and you can improve the odds by using a persistent GHC server that caches interface files in memory.

Interestingly most of the ways you make builds faster is by caching at some level, it’s just a case of choosing the right place(s) to get the most bang for buck.

3 Likes
28

I was very impressed by some of the talks I saw, but everything I looked at seemed to be experimental software that didn’t explain anything. How does one get started with Buck2? I failed to find a description of its theory of operation, or how to understand its syntax for naming targets, or… anything really, and gave up. (And I have Nix under my belt, so I feel like I have a decent stomach for thinly documented good ideas.)

29

The documentation has historically been lacking. It’s a bit better now, although some of the more obscure parts are still obscure - I have a lot of trouble with configuration resolution for example. But anyway, you could try looking at

and in particular for target naming: Build Target | Buck2

The essence of it all is staged computation. The first stage evaluates all the build files, which are written in Starlark (a pure subset of Python), and running these produces a graph of build rules. The implementation of each rule type is also written in Starlark, and the second stage evaluates the rules to produce an action graph: a graph of build actions where the edges are artifacts. This is what’s new about Buck 2 - in Buck 1 the rule implementations were written in Java in Buck itself, which made it really hard to modify and test any changes. In Buck 2 I can fiddle with the Haskell rules and get instant feedback. The language and API design rules out side effects in the BUCK files, and makes it really hard to write build code with untracked dependencies in the rule implementations.

6 Likes
30

See 6.5 in https://ndmitchell.com/downloads/paper-build_systems_a_la_carte_theory_and_practice-21_apr_2020.pdf
As for differences between buck and buck2, buck2 starts building as it is discovering the dependency tree, it is better at caching evaluation, it’s written in rust and it is much faster.

31

Update: I made a PR to start the discussion upstream in Cabal

32

Couldn’t you just use the external command system for the first iteration?

2 Likes
33

I wasn’t actually aware of that, but probably yes. It will have to link against the cabal-install library and I’m a bit worried about that API being unstable, but I’ll try it and see.

1 Like
34

Done: cabal-buck2: Build a Cabal project with buck2 (the `cabal buck2` external command)

5 Likes