# BitNet 1.58 in MicroHaskell: Exploring minimal LLMs and combinators (Experience Report)
**URL:** https://discourse.haskell.org/t/bitnet-1-58-in-microhaskell-exploring-minimal-llms-and-combinators-experience-report/14325
**Category:** Show and Tell
**Created:** [June 27, 2026, 10:46pm UTC](https://discourse.haskell.org/t/bitnet-1-58-in-microhaskell-exploring-minimal-llms-and-combinators-experience-report/14325 "2026-06-27T22:46:44Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Tritlo](https://discourse.haskell.org/u/Tritlo)
#### Post date: [June 27, 2026, 10:46pm UTC](https://discourse.haskell.org/t/bitnet-1-58-in-microhaskell-exploring-minimal-llms-and-combinators-experience-report/14325/1 "2026-06-27T22:46:44Z")
Some background: I’ve recently become obsessed with combinators, specifically iota, where:
`i x = x S K`
It’s a single combinator basis that gives you
`i i = I`, `i I = A (A x y = y)`, `i A = K`, `i K = S`
so you can recover `S`, `K`, and `I` and the full combinator calculus.
This of course leads to **Micro Haskell** , a project by Lennart Augustsson that compiles Haskell 2010 to combinators (with support for primitives/FFI to enable IO, fast math, etc.)
> **[GitHub - augustss/MicroHs: Haskell implemented with combinators](https://github.com/augustss/MicroHs)**
>
> Haskell implemented with combinators
I’ve been using MicroHs for a few projects, and I thought, can we do LLMs in pure combinators?
To explore this idea, I looked at BitNet 1.58 from Microsoft.
> **[The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://arxiv.org/abs/2402.17764)**
>
> Recent research, such as BitNet, is paving the way for a new era of 1-bit Large Language Models (LLMs). In this work, we introduce a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1}....
It’s an interesting idea: instead of using floating point weights, there are only three weights: `{-1, 0, 1}`.
Using these we get an interesting property: you don’t need multiplication at all!
When you do the classic dot product, say `[x1, x2, x3] x [-1,0,1]`, you get `(-1)x1 + 0*x2 + 1*x3`, which is of course `x1 - x3`. No multiplication required at all!
To get this back into the combinator world (where we really only have the naturals), you can model negative numbers as _difference pairs_: `(a,b) = a - b`, where `a` and `b` are church-encoded naturals.
So, can we then do the huge matrix multiplication in pure combinators? Turns out **yes**! It boils down to `sum $ map (\(x,w) -> if w == 1 then x else swap x) $ filter ((/= 0) . snd) $ zip xs weights`. on `-1`, we swap the pair (the `C` combinator), on `0` we drop it, and on `1` we keep it unchanged.
But it’s very slow. However! MicroHs has an FFI interface so we can define _kernels_ to do the numerical parts (not pure combinators anymore, but we have to live with that).
Here are my results:
(The tokenizer is mostly due to lack of any aeson in MicroHs).
By adding a few small kernels and using MicroHs for the rest, we got quite OK performance, ~10 tokens/second.
We’re not going to beat GPU accelerated matrix multiplication any time soon, but it was very interesting for me to see how simple things can be, and the fact that BitNet 1.58 (despite its name) essentially lives in the integer world. We can represent the weights of the LLM in ~22 Billion combinators (~98 billion iotas), which gives you a sense of scale (though with all the intermediate steps the number of iotas at any point can get _much_ bigger). If we imagine that the model was quantized to `int8` tensors, we get a clearer picture:
So, ~60 million combinators in total, almost _all_ of it data.
In any case, I thought this was a fun weekend project (Claude was surprisingly good at writing MicroHs and kernels), and if we ever run out of silicon, we can always manually reduce iotas to get our answers (though we’d need to figure out the tensors first). Maybe that’s how Deep Though came up with 42?
Here’s the codebase for the experiment for those interested:
> **[GitHub - Tritlo/microhsbitnet: BitNet 1.58 in MicroHs](https://github.com/Tritlo/microhsbitnet)**
>
> BitNet 1.58 in MicroHs
And, for good measure, there’s also an interactive combinator playground, if you want to play around with combinators and iotas in a more sandbox setting. You can even compile MicroHs programs straight to the canvas!
> **[Combinate — an ι / SKI combinator-calculus sandbox](https://combinate.app)**
>
> Drag ι, snap trees, and watch them reduce on their own. Discover combinators, compile Haskell to combinator trees, and golf.
---
### Author:  [@augustss](https://discourse.haskell.org/u/augustss)
#### Post date: [June 28, 2026, 2:57pm UTC](https://discourse.haskell.org/t/bitnet-1-58-in-microhaskell-exploring-minimal-llms-and-combinators-experience-report/14325/2 "2026-06-28T14:57:05Z")
I saw in the documentation that you’ve run in to some MicroHs bugs. Please report them when you run into them.
---
### Author:  [@Tritlo](https://discourse.haskell.org/u/Tritlo)
#### Post date: [June 28, 2026, 7:59pm UTC](https://discourse.haskell.org/t/bitnet-1-58-in-microhaskell-exploring-minimal-llms-and-combinators-experience-report/14325/3 "2026-06-28T19:59:50Z")
Turns out I was running on an older version, the bugs got fixed in 4b616112. Updated!
---
### Author:  [@ashokkimmel](https://discourse.haskell.org/u/ashokkimmel)
#### Post date: [June 28, 2026, 10:10pm UTC](https://discourse.haskell.org/t/bitnet-1-58-in-microhaskell-exploring-minimal-llms-and-combinators-experience-report/14325/4 "2026-06-28T22:10:56Z")
Btw, I think you meant `x3-x1`
---
### Author:  [@danielc777888](https://discourse.haskell.org/u/danielc777888)
#### Post date: [June 29, 2026, 6:31pm UTC](https://discourse.haskell.org/t/bitnet-1-58-in-microhaskell-exploring-minimal-llms-and-combinators-experience-report/14325/5 "2026-06-29T18:31:08Z")
Very interesting. Wonder if it could perform better with hardware accelerated graph reduction.
Here is a research project that does just that with FPGA’s and the custom Heron chip.
> **[HAFLANG - Hardware Acceleration of Functional Languages](https://haflang.github.io/)**