# Botan bindings devlog

**URL:** <https://discourse.haskell.org/t/botan-bindings-devlog/6855>\
**Category:** Uncategorized\
**Created:** [July 5, 2023, 6:17pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855 "2023-07-05T18:17:27Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 5, 2023, 6:17pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/1 "2023-07-05T18:17:27Z")

</div>

I’ve made some significant progress over the last few hours. In short order, I have:

- Sussed out the M1 macOS issues
- Used `sdl2` as an example of good C FFI
- Created a new `botan.cabal` that uses `pkgconfig-depends: botan-3 >= 3.0.0` instead of embedding `botan_all.h` and `botan_all.cpp` via monolithic `--amalgamation` and the `Custom.hs` build script
- Ditched `CPP` and `#include <botan/ffi.h>` in favor of `includes: botan/ffi.h` in `botan.cabal`
- Got it to print out the Botan 3 version string from FFI!

 ![Screenshot 2023-07-05 at 1.00.32 PM](https://us1.discourse-cdn.com/flex002/uploads/haskell/original/2X/2/2771dde77be3d835a535898a5d329ede44649afe.jpeg)

With this being a good step forward, I will keep moving in this direction.

Question: I would like to keep a devlog of this - I am sure to run into issues where I may need to ask more questions / discuss minutia, but I don’t wish to pollute this topic. Being a relatively recent refugee from reddit, I am unsure of the expected protocol here - shall I spin off another topic for specifically discussing this effort to create botan-3 bindings? This thread can then be updated with milestones as relevant.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [July 5, 2023, 6:31pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/2 "2023-07-05T18:31:31Z")

</div>

Hi @ApothecaLabs ,

Just keep writing! A mod can split this into a new discussion on your behalf. (I flagged it so a mod will see it.)

---

<div class="post-metadata">

**Author:** ![neil.mayhew](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/neil.mayhew/32/3467_2.png) [@neil.mayhew](https://discourse.haskell.org/u/neil.mayhew)\
**Post date:** [July 5, 2023, 6:31pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/3 "2023-07-05T18:31:52Z")

</div>

I think it would be helpful to split it off into a new topic, but I for one am following this with interest.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 5, 2023, 8:07pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/4 "2023-07-05T20:07:51Z")

</div>

I have created a repo, with the initial hello-world of printing botan’s version string:

> **[GitHub - apotheca/botan: Low-level bindings for Botan](https://github.com/apotheca/botan)**
>
> Low-level bindings for Botan. Contribute to apotheca/botan development by creating an account on GitHub.

_I cannot promise that it works on Windows, but I have filled it with my best guess._

Something to note is that it uses the old `ccall` convention instead of the newer `capi` - attempting to do so yields some errors, but I did notice that `sdl2` also still uses the `ccall` convention - I will need to investigate this before too long.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 6, 2023, 5:58pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/5 "2023-07-06T17:58:26Z")

</div>

Minor update: Things continue to go well, and I am mostly following [the botan ffi bindings in order](https://botan.randombit.net/handbook/api_ref/ffi.html). Thus far, I have:

- Filled out the `Botan.Version` module, dipping a toe in the water
- Completed the `Botan.Exception` module, to prepare for handling errors.
- Begin work on the `Botan.Utility` module.
- I have implemented `Botan.Utility.botanConstantTimeCompare`. It is probably my first proper / nontrivial use of `unsafePerformIO`, so I’ll be scrutinizing the internals of a few modules (such as `Data.ByteArray`) to get a better sense of it vs inlining as I continue.

 ![Screenshot 2023-07-06 at 1.00.29 PM](https://us1.discourse-cdn.com/flex002/uploads/haskell/original/2X/c/c9b7ace883655f858f930d4e221223a2f33942b6.jpeg)

The github isn’t updated with this quite yet, but I will commit and update when I do finish the `Botan.Utility` module.

---

<div class="post-metadata">

**Author:** ![Bodigrim](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bodigrim/32/1457_2.png) [@Bodigrim](https://discourse.haskell.org/u/Bodigrim)\
**Post date:** [July 6, 2023, 6:42pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/6 "2023-07-06T18:42:50Z")

</div>

Discourse does not allow me to like every post twice, so let me say that I’m extremely hyped about this development!

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 6, 2023, 9:09pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/7 "2023-07-06T21:09:27Z")

</div>

> [@Bodigrim](#):
>
> let me say that I’m extremely hyped about this development!

Allow me to return the favor - I am thrilled to have drawn interest from others!

It is a wonderful turn of circumstance for me, because it is not incidental that I have an interest in seeing good open-source cryptography bindings in Haskell. Though I am not a professional cryptographer by any means, I have had my fair share of hands-on experience with it, and I know to be careful with a dangerous tool lest I lose a finger or two.

The whole reason I took up Haskell was in order to figure out how to write some tricky code involving recursion and cryptography, and though I suspect my writing lay more heavily towards prose than formality, I _do have a point_ that I have been steadily working towards in [my blog series about recursion](http://discourse.haskell.org/t/blog-sourced-recursion-schemes/6731).

The next article involves implementing sparse merkle trees in a rather trite / concise way (because the approach is important for a greater discussion), and for that, and for that _I need good cryptography bindings_.

So you see, I am interested in more than just bindings to libraries - _there are specific things that I wish to do with it_ 🙂

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 8, 2023, 6:33pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/8 "2023-07-08T18:33:59Z")

</div>

The `Botan.Version`, `Botan.Error`, and `Botan.Utility` modules are now functionally complete, and [the repo](https://github.com/apotheca/botan) has been updated.

 ![Screenshot 2023-07-08 at 1.15.02 PM](https://us1.discourse-cdn.com/flex002/uploads/haskell/original/2X/a/aefdd07ba14c456ac89b07645f00be28df90a484.jpeg)  
This includes:

- Version string, and major / minor / patch numbers
- Error codes, exception types, and throwing methods
- Constant time equality checking and hex encoding / decoding methods

This is giving me a good hands-on idea of how to handle Botan’s buffer / return style, and although I’d like to complete one or two of the crypto modules first, at this point I’m growing fairly confident in my ability to complete the rest of the bindings, with the caveat that I need someone to go over them too to make sure my low-level pointer / unsafePerformIO shenanigans are safe.

My next module target is probably going to be `Hash`.

---

<div class="post-metadata">

**Author:** ![Vlix](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/vlix/32/1551_2.png) [@Vlix](https://discourse.haskell.org/u/Vlix)\
**Post date:** [July 9, 2023, 2:02am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/9 "2023-07-09T02:02:19Z")

</div>

How is Botan 3 in comparison to libsodium?

I’ve seen/read someone was working on libsodium bindings (Hecate?) so might it be a better idea to focus on one library together, or do they serve different purposes? 🤔

---

<div class="post-metadata">

**Author:** ![Ambrose](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ambrose/32/5672_2.png) [@Ambrose](https://discourse.haskell.org/u/Ambrose)\
**Post date:** [July 9, 2023, 2:07am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/10 "2023-07-09T02:07:35Z")

</div>

I think C bindings are generally pretty good value. The work is mostly mechanical, and the hard part is already done. FFI just grows Haskell for cheap.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 9, 2023, 2:30am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/11 "2023-07-09T02:30:23Z")

</div>

Speaking as a practical user of `libsodium` for many years, libsodium makes algorithm choices for you, which makes for a smaller, lighter, simpler library. You don’t need to make any decisions, eg you just have `hash :: ByteString -> ByteString`. It aims for a simple, friendly, no-mistakes interface, at the cost of nuance - great for developing new / self-contained systems, but not so great for interacting with something that doesn’t use libsodium.

I haven’t gotten too far into it (after all, I’ve just gotten started with the bindings) but `Botan` is a bit more heavyweight. It is slightly more state-y, and it offers choices of algorithm and things like full control over incremental hashing, but it still tries to offer a consistent interface, rather than just collecting algorithms in a pile. So far, it seems sensible.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 9, 2023, 4:30am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/12 "2023-07-09T04:30:34Z")

</div>

After [a bit of help with some puzzling pointers](http://discourse.haskell.org/t/questions-about-ffi-foreignptr-and-opaque-types/6914) from some wonderful people, I have the initial pieces of the `Botan.Hash` module functioning! I learned quite a bit about ForeignPtr today, and put it to good use:

 ![Screenshot 2023-07-08 at 11.08.02 PM](https://us1.discourse-cdn.com/flex002/uploads/haskell/original/2X/8/89b0881235d747c5565dc1246d828ee514c3c097.jpeg)

We have:

- A hash object can be initialized by algorithm name.
- It can be queried for its name (still in CString format), and its digest length in bytes.
- It is destroyed automatically when garbage collected.

This places us rather close to having our first-hash milestone!

---

<div class="post-metadata">

**Author:** ![Kleidukos](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/kleidukos/32/1213_2.png) [@Kleidukos](https://discourse.haskell.org/u/Kleidukos)\
**Post date:** [July 9, 2023, 7:50am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/13 "2023-07-09T07:50:48Z")

</div>

I do write libsodium bindings, but in the context of the [Haskell Cryptography Group](https://haskell-cryptography.org/), where all cryptography libraries have their place. 🙂

---

<div class="post-metadata">

**Author:** ![Vlix](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/vlix/32/1551_2.png) [@Vlix](https://discourse.haskell.org/u/Vlix)\
**Post date:** [July 9, 2023, 3:43pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/14 "2023-07-09T15:43:45Z")

</div>

Ah, thanks for the explanation. Sounds like a good addition, then 😁 I’ll be checking out your progress when you share it!

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 9, 2023, 9:13pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/15 "2023-07-09T21:13:19Z")

</div>

We have first hash 😃

 ![Screenshot 2023-07-09 at 4.08.25 PM](https://us1.discourse-cdn.com/flex002/uploads/haskell/original/2X/3/35a1c0c3fa0a36da4ef7e35393b2bc06ee7b9a31.jpeg)  
And a quick sanity check for comparison:  
 ![Screenshot 2023-07-09 at 4.11.39 PM](https://us1.discourse-cdn.com/flex002/uploads/haskell/original/2X/f/fd84bd894005abea9c124159ab00afc63cc39508.png)

---

<div class="post-metadata">

**Author:** ![hasufell](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/hasufell/32/1250_2.png) [@hasufell](https://discourse.haskell.org/u/hasufell)\
**Post date:** [July 10, 2023, 3:37am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/16 "2023-07-10T03:37:19Z")

</div>

How do we deal with APIs where the secrets leak into Haskell land, e.g. because that’s how the API is designed?

As long as everything is on the C/C++ side, we can be somewhat confident it’s secure (written by experts and audited).

I think this is a general issue with crypto bindings, especially if they are low-level.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 10, 2023, 1:36pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/17 "2023-07-10T13:36:24Z")

</div>

My mind goes about ten different ways when I want to answer this. I have a lot to say about this, because it is an unassumingly hefty question. The important parts are:

> [@hasufell](#):
>
> How do we deal with APIs where the secrets leak into Haskell land, e.g. because that’s how the API is designed?

The answer to the problem of secrets leaking is that it really is an application-level (or even os- and machine- level) problem. Libraries can’t solve it; cryptographic purity is not unlike functional purity - the surrounding context must restrain itself from reaching in, and there is very little that the cryptographic primitives themselves can do about it. Cryptographic purity is something that people are only recently willing to make the orders-of-magnitude-efficiency tradeoff required to do it properly, and although some day, we’ll give a yawn as we spin up full homomorphic encryption sandboxes for everything, right now our existing computer ecosystems from the transistors up are still built to take advantage of sharing and shortcuts for efficiency above all else, because our architectures are still built based on an era where fundamentally, that is what it took. 😬

> [@hasufell](#):
>
> I think this is a general issue with crypto bindings, especially if they are low-level.

It is.

* * *

That’s the thousand-foot perspective, anyway.

More realistically, Haskell’s crypto problem in particular is that it is easy for things to accidentally stay alive due to buildup of thunks; C and C++ are just as vulnerable to forgetting to `free` something if you’re handling allocations manually, _which you’ll notice that cryptographic primitives require you do_.

People used to write assembly and punchcards by hand, people used to write threading and graphics by hand, we still do cryptography ‘by hand’, in large part because our systems haven’t been adapted to these needs yet.

* * *

Cryptographic operations should be performed as atomically as possible, with everything zeroed and destroyed immediately after it is no longer necessary, rather than relying on the garbage collection.

Long-lived cryptographic data should be kept to a minimum, and should only be things like immediate session keys rather than the passwords used to generate said keys. It is better to ask for a password or cert store access again, than keep it in memory for the lifetime of the app. More and more, the OS is handling this sort of thing. For the duration that it is in memory, it should be treated specially, as to avoid accidentally printing it into something.

In part, this is why I [was using](https://github.com/apotheca/botan/issues/1#issuecomment-1627943672) `memory`, for `ScrubbedBytes`, but I’ll have to re-implement something like it for the `botan` bindings.

* * *

A lot of the practical answers to this will lie in how we design our higher-level API, which will probably involve constructs akin to `withStoredPrivateKey keyRef $ \ key -> ...`, and we’ll have to build stores and atomic cryptographic operations to guide the user away from ever handling raw / exposed keys and such.

---

<div class="post-metadata">

**Author:** ![chreekat](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/chreekat/32/2669_2.png) [@chreekat](https://discourse.haskell.org/u/chreekat)\
**Post date:** [July 10, 2023, 1:59pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/18 "2023-07-10T13:59:03Z")

</div>

Damn, so it sounds like there’s fertile ground for researching how to implement good memory-hygiene primitives for lazy functional languages!

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [July 10, 2023, 2:16pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/19 "2023-07-10T14:16:51Z")

</div>

In my mind, it is something like IO or STM, but a completely separate branch, being unable to perform side effects like print or even unsafePerformIO, and where secrets and keys cannot be returned directly, only their references. Ideally, operations should be provably halting, and should do as little as possible before returning control and the plaintext / result back to the user. Even the foreign call bindings could use this `Crypto` monad / DSL instead of `IO`.

This is mostly me summing up my internal model rather than dictating any formal structure, however 🙂

---

<div class="post-metadata">

**Author:** ![Probie](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/probie/32/3232_2.png) [@Probie](https://discourse.haskell.org/u/Probie)\
**Post date:** [July 10, 2023, 2:28pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/20 "2023-07-10T14:28:21Z")

</div>

I’ve always wondered if one can (ab)use a combination of unlifted types and linear types to get some safety guarantees. That would prevent things like accidentally holding onto a key when signing, since the data type holding the signature is unlifted (and therefore there is no thunk that might contain the key), and linearity can be used to make sure that the secrets don’t accidentally go any place they’re not meant to.

[Next page](https://discourse.haskell.org/t/botan-bindings-devlog/6855.md?page=2)