# Botan bindings devlog

**URL:** <https://discourse.haskell.org/t/botan-bindings-devlog/6855>\
**Category:** Uncategorized\
**Created:** [July 5, 2023, 6:17pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855 "2023-07-05T18:17:27Z")\
**Posts on this page:** 20\
**Page:** 7

<div class="post-metadata">

**Author:** ![jaror](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/jaror/32/3271_2.png) [@jaror](https://discourse.haskell.org/u/jaror)\
**Post date:** [January 31, 2024, 6:12pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/121 "2024-01-31T18:12:04Z")

</div>

Someone’s already done that: [raaz: Fast and type safe cryptography.](https://hackage.haskell.org/package/raaz)

I’m actually surprised this didn’t make the prior art of the tech proposal.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [January 31, 2024, 6:37pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/122 "2024-01-31T18:37:35Z")

</div>

@romes

It certainly helps to have every gear polished and lubricated, ready to fit into place…

@jaror

Oooh, I don’t know how I missed `raaz`, but I’mma have to give it the ol’ look-see-and-digest for a bit - I’m sure there’s things to learn from it. 😃

* * *

Small aside, _providing different backends for implementing the same algorithm_, and _providing typeclasses for implementing similar but separate algorithms_, are two distinct goals. They do work together quite nicely, however, as they are related.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 3, 2024, 5:42pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/123 "2024-02-03T17:42:00Z")

</div>

# Update: More gold-standard modules, imminent Hackage release

Hello, all! It is time for another update, but first:

I must apologize, for I have fallen prey to my tendency to keep my eyes on the horizon instead of on the finish line, and have ended up moving my own goalposts. Truth be told, `botan-bindings` and `botan-low` passed the finish line some time ago, and I really should have pulled the trigger on publishing them already. 😬

This is where I really appreciate the community framework; weekly meetings with José have been helping me to keep my updates more regular, and also keep me on track and appraised of any concerns. Frankly, _I don’t want to be a silo’d developer_, and communication is an essential part of that.

As such, a minor course correction: I’m switching focus, and the goal of the next update is going to be the publishing of `botan-bindings` and `botan-low` to hackage as a candidate package (alongside the third monthly status report, of course, which is also now due! Egads!).

I am a bit anxious, never having published a package to hackage before. I look at the project and its very easy for me to forget what I’ve done, and only see what still needs to be done, or could be done - perfectionist’s disease. Its fine, it doesn’t have to be perfect, it just has to be sufficient _for initial release_, and it is. We can still keep working on it _after we publish_.

There’s a few things that must happen before publishing:

**First** , according to hackage requirements, I need someone to [endorse me](https://hackage.haskell.org/user/ldillinger/endorse), so if a fine citizen or two could assist me in this, I would appreciate it.

**Secondly** , there’s the minor question of `pkgconfig-depends` vs `extra-libraries`. Right now, the stanzas in `botan-bindings.cabal` look like this:

```cabal
library
    ...
    includes:
        botan/ffi.h
    if os(windows)
        extra-libraries: botan-3
    else
        pkgconfig-depends: botan-3 >= 3.0.0
    ...

```

This has been working _so far_, but I want to ensure is configured _properly_, since it is how we link to the `Botan C++` library. There are more than a few different ways of handling this, spread over several discussions:

- [What's the benefit of pkgconfig-depends?](http://discourse.haskell.org/t/whats-the-benefit-of-pkgconfig-depends/8670)
- [Allowing pkg-config to fail · Issue #6771 · haskell/cabal · GitHub](https://github.com/haskell/cabal/issues/6771)
- [Backtrack when no pkgconfigdb is present by gbaz · Pull Request #7621 · haskell/cabal · GitHub](https://github.com/haskell/cabal/pull/7621)

I’d like to handle it properly - if I’ve read the discussions correctly, we can crank the cabal version dependency up and improve it. I’d appreciate any suggestions here.

**Thirdly** , I need to go over `botan-bindings` and `botan-low` with a fine-toothed comb and do a few things like establish dependency version constraints to make the package acceptable to hackage. I’ll be on this for the next few days, and then hopefully upload it as a candidate on Monday.

So, time to get it done!

* * *

Now, back to this update.

- The following modules have been completed to gold-standard, and conform to `Botan.Types.Class`:
  - Botan.BlockCipher
    - AES
    - ARIA
    - Blowfish
    - Camellia
    - CAST
    - DES
    - GOST
    - IDEA
    - Noekeon
    - SEED
    - Serpent
    - SHALCAL
    - SM4
    - Threefish
    - Twofish

- I’ve added a temporary BlockCipher128 typeclass while a proper blocksize constraint is developed.
- There have been improvements to error reporting, and the last exception message is now attached to any thrown Haskell exceptions.
- I’ve laid out a Botan.Easy module that exposes a `saltine`-like interface of recommended algorithms

That’s all for now, it’s been pushed to the [repo](https://github.com/apotheca/botan). We’ll be back on this stuff after we publish to hackage.

---

<div class="post-metadata">

**Author:** ![jackdk](https://avatars.discourse-cdn.com/v4/letter/j/9f8e36/32.png) [@jackdk](https://discourse.haskell.org/u/jackdk)\
**Post date:** [February 4, 2024, 1:06am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/124 "2024-02-04T01:06:29Z")

</div>

> `pkg-config`

I strongly recommend `pkgconfig-depends`, if botan upstream publishes `.pc` files. The main reason is that it means botan developers are deciding what the link flags are, rather than all their library consumers making a best guess. It also means you’re declaring a dependency on a native package instead of linker flags (which are more of an implementation detail), which seems more like the correct level of abstraction.

I would hope that you’d be able to use pkg-config on Windows but I haven’t played in that space for many years. Doesn’t GHC come with some sort of MSYS setup for this sort of thing?

---

<div class="post-metadata">

**Author:** ![jaror](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/jaror/32/3271_2.png) [@jaror](https://discourse.haskell.org/u/jaror)\
**Post date:** [February 4, 2024, 9:48am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/125 "2024-02-04T09:48:11Z")

</div>

Yes, pkg-config works on windows (or at least it is not any worse than other options). See my experience report: [Installing a library with C dependencies on Windows](http://discourse.haskell.org/t/installing-a-library-with-c-dependencies-on-windows/8557)

---

<div class="post-metadata">

**Author:** ![Bodigrim](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bodigrim/32/1457_2.png) [@Bodigrim](https://discourse.haskell.org/u/Bodigrim)\
**Post date:** [February 4, 2024, 12:43pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/126 "2024-02-04T12:43:38Z")

</div>

It’s worth to provide a fallback to `extra-libraries` (for the case when library is installed, but pkg-config isn’t) and a fallback with bundled C-sources (very helpful on Windows / CentOS and inevitable for cross-compilation). This requires two cabal flags: a manual one to force bundled sources and an automatic one to switch between pkg-config and extra-libraries.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 4, 2024, 5:38pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/127 "2024-02-04T17:38:40Z")

</div>

@jackdk @jaror @Bodigrim This is very helpful. I do intend to keep using `pkgconfig-depends` wherever possible, and wish to ensure that I am using it properly.

* * *

Alright so this is what I have so far, though my inference is incomplete and so I have a few questions.

First, we have our `pkgconfig` flag, and the first question is whether the preferred convention is to express it in the positive or the negative:

```cabal
-- We can express it in the positive
flag use-pkgconfig
    description: Enable pkgconfig
    default: True
    manual: False

-- Or we can express it in the negative
flag no-pkgconfig
    description: Disable pkgconfig
    default: False
    manual: False

```

This should be purely nomenclative, as the two flags should behave the same (with an appropriate negation): it should attempt to use `pkgconfig-depends` first, with the dependency solver automatically falling back on negating the flag if it fails.

Secondly, can you be more explicit about what you mean by `bundled sources`? I think I understand, but wish for clarification to avoid assumptions - I’d rather know exactly.

My inference is that we have our `force-bundled` flag:

```cabal
flag force-bundled
    description: Force bundled sources
    default: False
    manual: True

```

Unlike `use-pkgconfig` (or `no-pkgconfig`), force-bundled is `manual: True` (and `default: False`) in order to require the user to positively activate it.

Then we use the flags as such:

```cabal
library
    ...
    -- Or !flag(no-pkgconfig) && !flag(force-bundled) / !(flag(no-pkgconfig) || flag(force-bundled))
    if flag(use-pkgconfig) && !flag(force-bundled)
        pkgconfig-depends:
            botan-3 >= 3.0.0
    else
        includes:
            botan/ffi.h
        extra-libraries:
            botan-3

```

Combined with the `use-pkgconfig` flag, the dependency solver should 1) try to use `pkgconfig`, falling back if it fails 2) unless the `force-bundled` is set, in which case it skips to the falls back immediately.

_I may also need to bump `cabal-version: 3.8` because of [this issue](https://github.com/haskell/cabal/issues/6771)._

* * *

Is this understanding correct? Is this what you meant by forcing bundled sources? And which method is preferred / standard: `use-pkgconfig` or `no-pkgconfig`?

> NOTE: I have some other flags (eg, for `mtl` and `random` support, plus the `XFFI` flag) that will also try to follow the established convention, but I believe that they should be `flag no-foo` and `default: False` / `manual: True` (like `force-bundled`)

---

<div class="post-metadata">

**Author:** ![Vlix](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/vlix/32/1551_2.png) [@Vlix](https://discourse.haskell.org/u/Vlix)\
**Post date:** [February 4, 2024, 7:48pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/128 "2024-02-04T19:48:42Z")

</div>

I never like negation if it is not necessary.

The `default` is the thing that assumes if it’s on or not and you won’t get situations where you _disable_ the _no-foo_ flag… because then you get a double negation, which in programming you almost never want to encounter.

E.g. I’d rather `--flag botan:-pkg-config` than `--flag botan:no-pkg-config`. We did the same in the [`password`](https://hackage.haskell.org/package/password-3.0.4.0) library, where the default is that all 4 algorithms are enabled, and you have to **disable** them to not include them, instead of **enabling the negative flag**.

All my opinion, obviously. I wonder how other people feel about this.

---

<div class="post-metadata">

**Author:** ![Bodigrim](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bodigrim/32/1457_2.png) [@Bodigrim](https://discourse.haskell.org/u/Bodigrim)\
**Post date:** [February 4, 2024, 8:23pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/129 "2024-02-04T20:23:34Z")

</div>

@ApothecaLabs yes, correct. The full setup could look like this:

> <https://github.com/haskell/zlib/blob/4d48dda148e1baf144696298b1d9ef7da1f304f8/zlib.cabal#L65-L75>

> <https://github.com/haskell/zlib/blob/4d48dda148e1baf144696298b1d9ef7da1f304f8/zlib.cabal#L107-L122>

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 5, 2024, 9:29pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/130 "2024-02-05T21:29:27Z")

</div>

# Minor update: Readying for `0.0.1` release

Awrighty, I’ve got a minor update today, and its that I’ve gotten `botan-bindings` and `botan-low` ready for initial release.

This includes futzing with package flags, adding dependency version bounds, and getting low-level tutorials completed for critical path use cases. These tutorials are available in the [README](https://github.com/apotheca/botan?tab=readme-ov-file#tutorials), and will be making their way into the haddock module documentation in the near future.

It all looks good, passes `cabal check`, and I can `cabal sdist botan-bindings/botan-bindings.cabal` and `cabal sdist botan-low/botan-low.cabal` to get my package’s `*.tar.gz` archive. I’ve uploaded the package candidates:

- [https://hackage.haskell.org/package/botan-bindings-0.0.1.0/candidate](https://hackage.haskell.org/package/botan-bindings-0.0.1.0/candidate)
- [https://hackage.haskell.org/package/botan-low-0.0.1.0/candidate](https://hackage.haskell.org/package/botan-low-0.0.1.0/candidate)

I can also `cabal new-haddock --haddock-for-hackage botan-bindings` and `cabal new-haddock --haddock-for-hackage botan-low` to generate haddock documentation. ~~However, I don’t know how to preview the documentation on hackage (though I can look at it on my machine).~~ Documentation has been generated and uploaded as well. _It isn’t the greatest, but I’ll be working on it the next few days_

I’ve pushed this to [the repo](https://github.com/apotheca/botan), and I think we’re set for `0.0.1` aside from this question of documentation - what does everyone think? Do I pull the trigger and publish?

---

<div class="post-metadata">

**Author:** ![Bodigrim](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bodigrim/32/1457_2.png) [@Bodigrim](https://discourse.haskell.org/u/Bodigrim)\
**Post date:** [February 5, 2024, 9:32pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/131 "2024-02-05T21:32:17Z")

</div>

> [@ApothecaLabs](#):
>
> However, I don’t know how to preview the documentation on hackage (though I can look at it on my machine).

You can generate documentation locally and upload with `cabal upload -d`.

---

<div class="post-metadata">

**Author:** ![bradrn](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bradrn/32/264_2.png) [@bradrn](https://discourse.haskell.org/u/bradrn)\
**Post date:** [February 6, 2024, 7:23am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/132 "2024-02-06T07:23:27Z")

</div>

I clicked on a random module and ran into this:

 ![image](https://us1.discourse-cdn.com/flex002/uploads/haskell/original/2X/6/63a1e60332608b3907e7cb1f16bec8454268b461.png)

Surely that extra slash shouldn’t be there?

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 6, 2024, 1:42pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/133 "2024-02-06T13:42:05Z")

</div>

It should not! I am cleaning up Botan.Low’s documentation today 🙂

---

<div class="post-metadata">

**Author:** ![Bodigrim](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bodigrim/32/1457_2.png) [@Bodigrim](https://discourse.haskell.org/u/Bodigrim)\
**Post date:** [February 8, 2024, 9:20pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/134 "2024-02-08T21:20:27Z")

</div>

I think a convenient way to install everything through `cabal`, especially on Windows, would be crucial for wide adoption. It’s hard to imagine packages migrating from, say, `cryptonite` to `botan` otherwise; installing `botan` yourself is [quite a chore](https://github.com/apotheca/botan?tab=readme-ov-file#windows-from-source).

Prior art is packages like [`lzma-clib`](https://hackage.haskell.org/package/lzma-clib) and especially [`lzma-static`](https://hackage.haskell.org/package/lzma-static). The idea is that you can autoconf on several, most-popular OS / arch and package headers to switch between depending on conditionals in Cabal file.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 9, 2024, 4:46pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/135 "2024-02-09T16:46:06Z")

</div>

Agreed - I’ve been putting some time into looking at a custom Setup.hs and all that entails. This is a thornier issue for `botan`, however, because the source code is extremely large and complex compared to something like `lzma`: it is C++, has a ton of configuration and customization, and takes between 5 minutes to half-an-hour to build depending.

This is not to dissuade, of course. It will not be as simple as including the source files and mentioning them in the appropriate stanzas, but it would be incredibly valuable, especially for getting it working for other backends such as WASM. I’ll have to devote some time to investigating how readily this can be achieved. I do know that the previous `z-botan` managed some form of bundling.

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 12, 2024, 10:43pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/136 "2024-02-12T22:43:17Z")

</div>

# Major update: `0.0.1` release imminent

My laptop was in the shop over the weekend, and I am extremely happy to have it back. I have otherwise spent my time recently focusing on 2 things: Squishing a few critical bugs, and producing tutorials for everthing `botan-low`.

In that regard:

- Fixed memory leak in Botan.Low.PubKey.KeyAgreement
- Renamed pattern BOTAN\_PUBKEY\_PEM\_FORMAT\_SIGNATURE to BOTAN\_PUBKEY\_STD\_FORMAT\_SIGNATURE because it is not guaranteed to be PEM format.
- Created tutorials for:
  - Botan.Low.HOTP
  - Botan.Low.PubKey.KeyAgreement
  - Botan.Low.PubKey.KeyEncapsulation
  - Botan.Low.SRP6
  - Botan.Low.TOTP
  - Botan.Low.ZFEC

- Tutorials are now present in the README as well as their respective module’s haddock documentation

There are a few modules remaining left to document / produce tutorials for (Botan.Low.KDF, Botan.Low.PwdHash) but they are both non-critical modules, and will not hold up the release, which is planned for **tomorrow**! I’ll sneak them in if I can, but I don’t want to hold things up just for them!

The package candidates have been updated:

- [botan-bindings: Raw Botan bindings](https://hackage.haskell.org/package/botan-bindings-0.0.1.0/candidate)
- [botan-low: Low-level Botan bindings](https://hackage.haskell.org/package/botan-low-0.0.1.0/candidate)

So has [the repo](https://github.com/apotheca/botan). 🥳

---

<div class="post-metadata">

**Author:** ![Bodigrim](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bodigrim/32/1457_2.png) [@Bodigrim](https://discourse.haskell.org/u/Bodigrim)\
**Post date:** [February 12, 2024, 10:51pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/137 "2024-02-12T22:51:48Z")

</div>

> [@ApothecaLabs](#):
>
> It will not be as simple as including the source files and mentioning them in the appropriate stanzas, but it would be incredibly valuable, especially for getting it working for other backends such as WASM. I’ll have to devote some time to investigating how readily this can be achieved. I do know that the previous `z-botan` managed some form of bundling.

Yes, [`Z-Botan`](https://hackage.haskell.org/package/Z-Botan-0.4.0.0/src/Setup.hs) includes entire Botan source code and builds it during `Setup.hs`. Not bad, even while it requires Python and other tools installed, but I do not see what blocks us from distributing precompiled amalgamated Botan source at least for most popular platforms (foremost Windows, because this is where getting build tools is most cumbersome).

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 12, 2024, 11:10pm UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/138 "2024-02-12T23:10:05Z")

</div>

I don’t see anything blocking us per se, but from my efforts to extend it, I am aware that Botan C++'s internal build structure is highly unorthodox and does a bunch of include-path swizzling.

Even if it does give us Python and other tools as environmental dependencies, I’d rather we wrap Botan C++ entirely as-is (eg, as a submodule / subrepo pointed towards a specific botan commit), than create any sort of custom Botan C++ distribution unless it requires very little effort.

This subject (packaging C++ into a Haskell repo) is outside my area of experience, and I am not necessarily aware of all of the potential solutions to this problem, so I do appreciate any help or information.

Edit: I missed: _distributing **precompiled** amalgamated Botan source_ can you explain in a little more detail what you have in mind?

---

<div class="post-metadata">

**Author:** ![Vlix](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/vlix/32/1551_2.png) [@Vlix](https://discourse.haskell.org/u/Vlix)\
**Post date:** [February 13, 2024, 2:25am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/139 "2024-02-13T02:25:03Z")

</div>

Quick question, @ApothecaLabs

Does Botan have `PBKDF2`? I see it has `bcrypt` (even though you can’t supply your own salt), but other password hash functions like `scrypt` and `Argon2` also seem to be missing?  
Is that something that’s explicitly not in the `botan` library, or something they just haven’t gotten to yet?

---

<div class="post-metadata">

**Author:** ![ApothecaLabs](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/apothecalabs/32/3215_2.png) [@ApothecaLabs](https://discourse.haskell.org/u/ApothecaLabs)\
**Post date:** [February 13, 2024, 2:57am UTC](https://discourse.haskell.org/t/botan-bindings-devlog/6855/140 "2024-02-13T02:57:56Z")

</div>

Yes it does have them - several PBKDF algorithms can be found under `Botan.Low.PwdHash`:

- PBKDF2
- Scrypt
- Argon2d
- Argon2i
- Argon2id
- Bcrypt\_PBKDF
- OpenPGP\_S2K

The interface is a little more involved than in `Botan.Low.Bcrypt` and I don’t think it does fancy Modular Crypt Format, but it _does_ let you choose your salt, though according to notes, unit tests indicate that the parameter triplet ordering is inconsistent for `pwdhashTimed` compared to `pwdhash`, so it may take some finagling. See `botan-low/test/Botan/Low/PwdHashSpec.hs` for more information.

[Previous page](https://discourse.haskell.org/t/botan-bindings-devlog/6855.md?page=6)

[Next page](https://discourse.haskell.org/t/botan-bindings-devlog/6855.md?page=8)