[Haskell Cryptography Group] Botan: The First Milestone

After 8 months of work, we’ve reached an important milestone. Now we’re here to tell you about what is coming next in our Botan: The First Milestone blog.

With this milestone, the project enters a new phase in the software development lifecycle - maintenance and development. Your feedback and activity is important to us, and helps guide our decisions, so let us know what you think, even if it is only to say hello - every response you give is a tangible metric of interest that helps keep this project going!

(Sorry, the link preview failed)

11 Likes

I’m a crypton user who is interested in botan especially since it supports hardware-accellerated SHA2 and SHA3. At the moment, installing this looks rather hard; the C library first has to be installed (currently from source as it’s too new a version to get from a linux distribution). As many of my users install from source, and I have to deal with many platforms, this would be a significant impediment to using it.

If the C library were included in botan-bindings, it would make widespread use much easier, especially deep in dependency trees of the kind discussed in the blog post. See the sqlite package which I consider to handle this very well; it bundles a copy and has a build flag to use the OS vendor’s library.

As it is, I’ll consider using it eventually, but will have to start with it behind a build flag with crypton used by default…

2 Likes

One of the biggest pieces of feedback that we’ve received is the need for improved support for the installation of the botan3 C++ library. This was a recurring item, and we’ve heard you loud and clear.

Wrt bundling libraries, this is what happens:

You have to read the C source code to verify which version was actually bundled.


As such, both bundling botan C++ with the Haskell packages as well as doing automagic downloads of the latest version is inappropriate in my opinion. The latter is a can of worms and also breaks reproducible builds.

Instead, you should create a package that does nothing but provide the headers and the C source code. See bzip2-clib: bzip2 C sources as an example. The version of this package will then not follow PVP, but whatever botan upstream does. This way end users can also very easily track what they installed. It makes it easy for security audit tools to e.g. search for updates or warn users they’re using old versions of C libraries.

Then in the Haskell botan proper packages you introduce a flag system-botan, which:

  • is True by default
  • is automatic
  • uses pkgconfig-depends to search for system botan when True or falls back to the “botan-clib” package

This way you can precisely control which botan C++ library versions are supported and can have cabal just turn off system-botan if there’s no such package installed.

Then we need the haskell security response team to maybe come up with a strategy to monitor freshness of such clib packages (or someone else).

9 Likes

@hasufell I’ll have to remember to tag you into conversations more, you bring a lot to the table (and I believe you’ve mentioned this particular topic elsewhere as well). Balancing ease of use vs security vs staying up-to-date has been a recurring discussion and I want to make sure we get it right.

@joeyh It seems making ease of our next major goal was the right decision, then. You’re exactly the sort of user that I need feedback from, and want to make things easier for.

I don’t think automagic fetching via build-type: Configure is a great idea. It forces anyone who uses the package to run a configure script, even if they have a copy of botan available. It also seems finicky, particularly on Windows where you need a working MSYS.

1 Like

Btw… wrt windows: despite popular belief, you can install and link against botan as a system package just fine: Base Package: mingw-w64-libbotan - MSYS2 Packages

Both stack and ghcup document how to install system packages:

2 Likes

Oh and to be clear… people already asked me in private why I’m suggesting to bundle C source code: I’m not. The proper way is to link to system libraries always. But there are cases where:

  • maintainers have already done it, so there’s no way back (see bzlib/bzlib-conduit/bz2)
  • maintainers are planning to do it anyway one way or the other, which seems to be the case here to make adoption smoother

So if you’re already en route, I believe my suggestion is the least worst option.

1 Like

One thing I’d like to be a little explicit about - we’re committed to “Improved installation support”, and not any particular solution, so we’re definitely factoring this feedback into our approach - the end goal was always “we’d like for usage to be as easy as adding botan to your dependencies.”

With installation being such a hot topic, there are many differing needs, priorities, and opinions, so any proposed solution is to find out what needs, if any, it does or does not fill, and how to include it in the set of solutions - since solutions are not necessarily mutually exclusive, especially if we have things fall back appropriately.

This topic is worth a doing a writeup of its own in the future, eg “Practical security considerations for binding to C libraries” or something - it’d be great to have something to reference the next time we need to bind to a C library :slight_smile:


@hasufell @jackdk

One thing to note, is that if botan is not system-installed, then running the botan configure script may be an unavoidable necessity, even in the case of the “least worst option”. This is another factor of consideration that is easily overlooked. Practically speaking, this means that there may not be much of a difference pipeline-wise between fetching from a pinned commit version, and bundling the source code of that commit, as the steps afterward should be similar.

1 Like

I am not familiar with foreing dependencies and maybe I am not understanding the issue, but shouldn’t users be in charge of installing the botan3 C++ library themself? I mean, IIRC, when using (for example) haskell’s sdl bindings, you have to use your OS package manager to install libsdl, which is what I’d expected: cabal installs my haskell deps and just that.

Again, I am not a botan user, so I guess I am not understanding user’s needs. Any case, with cabal’s hooks (isn’t build-type: Configure in a deprecation cycle?), can’t you run a simple hook like “look for this system library, If it isn’t fail with user-friendly error message”?

Btw, congrats for the library!!

2 Likes

Ideally? Yes. But not every user is capable of managing the complex installation process, so we’d like to provide an alternative if they can’t.

That’s what the pkg-config flag and pkgconfig-depends stanza do already - it tries to find a system installation using pkgconfig as our first choice option, and falls back on other methods if that fails.

Currently, it immediately falls back to the last resort of requiring the user to pass include and lib dirs to cabal via flags --extra-include-dirs and --extra-lib-dirs flags (or permanently as stanzas, in your cabal.project or cabal.project.local file).

Any of the solutions being discussed now (that we do implement) will occur in between our first choice and last resort, in some order of precedence.

I don’t believe so - " The Configure build-type is fine because it can be implemented in terms of Hooks build type.". It appears several other build types like Custom are being deprecated, though.

Thanks! :partying_face: :cake:

1 Like

Even if we restrict our considerations to Linux, at the moment no users can install Botan 3 from OS package manager and this situation will not change any time soon. We are talking about, like, 10 years from now until all not-EOL’d distros upgrade to Botan 3. So the choice is really either a user building who-knows-what who-knows-how from sources or downloading a Hackage package which does it for them in a more controlled way. I’ll chose the second option every time.

3 Likes

Yeah, that’s in fact the reason ghcup was created. You should always install GHC from your package manager, but there’s that.

I opened Tracking cbits in the hackage ecosystem · Issue #162 · haskell/security-advisories · GitHub

2 Likes

this is slightly wrong nix has botan 3, but i guess your main point still stands.

2 Likes

Following the most popular distros, the situation is:

In that sense, mac and windows work fine (the platforms where people usually complain about system library shenanigans).

1 Like