`cabal-audit` - please try it out

Show and Tell
#1

Hi! I have made a small tool that can check your haskell applications against the security-advisories database kindly maintained by the haskell security working group.

It runs the cabal solver, then checks out the security-advisories repo and tells you about matches with the security-advisories database.

Output is available in human-readable and machine-readable form.

The repo can be found here: GitHub - MangoIV/cabal-audit: audit your cabal projects for haskell-specific security-advisories

Thank you for testing!

image
image871Ă—900 152 KB

this is what running cabal-audit on itself looks like :slight_smile:

29 Likes
#2

This is quite exciting and even more of a reason to put more effort into providing a way to install “arbitrary” binaries via ghcup: Intaller DSL · Issue #141 · haskell/ghcup-hs · GitHub

The other option is that we start providing some collection of cabal + cabal related tools (e.g. cabal-cache, cabal-fmt, cabal-audit) in one bindist.

8 Likes
#3

This is super interesting, thank you for building it!

I have some questions:

  • would it work with stack?
  • how does the cabal build plan look like?
#4

If you use stack’s freeze files for cabal, it’ll probably arrive at the same package set.

E.g. https://www.stackage.org/lts-22.22/cabal.config

Or give stack2cabal a shot.

#5

I’d be happy to help there. I’m currently trying to get cabal-audit on hackage such that this can be installed with cabal install cabal-audit but I’m also worried about not having trivial-ish distribution.

#6

What do you mean? How does the Haskell type look like? How does building the plan work? Or?

#7

As well as distribution by ghcup, we could think about making cabal capable of installing dev tools, which is pretty common as a feature. I couldn’t find an existing issue, so I made Install project-local dev tools from Hackage · Issue #10016 · haskell/cabal · GitHub.

2 Likes
#8

prior art: in cargo it appears to be as follows:

  • cargo_audit can be installed by running cargo install cargo_audit, cargo has a similar feature to cabal >= 3.12 where subcommands are invoked if there’s as cargo-<subcommand> command available in $PATH.
  • crates.io has a directive Development tools::Cargo plugins, we should probably add a documentation section in the cabal repository that asks the implementer of a plugin to tag their hackage upload with cabal-plugin (or cabal-plugins if you will)
  • we should probably, similarly to cargo (Third party cargo subcommands · rust-lang/cargo Wiki · GitHub) have a documentation page that tells the cabal user about available third party plugins
#9

If cabal had proper binary handling and a good install interface, this wouldn’t even be needed. You could simply create a “meta” package that depends on all those other packages. Then an install wouldn’t just install executables of the immediate target, but also of the dependencies.

This is what linux distros do too.

Some have the concept of package sets, which just act as aliases Package sets - Gentoo wiki

cabal.project is already an underspecified mess :upside_down_face:

#10

Hello @MangoIV , thank you for pushing that work! During the last HSRT meeting we talked about providing a better format for such a tool so that the end users don’t need to have git. For example, when updating the website, we could push a sqlite or a tarball with the relevant files, would that helps cabal-audit?

Also, I previously worked on another cabal-audit tool and I just pushed an update to the README to points at your implementation. You can find my experiment here: GitHub - TristanCacqueray/cabal-audit

#11

I proposed that in an issue to haskell/security-advisories repo and this got rejected, as far as I understood the issue was that this would be an issue security wise? (offer API to request security advisories · Issue #166 · haskell/security-advisories · GitHub)

edit: I misremembered, there was already the proposal of pushing a single artifact, I proposed an actual API

Also sorry, I wasn’t aware of your … quite elaborate? :smiley: experiment, super cool! How does it determine which declarations are affected by an advisory? (i.e. I didn’t see machine readable information about that in the advisories itself?)

#12

otherwise, yes that would be nice, I found it quite troublesome that some of the information about the advisories is hidden in the git history, i.e. the advisories DB is not parseable without knowing the git history as the timestamps (?) rely on that.

#13

I think there was some miss communications. Providing a machine readable format for the advisories is planed, it’s running a web service for it which was deemed out of scope. I think we are looking for a static file format to be published along with the website. See the following feature request: advisory distribution and caching mechanisms · Issue #170 · haskell/security-advisories · GitHub

This information is not yet available. We need to specify a format, for example:

  • Module.name
  • Module.Typeclass x Instance y method z
  • How to deal with CPP?
  • What about deriving strategies?
#14

I wonder if there’s really a complete way of doing this, sounds like an interesting problem, like, even if you manage to get no false positives for a single build, how do you avoid that you don’t get false negatives, I guess this would require the end user to run the auditing on all the different setups they also use the program on…

Currently, tbh, I’m more concerned about false negatives, especially as the amount of actual advisories isn’t so vast as that they can’t be checked for applicability by hand. Ideally any such advisory should be fixed anyway, no matter if the specific definition is used in your project, no?

#15

That’s right, a given executable target may produce different callgraphs depending on the setup, so you’ll have to audit each individual build. I agree with you, false negatives are more important right now and I think your approach is better for the cabal-audit tool.

However, it does not seems like HSEC-2023-0007 is going to be fixed, so I worry such a warning will reduce the effectiveness of the audit tool if it is always reported.

1 Like
#16

I’m curious why there are advisories that don’t get fixed… But I guess this is a different topic.

#17

It would be cool if ghcup would be able to install ghc plugins too, which are just libraries rather than executables.

#18

Since they are like libraries, doesn’t cabal's existing plugin support work for this?