Cradle: a simpler process library

#1

Hi all,

We just released cradle, a subprocess library (inspired by shake’s cmd) that aims to be simpler and more straightforward to use than process.

Our blog post describes the library and the motivation in a bit more detail.

Do let us know your thoughts!

13 Likes
#2

You write:

cradle also doesn’t have any shell-based functionality. We believe these are rarely, if ever, the right thing to use.

Is there any specific reasoning behind this belief?

2 Likes
#3

Took me a moment to realize but this is not a full replacement for process, it’s a wrapper on top of it.

4 Likes
#4

I haven’t had time to give the library a try yet, but the blog post makes it sound nice! Ironically, I spent a lot of time over the weekend wrestling with different process/streaming libraries trying to figure out how to do different things with stderr etc, so this is very welcome.

2 Likes
#5

Yeah, maybe I should have been more explicit. There are a few reasons:

  • The behavior is not really portable. Shell helpers often use /bin/sh (which is the only interpreter that’s required to be present by POSIX). But sometimes that’s bash masquerading as sh, sometimes dash, and sometimes sh, and the behavior is slightly different.
  • It’s not obvious what behavior to expect from these functions. In Python’s subprocess, for instance, the SHELL variable determines the shell. In Haskell’s process, it doesn’t (it’s just /bin/sh).
  • Most importantly, there are lots of security issues, especially around shell injection, that are easy to forget about

You can still, if you want to, use cmd "/bin/sh" & addArgs ["-c", "mycmd & etc"]. We just don’t make it as easy.

5 Likes
#6

Hey Julian :wave:!

I found this post surprising because I consider typed-process to be a kind of perfectly balanced API between the proliferation of names and overloading (to the extent I included it directly in Hell), and is what the maintainer of process points to as the better option in the package description.

The blog post doesn’t really motivate the need for an alternative interface, it compares to the process package, not typed-process. Its examples show that it needs generous sprinkling of type signatures to make a correct, self-contained invocation (without the hand of type inference), with similar verbosity to typed-process.

As a library, it doesn’t seem to do streaming out of the box, or support spinning up processes with withX patterns or startProcess, intentionally, or shells, which is a stance I’m in favor of (as I think it’s right to prefer native GHC concurrency and I never use shells), but does mean if you want to just learn and use one library, typed-process covers all your use-cases.

:thinking: I guess everyone has an itch to scratch. I made lucid when blaze-html was fine, I suppose! Guilty.

Thanks for sharing, I find it hard to make a design in this space, it’s all ergonomics and tradeoffs. Bold choices!

1 Like
#7

Consider using OsString for your API (e.g. program names and program arguments) instead of String.

Also see: Support `AbstractFilePath` · Issue #252 · haskell/process · GitHub

4 Likes
#8

I didn’t write cradle, so I can’t speak authoritatively about original motivation here, but I did use process a lot, typed-process for a while too (though had to switch back to process after hitting these bugs.), and now cradle, so I can compare them as a user.

I really enjoy working with cradle - it feels really ergonomic. I thought from the blog post that it’d be clear to most people that the ways in which that’s case, compared to process, are much the same as to typed-process, but apparently I was wrong. In particular:

  • You have a ton of different functions for each use case - reading just stdout, stdout and stderr, stdout and stderr but throwing exceptions… And then start and with versions for everything…

  • If you want to come up with another abstraction (such as the timing one in the blog post), you need to write a new function for each existing function. I find it really cool that with cradle you can write these, and it’s so trivial to use that functionality. Timing was one example, but others that come to mind:

    • newtype Json a: (Json MyType) <- run & cmd "foo" parses stdout with FromJSON and throws otherwise

    • newtype TempStdoutFile: (TempStdoutFile f) <- run & cmd "foo" creates a temp file, pipes stdout there, and returns the filepath (this needs a change that should be upcoming, though).

    • data SourceError = SourceError { file: FilePath, lineno: Int, colno: Int, msg: Text} (e :: SourceError) <- run & cmd "foo" parses stderr, using knowledge about common executables (ghc, gcc, cargo, etc) error formats, defaulting to maybe the GNU-style.

    I can’t really figure out how to replicate Timing, for example, in process/typed-process without either having a ton of new functions, or only supporting some of their APIs.
    To reiterate, these combine simply:

    (Exit e, Json parse, t :: Timing, s :: [SourceError]) <- run "nix" & addArgs ["build", ".#", "--json"]

Does the right thing. (In fact, even multiple uses of the same handle will work!). Of course, all of this is extra fancy stuff. 95% of the time it’s (StdoutTrimmed out) <- run "ls" or the like.

  • There’s the mostly aesthetic fact that a lot of functions ignore ProcessConfig
    parameters.

I don’t think any of these are really different from process and typed-process?

Regarding streaming and with*. As mentioned in the blog post, cradle generally has a compositional approach. If there’s already the wonderful async library (and the simple forkIO), that should be enough (we’ve definitely been using cradle that way.). And I already mentioned above the reasoning for not having shell, and how you can still mimic it pretty easily if you really want to.

Regarding type signatures: very rarely do you need them! I gave them in the blog post and above so the lines are self-contained and standardized. But of course the majority of the times where I gave them, type inference would quickly have settled things. E.g.:

run "cmd" >>= \case
  ExitSuccess -> doSuccess
  ExitFailure _ -> doFailure

In short, I definitely enjoy using cradle a lot more than I did typed-process. But of course, these are my own experiences. If you’re happy with typed-process I wouldn’t want to take that away from you, just like presumably typed-process shouldn’t take anything away from the people who were happy with process!

2 Likes
#9

Interesting! I still have reading to catch up to and maybe will soon find an answer, but do binaries looked up in PATH have the same characteristics as proper filepaths in this sense?

#10

It’s not really whether they are filepaths or not. Operating systems API doesn’t really treat them that much differently (although on windows there’s some magic parsing, which you can disable with the \\?\ prefix). It’s about the byte format that those APIs return. OsString preserves the original bytes, String does not, it decodes.

Also see bytestring parser · Issue #65 · pcapriotti/optparse-applicative · GitHub

2 Likes
#11

I don’t see any OsString functions in the process library, are you asking people to use CPP for this?

Okay, so after investigating this for a half an hour, the idea is that OsString is never going to expand beyond the (windows | unix) choice (every new platform will just use UTF-8 because it’s better), so the CPP here won’t magically break in the future.

I think it’d be better to have a -compat library for this for now. For example my command-line options parsing library accepts a list of strings, and the end users shouldn’t have to assemble parseArgs across two libraries just to be able to use it.

#12

If your library is a wrapper around process, then I guess we need to solve Support `AbstractFilePath` · Issue #252 · haskell/process · GitHub first