Convenience in the Haskell ecosystem

[sclv]

I’m starting to feel that a lot of really cool functionality is waiting on an exact-printing Cabal parser. It would be really nice to get a more coherent development effort around that. I don’t know what organisation exists around such things, though — is this the kind of project the Haskell Foundation would be willing to support?

There’s a pretty complex fleshed out discussion on a cabal ticket, which includes some recent activity of a few people interested in exploring this: Meta: Exact-printer Mega-issue · Issue #7544 · haskell/cabal · GitHub

If there was a person with demonstrated capacity and knowledge who made an HF tech proposal to get funding to do this, that would certainly be something the tech advisory group would seriously consider.

[bradrn]

@sclv

There’s a pretty complex fleshed out discussion on a cabal ticket, which includes some recent activity of a few people interested in exploring this: Meta: Exact-printer Mega-issue · Issue #7544 · haskell/cabal · GitHub

True, I’d forgotten that this issue exists.

[Liamzy]

About the cabal exact parser, there’s one standing partial solution right now.

GitHub - VeryMilkyJoe/haskell-language-server: Successor of ghcide &...

This is, iirc, missing the ability to prune data after the main text? Morever, I’m not sure about its general reliability; the parser tends to allow a lot of invalid Cabal files through (I’m told barring invalid Cabal files isn’t an objective). I think the HLS project is looking for someone to complete it, and this might be the fastest way to get an exact-parser.

I have my own, but it’s sort of embarrassing (I can parse whitespace and comments, and everything else as “raw”). GitHub - liamzee/flatparse-cabal-exact: A flatparse-based library for exact-parsing cabal files.

---

As for the project itself, a very skilled Haskeller could probably push out a prototype in 25-50 hours, then test and refine it in another 25-50 hours; i.e, I wouldn’t consider the parser that challenging.

The bigger problem, rather, is the information in the Github thread; that is to say, the Cabal file format seems eccentric, and is defined by the parser’s behavior.

The Cabal codebase dates all the way back to 2003, at the very least, and it’s a bit hard to be onboarded onto. Actually building a high-quality parser requires going through a medium-large codebase, understanding how the parser is called, and how the parser works, and the Cabal megarepo itself is eccentric:

It seems that originally, the Cabal executable (cabal-install) didn’t exist as a separate package, and Cabal was usable on its own via Setup.hs. cabal-install is actually hooking into Cabal, just like Stack, and there’s a bit of kludge in doing so; it works, it works well, but the codebase is harder to understand as a consequence.

---

The fastest way to do it would be just to have someone familiar with the Cabal codebase and moreover, the Cabal parser, rig something up in a few weeks.

The second fastest way to do it would be to find a more skilled Haskeller than me to build the exact-parser, possibly based off VeryMilkyJoe’s version; I think it might take a month at most to understand the Cabal parser in its native context, understand it, then a couple of days of work to build the parser, two weeks, tops.

However, I’m still going to try to do it as a personal challenge. As I am seeking medical care right now, I have a lot of spare time, and suspect I might be able to get something decent by February at the latest.

If someone beats me to the exact-parser, I’ll still hopefully understand the Cabal codebase by then and be able to help improve documentation for it.

[bradrn]

Let me quote @george.fst from another thread, since I think his comments are also relevant here:

@george.fst

tl;dr I think the records problem is basically solved, and we just need to encourage turning on the right extensions.

[malteneuss]

Thanks for that great initiative. Speaking of convenience: I’m working on improving the documentation in the Haskell ecosystem and it would be really nice to automatically also get a space to host companion pages with tutorials, how-to’s etc to not clutter the regular Hackage code reference documentation pages.
E.g. a setup like Aeson is hard to read, hard to find (bad SEO) and hard to navigate.

Currently, if you want to separate these pages, you have to manage and host the companion pages yourself like Cabal or Yesodweb do, which many package maintainers may not have the time for.

[Kleidukos]

That’s also a goal for the development of Haddock, to have a space for guides that is integrated in the generated HTML. Feel free to send me a message if you want to collaborate on this.

[tomjaguarpaw]

@malteneuss

I’m working on improving the documentation in the Haskell ecosystem and it would be really nice to automatically also get a space to host companion pages

Sounds like a great project! Do you have a link to what you’re working on? Perhaps the Haskell Foundation or Haskell.org committee could provide a space to host these docs. (I’m on the board of both, and happy to help try to make this happen.)

[sullyj3]

@rhendric

I don’t know what one would expect a cabal add to do with any moderately interesting Cabal file.

At the least, you could have it work for the simple case and print an error for more difficult cases. That would be a strict improvement. No need to let the perfect be the enemy of the good. Maybe an implementer looking at it in more detail might be able to figure out reasonable behaviour for the more complex cases; if so, bonus.

[rhendric]

@sullyj3

At the least, you could have it work for the simple case and print an error for more difficult cases. That would be a strict improvement.

It’s not necessarily a strict improvement if it’s a habit that newbies learn that they then have to unlearn later, or if it deters people from using Cabal to its best potential. I can anticipate the Medium posts: don’t do [moderately sophisticated thing with Cabal files] because it breaks cabal add.

I am all for bringing new people into the Haskell ecosystem, but we should do it by bringing them to the actual Haskell ecosystem, not by setting up a fenced-in newbie area of the ecosystem that they have to leave in order to join the rest of us.

[Ambrose]

@rhendric

I can anticipate the Medium posts: don’t do [moderately sophisticated thing with Cabal files] because it breaks cabal add.

I think the dream of cabal add is for greenfield projects with fresh cabal files. Although fundamentally, cabal add is just gonna be a CLI that reifies all the lenses into build-depends in a cabal file, right? It’s a command line DSL for editing a specific file.

I agree that all told, cabal add is not actual super useful. It isn’t hard to just…edit a text file. But (whether it’s real or dramatized) apparently that’s hard enough for people that they quit Haskell?

I say “apparently” not sarcastically but literally. Like it’s an actual surprise to me that that could be true

[hasufell]

There are only a small number of possible if conditions in Cabal files (compiler, compiler version, architecture, platform, flags… did I miss any?). It doesn’t sound hard allowing a cli interface to “navigate” that path down.

Auto-detection seems too hard. You could assume the state of plan.json, but that’s also just a guess.

[vanceism7]

The thing is, in most other languages, adding a dependency is trivial, and usually there’s a nice command to make it happen. With cabal, as a newcomer, it’s quite difficult initially to figure some of this stuff out.

The cabal user guide is huge, and it’s quite easy to miss the section on how to add dependencies (which btw, seems to only be simply demonstrated in the quick start guide, in a very brief fashion). This is aggravating for someone who has set out to work on creating software. We want to build programs that solve our problem domain, not flip through the manual of a build tool trying to figure out how to add a dependency when in most other languages, it’s tool add <package>.

When you have business people checking in with you daily to see what your progress is, you don’t want to report back to them “I got stuck trying to add a dependency for half the day”. When people hear that haskell is this amazingly productive language and then get stuck for half a day trying to figure out the build tool, they’re gonna quit and go back to blub where its easy to do it.

Cabal is a complex and powerful tool, but not everyone wants to know about it. cabal add streamlines the process so people can focus on their work. Adding dependencies in cabal is trivial once you know how to do it, but it isn’t before then.

(Sorry btw @Ambrose, I didn’t mean that rant specifically towards you, I just clicked the reply button too quickly lol)

[rhendric]

‘The documentation is too opaque; let’s enhance the tool’ is certainly an argument. :emoji_conveying_good_natured_humor_and_not_an_attack:

[vanceism7]

@rhendric

:emoji_conveying_good_natured_humor_and_not_an_attack

Yea haha, addressing the documentation is one way to remedy the situation, and definitely a reasonable one; but I think adding some convenience to the tooling is a good thing.

Convenience is never usually essential or even very impressive, but it’s so influential. I think software lives or dies by convenience.

[rhendric]

@vanceism7

The cabal user guide is huge, and it’s quite easy to miss the section on how to add dependencies (which btw, seems to only be simply demonstrated in the quick start guide, in a very brief fashion).

Snark aside, I think this is a very valid complaint and I’d love to see the Cabal docs reorganized to address it, regardless of whatever effort goes into designing and implementing an imperative interface on Cabal files.

[bradrn]

@rhendric

Snark aside, I think this is a very valid complaint and I’d love to see the Cabal docs reorganized to address it, regardless of whatever effort goes into designing and implementing an imperative interface on Cabal files.

I completely agree. Even as someone who knows what I’m doing, I find the Cabal docs quite hard to navigate. Perhaps I’ll have a think about how to reorganise them.

(Also, where is the best place to propose and discuss such documentation changes?)

[Lsmor]

@sullyj3

At the least, you could have it work for the simple case and print an error for more difficult cases.

what’s wrong with cabal add [pkg-name:][ctype:]component <dependencies>. For example

• cabal add my-executable optparse
• cabal add my-library containers
• cabal add all text

I think, It is a good cli interface. The only (not) problem is that newcommers should learn that cabal uses components to create a package, something that (based on stack overflow questions) is not very well known.

@Ambrose

I agree that all told, cabal add is not actual super useful. It isn’t hard to just…edit a text file. But (whether it’s real or dramatized) apparently that’s hard enough for people that they quit Haskell?

yes but when you get the classic Module.Name belongs to hidden package xyz, consider adding it to your dependencies you push people to edit a file with very specific ad-hoc format. If that faliure was run cabal add component xyz people will be faced with something very common, which is running something in the console. And that is a very good improvement

@rhendric

I think this is a very valid complaint and I’d love to see the Cabal docs reorganized to address it,

Notice there are current efforts in this direction.

[Bodigrim]

I might be missing something, but

hackage-cli add-bound DEPENDENCY VERSIONRANGE CABALFILE

works for years. This is a solved problem.

You don’t need any exact-printing infrastructure to add a dependency: it’s absolutely fine to have multiple build-depends sections in a single component.

[tonyday567]

@Bodigrim

hackage-cli add-bound DEPENDENCY VERSIONRANGE CABALFILE

Yes, but what’s missing in the ecosystem is something to tie this together.

Cabal via the compiler tells you to add a package, that it’s hidden, say. You have to then jump on to hackage to find out the latest version, try and remember PVP to make a range up. Or add it manually to the file, run gen-bounds, find a pencil, or copy and paste the result from the command line back into the file manually.

All of this information is already available to cabal to do the entire task, but there are all these human steps needed that feel like unnecessary points of manual labour and potential bug generators which feel like they could be easily automated.

[Bodigrim]

@tonyday567

Yes, but what’s missing in the ecosystem is something to tie this together.

Agreed, but my point is that there is no technical challenge to implement cabal add (or a standalone cabal-add executable), which would be the first step in the direction. This can be done today.