Convenience in the Haskell ecosystem

#63

Currently, a package set is a set of packages/versions that can all work together. Meaning, among other things, that you could use every module from every package at once, if you were so inclined—no module name collisions, no dependency conflicts. In my fantasy, a package set would just be that collection of modules—the concept of ‘package’ would be an implementation detail for publishers, not consumers. You wouldn’t download the entire package set, or individual packages; you’d ask the package set server for the modules that your project imports, and the package set server would get you their transitive closures.

If I’m the author of package foobar, publishing foobar entails submitting a request for the next package set to include a set of module files I provide. The important part is, I tag this set of module files with foobar primarily so that the assemblers of the package set know to remove the previous set of modules tagged foobar if they accept my new submission—consumers won’t access the module files I submit by their package name, only by their module names. (Package names and versions can still be presented to users so that they can report issues and whatnot—this is more about changing how modules are resolved, and making that a one-step package-set-to-module link instead of a two-step package-set-to-package-to-module link.)

1 Like
#64

Thanks, this is an interesting vision.

1 Like
#65

I think the problems I could foresee are:

  1. You still need packaging. Library authors leverage them and PVP. And package sets benefit from that work.
  • I guess this would only be useful for “applications” - but that kind of goes against the “everything is a library” mantra. Most Haskell applications also make themselves available as libraries.
  1. Every package set user ends up bringing in extra dependencies anyways. I guess they just become part of the global store?

Could be a cool feature though! I already like to do ghcWithPackages for Nix. This is in spirit the same but lazily builds modules as needed.

1 Like
#66

This is what it’s like using a monorepo. For my personal project, I build with shake, and I declare a list of ("name", "Path/To/Main.hs") and I can do mk build/opt/name. Whatever it imports, it imports. At work it’s similar except it’s a hermetic build thing so you have to declare all the files and deps so there are package declarations, and they add the ability to have private modules.

So haskell already doesn’t have to have packages, and the development experience is indeed better, but of course this only works in a system with global control. What if you want to share some modules with the outside world?

Authors are always breaking APIs and adding features, and users always want the new features from A without the new breaking API from B. A vetted snapshot of compatible packages like stackage is always too old or too incomplete, so a patchwork of semi-consistent versions seems unavoidable. So I don’t really see a way around in an uncoordinated world. But if you can make a little consistent island and live there, it’s nice. Making it is work though, the bigger it is, the more work!

But perhaps the Unison people have had some thoughts about how to do it. I see they have packages and versions, I haven’t actually used the language so I don’t know how they assemble mutually compatible subsets of them.

2 Likes
#67

5 years from now there might be different maintainers… or stack and cabal may have merged, or been superseded by something better than both ?

2 Likes
#68

Interesting ideas, and stack’s script command is a step in that direction (I believe it will automatically add some common packages based on your imports).

#69

Very interesting, thanks!

From what I can see, this file looks very repetitive… is there no way to compress it somewhat to reduce transfer and parsing time? It would be really great to figure out a way to include this in GHCup somehow.

From a purely user-centric perspective, this seems obviously the most convenient option. But given the very significant effort which would be required to coordinate this, it’s equally obvious that it can’t happen any time soon.

#70

Just to remember every one that sooner that we think, all of this will sort itself magically

User

cabal file for a project reading a csv and producing a plot

ChatGPT

To create a Cabal file for a Haskell project that reads a CSV file and produces a plot, you’ll need to specify the project dependencies in the .cabal file. You’ll likely want to use the csv library for CSV file handling and the gnuplot library for creating plots. Below is an example of a simple .cabal file for such a project:

cabalCopy code

-- Example.cabal

name:           csv-plot-example
version:        0.1.0.0
cabal-version:  >= 2.0
build-type:     Simple

executable csv-plot-example
  main-is:        Main.hs
  build-depends:  base >= 4.14,
                   text,
                   csv-conduit >= 0.7,  -- For CSV parsing
                   gnuplot >= 0.5      -- For plotting
  hs-source-dirs: src
  default-language: Haskell2010

In this example:

  • name and version are set to the name and version of your project.
  • build-type is set to “Simple” for a basic project structure.
  • executable section specifies the details for the executable you want to build, including its name (csv-plot-example), the main source file (Main.hs), and the list of dependencies.
  • build-depends includes the required dependencies: base for basic functionality, text for text processing, csv-conduit for CSV parsing, and gnuplot for plotting.
  • hs-source-dirs is set to “src” to indicate that your source files are in a directory named “src.”
  • default-language specifies the Haskell language version you want to use.

After creating this .cabal file, you’ll need to create the actual Haskell source code for your project, including a Main.hs file that reads the CSV and produces a plot using the mentioned libraries. Don’t forget to import the required modules from the dependencies in your Haskell code.

Once you have your source code ready and your .cabal file configured, you can use cabal build to build the project and cabal run to run the executable.

Here’s an example of what your Main.hs file might look like (assuming you have already written the code for CSV reading and plotting):

haskellCopy code

-- Main.hs

import Data.Csv
import Graphics.Gnuplot.Simple

-- Your CSV reading and plotting code goes here

main :: IO ()
main = do
    -- Read CSV and produce a plot
    -- Replace this with your actual code
    let dataPoints = [(1, 2), (2, 3), (3, 4)] -- Replace with your CSV data
    plotPath [] (Data2D [Title "CSV Data"] [] dataPoints)

Replace the placeholder code with your actual logic for reading the CSV file and creating a plot.

#71

It is more that just discoverability it is also about curating, the proposed packages have been selected by the community.

but little else

What else are you expecting ? I though you wanted something battery included so basically text, containers, possibly aeson, maybe cassava for csv, some networking lib, and sql one (you’ll need to chose between mysql, sqllite, psql etc …). Maybe a new Prelude, a simple hhtp server ?

There are already too many possibilities for

Listing them all and let the user uncomment is actually a very cheap but really effective way of doing so withouth having to manage the exponential complexity.

Fair enough. What about resurrecting Haskeleton (or equivalent) and have GHCup installing it ?

1 Like
#72

You might be surprised, but the actual transfer time is not the bottleneck.

Also note that you now coupled the following things with GHC:

  • release management
  • release CI

Whether you consider that a good or bad thing is up to you to decide. I’m skeptical.

I believe one good side effect of the decoupling is that we have a good argument for demanding a stable GHC API. If we move every major project into GHC that needs a lot of its “internals”, then what do we end up with?

2 Likes
#73

This is true: a commented-out Cabal file solves that much.

However, I am of the opinion that defaults matter — a lot. Basic types such as Text, Map, ByteString, State, FromJSON are really widely used in Haskell code, and I want to make it as easy as possible for people to get started using the Haskell ecosystem. Leaving packages commented-out by default does not achieve that goal.

I’d like the selection to be as unopinionated as possible. So: most (if not all) of the boot libraries, probably aeson and cassava, probably a networking library, maybe some other common utilities like optparse-applicative or even lens, but nothing related to webservers or SQL (because there’s too many competing options), and definitely not a new Prelude.

That would be fine with me as a first step. I’d be interested to hear @taylorfausak’s views on it (as the creator of Haskeleton).

Like I said, I’m also working on my own idea for a templating application: hopefully I can share a first version in a few days.

Hmm, then what is? (Surprise me!)

I agree with this. It’s a good point.

2 Likes
#74

Process spawning (we use curl, not non-audited TLS haskell libraries), establishing connection overhead (HTTPS etc), yaml parsing. Although parsing is fairly fast now too, even with multiple MB, so there’s a constant overhead and a very low increase with size. I don’t think compression will get us anywhere.

1 Like
#75

Haskeleton started as a repository that people could use as a template. Then I heard about hi and converted it to work with that. Then Stack added the ability to initialize projects with templates, so I added it to that.

All that to say, there are plenty of ways to make templates for Haskell projects. Haskeleton has done most of them, although I stopped maintaining it a long time ago. These days I would probably just make a template repository:

6 Likes
#76

It would be very nice to simplify the distribution story for HLS. In case anyone is interested, I attempted to summarise the issues here: HLS and the user's GHC · Issue #3671 · haskell/haskell-language-server · GitHub

7 Likes
#77

I feel like this has been a pretty productive conversation, with a couple of different actionable items that have come out of it. They’ve been summarized above atleast once, but I figured maybe I should re-summarize (along with ideas I’ve seen floating around in other posts)

I see a couple of different potential project ideas:

  • Some type of templating system for cabal/stack
  • cabal add <package> (for adding dependencies)
  • Some type of cabal gen-deps command which, from imported modules, can automatically insert all required packages into build-depends. (This seems like a natural extension of cabal add <package>)
  • A minimal fallback ide server for HLS (integrate dante into hls?)
  • A better story for editor support outside of vscode (like emacs)
  • Local Modules / Fixing the “duplicate field name/duplicate constructor” problem
  • Make code formatting tools easier to use

I dont know if something exists for this or not, but I can envision some type of “Haskell Convenience Committee” or “Haskell Tooling Committee”, or something! It would be nice if there was at least some type of Haskell Foundation official repo for tracking requests/conversations about reducing all the papercuts in the ecosystem.

These concerns are obviously very spread out amongst all the Haskell tooling; most of them would require the insight and cooperation of the various project maintainers to decide if the ideas are feasible, to help steer the ideas in a feasible direction, and to aid with guidance or implementation. But if there’s not already an official place for storing wish list items like stuff we’ve discussed here, I think it would be helpful to create one.

3 Likes
#78

There may well be benefits to having something more official, but in the meantime there’s @tomjaguarpaw’s Tilapia, which aims to track a lot of these cross-project issues.

1 Like
#79

That is correct: with stack script, if no packages are specified, all the required packages that are in the snapshot or are a GHC boot package are deduced by reference to the import statements in the source file. The base package associated with the version of GHC specified by the snapshot is always available.

1 Like
#80

@vanceism7 That’s a really great summary, thanks!

I don’t recall this being discussed, though; could you elaborate please?

I second this. The whole point of the Haskell Foundation is to coordinate such things, after all.

Oh, I didn’t know about this. It doesn’t seem to be too well-known, unfortunately, which limits its usefulness. A good first step might be to make this more ‘official’.

#81

Imho the best place for a lot of these things to live is as enhancement requests in the trackers for the individual projects. If its for projects that don’t yet exist, the issues section of the haskell tech proposals repo is a place to create tickets that might later involve into proposals seeking HF support Issues · haskellfoundation/tech-proposals · GitHub

#82

This wasn’t discussed here, but I referenced it in the neohaskell conversation. One of the very annoying bits of Haskell for me is the whole issue with duplicate record fields and duplicate data constructors. It’s quite annoying that i can’t do:

data Person = Person
  { name :: Text, ... }

data Company = Company
  { name :: Text, ... }

-- Or
data Breakfast = Eggs | Banana

data Lunch = Sandwich | Banana

Without running into duplicate name errors. Even if you use the duplicate fields extension, you still run into ambiguous type errors. The only way to currently work around this is to move the types into separate modules, or add prefixes to the names, which is a big pain. Richards local Modules Proposal would give namespaces to these things by default, so you could access them like Person.name or Breakfast.Banana.

Yea that’s true, but I also think there’s some value to aggregating these types of requests, so there’s a centralized place where people find all the different initiatives being worked on to improve QOL.

I didn’t know about Tilapia either, that’s really cool. Making it an officially recognized haskell foundation repo is probably a good next step. I think we need to do more to make it known that we care about this as a community, and maybe help focus efforts towards fixing some of this stuff

4 Likes
Scotty-0.20 released