Benchmarking Haskell dataframes against Python dataframes

Links
1
21 Likes
2

This sounds very impressive! Great work.

Your high level design looked great and knowing its performance is competitive is exciting. I’m very much looking forward to your updates here. For what it’s worth, this is what the code for the groupBy operation mentioned in the post looks like:

groupByHaskell :: IO ()
groupByHaskell = do
  df <- D.readCsv "./data/housing.csv"
  print $ df |> D.groupBy ["ocean_proximity"]
             |> D.aggregate [("median_house_value", D.Minimum), ("median_house_value", D.Maximum)]
             |> D.select ["ocean_proximity", "Minimum_median_house_value", "Maximum_median_house_value"]

(That’s beautiful! And fast!)

I haven’t experimented with it yet, but even the docs are already coming along nicely. I’d love to see your project gain more traction :slight_smile:

6 Likes
3

BTW I think it would be good to display at the end of the groupBy section the code in Haskell, because the link click to the code is too small and I think it deserves some attention in the main page :slight_smile:. It may help unknowing people see from a glance what using it would be like.

3 Likes
4

Congrats, that looks good.

I think you could address the overhead of spawning a process by making a dedicated executable and spawning a process for it. Or maybe just do cabal run, but that probably has an overhead too.

4 Likes
5

Thank you. I was working on changing the API to be slightly more general.

groupByHaskell :: IO ()
groupByHaskell = do
  df <- D.readCsv "./data/housing.csv"
  print $ df |> D.groupBy ["ocean_proximity"]
             |> D.aggregate [ F.alias "min" (F.minimum @Double "median_house_value")
                            , F.alias "max" (F.maximum @Double "median_house_value")]

That way you can have additional logic in your aggregation e.g F.alias "rate" (F.sum "x" / F.count "x") as oppose dot a single aggregation function. Will update the post!

1 Like
6

Yeah. Cabal run might be a fair comparison. In general I probably need to have a more sophisticated benchmark. Polars’ strengths come through in multi threaded environments - and I think more investment has been put towards the lazy API and query planning for that. As things get more mature I’ll think more seriously about how to compare these in earnest but for now things are acceptable.

4 Likes
7

I expect that any deeper data analysis ends up with lots of derived columns. Hence it would be interesting to benchmark a dataframe that is generated from some static (or random) data through a series of transformations (lazy and strict). Since your Expr type is a GADT, you could even randomly generate the expressions for derived columns.
That way, not only the underlying array implementation gets tested, but also the library code that transforms the data.

3 Likes
8

That’s a good point. That said a lot of the expressions still boil down to vector operations for sufficiently large n on an in memory implementation. For the deferred evaluation API expressions may have a more central role (there’d need to be an optimization step).

Nevertheless I’ll spend the next few months trying to design a good benchmark.

1 Like