# \[Initial feedback request\] DataFrame library

**URL:** <https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802>\
**Category:** Show and Tell\
**Created:** [November 25, 2024, 6:08am UTC](https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802 "2024-11-25T06:08:52Z")\
**Posts on this page:** 5\
**Page:** 2

<div class="post-metadata">

**Author:** ![artem](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/artem/32/56_2.png) [@artem](https://discourse.haskell.org/u/artem)\
**Post date:** [January 31, 2025, 12:48pm UTC](https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802/21 "2025-01-31T12:48:39Z")

</div>

I think it’s a good read for everyone thinking about this design space. Do I think it’s a holy graal and is correct on all cases? Certainly, not (and I don’t think such a thing exists).

---

<div class="post-metadata">

**Author:** ![mchav](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/mchav/32/5143_2.png) [@mchav](https://discourse.haskell.org/u/mchav)\
**Post date:** [January 31, 2025, 4:35pm UTC](https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802/22 "2025-01-31T16:35:43Z")

</div>

Didn’t read the paper yet but this seems like a good check list for operational completeness. Although it is missing transpose which the modin paper seems to suggest is a foundational data frame operator.

---

<div class="post-metadata">

**Author:** ![artem](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/artem/32/56_2.png) [@artem](https://discourse.haskell.org/u/artem)\
**Post date:** [January 31, 2025, 6:24pm UTC](https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802/23 "2025-01-31T18:24:59Z")

</div>

Transpose makes sense for matrices but not dataframes, which, in general, hold heterogenous data. Am I missing something?

---

<div class="post-metadata">

**Author:** ![mchav](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/mchav/32/5143_2.png) [@mchav](https://discourse.haskell.org/u/mchav)\
**Post date:** [January 31, 2025, 7:00pm UTC](https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802/24 "2025-01-31T19:00:59Z")

</div>

Around page 20 of the [modin paper](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-193.pdf). The argument is that a dataframe’s algebra can be distilled to a set of core operations. Transpose is one of them. In fact, most literature I’ve read suggests that a dataframe is a mix of matrix, relational, and spreadsheet algebra.

First discussed in the paper [Is a dataframe just a table?](https://drops.dagstuhl.de/storage/01oasics/oasics-vol076-plateau2019/OASIcs.PLATEAU.2019.6/OASIcs.PLATEAU.2019.6.pdf). I reached out to Hellerstein about these comments and he said his view on the issue have progressed and he largely accepts the definition proposed in Modin (granted he’s a co-author).

---

<div class="post-metadata">

**Author:** ![mchav](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/mchav/32/5143_2.png) [@mchav](https://discourse.haskell.org/u/mchav)\
**Post date:** [February 2, 2025, 10:19am UTC](https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802/25 "2025-02-02T10:19:51Z")

</div>

To be more precise: Modin’s definition of a dataframe is a 4-tuple (A\_mn, S\_n, C\_n, R\_m) where S\_n is a collection of schema induction functions (parsing function) from String to a Type. The underlying 2D array can be transposed. This would invalidate S\_n so it would have to be regenerated or initialized as a list of identity functions. Of course this is only a theoretical definition that’s almost impossible to implement efficiently. And I admit it does seem very post hoc but dataframes are weird…

Speaking to a lot of people that use dataframes as a way to clean data (as opposed to a query engine) this sort of operation is useful. And I think the example in the Modin paper is illustrative of an intuitive use of transposition.

[Previous page](https://discourse.haskell.org/t/initial-feedback-request-dataframe-library/10802.md?page=1)