Hasura migrating to Rust?

Links
#29

It’s not just about the language. It never is.

It’s about ecosystem, community knowledge, resources, reliability of performance, etc.

C++ has been used for these things for decades. Rust is nowhere near that. Yes, you can probably do similar things, but you’ll likely end up ffi’ing into C/C++ libs or writing your own wrapper libs (you can do that with Haskell anyway).

Some time ago I also remember talks that the generated llvm IR is inefficient and has a lot of unnecessary allocations. But that might have been fixed.

1 Like
#30

…perhaps because it was the least-worst option available - if it arrives as scheduled, the new GCC Rust front-section could change that in a big way.

Having said that…this isn’t the first attempt at a second Rust implementation. For Hasura to make their announcement now, before Rust is fully supported in GCC, is a rather speculative decision - if the current attempt at a GCC implementation also failed, they will be moving from one single-implementation language to another.

instead of remarks like e.g:

New All-Rust Codebase

  • Full Rewrite in Rust

…tend to suggest they haven’t switched entirely over to Haskell. Did Hasura give an estimated completion-date for this migration?

#31

Thanks for saving us from sheer speculations.

#32

Still, some of these are important to us Haskellers, e.g. ecosystem and hiring concerns. How can I start a Haskell project at my company if I can’t guarantee them a sufficiently large ecosystem and sufficient applicants? And if these aren’t given, then there is something that we as a community can improve on.

3 Likes
#33

Ehm. I think industry has to improve on that.

Start investing, let your employees work on open source, support the HF.

Stop taking things for granted.

6 Likes
#34

You’re absolutely right, morally. Industry should make steps here. I just meant that there might be potential to act.

2 Likes
#35

Honestly somehow this thread sounds like the thread about haskell dying…

2 Likes
#36

Almost every programming language intended for practical use that predates Rust will be in this predicament sooner or later…

1 Like
#37

One line in a blurb somewhere and a thread full of sheer speculation? :smiley: Sounds a bit like a rorschach test…

14 Likes
#38

So here’s something less speculative, and also based on experience:

3 Likes
#39

I’m interested in Python to Haskell conversions (I view Haskell as a Python in terms of expressivity relative to Rust, with much less performance penalties and actually greater correctness than Rust); the other conversion of note would be Key.me moving from Python to Haskell, but it was roughly a monkey move wherein they slapped XStrict on everything.

What have been the challenges with moving from Python to Haskell? Python is notoriously anti-functional, so has it been hard training Pythonistas in an FP idiom? You’ve mentioned where things have gone right, but where have things gone wrong?

Within your start-up experience, what’s Haskell’s key selling point for a sector wherein 67-85% of firms fail within 5 years?

3 Likes
#40

Before I answer, I need to emphasize that we didn’t convert any Python code to Haskell; broadly rewriting would not be pragmatic for us because:

  1. Our current Python codebase works fine in many cases;
  2. We aren’t ready to spend time/money training devs to learn Haskell;

What we’ve done is expanded the business into a new area where there is <10% overlap with our current business. I work for a proprietary trading firm, so concretely this means that we’re preparing to trade entirely new products where the code path will be >90% Haskell. This is done by a 2-person team part-time for now (this work often gets de-prioritized in favor of more pressing matters).

Such green-field projects are a much easier avenue to start using a different language because new code needs to be written anyways.

As an aside, I am under no illusion that the entirety of our trading operations will switch to Haskell in my lifetime; the cost-benefit analysis just doesn’t make sense:

  • All products we’re trading right now have lax latency requirements (many of our competitors actually still trade manually!), and so there’s no tangible performance benefit to switching from Python to Haskell;
  • Our current trading operations are acceptably stable, such that the opportunity cost of software failure is probably less than the cost of training devs to learn Haskell + a wholesale rewrite.

Now to your questions:

Python is notoriously anti-functional, so has it been hard training Pythonistas in an FP idiom?

We haven’t started training new devs yet. However, after discussing this possibility at informal events with my colleagues, I encountered a strong resistance in many of them to change from the typical object-oriented Python. I get it; I can’t expect everyone to be comfortable investing work-time into an unfamiliar programming paradigm, when programming is just a job for most people. I would estimate that only about 1/5 of devs would be interested to be trained.

Pragmatically, I will look for a different type of candidate in future hiring situations – prioritizing people with ( knowledge of Python ) AND ( (interest in learning Haskell) OR (at least basic knowledge of Haskell) ).

You’ve mentioned where things have gone right, but where have things gone wrong?

Apart from the silly memory leak I mentioned, nothing went wrong per se. One of the additional challenges with respect to Python was that we needed time-series data structures that weren’t readily available off of Hackage, so we had to build our own library (to be released officially in the next few months, probably after a community RFC). In general, one of the drawbacks of literally any programming language vs. Python is the breadth of the ecosystem. On the quantitative research side of the business (i.e. stats and machine learning), you can be sure that we’ll be using Python for years to come, simply due to the ecosystem.

Within your start-up experience, what’s Haskell’s key selling point for a sector wherein 67-85% of firms fail within 5 years?

If I had to summarize to a single key selling point, it would be “ease of refactoring”, or in business-speak, “high iteration velocity”. Startups trying to build a minimum viable product, or achieving product-market fit, or expanding to new lines of business (like us) are working with fuzzy requirements and many unknowns. In this context, the ability to adapt to changes in requirements quickly is worth key.

I’ll be sure to write a postpartum blog post once the system is live (probably by the start of 2024).

8 Likes
#41

So, “Haskell isn’t as pure as people make it out to be–pure is actually a technical term when it comes to functional languages. We can support both OOP and procedural programming via some language facilities and libraries, but idiomatically, we’re the most functional language out there” doesn’t fly?

#42

Are there good examples that showcase the use of compact regions?

One possible use-case is “deduplication”. A large series (10s of millions) of tasks have subtasks that can be part of multiple tasks, and need to be performed just once, with the result shared. The naive solution is to use a map of some sort to store subtask id to result (pending or final). As the map grows GC costs become quite noticeable.

Is there some way to use compact regions to tame the GC cost of progressively building a map with O(10M) keys (and associated values)? Is there a “compact” map type that holds all its keys and values in a compact region, how does it avoid hanging on to too much “garbage” as updates replace the root and internal nodes?

#43

I tried using it in my IRC client to store the message history. I never measured to see if it actually helped, but message history is most of what my client spends its memory on.

1 Like
#44

If Alexis King is a noob I’m off to Google “barrista courses”

6 Likes
#45

Just on the topic of performance and GC, it seemed to me like linear types have a big role to play there, but I haven’t seen much in the space, especially w.r.t. adoption by libraries. Is the linear types vision alive? Or does it just need to be louder in its efforts?

#46

Just on the topic of performance and GC, it seemed to me like linear types have a big role to play there, but I haven’t seen much in the space, especially w.r.t. adoption by libraries. Is the linear types vision alive? Or does it just need to be louder in its efforts?

Hi. The linear types project is still very much alive. The time that I, and my colleagues have, for it is less that I’d hope (hence the relative silence), but we’re making progress. Adoption is rather slow (at least, slower than I’d have hoped), and I must confess that there are some good reasons: integration of linear types doesn’t cover all of Haskell. The most embarrassing is that “let” bindings are not compatible with linear types! This is something that I’m, finally, about to propose a patch for (it was a surprisingly long journey), see #18461.

The next big thing, for me, will be to integrate the work that we’ve published last year on linear constraints, which I’m hoping to be great help for the design of comfortable APIs.

23 Likes
#47

I worked on around 25 industrial Haskell codebases during consulting at FP Complete. I also participated in groups of Haskellers learning Rust. This topic is very interesting to me. Some things that come to mind:

  • Every line of code, in any language, is a liability. Haskellers don’t treat code as a liability; especially juniors but also seniors, they rewrite everything in it, try out weird abstractions, use packages with a single maintainer for a core product feature, etc.
  • The novelty budget is a very apt guide. When you use up your novelty budget, it’s very high chance that your project will be dropped and rewritten in $popular_lang. Haskell takes a big chunk of the novelty budget already, but Haskellers use more of it than necessary.
  • The tooling problem is real. It is terrible and has been since I started in 2009. That makes Rust look attractive. Also, people don’t want to have to learn Nix to build their code projects. But ecosystem churn makes CTOs look back at Java.
  • The compilation speed problem is real. If you count your CI build time in tens of minutes to an hour, rather than seconds, chances are you’re using Haskell and not Go. That makes Go look attractive.

In my view, the best way to use Haskell in industry and avoid it being dumped or rewritten is to be parsimonious, use existing packages from other languages that already work, minimize the upkeep required (think Casio watch vs Tamagotchi). Otherwise that codebase will get the chop.

28 Likes
#48

Those are holy words. Code should never be treated as an asset.

5 Likes