Hasura migrating to Rust?

and being ultra-high performance is good for their competitiveness

I think for ultra-high performance, C++ is still the gold standard, no matter the safety features rust has. And there is a large market for C++ engineers, especially since the layoffs.

So, kinda odd.

Maybe it has more to do with the fact that both languages have an academic vibe, that you might convince your Haskell devs to switch (no re-hiring needed) and that it solves some scaling issues they couldn’t figure out how to solve easily with Haskell.

1 Like

Surely both languages would have comparable performance?

4 Likes

A big question I do want to ask, though, is why do Haskell start-ups fail?
Start-ups in general fail, but the lack of successful pure Haskell start-ups serving outside communities >(i.e, not FPComplete, not DigitallyInduced) is pretty alarming, considering that Haskell has much of >what’s needed to succeed.

Contrary to public opinion, there are loads of successful Haskell startups that don’t make too much noise about it. My favorite among them, Co-star doesn’t even look like your typical Haskell shop. They have a small team ( 2 - 3 Haskell devs, though you could argue Pavan Rikhi is worth an extra 3) and over ~7m active users.

6 Likes

The key quote is in:
Introducing Hasura version 3, Rahul Agarwal, at 6:30-7:00

New Engine Architecture
New Rust Based Codebase

  • Refactored to use Rust
  • Leverage best in class server technology
  • Better baseline performance than V2
  • Helps engage with the open source community, given that Rust has such a large and growing ecosystem

So Hasura rewrote the core system in Rust, which is a strategic decision that requires a big investment. Hasura is betting that Rust has a brighter future than Haskell, and they put real money on their bet.

12 Likes

I just think that for the application chosen, Rust is a better option, since they need performance.

I suspect that their mistake was trying to go with the junior dev Haskell route, but that’s actually a good experiment for the rest of us. What they showed was that they didn’t have the experienced engineers on staff to guarantee performance in time and space, and by the time they realized the problem, their codebase was likely too big and too filled with junior code that a rewrite, in this case, in Rust, was the correct solution.


On the other hand, let’s say they were building their application in Rust in the first place. Would it have been able to take off? Possibly, but they’d have been slower to get to the starting line since Haskell is more expressive and the FP paradigm is good for prototyping.

I hope Hasura will at least let us have the courtesy of suggesting to us that Haskell was the best choice at their initial stage of development, but between moving to linear types and going with Rust, Rust was the more mature option and more performant to boot.

4 Likes

I think there are a least a few good haskellers there. I am pretty sure Alexis King worked in Hasura for sometime, for example… Also, the code in github doesn’t look “noob” at all. So I don’t think this is the reason.

My guess is that there is a combo of “Rust fits our performance needs better” + “devs are eagger to work with rust”.

They have contribute a lot to the Haskell ecosystem (with ghc-debug). So that’s something we should be gratefull. It would be very good to know if some parts of the Haskell code base remains, and a little bit of ups and downs of using Haskell.

5 Likes
  • Total funding $136.5M per Crunchbase
  • Invested in the Haskell OSS ecosystem
  • Hired and trained many people, thus creating more Haskellers in the world (great use of some of that dosh)
  • Built a pretty popular bit of infrastructure (“10B+ API calls served / day”)

Sounds like a Haskell success story to me!


Again I want to reiterate - we as a community really shouldn’t over-fixate on corporate decisions as a form of validation for Haskell. The language is well beyond the point of “prove it” imo - I can confidently say I could build most things in Haskell, for instance.

Programming language choice really isn’t a key factor for business success. And PL choice is not purely technical. There are many less-than-technical reasons to pick a PL. Morale, attrition, fun, culture, ecosystem, office politics, hiring concerns, etc. And even if every Haskell and ex-Haskell company told you their 5 Whys behind their Haskell decisions, you can’t really take it at face value. Companies aren’t oracles who make optimized decisions. They’re messy webs of people usually primarily motivated in part by financial concerns.

10 Likes

I also underrated their revenues; I took the casual search of a Haskell basher at face value, Hasura is around 300 million in revenues via Zoominfo: https://www.zoominfo.com/c/hasura-inc/404445228


@Lsmor: What I heard was that they started out by hiring Indian IIT grads (which, note multiple, tend to be more selective than MIT, but not well-ranked globally) with no Haskell experience, then transitioned off toward hiring senior devs.

The code we see on Github is not necessarily all of their code, and it’s possible they have huge balls of mud hidden in their servers that a combination of Simon Marlow, Alexis King, and SPJ wouldn’t be able to fix.

But I agree with you on the positives of Rust. One of the big deals of Rust, is that while there are over 2 million Rustacaeans, there’s also historically a dearth of Rust jobs. I think in 2021, at least, the Haskell job market actually looked better than the Rust job market. Combine that with Rust ranking #1 in developer happiness, when Haskell is only middling now, and you have a huge arbitrage opportunity.

@Elorm
Well, Co-star, as an astrology firm, is likely to throw the ETO > IOHK crowd into even greater paroxysms.

There’s also Obsidian Systems (around 15m revenues), which is interesting to me because from what I know of them, their main selling point is pure prototyping speed; i.e, between their FRP libraries (Obelisk, Reflex) and Haskell’s safety, they can get applications built very quickly, and with Haskell’s safety and correctness guarantees to boot.

4 Likes

It’s not just about the language. It never is.

It’s about ecosystem, community knowledge, resources, reliability of performance, etc.

C++ has been used for these things for decades. Rust is nowhere near that. Yes, you can probably do similar things, but you’ll likely end up ffi’ing into C/C++ libs or writing your own wrapper libs (you can do that with Haskell anyway).

Some time ago I also remember talks that the generated llvm IR is inefficient and has a lot of unnecessary allocations. But that might have been fixed.

1 Like

…perhaps because it was the least-worst option available - if it arrives as scheduled, the new GCC Rust front-section could change that in a big way.

Having said that…this isn’t the first attempt at a second Rust implementation. For Hasura to make their announcement now, before Rust is fully supported in GCC, is a rather speculative decision - if the current attempt at a GCC implementation also failed, they will be moving from one single-implementation language to another.


instead of remarks like e.g:

New All-Rust Codebase

  • Full Rewrite in Rust

…tend to suggest they haven’t switched entirely over to Haskell. Did Hasura give an estimated completion-date for this migration?

Thanks for saving us from sheer speculations.

Still, some of these are important to us Haskellers, e.g. ecosystem and hiring concerns. How can I start a Haskell project at my company if I can’t guarantee them a sufficiently large ecosystem and sufficient applicants? And if these aren’t given, then there is something that we as a community can improve on.

3 Likes

Ehm. I think industry has to improve on that.

Start investing, let your employees work on open source, support the HF.

Stop taking things for granted.

6 Likes

You’re absolutely right, morally. Industry should make steps here. I just meant that there might be potential to act.

2 Likes

Honestly somehow this thread sounds like the thread about haskell dying…

2 Likes

Almost every programming language intended for practical use that predates Rust will be in this predicament sooner or later…

1 Like

One line in a blurb somewhere and a thread full of sheer speculation? :smiley: Sounds a bit like a rorschach test…

14 Likes

So here’s something less speculative, and also based on experience:

3 Likes

I’m interested in Python to Haskell conversions (I view Haskell as a Python in terms of expressivity relative to Rust, with much less performance penalties and actually greater correctness than Rust); the other conversion of note would be Key.me moving from Python to Haskell, but it was roughly a monkey move wherein they slapped XStrict on everything.

What have been the challenges with moving from Python to Haskell? Python is notoriously anti-functional, so has it been hard training Pythonistas in an FP idiom? You’ve mentioned where things have gone right, but where have things gone wrong?


Within your start-up experience, what’s Haskell’s key selling point for a sector wherein 67-85% of firms fail within 5 years?

3 Likes

Before I answer, I need to emphasize that we didn’t convert any Python code to Haskell; broadly rewriting would not be pragmatic for us because:

  1. Our current Python codebase works fine in many cases;
  2. We aren’t ready to spend time/money training devs to learn Haskell;

What we’ve done is expanded the business into a new area where there is <10% overlap with our current business. I work for a proprietary trading firm, so concretely this means that we’re preparing to trade entirely new products where the code path will be >90% Haskell. This is done by a 2-person team part-time for now (this work often gets de-prioritized in favor of more pressing matters).

Such green-field projects are a much easier avenue to start using a different language because new code needs to be written anyways.

As an aside, I am under no illusion that the entirety of our trading operations will switch to Haskell in my lifetime; the cost-benefit analysis just doesn’t make sense:

  • All products we’re trading right now have lax latency requirements (many of our competitors actually still trade manually!), and so there’s no tangible performance benefit to switching from Python to Haskell;
  • Our current trading operations are acceptably stable, such that the opportunity cost of software failure is probably less than the cost of training devs to learn Haskell + a wholesale rewrite.

Now to your questions:

Python is notoriously anti-functional, so has it been hard training Pythonistas in an FP idiom?

We haven’t started training new devs yet. However, after discussing this possibility at informal events with my colleagues, I encountered a strong resistance in many of them to change from the typical object-oriented Python. I get it; I can’t expect everyone to be comfortable investing work-time into an unfamiliar programming paradigm, when programming is just a job for most people. I would estimate that only about 1/5 of devs would be interested to be trained.

Pragmatically, I will look for a different type of candidate in future hiring situations – prioritizing people with ( knowledge of Python ) AND ( (interest in learning Haskell) OR (at least basic knowledge of Haskell) ).

You’ve mentioned where things have gone right, but where have things gone wrong?

Apart from the silly memory leak I mentioned, nothing went wrong per se. One of the additional challenges with respect to Python was that we needed time-series data structures that weren’t readily available off of Hackage, so we had to build our own library (to be released officially in the next few months, probably after a community RFC). In general, one of the drawbacks of literally any programming language vs. Python is the breadth of the ecosystem. On the quantitative research side of the business (i.e. stats and machine learning), you can be sure that we’ll be using Python for years to come, simply due to the ecosystem.

Within your start-up experience, what’s Haskell’s key selling point for a sector wherein 67-85% of firms fail within 5 years?

If I had to summarize to a single key selling point, it would be “ease of refactoring”, or in business-speak, “high iteration velocity”. Startups trying to build a minimum viable product, or achieving product-market fit, or expanding to new lines of business (like us) are working with fuzzy requirements and many unknowns. In this context, the ability to adapt to changes in requirements quickly is worth key.

I’ll be sure to write a postpartum blog post once the system is live (probably by the start of 2024).

8 Likes