Hasura migrating to Rust?

Yesterday Hasura (a GraphQL server generator written in Haskell) announced their upcoming version 3 during their yearly conference HasuraCon 2023. One of the highlights in the announcement was they were in the process of migrating the server implementation over to Rust.

I wonder if there are any insiders here who would like to share the reasoning and what they think it means for Haskell, the language and community.

17 Likes

Have we made enough cases for relentless pursuit of code purity? Probably it’s time for not holding back on promoting it more?

1 Like

While it’s always interesting to hear how the sausage is made, I want to quote a Reddit comment of mine from a little while ago:

fwiw, I assign pretty much 0 predictive value to corporate decisions vis-à-vis Haskell usage. Org charts make technology decisions for plenty of reasons unrelated to technology, and it’s impossible to know the real reasons even if you are a member of said org chart! That’s a lesson I learned years ago.

There’s been a mild obsession with (commercial) production usage as a signal of “legitimacy” in Haskell-land, so I think it’s important to push back against that culturally.

12 Likes

How about the predictive value regarding how easy it is to find jobs using Haskell?

12 Likes

Your Reddit comment presupposes that engineering reasons didn’t play a significant role in the decision. Unless you know that’s the case, that’s an unfortunate assumption. And if you know that’s the case, you are in a privileged position to answer OP’s question :slight_smile:

6 Likes

It would depend on the job, to an extend:

  • “greenfield” - starting an entirely new project using Haskell;

  • “brownfield” - the upkeep of an existing Haskell project;

  • “whitefield” (my terminology: covered in snow) - the abandonment of an old Haskell project, as Hasura did.

1 Like

I’ve found through first-hand experience that it’s actually a safer assumption about these things than to assume engineering reasons are the default reason :smirk: just spreading awareness, yknow?

You see this all the time. Rumors of a Haskell shop moving off Haskell. All you hear is stuff like “Dfinity switched to Rust” or “Target doesn’t use Haskell anymore” in passing on online forums. If you assume engineering reasons are the cause, you’d be missing the possibility that a senior manager ratf****ed the entire Haskell project. Or that inter-org political jockeying begat a needless rewrite.

Usually the people privy to the BS keep it to themselves. Because it’s annoying and easier to let it go and move on. And because professionally it’s wise to not burn bridges.

Not saying it’s the case here (I couldn’t know), but it’s something to keep in mind is all I’m saying. I find it important to publicly push back against this kind of assumption of good faith engineering reasons so maybe it’ll stop being the default assumption someday.

11 Likes

It’d make sense for Hasura to migrate to Rust, considering they seemed to have been targeting junior developer hiring at the outset, and there’s a substantial fraction of the Haskell community that believes that Haskell is not powerful enough to save junior developers from themselves.

Given that Hasura is also competing with Sumo Logic and Elastic, and they’re not winning (revenue 13 m vs 300 million or environs), it makes sense to attempt a switch to Rust and see how well that works out.


That said, of Indian start-ups, Juspay.in is more interesting and has a more serious use-case, given that they’re in financial services and do not need ultra-low latency or extreme performance. The only concern with Juspay.in is that purportedly during their Purescript era, their programmers were just copy-pasting code from the internet to get around with their utter inexperience with functional programming.


A big question I do want to ask, though, is why do Haskell start-ups fail?

Start-ups in general fail, but the lack of successful pure Haskell start-ups serving outside communities (i.e, not FPComplete, not DigitallyInduced) is pretty alarming, considering that Haskell has much of what’s needed to succeed.

It’s a pretty good “worse-is-better” balance between Python (concision, production speed), Rust (performance), and Idris (type-safety).

9 Likes

Off topic: but if this is true, I wonder if they will keep the Hasura name. I mean, it is our trademark :laughing:

Maybe they’ll rename to Rusura.

9 Likes

Anecdotally, the Haskell codebase (~100kloc) at my previous company is slowly dying. IMO the reasons for this are more complex than being purely technical or managerial. My biggest takeaway was that engineering decisions are not discrete events, and being the “right” or “wrong” tradeoff beyond a moment in time depends on the follow-through. Of course, I believe Haskell should have worked out there, but not without the proper commitment.

:point_up:

6 Likes

What is it about Haskell that needs the company to commit? I assume that if the company built a 100 kLOC code base, then their problem is not a lack of engineers who know Haskell. Or is the hurdle only the cost of hiring and training engineers who could replace those who leave?

What is the company replacing Haskell with?

2 Likes

If I recall correctly, their core engine and several microservices were built entirely in Haskell, but they seemed to be facing a lot of memory challenges and even had to rope in Well Typed to help them investigate. Not surprised by this move at all.

8 Likes

If that is true, well it is a shame. Understandable. Rust starts looking like a holy grail when you have memory issues.

I think Haskell projects get a pretty short lease in commercial “systems” projects. Which is a shame because it’s an exceedingly good systems language.

What’s interesting is I think the imperative language most comparable to Haskell out there today for systems work is Golang. And GHC in many ways has a superior RTS to Golang (especially when you consider the potential engineering-hour difference due to Golang being more commercially successful). And personally, I’ve shot myself in the foot due to Golang’s RTS and semantics more than I have GHC’s, despite having more experience with GHC in prod.

Is it just that GHC has rougher corners - or even steep valleys you can fall into? Maybe there are simple and elegant ways out.

I remember the Pusher article from 2016 (time flies). There was a lot of discussion about how Haskell is fundamentally flawed for certain workloads. Years go by and we now have two great new pieces of systems programming available to us: Compact regions and the non-moving GC. Not to say those were easy to implement (they were huge efforts), but they resulted in a stepwise increase in GHC’s viability in different situations. The Pusher of today would be able to make small changes to their code and config and potentially never had issues.

9 Likes

There were other factors too, but the Haskell team being understaffed was indeed one of the major problems. Over time engineers left and business struggles led to layoffs. Positions that needed to be backfilled weren’t for long periods of time, and when they were there was a refusal to hire below senior levels. I think it’s a testament to Haskell that a much smaller remaining team could continue maintaining everything and developing new features. But eventually there wasn’t enough bandwidth to improve core infrastructure and those strengths turned into bottlenecks.

Python

6 Likes

Yeah, it’s hard to compete with that initial simplicity of having effects everywhere, rather than e.g being seemingly-trapped behind an abstract interface which still defies a simple explanation. It will be interesting to see if they stay with Python in the long-term though, as the task of programming gets more difficult, not less.

1 Like

I would like to counter the gloom of this thread with a happy tale of conversion.

My work codebase was 100% Python until a few months ago, for the usual reasons:

  1. There is a large pool of Python developers;
  2. Python is easy to get started, so it feels productive at first;
  3. There is a large ecosystem of third-party libraries ( e.g. how many programming languages have a large weather science toolkit available ?

But of course, slowly, cracks in the foundation appeared. Python is slow for most things. People started dreading refactorings. Surprise “one-off” bugs in production due to a sentinel None happen at least once a week. All of this has contributed in limiting our technical velocity as the business scales.

After some experimentation, we’ve started using Haskell for a new key area where we want to grow, and our results are simply stellar. This area plays into Haskell’s strengths:

  1. Refactoring is a breeze, which has made us bolder. This is a huge plus for a startup;
  2. Parallel performance is unmatched, especially given that I put very little effort in it. We’ve been able to tackle problems at a scale we would never have touched before;
  3. The single-threaded performance bump has allowed us to re-wire our brain and express ourselves better, instead of trying to fit everything in a numpy and pandas context;
  4. An architecture based on monad transformers has made it possible to run the same code in separate contexts (e.g. experimentation vs production) in ways which has been a pain in our previous Python efforts;
  5. No sentinel None has crashed our system yet, which I’m very happy about :wink:

Sure we bumped against a nasty memory leak once, but to be fair I was being silly (import Data.Map as Strict instead of import Data.Map.Strict as Strict).

Overall I would say that there are many good reasons for a start-up to use Haskell, as long as the tradeoffs match the use-case.

34 Likes

Re: monads:

I would say it’s more the problem of the Haskell community:
https://tomasp.net/academic/papers/monads/monads-programming.pdf softly accuses Haskellers of gatekeeping with monads by calling it a “rite of passage”, and a different, similarly negative interpretation, is that our inability to explain monads quickly and clearly implies that we don’t really understand what monads are (“to really get a concept, you have to be able to teach it”).

On the other hand, there are multiple production Haskellers who claim they can explain monads to a reasonably competent programmer in 15 minutes, usually via the typeclass definition, and I gave it a whirl to explain monads are to a Clojurian (have to think in types, methods defined by types and laws, interfaces, useful to provide a standard interface to a diverse quantity of types, which in other languages would require language primitives, see Rust, provides the basis for an effect system, provides the basis for a standard eDSL interface, using Blaze and parser combinators as examples, what “a monoid in the category of endofunctors” means, and in their context, tradeoffs of monadic eDSL vs macro eDSL).


TBH I do have to wonder whether Hasura is actually ditching Haskell. I’ve done a bunch of searches, but I can find no evidence that their version 3 will be in Rust. It makes sense for them to bring it into Rust; they are providing server infrastructure and most Haskell-capable developers are potentially Rust-capable developers, and being ultra-high performance is good for their competitiveness.

Where can I find actual confirmation that Hasura is ditching Haskell?

4 Likes

Since the post mentions HasuraCon 2023, probably in one of these talks?