Commercial Haskell should go after Python / Julia, not Rust

#1

I’ve been reading up a lot on Haskell production, as well as translating some Python toy scripts into idiomatic Haskell, and it’s provided some ideas as to how Haskell can find more commercial adoption.

Here’s the big problem with Haskell in an age of Rust. Rust, while more verbose and significantly more imperative than Haskell, is more performant and provides more memory / space safety. While Haskell can be tweaked and forced to get within 70-80% of C, at least with toy examples, Rust can, first, do so more easily, second, when it comes to high-performance use cases, it is easier to hire or train a Rustacaean than it is to hire or train a Haskeller.

Haskell has tried to compete via its type system, but type-level Haskell is, first, arcane, and second, first-class dependent types look as though they’ll be very late, with rae having admitted that GHC needs substantial refactoring before dependent types can be added.

We are seeing Haskell losing in competition with Rust as a consequence; Meta, while still supporting Haskell Foundation, is more supportive to Rust (it is an approved language for new projects), Tsuru Capital moved from Haskell to Rust, and we are all familiar with Hasura’s migration from Haskell to Rust.

On the other hand, going after Python’s use cases is closer to seal-clubbing. Going after Julia’s use cases is not so one-sided, with Julia winning in terms of pure performance, but Haskell winning in maturity and reliability of ecosystem (Julia is going to be notorious for buggy libraries for quite some time), however, Julia in expressivity is a downgrade from Python, whereas Haskell, excepting the ecosystem problem, is almost a pure upgrade.

In terms of performance, Haskell knocks the socks off Python; even the slightly worn Haskell server libraries promise at least a 3x performance improvement over comparable Haskell.

When it comes to code correctness, the Pythonistas themselves are moving to gradual typing (with type-hinting, or static typing, being idiomatic for Python code), but their type system, as a bolt-on, is not going to be as expressive or powerful as Haskell’s.

In terms of memory safety, Haskell’s propensity to space leak at scale is probably equal to Python’s propensity to leak at scale, but we’ve known of this problem for a long time and have been developing workarounds, and the nuclear -XStrict option is still available.

Any time you go into more complicated data processing code, Haskell’s FP-optimized syntax is much more succinct and understandable than any imperative Python implementation, and even “functional” Python code loses out easily because Python is not designed to be a functional language and has no spare performance for paying the lambda tax.

The problem with this, of course, is that Haskell’s library ecosystem is still quite problematic; there is a survivorship bias where every time Haskell’s ecosystem is found wanting, and the persons involved lack the skill to write their own library, they cease to be Haskellers, so Haskell ends up being about library writers and people working within the cases where the Haskell ecosystem is satisfactory.

A further problem involved is marketing. Haskell, of course, is a “pure” (in the technical, not vernacular sense, by being referentially transparent with the overlap of call-by-value, call-by-name, and call-by-need semantics) functional programming language. It is foreboding to the average Pythonista (anecdotally, a few friends checked out the Python and Javascript Discords, and the average member of either community did not know what a pure function was), but when you trial monadic Haskell code to imperative programmers with zero Haskell experience, a common response is “I can’t believe it’s not Python”.

Historically, we’ve had the fake sieve and fake quicksort scandals. For pushing production use, it’s not that morally different from hiding Haskell’s fundamentally functional nature, limiting newbies to an eDSL (Gabriella Gonzalez suggests that Haskell’s killer app is as an eDSL builder in Gabriella Gonzalez – How to market Haskell to a mainstream programmer - YouTube ), then gradually explaining how to do computation functionally, building and extending the eDSL themselves.

This basically just amounts to how to sneak Haskell into the workplace; i.e, it is hard to sell based on effect systems and type-level, considering that the average IT manager likely has no idea what they are and will be hostile to those features for that reason, but it is easy to sell Haskell as “Python that’s fast, concurrent, and scales way better to medium-sized projects” when the Haskell ecosystem is suitable.

The only caveat is, of course, you need experienced Haskellers to protect the junior and newbie Haskellers from space leaks, IO monad abuse wrecking the code architecture, lazy IO giving you single-threaded concurrency bugs, but that’s sort of a win, isn’t it? It means that the population of experienced Haskellers becomes indispensable to make Haskell projects actually work.

And if this sounds vaguely Faustian, well, it’s about a way to get a starting point. The goal is that, if the Haskell projects actually take off within your organization, the production idiom aims at eventually upgrading to Simple Haskell with TyDD. Once that’s achieved, depending on the scale and the skill level of the developer base, type-level could be deployed in future projects, but that assumes that the developers involved will get to that skill level, and realistically, by the time you get to that point, ergonomic dependent types will probably be available and you’d rather go with dependent types rather than older Haskell type-level programming.

Ultimately, it just comes out to, have you ever heard of a Rust project being rewritten in Haskell? Even C++ will end up getting wrapped in Haskell, as with Meta’s Sigma and Standard Chartered’s Mu project (strictly a dialect of Haskell). Meanwhile, while the sample size is small, I’ve never heard of a Haskell project being rewritten in Python, simply because while Rust vs Haskell is a difficult challenge, Haskell vs Python, as long as the ecosystem is there, is seal-clubbing.

Key.me’s successful Python to Haskell transition:

8 Likes
#2

Marketing based on comparing yourself with other brands is a poor strategy.

I’m not usually one to link to CEO speeches, but this one struck me: Best marketing strategy ever! Steve Jobs Think different / Crazy ones speech (with real subtitles) - YouTube

Never did apple tell you in one of their ads/campaigns why they’re better than Microsoft. Never did Nike talk how they’re better than Adidas. Instead, they sell an idea, a vision.

As for programming languages, I think there are two ideas that are worth to market and Haskell can do those:

  • for the engineer: the conflict between expressivity and confidence
  • for the manager: the conflict of aggressive iteration and keeping technical debt low

Engineers love high expressivity, but they will soon regret it if they lose confidence in the things they built. This happens with languages like C++ quickly, in my opinion. Soon you’ll need very disciplined engineers, so your project doesn’t derail and becomes unmaintainable.

Managers love the idea of iteration (agile is all over the industry) and yet they’re scared of technical debt for long-running projects as it’s slowly regressing productivity, burning out employees and requiring a rewrite down the line.

Haskell has a unique stand in both conflicts, IME.

22 Likes
#3

Thanks for deigning to respond, hasufell.

First, I think comparison is useful as a way to identify potential use cases. Rust will always beat Haskell in certain fields (systems programming, for instance) because of its core values, but in fields where it should be more of a toss-up, Haskell’s advantages (expressivity, ease of refactoring, correctness) can’t cancel out the deeper Rustacaean developer base and depth of ecossytem.

However, for Pythonic or Julian use cases, the relative strength of Haskell is such that, barring issues with the ecosystem, Haskell’s greater performance, correctness, and expressivity cancels out the relative dearth of Haskell developers.

Second, the point of comparing Haskell to Python is more that, Haskell currently has a reputation for being extremely difficult to learn. If, say, you want to understand all the language extensions, all the possible design patterns and architectures in Haskell, this is 100% true.

However, if we’re talking about the minimum dialect of Haskell that’s useful and idiomatic (i.e, does not have IO eating up the entire program, does not abuse mutable references, has proper division between effect-free and effectful code), it is not that much more challenging than Python.

Python is notorious for just having libraries and library functions for everything. Ironically, this maps pretty well to the Haskell idea of pure-impure separation; the functional pattern of having effectful code that calls effect-free code is pretty much just Python “just use a library function” except that you are writing the libraries yourself.

If the ecosystem supports the use case, and there are competent Haskellers on the team, for a junior developer, the Python to Haskell transition does not need to be painful.

And in reality, there’s like multiple blog posts already talking about individuals transitioning from Python to Haskell:

How to read Haskell like Python : ezyang’s blog ← Thanks for backpack!

Haskell for Python Programmers. Python is a functional programming… | by Noah Hradek | Medium <— the assertion in the first few paragraphs are technically wrong, but the point is that Python has declarativeness as a goal, and Haskell’s pure FP is a subset of declarative programming, so the confusion should at least be understandable.

#4

@Liamzy Your mention of Python (ultra-popular) and Julia (pretty niche) makes me think of the following: the target user of both languages is not a formally-trained software engineers.

The following observation is limited to my experience, i.e. physical sciences and finance, so take this with a grain of salt.

The reason I didn’t keep using Julia during my graduate studies (like many of my fellow students) was that there was too much friction to get started and keep going. This includes a lack of appropriate learning material aimed at non-software people and language-specific issues (in the case of Julia, time-to-first-plot).

As the demand for software knowledge has increased – and the supply of formally-trained software engineers is somewhat constrained – many software developers aren’t knowledgeable about the existence of many technologies, let alone their pros and cons. As an example, at my work, <15% of developers are software engineers, who presumably have had a class on type systems. The others are smart cookies who are no less effective, but have little-to-no knowledge of type systems / mathematical purity / algorithmic complexity.

Therefore, adoption of a technology goes by the path of least resistance. In this optic, Julia hasn’t reached great penetration because of a thousand paper cuts.

In order to increase industrial Haskell adoption, we need to remove friction from the perspective of non-formally-trained software developers. The breadth of the ecosystem is a pain point, but there are easier wins:

  • Having good tooling (which has improved so much recently!);
  • Appropriate learning material (I’m a big fan of ‘Haskell Programming from First Principles’ and ‘Production Haskell’, but buying a book requires some level of motivation);
  • A focus on documentation, including API documentation but also tutorials and examples;
8 Likes
#5

I agree. My experience also is about mindset. Some people are interested in programming languages for the sake of it. Some just want to get things done, programming language is just a tool and they learn the minimum they need to.
Of course you can use Haskell without understanding Monads, but would you hire someone to work on a haskell code base which doesn’t want to learn and understand Monads ?
Most Haskell enthusiasts learned Haskell are interested in languages in general (I’ve been progamming for 35 years and tried pretty much every language possible, just because I like learning new languages and see how they work). Some even started because they’ve heard about a Monad and fell into the rabbit hole. But trying to sell Haskell to people which are not interested won’t just work.

3 Likes
#6

I’m sorry for the selective quoting, but I don’t think I’m taking you out of context here.

I see a discrepancy between these two statements:

From my perspective, monads are an extremely useful way to structure a wide variety of APIs. Abstracting over arbitrary monads is good not only because it’s fun and enjoyable, but also because it enables a great deal of reliable code re-use. Programming languages that don’t make it convenient to abstract over arbitrary monads are missing a proven, useful tool. The slow spread of the concept to a variety of other programming languages is evidence that it’s not just for programming language enthusiasts. I think your post implies otherwise, though, and that really doesn’t match my experience.

4 Likes
#7

Re: LaurentRDC: I’m worried about how well the Haskell community might take an influx of developers with less formal software developer training, but I think both yours and maxigit’s post has brought up interesting perspectives.

First, Haskell is unusually valuable to outsider developers, because learning Haskell, to an extent, is trying to pick up parts of a formal computer science education, given that many pre-existing Haskellers are very experienced, knowledgeable, and skilled software developers and engaging with Haskell is engaging the culture.

Second, I think of this blog post:

Outsider developers are often competitive in software engineering because they are domain experts and understand a necessary domain better than formally-trained developers, who know how to develop software but not necessarily the domain the software is being developed for.

Incidentally, these domains are often mathematical, and Haskell’s mathematical culture matches better to these non-formally trained developers than things like SOLID or Agile. A notorious meme concerning Haskell is the OOP-trained developer, who is often very experienced, rejecting Haskell because their domain knowledge concerning imperative and OOP programming doesn’t translate well to a functional programming paradigm. This issue doesn’t apply to domain experts, who don’t know SOLID or agile in the first place. And for a non-formally trained developer, Haskell helps them address imposter syndrome, fill lacuna in their software development knowledge, and gain respect within the software development community.

Likewise, for formally-trained software developers working in shops with lots of domain experts, Haskell’s tendency toward “pit of success” design provides benefits in that the domain experts, without software engineering training, are more likely to use proper software engineering practices.

That said, yeah, documentation is written by and for experienced Haskellers. Consider Network.HTTP.Simple for instance. You and I can read the documentation and figure out how to use the “simple” library, but it’d take a substantial amount of time, or at least dedicated instruction, for the average Haskell newbie to figure out how to get access to the body of an HTTP response.

“Why can’t I just strip the IO type off? The example isn’t in do notation! What the hell is a ByteString?” What does the Response a → a type of getResponse body mean? What’s the >>= operator?"

Another example might be Wreq, whose documentation uses lenses.

https://hackage.haskell.org/package/wreq-0.5.4.0/docs/Network-Wreq.html

1 Like
#8

“Late” is good. The alternative is “never”. The entire initiative hangs by a thread, as there aren’t many people working on adding dependent types to GHC.

I’m rather lucky to be able to work on Dependent Haskell at Serokell, but if I get hit by a bus or if the funding runs out, I can only hope that someone else will pick up where I left off.

14 Likes
#9

I made a type

Of course you can use Haskell without understanding Monads, but would hire someone to work on a haskell code base which doesn’t want to learn and understand Monads

Should have been “would you hire somene … ?”

Maybe it makes a difference.

Anyway, I don’t think I disagree. What I am just trying to say is that people who find monads “fun and enjoyable” (I’m one of them) are usually not the one who learn a program language reluctantly.

1 Like
#10

I’m more oriented on breaking down resistance to Haskell; i.e, there can be dialects of Haskell that resemble what people are used to, without alienating industrial users by telling them they have to complete a master’s degree in functional programming (which, of course, doesn’t exist) before they can start using Haskell.

The idea is more slow infiltration; i.e, start with main = do, wire together library functions, then learn how an FP language can achieve the same things an imperative language can, as opposed to throwing HaskellBook at someone and hoping it sticks.

Given the average knowledge of industrial users (Haskell has been attacking at the high-end, but Rust is starting to block that off), showing Haskell that the average developer or IT manager can understand is useful.

Re: @hasufell: I agree with your points concerning how Haskell can be branded. I just disagree strongly with the notion of introducing and pushing Haskell through its most cutting edge and experimental features; i.e, the average developer or IT manager would feel professionally threatened by what is really a completely different paradigm, rendering much of their existing knowledge useless, and infiltrating Haskell by its similarity to what they already know would be more successful.

1 Like
#11

Are you sure pythonistas are willing to move to Haskell (or any other static typed lang)?. I’ve been a python developer (exclusively in the field of data science) for 8 years, and whenever I proposed to my colleghes to use a different language (commonly scala's spark) or to use python’s type hints I’ve got the response: “I prefer to quit than using types”

Moreover, not just one, but two managers have told me “stop proposing typed language” (they did kindly, written down seems more dramatic that the actual conversation). The simple reason: most developers I work with do only know python. They are completely unaware that languages with types exists, and when they see type hints in a python code, the tend to frustrate because they add verbosity to the language.

Notice that me background is exclusively data science/data engineering, and my sample of coworkers are phicisits, mathematicians, electric engineers, etc… probably just 5% of my coworkers have had a degree in CS. So probably I am biased, but I’d say pretty much 0 pythonistas would like work with Haskell

6 Likes
#12

Yes, some people don’t like type. I remember when I was writting C++ (20 years ago).
People hated compilation errors but like runtime error. With compilation errors, they felt they hit a wall . They frustrated with the compiler refusing to understanding them. Rutime errors was a different story, they could debug and see what was going wrong. Telling them that the compiler was their friend trying to help them didn’t work.

2 Likes
#13

I’ve noticed the same thing. Large amount of resistance to consider anything else but the status quo.

However, I refuse to believe that there are people out there who are so set in their ways that they wouldn’t consider an alternative. Rather, there needs to be a very high upside for some people.

I can’t help with data scientists, quants and data engineers, but here’s something for managers.

I work in proprietary trading, and I’ve been tasked with assessing the cost of production bugs. Every bug in production costs two things:

  • A loss in case we’ve traded the wrong thing;
  • An opportunity cost; the loss of the money we would have made, had the bug not happened.

We have a failure rate that’s pretty low. I can’t give the numbers exactly, but think in units of bugs / month. And yet, in the past year, just in terms of Python’s TypeError and related errors, i.e. things that static typing would have caught at compile-time, we could have saved something on the order of person-years. Assuming that new bugs pop up as new ones get squashed – which is likely given that we keep expanding to new products and markets --, there is a tangible financial incentive to consider static typing. In fact, we have been experimenting with Haskell for this purpose, and now we have firm numbers backing our decision.

5 Likes
Hasura migrating to Rust?
#14

TBH, while this isn’t proper TyDD, a funny thing you can do with HLS is just write a function or value, and watch the shadowed type signature change. When you are doing something wrong, the types make no sense. When you’ve completed your definition, just click on the code shadow and your type signature shows up.

One of the big advantages of Haskell’s approach to type signatures is that we don’t actually need the type signature in code most of the time. The type inference engine is powerful enough to guess the types most of the time, and an type-sigless style can work, albeit be smelly.

Do your Pythonista coworkers accept docstrings? Knowing what I know of the Python community, I wouldn’t be surprised if there were Pythonistas who hate docstrings, but introducing a type signature, something that strictly exists outside the code and comments existing code, might work.

I hope this isn’t too shocking, and I hope no one actually ships code into production without type signatures, but as a way of getting crap under the radar, the ability to avoid type signatures 95-99% under the time IS a way to sell strict static typing to dynamic typing addicts. Gradually, they’ll discover that it’s better to have a type signature most of the time, just to get the compiler to give better errors on what went wrong, just as some Haskellers go so far as to type sig all their where-clause code.

Also, for Python code golfers, one of the interesting parts of Haskell to me is that it is actually quite easy to get idiomatic Haskell code to be so terse as to be unreadable. We literally introduce garbage and unnecessary names just so that someone reading the code knows what the hell is going on. That’s a sellable feature.

#15

I wouldn’t be surprised if there were Pythonistas who hate docstrings, but introducing a type signature, something that strictly exists outside the code and comments existing code, might work.

If we’re being pedantic (pydantic?) “type signatures” (I think the normal nomenclature in Python is “type hints”) are available at run time in Python, and are able to be used to determine behaviour. I have worked with Python programmers who never run a type checker like mypy over their code, but write types, just so they can use libraries like pydantic or fastapi.

1 Like
#16

No type inference, way less useful typing errors, way more cryptic messages, I think during these 20 years there has been great progress in static typing and compilers (and not only in Haskell/FP).

1 Like
#17

Haskell has type sigs. We use them all the time because they’re idiomatic, but your code will often run if you leave them off. This idiom mostly happens with newbies and people who are learning Haskell, because taking the type signature off makes Haskell a bit simpler to read and write.

#18

I think the main problem comparing with Python is that Python has many target users. For example, Python is a great scripting language, and a great language for prototyping. The thing I love about Haskell is it forces you to think about the types you actually want and handle all the cases, but that’s not always what you want. If I want a quick script that’s mostly portable, fairly small/limited in scope, and I’m fine with making assumptions, Python is perfect.

If we’re talking about production systems, etc etc, I’m 100% on board with evangelizing Haskell more. But we should recognize that there are valid use cases for keeping things in Python, even as a die-hard Haskell enthusiast as myself

3 Likes
#19

I just wanted to both expose a class of users you may not have been aware of, as well as to clarify that type hints in Python are very much things that are used at run time (which seemed at odds with the statement “strictly exists outside the code”), in contrast to languages like Typescript or Haskell (after typeclass dictionaries have been inserted) where they are erased.

Taking the type signatures off often makes Haskell a lot harder to read IMO, but that’s a discussion for another time. However, even ignoring ambiguity and all the language extensions that interfere with inference, there are cases where they are genuinely mandatory, for example polymorphic recursion.

data Splitting a = Val a | Split (Splitting (a,a)) deriving Show
collect :: Splitting a -> [a] -- See what happens when you remove this
collect (Val x) = [x]
collect (Split x) = concatMap (\(a,b) -> [a,b]) (collect x)
2 Likes
#20

Why do you … “want” more commercial adoption ?

  • As maxigit said you might be interested in the langage per se/for language research
  • Would you accept waiting for more adoption instead of actively seeking it ?

IIRC : In this talk Uncle Bob says “Memory is cheap, dirt cheap - everyone has in its pocket a thumb drive or a phone with an sdcard orders of magnitude larger than our wildest dream in 2000 … the real next fight of moore’s law is multicore programming … and Haskell (and FP) shines at it”