RFC: Haskell-in-Production Experience Reports

Haskell Foundation
#1

Motivation

In my opinion, much of the (justified) fear and apprehension that I see in the “I feel like Haskell is dying” thread is due to poor accessibility to Haskell production users’ experiences. The resulting mismatch of information appears to even affect core GHC contributors, and it appears that most non-Haskell-employed Haskellers feel that the language and its job market are dead or dying.

One of the biggest rebuttals we see from Haskell-is-alivers are the strongly-held claims that the Haskell job market is up to 10x larger than it was 10 years ago. This generally aligns with my experience, but I have no real data to go off: linking to some “usage” metrics is obviously not going to cut it - those sites are an awful estimate of usage or language-death and are without a doubt not a substitute for lived experiences.

So in order to ameliorate this imperfect information exchange collect subjective but standardized experience reports of real world Haskell. I want these reports to accumulate into reference points that might help non-visionaries believe in the economic utility of a language like Haskell [1].

Haskell Experience Reports

The following is all up for debate: I am stating everything as fact to make it easier to discuss, but please push back on anything and everything

Goals

A Haskell Experience Report will convey the lived experiences of using Haskell in a production setting with money and full-time employees involved [2]. The target audience will be anyone no more familiar with Haskell’s ecosystem than a newbie Haskeller: think Hacker News, not r/haskell.

I want Haskell Experience Reports to make Haskell a language that when an outsider says “yeah but who actually uses Haskell”, instead of linking to pandoc, we link to a single webpage (or the HF website) and they see myriad experience reports from a variety of domains, markets, and company sizes.

Audience

The target audience will be non-technical non-Haskellers. The reports should exclude as few readers as possible and the reports should be able to communicate the value of Haskell whilst keeping technical details to a minimum.

For example, it might be reasonable to mention using the Polysemy library and that it made code more maintainable and easier to use. It would likely not be relevant to discuss different encodings of monads or the differences between algebraic effects and monad transformer architecture.

Report Style/Format Principles

The experience reports should be rigidly formatted and non-casual. These features will aid in conveying professionalism and will communicate a data-minded approach to the experiences offered by the authors. To be clear, the content will still be fully subjective, but when comparing any two experience reports, the user should be able to easily observe the similarities in format and writing style.

Tone

Reports should use a formal tone, and attempt to extract value-oriented judgments of the usability of Haskell and its ecosystem, even in social/emotional evaluations. For example, instead of, “I really liked using lens - it was fun and made waking up easier”, it would be preferable for a report to say, “We found that lens had the benefit of improving engineer morale, who reported that working on the code base was more motivating and exciting as a direct result of using it”. The reason for choosing a formal tone is because we would like these reports to be some sort of match to insert_favourite_PL_usage_graph, and so appealing less-so emotion (where appropriate) is critical. A formal tone also communicates a business-oriented mindset that readers might not have seen from Haskell users.

Format

The format as I have imagined it should be rather rigid. There will likely be at least one catch-all Other Thoughts section, but when addressing major evaluation metrics, such as hiring or performance, there will be designated heading/subheadings. Such rigidity enables a manager considering Haskell to efficiently find/consume the information they need in a reports, which is greatly aided by a rigid format. Another added bonus of a rigid format is that it makes experience reports far faster to write and we could therefore (hopefully) get more content.

Grading section

There will be a final letter grading (e.g. D <=> A) at the end of the report for a number of metrics (no suggestions: totally up for debate), so that we can give quick glance summaries of the report’s subjective opinions. I think such a section adds readability/fun factor, and it gives a good summary for someone skimming the article. If the grades are colored and attractive, a user might say to themselves, “Ooh Haskell got an A for hiring/team-building; what’s that about?” and then click directly to that section. I’m interested to see what others think here.

Some suggested sections for report

This section is very open to debate, and so I don’t want to push the conversation in any direction, other than coming up with some ideas on section headings:

  • Problem Domain (perhaps standardized to keywords or a word description)
  • Process of choosing Haskell and other (dismissed) choices
  • Team/Employment/Hiring/Retainment
  • Engineering experience:
    • Onboarding
    • Code review
    • Maintainability
  • Haskell ecosystem:
    • Tooling (IDEs etc.)
    • Libraries: docs, quality etc. (perhaps this should be a separate section?)
    • Stability
  • How did Haskell work post-deployment:
    • Runtime performance
    • Errors/bugs/customer experience
  • Biggest successes (not mentioned above)
  • Biggest gripes/regrets/lessons (not mentioned above)
  • Future of Haskell at company

Hosting/publishing reports

I don’t want to talk about this too much because I think this should be discuss later. Some general ideas to help orient us is that I think we might publish these reports every other week on public forums, and that their content would be hosted on the Haskell foundation website, or some other larger Haskell organization.

Out of Scope

I’ll add to this section as we go:

  • For now, I want to limit to in-production reports. Future iterations might add reports for educators or open-source projects, but we won’t discuss that today.
  • How the articles are written (raw text; markdown). These are implementation details for later down the road.
  • Who will take on the work. I would happily take on a lot (all?) of the work. But perhaps someone would take it off my hands - we can decide later :slight_smile:
  • How reports are chosen for publication.

Open Questions

  • Who is eligible for a Haskell Experience Report? Should we have a minimum time limit on Haskell usage? Should there be a minimum team size?
  • Perhaps these reports should happen only at major time intervals for a team; say 6 months, 2 years, 5 years? Perhaps they can happen at any time for any team? I’m inclined to say they can happen whenever…

[1] Gabriella Gonzalez – How to market Haskell to a mainstream programmer
[2] For now, I am limiting the scope of experience reports to for-profit organizations, but future additions to the format could support open source and full-time volunteer work.

10 Likes
#2

@rebeccaskinner and others from the Haskell.org committee sourced testimonials which are displayed on the front page of the website. Perhaps the same companies would be willing to provide experience reports.

3 Likes
#3

Great plan! The Haskell Foundation would be a natural place for hosting etc.

My only question would be: would we find enough authors? People using Haskell in production are generally busy with production :slightly_smiling_face:

Simon

5 Likes
#4

A very good presentation, but a bit simplistic. There’s a middle kind of person, like myself, who’s looking for a competitive edge and wants to know how the new thing will give them that. Also, we middle people want to hear something more about how the code will be maintained. Most of the problems start with “It doesn’t matter if I’m the only person who can maintain the code, because it’s never going to be used much after I’ve written it”.

#5

“OK, buddy. What’s so great that you cannot spare me a few minutes to return this questionnaire?”

#6

I agree with this concern - which is why I want to make sure that whatever we come up with be as boilerplate/rigidly structured as possible. In an ideal world, writing out an experience report would be a question of basically filling in a questionnaire but wouldn’t read as such… Perhaps this means that the authors submit short form questionnaires; I fill in the gaps; then they okay the resulting article. This would be worst case scenario.

I would happy to lead the way at my team with my free time :slight_smile:

But one thing I want to point out is that writing out an experience report could be a big help in a hiring effort. I could imagine that for some companies the hiring-outreach of a front-page hacker news post (if these posts are popular, which similar posts appear to be) would be outweighed by 5 hours of a managers time. The cost might not be fully offset - but it could help to ameliorate the downsides.

Lastly, I think there’s another upside here, which is to try to build a culture around such reports in the Haskell community. Perhaps this could be that when you have a Haskell friend working in production, you pressure them into giving up an evening to promote Haskell. Optimistic? Perhaps, but places such as Serokell/Tweag already spend far more time writing lower-reach blogs, and Haskellers generally love to talk Haskell :slight_smile: ; so I think we could build some kind of community culture around these reports if done right…

2 Likes
#7

Something else I want to say is that in the Haskell community we are often focussed on improving the language rather than growing the userbase. In my previous company, I believe 50% of our company were sales. I would argue that the Haskell community is something like 5-10% sales, which I think is too low. If we can put time in now, we will all feel the benefits of growth and improved public perception later down the line. Again, this is a cultural shift which I would like to promote with ideas such as experience reports

2 Likes
#8

I agree, but I think collecting data on the number of Haskell job postings to Reddit over time would cut it and would probably be less work to gather those statistics than to gather the proposed experience reports. On the other hand experience reports would have many other benefits besides.

#9

Perhaps in a separate proposal we could try to centralize Haskell job postings around the HF. This might be too much grind for little payoff, however…

While I do think job postings would help Haskellers’ perception of Haskell liveliness, I don’t think they will do as much for non-Haskellers as an experience report, since people really want to hear experiences from fellow senior engineers. Not that the two are exclusive - just that I think experience reports give a more promotional component on top.

#10

This all sounds great. Since you say you’re willing to take on a lot (all?) of the work, what do you need in order to get started? After all, this kind of thing doesn’t, per se, need resources or signoff from anyone else. You could just launch yourself into gathering these reports and publish them on a simple website. But perhaps there are things we can help with. For example

  • Are you looking for opinions on whether this is worthwhile?
  • Are you looking for critique of the ideas?
  • Do you need funding to do the work?
  • Do you want this to become an “HF official” project, so that companies are more likely to respond?
  • Are you asking for people to put you in contact with industrial Haskell users?
1 Like
#11

Ohh… I want to express sorry about making the thread again. Was a huge mistake… That said, some kind of experience reports would be great to have. It would also be helpful for people potentially looking for haskell jobs.

1 Like
#12

Yes - as you say the main thing I need is “this sounds like a good idea”. It’s early, but I feel I have that :slight_smile:

The biggest thing I need help with is to make sure that the report format hit all the biggest bulletpoints you would want when hearing the experience of another using a more-experimental technology. So some of the most valuable feedback might be: “if I were a non-Haskeller manager it would really help me to hear that XXX” or “I really need to now about XXX before I jump into using Haskell”. I am not a manager - so experienced Haskellers could really help me out here if I’ve missed something, or by harping on something they feel is critical.

I don’t need funding :slight_smile: if the work grows too large I can revise this :wink:

Do you want this to become an “HF official” project, so that companies are more likely to respond?

I don’t know that it has to become one prior, but vocal community support would be important so we can get some initial momentum. On my own, it would be a much harder ramp-up than some community feedback and a first round with some community buy-in. I feel I am already getting that based on these few comments, so that is going well. On the next step, where I have a decided format, I would hope the HF would get on board.

Are you asking for people to put you in contact with industrial Haskell users?

Yes! But I would like to make sure everyone agrees on at least some of the details (format etc.) so I can reasonably create an example and give time estimates to people volunteering to write. I think it will be a very dynamic recruitment process once I start - any private messages here or to u/santiweight on Reddit would be a great place to signal interest to me :slight_smile:

1 Like
#13

An apology is not needed! You were being yourself and honest. Without it I would not have thought of this idea :slight_smile: Also “fear and defeatism” is not meant in a judgmental way - I am making this idea because I fear your feelings are quite justified!

1 Like
#14

In that case I suggest you find one single industrial Haskell user willing to contribute to a report, you interview them and write up the report, in whatever format you first think of. Then share back with us and we can give our opinions, e.g. “that bit’s really useful, please expand on it next time”, “that bit is not really relevant”, and then you can iterate.

It feels useful to discuss things until all the fine details are worked out, but in my experience nothing beats actually practically doing one small thing, learning from it, and iterating.

4 Likes
#15

Thanks for the suggestion. I honestly wasn’t expecting everyone to be so okay with the idea immediately… If anyone wants to be the first guinea pig:

  • I will come up with a concrete questionnaire format in markdown
  • We can discuss on the phone for 30 minutes about content what you want communicate
  • I’ll write up the report (or collaborate on it if you want)
  • We can discuss the result and subsequently get feedback on the first report on reddit/discourse
  • Move forward from there…

I have time this week - so I can move forward pretty quickly…

Feedback is still welcome from all about the experience report and content you want hit on - but the above plan seems like a good place to start working. Thanks for the suggestion @tomjaguarpaw

1 Like
#16

That’s great. In which case I encourage willing industrial Haskell users to make themselves known to @santiweight!

#17

@santiweight, amazingly thorough write up, good on you :slight_smile:

I have two thoughts to share:

  1. Experience reports from “in-production haskellers” are definitely useful for selling haskell to new and interested parties, and that’s good, however I think there is a treasure trove of useful info and feedback from users who failed to bring haskell to bear on their problems. This information is difficult to gather, but extremely useful feedback. Optimizing initial experience on haskell.org for the first-time Haskell user · Issue #136 · haskell-infra/www.haskell.org · GitHub is a good example of such feedback (thank you to @tomjaguarpaw for facilitating that!). One of our problems is that we argue over what should be on the downloads page, but we don’t use the feedback we have received to improve the onboarding experience. On top of that, when we have experienced haskellers stand up to help improve that experience, we burn them out and lose their contributions.
  1. While I agree with this perspective, I think “haskell sales” come naturally when there’s a good onboarding story. Increasing our efforts towards sales will be wasted energy if we don’t also do something about the onboarding story. I think the HF should be involved in addressing this problem, and the haskell.org committee should be supportive of such an effort.
#18

I find it disappointing that we’re unable to discuss the ways that we’ve lost great contributors, or how unresolved rifts in the community continue to affect us (especially onboarding new haskellers) - or that such discussions are classified as fear mongering, defeatism, etc. If we continue to ignore or shun meaningful discussion of these topics, we set ourselves up to continue repeating that history.

#19

I think there is a treasure trove of useful info and feedback from users who *failed to bring haskell to bear on their problems

Reiterating what I said to santiweight above:

this kind of thing doesn’t, per se, need resources or signoff from anyone else. You could just launch yourself into gathering these reports and publish them on a simple website.

It would be great to see a report that summarised this trove of useful info. If anyone wants the community to benefit from it then all they have to do is to produce the report!

#20

Yes, please! There is a good group of people already talking details, so I don’t feel the need to get into the weeds on this one, but I just wanted to pipe up to offer my excitement at pulling this together.

2 Likes