Over the years, many have groused about the quality of GHC’s error messages. While quality of error messages is, actually, a major driver of the design and implementation of GHC, many users are still left unsatisfied. @ketzacoatl and I have been scheming about doing something about this, leading to this proposal.
The main idea: create a new repo on GitHub, https://github.com/haskell/errors, which will be used only to track suggestions around improvement of error messages. Details:
- The content of the repo would initially only be a README, laying out the goals of the repo and referring to the Guidelines for Respectful Communication for all participants. Perhaps someday, the repo would also contain agreed-upon high-level guidelines for how to construct good errors, but that’s a stretch goal.
- The main event will be the associated issue tracker. This issue tracker will invite the larger Haskell community – explicitly including people on their first day of Haskell – to post about confusing error messages. There will be some amount of help offered to posters in resolving their errors, but the primary goal here will be to decide on concrete improvements to the error text. Once debate subsides and a clear, concrete direction forward (with specific new suggested wording for a message or messages) has been found, then someone will post a bug on the issue tracker for the relevant project.
- @ketzacoatl and I envision this issue tracker to encompass a wide range of Haskell tools (hence the rather central location), though it seems starting small and growing with experience is good. The scope will thus be loosely restricted to GHC errors and not, say, stack or cabal errors, at first. By “loosely restricted”, I mean that posts about cabal or stack errors will be greeted warmly (some users may not know the difference between a GHC error and an install-tool error), but perhaps the community will not spend as much time debating a concrete improvement – at least until we have buy-in from the respective developers of cabal and stack (and perhaps other tools, too).
- The goal of this effort is to increase the bandwidth and quality of information flowing from users to developers. Right now, a GHC user who is frustrated with an error message can only post a GHC bug. I claim without evidence that most GHC error messages are factually accurate, and so GHC developers might write on the bug report that the error is correct as GHC dishes it out. While this statement is true, it’s not necessarily helpful to the poster, who was likely befuddled. Even if the responding GHC dev is interested in improving the understandability and actionability of the message, it is hard to source a diversity of opinions and settle on a path forward that we can be confident in. With this new issue tracker, we hope that the lower barrier-to-entry (the GHC tracker is surely intimidating to many!) will be able to get the voices we need – routine and novice Haskell users with a diversity of backgrounds – to share their thoughts and help craft better messages. Only when a concrete path forward is identified does the (often busy and easily distracted) GHC dev step in to implement the change.
- The main barrier to this effort is simply time and dedication. That is, success will depend on an individual or smallish group of individuals to continue to give this effort love and attention over a period of time, creating a welcoming culture of helpfulness and guiding discussions to useful, actionable, concrete suggestions for tool developers. I am pleased to say that @ketzacoatl has volunteered to be this initial leader, though they will likely need help to pull this off. My role in this is as midwife, ardent supporter, and sympathetic GHC dev. I will likely comment on the repo, etc. (I care deeply about error messages and learnability), but will not be running the show or doing broad organization.
- Prior art: Elm does this (https://github.com/elm/error-message-catalog), and Evan Czaplicki (the leader behind Elm) thinks the approach here has worked well for Elm.
So, what do we think? Is this a good idea? Should we / can we put this in the
haskell GitHub org? (An alternative is to host at, say,
gitlab.haskell.org/ux/errors in a new, tantalizingly-named
ux group. But perhaps we get more contributors at GitHub.)
And, most importantly: are you interested in helping out in community building, curation, and organization?
(Caveat: though @ketzacoatl and I have indeed been scheming together there, they have not reviewed this proposal text and may differ with me on some finer points.)