So you want also remove ghc itself from your dependency graph ?
I’m sympathetic to the concern. The category difference between generation prior to LLMs is that:
- it was almost always clearly marked (pgmF, template haskell, the glib-gens, etc) with limited scope, and
- as a result, the output was predictable and regular, so no one ever said “I dunno, it was generated” in a deferential way
LLM use flips both of these, where it is unlimited in scope, not clearly delineated, and many (not all, but I’ve seen smarter devs than me) become deferential with a sense of justified ignorance. “I don’t know how it does what it does, why it’s like that,” both to peer review and when a bug happens. Neither of these are surprising, but it does change the relationship and the social contract (open source is primarily a social thing) between people in a real way, regardless of any value judgment. I’m neutral on LLMs, they obviously have power and obviously have a cost (medium is the message, etc).
How to recognise, manage and mitigate that while retaining quality, joy, personal stake, etc. within a community seems like a reasonable thing to start pondering openly.
Apologies if this doesn’t advance the discussion, just wanted to say it’s on my mind too, @hasufell.
Really, it’s less a problem about vibecoding than careless & sloppy code. The Hackage rating mechanism is kind of underused, and given that the Haskell community tends to have good standards in terms of code quality, encouraging people to use it more might help mitigate the issue. Ratings might correlate well with the quality, and given that a lot of people in the comments share your sentiment about AI, it will also filter out projects that are mostly AI slop. You could just filter out packages with low rating with some script, regardless of whether it’s a bitrotten, vibecoded, or just someone’s pet project that’s unlikely to be any good.
Without any bot protection mechanisms the rating system could be subverted. An example would be a bad actor wanting to replace ms-azure-api with a malicious API where they downrate the good library and uprate their malicious library.
Thanks for expressing your thoughts on this. I definitely appreciate it.
And I agree, LLMs aren’t just another technology. It’s not like google, stackoverflow or previous code generation utilities.
My current perception is that they can subvert the judgement and agency of people I used to trust. I say “can”, because that’s definitely not a generalization. For example, I’m quite confident that Lennart knows what he’s doing. And maybe most Haskellers do.
And yet, “I looked over the code, promise” and “but I take responsibility anyway” aren’t particularly convincing to me.
I don’t really know what it takes to use these tools without falling into the many traps that they expose us to. Erosion of skills, loss of agency, cognitive fatigue, etc. don’t just affect the new generation of engineers I think. It’s very hard to navigate what is happening in real time. And I think transparency is one way we can make it easier to navigate.
Agreed. The degree of LLM involvement in a project is not a good proxy measure for package quality.
So perhaps (derailing the particular task the OP set out to solve) we should look for established measures of code quality in other ecosystems. I presume much literature has been written on that subject.
In one particular ecosystem I happen to know a little, a linter is marketed that gauges
- Mc Cabe Complexity
- Code lines per source file / Libs
- Code lines per function
- Number of functions per file / libs
- Number of parameters per function
- Number of files per folder
- Dead code
These metrics could help to detect LLM slop as well as sloppy human coders alike. There is a study that has produced promising results.
Once the LLMs adapt and write well-organized, maintainable code to subvert the sloppiness filters, we can declare a victory.
The main difference is intent. I can’t really go talk to the LLM and ask about specific intentions regarding specific pieces of code. The “prompt engineer” might never have seen that piece of code and might only vaguely remember the instructions that led to that piece of code.
There might be no particular reason why something is the way it is. It may just be the model weights, the training set, the prompt input, the context. Whatever.
If it was written by a human (even if the quality is sub-par), there’s a high chance I can go talk to them and they might be able to enlighten me.
Even if that’s not possible, I find slop code more alienating. It’s a wild combination of mosaic, unless the prompt was super specific and the context very narrow.
But fair enough… let’s take the prime example of badly written Haskell code, which is in my opinion Cabal. It is the way it is not because the contributors were sloppy coders, but because it had too many generations of maintainers that passed by with huge contributions and then moved on without ever having:
- socialized for an extended period of time
- allowed other maintainers to absorb their ideas, coding style and visions
- taught the next generation and conveyed the intent and caveats of the current codebase
I don’t want to sound too romantic… like programmers gathering around a lake, drinking beer and sharing ideas… oh wait, that was ZuriHac, nevermind.
I mean that we can now witness in Cabal how hard it is to maintain institutional knowledge and visions amongst humans. If you remove the human element, you create that type of legacy nightmare in an instant, where interacting with said code becomes an archeological exercise and causes psychological damage.
I think we all experience this to some degree at work. Long pieces of code, generated by an LLM which nobody really understand. Those pieces become a source of problems every day. At least, that my experience.
Reallistically, LLMs are here to stay, so It would be very unpractical to filter dependencies out. Also, LLM generated code is fine in the right hands, and many times the person behind the clanker, will understand the generated code.
I somewhat sympathise with you, but I am not sure about the approach.
Reallistically, LLMs are here to stay
I was told the same thing about NFTs and the Metaverse. I hope you don’t have a lot of money staked on LLMs staying available under the same conditions forever
Will there be nation-state-level investment in infrastructure committed to them? Probably not for free forever.
Local ones will probably always be around (a decent longterm bet). Diminishing returns for coding help here. Things just have to get good enough.
Will they stick around for coding? Probably, for the same reason they will stick around for other stuff: Dopamine.
That’s why I see this vibe coding craze as a sort of Idiocracy-aligned future. It was the high fructose dopamine hits that did that society in.
Did you have particular vibe-coded packages in Hackage in mind, that exhibit low code quality or a lot of defects?
I started this project: GitHub - hasufell/vibecode-scanner · GitHub
It’s very early stage and the major limitation is how to find the git repo of a cabal project (I utilize cabal get --source-repository=head). Because without that, you can’t do any interesting checks anyway.
Example:
$ cabal --verbose=0 run vibecode-scanner -- scan hackage hatter
{
"scannedAgents": [
{
"commits": 371,
"directories": [],
"files": [],
"name": "Claude Code"
}
],
"scannedPkg": "hatter"
}
The agents definition is here. I plan to allow to extend/overwrite that with a Dhall config.
Please filter out GHC also 
uh oh! luckily GHC has good standards and stewards
Here’s the report:
{
"scannedAgents": [
{
"commitdetails": [
"2df604e98d (Sylvain Henry): Introduce TargetInt to represent target's Int (#15973)",
"3981db0ccf (Sylvain Henry): Add more canned GC functions for common register patterns (#27142)",
"f8de456fb9 (Sylvain Henry): STM: don't create a transaction in the rhs of catchRetry# (#26028)",
"5ebcfb5767 (Benjamin Maurer): Generate assembly on x86 for word2float (#22252)",
"a5ec467ee3 (ARATA Mizuki): rts: Align stack to 64-byte boundary in StgRun on x86"
],
"commits": 5,
"directories": [],
"files": [],
"name": "Claude Code"
},
{
"commitdetails": [
"371ef200bc (Cheng Shao): ghci: use ShortByteString for MkCostCentres message",
"8dd6f45334 (Cheng Shao): ghci: use ShortByteString for LookupSymbol/LookupSymbolInDLL/LookupClosure messages",
"148d36f367 (Cheng Shao): compiler: avoid unneeded traversals in GHC.Unit.State",
"3f5e8d80b3 (Cheng Shao): ci: only push perf notes on master/release branches",
"4c784f00a4 (Cheng Shao): Fix testsuite run for +ipe flavour transformer",
"ca79475f6b (Cheng Shao): testsuite: avoid re.sub in favor of simple string replacements",
"b18b2c42c3 (Cheng Shao): llvm: fix split sections for llvm backend",
"7b616b9f02 (Cheng Shao): compiler: fix regression when compiling foreign stubs in the rts unit",
"2dafc65a48 (Cheng Shao): Tree-wide cleanup of cygwin logic",
"244d57d79a (Cheng Shao): compiler: fix split sections on windows",
"ec3cf767b2 (Cheng Shao): make: remove unused Makefiles from legacy make build system",
"50a5875763 (Cheng Shao): hadrian: only install js files to libdir for wasm/js targets",
"7971f5ddaa (Cheng Shao): deriveConstants: clean up unused constants",
"414d1fe1a5 (Cheng Shao): compiler: fix LLVM backend pdep/pext handling for i386 target",
"53038ea9e8 (Cheng Shao): hadrian: drop unused logic for building ghci libraries",
"81d1013467 (Cheng Shao): configure: remove dead code in configure scripts",
"b26d134a66 (Cheng Shao): rts: opportunistically reclaim slop space in shrinkMutableByteArray#",
"f69c5f1492 (Cheng Shao): wasm: fix handling of ByteArray#/MutableByteArray# arguments in JSFFI imports",
"1ca4b49a56 (Cheng Shao): compiler/rts: fix ABI mismatch in barf() invocations",
"bd38b76cff (Cheng Shao): testsuite: improve coverage of foundation test",
"6a2b43e339 (Cheng Shao): compiler: clean up redundant LANGUAGE pragmas",
"3c0013778b (Cheng Shao): ci: use treeless fetch for perf notes",
"c9b8465c2c (Cheng Shao): wasm: workaround WebKit bug in dyld",
"ab180104b6 (Cheng Shao): driver: emit single LoadDLLs message to load multiple DLLs",
"a4d664c786 (Cheng Shao): compiler/ghci: replace the LoadDLL message with LoadDLLs",
"8cbe006ad0 (Cheng Shao): hadrian: fix GHC.Platform.Host generation for cross stage1",
"867c26755e (Cheng Shao): wasm: fix dyld handling for forward declared GOT.func items",
"0a3a4aa308 (Cheng Shao): docs: document :shell in ghci",
"06d2562362 (Cheng Shao): ghci: add :shell command",
"df4ee9b42c (Cheng Shao): compiler: use zero cost coerce in GHC.CmmToAsm.CFG.loopInfo",
"91f4faaab4 (Cheng Shao): configure: check python3 version and require minimal 3.7",
"0335d8996e (Cheng Shao): driver: use UniqMap ModuleName in the finder",
"b29147977d (Cheng Shao): driver: use UniqSet for hiddenModules in DynFlags/FinderOpts",
"39b2e3823d (Cheng Shao): compiler: only use `Name` instead of `Id` in `SptEntry`",
"783cd7d67d (Cheng Shao): compiler: use `UniqMap` instead of `Map` for `BCEnv` in bytecode compiler",
"894a04f3a8 (Cheng Shao): compiler: fix GHC.SysTools.Ar archive member size writing logic"
],
"commits": 36,
"directories": [],
"files": [],
"name": "Codex CLI"
}
],
"scannedPkg": "unknown-unknown"
}
I like it! To avoid reinventing some wheels, you might want to look at some existing tools that do dependency-scan-type things, e.g. depends, dependabot etc.
@hasufell thanks a lot for pushing the tool out!
The main use-case for me is moderating the copyright erosion and keeping software free&maintainable.
There is precedent for this. Luanti’s ContentDB requires uploaders to declare whether AI was used to produce a package and in what measure.
And from what I’ve seen packages are being tagged accurately. People have said that LLM users are just going to lie and mark their packages as not generated, but I think that’s just a minority of people that’s projecting their attitude on others.
That seems like a very coarse set of options. If I were to hash out a design in a chat session, but type the entire thing in by hand, what do I answer?