Since people seem to already publish vibe coded projects to hackage I’m starting to wonder how I can make sure my dependency graph never includes such packages.
There’s no label on hackage and most READMEs have no disclaimers. The best thing you can do is to clone the repo and check whether there are claude code commits or similar.
Maybe it would be possible to maintain a remote cabal.project file that severs as a blacklist (through constraints). But that would have to be maintained manually.
If that would count for a black mark, people would scrub this signal and you end up with vibecoded deps anyway.
OTOH merely having a bot in coauthors is not that strong of a signal. Some repos are 100% pure crystalline slop. The others may used it for bug triage. Those are not the same.
I reckon the best way to find out whether a project is through a combination of scripting and calls to some analysis tool. High level steps:
Write a script that walks the dependency tree. Either with cabal/hackage, or something that walks over Nix expressions to find the source (hackage, GitHub repo, pick whatever you prefer). This first step produces package metadata.
A second part is to iterate over the metadata, and then download the source code, or clone a Git repo, or Darcs, pick whatever
Third, write down a comprehensive definition of “vibe coded". What signals are indicative, what signals are counterindicative. How do different signals should combine to a score and/or conclusion. And crucially: define as precisely as you can, when something falls in the vibe coded bucket and when it does not.
The fourth step is running some coding agent on the downloaded code. Depending on your definition, you can give it access to just reading files, or also some specific Git log/Git search commands. Tell it to output JSON results with signals, location of signals (e.g. file locations or Git commit messages) in a very specific format.
Note to iterate on the above. Let it do some analysis on a known good and known bad package, make up your own judgement, and modify the definition until the agent gives a satisfactory output.
Let the script run on all packages
Then write a final script to summarize the JSON reports into a report
This is another point to iterate: spot check some packages, compare to your own judgement and iterate on the definition and prompt to fine tune
Optionally write a script to create issues on all affected repositories to voice your concerns
The biggest challenge I reckon will be defining what constitutes vibe coded, as that line gets quite blurry fast.
And then the next challenge will be convincing anyone that blacklisting packages based on your specification is a morally defensible choice.
This is already effectively impossible. It’s only going to get harder. It’s like trying to forbid packages that were developed on Windows. There may be some signs, but they’re not definitive and they’re easy to conceal.
I might treat code from LLMs the same way I treat code from third-rate programmers. (The Yonneda lemma says that I may simplify that to: I might treat LLMs the same way I treat third-rate programmers!)
How do I blacklist third-rate programmers? By whitelisting trustworthy programmers instead. Then it is a network of trust. People or code I have directly seen to be good. Word of mouths from them to indirectly infer about other people and code.
(Hell, LLMs in the hands of good programmers can generate good and trustworthy code too. Just look at Terrance Tao… )
Then you should probably not use tower-hs then. Its been actively developed with claude code, interactively, but given the vague terms of vibe coding, its extreme hard to distinguish between the two types of development. In one hand with pure vibe coding, the developer (or the project “manager”) doesnt know ANYTHING about the code, hopefully just that it works for him. In my case and the other use of a AI, i actually read through everything claude suggests and try to understand it and if it makes sense. I dont blindly accept anything just to “get something working” because then it can work or not work in ways its not intended to.
btw i have customInstructions and CLAUDE.md “donts” that explicitely say “no co auth”. If i get help from google or stack overflow or forums, i dont add " Co-Authored-By Haskells discourse forum" hell no. I also am pretty strict to review commits and if some contain co-auth crap i might rewrite the whole history if im not 100% committed yet. Or shed a tear and move on if it reached master. Its MY responsibility. It doesnt HELP to blame or credit claude code. Think about, if you use a special keyboard for development, do you attribute every commit with “co developed with Foobar keyboard”? nope. Claude code is just a tool. It wont take over the world. (just improve the speed of which we get new libraries, which before was EXTREMELY tedious and needed at least an awesome set of devs, or a very very persistent single one). (yes i got a bit carried away there maybe)
But I think that’s not really the point. There’s something deeply untrustworthy about it. Not just about the nature of LLMs, but also the companies behind it.
A swarm of vibe coders who have never coded anything real in their entire lives are suddenly let loose onto the world and the open source ecosystem. I’m not sure how you can say “but there are bad software engineers too” and brush that problem off as non-existent.
I think open source ecosystems indeed are going to be faced with the problem of how to protect themselves from slop generated garbage. It’s not the occasional dude who’s uploading university exercises to hackage. And for that, I already have a personal blacklist anyway. I just want one for all vibe-coded projects, regardless of their authors.
I suspect a lot of the vibe coding is more work than people let on. But they wanna sell the magic.
I tried to have Claude code up a tool that would use Claude API to fill typed holes. I did it twice and used /plan. It made pretty egregious errors both times (not feeding errors back into the context, not reacting to compilation failures at all, not parsing GHC diagnostic JSON correctly somehow)
I could’ve just iterated with Claude. But now that’s already programming with extra steps to me. It’ll be easier just to program it myself.
i also could’ve instructed it to go a module at a time. but that’s so much work - i was promised magic.
also..it produced like >1k LoC for such a simple tool. That’s approaching my game jam game sizes..and a game is way more involved that the tool I wanted.
So yeah..programming Haskell is so easy idk why ppl are vibing outside of lack of skill [1]
[1] I wanna clarify here: I am not being flippantly mean or rude when I say this. I mean it literally. if you lack the skill but still want Haskell code, you can use Claude to get that.
Although the skill isn’t that hard to attain..reading half of LYAH in a week was enough for 22yo me in college to be dangerous and program my capstone project in Haskell. I honestly think if you teleported Claude to 2014, it still wouldn’t have helped novice me do Haskell better lol.
Think of it instead as a tacit acknowledgement of powerlessness. LLMs are merely the hot new addition to the smouldering pile of vomit that is today’s Internet, and if there’s one lesson to learn from all the previous additions it’s that none of them ever go away.
One thing you can do is to keep a list of LLM-oriented files, such as CLAUDE.md, GEMINI.md etc. and look for those file names both in the repository and in the .gitignore (some files, such as CLAUDE.local.md,are not meant to be versioned). The presence of these files is a strong indicator that LLM’s were somehow involved in the development.
Some of my C code (the base64 transducer) was vibes coded. It’s clearly says so in the file. The code is of the same quality as the other transducers. I don’t see what the problem is with this.
Similarly, the YAML parser used by MicroCabal was vibes coded. It’s ugly, but less buggy than my hand written parser.
I’m not ashamed of this, and I still feel responsible for the code I upload.
Just to be clear. I’m not trying to shame anyone. But I think “vibe coded” in general is a very strong indicator for “poor quality” or “untrustworthy”.
I mean, just look at the leaked claude code source code. It’s utter garbage. And this is produced by one of the biggest AI companies in the world. The incident also speaks for itself. It seems even them with their insane resources can’t manage any sort of minimal QA.
Sure, this is just an indicator and it can be wrong. But I still think it’s useful to have a list of vibe-coded dependencies, even if it’s just for me to take a closer look at them.