# Safety of LLM generated code

**URL:** <https://discourse.haskell.org/t/safety-of-llm-generated-code/14562>\
**Category:** Uncategorized\
**Created:** [August 15, 2026, 11:27am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562 "2026-08-15T11:27:21Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![philh](https://avatars.discourse-cdn.com/v4/letter/p/7ab992/32.png) [@philh](https://discourse.haskell.org/u/philh)\
**Post date:** [August 15, 2026, 11:27am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/1 "2026-08-15T11:27:21Z")

</div>

> [@\[ANN\] ychr is on Hackage](https://discourse.haskell.org/t/ann-ychr-is-on-hackage/14559/8):
>
> I wrote such a bug myself (arbitrary file deletion) and it took me months to spot it

That seems like probably a very different failure mode to me?

Like, the kind of bug that allows arbitrary file deletion in a program that’s intended to be able to delete specific files, is very different from the kind of bug that allows arbitrary file deletion in a program that’s not intended to be able to delete files. I’m imagining you were working on the first kind of program, and ychr is the second.

I’m imagining that ychr has a handful of uses for IO: reading files, mutable variables (IORef/MVar/STM/…), exceptions, printing to stdout/stderr. Maybe one or two I’m forgetting. But no writing to the filesystem, and no network access. Assuming that’s right (which it may well not be!) then I think “ruling out hidden destructive bugs to sufficient degree of confidence” isn’t all that difficult.

---

<div class="post-metadata">

**Author:** ![hasufell](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/hasufell/32/1250_2.png) [@hasufell](https://discourse.haskell.org/u/hasufell)\
**Post date:** [August 15, 2026, 11:42am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/2 "2026-08-15T11:42:00Z")

</div>

> [@philh](#):
>
> I’m imagining that ychr has a handful of uses for IO:

I think our imagination is irrelevant when dealing with slop. And it doesn’t matter what you prompted. It’s known that LLMs sometimes ignore instructions and that unrelated code can make it into the implementation.

I don’t think any amount of effects system or safe haskell would convince me.

Either you own the output and understand it or you’re playing code roulette.

---

<div class="post-metadata">

**Author:** ![philh](https://avatars.discourse-cdn.com/v4/letter/p/7ab992/32.png) [@philh](https://discourse.haskell.org/u/philh)\
**Post date:** [August 15, 2026, 12:24pm UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/3 "2026-08-15T12:24:18Z")

</div>

I’m not imagining details about what the LLM generated, I’m imagining details about how to check what the LLM generated.

Like, if you know that a codebase shouldn’t have any IO at all, then checking it doesn’t is fairly straightforward. Do you agree with that?

---

<div class="post-metadata">

**Author:** ![gregbm](https://avatars.discourse-cdn.com/v4/letter/g/4da419/32.png) [@gregbm](https://discourse.haskell.org/u/gregbm)\
**Post date:** [August 16, 2026, 2:53am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/4 "2026-08-16T02:53:01Z")

</div>

> How do I know your program doesn’t delete my home directory?

Same way you’d know for human-generated code on hackage, probably.  
Speaking of which, what defenses does hackage have against malicious packages?

---

<div class="post-metadata">

**Author:** ![hasufell](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/hasufell/32/1250_2.png) [@hasufell](https://discourse.haskell.org/u/hasufell)\
**Post date:** [August 16, 2026, 3:18am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/5 "2026-08-16T03:18:55Z")

</div>

> [@gregbm](#):
>
> Same way you’d know for human-generated code on hackage, probably.

That’s a pretty strange way to dismiss the “lack of agency” issue. And I don’t think it can be taken seriously.

> **[Vibe Coding’s Security Debt: The AI-Generated CVE Surge](https://labs.cloudsecurityalliance.org/research/csa-research-note-ai-generated-code-vulnerability-surge-2026/)**
>
> Key Takeaways Empirical research across Fortune 50 enterprises found that AI-assisted developers produce commits at three to four times the rate of their peers but introduce security findings at 10…

> - Empirical research across Fortune 50 enterprises found that AI-assisted developers produce commits at three to four times the rate of their peers but introduce security findings at 10x the rate, creating a security debt that accumulates faster than organizations can remediate it [7].

> **[Understanding the (In)Security of Vibe-Coded Applications](https://arxiv.org/abs/2606.23130)**
>
> Recent advances in large language models (LLMs) have enabled vibe coding, an emerging software development paradigm in which users create applications primarily through natural-language interactions with AI agents. Due to its low barrier to entry,...

> Our study suggests that securing vibe-coded appli-  
> cations requires treating vibe coding as a workflow-level security problem more than a code-generation problem

---

<div class="post-metadata">

**Author:** ![Ambrose](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ambrose/32/5672_2.png) [@Ambrose](https://discourse.haskell.org/u/Ambrose)\
**Post date:** [August 16, 2026, 3:55am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/6 "2026-08-16T03:55:35Z")

</div>

LLMs change the game here fundamentally sadly. They rework distributed trust because there is less of a hurdle of effort to get something compiling and plausibly working.

I see this myself in industry with junior devs. Before, they would get stuck and be unable to cut a PR that looked reasonable. Now, the LLM can give them that by default. But the LLM PRs have some pretty silly issues that trace back to the juniors’ lack of understanding.

---

<div class="post-metadata">

**Author:** ![gregbm](https://avatars.discourse-cdn.com/v4/letter/g/4da419/32.png) [@gregbm](https://discourse.haskell.org/u/gregbm)\
**Post date:** [August 16, 2026, 4:14am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/7 "2026-08-16T04:14:56Z")

</div>

With regards to trusting code on hackage, I am more worried about intentionally-introduced malware than I am about vibecoded packages, so I would be interested in a verification mechanism that isn’t just scanning for signs of AI-generated code.

---

<div class="post-metadata">

**Author:** ![hasufell](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/hasufell/32/1250_2.png) [@hasufell](https://discourse.haskell.org/u/hasufell)\
**Post date:** [August 16, 2026, 5:03am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/8 "2026-08-16T05:03:52Z")

</div>

> [@gregbm](#):
>
> I am more worried about intentionally-introduced malware than I am about vibecoded packages

Models can very well be compromised by state actors.

---

<div class="post-metadata">

**Author:** ![lortabac](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/lortabac/32/4597_2.png) [@lortabac](https://discourse.haskell.org/u/lortabac)\
**Post date:** [August 16, 2026, 11:51am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/9 "2026-08-16T11:51:49Z")

</div>

My two cents:

- The same models that empower inexperienced developers to ship sloppy code, are also **extremely** good at discovering bugs. This week alone, with the help of AI I caught and fixed 5 different subtle bugs at work. These bugs had been introduced by competent Haskell developers before LLMs were popular.
- Yes, models can be compromised, even by state actors. I believe independent reviews between models of different companies should become more common. E.g. an American model implements, a Chinese model reviews, and viceversa. This is not only for paranoia, but also because different models tend to focus on different aspects.

---

<div class="post-metadata">

**Author:** ![Ambrose](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ambrose/32/5672_2.png) [@Ambrose](https://discourse.haskell.org/u/Ambrose)\
**Post date:** [August 16, 2026, 3:51pm UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/10 "2026-08-16T15:51:59Z")

</div>

Yeah I am a fan of LLM code review and using it to navigate a large codebase. Those two things feel like free cheese to me.

I fix the bugs by hand though. So it kind of feels like a minigame. And because I’m in there by hand, it’s also just some grease for me to polish things up in general.

> <https://github.com/hawkw/mycelium/blob/b77e854bcbbe7c64f644e4c6ceeaebb74c80c79a/CONTRIBUTING.md#ai-policy>

^ that’s why i really like this AI policy. “if i can’t smell the LLM on the code, i’ll accept it.” it’s a nice forcing function.

(and the package in the thread this one forked one sadly did not pass that sniff test for me upon opening Hackage not knowing LLM or nah)

---

<div class="post-metadata">

**Author:** ![hasufell](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/hasufell/32/1250_2.png) [@hasufell](https://discourse.haskell.org/u/hasufell)\
**Post date:** [August 17, 2026, 4:40am UTC](https://discourse.haskell.org/t/safety-of-llm-generated-code/14562/11 "2026-08-17T04:40:34Z")

</div>

> [@lortabac](#):
>
> - The same models that empower inexperienced developers to ship sloppy code

I’m not sure you read the studies I linked above, but “**Empirical research across Fortune 50 enterprises […]**” doesn’t read like “inexperienced developers” to me.

> [@lortabac](#):
>
> are also **extremely** good at discovering bugs

Sure, I’ve also considered that use case: ask an LLM to find bugs in my codebase prior to a release. If it doesn’t find any bugs, I didn’t really lose anything (LLM not finding bugs doesn’t mean your application is secure… this is not how it works).

But your calculation doesn’t really compute… this is for cases where there’s actually an author who understands the code and is familiar with it. You can’t claim that the bug-searching capabilities of AI is enough to replace the need for someone to understand the code. And the research I linked supports this.