Despite my quip above. Iāve used LLMs with Haskell, Python, Rust, Typescript as well as other odds and ends so far.
The first time I tried those tools they were usable on python/typescript and worthless on Haskell. Iāve started another trial run a few months back and they seem to work decently on Haskell now.
I donāt know if Jappie used those tools to any meaningful degree in other languages. Because while I agree that encoding and expressing invariants in types is useful. In practice I canāt say I feel like this translates to generated Haskell code being particularly more reliable than generated python code, rust code or anything else.
At least if I take the same amount of effort in expressing invariants. Itās just that instead of adding (partial) type signatures I will add type hints, assertions or plain comments for python. It does make it a bit easier to reason about generated code if it includes type signatures afterwards. And I would rather manually patch up Haskell code than Python code if I have the choice. But that might just be me prefering Haskell in general.
Maybe I just havenāt tried to apply it to complex enough problems in python yet. But so far I canāt say I feel like those tools work better with Haskell. I might think the output, eventually, is better since itās a Haskell program rather than a python program. But that feels somewhat orthogonal.
> That final 10% of the implementation is now my job, which is getting it correct. I have to ensure the software Claude produces works. I have to think in terms of verification.
Personally I often found that doing the āeasyā part of the implementation also makes the verification much easier. I naturally think about the invariants, preconditions and postconditions of various parts of the code while writing it. And often without it adding much more effort to the implementation. And even when using LLMs I think one is much better off when thinking about how to ensure a program is - mostly - correct by construction from the start rather than trying to verify it behaves as expected afterwards. Last but not least because often when letting the LLM do much of the work itās not even that clear what the expect behavior is, making it even trickier to ensure correctness!
Maybe itās still faster overall. But I feel like whenever I used these tools for a non-trivial task where I actually cared about it being correct I didnāt get much of a productivity benefit. One of scripts? Random stuff I donāt care if it crash and burns? Sure. But taking any large AI generated diff and ensuring there arenāt bugs is hard work. And sometimes I found myself thinking it would have been easier if I had written it myself in these cases.