Hey folks, I’m working on some bindings for llama.cpp, and before I try pushing this to Hackage I was hoping to get some feedback from others in the community. Long-term my goal is to try embedding some LLM functionality in my own home automation system, but I’m relatively inexperienced at both FFI and anything to do with LLMs, so I’d appreciate any feedback anyone has.
I also figured others may be interested in messing around with this and helping me figure out how to approach an intermediate API layer on top of the low-level FFI interface I’ve got now–I haven’t yet figured out what a good intermediate wrapper would look like for this. I’d also like to add more examples mimicking what is present in llama.cpp…and make the main example feature complete with what is in llama.cpp…and add tests…etc. etc.
In any case, if anyone wants to poke at it with me let me know, or just give me some feedback in the github repo, etc. Thanks for taking a look!
(P.S. here is where I shamelessly mention that I’m looking for employment, in Haskell or otherwise, if anyone is looking for a curious, experienced, friendly developer )