Hasql v2 is out. It can now run natively in Haskell with no external dependencies, or the same way it always has, using “libpq”. It’s the user’s choice now. No performance degradation and minimal changes to the API.
I’ve written up the post explaining the reasoning and how it was achieved:
What’s the difference between pqi and hpgsql? cc @mzabani
AFAICT (and quoting the docs a bit) pqi reproduces the API surface of the postgresql-libpq package without any ties to libpq or any particular implementation of that API itself.
hpgsql provides a much higher-level API than anything trying to replace postgresql-libpq - it’s more of a cousin to postgresql-simple and hasql.
hpgsql has its own internal and lower-level APIs, of course, that I evolved while developing hpgsql to have the things I wanted, including for example interruption safety, thread safety and streams. They are very different from postgresql-libpq’s, and I honest don’t know how easily it could be retrofitted to a libpq-style API, though I think it should be possible (maybe with minor tradeoffs).
My (biased and actually not that well informed about libpq) opinion, however, has been that the PostgreSQL ecosystem in Haskell would benefit from moving away from libpq-style APIs. We have wonderful abstractions and luxuries - from Streams to interruption safety and just plain pure code -, and though I didn’t look too deeply, while I was developing hpgsql it felt like not having the libpq-style APIs forced on me gave me a lot of freedom to achieve those goals, not to mention a lot of performance work (and there’s still a lot more of that coming!).
I of course understand that asking other library authors to reimplement their libraries is a huge effort and risk, and that there’s a long way towards native postgres implementations to support all the auth methods and TLS encryption, so I applaud Nikita’s effort to make a call towards a common interface. And if I ever expose one from hpgsql, I’ll make sure to adapt it to pqi.
FWIW, I successfully built my DNSSEC/DANE survey engine with each of pqi-ffi and pqi-native. For the latter I had to add support for unix-domain socket connections (as noted in the issue and pull request).
Once both were built, though “native” produces correct results I quickly found that performance of “native” is 3–6x slower when processing a bulk report fetching 10k rows at a time from a cursor for a total of around 20M rows.
The inner loop uses Hasql.Decoders.rowVector to decode and process the rows returned by each fetch. This is perhaps expected given the initial focus on correctness.
I’ve uncovered a usecase of pqi that I did not intend. It’s trivial to implement all sorts of proxies between hasql and the underlying transport implementation (pqi-native, pqi-ffi).
One use case that it opened for me is testing edge cases of hasql against a mock pqi adapter.
But now I can imagine people implementing diagnostics and observability proxies that are specialized to pqi and thus are compatible not just with hasql, but any lib that integrates with pqi in the future.
Think wai for postgres.