The 2.0 release of Amazonka, the Haskell AWS SDK, has been uploaded to Hackage. A full release announcement, including a link to a migration guide, is available as a GitHub discussion.
Thank you all for your patience, this has been a lot of work.
22 Likes
Is there a blessed streaming abstraction for services where it makes sense? I’ve written some FS2 wrappers around chunks of the Java SDK for Scala, and I wouldn’t mind taking a crack at something similar with streaming or streamly, but I don’t wanna duplicate work.
Where botocore (AWS’s IDL which describes services) declares something we can paginate, amazonka's paginate function will stream results out using conduit. Similarly, endpoints which can accept streaming inputs (currently only S3 - it looks like we can’t stream to glacier:UploadArchive even though we should be able to; I think this is a bug) can accept a conduit to provide a streaming body.
There’s nothing stopping you from writing your own version of paginate which emits a stream in your favourite streaming library. Providing a streaming body using other libraries might be a bit harder. The easiest thing for now is probably to convert to conduit just before you hand it off to amazonka. I’ll make an issue (soon) and have a think (later).
3 Likes
Streaming responses are much more common in my use cases, so that’s totally good. I’ll play around with it sometime soon, undoubtedly. I’ve been wanting to demo some proof of concept code at work for something semi-realistic, but I was waiting for the big release to dig in too hard. Thanks to all the contributors for getting this over the line.
Also, on the upload side, there is amazonka-s3-streaming which (despite the name) consumes ByteStrings from a conduit and creates S3 multipart uploads.
3 Likes
Oh, one other question: I know previously (and I seem to infer from your comments here) that the generator worked off of the definitions in botocore. Has Amazon’s use of the Smithy IDL matured enough that it might provide a less painful future path?
There’s some prior art in Scalaland based on https://github.com/aws/aws-sdk-js-v3/tree/main/codegen/sdk-codegen/aws-models from the Disney Streaming folks, though I think that’s just packaging the specs so that smithy4s users don’t have to vendor the specs themselves.
As yet I’m not aware of anything with the scope of Amazonka, anywhere, so if that would be a helpful direction to explore I may be able to devote some cycles to it in the next year. Having an up to date and complete native SDK on a regular release cadence would remove a fairly major hurdle for Haskell adoption, to my mind.
1 Like
I haven’t thought much about smithy, but the generator will need a thorough overhaul before too long: we can only generate against a specific botocore revision because any others change services in a way which produces non-compiling code. This isn’t AWS’ fault, but because of the way the generator currently handles cycles. See: deriving clauses of Ptr are not calculated correctly and track botocore order of shape fields in generator. Everything is currently working correctly with this specific ref of botocore and hashable-1.3; anything else changes the traversal order of the HashMaps too much.
I want to decouple the generator from botocore anyway so that people can use it for other purposes. If we had a standalone generator, then anyone using IAM authentication with API Gateway would be able to generate their own Amazonka-style SDKs. This includes things like Amazon’s Selling Partner API (SP-API), which provides endpoint descriptions via Swagger.
Because Amazonka has accumulated a lot of botocore fixes, I can’t see us moving directly to smithy, but once we have a decoupled data model for the generator, it may be worth integrating a Smithy reader. I would have to think more about how they do protocols and stuff before I understand whether we can use more than just their shape definitions.
4 Likes
Awesome! Thank you very much!
1 Like
I wrote up a retro of sorts for the 2.0 journey, which I hope will be useful to other people looking to resurrect stuck projects they care about: The Road to Amazonka 2.0 | Blog | jackkelly.name
9 Likes