I’m having difficulties implementing streaming of s3 object through servant endpoint using amazonka-s3.
I boiled down my code to this standalone single-module cabal project:
The currently working solution is inspired by this (5 years old) example from Domen Kožar.
The issue is that the “intuitively looking” implementation
runResourceT $ do resp <- send env req pure $ toSourceIO $ respBodyConduit resp
which just unwraps the conduit from amazonka’s GetObjectResponse’s body and converts it to servant’s
SourceIO seems to close HTTP connection even before streaming starts.
I suspect (though not sure) that the issue stems from the fact that
runResourceT which is required to run
send from amazonka somehow finalizes the response body’s conduit before this
runResourceT from servant-contuit has a chance to run the stream.
The workaround that seems to work creates resourcet’s
InternalState explicitly, and adds the
closeInternalState as last “step” of the conduit, which makes sure that resources are finalized after the conduit finishes streaming.
But I must say this feels very non-idiomatic and somewhat unsatisfactory.
Isn’t this solution exception unsafe (e.g. what is the conduit from the response throws an exception? Wouldn’t this mean that finalization of resources is not performed?)
I tried using bracketP in an attempt to make this code more exception safe, but this feels hacky (because the resourcet’s
InternalState needs to be initialized outside of bracketP to make it possible to run amazonka’s
st <- createInternalState resp <- runInternalState (send env req) st pure $ toSourceIO (bracketP (pure () ) (\() -> closeInternalState st) (\() -> respBodyConduit resp))
Has anyone implemented something similar in nicer way? Is there more “canonical” looking solution to this? Or do you think that workaround is safe (as in “no resources will be leaked in case that response conduit throws an exception”)?