I’m having difficulties implementing streaming of s3 object through servant endpoint using amazonka-s3.
I boiled down my code to this standalone single-module cabal project:
The currently working solution is inspired by this (5 years old) example from Domen Kožar.
The issue is that the “intuitively looking” implementation
runResourceT $ do
resp <- send env req
pure $ toSourceIO $ respBodyConduit resp
which just unwraps the conduit from amazonka’s GetObjectResponse’s body and converts it to servant’s SourceIO
seems to close HTTP connection even before streaming starts.
I suspect (though not sure) that the issue stems from the fact that runResourceT
which is required to run send
from amazonka somehow finalizes the response body’s conduit before this runResourceT
from servant-contuit has a chance to run the stream.
The workaround that seems to work creates resourcet’s InternalState
explicitly, and adds the closeInternalState
as last “step” of the conduit, which makes sure that resources are finalized after the conduit finishes streaming.
But I must say this feels very non-idiomatic and somewhat unsatisfactory.
Isn’t this solution exception unsafe (e.g. what is the conduit from the response throws an exception? Wouldn’t this mean that finalization of resources is not performed?)
I tried using bracketP in an attempt to make this code more exception safe, but this feels hacky (because the resourcet’s InternalState
needs to be initialized outside of bracketP to make it possible to run amazonka’s send
…)
st <- createInternalState
resp <- runInternalState (send env req) st
pure $ toSourceIO (bracketP (pure () ) (\() -> closeInternalState st) (\() -> respBodyConduit resp))
Has anyone implemented something similar in nicer way? Is there more “canonical” looking solution to this? Or do you think that workaround is safe (as in “no resources will be leaked in case that response conduit throws an exception”)?