`Servant` and `servant-reflex`: debugging a "bad request"

[rubenmoor]

I am running out of ideas while looking for an error in my code. I am using Obelisk with Servant and servant-reflex to have my XMLHttpRequests generated automatically.

I just added a route to my api, like this:

type RoutesAdmin =
  AuthRequired "jwt" :> "journal" :> ReqBody '[JSON] (UTCTime, UTCTime) :> Get '[JSON] [Journal]

… to realize that, when testing in my browser, the server responds with “bad request”. Checking the payload of the request (also in the web browser), I find nothing. So it looks like, indeed, the request is malformed.

But how?

This is the code where servant-reflex generates my client code:

getJournalAll
    :: SupportsServantReflex t m
    => Dynamic t (Either Text (CompactJWT, Text))
    -> Dynamic t (Either Text (UTCTime, UTCTime))
    -> Event t ()
    -> m (Event t (ReqResult () [Journal]))

((postConfigNew :<|> getDocDEPatternAll :<|> getDocDEPattern :<|> getDictDE :<|> getDictDENumbers) :<|> (getJournalAll) :<|> (postAuthenticate :<|> postAuthNew :<|> postDoesUserExist :<|> postLogout) :<|> ((postAliasRename :<|> getAliasAll :<|> postAliasSetDefault) :<|> (getAppState :<|> postAppState)) :<|> (postEventViewPage :<|> postEventStageCompleted))
    = client (Proxy :: Proxy RoutesApi)
             (Proxy :: Proxy (m :: * -> *))
             (Proxy :: Proxy ())
             (constDyn (BasePath "/"))

The fact that my codes typechecks and then generats a “bad request” really seems weird. It doesn’t seem like I have a lot of room for error.

• a type mismatch causes a compiler error
• omitting arguments in the client or the backend code caues a compile error
• having a function argument not ready, is reflected by the Either type and doesn’t cause a “bad request”

It seems like the client builds the request code, ignoring the request body.

My custom-made combinator AuthRequired could be the culprit, but I use other routes with AuthRequired and ReqBody just fine.

Any idea how to debug this?

[atravers]

Based on this:

Help needed narrowing down an IO exception

I found the cause of my demise: verifyClaimsAt indeed fails to verify, because I set the expiry to 30 seconds instead of 30 days. So far so good, otherwise I would have noticed the problem in 30 days. Jumping out of the function verifyCompactJWT, it gets evaluated via runExceptT to Left JWTExpired. This I never see because I then move on to call throwError :: MonadSnap m => ServantError -> m a … which apparently cannot be used safely with evalSnap (it seems to work fine in the normal case, w…

…maybe you’ve just found another “[dark] corner case” for Servant (or Obelisk). If not, and the docs for Servant don’t mention it, find the smallest example which causes the error and attach it to a polite message to the Servant devs - it could be a new problem for them too.

[rubenmoor]

That IO exception of snap could be avoided by changing evalSnap :: Snap a to something like evalSnapSafe :: Snap (Either Text a) … I made a small pull request, documenting how evalSnap is meant to be used, to save future users the headache I had.

This “bad request” problem now is about servant and servant-reflex. I agree that I will probably need a minimal example that reproduces the problem. But at the moment I am just lost in the dark:

• Why does this one route cause a malformed request and not any other? For a minimal example that reproduces the malformed request, I would need to narrow down the cause of the problem a bit more.
• Why am I the first that ever encounters this? Googling “servant bad request” doesn’t give a lot of hits. It seems likely that there is a problem with my code rather with any of the libraries … but at the same time, my code isn’t awfully special.

I tried around for a bit, but to no avail:

• moving the offending route up and down in the api
• deleting other routes
• changing arbitrary parameters of the route
• changing the type of the request body from ReqBody '[JSON] (UTCTime, UTCTime) to a custom newtype, to (Day, Day), to Bool

… but the problem is persistent: As soon as I want to send data via ReqBody '[JSON] the result is a malformed request (without payload).

[atravers]

I tried a longer search term:

haskell "servant" browser "bad request" "JSON"

A search for "JSON" here:

CORS support

brought up this message, complete with a a small working (at that time) example - you may be able to use it as a test-rig for your bug-hunt.

Reading some more brought up a subsequent message with the following remark:

simpleCors uses simpleCorsResourcePolicy. If you read it’s documentation, you can see this:

If the request is a POST request the content type is constraint to simple content types (application/x-www-form-urlencoded, multipart/form-data, text/plain)

So you can’t use application/json (which is what is used for ReqBody '[JSON] Player)

…so it could be one or more libraries Servant is using that’s causing the error because they don’t support the JSON format.

[rubenmoor]

Thanks a lot @atravers

I found the problem:

HTTP GET with request body

I am using a request body in an HTTP GET request, which

• is allowed according to the latest HTTP standard
• … but according to same standard it is also non-sensical as the response to a GET request isn’t supposed to take any request body into account.

Finding the source of my problem has been complicated by the fact that Servant.API is very permissive and doesn’t indicate any problem with a request body for a GET request (I don’t blame them for that) … while Servant-Snap thinks a GET request with a request body is a “bad request” - which isn’t entirely false either. Just the combination of these two factors - for someone who isn’t well acquainted with HTTP standards and best practices - leads to this confusing situation.

Immediate solutions are:

• switch to a POST request (no downside apart from the fact that the semantics don’t fit quite well)
• using query parameters to send options along with a GET-request (my preferred choice)

[rubenmoor]

cf. https://github.com/haskell-servant/servant-snap/issues/33

my suggested solution for servant-snap