Experimental Auth and reading the request body

31 views

Skip to first unread message

Asa

unread,

Feb 20, 2017, 6:51:02 PM2/20/17

to haskell-servant

Hello,

I have worked around my problem, so this is more feedback than a request for help.

I had a weird problem where my json request bodies were not able to be parsed, even though I verified that they parse correctly in ghci. It turns out it was because I implemented the experimental generalized auth to enable verifying webhooks from github. The short of it is that github webhooks are verified by HMACing the request body with a shared secret, which means I need to read the request body off of the wai request. When I do this, then the JSON parsing part failed because it read 0 bytes from the wai request.

My workaround is to just produce the request body I read in the auth handler and remove `ReqBody '[JSON] ...` from my API types and then parse the JSON manually in my handler functions.

I know the generalized auth is still experimental, but I wanted to give this feedback in case you hadn't heard of this problem before. It's a somewhat subtle and pernicious problem to have and a better API to deal with this is probably necessary before generalized auth is ready for primetime.

Asa

Alp Mestanogullari

unread,

Feb 21, 2017, 8:00:51 AM2/21/17

to Asa, haskell-servant

I have never thought about this particular scenario, to be honest. What's probably going on is that when you manually read the request body from the 'Request', it "consumes" it entirely. This fits well with chunked request bodies that are large, that you get piece by piece. I think your workaround is fine for now, but it would be nice to give you a more satisfactory answer here.

One possible lead would be to write a custom combinator that would replace both ReqBody and the experimental Auth combinator, say SignedBody, which would extract _and_ verify the request body (you could use the context to put the shared secret there, so that the SignedBody HasServer instance can access it), erroring out when verification fails.

Another lead would be to see if you don't have any way to read the request body without consuming it, but IIRC the API in wai for that is based on IO actions and unless you use something like an IORef to store what you've read and reuse that further down the road. Not sure it's possible, and it most definitely is a bit ugly.

I would definitely recommend exploring the SignedBody approach, if you ever feel like spending the required amount of time to get it working. Feel free to reuse this thread for questions/problems encountered while trying to implement it, of course.

--
You received this message because you are subscribed to the Google Groups "haskell-servant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to haskell-servant+unsubscribe@googlegroups.com.
To post to this group, send email to haskell-servant@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/haskell-servant/d5136285-2fc5-425e-aaf9-e1437bfa4066%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Alp Mestanogullari

Asa

unread,

Feb 21, 2017, 10:42:36 AM2/21/17

to haskell-servant, asa....@gmail.com

Yea, I am fairly sure the reading the request body in IO is doing the consuming as you mention.

I thought about the custom Content Type but custom content types can't access other headers, where in my case I need the value in the "X-Hub-Signature" header (this is a github integration) to complete the validation. And going in deeper to write a component that has full access to the request is a bit more intimidating ;)

If I take up the challenge I will certainly use this thread to solicit advice. Thanks for the offer!

Asa

On Tuesday, February 21, 2017 at 8:00:51 AM UTC-5, Alp Mestanogullari wrote:

I have never thought about this particular scenario, to be honest. What's probably going on is that when you manually read the request body from the 'Request', it "consumes" it entirely. This fits well with chunked request bodies that are large, that you get piece by piece. I think your workaround is fine for now, but it would be nice to give you a more satisfactory answer here.

One possible lead would be to write a custom combinator that would replace both ReqBody and the experimental Auth combinator, say SignedBody, which would extract _and_ verify the request body (you could use the context to put the shared secret there, so that the SignedBody HasServer instance can access it), erroring out when verification fails.

Another lead would be to see if you don't have any way to read the request body without consuming it, but IIRC the API in wai for that is based on IO actions and unless you use something like an IORef to store what you've read and reuse that further down the road. Not sure it's possible, and it most definitely is a bit ugly.

I would definitely recommend exploring the SignedBody approach, if you ever feel like spending the required amount of time to get it working. Feel free to reuse this thread for questions/problems encountered while trying to implement it, of course.

On Tue, Feb 21, 2017 at 12:51 AM, Asa <asa....@gmail.com> wrote:

Hello,

I have worked around my problem, so this is more feedback than a request for help.

I had a weird problem where my json request bodies were not able to be parsed, even though I verified that they parse correctly in ghci. It turns out it was because I implemented the experimental generalized auth to enable verifying webhooks from github. The short of it is that github webhooks are verified by HMACing the request body with a shared secret, which means I need to read the request body off of the wai request. When I do this, then the JSON parsing part failed because it read 0 bytes from the wai request.

My workaround is to just produce the request body I read in the auth handler and remove `ReqBody '[JSON] ...` from my API types and then parse the JSON manually in my handler functions.

I know the generalized auth is still experimental, but I wanted to give this feedback in case you hadn't heard of this problem before. It's a somewhat subtle and pernicious problem to have and a better API to deal with this is probably necessary before generalized auth is ready for primetime.

Asa

--
You received this message because you are subscribed to the Google Groups "haskell-servant" group.

To unsubscribe from this group and stop receiving emails from it, send an email to haskell-serva...@googlegroups.com.
To post to this group, send email to haskell...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/haskell-servant/d5136285-2fc5-425e-aaf9-e1437bfa4066%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.