Documentation on how to use rodauth in JSON mode

[Tiago Cardoso]

I'm trying to setup jwt authentication, and have been having issues with the required dependencies. Usually, rodauth requires rack_csrf, but I've read that this is not needed when roda is in "json" mode. What I didn't find out was documentation on how to enable this json mode, and how to proceed to fetch the token using the JSON API (the endpoints). 

Does this documentation exist somewhere? If not, how to best help?

[Jeremy Evans]

On Wednesday, September 14, 2016 at 6:28:38 AM UTC-7, Tiago Cardoso wrote:

I'm trying to setup jwt authentication, and have been having issues with the required dependencies. Usually, rodauth requires rack_csrf, but I've read that this is not needed when roda is in "json" mode. What I didn't find out was documentation on how to enable this json mode, and how to proceed to fetch the token using the JSON API (the endpoints). 

Does this documentation exist somewhere? If not, how to best help?

Enabling support for json can be done via passing the :json option when loading the rodauth plugin into your roda app.  If you only want to support json, you can use :json=>:only, which will skip loading some other plugins.  See http://rodauth.jeremyevans.net/rdoc/files/README_rdoc.html#label-JSON+API+Support

If you want to see some examples of using the jwt feature, check Rodauth's specs.  Basically, your JSON API should check the response Authorization header for the Token, and use that token in the Authorization header for future requests in it is set.  I agree, this should be better documented, I'll add more documentation for it shortly.

Thanks,

Jeremy

[Tiago Cardoso]

I checked that page, but it still didn't help me solve certain issues I had setting it up. I'll try to summarize them:

* Yes, I used the specs (hard to find) and the official documentation. I managed that json: :only setup and the login/logout enabling (and even prefix, as I need an "api/login" endpoint). After that, I proceed testing it up with curl. 

* Testing with curl:

  * "curl -X POST http://localhost:3000/api/login" 500 breaks with error "NoMethodError: undefined method `flash' for " in the backtrace. If it's JSON only, why is it trying to redirect me with flash? With json: true, it at least 302 redirects me to api/login. Shouldn't it assume Content-Type: "application/json" in json: :only mode?

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -H "Accept: application/json"" breaks with 405: Bad Request... with Content-Type set to text/html. the payload was JSON. Is this ok? Do I have to explicitly set up the content-type header in roda when I'm using the json plugin?

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -u ad...@mail.com" blows up on the server side. Appparently this is because basic authentication creates an Authorization HTTP header, and this is assumed by the jwt feature to be the jwt token, when in fact (?) it should ignore the "Basic" tokens. The Basic token credentials would have therefore to be used to login the user into the session and provide a JWT token in the response, but that is not happening. Does rodauth support this type of login strategy?

  * "curl -H "Accept: application/json" -H "Content-Type: application/json" -d '{"login":"ad...@mail.com", "password" : "secret" }' -X POST http://localhost:3000/api/login" finally does what's expected (in json-only mode it doesn't accept non-json request bodies, which seems ok). However, after sending a request to any other route with the obtained JWT token, the same flash error from above comes. How can I turn this off? Because I didn't explicitly set any flash plugin.

I have to say that the last one only comes because I'm calling "rodauth.require_authentication before the endpoints which have to be token-authenticated. Should I be using some other function for this? 

All of this to say that, at least what API-only support concerns, the documentation hasn't been guiding me in the most efficient way. Hopefully this issue description helps. 

[Tiago Cardoso]

I've just found out that the flash errors are removed once I use the header Content-Type: application/json as well. I do feel that this is sort of a bug, because if I already set "Accept: application/json", then the response has to be set as json, right? So having this here (https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/jwt.rb#L55) should have been testing for json_response?, IMO . 

[Jeremy Evans]

On Thursday, September 15, 2016 at 2:01:21 AM UTC-7, Tiago Cardoso wrote:

I checked that page, but it still didn't help me solve certain issues I had setting it up. I'll try to summarize them:

First, thank you for posting exactly what you did and the problems it caused, it is very helpful to get a new users perspective to consider possible improvements to the API.

 

* Yes, I used the specs (hard to find) and the official documentation. I managed that json: :only setup and the login/logout enabling (and even prefix, as I need an "api/login" endpoint). After that, I proceed testing it up with curl. 

* Testing with curl:

  * "curl -X POST http://localhost:3000/api/login" 500 breaks with error "NoMethodError: undefined method `flash' for " in the backtrace. If it's JSON only, why is it trying to redirect me with flash? With json: true, it at least 302 redirects me to api/login. Shouldn't it assume Content-Type: "application/json" in json: :only mode?

I think it's reasonable to assuming json input even without content type if you are in json: :only mode, though you are correct that it doesn't currently do that.  I'll make that change.

The reason for the NoMethodError is that currently if you use json: :only, it doesn't load the flash plugin, but if it isn't a json request, flash calls super, which fails.

 

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -H "Accept: application/json"" breaks with 405: Bad Request... with Content-Type set to text/html. the payload was JSON. Is this ok? Do I have to explicitly set up the content-type header in roda when I'm using the json plugin?

The 405 is expected, since the JSON API is POST-only.  However, I think the content type handling is a bug, it should also set the response content type to application/json in this case.  I'll fix this.

 

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -u ad...@mail.com" blows up on the server side. Appparently this is because basic authentication creates an Authorization HTTP header, and this is assumed by the jwt feature to be the jwt token, when in fact (?) it should ignore the "Basic" tokens. The Basic token credentials would have therefore to be used to login the user into the session and provide a JWT token in the response, but that is not happening. Does rodauth support this type of login strategy?

Rodauth currently does not support Basic authentication by default.  You can use the jwt_token configuration method if you want to try to handle this.  I think it makes sense to recognize when the request Authorization header starts with Basic or Digest and assume no token in this case.  I'll make that change.

 

  * "curl -H "Accept: application/json" -H "Content-Type: application/json" -d '{"login":"ad...@mail.com", "password" : "secret" }' -X POST http://localhost:3000/api/login" finally does what's expected (in json-only mode it doesn't accept non-json request bodies, which seems ok). However, after sending a request to any other route with the obtained JWT token, the same flash error from above comes. How can I turn this off? Because I didn't explicitly set any flash plugin.

Requesting any other route should work if you pass the JWT token in the Authorization header and set the Content-Type appropriately.

 

I have to say that the last one only comes because I'm calling "rodauth.require_authentication before the endpoints which have to be token-authenticated. Should I be using some other function for this? 

No, rodauth.require_authentication is what you should be using.  Hopefully with the changes mentioned above, it will work correctly for you.

 

All of this to say that, at least what API-only support concerns, the documentation hasn't been guiding me in the most efficient way. Hopefully this issue description helps. 

Agreed.  In addition to making the changes mentioned above, I'll see if I can better document how to use this feature.

Again, thank you very much for your detailed feedback.

Jeremy

[Tiago Cardoso]

Glad to help. I'll leave you with some comments regarding other stuff.

quinta-feira, 15 de Setembro de 2016 às 20:59:14 UTC+3, Jeremy Evans escreveu:

On Thursday, September 15, 2016 at 2:01:21 AM UTC-7, Tiago Cardoso wrote:

I checked that page, but it still didn't help me solve certain issues I had setting it up. I'll try to summarize them:

First, thank you for posting exactly what you did and the problems it caused, it is very helpful to get a new users perspective to consider possible improvements to the API.

 

* Yes, I used the specs (hard to find) and the official documentation. I managed that json: :only setup and the login/logout enabling (and even prefix, as I need an "api/login" endpoint). After that, I proceed testing it up with curl. 

* Testing with curl:

  * "curl -X POST http://localhost:3000/api/login" 500 breaks with error "NoMethodError: undefined method `flash' for " in the backtrace. If it's JSON only, why is it trying to redirect me with flash? With json: true, it at least 302 redirects me to api/login. Shouldn't it assume Content-Type: "application/json" in json: :only mode?

I think it's reasonable to assuming json input even without content type if you are in json: :only mode, though you are correct that it doesn't currently do that.  I'll make that change.

The reason for the NoMethodError is that currently if you use json: :only, it doesn't load the flash plugin, but if it isn't a json request, flash calls super, which fails.

To be fair, this should be a rack concern. But as rack currently doesn't handle response objects, either the framework provides the option of setting the default response content type, or I have to use my own rack ContentType middleware, one of the worst of them. And this wouldn't solve the flash object, of course. However, will be looking forward to whichever solution you implement for roda. 

  

 

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -H "Accept: application/json"" breaks with 405: Bad Request... with Content-Type set to text/html. the payload was JSON. Is this ok? Do I have to explicitly set up the content-type header in roda when I'm using the json plugin?

The 405 is expected, since the JSON API is POST-only.  However, I think the content type handling is a bug, it should also set the response content type to application/json in this case.  I'll fix this.

Ditto for this, as I don't think that the Content-Type header should have influence on the response content type. More on that below. 

 

 

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -u ad...@mail.com" blows up on the server side. Appparently this is because basic authentication creates an Authorization HTTP header, and this is assumed by the jwt feature to be the jwt token, when in fact (?) it should ignore the "Basic" tokens. The Basic token credentials would have therefore to be used to login the user into the session and provide a JWT token in the response, but that is not happening. Does rodauth support this type of login strategy?

Rodauth currently does not support Basic authentication by default.  You can use the jwt_token configuration method if you want to try to handle this.  I think it makes sense to recognize when the request Authorization header starts with Basic or Digest and assume no token in this case.  I'll make that change.

But does it support at all (http basic auth)? I didn't see any feature with such support. I think that adding it could be easily done by rewriting such lines (https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/login.rb), subclassing Login Feature and adding logic to inspect the Authorization header for Basic and Digest? If you would agree on such a feature, I could take a stab at PR'ing it on github. 

Also, the Authorization header defines an optional left key (like "Basic: hash" or "Bearer: hash"). The jwt token should therefore try to parse this possible "KEY:\s+" part out of the token and use the remainder (here is how rails solves it: https://github.com/rails/rails/blob/3fc0bbf008f0e935ab56559f119c9ea8250bfddd/actionpack/lib/action_controller/metal/http_authentication.rb#L118). It would of course not be able to decode anything in the case of basic and Digest, but I think that the jwt feature should be guarding against decoding errors (or token expiration). A very naive look here: https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/jwt.rb#L24-L37 tells me that that is not being done, which means that rodauth will never respond back with 498 token expired. Is this correct? I'll open an issue regarding this if so. 

 

 

  * "curl -H "Accept: application/json" -H "Content-Type: application/json" -d '{"login":"ad...@mail.com", "password" : "secret" }' -X POST http://localhost:3000/api/login" finally does what's expected (in json-only mode it doesn't accept non-json request bodies, which seems ok). However, after sending a request to any other route with the obtained JWT token, the same flash error from above comes. How can I turn this off? Because I didn't explicitly set any flash plugin.

Requesting any other route should work if you pass the JWT token in the Authorization header and set the Content-Type appropriately.

I don't agree with the Content-Type part. If you check https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html section 14.17, you'll see that the Content-Type in such a scenario identifies the Content-Type of the request payload, and nowhere there it says that this should be used to imply the response content type (see also https://www.w3.org/Protocols/rfc2616/rfc2616-sec7.html 7.2.1). The Accept header (see section 14.1) should be therefore used to get the preference of the user on the type of content he wants to get back. If none is present, it is assumed that the client accepts all media-types, and the server can choose a preferred format. If more than one media type is present, it should be handled right-to-left in order of preference. Again, and referring to my last comment that wasn't commented, I do feel this is a bug from roda, and will proceed to report it on github, and we can continue there. 

 

 

I have to say that the last one only comes because I'm calling "rodauth.require_authentication before the endpoints which have to be token-authenticated. Should I be using some other function for this? 

No, rodauth.require_authentication is what you should be using.  Hopefully with the changes mentioned above, it will work correctly for you.

 

All of this to say that, at least what API-only support concerns, the documentation hasn't been guiding me in the most efficient way. Hopefully this issue description helps. 

Agreed.  In addition to making the changes mentioned above, I'll see if I can better document how to use this feature.

Again, thank you very much for your detailed feedback.

Jeremy

Again, thank you for the amazing support and tools. Just glad to help. 

[Jeremy Evans]

On Friday, September 16, 2016 at 6:41:22 AM UTC-7, Tiago Cardoso wrote:

Glad to help. I'll leave you with some comments regarding other stuff.

quinta-feira, 15 de Setembro de 2016 às 20:59:14 UTC+3, Jeremy Evans escreveu:

On Thursday, September 15, 2016 at 2:01:21 AM UTC-7, Tiago Cardoso wrote:

I checked that page, but it still didn't help me solve certain issues I had setting it up. I'll try to summarize them:

First, thank you for posting exactly what you did and the problems it caused, it is very helpful to get a new users perspective to consider possible improvements to the API.

 

* Yes, I used the specs (hard to find) and the official documentation. I managed that json: :only setup and the login/logout enabling (and even prefix, as I need an "api/login" endpoint). After that, I proceed testing it up with curl. 

* Testing with curl:

  * "curl -X POST http://localhost:3000/api/login" 500 breaks with error "NoMethodError: undefined method `flash' for " in the backtrace. If it's JSON only, why is it trying to redirect me with flash? With json: true, it at least 302 redirects me to api/login. Shouldn't it assume Content-Type: "application/json" in json: :only mode?

I think it's reasonable to assuming json input even without content type if you are in json: :only mode, though you are correct that it doesn't currently do that.  I'll make that change.

The reason for the NoMethodError is that currently if you use json: :only, it doesn't load the flash plugin, but if it isn't a json request, flash calls super, which fails.

To be fair, this should be a rack concern. But as rack currently doesn't handle response objects, either the framework provides the option of setting the default response content type, or I have to use my own rack ContentType middleware, one of the worst of them. And this wouldn't solve the flash object, of course. However, will be looking forward to whichever solution you implement for roda.

Yesterday I implemented fixes for the issues discussed.  So now json is assumed if you are in :json=>:only mode.  I think this should handle the issue, if you don't think so, please let me know.

 

 

  

 

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -H "Accept: application/json"" breaks with 405: Bad Request... with Content-Type set to text/html. the payload was JSON. Is this ok? Do I have to explicitly set up the content-type header in roda when I'm using the json plugin?

The 405 is expected, since the JSON API is POST-only.  However, I think the content type handling is a bug, it should also set the response content type to application/json in this case.  I'll fix this.

Ditto for this, as I don't think that the Content-Type header should have influence on the response content type. More on that below.

I responded to this in the GitHub issue you opened.  Basically, Rodauth always responds to JSON requests with JSON responses, and never responds to non-JSON requests with JSON responses.  Because of this, it doesn't matter what the Accept header indicates.

 

 

 

  * "curl http://localhost:3000/api/login -I -H "Content-Type: application/json" -u ad...@mail.com" blows up on the server side. Appparently this is because basic authentication creates an Authorization HTTP header, and this is assumed by the jwt feature to be the jwt token, when in fact (?) it should ignore the "Basic" tokens. The Basic token credentials would have therefore to be used to login the user into the session and provide a JWT token in the response, but that is not happening. Does rodauth support this type of login strategy?

Rodauth currently does not support Basic authentication by default.  You can use the jwt_token configuration method if you want to try to handle this.  I think it makes sense to recognize when the request Authorization header starts with Basic or Digest and assume no token in this case.  I'll make that change.

But does it support at all (http basic auth)? I didn't see any feature with such support. I think that adding it could be easily done by rewriting such lines (https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/login.rb), subclassing Login Feature and adding logic to inspect the Authorization header for Basic and Digest? If you would agree on such a feature, I could take a stab at PR'ing it on github. 

Also, the Authorization header defines an optional left key (like "Basic: hash" or "Bearer: hash"). The jwt token should therefore try to parse this possible "KEY:\s+" part out of the token and use the remainder (here is how rails solves it: https://github.com/rails/rails/blob/3fc0bbf008f0e935ab56559f119c9ea8250bfddd/actionpack/lib/action_controller/metal/http_authentication.rb#L118). It would of course not be able to decode anything in the case of basic and Digest, but I think that the jwt feature should be guarding against decoding errors (or token expiration). A very naive look here: https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/jwt.rb#L24-L37 tells me that that is not being done, which means that rodauth will never respond back with 498 token expired. Is this correct? I'll open an issue regarding this if so. 

Rodauth does not currently support HTTP Basic auth in the login feature.  I'm open to supporting it, but as a separate feature (http_basic_auth), not part of the login feature.

I changed the jwt feature to ignore Authorization headers starting with Basic or Digest yesterday.  I'm open to making the regexp for ignoring the header user configurable if you think it is necessary.

 

 

 

  * "curl -H "Accept: application/json" -H "Content-Type: application/json" -d '{"login":"ad...@mail.com", "password" : "secret" }' -X POST http://localhost:3000/api/login" finally does what's expected (in json-only mode it doesn't accept non-json request bodies, which seems ok). However, after sending a request to any other route with the obtained JWT token, the same flash error from above comes. How can I turn this off? Because I didn't explicitly set any flash plugin.

Requesting any other route should work if you pass the JWT token in the Authorization header and set the Content-Type appropriately.

I don't agree with the Content-Type part. If you check https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html section 14.17, you'll see that the Content-Type in such a scenario identifies the Content-Type of the request payload, and nowhere there it says that this should be used to imply the response content type (see also https://www.w3.org/Protocols/rfc2616/rfc2616-sec7.html 7.2.1). The Accept header (see section 14.1) should be therefore used to get the preference of the user on the type of content he wants to get back. If none is present, it is assumed that the client accepts all media-types, and the server can choose a preferred format. If more than one media type is present, it should be handled right-to-left in order of preference. Again, and referring to my last comment that wasn't commented, I do feel this is a bug from roda, and will proceed to report it on github, and we can continue there. 

This definitely isn't a bug in Roda (that's the web framework Rodauth uses).  I don't consider it a bug in Rodauth, as I explain on the GitHub issue, the Accept header should be consulted when the software supports multiple types of responses.  If the server only supports a single response type for the request, the Accept header should be ignored, or at most you should return a 406 error indicating the response was not acceptable.

 

Thanks,

Jeremy