JSON-RPC over HTTP and Batch-related questions
516 views
Skip to first unread message
Ted M. Young
Mar 18, 2013, 2:24:47 PM3/18/13
to json...@googlegroups.com
Hi,
I'm implementing a prototype of JSON-RPC for Gosu and have a few questions.
1. It seems that most (all?) implementations of JSON-RPC 2.0 are assuming HTTP as the transport (I haven't looked at every implementation, so I'd be interested in hearing about those that use a different transport), however, there are requirements of HTTP that would be useful to specify in order to properly create inter-operable services, e.g., what content type to use (application/json? nothing?), what to status code to return for a Notification Request (204 or 200?), etc. Is there such a specification (or "sub-specification", not sure how that would be classified)? We could then implement some automated tests ("compatibility kit"-like) using a basic web page.
2. I see that some (many?) implementations of 2.0 don't implement every piece, the batch is the typical one left out. It's unclear to me whether the Batch capability is a requirement of the server. I see that "the Client MAY send an Array filled with Request objects", and "The Server should respond with an Array containing the corresponding Response objects", but it doesn't say that the Server MUST respond in that way to a Batch call. It's implied, but given implementations that don't support it, I'm not sure whether people are simply not bothering to implement the whole spec, or see Batch calls as optional and therefore see their implementations as fully 2.0 compliant. I ask because I'd rather not implement Batch (because it requires me to detect whether I'm getting a JSON block, or an array of JSON blocks, which isn't trivial).
3. With the Batch call, any JSON objects in the array that have errors will be represented in the response with an Error Object. It'd be nice if, for Batch at least, the Error Object included the Request Object contents, if there's no ID for the Request, or the ID if one exists. This would make it easier for the caller to figure out which Request Objects in the Batch Array were malformed.
Thanks!
;ted
--
http://about.me/tedmyoung
I'm implementing a prototype of JSON-RPC for Gosu and have a few questions.
1. It seems that most (all?) implementations of JSON-RPC 2.0 are assuming HTTP as the transport (I haven't looked at every implementation, so I'd be interested in hearing about those that use a different transport), however, there are requirements of HTTP that would be useful to specify in order to properly create inter-operable services, e.g., what content type to use (application/json? nothing?), what to status code to return for a Notification Request (204 or 200?), etc. Is there such a specification (or "sub-specification", not sure how that would be classified)? We could then implement some automated tests ("compatibility kit"-like) using a basic web page.
2. I see that some (many?) implementations of 2.0 don't implement every piece, the batch is the typical one left out. It's unclear to me whether the Batch capability is a requirement of the server. I see that "the Client MAY send an Array filled with Request objects", and "The Server should respond with an Array containing the corresponding Response objects", but it doesn't say that the Server MUST respond in that way to a Batch call. It's implied, but given implementations that don't support it, I'm not sure whether people are simply not bothering to implement the whole spec, or see Batch calls as optional and therefore see their implementations as fully 2.0 compliant. I ask because I'd rather not implement Batch (because it requires me to detect whether I'm getting a JSON block, or an array of JSON blocks, which isn't trivial).
3. With the Batch call, any JSON objects in the array that have errors will be represented in the response with an Error Object. It'd be nice if, for Batch at least, the Error Object included the Request Object contents, if there's no ID for the Request, or the ID if one exists. This would make it easier for the caller to figure out which Request Objects in the Batch Array were malformed.
Thanks!
;ted
--
http://about.me/tedmyoung
Roland Koebler
Mar 18, 2013, 4:54:29 PM3/18/13
to json...@googlegroups.com
Hi,
> 1. It seems that most (all?) implementations of JSON-RPC 2.0 are assuming
> HTTP as the transport
no, they don't, and JSON-RPC 2.0 explicitely does not depend on HTTP.
I'm using JSON-RPC 2.0 over TCP and Unix Domain Sockets, and my Python-
implementation will soon support multiple JSON-RPC-requests per connection
and JSON-RPC over netstrings.
AFAIK, Kamailio also uses JSON-RPC over netstrings.
> there are requirements of HTTP that would be useful to specify in order to
> properly create inter-operable services, e.g., what content type to use
> (application/json? nothing?), what to status code to return for a
> Notification Request (204 or 200?), etc. Is there such a specification (or
> "sub-specification", not sure how that would be classified)?
There is an old JSON-RPC-over-HTTP-draft (http://www.simple-is-better.org/json-rpc/jsonrpc20-over-http.html),
but I think this draft has some flaws, especially the mapping of errors
(and so mixing JSON-RPC and HTTP) is very ugly in my opinion.
I would suggest (and would write a JSON-RPC-over-HTTP-spec), to:
- simply tunnel JSON-RPC through HTTP POST
- Content-Type: I would suggest "application/json" or maybe "application/json-rpc".
Opinions?
- Status-Codes:
- 200 if HTTP succeeded
(Both for JSON-RPC-Responses and JSON-RPC-Errors;
There should be *no* error-mapping between JSON-RPC-Errors and HTTP errors!)
- 200 or 204 as answer to JSON-RPC Notifications.
- HTTP-error-code if HTTP failed
- HTTP GET is discouraged.
If *really* needed, we may specify that a JSON-RPC-RPC request may be
encoded in a URL in URL-Encoded form.
> 2. I see that some (many?) implementations of 2.0 don't implement every
> piece, the batch is the typical one left out. It's unclear to me whether
> the Batch capability is a requirement of the server.
I'm currently not sure. I thought that it normally should be implemented,
but may be omitted in special circumstances, in the sense of "SHOULD"
of RFC 2116 (http://www.ietf.org/rfc/rfc2119.txt). But I'm not sure if
that was still the case when JSON-RPC 2.0 was released.
> I ask because I'd rather not implement Batch (because it
> requires me to detect whether I'm getting a JSON block, or an array of JSON
> blocks, which isn't trivial).
Really? Since you have to de-serialize the JSON-string, it should be
easy to detect if it is an JSON-array or JSON-object.
> 3. With the Batch call, any JSON objects in the array that have errors will
> be represented in the response with an Error Object. It'd be nice if, for
> Batch at least, the Error Object included the Request Object contents, if
> there's no ID for the Request, or the ID if one exists.
The Error-Object always contains the id of the Request (as long as the
id could be extracted from the Request), and the id should be used to
match Responses to the Requests. So, if you want to match Errors to
Requests, you should use appropriate ids.
Adding the Request to the Error-Object may be an option, but this is
implementation-dependent, and I'm not sure if we should recommend this
behaviour in the spec.
Roland
> 1. It seems that most (all?) implementations of JSON-RPC 2.0 are assuming
> HTTP as the transport
I'm using JSON-RPC 2.0 over TCP and Unix Domain Sockets, and my Python-
implementation will soon support multiple JSON-RPC-requests per connection
and JSON-RPC over netstrings.
AFAIK, Kamailio also uses JSON-RPC over netstrings.
> there are requirements of HTTP that would be useful to specify in order to
> properly create inter-operable services, e.g., what content type to use
> (application/json? nothing?), what to status code to return for a
> Notification Request (204 or 200?), etc. Is there such a specification (or
> "sub-specification", not sure how that would be classified)?
but I think this draft has some flaws, especially the mapping of errors
(and so mixing JSON-RPC and HTTP) is very ugly in my opinion.
I would suggest (and would write a JSON-RPC-over-HTTP-spec), to:
- simply tunnel JSON-RPC through HTTP POST
- Content-Type: I would suggest "application/json" or maybe "application/json-rpc".
Opinions?
- Status-Codes:
- 200 if HTTP succeeded
(Both for JSON-RPC-Responses and JSON-RPC-Errors;
There should be *no* error-mapping between JSON-RPC-Errors and HTTP errors!)
- 200 or 204 as answer to JSON-RPC Notifications.
- HTTP-error-code if HTTP failed
- HTTP GET is discouraged.
If *really* needed, we may specify that a JSON-RPC-RPC request may be
encoded in a URL in URL-Encoded form.
> 2. I see that some (many?) implementations of 2.0 don't implement every
> piece, the batch is the typical one left out. It's unclear to me whether
> the Batch capability is a requirement of the server.
but may be omitted in special circumstances, in the sense of "SHOULD"
of RFC 2116 (http://www.ietf.org/rfc/rfc2119.txt). But I'm not sure if
that was still the case when JSON-RPC 2.0 was released.
> I ask because I'd rather not implement Batch (because it
> requires me to detect whether I'm getting a JSON block, or an array of JSON
> blocks, which isn't trivial).
easy to detect if it is an JSON-array or JSON-object.
> 3. With the Batch call, any JSON objects in the array that have errors will
> be represented in the response with an Error Object. It'd be nice if, for
> Batch at least, the Error Object included the Request Object contents, if
> there's no ID for the Request, or the ID if one exists.
id could be extracted from the Request), and the id should be used to
match Responses to the Requests. So, if you want to match Errors to
Requests, you should use appropriate ids.
Adding the Request to the Error-Object may be an option, but this is
implementation-dependent, and I'm not sure if we should recommend this
behaviour in the spec.
Roland
Yitzchak Scott-Thoennes
Mar 18, 2013, 5:00:48 PM3/18/13
to json...@googlegroups.com, Ted M. Young
On Mon, 2013-03-18 at 11:24 -0700, Ted M. Young wrote:
> Is there such a specification (or "sub-specification", not sure how
> that would be classified)?
"extension". There was such a specification for JSON-RPC 1.2:
> Is there such a specification (or "sub-specification", not sure how
> that would be classified)?
http://jsonrpc.org/historical/json-rpc-over-http.html
It provides useful guidance for what to do with status codes, content
type, etc.
I've heard its provision for GET requests was greatly vilified, though.
Someone updating it to the extent necessary for 2.0 and submitting it
here for comments would be Good Thing.
Ted M. Young
Mar 20, 2013, 12:37:19 PM3/20/13
to json...@googlegroups.com
> 1. It seems that most (all?) implementations of JSON-RPC 2.0 are assuming
> HTTP as the transport
no, they don't, and JSON-RPC 2.0 explicitely does not depend on HTTP.
I'm using JSON-RPC 2.0 over TCP and Unix Domain Sockets, and my Python-
implementation will soon support multiple JSON-RPC-requests per connection
and JSON-RPC over netstrings.
AFAIK, Kamailio also uses JSON-RPC over netstrings.
Perhaps I was premature, many of the Java libraries I looked at used HTTP as the transport. It's good to see other transports being used.
There is an old JSON-RPC-over-HTTP-draft (http://www.simple-is-better.org/json-rpc/jsonrpc20-over-http.html),
but I think this draft has some flaws, especially the mapping of errors
(and so mixing JSON-RPC and HTTP) is very ugly in my opinion.
I see in another recent thread that someone (you?) posted http://www.simple-is-better.org/json-rpc/extension_transport.html so I'll take a look at that.
> 2. I see that some (many?) implementations of 2.0 don't implement every
> piece, the batch is the typical one left out. It's unclear to me whether
> the Batch capability is a requirement of the server.
I'm currently not sure. I thought that it normally should be implemented,
but may be omitted in special circumstances, in the sense of "SHOULD"
of RFC 2116 (http://www.ietf.org/rfc/rfc2119.txt).
Yeah, I didn't see a good explanation why those implementations didn't support Batch, i.e., they didn't support a "SHOULD" (recommended) feature and didn't reveal any "valid reasons in particular circumstances to ignore" it (quoting from rfc2119).
> I ask because I'd rather not implement Batch (because it
> requires me to detect whether I'm getting a JSON block, or an array of JSON
> blocks, which isn't trivial).
Really? Since you have to de-serialize the JSON-string, it should be
easy to detect if it is an JSON-array or JSON-object.
Yes, really. I have to know up-front whether I'm expecting a JSON Structure Object, in which case it gets decoded into a Map, or if it's a JSON Array, in which case it becomes an array (or List) of Maps. It's not complex, but it's a decision that my code has to make that I can't leave up to the JSON processor.
Adding the Request to the Error-Object may be an option, but this is
implementation-dependent, and I'm not sure if we should recommend this
behaviour in the spec.
I was particularly referring to the case where there was no ID in the Request, so the Response couldn't provide an ID to track the error back to the source. It seems that quoting back the erroneous Request only in the case where there's no ID found would be useful. I will likely do this in my implementation when/if I implement Batch calls.
;ted
Roland Koebler
Mar 20, 2013, 1:28:02 PM3/20/13
to json...@googlegroups.com
Hi,
> Yes, really. I have to know up-front whether I'm expecting a JSON Structure
> Object, in which case it gets decoded into a Map, or if it's a JSON Array,
> in which case it becomes an array (or List) of Maps. It's not complex, but
> it's a decision that my code has to make that I can't leave up to the JSON
> processor.
oh, ok. Then maybe you could use a workaround: Look at the 1st
non-whitespace character in the string -- if it is "[", then it's
an Array, if it is "{", it's an Object. Or try to decode an Object,
and if this fails, try to decode an Array.
And if you don't implement Batches: please clearly document it.
> I was particularly referring to the case where there was no ID in the
> Request,
No ID, or no unique ID?
> so the Response couldn't provide an ID to track the error back to
> the source. It seems that quoting back the erroneous Request only in the
> case where there's no ID found would be useful.
Maybe, but probably only for logfiles.
If you use Requests (which always contain an id), matching is much easier
if you use Batch-unique ids (which should be really simple).
If you use Notifications (=no id), then you only get error-respones on
Parse Error/Invalid Request -- and this means that either something bad
happened to the data on the network (and matching doesn't really help)
or your JSON-RPC-client-implementation is buggy and should be fixed.
But I'll think about it, since I want to improve the error-messages in
my JSON-RPC implementation.
Roland
> Yes, really. I have to know up-front whether I'm expecting a JSON Structure
> Object, in which case it gets decoded into a Map, or if it's a JSON Array,
> in which case it becomes an array (or List) of Maps. It's not complex, but
> it's a decision that my code has to make that I can't leave up to the JSON
> processor.
non-whitespace character in the string -- if it is "[", then it's
an Array, if it is "{", it's an Object. Or try to decode an Object,
and if this fails, try to decode an Array.
And if you don't implement Batches: please clearly document it.
> I was particularly referring to the case where there was no ID in the
> Request,
> so the Response couldn't provide an ID to track the error back to
> the source. It seems that quoting back the erroneous Request only in the
> case where there's no ID found would be useful.
If you use Requests (which always contain an id), matching is much easier
if you use Batch-unique ids (which should be really simple).
If you use Notifications (=no id), then you only get error-respones on
Parse Error/Invalid Request -- and this means that either something bad
happened to the data on the network (and matching doesn't really help)
or your JSON-RPC-client-implementation is buggy and should be fixed.
But I'll think about it, since I want to improve the error-messages in
my JSON-RPC implementation.
Roland
Mo
Mar 20, 2013, 2:08:49 PM3/20/13
to json...@googlegroups.com
On Monday, March 18, 2013 2:24:47 PM UTC-4, Ted M. Young wrote:
Hi,
I'm implementing a prototype of JSON-RPC for Gosu and have a few questions.
1. It seems that most (all?) implementations of JSON-RPC 2.0 are assuming HTTP as the transport (I haven't looked at every implementation, so I'd be interested in hearing about those that use a different transport), however, there are requirements of HTTP that would be useful to specify in order to properly create inter-operable services, e.g., what content type to use (application/json? nothing?), what to status code to return for a Notification Request (204 or 200?), etc. Is there such a specification (or "sub-specification", not sure how that would be classified)? We could then implement some automated tests ("compatibility kit"-like) using a basic web page.
With the Ruby JSON-RPC library https://github.com/movitto/rjr you can invoke json-rpc over tcp, amqp, websockets, http, and more.
Ted M. Young [@jitterted]
Mar 20, 2013, 2:34:49 PM3/20/13
to json...@googlegroups.com
>oh, ok. Then maybe you could use a workaround: Look at the 1st
>non-whitespace character in the string -- if it is "[", then it's
>an Array, if it is "{", it's an Object.
>non-whitespace character in the string -- if it is "[", then it's
>an Array, if it is "{", it's an Object.
Yeah, that's the way I plan on handling it.
>> I was particularly referring to the case where there was no ID in the
>> Request,>No ID, or no unique ID?
No ID at all. This would most likely be due to a bug on the caller side, which is why I'd like to provide more information to help the user debug their code. If there's an ID provided, then I wouldn't provide the additional information because the ID . Though if a non-unique ID is provided, I might also want to provide the additional information.
Another option is to add a "debug" parameter into the JSON payload, but I'm not sure that's a good idea.
;ted
Roland
--
You received this message because you are subscribed to a topic in the Google Groups "JSON-RPC" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/json-rpc/DxjXzwNHgeg/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to json-rpc+u...@googlegroups.com.
To post to this group, send email to json...@googlegroups.com.
Visit this group at http://groups.google.com/group/json-rpc?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Roland Koebler
Mar 20, 2013, 3:04:30 PM3/20/13
to json...@googlegroups.com
Hi,
> No ID at all.
Erm, than it's a notification, so the only errors you get are
"Parse Error" and (maybe) "Invalid Request".
> This would most likely be due to a bug on the caller side,
> which is why I'd like to provide more information to help the user debug
> their code.
So, what you really need is a JSON-RPC syntax checker, right?
The best way to debug it is to:
- manually look at the created JSON-string and compare it with the
specification,
- run the Request through a JSON decoder (to detect JSON syntax errors),
- check if the JSON-Object contains the required members.
- And maybe create/use a special, non-productive, simple JSON-RPC-server
implementation for debugging with extensive error-logs (=which
problem occurred when and why on which position of the Request).
I don't think it's reasonable to add functionality to normal
JSON-RPC-servers to debug JSON-RPC-client-implementations.
Especially if the client is broken, it doesn't really help to
know which Request is wrong, since probably all are wrong.
Roland
> No ID at all.
Erm, than it's a notification, so the only errors you get are
"Parse Error" and (maybe) "Invalid Request".
> This would most likely be due to a bug on the caller side,
> which is why I'd like to provide more information to help the user debug
> their code.
The best way to debug it is to:
- manually look at the created JSON-string and compare it with the
specification,
- run the Request through a JSON decoder (to detect JSON syntax errors),
- check if the JSON-Object contains the required members.
- And maybe create/use a special, non-productive, simple JSON-RPC-server
implementation for debugging with extensive error-logs (=which
problem occurred when and why on which position of the Request).
I don't think it's reasonable to add functionality to normal
JSON-RPC-servers to debug JSON-RPC-client-implementations.
Especially if the client is broken, it doesn't really help to
know which Request is wrong, since probably all are wrong.
Roland
Reply all
Reply to author
Forward
0 new messages