Google APIs and JSON-RPC

719 views
Skip to first unread message

Louis Ryan

unread,
Sep 2, 2010, 11:19:36 PM9/2/10
to json...@googlegroups.com
Folks,

As announced at Google I/O were working on launching JSON-RPC support for most of our newer APIs. We've had success with using JSON-RPC for the client with OpenSocial so we are broadening our support for it. In particular JSON-RPC has proven useful as a dispatch mechanism for it's very obvious batch support and naming and parsing conventions that work well in Javascript clients. This by no means indicates that we are abandoning RESTful APIs or that we design first as RPC, quite the opposite in fact. It's simply that for newer APIs we support there is no reason to limit ourselves and our clients to a single transport mechanism for all use cases. 

There are a few areas where we are just non-compliant and have some questions about the value of what the spec is requiring. Let me know what you think as we'd like to launch fairly soon and we're very interested in the communities feedback.

Cheers
Louis

JSON-RPC version

The specification sections 4 & 5 mandates that the value
“jsonrpc” : “2.0”
be provided at the top-level of every request and response. In our case all our APIs are JSON-RPC 2.0 or above so filling this in is wasteful so while we’re happy to accept it on input, and even reject requests that specify a value other than “2.0” we don't reject requests that omit this value and we do not populate it in the output.

Absence of ‘id’ to indicate that no response should be produced

The specification section 4 defines any request that does not contain an ‘id’ to be a ‘notification’. A notification is a request the response to which the caller does not care about and so the system should not produce a response object. In the case of a batch of requests this would mean that the cardinality of the batch response may not necessarily match that of the request. Our current behavior is to always produce a response even if an id is omitted primarily because even if something is dispatching an asynchronous job there is still the need for parameter validation which should be confirmed to the caller. Does anyone make significant use of the behavior described in the spec of perceive any performance advantage from it?

Error code alignment

We produce codes per the specification for:
-32700 - Invalid JSON
-32600 - Invalid request object

however we vary from the spec by default for the following codes:
-32601 - Method not found. Instead we use the HTTP code 404 NOT FOUND
-32602 - Invalid parameters. Instead we use the HTTP code 400 BAD REQUEST
-32603 - Internal error. Instead we use the HTTP code 500 INTERNAL SERVER ERROR for failures in the Apiary stack and the spec code for failures caused by backend errors.

This variance is maintained primarily for compatibility with the OpenSocial JSON-RPC implementation in Apache Shindig which followed the same pattern. I think the error codification in the spec is one of its weaker areas about which there is good discussion on one of the other threads here. We use the 'data' field inside an error response to carry an array of more detailed error objects as our system allows for APIs to report secondary errors.

- location and name of invalid parameters
- textual namespaced error codes (see the other thread for discussion)
- debug information and identifiers for request tracing
- help links to reference documentation if requested

apiVersion:

The specification doesn’t provide a mechanism for describing different versions on the same API. We added support for this by defining an ‘apiVersion’ field at the top level of the request which contains a string literal of the version name. E.g
{
 id : ‘12345’,
 method : “language.diacritize.get”,
 apiVersion : ‘v2’
...
}
We do not currently echo back the apiVersion in the response and it’s worth discussing whether we should do so particularly when no apiVersion is specified in the request and the server chooses one by default.


Max Kanat-Alexander

unread,
Sep 2, 2010, 11:39:29 PM9/2/10
to json...@googlegroups.com
On 09/02/2010 08:19 PM, Louis Ryan wrote:
> The specification sections 4 & 5 mandates that the value
> “jsonrpc” : “2.0”
> be provided at the top-level of every request and response. [snip]

I agree that this is wasteful and probably unnecessary in the response.

I have a server system that supports 1.0, 1.1, and 2.0, so if this
field is in requests, it's the simplest way for my system to understand
what sort of response the client wants.

If a client supports multiple versions, having it in the response helps
the client understand how it should parse the response (particularly for
errors, although the JSON-RPC specs *can* all be compatible with each
other in error formats, theoretically).

> In our case
> all our APIs are JSON-RPC 2.0 or above so filling this in is wasteful so
> while we’re happy to accept it on input, and even reject requests that
> specify a value other than “2.0” we don't reject requests that omit this
> value and we do not populate it in the output.

I think that more or less, it's safe to ignore it. In my current server
implementation, I just assume that a request is 2.0 if it doesn't
specify otherwise, as well.

> The specification section 4 defines any request that does not contain an
> ‘id’ to be a ‘notification’.

I agree that this is useless and unfortunate--I wrote to the list about
it a few months back, also.

> however we vary from the spec by default for the following codes:
> -32601 - Method not found. Instead we use the HTTP code 404 NOT FOUND
> -32602 - Invalid parameters. Instead we use the HTTP code 400 BAD REQUEST
> -32603 - Internal error. Instead we use the HTTP code 500 INTERNAL
> SERVER ERROR for failures in the Apiary stack and the spec code for
> failures caused by backend errors.

I think that's pretty much in line with the JSON-RPC Over HTTP spec, no?

> We do not currently echo back the apiVersion in the response and it’s
> worth discussing whether we should do so particularly when no apiVersion
> is specified in the request and the server chooses one by default.

Mmm, since your APIs are centralized, I don't think anybody's going to
be implementing a client that attempts to support more than one API
version at once, right? If there are identical methods across different
APIs that would possibly get updated separately, though, there might be
a value in specifying the apiVersion in the response.

-Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.

Gaetano Giunta

unread,
Sep 3, 2010, 5:36:22 AM9/3/10
to json...@googlegroups.com
Louis Ryan wrote:
Folks,

As announced at Google I/O were working on launching JSON-RPC support for most of our newer APIs. We've had success with using JSON-RPC for the client with OpenSocial so we are broadening our support for it. In particular JSON-RPC has proven useful as a dispatch mechanism for it's very obvious batch support and naming and parsing conventions that work well in Javascript clients. This by no means indicates that we are abandoning RESTful APIs or that we design first as RPC, quite the opposite in fact. It's simply that for newer APIs we support there is no reason to limit ourselves and our clients to a single transport mechanism for all use cases.�

There are a few areas where we are just non-compliant and have some questions about the value of what the spec is requiring. Let me know what you think as we'd like to launch fairly soon and we're very interested in the communities feedback.

Cheers
Louis

JSON-RPC version

The specification sections 4 & 5 mandates that the value
�jsonrpc� : �2.0�
be provided at the top-level of every request and response. In our case all our APIs are JSON-RPC 2.0 or above so filling this in is wasteful so while we�re happy to accept it on input, and even reject requests that specify a value other than �2.0� we don't reject requests that omit this value and we do not populate it in the output.

Absence of �id� to indicate that no response should be produced

The specification section 4 defines any request that does not contain an �id� to be a �notification�. A notification is a request the response to which the caller does not care about and so the system should not produce a response object. In the case of a batch of requests this would mean that the cardinality of the batch response may not necessarily match that of the request. Our current behavior is to always produce a response even if an id is omitted primarily because even if something is dispatching an asynchronous job there is still the need for parameter validation which should be confirmed to the caller. Does anyone make significant use of the behavior described in the spec of perceive any performance advantage from it?

Error code alignment

We produce codes per the specification for:
-32700 - Invalid JSON
-32600 - Invalid request object


however we vary from the spec by default for the following codes:
-32601 - Method not found. Instead we use the HTTP code 404 NOT FOUND
-32602 - Invalid parameters. Instead we use the HTTP code 400 BAD REQUEST
-32603 - Internal error. Instead we use the HTTP code 500 INTERNAL SERVER ERROR for failures in the Apiary stack and the spec code for failures caused by backend errors.

This variance is maintained primarily for compatibility with the OpenSocial JSON-RPC implementation in Apache Shindig which followed the same pattern. I think the error codification in the spec is one of its weaker areas about which there is good discussion on one of the other threads here. We use the 'data' field inside an error response to carry an array of more detailed error objects as our system allows for APIs to report secondary errors.


- location and name of invalid parameters
- textual namespaced error codes (see the other thread for discussion)
- debug information and identifiers for request tracing
- help links to reference documentation if requested


apiVersion:

The specification doesn�t provide a mechanism for describing different versions on the same API. We added support for this by defining an �apiVersion� field at the top level of the request which contains a string literal of the version name. E.g
{
�id : �12345�,
�method : �language.diacritize.get�,
�apiVersion : �v2�
...
}

This change looks the most "hard" to implement in generic jsonrpc clients (or the more specific one).
Could you not simply put that field as 1st (or last) element in the name of the functions themselves, eg: language.diacritize.get.v2 ?

We do not currently echo back the apiVersion in the response and it�s worth discussing whether we should do so particularly when no apiVersion is specified in the request and the server chooses one by default.


--
You received this message because you are subscribed to the Google Groups "JSON-RPC" group.
To post to this group, send email to json...@googlegroups.com.
To unsubscribe from this group, send email to json-rpc+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/json-rpc?hl=en.

Henry Unger

unread,
Sep 3, 2010, 12:46:26 AM9/3/10
to json...@googlegroups.com

With respect to Louis Ryan’s  comments:

 

Absence of ‘id’ to indicate that no response should be produced

 

The specification section 4 defines any request that does not contain an ‘id’ to be a ‘notification’. A notification is a request the response to which the caller does not care about and so the system should not produce a response object. In the case of a batch of requests this would mean that the cardinality of the batch response may not necessarily match that of the request. Our current behavior is to always produce a response even if an id is omitted primarily because even if something is dispatching an asynchronous job there is still the need for parameter validation which should be confirmed to the caller. Does anyone make significant use of the behavior described in the spec of perceive any performance advantage from it?

 

We use such notifications extensively for client/server applications where the server asynchronously notifies the client of state changes of other clients of the server. These state changes are published to all clients. There is no benefit in having the clients respond to the server notifications, and performance would suffer if the server had to field responses from each of the clients, especially when the the server is pushing notifications every second or two to each of a thousand clients.  In addition, in a common carrier mobile environment where the carrier charges for traffic, or traffic in excess of  a monthly limit, producing a response to the notification may double the cost or possibly worse.

 

Best regards,

 

Henry Unger

Hitech Systems, Inc.

http://www.hitech.com

Lorenzo Pastrana

unread,
Sep 3, 2010, 9:56:48 AM9/3/10
to json...@googlegroups.com
On Thu, 2010-09-02 at 20:19 -0700, Louis Ryan wrote:
Folks,


As announced at Google I/O were working on launching JSON-RPC support for most of our newer APIs.
Hi Louis, first of all that's great news :D


There are a few areas where we are just non-compliant and have some questions about the value of what the spec is requiring. Let me know what you think as we'd like to launch fairly soon and we're very interested in the communities feedback.


JSON-RPC version

The specification sections 4 & 5 mandates that the value
“jsonrpc” : “2.0”
be provided at the top-level of every request and response. In our case all our APIs are JSON-RPC 2.0 or above so filling this in is wasteful so while we’re happy to accept it on input, and even reject requests that specify a value other than “2.0” we don't reject requests that omit this value and we do not populate it in the output.
That could sound, well ... conservative in your own case, but obviously implies that 2.0 is your default dialect for now, however there are a couple of points to be signaled :

JSON-RPC 2.0 has (wery few but some) antecedents in 1.0 :

>From the specs, 3 - Compatibility :
JSON-RPC 2.0 does not work with JSON-RPC 1.0 clients or servers.
[snip]it is easy to distinguish between the two versions as 2.0 always has a member named "jsonrpc" with a String value of "2.0" whereas 1.0 does not[snip]

The term "Compatibility" in that paragraph is intended "backward" and that's the only place where it was addressed (in an implicit way thus). Fact is that "the future's not ours to see" ... and neglecting that could turn in an unforeseen bullet heading slowly toward your foot ...

The jsonrpc member requirement is mostly intended for future compatibility, and future compatibility has been a strong concern all along the specs elaboration. Chances are that you'll be blessing that 10 bytes spent someday.

Absence of ‘id’ to indicate that no response should be produced

In the case of a batch of requests this would mean that the cardinality of the batch response may not necessarily match that of the request.

I'd consider the cardinality of a batch being actually N - number_of_notifications


Our current behavior is to always produce a response even if an id is omitted primarily because even if something is dispatching an asynchronous job there is still the need for parameter validation which should be confirmed to the caller.

If you do need any kind of validation / response, why do you omit the request ID in the first place ?
How an asynchronous client could ever dispatch a response without a request ID ?


Does anyone make significant use of the behavior described in the spec of perceive any performance advantage from it?

The main purpose of this kind of feature, as said Henry, is 'state change' notification ... A simple example would be a MMORPG where the client would periodically notify the server about it's player direction and speed so the server can compute collisions, encounters and so on.

When the player takes an action that could fail the client would use a request ID and hope for a response.

Error code alignment


See the thread running ...



apiVersion:

The specification doesn’t provide a mechanism for describing different versions on the same API.

Probably because JSON-RPC is a protocol, not an API :D


We added support for this by defining an ‘apiVersion’ field at the top level of the request which contains a string literal of the version name. E.g
{
 id : ‘12345’,
 method : “language.diacritize.get”,
 apiVersion : ‘v2’
...
}

This sounds to me like an "encapsulation error" but for communications, as if rails where to know if the train carried cargo or people ...

Why not :

{
  id : ‘12345’,
  method : “v2.language.diacritize.get”,
...
}

Or just a specific entry point ?

v2.some.api.server.google.com/call


{
  id : ‘12345’,
  method : “language.diacritize.get”,
...
}

Cheers,

Lorenzo

Dave

unread,
Sep 3, 2010, 4:54:11 PM9/3/10
to JSON-RPC
Glad to see this being adopted broader by Google. Here are my thoughts
on a few of your comments:

-Notifications / id field / batch requests:
I think notifications are very valuable as one-way, low-overhead
requests. It's a unique and cool feature that json-rpc has that most
RPC systems do not. As far as batch requests go, as Lorenzo mentioned,
the # of responses is simply equal to the number of non-notification
requests in the batch. Quite easy to count if you need to match up
that sort of thing. The responses can come back in the batch in any
order, so "ordering" of the requests/responses and gaps from
notifications never enters the picture.

-Error codes:
I think my views on this are clear from the other thread, but I'd be
interested to hear your perspective on allowing error code to be a
string field instead of numeric. Obviously it could still be used with
numeric codes (for HTTP errors for example) but in terms of your API
errors, wouldn't you prefer to be able to give specific error code
responses from each application rather than using a 500 response for
every application error then burying the details in the data field?

-API versions:
I don't really see a need to embed version information at the
individual request level. If you want to have multiple versions of a
method available on the same endpoint, using method names with
versions seems reasonable, however I generally prefer the approach of
giving each version of the API it's own endpoint, which is the
approach used by most web services these days. Given that there is no
concept of "api versions" defined or described in the spec, I think
it's a rather heavyweight concept to add just to allow multiple method
signatures with the same name (which is the net result of your
suggestion).

-Dave


On Sep 2, 11:19 pm, Louis Ryan <lr...@google.com> wrote:
> Folks,
>
> As announced at Google
> I/O<http://code.google.com/events/io/2010/sessions/how-google-builds-apis...>were
> working on launching JSON-RPC support for most of our newer APIs.
> We've
> had success with using JSON-RPC for the client with OpenSocial so we are
> broadening our support for it. In particular JSON-RPC has proven useful as a
> dispatch mechanism for it's very obvious batch support and naming and
> parsing conventions that work well in Javascript clients. This by no means
> indicates that we are abandoning RESTful APIs or that we design first as
> RPC, quite the opposite in fact. It's simply that for newer APIs we support
> there is no reason to limit ourselves and our clients to a single transport
> mechanism for all use cases.
>
> There are a few areas where we are just non-compliant and have some
> questions about the value of what the spec is requiring. Let me know what
> you think as we'd like to launch fairly soon and we're very interested in
> the communities feedback.
>
> Cheers
> Louis
> JSON-RPC versionThe specification sections 4 & 5 mandates that the value
> “jsonrpc” : “2.0”
> be provided at the top-level of every request and response. In our case all
> our APIs are JSON-RPC 2.0 or above so filling this in is wasteful so while
> we’re happy to accept it on input, and even reject requests that specify a
> value other than “2.0” we don't reject requests that omit this value and we
> do not populate it in the output.
> Absence of ‘id’ to indicate that no response should be producedThe
> specification section 4 defines any request that does not contain an ‘id’ to
> be a ‘notification’. A notification is a request the response to which the
> caller does not care about and so the system should not produce a response
> object. In the case of a batch of requests this would mean that the
> cardinality of the batch response may not necessarily match that of the
> request. Our current behavior is to always produce a response even if an id
> is omitted primarily because even if something is dispatching an
> asynchronous job there is still the need for parameter validation which
> should be confirmed to the caller. Does anyone make significant use of the
> behavior described in the spec of perceive any performance advantage from
> it? *
> *Error code alignmentWe produce codes per the specification for:

Max Kanat-Alexander

unread,
Sep 3, 2010, 11:35:34 PM9/3/10
to json...@googlegroups.com
On 09/02/2010 09:46 PM, Henry Unger wrote:
> We use such notifications extensively for client/server applications
> where the server asynchronously notifies the client of state changes of
> other clients of the server.

So, I've been thinking about this, because I'm not opposed to
notifications--I'm simply opposed to the semantics of "no id means this
is a notification".

Instead of this, why don't we include a field "notification: true" that
means "this SHOULD NOT get a response"?

That would preserve notification functionality while allowing the
majority of requests to be simpler by not containing an id.

Derrell Lipman

unread,
Sep 4, 2010, 10:52:23 AM9/4/10
to json...@googlegroups.com
On Fri, Sep 3, 2010 at 23:35, Max Kanat-Alexander <mka...@bugzilla.org> wrote:
       Instead of this, why don't we include a field "notification: true" that
means "this SHOULD NOT get a response"?

       That would preserve notification functionality while allowing the
majority of requests to be simpler by not containing an id.

I have no problem with your proposed "notification: true" field. I think that makes it more explicit, and would make the id field optional for a notification. I am interested, though, in whether you meant that you'd like to not send an id in ("the majority of") requests that do require a response.  An id allows the responses to multiple async requests to be associated with their request when the response is received. If no id is sent with a request that requires a response, only one could be sent at a time or there would be no way to correlate the responses to their requests.

Derrell



Henry Unger

unread,
Sep 4, 2010, 2:07:15 PM9/4/10
to json...@googlegroups.com

Given that responses may be asynchronous and unordered, I agree that having the id field should be mandatory when a response is expected, otherwise it would be impossible for a client to correlate a response with its request at the protocol level. From a conceptual basis, I have been taking advantage of this type of behavior (presence of id means response, absence means notification) for almost 25 years in public safety applications.

 

Henry Unger

Max Kanat-Alexander

unread,
Sep 4, 2010, 4:50:40 PM9/4/10
to json...@googlegroups.com
On 09/04/2010 07:52 AM, Derrell Lipman wrote:
> An id allows the responses to multiple async requests to be
> associated with their request when the response is received. If no id is
> sent with a request that requires a response, only one could be sent at
> a time or there would be no way to correlate the responses to their
> requests.

That's true, but:

(a) I've never needed the id. For example, I'm often using the YUI
DataSource module, which keeps track of requests separately. In fact,
any good async framework keeps track of requests and responses in some
way of its own.

and

(b) It's essentially putting an implementation detail into a protocol spec.

So what I'm saying is that I'd like it to simply be optional.

Derrell Lipman

unread,
Sep 4, 2010, 5:27:54 PM9/4/10
to json...@googlegroups.com
On Sat, Sep 4, 2010 at 16:50, Max Kanat-Alexander <mka...@bugzilla.org> wrote:
On 09/04/2010 07:52 AM, Derrell Lipman wrote:
> An id allows the responses to multiple async requests to be
> associated with their request when the response is received. If no id is
> sent with a request that requires a response, only one could be sent at
> a time or there would be no way to correlate the responses to their
> requests.

       That's true, but:

       (a) I've never needed the id. For example, I'm often using the YUI
DataSource module, which keeps track of requests separately. In fact,
any good async framework keeps track of requests and responses in some
way of its own.

       and

       (b) It's essentially putting an implementation detail into a protocol spec.

Hmmm... I'm not convinced. There needs to be some way to associate a particular response with the request which is being responded to. Certainly various servers could implement their own way of doing this, but that would mean that there could never be a consistent method of issuing remote procedure calls that would work regardless of which server provided the remote procedures. The client side (or whomever is the issuer of the request) would need to be customized for each server it wants to make requests of.

I believe that having the id field as part of every request and response (with the possible exception of the special type of request called a  notification) adds to the standardization that allows clients to be used with multiple servers.

Derrell

Max Kanat-Alexander

unread,
Sep 5, 2010, 12:00:51 AM9/5/10
to json...@googlegroups.com
On 09/04/2010 02:27 PM, Derrell Lipman wrote:
> Hmmm... I'm not convinced. There needs to be some way to associate a
> particular response with the request which is being responded to.

Yeah, I agree, and it should be optional. If I personally don't need a
way to associate requests with responses, then why force me to include
the id?

> The client side (or whomever is
> the issuer of the request) would need to be customized for each server
> it wants to make requests of.

Yes, if there wasn't an id field at all in the spec, that would be
true. But there is an id field, I just want it to be optional.

> I believe that having the id field as part of every request and response
> (with the possible exception of the special type of request called a
> notification) adds to the standardization that allows clients to be
> used with multiple servers.

Notifications would also work with a "notification: true" field.

Burak Yiğit KAYA

unread,
Sep 5, 2010, 3:30:12 AM9/5/10
to json...@googlegroups.com
Then the common ground might be "the client SHOULD provide an 'id' value IF it knows that the server will return the request results asynchronously or in other channel".

The point here is, if you are making an XHR then both server and client already know that your request's answer should in that very same session and unless you are making a batch request, using an id here is meaningless. Though if you are using or will use the new "WebSockets" API you still should provide an id which makes me think that the only use case of omitting the id seems this XHR thing and adding a relatively long name like "notification" just for this case not sounds plausible to me.

Sean Kinsey

unread,
Sep 5, 2010, 5:13:53 PM9/5/10
to JSON-RPC
It seems that most people are starting to forget that this is an
implementation agnostic protocol, both when it comes to the endpoint
implementations and the transport.

If you want to evolve the spec further then you need to weight the pro
and the cons of every suggestion carefully.
'Use id to signalize a notification'
Pro: No overhead
Con: Not clear for readers, uses inference. No possible
feedback on invalid calls (you cannot come with 'but I use [insert
name of framework] and it handles this for me). Notification cannot be
logged easily.

'Always use an id, use a flag to signalize a notification'
Pro: Enables feedback on invalid calls. Notifications can be
logged.
Con: A little overhead for most scenarios.

Anyone want to add to the list?

Sean

Max Kanat-Alexander

unread,
Sep 5, 2010, 6:28:36 PM9/5/10
to json...@googlegroups.com
On 09/05/2010 12:30 AM, Burak Yiğit KAYA wrote:
> Then the common ground might be "the client SHOULD provide an 'id' value
> IF it knows that the server will return the request results
> asynchronously or in other channel".

That sounds good.

> The point here is, if you are making an XHR then both server and client
> already know that your request's answer should in that very same session
> and unless you are making a batch request, using an id here is
> meaningless. Though if you are using or will use the new "WebSockets"
> API you still should provide an id which makes me think that the only
> use case of omitting the id seems this XHR thing and adding a relatively
> long name like "notification" just for this case not sounds plausible to me.

There could be lots of scenarios, though. I think that when designing a
protocol like this, it's definitely important to have reference
implementations and an idea of what the real-world applications would
be, but it's also important not to design it explicitly for a list of
valid situations, because there's always going to be situations you
can't predict. (See also:
http://www.codesimplicity.com/post/designing-too-far-into-the-future/ )

So, instead, it's best to design it to be as flexible as reasonably
possible. Making "notification" an explicit element leaves much more
flexibility in the protocol than re-using the "id" field for that purpose.

I think that a shorter name than "notification" would be good, though,
if anybody has some ideas.

Josh Marshall

unread,
Sep 5, 2010, 7:13:21 PM9/5/10
to json...@googlegroups.com
Rather obvious idea: 'notify': true, or 'notify': 0 / 1 if we're just going for 'truish' statements and /really/ want to minimize size. (I prefer the strict true / false though.) Although I'm not strongly against the 'id' presence check, I think this would make implementations a bit clearer . I do think required 'id' key should be required if 'notification' / 'notify' / whatever is used -- the spec states out of order requests as fundamental, and I think that's perfect because it writes the possibility of asynchronous / threaded / distributed / whatever processing right in the spec. Sure, a matching client and server library can interpolate these appropriately, but in my opinion this is the most cross-implementation-friendly manner to do it. (But I might be missing something.)

Josh Marshall


Max Kanat-Alexander

unread,
Sep 5, 2010, 7:19:17 PM9/5/10
to json...@googlegroups.com
On 09/05/2010 04:13 PM, Josh Marshall wrote:
> Rather obvious idea: 'notify': true, or 'notify': 0 / 1 if we're just
> going for 'truish' statements and /really/ want to minimize size.

I thought about that, too, but that effectively makes the "notify"
field mandatory if you want a reply, which adds an extra unnecessary
item in every call and also would break backwards-compatibility with 2.0.

Josh Marshall

unread,
Sep 5, 2010, 7:25:53 PM9/5/10
to json...@googlegroups.com
I was thinking 'notify':true means it's a notification, and no response is necessary from the server, (so 'notify' is actually unnecessary UNLESS it's a notification, sort of the inverse of the current 'id' field) but maybe that's unintuitive because it is a verb. I use that as a method in my python jsonrpclib library, so if it's not intuitive I'd love to know that so I can change it. :)

Josh Marshall


Omi

unread,
Sep 5, 2010, 7:35:58 PM9/5/10
to JSON-RPC
I thought we had settle this part of the spec.
A notification is similar to a call to:

void myFunction(....)

while a normal call is similar to a method call expecting a result.
To allow the client to understand which request response it is
receiving, the "id" field exists. For example in the TCP/IP scenario
where the server supports asynchronous responses, the other of the
responses might be different from the other of the requests. In the
HTTP transport scenario, it might not be of much use especially if
each connection waits for its response rather than piling all of the
responses onto a stack. I've noticed that so far in my use of JSON RPC
2.0, I've not needed to use the id field (on either the client or
server) but I've kept with it partly to maintain compliance and partly
to make sure it doesn't become a boulder smacking me in the face when
it becomes needed.
> > json-rpc+u...@googlegroups.com<json-rpc%2Bunsubscribe@googlegroups.c om>
> > .

Max Kanat-Alexander

unread,
Sep 5, 2010, 8:51:44 PM9/5/10
to json...@googlegroups.com
On 09/05/2010 04:25 PM, Josh Marshall wrote:
> I was thinking 'notify':true means it's a notification, and no response
> is necessary from the server,

Ahhh, okay. Yeah, I misunderstood and thought that "notify: true" meant
"send me a response". :-)

> but maybe that's unintuitive because it is a verb.

Yeah, there might be some better term to use....

Matt (MPCM)

unread,
Sep 5, 2010, 9:13:49 PM9/5/10
to JSON-RPC
Hey all,

This list seems to have exploded with lots of discussion all of a
sudden, which is exciting. But a lot of the discussion had immediately
jumped into what changes should be brought to a implied future 2.1
version without much discussion of if the changes make sense.

The way we have dealt with proposed changes in the past is to have
them defined in an extension specification (json-rpc+mycustom_blah).
The goal is to keep the core spec very simple. Even to the extent that
we do not directly even address transport issues.

Changes to the spec are a long process by intension, usually after
being defined via an extension specification. The managers of this
group usually take an informal vote as we propose changes, and have a
chance to be vocal over time about the proposed changes. Getting to
2.0 took a long time to keep it simple and extract a lot of implied
meaning that appeared in a 1.1 version. Prior to google groups we were
on yahoo groups, some of us for quite a few years dealing with these
concepts at this point. Lots of us rely on this is production
environments day in and day out.

So, with all that said... discussions about what belongs in the next
version of the specification are probably premature. Discussion about
extension specs which may be moved into a future version are not
however. Well discussed and defined specifications about anything that
can be leveraged by json-rpc (and other specs) are beneficial to all.
Using json-schema as a SMD mechanism is a good example.

I'm at home tonight, so I'll try and catch up and post some specific
points about some of the recent posts. The recent interest is very
exciting, just stick around for the discussions and be prepared to
justify your thoughts and approaches, and please keep the thoughts
separate from the core spec, as one or more people are likely to
always object to direct changes to the spec, but extension spec gives
room to prove your points while letting people use and try out the
concepts immediately.

--
Matt (MPCM)

Matt (MPCM)

unread,
Sep 5, 2010, 9:27:00 PM9/5/10
to JSON-RPC
id's are required to signify the need for a response object being sent
back. The lack there of signifies the client not needing a response
object. Changing this should not be done lightly and without really
good reason. This is about the client stating its intentions.

A client can not reply on one request object per transport instance as
a fact. The client could easily be sending request objects that
outlive the immediate transport instance and thus that scope
separation might not out survive your request objects existence. Ajax-
type one-off requests make this hard to see, but the id is vital to
the client sorting out its many messages. It has little to do with the
server, and probably never should.

--
Matt (MPCM)

Burak Yiğit KAYA

unread,
Sep 6, 2010, 2:40:43 AM9/6/10
to json...@googlegroups.com
On Mon, Sep 6, 2010 at 04:27, Matt (MPCM) <ma...@mpcm.com> wrote:
Ajax-
type one-off requests make this hard to see, but the id is vital to
the client sorting out its many messages.

Actually they don't since AJAX requests are also asynchronous and have to be ordered. The thing here is the HTTP protocol. Since it only allows a single shot, you can sort the requests and responses according to the transport objects(XHR object in this case). Since I'm not aware of any other "one shot" protocol which is commonly used, I cannot strongly support making the id field optional though I believe that this only case should be taken into account since it will be a major usage area of JSON-RPC.

Burak Yiğit "BYK" Kaya

Max Kanat-Alexander

unread,
Sep 6, 2010, 3:15:15 AM9/6/10
to json...@googlegroups.com
On 09/05/2010 06:27 PM, Matt (MPCM) wrote:
> the id is vital to the client sorting out its many messages.

True in some situations, but not in all situations.

> It has little to do with the server, and probably never should.

Okay. In that case, the protocol should also account for servers that
don't need it. I can tell you that thus far in all of the situations in
which I have used JSON-RPC, I have never needed the "id" field in a
conceptual or factual way.

Lorenzo Pastrana

unread,
Sep 6, 2010, 3:47:11 AM9/6/10
to json...@googlegroups.com

Hi Matt,

I just stumbled over the questioning explosion of this WE ... thanks for the cooling ;D

On Sun, 2010-09-05 at 18:27 -0700, Matt (MPCM) wrote:
id's are required to signify the need for a response object being sent
back. The lack there of signifies the client not needing a response
object. Changing this should not be done lightly and without really
good reason. This is about the client stating its intentions.
There's a simple question I asked in my first reply to Louis but nobody seemed to answer instead of speculating on notification mechanism ...

Given the current "full call" / notification mechanism, I'd like anyone trying to question that point to answer this question :

[question]
    Why would someone needing a response to a call would ever fire a notification in the first place ?
[/question]

This is just plain misconception of the actual pattern.

If you need a response, ack, or logging, whatever ... well provide an ID and forget about notifications, that's as simple as that, even if your call could be interpreted as a simple "bang". I could rephrase : providing an ID is not a big deal as everybody seems to agree, so just do it if you do need feedback.

(As there seem to be lots of new posters / listeners I'd like to re-state that as usual my intention is not being harsh despite some people could think so)

Cheers,
Lo.

Max Kanat-Alexander

unread,
Sep 6, 2010, 6:45:30 AM9/6/10
to json...@googlegroups.com
On 09/06/2010 12:47 AM, Lorenzo Pastrana wrote:
> providing an ID is not a big deal as everybody seems to agree, so just
> do it if you do need feedback.

It's a very significant and unnecessary complexity in an HTTP GET
situation. Generally, imagine telling people that your API looks like this:

rpc?method=Hello.world&jsonrpc=2.0&id=1

versus telling people that your API looks like this:

rpc?method=Hello.world

In fact, #2 can be implemented as a simple REST call, like:

rpc/Hello/world

Whereas #1 cannot be (without significant additional and unnecessary
complexity).

This is just one example.

Lorenzo Pastrana

unread,
Sep 6, 2010, 9:46:28 AM9/6/10
to json...@googlegroups.com
On Mon, 2010-09-06 at 03:45 -0700, Max Kanat-Alexander wrote:
> providing an ID is not a big deal as everybody seems to agree, so just
> do it if you do need feedback.

	It's a very significant and unnecessary complexity in an HTTP GET
situation.
Ah, http ?... mmmh, GET !?

Nothing related to one parameter or another then, HTTP GET won't support much more than hello world situations without "very significant and unnecessary complexity", for obvious reasons, the number one being "Where is JASON ?" :P

The "very significant and unnecessary complexity" is actually using HTTP GET for RPC ! Please use POST for any RPCs more complex than hello worlds :

Staying in HTTP domain, let's talk about a rather elementary json-rpc call

{
    "jsonrpc": "2.0",
    "method": "products.getinfo",
    "params": [
        123,
        234,
        345,
        456,
        567
    ],
    "id": 987654321
}
Feed it to any endpoint :

Content-Type: application/json-rpc
Content-Length: ...

paste the json blob and ... VOILA !
Try doing the same by GET :

3.5.1   Encoded Parameters

Encoding Steps:

   1. Base 64 param value
   2. URL Encoded Base 64 param value

Pre-Encoded Params:

http://endpoint?jsonrpc=2.0&method=products.getinfo&params=[123,234,345,456,567]&id=987654321

Encoded Request:

http://endpoint?jsonrpc=2.0&method=products.getinfo&params=WzEyMywyMzQsMzQ1LDQ1Niw1Njdd&id=987654321

See http://groups.google.com/group/json-rpc/web/json-rpc-over-http for more ... I'd like these specs explicitely and strongly discourages the use of GET endpoints (but that's another topic)

Shouldn't we refocus on properly partitioned matters ?

1 - json rpc - cristal clear, rock solid, transport agnostic core
    let's not forget the 2 main concerns :
     a) JSON
     b) RPC
2 - transport dependent considerations
3 - extensions

My 2.5 cents.

Lo.

Yitzchak Scott-Thoennes

unread,
Sep 6, 2010, 1:08:02 PM9/6/10
to json...@googlegroups.com
On Sun, 2010-09-05 at 18:27 -0700, Matt (MPCM) wrote:
> id's are required to signify the need for a response object being sent
> back. The lack there of signifies the client not needing a response
> object. Changing this should not be done lightly and without really
> good reason. This is about the client stating its intentions.

The fundamental "code smell" here is that id has two distinct purposes:

1) the id to pass back in the response, to be available to the client
to associate responses with requests.

2) indicates a non-notification by being present

Any time you dual-purpose a field this way, you are likely to end up
with boundary conditions where the dual-purposing doesn't make sense.
I believe this is the underlying reason people are getting so het up
over this seemingly trivial detail.

A far more sensible arrangement would be a separate "notification"
parameter (defaulting to false if not present).

This would then allow "id" to be optional, and to be omitted when the
client doesn't need it.

I'd be happy to write this up as an extension to the spec, but I'm
not really sure whether such a rule change would actually be welcome
to those writing json-rpc libraries without an actual version bump
to json-rpc itself.

Max Kanat-Alexander

unread,
Sep 6, 2010, 5:19:51 PM9/6/10
to json...@googlegroups.com
On 09/06/2010 06:46 AM, Lorenzo Pastrana wrote:
> Ah, http ?... mmmh, GET !?

Yeah. Did you read the part at the bottom where I said that it was only
one example?

> 1. Base 64 param value

That's unnecessary and I've discussed how unnecessary it is in another
thread, and we all seemed to agree.

In response to the rest of your post, and as a completely unrelated
side note to this thread, HTTP GET support is very useful and
considerably simplifies things for many consumers. It's also an
excellent foundation to build a REST-based JSON-RPC service on.

Robert Goldman

unread,
Sep 6, 2010, 5:23:25 PM9/6/10
to json...@googlegroups.com, Max Kanat-Alexander
On 9/4/10 Sep 4 -3:50 PM, Max Kanat-Alexander wrote:
> On 09/04/2010 07:52 AM, Derrell Lipman wrote:
>> An id allows the responses to multiple async requests to be
>> associated with their request when the response is received. If no id is
>> sent with a request that requires a response, only one could be sent at
>> a time or there would be no way to correlate the responses to their
>> requests.
>
> That's true, but:
>
> (a) I've never needed the id. For example, I'm often using the YUI
> DataSource module, which keeps track of requests separately. In fact,
> any good async framework keeps track of requests and responses in some
> way of its own.

Can you clarify this? It doesn't agree with my understanding. My
understanding is that JSON-RPC /is/ the async framework, and the id /is/
the way it keeps track of requests and responses.

I have worked on both a Common Lisp JSON-RPC library and a
Firefox-specific one, run over sockets. Both use the id field to manage
the response and request tracking, and neither has any alternative means
of tracking them.

The whole beauty of JSON-RPC, from my POV, is that it provides a dead
simple asynchrony framework, and the way it does it is by using the id
as its mechanism for tracking.

I would not like to see this made optional for requests that require
responses.

best,
r

Max Kanat-Alexander

unread,
Sep 6, 2010, 5:31:04 PM9/6/10
to json...@googlegroups.com
On 09/06/2010 02:23 PM, Robert Goldman wrote:
> Can you clarify this? It doesn't agree with my understanding. My
> understanding is that JSON-RPC /is/ the async framework, and the id /is/
> the way it keeps track of requests and responses.

JSON-RPC is a protocol, not a transport. Many transports have a method
of associating requests with responses on their own.

I personally have never needed to use the "id" field, and my
implementations will return responses whether you specify an "id" field
or not, at least partially in the spirit of "be liberal in what you accept".

> I would not like to see this made optional for requests that require
> responses.

Why do you require an id field for requests that require responses in
your implementation? Why would the "notification: true" system not work
for you?

Josh Marshall

unread,
Sep 6, 2010, 6:05:58 PM9/6/10
to json...@googlegroups.com
On Mon, Sep 6, 2010 at 4:31 PM, Max Kanat-Alexander <mka...@bugzilla.org> wrote:
>
>        I personally have never needed to use the "id" field, and my
> implementations will return responses whether you specify an "id" field
> or not, at least partially in the spirit of "be liberal in what you accept".
>

I'm assuming you aren't talking about batch requests here. Based on
the spec, it is completely permissible to request:

[
{"jsonrpc": "2.0", "method": "sum", "params": [2,4], "id": "1"},
{"jsonrpc": "2.0", "method": "email", "params": ['subject',
'message'], "id": "2"},
{"jsonrpc": "2.0", "method": "random", "id": "3"}
]

and receive:

[
{"jsonrpc": "2.0", "result": true, "id": "2"},
{"jsonrpc": "2.0", "result": 6, "id": "1"},
{"jsonrpc": "2.0", "result": 0.3577654, "id": "3"}
]

Without ids, that would look like:

[
{"jsonrpc": "2.0", "result": true},
{"jsonrpc": "2.0", "result": 6},
{"jsonrpc": "2.0", "result": 0.3577654}
]

I'm not sure how you would implement that without ids. (But I might be
missing something.) Now, obviously for a single request -> single
response in a shared-nothing environment, ids are not /as/ necessary,
but I still think they are intuitive and should be required for
optimum compatibility between implementations. Also, this keeps the
differences between single request and batch requests to a minimum,
which just makes the spec that much easier for people to pick up.
That's just my interpretation of the conversation, so apologies if
I've overlooked something.

Now notifications, those are a different argument (although there is
potentially impact on the id absence / requirement question.)

Josh Marshall

Max Kanat-Alexander

unread,
Sep 6, 2010, 6:35:13 PM9/6/10
to json...@googlegroups.com
On 09/06/2010 03:05 PM, Josh Marshall wrote:
> I'm assuming you aren't talking about batch requests here.

I'm not. I'm talking about single requests.

> I'm not sure how you would implement that without ids.

I agree--if you're going to do batch requests, you should probably add
ids, although I think that that should be up to the implementor--a
"SHOULD", not a "MUST".

Matt (MPCM)

unread,
Sep 6, 2010, 7:15:08 PM9/6/10