JSON-RPC batching out of order

[Sean Colsen]

Question for Brent and Pavish,

I happened to notice that our RPC batching code on the front end does not appear to conform to the JSON-RPC spec. So I wanted to check with you two about this.

To be clear: I’ve not observed any bugs — my concern is only speculative. I’m just a tad worried there could be an intermittent or latent bug lurking which would be incredibly difficult to troubleshoot if it did crop up. We don’t actually use RPC batching very much in Mathesar. So it does seem conceivable to me that a bug could potentially exist here.

The JSON-RPC spec says:

The Response objects being returned from a batch call MAY be returned in any order within the Array. The Client SHOULD match contexts between the set of Request objects and the resulting set of Response objects based on the id member within each Object.

But in looking here (and at the surrounding code), it seems to me that we only match RPC responses with their requests based on their index. We do assign a unique id to each request. To me it seems that (in order to conform with the spec) we need to match up each response with its request using those id values.

Pavish implemented RPC batching 10 months ago as part of DB page - role configuration, collaborators, roles. Since I was away (with a big family move) at the time, I never actually gave that PR more than a cursory skim.

Pavish, I’m wondering if you had a reason for skipping this id-based matching. Or maybe I’m missing something else.

I also wouldn’t be surprised if the backend is (intentionally or incidentally) enforcing a consistent order of RPC responses to their corresponding requests.

So Brent, I’m curious if you might know anything about this behavior on the backend. If we can safely assume that the backend consistently orders its batch responses to align with the batch requests, then we might choose to essentially ignore this for now.

​

[Brent Moran]

It seems we're okay for the moment. We're using django-modern-rpc, and you can see the code that processes batch requests here. It seems the processing is done serially, in the order in the submitted JSON array, and the results are joined together in the same order. So, we're lucky.

However, that's an implementation detail which I don't think we should count on in the long run, since it's not guaranteed by the spec, and it's out of our control (i.e., the order in the response is completely determined by a 3rd party library).

Brent Moran

Technical Strategy Lead, Mathesar

[Pavish Kumar Ramani Gopal]

> I’m wondering if you had a reason for skipping this id-based matching. Or maybe I’m missing something else.

It looks like I missed this check while implementing batching. Thanks for catching this, it would have been a pain to troubleshoot if we had hit bugs because of this.

Though the backend currently sends it in order, this fix for this was relatively small and I decided to raise a PR fixing it, rather than discussing if we need to work on it. Here's the PR, ready for review: https://github.com/mathesar-foundation/mathesar/pull/4583