Where can I find documentation for the real time API for Firestore?

1,052 views
Skip to first unread message

Samuel Elgozi

unread,
Dec 3, 2019, 3:16:31 PM12/3/19
to Firebase Google Group
Hi everyone,
About a month ago I wrote a light-weight "low level" SDK for Firestore. I got into the point in which only one feature is missing, which is real-time updates.
The API used by the SDK's is not documented anywhere, until this point I've used the public REST API for all other features, but the real-time updates aren't part of it.
I guess I can reverse engineer it as I did with my Storage and Auth SDKs, but this one seems harder.

So can anyone here explain to me how it works?
I do know that the current JS SDK uses polling, but I couldn't figure out what are the right arguments and the protocol.
If there is another way(web socket) used by a different SDK that would be preferable(I only support modern browsers anyways)

Side note:  If you were wondering why I rebuilt the library then its because I needed my web app to be very fast and performant, so by rewriting it I managed to have all features with a bundle of about 5KB~ for Firestore,
Storage is about 1.5KB~ and Auth is at about 1KB~ but not fully featured. And performance-wise it's in orders of magnitude faster and more memory efficient than the official ones.

Thanks in advance!

Hiranya Jayathilaka

unread,
Dec 3, 2019, 9:44:57 PM12/3/19
to fireba...@googlegroups.com
JS SDK uses gRPC streaming to receive real time updates. Here's the respective proto: https://github.com/firebase/firebase-js-sdk/blob/master/packages/firestore/src/protos/google/firestore/v1/firestore.proto#L148

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/dafa936d-676b-4761-96f4-9fdab8f47922%40googlegroups.com.


--

Hiranya Jayathilaka | Software Engineer | h...@google.com | 650-203-0128

Samuel Elgozi

unread,
Dec 4, 2019, 10:41:00 AM12/4/19
to Firebase Google Group
I've been trying to figure out how to map the gRPC request to HTTP for a few hours now, but it doesn't seem to work with this specific endpoint.
I looked at these resources for help:

I've used the latter "guide" to map other endpoints and it worked, but the "listen" endpoint doesn't seem to be mapped following these rules.
I also looked into the requests made by the official SDK, and it seems to follow a different protocol.
The requests made by the official SDK looks like this:
:authority: firestore.googleapis.com
:method: POST
:path: /google.firestore.v1.Firestore/Listen/channel?database={database}&VER=8&RID={rid}&X-HTTP-Session-Id=gsessionid&%24httpHeaders=X-Goog-Api-Client%3Agl-js%2F%20fire%2F7.5.0%0D%0A&zx=w2nvzzg08rob&t=1

[Parsed form data]
count: 1
ofs: 0
req0___data__: {
  "database":"projects/{projectId}/databases/(default)",
    "addTarget":{
    "documents":
      {
        "documents": ["projects/{projectId}/databases/(default)/documents/public/AKoaGL7w6GxCCF97IVIP"]
      },
    "targetId":2
  }
}


And the response body looks like:
51
[[0,["c","lyAOqbbp6su3UYBQD2dvQg","",8,12,30000]]]

I have no idea what are all the query params in the request, and how to use the response.
I know that after this request the SDK starts doing polling with some data from the response, but no idea how it works.

My attempt following Hiranya Jayathilaka response looks like this:
POST https://firestore.googleapis.com/v1/projects/{projectID}/databases/(default)/documents:listen

[json body]
{
"addTarget": {
"documents": [
"projects/sandbox-6b679/databases/(default)/documents/public/AKoaGL7w6GxCCF97IVIP"
]
}
}

But the response I get is:
[
  {
    "error": {
      "code": 400,
      "message": "Invalid value (Object), ",
      "status": "INVALID_ARGUMENT",
      "details": [
        {
          "fieldViolations": [
            {
              "description": "Invalid value (Object), "
            }
          ]
        }
      ]
    }
  }
]

I have two assumptions. Either the "listen" API wasn't mapped to REST yet and thats the official SDK is using the "other method", or im just doing it wrong.

Can someone give me a hand, please?
Thanks!


On Wednesday, 4 December 2019 04:44:57 UTC+2, Hiranya Jayathilaka wrote:
JS SDK uses gRPC streaming to receive real time updates. Here's the respective proto: https://github.com/firebase/firebase-js-sdk/blob/master/packages/firestore/src/protos/google/firestore/v1/firestore.proto#L148

On Tue, Dec 3, 2019 at 12:16 PM Samuel Elgozi <samu....@gmail.com> wrote:
Hi everyone,
About a month ago I wrote a light-weight "low level" SDK for Firestore. I got into the point in which only one feature is missing, which is real-time updates.
The API used by the SDK's is not documented anywhere, until this point I've used the public REST API for all other features, but the real-time updates aren't part of it.
I guess I can reverse engineer it as I did with my Storage and Auth SDKs, but this one seems harder.

So can anyone here explain to me how it works?
I do know that the current JS SDK uses polling, but I couldn't figure out what are the right arguments and the protocol.
If there is another way(web socket) used by a different SDK that would be preferable(I only support modern browsers anyways)

Side note:  If you were wondering why I rebuilt the library then its because I needed my web app to be very fast and performant, so by rewriting it I managed to have all features with a bundle of about 5KB~ for Firestore,
Storage is about 1.5KB~ and Auth is at about 1KB~ but not fully featured. And performance-wise it's in orders of magnitude faster and more memory efficient than the official ones.

Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireba...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages