Does Firebase always guarantee added events in order?

[Morgan Jung]

Hello. 

I am developing messenger IOS app  based on Firebase Realtime Database.

I want that all messages to be ordered based on timestamp. 

There is a scenario as like below.

------------------------------------------------------------------------------------------------------------------------------------

There are 3 clients. A, B and C.

1) All clients register <figure-1> listener to receive messages from others.

<figure-1>

ref.queryOrdered(byChild: "timestamp").queryStarting(atValue: startTime).observe(.childAdded, with:
{
    ....
    // do work for the messages, print, save to storage, etc.
    ....

    // save startTime to storage for next open.     
    startTime = max(timeOfSnapshot, startTime)
    saveToStorage(startTime)
}

2) 

Client A write  message 1 to server with ServerValue.timestamp().

Client B write  message 2 to server with ServerValue.timestamp().

Client C write  message 3 to server with ServerValue.timestamp().

Client B has low speed wifi. 

So, finally. Server data saved like <figure-2>

<figure-2>

text : "Message 1",  timestamp : 100000001
text : "Message 3",  timestamp : 100000002
text : "Message 2",  timestamp : 100000003

------------------------------------------------------------------------------------------------------------------------------------

As my listener's code, i keep messages on storage and next listening timestamp for preventing downloading duplicated messages.

In this case.

Does Firebase always guarantee to trigger callback in order as like below?

Message 1

Message 3

Message 2

If it is not guaranteed, my strategy is absolutely wrong.

For example, some client received messages as like below.

Message 2  // the highest timestamp.

// app crash or out of storage

Message 3

Message 1

The client do not have chance to get message 3, 1 anymore.

I have searched stack overflow and google and read official documents many times. However, i could not find the clear answer.

I have almost spent one week for this. Please give me piece of advice.

[Frank van Puffelen]

Also posted on https://stackoverflow.com/questions/53280940/does-firebase-always-guarantee-added-events-in-order

[Kato Richardson]

Hi Morgan,

Firebase does guarantee that for multiple docs like this that they will be ordered by the field you specify. The source of truth for server timestamps would be when the writes reach the server of course (if you think this through in detail, there's probably no other easy and effective way to manage this; there's nothing you could use client side that's foolproof and not subject to drift on the device, so the server is likely the best authority).

So if B write arrives at the server after A write, then C would appear before B by virtue of the fact that it has a later timestamp. That's logical and works out for most cases. There are probably a few gotchas if the client has been offline for a long while and returns at some later date. I can probably think up solutions if you need to handle that extreme edge case (you probably wouldn't).

This works for nearly every case outside of game enviroments (which do a lot of magic for write contention I don't pretend to understand).

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/7d35643c-045e-48f2-8b72-6a12e9f7f011%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Morgan Jung]

Hello Kato.

Thank you for replying.

I understood what is Server Timestamp. 

I just want to know if Firebase is generating events in a sequence that is consistent with the server's data.

2018년 11월 14일 수요일 오전 12시 14분 6초 UTC+9, Kato Richardson 님의 말:

[Aswath Satrasala]

Hello Kato,

The message to firebase will be recieved by various servers hosted by Google.  Is there a mechanism to ensure that all servers will have exactly same time.  I was earlier looking into this kind of issue while working on Google appengine.  There was discussion, that servers time may be little different.

Can you confirm more on this.

Cheers.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CADypTEZD-AMWBqcqc23bOOKY1FrDtna6-Rj3EzjpPpaU8JJP6w%40mail.gmail.com.

[Kato Richardson]

Given that Firebase is using similar internal infrastructure to App Engine, I would expect the behavior here to be the same. This isn't my area of expertise though.

Again, I'm not really sure what sort of app you're building that the times could matter this much. Game developers may have thought up some clever solutions here, but for general app building, I don't think you can't reasonably (maybe not even possibly) overcome internet latencies anyway, so any device will be subject to some minor skew based on connectivity speeds. I'm sure you've thought all this through, but do be sure you're not prematurely optimizing here by solving an extreme edge case that may not matter with premature optimization.

I can't think of any good alternatives. My first temptation was to recommend a Functions endpoint, but those also run on app engine and could have slight variance.

☼, Kato

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAKFF5G4gH9tpPMLMPU0t5_6zJcRGmtnEZUTq7gYm0evCKwXqtw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Alan deLespinasse]

It sounds to me like the original question is: will the 3 clients get their childAdded events in the same order? If so, the answer is no. Client B will receive the callback for its own Message 2 instantly, but will probably not receive the other two for a while because of the slow connection.

[Patryk Lesiewicz]

Hi Aswath,

I'd like to clarify some confusion around "server timestamp" in both Realtime Database and Firestore. It provides much stronger guarantees than just the time when one of many servers received the request. 

firestore.FieldValue.serverTimestamp() is a TrueTime of when the write was committed. It provides external consistency guarantees, which means that "the system behaves as if all transactions were executed sequentially".

The same holds for Realtime Database.

To answer the original question from Morgan more specifically, you will receive local notifications out of order because of "latency compensation". As Alan mentioned, your local writes will emit events instantly, even though the message hasn't reached the server yet, and didn't resolve the server timestamp yet.

Here's a great answer to a similar question by my colleague Gil: https://github.com/firebase/firebase-js-sdk/issues/192#issuecomment-336517226. It talks about issues with "server timestamps" in client SDKs in both firestore and RTDB.

Hope that helps,

Patryk

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/ef80de1c-5bf6-430a-81fa-5aae25c8aca2%40googlegroups.com.