Hoping Someone Can Share Best Practices on Enforcing Referential Integrity in Firebase

[John Tucker]

One of the challenges that I am struggling with in Firebase is that because the client (untrusted) is in control, how can I assure that I maintain referential integrity across changes that require multiple operations, e.g., delete a node and THEN update another node, e.g., updating an index in another node.

Since this is two updates; I cannot see how I can use the validation rules to ensure this.  My current solution is to use the onDisconnect mechanism to back out changes if something were to fail, e.g., 

1) Setup a onDisconnect to restore the index.

2) Delete from index.

3) Setup a onDisconnect to restore the node

3) Delete the node

4) Cancel the onDisconnects across the entire Firebase (would cancel both onDisconnects in one fell swoop).

note: This could get pretty ugly if I had to do a bunch of operations.

Any other tricks?

I some other posts about queues and priv-workers (presumably on my application server); but I am trying to get away from having my server.

Like the queue, I could simply have the client work through my my application server (which then updates Firebase).  Again, I do not want my server.

John

[Kato Richardson]

John,

There are no simple answers to this highly proprietary and vastly complex problem. Ideally, Firebase would provide a solution for this at some point and I'll add a vote for this feature to the discussion list.

What you are presenting here is the proposed solution but not the underlying use case, which may help to identify a best approach. I'll go ahead and offer some nebulous advice on the topic.

In general, the simple options are to combine the data into a single path so that it can be written as a single operation, or to use the update command.

Often, it's possible to identify a single source of truth (SSoT) and then reference this from the other write paths. If that is the case, then you can write to this path first, including some unique update counter, and then as you attempt to write to the other paths, you use that update counter to enforce validation (they only succeed if the master succeeded). If one of the dependent paths falls out of sync, that's usually easily corrected since you would consult the SSoT during reads and verify integrity (and do any cleanup). This is often much simpler than enforcing atomic writes and ultimately more successful.

The last option is update counters and rollbacks, as you've been tinkering with. I experimented with a rollback approach a while back and wrote firebase-multi-write. It has a few edge cases left to resolve, but it may meet your needs, or provide a launching point for your own work.

I hope that helps!

Cheers,

Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/59517d41-baab-4e2e-8c6c-c0db73dfcbe5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Don Barthel]

In the 90s I worked with a multi-user DOS-based accounting system called ACCPAC Plus that had the same issue. They solved it by not worrying too much about it at run time and then provided a separate "Data Integrity Check" (DIC) function that was meant to run periodically (typically nightly). That function would look for anomalies in the data and either just report the errors or report and fix the errors.

For example, an "order" had a single header record and multiple line records (one for each line on the order). The header record contained a line count field. The DIC would count up the lines and fix the line count field in the header if it was wrong. It would also check indexes, which were separate files just like in Firebase) and rebuild those if they were wrong.

What I'm suggesting is that another potential solution is to write your own data integrity check function. It would run within a known account UID and your Firebase rules would give it special privileges to massage the data back to correctness.

[Kato Richardson]

Great point, Don! Thanks for adding that. 

I've seen some similar strategies for detecting complex cheating scenarios in games and voting apps, so this is definitely a viable approach for Firebase.

Cheers,

Kato

--

You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/bbda1048-bae7-4bd0-bc95-13dd29af129a%40googlegroups.com.

[Jacob Wenger]

To follow up on this thread, atomic, multi-location updates have just landed in all of our Firebase clients! This now makes possible some of the things you were trying to do. Namely, you can delete a node (aka set it to null) and updated another node all in one, atomic write. You can read about it here.

Cheers,
Jacob