Guide: 5. Structuring data

[Cem Turan]

This is regarding the "extra" index to track Mary's membership mentioned in: https://www.firebase.com/docs/android/guide/structuring-data.html#section-indices
While I do understand the benefits of this approach, it's quite unclear on how one should update of the index?
As far as I can understand it's not possible to link values/references so one would have to update the value both places.
Is there any recommended way of doing this, such as a transaction/atomic update or does the Firebase client deal with this?

The jsfiddle example only updates the membership status in one of the two places as far as I can tell. Is this on purpose or is this example incomplete/misleading with regards to the above?

[Frank van Puffelen]

Hey Cem,

I wrote the most common approaches up on StackOverflow a while ago. 

http://stackoverflow.com/questions/30693785/how-to-write-denormalized-data-in-firebase/30699277#30699277

Say we have a chat application, where we store two entities: messages and users. In the screen where we show the messages, we also show the name of the user. So to minimize the number of reads, we store the name of the user with each chat message too.

users

  so:209103

    name: "Frank van Puffelen"

    location: "San Francisco, CA"

    questionCount: 12

  so:3648524

    name: "legolandbridge"

    location: "London, Prague, Barcelona"

    questionCount: 4

messages

  -Jabhsay3487

    message: "How to write denormalized data in Firebase"

    user: so:3648524

    username: "legolandbridge"

  -Jabhsay3591

    message: "Great question."

    user: so:209103

    username: "Frank van Puffelen"

  -Jabhsay3595

    message: "I know of three approaches, which I'll list below."

    user: so:209103

    username: "Frank van Puffelen"

So we store the primary copy of the user's profile in the users node. In the message we store the uid (so:209103 and so:3648524) so that we can look up the user. But we also store the user's name in the messages, so that we don't have to look this up for each user when we want to display a list of messages.

 

So now what happens when I go to the Profile page on the chat service and change my name from "Frank van Puffelen" to just "puf".

 

Transactional update

Performing a transactional update is the one that probably pops to mind of most developers initially. We always want the username in messages to match the name in the corresponding profile.

 

So when the user change's the name in their profile:

var ref = new Firebase('https://mychat.firebaseio.com/');

var uid = "so:209103";

var nameInProfileRef = ref.child('users').child(uid).child('name');

nameInProfileRef.transaction(function(currentName) {

  return "puf";

}, function(error, committed, snapshot) {

  if (error) { 
    console

    console.log('Transaction failed abnormally!', error);

  } else if (!committed) {

    console.log('Transaction aborted by our code.');

  } else {

    console.log('Name updated in profile, now update it in the messages');

    var query = ref.child('messages').orderByChild('user').equalTo(uid);

    query.on('child_added', function(messageSnapshot) {

      messageSnapshot.ref().update({ username: "puf" });

    });

  }

  console.log("Wilma's data: ", snapshot.val());

}, false /* don't apply the change locally */);

Pretty involved and the astute reader will notice that I cheat in the handling of the messages. First cheat is that I never call off for the listener, but I also don't use a transaction.

 

If we want to securely do this type of operation from the client, we'd need:

• security rules that ensure the names in both places match. But the rules need to allow enough flexibility for them to temporarily be different while we're changing the name. So this turns into a pretty painful two-phase commit scheme.

1. 1. change all username fields for messages by so:209103 to null (some magic value)
   2. change the name of user so:209103 to 'puf'
   3. change the username in every message by so:209103 that is null to puf.
   4. that query requires an and of two conditions, which Firebase queries don't support. So we'll end up with an extra property uid_plus_name (with value so:209103_puf) that we can query on.

• client-side code that handles all these transitions transactionally.

This type of approach makes my head hurt. And usually that means that I'm doing something wrong. But even if it's the right approach, with a head that hurts I'm way more likely to make coding mistakes. So I prefer to look for a simpler solution.

 

Eventual consistency

The second approach depends on splitting the user action ("I want to change my name to 'puf'") from the implications of that action ("We need to update the name in profile so:209103 and in every message that has user = so:209103).

 

I'd handle the rename in a script that we run on a server. The main method would be something like this:

function renameUser(ref, uid, name) {

  ref.child('users').child(uid).update({ name: name });

  var query = ref.child('messages').orderByChild('user').equalTo(uid);

  query.once('value', function(snapshot) {

    snapshot.forEach(function(messageSnapshot) {

      messageSnapshot.update({ username: name });

    })

  });

}

Once again I take a few shortcuts here, such as using once('value' (which is in general a bad idea for optimal performance with Firebase). But overall the approach is simpler, at the cost of not having all data completely updated at the same time. But eventually the messages will all be updated to match the new value.

 

Not caring

The third approach is the simplest of all: in many cases you don't really have to update the duplicated data at all. In the example we've used here, you could say that each message recorded the name as I used it at that time. I didn't change my name until just now, so it makes sense that older messages show the name I used at that time. This applies in many cases where the secondary data is transactional in nature. It doesn't apply everywhere of course, but where it applies "not caring" is the simplest approach of all.

 

Summary

While the above are just broad descriptions of how you could solve this problem and they are definitely not complete, I find that each time I need to fan out duplicate data it comes back to one of these basic approaches.

[Cem Turan]

Hi Frank,

Thanks, that clarified a few points, but I still believe the guide could be updated to take this into account or at least the jsfiddle example.
As I'm dealing with a scenario/schema similar to the membership example, the option of "not caring" is ruled out.
Eventual consistency seems like the way to go. You mention running this on the server and I believe the reason is that we are dealing with 1-N relationships? If I were to update the status of a membership in only two places would the recommended way of doing so be through two subsequent updates?

Also, something like JSON graph seems like a good way to solve the above mentioned problems: https://www.youtube.com/watch?v=2xX5JTHWw4Q&index=3&list=PL-7Rk5Igg3dfdxlNKNSMJHfK_yG5ceiZf
Is that something you have on the roadmap?

[Kato Richardson]

Cem,

Thanks for the feedback! This is an amazing community and people like you who take the time to contribute are a huge part of what makes it so awesome.  I'll put this on the list for discussion and see what we can do here.

Have a great day,

Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/9270aa53-b77a-4334-95ef-17eb1da3adb3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.