Pros & cons of copying data in on() vs heavy use of once()?

[David Hart]

When I first started using Firebase, it seemed most straightforward to use on('child_added'...) and in the callback, grab the data and stuff it somewhere. Then at some point it became clear I could use once('value'...) callbacks to always traverse Firebase's local copy, and never copy any data out of on() callbacks. On() callbacks in this scenario could serve only as event notification, not data delivery. Ever since starting to think this way, I've been waffling on whether to rely completely on once() for reading memory, or not.

I'm curious if there are opinions, or better yet accrued wisdom & evidence, for or against sticking to using once() for re-reading data from memory, versus copying the data locally to a separate structure in memory, and what the pros and cons are - specifically if there might be better practice here or large advantages or disadvantages that I'm not considering.

In my case, I want to be able to re-play a lot of data, and very often, and I want it to re-play very quickly, on the same order as if I were reading my own local copy, and without stalling for any reason. Is it reasonable to ask this much of once(), assuming the ref has already been synced at least once? Here are the main issues that I can see with using once() like this. Am I right, and are there others?

Pros of using once() for re-reading local data:

- Completely avoids the problem of trying to keep a local copy and remote copy in sync.

- Less code, easier development.

- Firebase has a copy of the data. If I copy it in on() callbacks, I have 2 copies, and I'm wasting a bunch of memory.

Cons of using once() for re-reading local data:

- Somewhat binds the code to the Firebase schema. Could make it harder to incorporate local storage, or to use or migrate to or from other remote databases.

- I'm not sure about the performance of once(). Its relatively fast, but is getting the value of a snapshot a lightweight o(1) wrapper around returning something already sitting in memory? Or is it doing a deep-copy or something else o(n) that also duplicates memory and I have to pay for the copying every time I read memory, instead of only once if I copy in an on() callback? And in either case, is once() going to stay at least that fast, or could it in the future result in additional transfer of data, e.g., from data that was pushed out of a local cache?

- Getting the data I need using once() can be a little taxing. Its slightly heavier to code every time I need to access something. And since the callback is async, it sometimes means a larger refactoring of an algorithm to get all the code in one place (inside the callback). Reading from multiple refs simultaneously means nested once() callbacks. Incidentally, I have been imagining a helper function, at least in javascript, that does un-currying of nested once() callbacks, e.g., a single callback that gets all the snapshot values from a list of refs. Has anyone done something like that?

--

David.

[Michael Lehenbauer]

Hey David,

This is a good question and starts to touch on some of the internal performance characteristics of Firebase.

The #1 biggest thing to be aware of is that Firebase's current caching behavior is to only keep data cached that's actively being observed with a .on() event listener.

That means that this will be nice and fast:

var currentSnapshot = null;

ref.on('value', function(snapshot) {

  currentSnapshot = snapshot;

});

setInterval(function() {
  console.log(currentSnapshot.val();
}, 1000);

But this will be slower and use much more bandwidth:

setInterval(function() {

  ref.once('value', function(snapshot) {
    console.log(snapshot.val();

  });
}, 1000);

However, you can very easily "fix" this by just adding a no-op .on() observer, like:

ref.on('value', function(snapshot) { });

After that, any .once() calls will be fast, since we'll have the data cached.  So if you're going to be accessing data frequently, you probably want to make sure that you have an outstanding .on() event listener on that data at all times.

So now, assuming you've done that, which of the two code snippets are better? Manually caching the snapshot in a .on() callback, or repeatedly calling .once() when you need a snapshot?

The answer is that they're really equivalent in terms of performance.  There's no extra work required to give you a copy of our cache when you call .once(), so it's really just a matter of your code structure and what fits better for you.  I think using .once() is slightly cleaner in general, but either approach is completely fine.  A couple more relevant details:

• Storing a reference to a snapshot is very cheap.
  Firebase internally uses immutable snapshots for its cache and the snapshots that it gives to you.  So when you do "ref.on('value', function(s) { currentSnapshot = s; }" you are basically just storing a reference to a snapshot of our internal cache.  It's very cheap, and there's very little extra memory overhead.
  
  
• Calling .val() is somewhat expensive.
  .val() is an O(n) operation that crawls the snapshot and creates an equivalent JavaScript object.  In addition to the O(n) algorithmic work, you're now storing two copies of the data (our internal snapshot and the copy you just created with .val()), so there's extra memory usage.
  
  
• Corollary: snapshot.val().foo.bar is more expensive than snapshot.child('foo').child'(bar').val()
  Because .val() has to do an O(n) copy of the whole snapshot, it's better to call it at deeper levels if possible.

So this means if you're worried about perf you shouldn't hesitate to store copies of snapshots and/or use .once() aggressively (assuming you have an outstanding .on() as well), but you should make some effort to reduce calls to .val() and/or call .val() on deeper nodes rather than higher up.

Hopefully this helps at a high-level.  I know I didn't touch on all of your specific points, but I'm hoping this sheds some light.  Holler if there's anything unclear or a specific question you'd still like answered.

Thanks,

-Michael

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[noah]

A couple follow up questions (w/ v3.x syntax).

1) If a different reference is used, do we still get the caching? For example:

var ref = firebase.database().ref('posts/');

ref.on('value', function(snapshot) { });

var ref2 = firebase.database().ref('posts/');
ref2.once('value', function(snapshot) {
  console.log(snapshot.val());
});

2) If 1 is true & we retrieve data from a deeper level, but still under the cached location, will it also be retrieved from the cache? For example will the data in the 2nd call below be pulled from cached data:

var ref = firebase.database().ref('posts/');

ref.on('value', function(snapshot) { });

var ref2 = firebase.database().ref('posts/first-post');
ref2.once('value', function(snapshot) {
  console.log(snapshot.val());
});

Thanks!

[Kato Richardson]

Hi Noah, both are true. The local cache is synced independently of notifying the individual listeners, and if you listen on a higher path in the same tree, then the local cache already exists, so no server request is needed.

--

You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/10be8837-c039-4717-af2d-e436de09a4b5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[noah]

Awesome, thanks Kato!