Scaling a social feed with Firebase

[Selwyn Jacob]

Hello,

I’m developing a social app with Firebase as a backend. I found the Firefeed example to be very beneficial as my app somewhat follows a similar structure. One aspect that I’m really concerned is that when a user X makes a new post, its link is saved into the ‘feed’ list of every user that X follows. In other words, if X follows 1000 users, then 1000 writes have to be made every time X makes a new post. I understand that this approach is fantastic for reading, as we need to look in just one place, but it becomes a very write intensive task especially when the user follows a large number of people. Is there a better approach to scale this? May be a different schema? 

Thanks,

Selwyn

[Nick Halper]

Hi Selwyn,

I suspect that you can do 1000 writes very quickly in Firebase, especially if the data posted is small.

If you have a large amount of data, then I would instead push a reference to that data instead.

I really don't think it will be much of an issue until you get past the 100,000's of users, and at that point you can always branch off a script or two that completes the pushes in a queue. For a twitter-like app it's probably not a big deal for users with 100,000 of follows to have a delay of a few seconds or a minute or two until all their followers are notified.

Nick

[Selwyn Jacob]

Hello Nick,

Thank you very much for the clarification. On a similar note, is it advisable to do this on the client side (say an Android app) like this?

// Add spark ID to the feed of everyone following this user.
  currentUser.child("followers").once("value", function(list) {
    list.forEach(function(follower) {
      var childRef = firebase.child("users").child(follower.name());
      childRef.child("feed").child(sparkRefId).set(true);
    });
  });

Would every 'childRef.child("feed").child(sparkRefId).set(true);' in the loop make a network call to my Firebase instance? In that case it might not be responsive on the client side. The other alternative that I could think of is to writing a REST endpoint or an Android service that would handle Firebase interactions, and the client would make just one call to that endpoint when a post is made. Any thoughts on this?

Thanks,

Selwyn

[Michael Lehenbauer]

It's worth noting that the Firebase SDK will maintain a persistent connection to Firebase and all of the writes will be pipelined over that connection, so you won't incur 1000 round-trips and it should be reasonably fast to do the writes.

That said, you do have the risk that the android app is closed halfway through writing all of the feed entries, so if you want a more robust solution, having a backend process may be a good option.

-Michael

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/7e9268c7-0f55-4e38-9e54-ff28c224d956%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Selwyn Jacob]

Hello Michael,

That makes a lot of sense now! thank you very much :)