Designing a chat application with Firebase database

[Gary Huang]

I was hoping I could get some feedback on firebase practices and how it would be used to design a chat application where a user would have to listen to many chats simultaneously. 

For each chat, the client will need to listen for updates regarding new messages, chat detail changes, chat deleted, and new users being added or removed from chat.

The basic structure could be something like

GroupChat -list of chats made by everyone

ChatMessages -the messages for each chat where each entry's key is the group chat id and the entry holds a children of messages

ChatUsers -the users belonging to each chat where each entry's key is the group chat id and the entry holds a children of the users

In the Firebase documentation, I saw they only showed examples listening to one post or chat. With one chat, I'd have 4 listeners where I'd listen to GroupChat for children and value changes, then listen to ChatMessages, and ChatUsers  by the GroupChat key. Now, if I had 20 group chats, I'd have to grab all the keys of my group chats and attach all these listeners per chat. It'd be around 20 * 4 = 80 listeners. Plus I'd have to keep track of all the listeners and manage them. Is this the proper way to use Firebase to handle listening to so many simultaneous chats? Is Firebase made to handle all these listeners per client? Or should I be looking at restructuring the data to limit the listeners?

As an alternative design, I could structure Firebase to be used as a queuing system where each user has their own individual message queue in the database. They would listen to any children changes to their queue. As new messages or events are pushed to their queue, the user can process each update individually. Only one listener is needed. The downside would be for every message in a chat, the message has to be written to all the queues of the chat users, increasing writing and creating redundant data.  Old queue messages would ideally be cleared constantly as well. Is this heavy writing something firebase suitable for?

Which one would be better design and why? Or are there some better solutions than what I have come up with? 

[Kato Richardson]

Hi Gary,

We built a complete code lab that walks you through best practices for creating a chat app. We also have a sophisticated plug-and-play chat app called Firechat. Between the two, you should be able to derive some great ideas on data structure and approach.

Not sure what the use case is for listening to 20 group chats at once. That would probably make me a bit schizophrenic. :) But more seriously, you'll find the bytes sent eclipses the number of listeners. It's fine to listen on 20 paths if you have a legitimate use for that data, and if the byte count is reasonably small. By far your best optimization is to limit the downloaded data to only things the user will really read and use, and to reduce the payload of each message.

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/2fc230cf-45c8-4e08-8f8f-988e774457b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Gary Huang]

Hi Kato,

Thanks for the details. The use case for listening 20 chats at once would be how What's app works, where you can be looking at one chat and still receive messages and notifications for the other chats you are a part of. Does that seem less schizophrenic?  Good to hear li it's the data transferred that bottlenecks things and not the number of listeners.

[Kato Richardson]

Okay, so if you're only interested in new messages, you can reduce the scope considerably using queries. For example, I only need to listen on background chats for messages that come in since the last time I viewed it.

Alternately, if we're thinking WhatsApp or Twitter scope (which is more streams than chat rooms), fanning out new messages to each follower's individual feeds would be better. Then you simply monitor those. In either case, a much smaller byte count than all messages in the feed/room.

Great stuff. Let me know if you have any other questions not covered in the code labs and Firechat app.

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/dd8abdd0-8bd2-491e-be83-398821c1d58c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gary Huang]

Interesting, so the feeds would be more similar to the second alternative design I mentioned. But could you explain why it would be less byte count to do it this way? It seems using individual feeds would have at least equal or more data transferred. Since in either design, the feeds or multiple listeners, the users still receive the same amount of messages. However for feeds, we have to pass in a lot more duplicate data to patch each users individual queue

On Friday, September 16, 2016 at 9:07:59 AM UTC-7, Kato Richardson wrote:

Okay, so if you're only interested in new messages, you can reduce the scope considerably using queries. For example, I only need to listen on background chats for messages that come in since the last time I viewed it.

Alternately, if we're thinking WhatsApp or Twitter scope (which is more streams than chat rooms), fanning out new messages to each follower's individual feeds would be better. Then you simply monitor those. In either case, a much smaller byte count than all messages in the feed/room.

Great stuff. Let me know if you have any other questions not covered in the code labs and Firechat app.

☼, Kato

On Fri, Sep 16, 2016 at 12:51 AM, Gary Huang <gkh...@gmail.com> wrote:

Hi Kato,

Thanks for the details. The use case for listening 20 chats at once would be how What's app works, where you can be looking at one chat and still receive messages and notifications for the other chats you are a part of. Does that seem less schizophrenic?  Good to hear li it's the data transferred that bottlenecks things and not the number of listeners.

On Thursday, September 15, 2016 at 8:30:12 AM UTC-7, Gary Huang wrote:

I was hoping I could get some feedback on firebase practices and how it would be used to design a chat application where a user would have to listen to many chats simultaneously. 

For each chat, the client will need to listen for updates regarding new messages, chat detail changes, chat deleted, and new users being added or removed from chat.

The basic structure could be something like

GroupChat -list of chats made by everyone

ChatMessages -the messages for each chat where each entry's key is the group chat id and the entry holds a children of messages

ChatUsers -the users belonging to each chat where each entry's key is the group chat id and the entry holds a children of the users

In the Firebase documentation, I saw they only showed examples listening to one post or chat. With one chat, I'd have 4 listeners where I'd listen to GroupChat for children and value changes, then listen to ChatMessages, and ChatUsers  by the GroupChat key. Now, if I had 20 group chats, I'd have to grab all the keys of my group chats and attach all these listeners per chat. It'd be around 20 * 4 = 80 listeners. Plus I'd have to keep track of all the listeners and manage them. Is this the proper way to use Firebase to handle listening to so many simultaneous chats? Is Firebase made to handle all these listeners per client? Or should I be looking at restructuring the data to limit the listeners?

As an alternative design, I could structure Firebase to be used as a queuing system where each user has their own individual message queue in the database. They would listen to any children changes to their queue. As new messages or events are pushed to their queue, the user can process each update individually. Only one listener is needed. The downside would be for every message in a chat, the message has to be written to all the queues of the chat users, increasing writing and creating redundant data.  Old queue messages would ideally be cleared constantly as well. Is this heavy writing something firebase suitable for?

Which one would be better design and why? Or are there some better solutions than what I have come up with? 

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/dd8abdd0-8bd2-491e-be83-398821c1d58c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Kato Richardson]

Hi Gary,

To some extent, we're entering the abstract now, and we'd need to sit down with a real data model, some debug logging output, and do a bit of serious math.  But to continue talking in generalities, the primary difference is in how much historical data you're fetching. If you're only interested in new entries, you can actually store those separately, reducing the amount of data being accessed and downloaded accordingly.  Note that storing the entire message or just a list of message ids referencing a message are both perfectly valid and have tradeoffs (space vs coding complexity and a slight bandwidth margin).

You can probably achieve similar results by using queries based on timestamps, but keep in mind that in the current iteration of the Firebase database, the server still warms all the data (or at least a significantly larger chunk) into memory to do indexing. For example, if you have a million chat messages stored in `messages/all` and want to get the five newest ones, the responses can be significantly faster if we store the new messages separately (e.g. `/messages/new` and `/messages/viewed`), rather than querying `/messages/all` based on timestamp. We could (and probably will) change this in the future to include even better server-side optimizations.

☼, Kato

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/1df40a16-34e2-4ec2-9b16-0ef35bc287aa%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.