Firebase scalability?

[Orbyt O]

Hi all. Were looking at using Firebase for its realtime functionality. Now I now Firebase can somehow tie in to App Engine. App Engine has autoscaling, im wondering if Firebase also has autoscaling?

[Kato Richardson]

See this discussion. You may only use Firebase with manual scaling. So you won't be able to take advantage of the autoscale features.

Note that your can probably get a great deal more bang for your buck by inverting the process, and having users write data into Firebase and turning your GAE worker into a privileged consumer. This eliminates the need for auth, security, and a lot of other complexities on your worker. See firebase-queue for some ideas on this strategy.

☼, Kato

On Tue, Jan 19, 2016 at 3:57 PM, Orbyt O <pbfre...@gmail.com> wrote:

Hi all. Were looking at using Firebase for its realtime functionality. Now I now Firebase can somehow tie in to App Engine. App Engine has autoscaling, im wondering if Firebase also has autoscaling?

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/4a4b4595-d5d3-470c-93f2-049721ec7159%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Orbyt O]

Thanks for the info. So App Engine aside, how many simultaneous devices can Firebase push realtime updates to?

[Ibo Roma]

I am new to your API, so pardon my ignorance.

But isn't Firebase supposed to auto-magically scale. From https://www.firebase.com/features.html:

Automatically scales with your app. When your app is a breakout hit, you don't have to worry about scaling your server code or provisioning extra capacity — Firebase handles that automatically for you. Our servers manage millions of >concurrent connections and billions of operations per month.

The part that is unclear is that we (users) need to connect to specific servers (s-dal5-nss-20). 

If firebase scales auto-magically why do we need to connect to specific servers? 

More importantly, these servers seems to go down (e.g. they could be down for many minutes). Isn't auto-scalability supposed to improve service reliability. That is, if some of your servers go down we can still keep on using the service. 

Altogether a bit confused about scalability and reliability aspects of Firebase and would appreciate any light you can shed on this. 

[Andrew Lee]

Ibo -

[Firebase founder here]

The Firebase Database service is built fairly differently than most web services since it uses long-lived, stateful network connections (usually websockets) instead of normal HTTP traffic. This precludes the use of typical load balancing techniques, and instead we terminate each web socket on a single “shard” on our side.

This has a couple implications.

First, it means if that shard that your app is assigned to goes down, your app backend goes down. We have the ability to move connections around between shards (and we do do this regularly, to balance load across our service), but unfortunately that currently requires manual intervention by our ops team, which means if we lose a server, you’ll see a short downtime until we re-balance traffic. We’re working to make that failover automatic, though it’ll be a while still.

Second, it does mean that scaling beyond some (generally quite large) limits requires manual work. We power some very large sites and apps like Twitch.tv and apps from CBS, and the vast majority of our customers never hit any of our scaling limits.  For customers that do run into these limits, we shard those customers across multiple app instances. This does (today) mean some manual work on both sides. One note that you may find helpful is that we generally scale quite well to very large numbers of connected devices (100k+ simultaneous connections), but we are more limited in I/O throughput. So if you're building an app that you expect to be writing a lot of data, I'd suggest you ping us beforehand so we can discuss the specifics and make sure we're prepared.

I admit that the wording on the website doesn’t reflect reality very well right now. While we do scale, it’s not quite as automatic today as we’d like. We’ll get the website fixed.

I hope that helps you understand our capabilities and limitations better -

-Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/6f40da22-7210-4664-b485-fd18b79fc03b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Lee | Firebase Cofounder | and...@firebase.com | (805) 426-9663