Does querying Firebase with once('value') count as a connection?

[Ruben Gonzalez Torres]

Hello everyone,

If I query my Firebase DB using once('value') from my Lambda functions or Google Cloud Functions, does that count as a connection? I am asking because of the limitation of 100k simultaneous connections to a DB and my API is going to be heavy on requests.

Thanks in advance

Ruben

[Hiranya Jayathilaka]

Yes, once() does establish a connection to the database. But each App instance would typically establish only one connection to the database, and multiplex all RTDB API calls on to that.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/0fdb8e93-c671-4828-92b4-c91d917e50e9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Hiranya Jayathilaka | Software Engineer | h...@google.com | 650-203-0128

[Ruben Gonzalez Torres]

Thanks Hiranya.

Do you know if multiple instances of the same Google Cloud Function or Amazon Lambda function count as one App instance or multiple? I am asking because I know that they sometimes reuse the same memory for multiple calls, maybe they internally keep only one connection. I am trying to figure this out because if they create multiple connections for multiple function calls, then I will need to use other DB because of the maximum number of simultaneous connections.

[Samuel Stern]

Hi Ruben,

Just to be clear, once() sets up and then tears down a connection very quickly (assuming there are no other uses of the connection).  So you'd need to have 100,000+ functions executing at the same time to get into a situation where you'd have too many concurrent connections.  Do you expect this to be the case?

- sam

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/dc42650b-52c7-42c2-a956-6f831670d96e%40googlegroups.com.

[Ruben Gonzalez Torres]

Hmm, I see. Yes, I am expecting that is the case in the short-term future. I am going to sell to tech companies and the API I am building is going to be used by their mobile and web apps. That mean, if my client has x DAU then I am going to have x daily requests to my API just for that client. I know the limit of firebase is 100k at the same time but I am worried that firebase doesn't scale well for my use case. Right now my potential clients have 10k & 100k DAU. 

I am using firebase because I have a web dashboard to allow the company manage and see reports from the API calls, and a Real Time, Easy to use DB is awesome, but maybe I will have to use another DB for this particular use case. 

Do you have any recommendation about how can I tackle this use case with firebase? Or maybe this is just one use case where firebase is not a great fit?

Best

Ruben

[Samuel Stern]

Hi Ruben,

A few final things:

• Do your API consumers need to share data?  If not, you could make a separate project or database (in the same project) for each one.  This gives you the 100,000 limit per database, rather than global.
• I am told that Cloud Functions actually does "cache" the connection to Firebase between invocations on the same instance, and the maximum number of instances for a single named function is ~1000 so it would be pretty hard for you to break the 100k RTDB concurrent limit through functions access alone.

- Sam

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/de073c60-b598-484f-bfce-c1ec6e6ce5b6%40googlegroups.com.

[Ruben Gonzalez Torres]

Thanks Samuel for your reply. These are my current thoughts

- Microservices: Because the limitation of Cloud Functions, I am going to use Lambda function which also has an initial limit but I can increase it very easy.

- Database: I really want the Realtime capabilities of Firebase Database for the dashboard, to have several people from the same team managing their APIs.

- API calls: I am going to start with Lambda Functions calling once('value', ...) on firebase and when I start having a lot of concurrent requests then I will use an In-Memory DB to stop calling Firebase.

- Latency: If it is too slow to query FB from Lambda then I will need to create the In-Memory DB sooner.

Question:

- How is the best way to keep an eye or set an alert for the number of Firebase Concurrent Connections?

Best

Ruben

[Kato Richardson]

If you're not going to use any of the realtime capabilities, and your goal is solely to avoid concurrent connections, you would probably be better of just using REST GET calls instead of using the SDK to make once() calls. REST calls don't count against your concurrents since they do not open a socket connection.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/b28d5548-9a03-4b65-b50c-a614e13a43dd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Ruben Gonzalez Torres]

Thanks Kato, I completely forgot about the REST API. I think this is the best approach to avoid concurrent connections. Thanks again. 

[David DeRemer]

Hey Kato,

Quick question on this... when using Cloud Functions to interact with the RTDB, the preferred method seems to be to use the Admin SDK, not the REST API... or am I wrong here? Would using the REST API inside of cloud functions be beneficial?

I've also been thinking recently that I'm not totally sure when a "connection" is established to the RTDB when using the admin SDK.

Does a connection get established with initializeApp, and after that it multiplexes that connection?

Or, does a connection get established each time we do something like `await admin.database().ref('some_ref').once('value')`

I've also been told that, for RTDB triggered functions, using snap.ref.root.child may be preferable over admin.database().ref('...')

It's all a tad confusing and it'd be useful to know techniques to optimize connections to the database.

[Kato Richardson]

That really depends on your use case. The question was how to optimize for number of connections. The REST API is ideal for that scenario.

If we're not optimizing for connections, and working from a server environment, the admin SDK is probably simpler, since the SDK abstracts connectivity, reduces transactional overhead by using sockets, provides error handling abstraction, and perhaps most importantly, auth via REST (and in Functions) offer some additional complexities.

If you're doing a straight up, single write event from Functions, and auth is a solved problem (i.e. you have an auth token on hand that you can throw in the POST/PUT/PATCH), then it's probably a wash, although I could argue that REST is probably just a bit faster and does avoid the concurrent connection. But the simplicity of using the Admin SDK here is probably going to be a win over premature optimization.

☼, Kato

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/83a4e9f4-54bc-495a-9b28-7a7bd69e0cd5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[David DeRemer]

Thanks Kato. Super helpful info! Admin SDK inside of CF is ace and we've never come close to the max connections limit... we've generally hit other quotas long before that would be a problem. So we'll be sticking with admin SDK :-)