How to stop restarting Firebase Functions each request?

[Eugene Krevenets]

Hi all!

I'm developing Actions on Google and use Firebase Functions to host Node.js app. But I have found that Firebase Functions restarts app for almost all requests (once in 10-20 requests it persists to use already run instance). And because I need to load some resource (just few small files) it takes about 3-4 seconds to restart Function each request what drops response time from ~500ms to ~5000ms (10 times slower). 

How could I persist to not restart node.js app?

[Eugene Krevenets]

Btw, I have researched recommendations about the cold start (https://cloud.google.com/functions/docs/bestpractices/tips) and we don't have

- memory leakage 

- app uses less than 50MB

- we don't have high load - only about one request in one minute

So I don't understand, why do we have a cold start here.

Is there any way to get which signal has forced to restart my Function?

[Eric White]

I have not explored this specifically, but by definition, the cloud functions are designed to execute and complete.  Hence recommendations to clean up temp objects.

You also have no guarantee that calls will be in the same execution space.

Did you try the recommendation to define global scope on your expensive tasks?

cheers.

Eric

[Eugene Krevenets]

Hi Eric! Good point, yes that's true our app is made in the purely functional way, so it doesn't persist state between sessions.

But the problem that Node.js is not very effective on restarting each request (in contrast to for example php) - It gives performance benefits when it persists between sessions. And actually, it is written in docs of firebase functions that there are ways to reduce cold start, and we follow them. But we still have the cold start.

> Did you try the recommendation to define global scope on your expensive tasks?

Interesting, Maybe I have missed something in the firebase docs. Do you mean global variables (https://firebase.google.com/docs/functions/tips#use_global_variables_to_reuse_objects_in_future_invocations)? Yes, because we need to process some actions on app start we definitely store data globally in the app between session (what took about 2-3 seconds). But it doesn't work when we have cold start.

[Doug Stevenson]

Eugene,

Cloud Functions will shut down a running instance if it hasn't been asked to run its function in a while.  The actual idle time before shutdown is an implementation detail, and may changed over time.  If your code isn't executed very often, and it depends heavily on some loaded state, you will definitely feel the pain of a cold start more often.

There is really no way to stop cold starts altogether with Cloud Functions.  Even if you manage to keep one instance alive, when load increases, a second instance may be spun up, and it will also have a cold start.

If cold starts are unacceptable, you'll have to manage your own servers that run 24/7, and make sure there are always enough warm to handle your expected load.  This is obviously going to cost more time and money, but that's the tradeoff between traditional and serverless backends.

Doug

[Eugene Krevenets]

Thank Doug!

Ok, thanks for your explanation! I'm not sure whether idle time really shutdowns my app, because I found many cases when a cold start was with an idle time less than a second, but in other cases, Firebase didn't shut down the app with idle time more than few seconds. 

It's strange that I can't control idle time before shutdown. If I would be able to set it to 10 minutes or so it would be much more helpful.

And yes, 2-3 seconds delay before a response unacceptable. Because we are working with Actions on Google with Dialog Flow, which don't allow big timeouts. So maybe there won't be other solution then move to our hosting.

Best,

- Eugene

[Hiranya Jayathilaka]

Aggressively shutting down idle instances is an inherent quality in serverless functions. Also sometimes it may spawn multiple instances to handle concurrent requests, which may explain why you see cold start delay in requests that are only 1-2 seconds apart (they got routed to multiple new instances).

If you need more control over how instances are shutdown, look at App Engine: https://cloud.google.com/appengine/docs/standard/nodejs/config/appref#scaling_elements

With basic scaling you can explicitly configure max_instances and idle_timeout. Depending on your traffic pattern, even auto scaling may be good enough for you.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/1c69cb1d-35db-4dbb-9322-aa36c3d2e5a1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Hiranya Jayathilaka | Software Engineer | h...@google.com | 650-203-0128

[Eric White]

I was going to make the same suggestion.

Now that AppEngine supports Node on GAE standard it should be a simple migration, and default idle_timeout is 5 min.

cheers.

[Eugene Krevenets]

Hi Hiranya,

Yes, it is clear. But what is the sense to give "tips and tricks" of how to fight against cold start (https://firebase.google.com/docs/functions/tips) if we can't control it and it is "inherent quality"?

In addition, there is information about global variables https://firebase.google.com/docs/functions/tips#use_global_variables_to_reuse_objects_in_future_invocations which doesn't work on cold start. For me, it still looks like cold start isn't desirable behavior for Firebase Functions too.

it would be much more useful to admit that cold start is very often event and we can't influent on it, so please focus on optimizing start time and forget on persisting between sessions.

[Doug Stevenson]

Eugene,

The moment you are able to start exercising this level of control over a serverless system, an important feature of the system stops becoming a benefit:  You only pay for what you use.

With Cloud Functions, notice that you're only billed for function executions and resources that actually occur during execution.  This is what makes serverless architectures cost effective for many apps.  You're not paying to keep a server up and idle for unknown amounts of time (which has real world cost, often at little to no benefit).  The reason you can't control your idle uptime is because every single developer would always opt into the maximum amount of free idle time!  If you're not billed for a resource, of course you would ask for as much of it as possible (at least I would - that's just good pocketbook economics).  Given that you're not being billed for idle time at all, any idle time that you get is actually a courtesy of the system - a cost that the serverless provider willingly pays for all customers.

I know that the engineering team has put effort into speeding up cold starts.  It actually used to be worse!  There have been both improvements at the service layer and at the software library layer.  I'm sure there's still more work that could be put into it, but unfortunately, all work does need to be prioritized.  For example, there is a lot of demand for other runtimes, such as python (very recently now in beta), and JVM.  JVM runtimes will definitely have even worse cold start times, just because of the way that JVMs work.  But there is enough demand for serverless JVMs that greater cold start costs are actually acceptable.  I know that seems difficult to believe, but these developers would be saving large amounts of time and money not having to port their Java/Kotlin/Scala/JRuby/Groovy/etc and all of their dependences to a completely different language.  As an Android developer myself, I'm actually one of them!  I'm secretly rooting that they divert resources into the JVM.  Well, I guess that's not a secret any more.  :-)

I'd encourage you to look into App Engine or some other backend that gives you the level of control that you want (accepting the cost that it incurs), while the engineering team continues to make progress on performance and other features.  It can only get better from here!

Doug

You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/MZjwG3XEKus/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/ae3bf8b4-60bd-449e-a85e-70752118287a%40googlegroups.com.

[Hiranya Jayathilaka]

Doug has given a comprehensive answer pointing out the cost-performance tradeoff of serverless functions (Functions-as-a-Service). I will just add a bit more context here:

FaaS systems are optimized for cost. If the traffic is intermittent, the developer has to pay very little (often nothing), but that comes at the expense of poor cold start performance. There are methods to reduce/manage that overhead, but it cannot be fully eliminated. This is why performance is top on the list of disadvantages in this Wikipedia article about serverless runtimes: https://en.wikipedia.org/wiki/Serverless_computing#Disadvantages. There's actually a lot of ongoing research in this area, and we can expect performance of FaaS systems to get better with time. 

App Engine on the other hand is your classic Platform-as-a-Service cloud -- tried and tested for a decade (although the Node.js runtime is relatively new). It gives you more control over instance setup and teardown, so you can fine tune the deployment for your exact traffic pattern (you can even have some number of dedicated instances always listening for requests). This of course comes at a higher monetary cost to the developer.  But if fast response times are critical for your app, and your traffic pattern is not amenable to benefit from the instance retention in a FaaS environment, this is a pretty good option.

I guess the key takeaway here is the cost-performance tradeoff. Of course we'd love to have best performance at minimum cost, but often in real world systems one comes at the expense of the other.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/ae3bf8b4-60bd-449e-a85e-70752118287a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.