Best way to deal with 30 day "Max inactivity time" quote for Cloud Functions?

[David DeRemer]

The problem:

Firebase/Cloud Functions that are not invoked for 30 days will no longer be triggered. There is no warning, error, or indication that this has happened anywhere on GCP (that I have found). The functions will simply not be triggered.

This is explicitly stated in the quotas, but I think it is a confusing and not very well known limitation of Cloud Functions. This has serious implications for cloud functions that are infrequently utilized. For instance, we have some features that require an import of new content every 1-2 months, which utilize cloud functions to normalize the data. Currently, the only way to get the functions to work again is to re-deploy. So, we have to remember to re-deploy the functions to ensure they are alive or face issues in the import.

The question:

Does anyone have any recommendations for best practices to keep infrequently invoked but crucial cloud functions alive and responsive indefinitely?

Some ideas we've had:

1. Set up a CRON job that 1x per week at a low volume time, triggers a Travis build that includes a deploy script to re-deploy all functions.

The problem with this is that we really don't like automated deployments to production. If something goes wrong with the deployment, nobody will be actively monitoring it. Depending on package.json, the functions might additionally pick up new and untested minor version updates.

2. Set up a CRON that triggers each CF but does no op, so that the function is triggered in a benign way and resets the 30 days timer.

Issue here is that you have to actually trigger the function (i.e., change an auth object, storage object, RTDB node, etc.)

IMO, this max inactivity time quota is the single biggest achilles heel of cloud functions. I've raised this to Firebase support in the past and they are aware of the limitation and this doesn't seem to be something they will change. Perhaps because invocations use PubSub under the hood? So, at a minimum, I think it'd be nice to have a best practice for how to keep important but infrequently cloud functions alive without redeploying every 30 days.

Thanks in advance for your help!

[Samuel Stern]

Hi David,

Thank you for the really detailed feedback!  The good news is we consider this to be a bug (a bad one) and we have a fix almost ready.  If you've been on this list, you'll know we never offer hard timelines so all I can say is expect news here "soon".

In the meantime, your workaround of just hitting your functions with a no-op once a month is probably the best/most reliable thing to do.  Although you're totally right that this is much harder for some trigger types (auth, analytics) than other (HTTP, PubSub).

Also while we're here: I'd love to hear more about your functions work load.  How many functions do you have that are triggered less than once a month and what do you use them for?

- Sam

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/89509cfe-313e-44fe-924b-14db02a70529%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David DeRemer]

Hi Sam,

Thanks for the reply! I'm so excited you feel the same way I do and that a fix is in the works. I had given up hope that it could be resolved in a reasonable way. So hooray and thank you!

We currently have 63 unique functions. I'd say ~5 are affected by this. Here are a few real use cases we have:

1. Data imports

We have a feature where we periodically have to import new content to keep the feature interesting. We get this data as a CSV file. We currently parse the CSV and update nodes in the RTDB with its contents. The RTDB writes trigger some cloud functions that do additional work on the data and fanout it out to various locations where it is needed. We did it this way because we wanted to separate the concerns of the import/upload from the background db fanouts. We also wanted that work to be done regardless of the import source.

You can also imagine a world where the CSV is simply dropped into Storage and an import function is automatically invoked. If we added that to our CMS now, it'd be a huge problem because the admins would upload new files and nothing would happen if it's been more than a month. So, for now we just have to do the import manually so we can babysit it.

2. Custom claims management

We have a function that adds or removes some custom claims on admin users based on role changes in the CMS. As there aren't many admins and this doesn't change often, it's almost guaranteed the function will be dead by the time they make a change. You can see how this would be problematic as the user would just expect to change this setting, not redeploy functions functions (that have no idea what's happening behind the scenes).

3. Metadata fanouts

We have some functions that listen to different content types in the DB, and if there are changes may sync to/from another node we use for content to be shown on the homepage. Some of the content is only updated every couple of months, so it's a problem. Admins may make a change and then call us complaining they're not seeing it on the homepage.

Now, I recognize that we could add the work that these functions are doing into the original request/function that triggers it. However, we prefer to separate concerns and push anything that is not essential to the user completing their task to a background job. One of the amazing things we love about cloud functions is we can focus the user path on the critical path to capture and event and let the CF's handle the rest in the background. For instance, we set a record that indicates a user action, and rather than the client or the initial trigger doing all the work, we let a series of background functions do it instead. Much better maintainability.

Fortunately we have the scale where most of our functions are triggered within the window. But I'd imagine this would be even worse for small apps. For instance, consider an app with only a couple hundred users. Consider a function to delete user data if their auth object is deleted...if this doesn't happen frequently enough, then the user's auth will be deleted but their data will not be deleted and the app owner will have no idea that it didn't happen. Yikes!

So, thank you for working on this, and if you don't mind (i.e., happen to remember), I'd love an update here when it's live.

Thanks!

[David DeRemer]

Hi Sam,

I noticed that Google Cloud PubSub now has a "Subscription expiration" setting that defaults to "Off". And more importantly that the 30 day inactivity limit has been removed from: https://cloud.google.com/functions/quotas#time_limits

Is it safe to celebrate and say that this issue is behind us? If so, thanks to the Google team for fixing this important issue!

[Samuel Stern]

Yes David this issue has been resolved, sorry I forgot to update this thread.  You can now have inactive functions for >30 days.  

We're all super excited to put this bug behind us :-)

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/473a9851-f5c8-490a-82a3-6b1ed0ac70fa%40googlegroups.com.