Cloud Function performance, and CF roadmap

[hrair.m...@foundersfactory.co]

Hi all

First of all, let me say that I love Firebase. I think it provides a great alternative to AWS, cutting corners on flexibility/features in well chosen areas in exchange for developer productivity. Cloud functions are a big part of this. I only started using firebase after CF was introduced because I did not feel comfortable building a company's tech on a suite which had no fallback to a traditional HTTP back-end/API.

But... there is one issue that stands in the way of me embracing and recommending your stack to others. In fact, it's a combination of two issues:

1) Cloud Function performance is unpredictable / slow.

2) There is a lack of transparency / roadmap regarding when (1) is going to be resolved, if at all.

When I mention to my colleagues that functions can take up to 10 seconds to execute due to cold starts, I get blank stares. I've spent quite a bit of time searching for others experiences on this and they mirror mine. I also spend time in the firebase community slack channels. A lot of people are seeing the same issue and there is no official workaround/solution that's come from firebase developers themselves.

I'd love to get a canonical response on this topic here, and I'd be happy to spread it around to the rest of the community.

Thank you!

[Peter Svensson]

Hi Hrair, this mirrors my own experience and concerns. Everyone understand that it's hard to create something this big, securely and reliably, so what have to give is probably speed. But what we as possible and budding first-arrivals and evangelists would need is some kind of .. something. 

(Directing this now to the FB/Cloud team(s));

Maybe just a blog a week about what has gone as it should and what you're still working on. No need to pull all the boxers down, just something. There are well written blog coming out once a month on various FB features, but I'm talking ab out more frequent, less laid out and more neckbeard. Highlight the areas you're working on improving, soliciting feedback in the form of .. forms perhaps, just so we get a feeling we see where you're going. It feels like sitting at a live sports event, having the arena blacked out and jsut seeing scores now and then.

Cheers,
PS

[Bartholomew Furrow]

I'm glad you sent this; I just made an exhaustingly-researched post[0] about this being a problem for using CF to distribute computations.

As an aside, one of the challenges of cold starts is that with the way functions are uploaded through Firebase, all your functions use the same modules. So if you have just one function that uses, say, cloud storage, then every function's cold starts become just a little bit slower.

Cheers,

Bartholomew

[0] Note that the research was exhausting, not exhaustive.

[Kato Richardson]

Hello Hriar, Peter, and Bartholomew,

Thanks for this. I wanted to ack this conversation and let you know that we have talked about it internally. Functions is a huge initiative involving a lot of internal resources, so it's a big discussion to see how we can coordinate more frequent updates, but we hear you loud and clear.

We're looking at at least an update on what's happening right now, if not a more cyclic report.

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/a082bd55-9608-4657-8b25-0fed4d19da7f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Michael Bleigh]

@Bartholomew — one thing you can do to minimize code loading time is to require modules inside functions instead of outside. For instance:

exports.myExpensiveFunc = functions.https.onRequest((req, res) => {
  const expensiveModule = require('expensive-module');
  expensiveModule(req.body).then(result => res.send(result));
});

This will only incur the parse/load time for the specific function when it’s run.

​

On Thu, Mar 29, 2018 at 12:41 PM 'Kato Richardson' via Firebase Google Group <fireba...@googlegroups.com> wrote:

Hello Hriar, Peter, and Bartholomew,

Thanks for this. I wanted to ack this conversation and let you know that we have talked about it internally. Functions is a huge initiative involving a lot of internal resources, so it's a big discussion to see how we can coordinate more frequent updates, but we hear you loud and clear.

We're looking at at least an update on what's happening right now, if not a more cyclic report.

☼, Kato

On Mon, Mar 26, 2018 at 4:00 PM, Bartholomew Furrow <fur...@gmail.com> wrote:

I'm glad you sent this; I just made an exhaustingly-researched post[0] about this being a problem for using CF to distribute computations.

As an aside, one of the challenges of cold starts is that with the way functions are uploaded through Firebase, all your functions use the same modules. So if you have just one function that uses, say, cloud storage, then every function's cold starts become just a little bit slower.

Cheers,

Bartholomew

[0] Note that the research was exhausting, not exhaustive.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/a082bd55-9608-4657-8b25-0fed4d19da7f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CADypTEZkcoJDob63g5DCoZC5mkL%2Be8xFUCzk6hUzSVEtWSNsPQ%40mail.gmail.com.

[Jason Polites]

Hi folks,

PM for Cloud Functions here, hoping I can shed some light.

Cold start time is a function (no pun) primarily of two things:

1. The amount of "work" needed to be done when the instance is started
2. The ability of the instance to do the work

In GCF, the "work" includes the following:

1. Starting the Node process
2. Starting an HTTP server to listen for requests and loading/executing its dependencies
3. Loading the user code (your code), which (in Node) also means globally scoped code is executed

You can't really affect #1 & #2, but you can have an impact on #3.  The less code that is both loaded (i.e. the fewer STAT et. al operations done), and executed; the less time it will consume.  Moving some of this into function scope will help in the literal interpretation of "cold start", but often you'll still feel it because you'll see it in "first request" which is usually when you're seeing cold start (although there IS an anti-pattern here, which I'll talk more about below).

The real question is, "what's a reasonable cold start time?".  You can get a baseline by doing steps 1-3 above on your local machine and timing it.  It may be that you observe something less than ~10 seconds, but it's important to recognize that a 128MB function has far fewer resources available to it than your local machine (related to the first #2: "The ability of the instance to do the work"). 

A further important consideration is not so much how long do cold starts take, but also how frequently do they occur.  A slow start time that occurs infrequently might be reasonable (VMs.. I'm looking at you), an in many production cases they are actually fairly sparse, but you tend to see them when you're building/testing because you're often going from 0 to 1. Obviously sparse workloads (ones where traffic is infrequent or spikey) are going to be naturally prone to more frequent cold starts, but having a good understanding of what percentage of requests are cold is also important.

So.. what to do...

We are indeed working on several efforts to improve both cold start time and frequency, but we don't have anything we can announce as yet.  The best thing to do for now is try to optimize the startup time of your function.  In particular some node modules take a loooong time to require and in some cases this can be mitigated by referencing the specific sub-module you actually depend on.  Also, following on from Michael's suggestion below, there is an anti-pattern that may occur more frequently in the Firebase use case (some discussion about this is here: https://cloud.google.com/functions/docs/bestpractices/tips).

If you have more than one function in your module (common in Firebase), you might be tempted to bunch all your require statements at the top, in global scope. This is fine if every function uses every module every time, but if not you're imposing the cost of loading these to every cold start, including the ones which never use that module.  The better approach is to lazy-load these modules only in the functions that need them. This is basically the same thing Michael was suggesting, but it's even more likely to happen in the multi-function-per-module use case.

Hope this helps.

Cheers,

Jason.

[Bartholomew Furrow]

Kato, Michael and Jason: As always, I appreciate the responsiveness of the Firebase team!

I'm glad to hear that there are efforts underway for Jason's #1 and #2.

Vis-a-vis Michael's suggestion, I don't know enough about nodejs to know this: if I require() in a function, will that be cached across multiple calls to that function, or does the work of requiring get done every time the function is called? It would be nice if JS simply had a "lazy" keyword like Swift.

I can pile on a little to the discussion of cold start times: I chopped a whole second off the cold-start time of every one of my functions by removing just two calls to gcs.bucket(bucketName). Here's a pattern people can use if they'd like.[0]

One last question. Can someone take a look at this thread and comment on the sorts of super-slow starts I'm seeing when I try to send a lot of requests to a cold https function at once? The thing I'm looking at specifically is how the last shard can start more than 15 seconds after it was requested.

Thanks again!

Bartholomew

[0] Here's how I took my call to gcs.bucket(stagingBucketName); out of the top-level:

const stagingBucket = (() => {

    var instance = null;

    return {

        getInstance: () => {

            if (instance === null) {

                instance = gcs.bucket(stagingBucketName);

            }

            return instance;

        }

    };

})();

Then:

stagingBucket.getInstance().file(filePath).download({

    destination: downloadPath

})

You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/yddyVVMbbgo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/44f4b486-298b-4313-aa52-bd8707c90958%40googlegroups.com.

[Michael Bleigh]

require() caches, so it can safely be called over and over for the same module

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHaiWHNqBnjhcque5mAnFE4wSV2_WAdb6QTXLox4F-4oaUWvsQ%40mail.gmail.com.

[Brian Woodward]

While building out some CLI tools, we created this module: https://github.com/jonschlinkert/lazy-cache

The intent was to be able to only require in modules when a command or other piece of code actually needed them, but to also be able to use a pattern of requiring modules at the top of a file.

I think this can also be used in Cloud Functions since the code is not statically bundled before hand. I'd recommend having a `utils.js` file to include your dependencies:

```

const utils = require('lazy-cache')(require);

utils('express');

utils('some-dependency', 'myDep');

module.exports = utils;

```

Then you can use them in your functions file like this:

```

const utils = require('./utils');

module.exports.myfunction = functions.https.onRequest(function(req, res) {

  res.send(utils.myDep('do-something'));

});

```

We also use a pattern in the utils.js file to allow the file to be browserified if necessary (seen in this file https://github.com/doowb/npm-api/blob/master/lib/utils.js) :

```

const utils = require('lazy-cache')(require);

const fn = require;

require = utils;

require('express');

require('some-dependency', 'myDep');

require = fn;

module.exports = utils;

```

I hope this helps speed up cold starts.

Thanks,

Brian

On Sat, Mar 31, 2018 at 12:34 AM, 'Michael Bleigh' via Firebase Google Group <fireba...@googlegroups.com> wrote:

require() caches, so it can safely be called over and over for the same module

On Fri, Mar 30, 2018, 7:18 PM Bartholomew Furrow <fur...@gmail.com> wrote:

Kato, Michael and Jason: As always, I appreciate the responsiveness of the Firebase team!

I'm glad to hear that there are efforts underway for Jason's #1 and #2.

Vis-a-vis Michael's suggestion, I don't know enough about nodejs to know this: if I require() in a function, will that be cached across multiple calls to that function, or does the work of requiring get done every time the function is called? It would be nice if JS simply had a "lazy" keyword like Swift.

I can pile on a little to the discussion of cold start times: I chopped a whole second off the cold-start time of every one of my functions by removing just two calls to gcs.bucket(bucketName). Here's a pattern people can use if they'd like.[0]

One last question. Can someone take a look at this thread and comment on the sorts of super-slow starts I'm seeing when I try to send a lot of requests to a cold https function at once? The thing I'm looking at specifically is how the last shard can start more than 15 seconds after it was requested.

Thanks again!

Bartholomew

[0] Here's how I took my call to gcs.bucket(stagingBucketName); out of the top-level:

const stagingBucket = (() => {

    var instance = null;

    return {

        getInstance: () => {

            if (instance === null) {

                instance = gcs.bucket(stagingBucketName);

            }

            return instance;

        }

    };

})();

Then:

stagingBucket.getInstance().file(filePath).download({

    destination: downloadPath

})

To unsubscribe from this group and all its topics, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/44f4b486-298b-4313-aa52-bd8707c90958%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHaiWHNqBnjhcque5mAnFE4wSV2_WAdb6QTXLox4F-4oaUWvsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CANPY8KXgwTorALz7u1E-afbOhuNQioJgXdeYJUyRx%3DyJ-x-dEA%40mail.gmail.com.

[Bartholomew Furrow]

Thanks, Brian! I had already implemented Michael's solution before I saw yours, but I'll bear it in mind when I get to something more complicated. For what it's worth, my cold-start time is now down to about 2 seconds from about 4, after removing:

const request = require('request');

const request = require('request-promise-native');  // Yes, I was loading both

const crypto = require('crypto');

const gcs = require('@google-cloud/storage')();

const tmp = require('tmp-promise');

const moment = require('moment');

...from every request.

To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/44f4b486-298b-4313-aa52-bd8707c90958%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHaiWHNqBnjhcque5mAnFE4wSV2_WAdb6QTXLox4F-4oaUWvsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CANPY8KXgwTorALz7u1E-afbOhuNQioJgXdeYJUyRx%3DyJ-x-dEA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/yddyVVMbbgo/unsubscribe.

To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CABvc%3DwvObZX%2BjVWxTGeKYSjm6JQqmPNPn7a0ESx1338Ryp%2BGWQ%40mail.gmail.com.