Cloud functions with cloud tasks

[Jean Omberto]

Dear Sam and all,

I am working on a project that fetches email addresses from one collection with various sub collection references and dispatch an email to these addresses every morning.

So far with 25 email addresses from 5 collection references,  all seems ok. ( each collection reference  provides 5 addresses).

I m trying to simulate a scenario where 5000 addresses have to be collected from 1000 collection references.

Question :
Considering that a cloud function has a 60 second timeout limit, should i rather involve cloud tasks for this ?

My approach is to use a loop that navigates around the references then for each reference,  send an email.
That is about 1000 tasks from 5000 references.

Your advise is appreciated.
Regards,
Jean O.

[Sam Stern]

Hi Jean,

You're right that you definitely don't want to send 5000+ emails from a single Cloud Function! However you can "fan out" to get this done:

1. Have one central scheduled function which reads the list of email addresses
2. For each email address it finds, send a Cloud PubSub message
3. Have a second cloud function which subscribes to those PubSub messages and actually delivers the email

The function in (3) will scale up to 1,000 concurrent copies which means your emails will get sent quickly and because each function only has to send one email it will not have to worry about timing out.

Here are the things you'll have to consider:

• Will the central function (1) finish in time? You may want to increase the timeout on that function to 540s, which is the maximum.
• This will increase your costs due to running more parallel functions (3). If you want to mitigate this somewhat, you could batch more than one email into each PubSub message so that instead of a 1:1 fanout you maybe have each function send 5-10 emails. 

Hope that makes sense!

Sam

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CACP%2B_1jHo9N6cmFGvtWv5RndNacmtxFtJ9mjZMtLGhd1AbkCtA%40mail.gmail.com.

[Jean Omberto]

Dear Sam,
Will give a try with cloud pubsub as advised.
It actually sounds interesting because during my GCP training, there was a theorical section on Cloud Pubsub. (Architecting with GCP Specialization).
This is an opportunity for a section review and practical.
Many thanks.
Jean O.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CACP%2B_1jHo9N6cmFGvtWv5RndNacmtxFtJ9mjZMtLGhd1AbkCtA%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHafJBpkvm64q6tQ2954bbMz9i6u4m%3DffAcp_a9mmUUZcx%3DKRw%40mail.gmail.com.

[Kato Richardson]

If cost is a concern, a more economical approach might be to use a third party like SendGrid for the actual sending, and have the function just look up the metadata and call their API. Most services like this can do bulk messages with email templates and some mail merge functionality.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CACP%2B_1jHo9N6cmFGvtWv5RndNacmtxFtJ9mjZMtLGhd1AbkCtA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHafJBpkvm64q6tQ2954bbMz9i6u4m%3DffAcp_a9mmUUZcx%3DKRw%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CACP%2B_1gWywqyrX1vqB3XD6L%2BO0eow7ppPUNVnt%3DcnjRu5h8sGw%40mail.gmail.com.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Jean Omberto]

Dear Kato,
Yes, Twilio Sendgrid is being used in the project.
The daily email content is generated from activities recorded in separate collections before dispatch.
The first motivation  for cloud task was that it is cheap.
I will take Sam's  suggestion and compare its price implication with the cloud task idea.
Since Gcp offers 1 million tasks per month in the free tier and 5 billion tasks for less than half a dollar, that was very appealing to me.  The email content is less than 64k so only a maximum of 2 billable operations are expected.
Regards,
Jean O.

On 22 Jun 2021 16:23, "'Kato Richardson' via Firebase Google Group" <fireba...@googlegroups.com> wrote:

If cost is a concern, a more economical approach might be to use a third party like SendGrid for the actual sending, and have the function just look up the metadata and call their API. Most services like this can do bulk messages with email templates and some mail merge functionality.

On Tue, Jun 22, 2021 at 4:02 AM Jean Omberto <ombe...@gmail.com> wrote:

Dear Sam,
Will give a try with cloud pubsub as advised.
It actually sounds interesting because during my GCP training, there was a theorical section on Cloud Pubsub. (Architecting with GCP Specialization).
This is an opportunity for a section review and practical.
Many thanks.
Jean O.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CACP%2B_1jHo9N6cmFGvtWv5RndNacmtxFtJ9mjZMtLGhd1AbkCtA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHafJBpkvm64q6tQ2954bbMz9i6u4m%3DffAcp_a9mmUUZcx%3DKRw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CACP%2B_1gWywqyrX1vqB3XD6L%2BO0eow7ppPUNVnt%3DcnjRu5h8sGw%40mail.gmail.com.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CADypTEadUwR9e04aJDC3Jmd%3DSFc4OCDc_j_3e2mqRP%2BiwmqouA%40mail.gmail.com.