Realtime database and Cloud functions event handler

[Laurent Pellegrino]

Hello,

I have a use case where the Firebase realtime database is used to perform a synchronization between multiple instances of a same cloud function F1. 

This function F1 initializes or decrements a counter in a specific path P/id/counter on the realtime database by using a transaction (id is shared between F1 instances).

Another function F2 is configured to be triggered when a write to P/{id} is performed. The purpose of F2 is to perform an action when the counter value is equal to 0 (meaning all F1 executions for a given id have completed). Thus, F2 returns an empty promise if the counter is greater than 0. Otherwise, the action is triggered and then the database entry P/id removed, which retriggers F2. In order to avoid the circular dependency, I detect with the following piece of code if the current snapshot contains any data:

if (!snapshot.exists()) {
    // function triggered by a delete
    return emptyPromise();
}

The flow works pretty well. However, each synchronization requires a dummy execution for F2.

I wonder if there a mean to better handle such a scenario? at least having the possibility to differentiate when a function is triggered (onWrite vs {onInsert, onUpdate or onRemove}) would help to resolve this issue.

Is there new database event handlers planned in the new future? or something that could help for creating rendez-vous between functions? 

Kind Regards,

Laurent Pellegrino

[Kato Richardson]

Hi Laurent,

It seems, at first glance, that this may be overly complicated. But it's going to be hard to be very useful without a better understanding of the underlying use case instead of just the implementation details (see XY problem)

Some initial ideas to reduce function invocations:

• Move the code from F2 into F1 in an if( counter == 0 ) block
• Store the counter in a separate path and monitor that in F2 so it's not retriggered on remove
• Have F1 write a "remove request" or other trigger to another path, and have F2 monitor there

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/cf5009ec-31c3-4b37-b874-e29fffbe7c6f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Laurent Pellegrino]

Hello Kato,

Thanks for your answer.

Here is more information about the big picture.

I have a service that requires handling "requests" every 30s. End users may register thousands of requests. A request is a small set of actions. Each action is independent and requires up to 15s to complete. At the end, an action produces a result that is true or false for the sake of the explanations. 

Depending on the number of results that are false, I need to perform a final action. 

Since actions are independent, they can be executed in parallel. Results produced by actions are used only for deciding if a final action is required.

As a side note, I am looking for a good compromise between performance and storage in the realtime database.

> Move the code from F2 into F1 in an if( counter == 0 ) block

It is not acceptable because it means that all actions are executed in series within F1. Thus, taking up to 45s if 3 actions, instead of 15s + synchronization time if in parallel, which is taking far less than 45s with the tests I made.

> Store the counter in a separate path and monitor that in F2 so it's not retriggered on remove

If I understand correctly, it is the same as not removing the counter value once the synchronization has been performed. This is something I would like to avoid since storage will growth as a function of the "requests" that are handled times the time elapsed. The bill will be really salted at $5/GB. Deleting daily, weekly or monthly will help if it is possible to delete without triggering Cloud Function events.

> Have F1 write a "remove request" or other trigger to another path, and have F2 monitor there

In my use case, I don't really see the difference with the previous suggestion. My purpose is to delete transient data used for synchronization. Having more fine grain Cloud function event triggers, such as the one supported by the Firebase API (child_added, child_removed, child_changed, child_moved) would help.

Any ideas or suggestions are welcome.

Kind Regards,

Laurent Pellegrino

Le jeudi 1 juin 2017 17:40:18 UTC+2, Kato Richardson a écrit :

Hi Laurent,

It seems, at first glance, that this may be overly complicated. But it's going to be hard to be very useful without a better understanding of the underlying use case instead of just the implementation details (see XY problem)

Some initial ideas to reduce function invocations:

• Move the code from F2 into F1 in an if( counter == 0 ) block
• Store the counter in a separate path and monitor that in F2 so it's not retriggered on remove
• Have F1 write a "remove request" or other trigger to another path, and have F2 monitor there

☼, Kato

On Mon, May 29, 2017 at 5:10 AM, Laurent Pellegrino <laurent.p...@gmail.com> wrote:

Hello,

I have a use case where the Firebase realtime database is used to perform a synchronization between multiple instances of a same cloud function F1. 

This function F1 initializes or decrements a counter in a specific path P/id/counter on the realtime database by using a transaction (id is shared between F1 instances).

Another function F2 is configured to be triggered when a write to P/{id} is performed. The purpose of F2 is to perform an action when the counter value is equal to 0 (meaning all F1 executions for a given id have completed). Thus, F2 returns an empty promise if the counter is greater than 0. Otherwise, the action is triggered and then the database entry P/id removed, which retriggers F2. In order to avoid the circular dependency, I detect with the following piece of code if the current snapshot contains any data:

if (!snapshot.exists()) {
    // function triggered by a delete
    return emptyPromise();
}

The flow works pretty well. However, each synchronization requires a dummy execution for F2.

I wonder if there a mean to better handle such a scenario? at least having the possibility to differentiate when a function is triggered (onWrite vs {onInsert, onUpdate or onRemove}) would help to resolve this issue.

Is there new database event handlers planned in the new future? or something that could help for creating rendez-vous between functions? 

Kind Regards,

Laurent Pellegrino

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/cf5009ec-31c3-4b37-b874-e29fffbe7c6f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.