Firestore functions create trigger fired multiple times, any plans for max trigger counts?

[a...@21risk.com]

Today I experienced multiple events for a firestore create trigger.

According to the firebase documentation here:

... we plan to guarantee "at least once" delivery.

My question is: Are you also working on a "max one trigger on create"? Or should I rely on callable functions for such a use-case?

[Tom Larkworthy]

Events have an event_id which is preserved on multiple deliveries. I don't think even relying on http functions solves the issue. The function could start, then crash before it posted the result. The caller would see a failure, and retry and now you end up with stuff being done twice. The ideal design is to make sure your operations inside functions are safe to do twice (idempotent). The event_id is a useful primitive for deduplicating downstream operations.

Some words on idempotent and the impossibility of exactly-once semantics:

https://bravenewgeek.com/you-cannot-have-exactly-once-delivery/

Though to confuse things Kafka say they have solved it:

https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/ 

Anyway, engineering wise getting to at-least-once is fairly hard which I think we have nearly reached if you occasionally see events twice. I think you should be designing on the basis of at-least-once for the time being, exactly-once is maybe impossible, but at the minimum very hard.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/e40861fa-55be-4f13-9cfd-21417a0d97ca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[a...@21risk.com]

Hi Tom. 

Thanks for the quick answer. I have multiple commens.

1. What do you mean with "Events have an event_id which is preserved on multiple deliveries" ? Does this mean that if an "create-event" triggers multiple times, the event_id will be the same?
   
2. In firebase-functions, we are talking about this event_id right?https://firebase.google.com/docs/reference/functions/functions.EventContext#eventId
3. I have a scenario, and I could imagine many other developers have similar, where I wan't to build this chain of events: a) A user creates a setup doc, saved to firestore. b) This triggers an create event c) A firebase functions uses this create event to create some "related documents". Now if there is a "duplicate" create trigger there would be created extra "related documents". What strategy would you use to make this function idempotent? Maybe use firestore transactions to check if another function, already processed that eventId or?

Thanks for your answer so far. This trigger-based programming paradigm just became a bit more advanced ;) 

[Tom Larkworthy]

1. yes

2. yes

> I have a scenario, and I could imagine many other developers have similar, where I wan't to build this chain of events: a) A user creates a setup doc, saved to firestore. b) This triggers an create event c) A firebase functions uses this create event to create some "related documents". Now if there is a "duplicate" create trigger there would be created extra "related documents". What strategy would you use to make this function idempotent? Maybe use firestore transactions to check if another function, already processed that eventId or?

If the action the function takes is all to do with Firestore, yes, you have an ideal deduplication primitive of a collection of "ProcessedEvents". You can do a read-write transaction where in the read you ensure the ProcessedEvents does not contain event_id, and in the write, write the event_id plus all the other docs that are part of the work.

> My question is: Are you also working on a "max one trigger on create"? 

No. We are still working on the previous engineering challenge of at-least-once and expanding the number of event providers (and I imagine we will be for a while).

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/1546f5c2-2b5c-4fba-ae15-69b4087a177a%40googlegroups.com.

[a...@21risk.com]

Awesome that the eventId is the same.

So based on your recommendations and my new knowledge, I played around with the idea of writing a generic function to avoid duplicate create or update events.

It's an async function, with the name "isDuplicateCreateOrUpdateEvent",  that uses a helper collection called "utils", to keep track of events. Each document in the collection, can have properties like: "create_lastEventId", "update_lastEventId", and "deleted". The goal is to never run a "duplicate" create or update event.

Do you think this approach is good / would work ( I don't have so much experience with transactions)?:

 

async function isDuplicateCreateOrUpdateEvent(change: Change<FirebaseFirestore.DocumentSnapshot>, context: EventContext): Promise<boolean> {
    try {
        const firestoreEventType: 'create' | 'update' | 'delete' = checkEventType(change);
        logger.debug(`isDuplicateCreateOrUpdateEvent called, analyzing event from ${change.after.ref}, eventType: ${firestoreEventType}`);
        if (firestoreEventType === 'delete') {
            // We don't worry about multiple delete events. Just mark the helper doc, with deleted: true
            await afs.doc(`utils/${change.before.id}`).set({deleted: true}, {merge: true});
            return false;
        }
        const t = await afs.runTransaction(async (transaciton: FirebaseFirestore.Transaction) => {
            logger.debug(`Transaction running inside isDuplicateCreateOrUpdateEvent`);
            // Start by extracting the utils document
            const metaDocRef = afs.doc(`utils/${change.after.id}`);
            const metaDoc = await transaciton.get(metaDocRef);
            const lastEventId = metaDoc.data()[`${firestoreEventType}_lastEventId`];
            if (lastEventId === context.eventId) {
                // If lastEventId === process.env.eventId, we processed this trigger before, i.e. a duplicate event
                return true;
            }
            // If we reach here, it's not a duplicate event. Save the lastEventId, to prevent duplicate future triggers
            await transaciton.set(metaDocRef, {[`${firestoreEventType}_lastEventId`]: context.eventId}, {merge: true});
            return false;
        });
        return t;
    } catch (error) {
        // When will we execute here? Hmm.. If the transaction fails, is this gonna emit?
        // Read documentation here: https://firebase.google.com/docs/firestore/manage-data/transactions
        logger.error(`isDuplicateCreateOrUpdateEvent failed with error: ${error.message}`);
        throw error;
    }
}

I could then use isDuplicateCreateOrUpdateEvent(), in all functions that should be protected from running multiple times on create an update.

And thanks so much for all the help :)