onRead - Firestore function trigger

[Rui Faria]

- onRead function trigger

- context.auth.token  in function triggers

In my case i use one firestore account to provide services to my customers that have their accounts, and i need to control them individually.

if i have any type of problem or excessive use by a customer i cannot suspend all firestore account and block all the customers.

In most cases i use live queries and i cannot replace by generic google functions or api in the middle.

Using custom tokens,onWrite function trigger and security roles i have the control of writes.

without onRead function triggers i cannot have the control of the reads.

using onRead Triggers/auth.token we developers can have a more detailed control of reads implement read counters and block access to regions/cities or even IPs

using onRead triggers we developers can sleep well without afraid of huge bills that can kill our business

Please help me pushing firestore to implement:

- onRead function trigger

- context.auth.token  in function triggers

Rui Faria

[Sam Stern]

Thanks for the feature request!  It's something we've thought about but there are a few current issues:

• Cloud functions are currently fired asynchronously after the triggering event which means an "onRead" function would be fired after the read is already allowed.  So it wouldn't be a good place to do read security.
• Even if we changed the behavior above and made the function fire first, I don't think it would be fast enough with the current system to run a Cloud Function before every Firestore read.

Of course we are looking into ways to achieve what you want.  We know that developers want to do more in their security rules than we allow and we are trying to find the best way to do it!  Thanks for the feedback.

- Sam

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/62854b03-5991-4e2b-90da-ad8af9fbd757%40googlegroups.com.

[Rui Faria]

There is no problem of process onRead asynchronously , i know that and i can miss a couple of reads before them are blocked on security rules.

I already have that onWrites,
My concern is not a question of 10-20 reads,my concern is a question of thousands

Even if the onRead are synchronously i must implement the writes on the counter document asynchronously because of the one second write limitation.

The onRead  asynchronously  is the perfect place to do that

in attach:

my current increment code for onWrite

To unsubscribe from this group and stop receiving emails from it, send an email to fireba...@googlegroups.com.

[Paul Renshaw]

What about something like this:

When your user reads documents, increment a counter in the database e.g. (for snapshots)

  const collection = 'things'
const userId = firebase.auth().currentUser.uid

  firebase.firestore().collection(collection).get().then(snapshot => {

    // your existing logic

    firebase.firestore().collection('users').doc(userId)

      .collection('operationCounts').doc(collection).update({reads: firebase.firestore.FieldValue.increment(snapshot.docChanges().length)})

  })

In firestore rules:

match /{collection}/{document} {

  allow read: if get(/databases/$(database)/documents/users/$(request.auth.uid)/operationCounts/$(collection)).data.reads  
    < get(/databases/$(database)/documents/users/$(request.auth.uid)/operationLimits/$(collection)).data.reads

}

I have tried this and it works. As soon as the reads value in /users/{userId}/operationCounts/things exceeds the reads value in /users/{userId}/operationLimits/things an error is throw for insufficient permissions. No background functions needed just values in the database and clever rules.

Maybe also have a scheduled function that resets the operation limits values to 0 every so often

Hope this is helpful.

Regards,

Paul

[Hiranya Jayathilaka]

On Thu, Oct 3, 2019 at 8:41 AM Paul Renshaw <paulr...@time-lapse-systems.co.uk> wrote:

What about something like this:

When your user reads documents, increment a counter in the database e.g. (for snapshots)

  const collection = 'things'
const userId = firebase.auth().currentUser.uid

  firebase.firestore().collection(collection).get().then(snapshot => {

    // your existing logic

    firebase.firestore().collection('users').doc(userId)

      .collection('operationCounts').doc(collection).update({reads: firebase.firestore.FieldValue.increment(snapshot.docChanges().length)})

  })

In firestore rules:

match /{collection}/{document} {

  allow read: if get(/databases/$(database)/documents/users/$(request.auth.uid)/operationCounts/$(collection)).data.reads  
    < get(/databases/$(database)/documents/users/$(request.auth.uid)/operationLimits/$(collection)).data.reads

}

I have tried this and it works. As soon as the reads value in /users/{userId}/operationCounts/things exceeds the reads value in /users/{userId}/operationLimits/things an error is throw for insufficient permissions. No background functions needed just values in the database and clever rules.

This is pretty nifty. But note that it's also fairly easy to abuse. The client has write access to the users and operationCounts collections and therefore can easily change/manipulate their own counts to gain access to the protected content. Also this essentially triples the read volume of the app. Each read results in 2 additional reads to perform the rules check. The only safe and reliable way to implement this use case today is to have a backend middleware layer that acts as a proxy for all database reads.

As for implementing an onRead trigger, I think there are challenges in providing such functionality. Unlike writes, there's no reliable means of detecting if a user has actually read something. Even if a query has executed in the backend, the user still may have not received the data due to a network error. Should that result in the onRead trigger being fired? Also SDKs do a lot of caching, provide offline support etc -- this further muddies up the waters. And then there's the question of asynchrony that Sam has already pointed out. Cloud Functions triggers are retroactive and even a few seconds delay could be disastrous if the correctness of the app depends on it. It's not that implementing this is impossible -- obviously Firestore does its own read counting for quota enforcement purposes. But exposing that logic to developers brings up a lot of questions that are difficult to answer. Just my $0.02.

Thanks,

Hiranya

 

Maybe also have a scheduled function that resets the operation limits values to 0 every so often

Hope this is helpful.

Regards,

Paul

On Tuesday, October 1, 2019 at 7:25:40 AM UTC+1, Rui Faria wrote:

- onRead function trigger

- context.auth.token  in function triggers

In my case i use one firestore account to provide services to my customers that have their accounts, and i need to control them individually.

if i have any type of problem or excessive use by a customer i cannot suspend all firestore account and block all the customers.

In most cases i use live queries and i cannot replace by generic google functions or api in the middle.

Using custom tokens,onWrite function trigger and security roles i have the control of writes.

without onRead function triggers i cannot have the control of the reads.

using onRead Triggers/auth.token we developers can have a more detailed control of reads implement read counters and block access to regions/cities or even IPs

using onRead triggers we developers can sleep well without afraid of huge bills that can kill our business

Please help me pushing firestore to implement:

- onRead function trigger

- context.auth.token  in function triggers

Rui Faria

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/cfbdc1e7-5a49-4094-b480-2283f30884b4%40googlegroups.com.

--

Hiranya Jayathilaka | Software Engineer | h...@google.com | 650-203-0128

[Joaquim Guerra]

Hello guys,

Do you have any news about this feature request?

I understand this trigger is not a good place to block a read request but it would be a nice feature to anticipate attacks which can cause huge amounts of Reads and, consequently, huge bills.

Without a mechanism like this, I can't rely 100% putting a service Online.

I understand Paul's suggestion but I would prefer to do the countings on the server side instead.

Thanks.

Joaquim Guerra

To unsubscribe from this group and stop receiving emails from it, send an email to fireba...@googlegroups.com.

[Sam Stern]

Hi Joaqim,

This feature is not likely to come to Firestore in the near future.  Maybe you could use Firestore's StackDriver monitoring integration to detect abnormal read/write volume?   

- Sam

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/e874fbe8-8466-4011-99f5-ec6440a0a5c3%40googlegroups.com.

[Joaquim Guerra]

Hi Sam,

Thanks for your answer.

I understand the possibilities with your Monitoring platform but, with that, I can't see an option to monitor and block access to a specific user account.

We can get an alert an stop the entire service but we don't have the option to analyse the usage per user.

Do you have a solution to, programmatically, trace the Reads per account and block individual users?

Thanks.

Joaquim Guerra

segunda-feira, 24 de Fevereiro de 2020 às 16:57:47 UTC, Samuel Stern escreveu:

Hi Joaqim,

This feature is not likely to come to Firestore in the near future.  Maybe you could use Firestore's StackDriver monitoring integration to detect abnormal read/write volume?   

- Sam

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/e874fbe8-8466-4011-99f5-ec6440a0a5c3%40googlegroups.com.

[Sam Stern]

Hi Joaquim,

I appreciate the feedback and I agree with you, Firestore is definitely missing this feature.  Assuming you are accessing your Firestore data directly from your client application (instead of putting an API in front of it) there is no great way to detect and limit per-user reads.   

One thing I would suggest is using Security Rules to limit bad behavior.  First make sure that your rules require users to be authenticated, which will prevent major types of request spamming.  Also in rules you can check rules.query.limit to get the .limit(x) part of incoming queries.   You could mandate that all incoming queries add a limit clause of some reasonable size, maybe 100 documents on most collections?  Then even a theoretically sophisticated spammer who manages to steal a user's auth token (unlikely) will be not be able to easily read huge amounts of data from your database.  Instead they'd have to make many small requests or they will get a permission denied error (which will not tell them why so they'd have to guess).  I think at this point most spammers would look for a different way to bother you.

Hope that's helpful!

- Sam

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/96635314-db3e-4d0e-883a-d52544f12878%40googlegroups.com.

[abdullah abrahams]

When your user reads (gets) a document, increment a counter in the document.

// for example, from your client

firebase.firestore().collection(collection).get().then(snapshot => {

    // your existing logic

    // then call an http cloud function with the doc id for example

  http.get(`https://us-central1-${url}.cloudfunctions.net/functionName?doc_id=${doc_id}`)

}

// and then in the cloud function something like this

export const functionName = https.onRequest(async (req, response) => {
    // ref to the parent document
    const docRef = firestore().collection('"collection').doc(req.query.doc_id);

   return userRef.update({views: firebase.firestore.FieldValue.increment(1)})

     .then(()=> {

         // more logic

     })
 }