Firestore Write Limits in Subcollections with CollectionGroupQuery

[John X]

Following up on my previous post learning about write limits to individual collections

Firestore 500+ Writes Per Second to One Collection

How does this 500ps limit apply to CollectionGroups that were recently introduced? (btw...Todd didn't get a loud enough cheer here! :P)

https://firebase.google.com/docs/firestore/query-data/queries#collection-group-query

Firestore limits still only reference individual collections:

https://firebase.google.com/docs/firestore/quotas

Maximum write rate to a collection in which documents contain sequential values in an indexed field: 500 per second

Dan McGrath in my previous post said this is because indexing for Individual Collections

will be limited by the load a single server can handle.

Example data model:

• Collection: users
• Doc: userId1
• Subcollection: posts
• Doc: postId1 (fields for postId, img, caption, date)

If I create a CollectionGroup Index for posts (query across all subcollections called "posts") - order DESCENDING by date

Would my write limit across all those subcollections go down from 10k per second to 500 per second? How is CollectionGroup indexing handled? By the same "individual server"?

If so...this essentially means theres no benefit to putting these "posts" in separate subcollections vs. one individual root level collection from a write limit perspective.

[John G]

I'm interested in this as well. It could dramatically effect how I structure my database. Should be an easy answer??? Google team?

[Sam Stern]

Hi John,

Sorry for the late reply.  As with many limitations of Cloud Firestore, this limit is based on our ability to index data and maintain strong consistency.  So yes: if you have a CollectionGroup index on "date" and your date field has sequential values, you should not exceed 500QPS write rate for updates for "posts" documents.

- Sam

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/506da511-35a3-4593-8624-8425a351fcdf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Petri Kivikangas]

What happens if the limit of 500 writes/s is exceeded? I don't care if a CollectionGroup read query doesn't catch documents immediately after write operations, but if there's a possibility that the write operations fail, I want to know if I need to have some kind of retry logic.

To unsubscribe from this group and stop receiving emails from it, send an email to fireba...@googlegroups.com.

[Kato Richardson]

There is some flex room in the write limits and write bursts will just be delayed. However, at some point, if you're doing consistent writes over 500 per second for something measured in handfuls of minutes, you'll get errors.

If you're concerned with running into the limits, rather than retry logic (which will probably not be effective considering that you'll just be adding to the write volume), you should consider buffering writes at the clients, segmenting data a bit more effectively (by group, discussion, et al), or some other strategy. 

> this essentially means theres no benefit to putting these "posts" in separate subcollections

Should work; if the collections have different names, they do not share an index and wouldn't be subject to the limit.

☼, Kato

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/2267d0e5-adb2-4e04-9515-757ed975dfd6%40googlegroups.com.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398