Valid use case for Firestore?

[Tyler Turner]

I've been building my web application with Firestore, and I'm concerned that my approach will not be cost-effective or user-friendly because of the number of read operations required to piece together data from different collections. The application allows users to upload songs which are rendered as music notation for other users to play along with and record. Users share their recordings with the community, and community members can then combine these performances together into mixes which are also shared with the community. The data has a lot of relationship associations, so that for example when accessing a song, we read a list of associated performances and mixes, which in turn have a list of associated users. Without duplicating data across collections (ie. including a user's profile pic URL and display name in a performance document), I can see where a single user could generate hundreds of Firestore reads in a single visit.

So I guess my question is whether there are best practices for DB design I could employ to work with this type of connected data, or if I'm really doing something that shouldn't be done with Firestore or NoSQL in general. Is it normal to do a lot of data duplication or caching in some other medium for use cases like this? I'm very new to this. I also apologize if this belongs on Stack Overflow - I thought it might be too opinion-sounding for that.

[Kato Richardson]

Hi Tyler,

If a single user did generate hundreds of reads per visit (which could most likely be optimized considerably), and visited twice a day, and you had half a million users, that would come out to 100M reads a month. 

The pricing calculator says that $60 / month for 500k MAU. What's so unreasonable?

One thing to consider is what content is static and what content is dynamic and changes in near realtime. For the static content, you should probably offload that to Google Cloud Storage and just keep meta info (that changes frequently) in Firestore.

☼, Kato

On Thu, Apr 12, 2018 at 12:53 PM, 'Tyler Turner' via Firebase Google Group <fireba...@googlegroups.com> wrote:

I've been building my web application with Firestore, and I'm concerned that my approach will not be cost-effective or user-friendly because of the number of read operations required to piece together data from different collections. The application allows users to upload songs which are rendered as music notation for other users to play along with and record. Users share their recordings with the community, and community members can then combine these performances together into mixes which are also shared with the community. The data has a lot of relationship associations, so that for example when accessing a song, we read a list of associated performances and mixes, which in turn have a list of associated users. Without duplicating data across collections (ie. including a user's profile pic URL and display name in a performance document), I can see where a single user could generate hundreds of Firestore reads in a single visit.

So I guess my question is whether there are best practices for DB design I could employ to work with this type of connected data, or if I'm really doing something that shouldn't be done with Firestore or NoSQL in general. Is it normal to do a lot of data duplication or caching in some other medium for use cases like this? I'm very new to this. I also apologize if this belongs on Stack Overflow - I thought it might be too opinion-sounding for that.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/37c9e2d1-227d-4356-af8b-867162a69f4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Tyler Turner]

Thanks for the reply! I might be completely misunderstanding the pricing. With a million visits per day, each generating 100 reads, my understanding was that I'd be charged for 100 million reads per day, and thus ~3 billion reads per month (around $1800). Do I have this wrong? The pricing calculator confused me a little since it seems to be in terms of the month while the actual price listing was by day. To be sure, I don't anticipate being this successful with my website. :-)

I can definitely see opportunities to cache certain content in storage. I'll try to think how to approach that. Thanks again!

On Friday, April 13, 2018 at 1:27:10 PM UTC-6, Kato Richardson wrote:

Hi Tyler,

If a single user did generate hundreds of reads per visit (which could most likely be optimized considerably), and visited twice a day, and you had half a million users, that would come out to 100M reads a month. 

The pricing calculator says that $60 / month for 500k MAU. What's so unreasonable?

One thing to consider is what content is static and what content is dynamic and changes in near realtime. For the static content, you should probably offload that to Google Cloud Storage and just keep meta info (that changes frequently) in Firestore.

☼, Kato

On Thu, Apr 12, 2018 at 12:53 PM, 'Tyler Turner' via Firebase Google Group <fireba...@googlegroups.com> wrote:

I've been building my web application with Firestore, and I'm concerned that my approach will not be cost-effective or user-friendly because of the number of read operations required to piece together data from different collections. The application allows users to upload songs which are rendered as music notation for other users to play along with and record. Users share their recordings with the community, and community members can then combine these performances together into mixes which are also shared with the community. The data has a lot of relationship associations, so that for example when accessing a song, we read a list of associated performances and mixes, which in turn have a list of associated users. Without duplicating data across collections (ie. including a user's profile pic URL and display name in a performance document), I can see where a single user could generate hundreds of Firestore reads in a single visit.

So I guess my question is whether there are best practices for DB design I could employ to work with this type of connected data, or if I'm really doing something that shouldn't be done with Firestore or NoSQL in general. Is it normal to do a lot of data duplication or caching in some other medium for use cases like this? I'm very new to this. I also apologize if this belongs on Stack Overflow - I thought it might be too opinion-sounding for that.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/37c9e2d1-227d-4356-af8b-867162a69f4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joel Margolese]

You also mention: " Is it normal to do a lot of data duplication or caching in some other medium for use cases like this?"  To cut down on traffic you may want to duplicate data within Cloud Firestore.  I'm not sure if it's viable, but if you create some collections with duplicated data you may be able to cut down on the reads without having to use Cloud Storage for storing data.  This does increase effort in maintaining the data and may or may not be worth it.

[Tyler Turner]

Thanks, that helps. I have a better idea now how to evaluate this.