firestore projection

[Martin Kuhn]

Is it possible to do projections with firestore (specifying which fields are returned by a query)?

[Samuel Stern]

Hi Martin,

This is not yet possible using the mobile SDKs (Android, iOS, and Web) but it is possible using the select() method of the server-side SDKs, here's the reference docs for select() in Node.js:

https://cloud.google.com/nodejs/docs/reference/firestore/0.8.x/Query#select

So for example:

db.collection("users").where("age", ">=", 13).select("name", "age").get()

- Sam

On Wed, Oct 18, 2017 at 7:44 AM Martin Kuhn <marti...@gmx.at> wrote:

Is it possible to do projections with firestore (specifying which fields are returned by a query)?

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/ae90acb8-4bdd-4783-ae70-e93e9808262f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kristian]

Hello. I am very interested in the select method. Does select avoid having to download fields not specified? Or is it filtering on the client?

Also, are there plans to add select to the mobile SDKs? I am curious as to why it is not available yet. I looked at the REST API and I would think you could make a REST call using a DocumentMask to achieve the same result.

Thanks,

Kristian

[Michael Lehenbauer]

select() avoids downloading the extra fields.  There aren't concrete plans to add it to the mobile SDKs at this time. You're right that the mobile SDK could send the DocumentMask to achieve the same result, but since the mobile clients are designed to cache server data and work offline things become problematic if they don't have the entire document contents.  For instance if you did db.collection('users').select("name").orderBy("age").get(), the client would not know how to order the documents it got from the server, since the "age" field was not included.

If you have a specific use case you're trying to solve, I'd be curious to here about it. In particular I'd like to know what you're trying to achieve with select(), and if you would want the query to work while offline, with latency compensated local writes, etc. In the future we may explore ways to provide similar functionality (perhaps only in specific circumstances) in the mobile SDKs to achieve similar results.

Thanks,

-Michael

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/49126bc4-dead-48f7-9500-4f797269a47f%40googlegroups.com.

[Kristian]

Hello Michael,

Thank you for the response. My use case is a chat application on a mobile device. In the application, I want to have a screen that gives a summary of unread messages for a given user. I have tried different approaches but this query has always been a challenge.

Using firestore's new better queries and rules, I came up with the following query and structure:

There is a collection of all messages. Each message document contains an 'isUnread' map that has every user id that has that message as unread. The value of user id key is the creation date of the message. 

So a message would look like this:

message {

  text: 'Hello. This is a message',

  createdAt: 1506020743055,

  isUnread: {

    QLGqZs526rf5Bj6g1lRqzhJq0e82: 1506020743055,

    H8azi90yo4WovxUD4l22Pj2feCc2: 1506020743055,

    ...

  },

}

And the query would look like this:

// Getting all messages that are unread for a user in one one fetch:

db.collection('messages').where(`isUnread.${userId}`, '>', 0).orderBy(`isUnread.${userId}`).get()

Now my concern about this, is that the size of message documents scales with the number of users. It is still a single fetch so, I expect it to stay fast. But the main concern becomes data usage on mobile devices. Now if I could use select to not have to download the 'isUnread' map and just have the server do filtering I would not have to worry about this scaling. Now I can understand how this might mess up caching, due to the client not having the 'isUnread' map but I think I would be okay with this. Also offline and local writes are not necessarily a concern for me.

I tried other approaches in the past:

I found that a single fetch works much better than multiple round trips. The performance is okay on web but suffers greatly on react native (I think this was due to the react native bridge being invoked for http requests). To have a single fetch on old firebase, I denormalized message data so that there is a unread messages path for each user with the message data duplicated. The problem with this approach is that every time someone writes a message it has be written for each user. Again the main concern is the data usage on mobile devices. There could be hundreds of users for a chat. The firestore approach should have better scaling.

Do you have any suggestions?

Thanks for you time,

Kristian

[Michael Lehenbauer]

Thanks for explaining Kristian,

Keeping accurate up-to-date per-user "unread messages" counts seems challenging and I don't have any magic suggestions. I'd normally expect to just calculate it on demand (do a query to find unread messages and count the results) though I understand that would be more expensive. We'll keep your select() / bandwidth use case in mind for the future.

-Michael

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/80afd021-6481-4f74-bfcb-ef1f35cf19f4%40googlegroups.com.

[dema...@sfeir.com]

I also need a select() feature for a PWA app.

In my application I have a collection of posts with { title, desc, content, date } and I just want to display a summary of recent posts, so I just want to display { title, desc, date } in the listing page and { title, content, date } in the post page.

This is important for performance on mobile app because { content } may be bulky.

[swftvsn]

Our use case is pretty close: we have one property named "search" that is used to hold a lot of denormalized data that is only used to filter documents in our advanced search. It's completely unnecessary and waste of bandwidth to download that (the size of the search entry is 10 - 100x the size of the actual data). Thus it would be nice to be able to say that 

a) for this query, I'm willing to bypass offline support
b) for the query mentioned in a), please bring home only fields x, y and z

[Michael Lehenbauer]

Hey all,

Thanks for the use cases!  I may reach out for more details in the future, but just FYI- I think the current workarounds are:

1. Downloading the entire document (taking the bandwidth hit).
2. Splitting the document into the "light" part and the "heavy" part so you can optimize for bandwidth. You could store the "heavy" part (content, search data, etc.) in a separate collection and read it on-demand.

We'll let you know if the situation changes in the future.

Thanks!

-Michael

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/d68c3530-125b-40ef-abb9-a2824aa14f0c%40googlegroups.com.