Efficient Fetching for Denormalized Data?

[VAlexander]

Hi,

I'm working on an iOS chat app and and have denormalized my data in accordance to what the Firebase docs recommend. I've ran into a bit of a problem though. Correct me if I'm wrong, but it seems that I'd have to make a fetch request for every single chat object I need to download (that seems highly inefficient). Here's an image of what my data structure looks like when I try to fetch everything in one go (no listeners): 

The first thing I do is given a userId I fetch all the groups the user belongs to (requires 1 call). I then store these groupIds in an array and using a for-loop I fetch each chat using the supplied groupId (1 call for every chat object). Is there I way I can query using something like `queryContainedInArray`. I feel that would solve the problem. I'm open to any suggestions or improvements. Thanks!

//Setup a Firebase Reference
Firebase *firebase = [[Firebase alloc] initWithUrl:kFirebaseBaseURL];

//Retrieve All The Chat's a User is a Member of
NSString *pathString = [NSString stringWithFormat:@"members/%@/groups", user.objectId];
[[firebase childByAppendingPath:pathString] observeSingleEventOfType:FEventTypeValue withBlock:^(FDataSnapshot *snapshot) {

   //Get Chat Ids
   NSArray *chatIds = [(NSDictionary *)snapshot.value allKeys];
   NSMutableArray *chats = [NSMutableArray new];

   //Fetch All Chats
   __block BOOL exitLoop = NO;

   //Fetch Each Chat
   for (NSString *chatId in chatIds) {

     //Helper Method to Fetch Chat
     [self fetchChatWithId:chatId inBackgroundWithBlock:^(Chat *chat, BOOL succeeded, NSError *error) {
 
       if (!error) {

         //Add Chat to Array
         [chats addObject:chat];

       } else {

         //Exit Loop
         exitLoop = YES;
       }
     }];
 
    if (exitLoop) {
 
       //Remove All Objects (Incomplete)
       [chats removeAllObjects];
 
       //Exit Loop
       break;
     }
   }

   //Return block
   block(chats, YES, nil);
}];

[VAlexander]

Now that I'm thinking about this... would the best solution be to just pull in all the chats, and use security rules to only return chats which the member is a part of?

[Frank van Puffelen]

That won't work. 

Firebase security rules work on the location where you execute the read/write operation.

So if you try to read /chats and a user only has access to specific chat rooms, the operation will fail.

See rules are not filters in the documentation and this eerily similar question on StackOverflow today.

[VAlexander]

Interesting. Thanks Frank. In that case, it leads me back to my original question. Is there a more efficient way to pull in an array of objects when the data is denormalized? I did read the following on Firebase:

Is it really okay to look up each record individually? Yes. The Firebase protocol uses web sockets, and the client libraries do a great deal of internal optimization of incoming and outgoing requests. Until we get into tens of thousands of records, this approach is perfectly reasonable. In fact, the time required to download the data (i.e. the byte count) eclipses any other concerns regarding connection overhead.

That definitely answers some of it however, what would one do when you do get into the tens of thousands of records? What would be the recommended approach then? Perhaps I'm wrong, but I really do feel that the ability to query for any value within a provided array would be the way to go (something like queryEqualToValueInArray:). Thoughts?  

Thanks for your time!

[Frank van Puffelen]

What type of application are you building?