Feat Request: Unique Value in Real Time Database

[Sebastian]

I think it would be really helpful if we could do something like 

data.val().unique()

in the database rules, to enforce a field to be unique. 

[Jacob Wenger]

Hey Sebastian,

If you want to enforce a field to be unique, you can store those values as the keys to a node which all map to true, like this:

{

  "usernames": {

    "fred": true,

    "barney": true,

    "dino": true,

    // ...

  }

}

And then, in your Security Rules, you can enforce that the existing username cannot be overwritten:

{

  "usernames": {

    "$username": {

      ".write": "!data.exists()"

    }

  }

}

You also can check from the client if a username is already in use:

var username = "fred";

firebase.database.ref("usernames").child(username).once("value", function(snapshot) {

  if (snapshot.exists()) {

    console.log("Username already in use");

  } else {

    console.log("Username is not in use");

  }

});

Hope that helps!

Jacob

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/f0b93be2-35a6-43dd-86bd-f88a87dfaa3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sebastian]

Hey Jacob

I'm absolutely aware of how we recommend it to solve it currently. Still considering the overall ease of using firebase, I find this particular issue really cumbersome in comparison.

For example, when the user wants to change their username, we need to update this in both places (given you have a node 'users' and a node 'usernames' as in your example). We also can't update a key, so we have to copy its values, delete the key, and create a new one. All this, while basicly every database out there has some kind of "uniqueness" flag in their schema.

Since you guys added exists(), I thought it could make sense to have unique() as well.

Thanks for your swift response in either case!

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

[Sebastian]

Can you imagine something like this in comparison

"profiles": {
  ".read": true,
  ".write": "auth !== null",
  "$uid": {
   "username": {
     ".validate": "data.unique()"
    }
  }
}

On Tuesday, March 21, 2017 at 5:40:08 PM UTC+1, Jacob Wenger wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

[Kato Richardson]

Hi Sebastien,

Great stuff and thanks for the feedback!

while basicly every database out there has some kind of "uniqueness" flag in their schema.

Ah, well here's the rub. As I see it, this would mean keeping a map of all nodes in the database, keyed by each unique field, and checking this map on every write operation. Besides being slow to parse and prepare this index at scale (keep in mind that we're talking realtime data, delivered in sub 100ms to gadzooks numbers of listeners, with consistency guarantees), consider also that at scale you usually want to keep collections under 10k-ish in size, so you're going to be segmenting data across paths by month/year/group or some similar archiving criterium, and now uniqueness becomes suddenly very proprietary and complex (i.e. something you're probably best defining yourself rather than trying to do via some sort of baked-in solution).

I'd love to hear your thoughts on solving those complexities and, if you have examples of realtime, scalable dbs that have succeeded here, while providing a simple and straightforward API to do so, I'd love to hear thoughts about how it was done.

☼, Kato

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/361e5966-3d18-4572-90f2-b03fca886bad%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Marek Olszewski]

Hi Kato,

Thanks for your response. 

I'm still at a loss at how to write all the security rules on the database side to safely enforce a unique username in the presence of a malicious client. In Jacob's response, how do you prevent a malicious or misbehaving client from writing a huge amount of random usernames to the 'usernames' node, effectively claiming them and preventing others from registering using their desired username?

As far as I can tell, even if you make that node map to a userid, you can't enforce that only one field exists with that mapping because, once again, you would need to guarantee uniqueness on the userid field.

Am I missing something here?

Best,

Marek

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/361e5966-3d18-4572-90f2-b03fca886bad%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Kato Richardson]

Well, if simple is the goal, using Functions can make this very straightforward:

// Listens for changes to /users/<user id>/username and stores them in an index
// for testing uniqueness
exports.indexUsernames = functions.database.ref('/users/{userId}/username')
    .onWrite(event => {
       // Grab the current username written to the /users path
       const username = event.data.val();
       const uid = event.params.userId;

       // store it in the index
       const indexRef = event.data.adminRef.parent.parent.parent.child('username_index').child(username);
       return indexRef.set(uid);
    });

Now my security rules can simply consist of something like this, depending on your specific needs:

"users": {
   "$userid": {
       "username": {
           ".validate": "newData.val() === data.val() || !root.child('username_index').child(newData.val()).exists()"
       }
   }
}

You may also want to consider how much effort you need to put into perfection here. There’s a lot you can do to enforce username uniqueness and most of it’s unnecessary unless there’s actual benefit to gaming the system (i.e. it’s Twitter or Candy Crush).

If you’re in need of more and really determined to make this a production, you can add the following to make sure people can’t write multiple usernames, based on Jacob’s rules:

"users": {
   "$userid": {
       "username": {
           // I cannot change my username, preventing me from creating multiple usernames in the db
           ".validate": "newData.val() === data.val() || (!data.exists() && !root.child('username_index').child(newData.val()).exists())"
       }
   }
}

"username_index": {
   "$username": {
       // I can only write my own username to the index
       ".validate": "root.child("users").child(auth.uid).child('username').val() === $username && newData.val() === auth.uid"
   }
}

Hopefully you can work the rest out from here.

☼, Kato

​

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/5973444d-9461-4cfc-a967-047b5db93f28%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Marek Olszewski]

Hi Kato,

This is very helpful. I didn't think of using cloud functions, but they obviously would be very helpful.

Are you certain your solution is not prone to race conditions though? Couldn't both writes to the username field in the user node succeed, and then have one of the cloud functions fail to complete the second write to the username_index node? Or, are the cloud functions executed atomically with the original writes?

Also, with regards to locking things down further, I'm hoping to find a solution that allows people to change their username as well, so long as it's unique. Once you allow that, I'm struggling to find a way to prevent a malicious client from claiming more than one username.

I know that it's unlikely that anyone will maliciously try to claim usernames for a small application. I'm just using usernames as an example here. I'm really interested in the correct way to create any non-key unique field (that can be modified) in Firebase as it comes up a lot and nobody has a good answer when I google it. I'm hoping we can come up with a robust template that anyone can use for usernames or any other data. Unique non-key fields come up frequently in my experience and judging by even this thread, they are tricky to get right in Firebase. 

Best,

Marek

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/5973444d-9461-4cfc-a967-047b5db93f28%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Kato Richardson]

 > Are you certain your solution is not prone to race conditions though?

Since the username can only be set once, according to the rules example I provided, and the function only triggers after it is written, not sure what you mean by race conditions. 

 

> Or, are the cloud functions executed atomically with the original writes?

No, they are triggered by the original write.

> Also, with regards to locking things down further, I'm hoping to find a solution that allows people to change their username as well, so long as it's unique.

I think you need to pick one. They can have multiple usernames or they can't. Also, what happens after I change a username? Does the old one become available for others to use? If so, spoofing is a problem here. This gets very complex, very quickly. Allow multiples or prevent creating multiples, but don't attempt both unless you have some specific, highly proprietary needs here.

But yes, you could do something like allow them a certain number of usernames and track how many they've created, or even doing a rolling window and allow them based on time period. All possible, some assembly required. Well, lots of assembly.

 

> I'm really interested in the correct way to create any non-key unique field (that can be modified) in Firebase

The rules above are the way to do this. Functions will make it simpler. Again, some assembly required as almost every use case is slightly different and proprietary to your app.

[Marek Olszewski]

On Friday, April 21, 2017 at 11:57:32 AM UTC-7, Kato Richardson wrote:

 > Are you certain your solution is not prone to race conditions though?

Since the username can only be set once, according to the rules example I provided, and the function only triggers after it is written, not sure what you mean by race conditions. 

I think a race condition could occur here if two people try to simultaneously claim the same username at the same time. Both would successfully write their usernames to their respective user nodes, before each of the two cloud function invocations have the time to update the username_index node. If I'm right, I don't believe your solution above is correct and you may not want to recommend it to Firebase users.

 

> Also, with regards to locking things down further, I'm hoping to find a solution that allows people to change their username as well, so long as it's unique.

I think you need to pick one. They can have multiple usernames or they can't. Also, what happens after I change a username? Does the old one become available for others to use? If so, spoofing is a problem here. This gets very complex, very quickly. Allow multiples or prevent creating multiples, but don't attempt both unless you have some specific, highly proprietary needs here.

But yes, you could do something like allow them a certain number of usernames and track how many they've created, or even doing a rolling window and allow them based on time period. All possible, some assembly required. Well, lots of assembly.

Changing usernames and letting other people re-use previously used usernames is pretty standard, and would be trivial if Firebase supported a unique database rule as suggested above. I know it's obviously not a trivial thing to add and would have performance implications, but unless you can give a correct workaround, it seems like a pretty big omission. Again, I've seen many non-username use cases for unique non-key fields, so I wouldn't be surprised if this topic kept coming up on this group.

That said, I'm very new to Firebase so perhaps there still is a way to do it. I really don't mind 'some assembly required'. I'm just looking for a correct workaround.

 

[Kato Richardson]

Marek,

You can add a !data.exists() into the username index to prevent it from being overwritten by other users who might try to write the same name at the same time (unlikely). That would have been good to add to the example. I didn't solve for all the edge cases. An error here seems plenty fine, given the rarity that two people would type in the same username at the same time--something like "couldn't claim that username, try another one" is probably plenty.

You should of course verify in the UI that the username doesn't exist before submitting for write and probably check that and throw an error in the function/process, and consider any other caveats that might come up in the workflow.

☼, Kato

--

You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/e79244c7-07b1-454f-9968-2c9e17472a9d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.