If you're seeing more users in the experiment because you have more users and they match your targeting conditions, that's expected and the results should still be accurate. However, let's dig in a little bit:
- Are you seeing this in the Firebase console, via inspecting BigQuery results
, or both?
- What targeting conditions are you using/what percentile/how are you assigning the users to experiment groups?
Also note that "users" are assigned to experiment variants according to a hash of the experiment ID and the user’s Firebase installation ID
, so you could have one user (person), but three experiment members (installation IDs) if the same user accesses your app using three different devices. If you're not doing so already, you might want to look at user audience or user properties to control experiment membership (note that there's some latency for new audiences and for user audience enrollment
If these extra users don't match the targeting conditions you set and aren't the result of different installation IDs, can you send more detail, including project, experiment ID, targeting conditions, and any other relevant detail that would help the team troubleshoot to Firebase Support
) so that our team can take a closer look?
Thanks a bunch in advance!