Remote Config A/B Testing: How does a user get assigned to a non-overlapping user group and how long for?

[Ronak Badheka]

How long a user will be stay in a particular non-overlapping user group? Will the random allocation take place one more time for every single users when value conditions and/or priorities are changed? 

Because we are using Remote Config to set up an A/B testing experiment. We have three test groups for Remote Config parameter name MyExperiment:

Parameter Name:

MyExperiment

Value Conditions:

test-A : Users in random percentile <= 33%

test-B : Users in random percentile <=66% 

control : default.

Scenario-1:

Now based on this when the first fetch call is made to get Remote Config a user gets assigned in test-B user group based on random percentile allocation.  What happens to that user when the next fetch call is made? Does the random allocation take place one more time for the same user or that user will continue to be a part of the test-B user group even when the further fetch calls are made?

Scenario-2:

Now let's we have updated the value conditions on Firebase > Remote Config console. Now they look like following. 

Value Conditions:

test-A : Users in random percentile <= 66%

test-B : Users in random percentile <=100% 

control : default.

So what happens in this case to that same user who was previously a part of test-B user group? Will he be assigned a new user group eg. test-A? There is a 66% chance after all. 

I'm trying get clarity around how long a user will continue to be a part one testing user group and what happens when the values are updated on the Firebase console. Please explain! 

Thank you.

[Safa Alai]

1.  Random percentiles are based on a random seed.  The seed stays stable while any random percentile conditions are present.  If a template is published with no random percentiles, the next set of percentiles will use a new seed and be completely unrelated.

2.  As you have specified here there is actually overlap between test-A & test-B --i.e. it is possible to have both be true.  To avoid this specify a lower bound for test-B

     e.g. test-B Users in random percentile > 33% AND Users in random percentile <66%