The firebase A/B test doesn't run when app first open

[Bara Brahmantika]

Hi,

I'm currently using a Firebase A/B and Firebase Remote Config.

Our current app is implementing strategy 3 for loading remote config value. So the fetch happens in async and is activated after the next app launch.

This works for our app most of the time, but I think we ran into a problem when we started to use Firebase A/B.

What I noticed happens when the running experiment is that when we split the group into A and B, where B becomes a treatment group.

When people in the B group open the app, the fetch is happening in the background, so while the value has changed as instructed in the Firebase A/B, the actual UI is not. (UI only updated after the second launch). But what I'm afraid of is that while the UI has not changed, Firebase A/B already considered this group as a treatment group even though they are still receiving and interacting with older UI (the same UI that the control group received)

Is my concern valid? Is Fetching Strategy 3 (async fetch) not compatible with Firebase A/B test?

What are the best practices for this? We are considering changing the implementation to strategy 2, where we show a loading screen and then fetch and activate simultaneously. But unsure how long the time out should be? How long does generally Remote Config response time take? How long is too long that it becomes cumbersome from the user's perspective?

Sincerely,