Firebase A/B test retention rate between platforms

Skip to first unread message

Matt King

May 9, 2022, 10:30:44 AMMay 9
to Firebase Google Group
We've been running Android and iOS A/B tests with Firebase successfully for the past year or so, but we've noticed an anomaly recent tests.

Normally our 4-7 day retention rate for A/B tests are consistently in the 30-40% range. It is usually similar across both iOS and Android. We ran a new experiment and iOS has dropped to 1-2% while Android remained at the expected rate.

We thought it might have been an implementation issue, so we just ran a quick A/A test across all users on both platforms, and iOS still remained at 1-2%.

Has anyone else noticed this in their experiments?

Chip is a trading name of Chip Financial Limited. Chip is authorised by the Financial Conduct Authority under the Payment Service Regulations 2017, firm reference number no. 911255, for the provision of payment services. 
‍Note that Chip is not a bank. Only the money held in the Chip Savings Account powered by Clearbank (Chip Savings Account), our Interest Accounts and the Chip+1 account are covered by the Financial Services Compensation Scheme (FSCS) for deposits. To find out more and to check your eligibility please visit: 
Chip is an agent of Prepaid Financial Services Limited. Prepaid Financial Services is authorised by the Financial Conduct Authority under the Electronic Money Regulations 2011, firm reference number 900036, for the issuance of electronic money and provision of payment services. Registered Office: Fifth Floor, Langham House, 302-308 Regent Street, London, W1B 3AT. Funds held in your e-money wallet are not covered by FSCS. This money is safeguarded with Prepaid Financial Services Limited.

For our Investment Platform business only Chip is an Appointed Representative of P1 Investment Management Ltd, which is authorised and regulated by the Financial Conduct Authority, under Firm Reference Number 752005.
Your money may be protected up to £85,000 by the Financial Services Compensation Scheme (FSCS) under their Investment Scheme. This means that in the event Chip Financial Ltd is declared to be in default and cannot meet their obligations, you may be able to claim compensation. It’s important to understand that the FSCS doesn’t cover you in the event that your investments go down and you get back less than what you put in. Further information is available from the FSCS website (
Our registered office is: Chip Financial Ltd, 7 Bell Yard, London, WC2A 2JR

Reply all
Reply to author
0 new messages