Finch alert: Study BeforeunloadEventCancelByPreventDefault - heartbeat metrics differences first seen on 2023-07-17

[finch...@google.com]

Chirp thinks your experiment is having an effect.
Analysis based on 3 days' data from 2023-07-15 to 2023-07-17

WebVitals.CumulativeLayoutShift3 (0.5 quantile)

Platform	Experiments	Quantile	Change	Effect size	Samples (approx.)	Test
M117 on Mac (CANARY)	Enabled vs Control	50	⇘12 (22 +/- 21) scorex10000	big	381 k	T-test with jackknife resampling

Statistical significance: p=3.33e-03 or lower across all results.

You can update this incident to associate it with a bug or snooze it.

Feedback?

---

Terminology explained...

• Samples (approx.): Approximate number of observations used in the test. Note that usually each client contributes many observations each day.
• Change: How much the observed variable changed between groups in absolute terms. The median and deviation of the entire population are given in parentheses for reference.
• Effect size: It is a relative measure of the strength of the relationship between group assignment (the independent variable here) and the metric we look at (the dependent variable).
• Test: The statistical hypothesis test used.
• Warning! Chirp will not alert again for a week unless something changes.