Standard vs Normalized metrics

[Thibault Lbt]

Hi there,

I was wondering what is the difference between Normalized Good LCP and Good LCP? 

I've noticed each metric in the CrUX report has its "Normalized" equivalent (good/NI/bad LCP, CLS and FID) but not sure which one I should use (although I don't see a lot of difference between each pair)?

Thanks!

[Rick Viscomi]

Data in the CrUX Dashboard is segmented by device (desktop, phone, tablet) and the experiences for any given metric always add up 100%.

So for example an origin might have its LCP performance segmented like this:

Device	Good	Needs Improvement	Poor
desktop	50%	20%	10%
phone	0%	10%	10%
tablet	0%	0%	0%

The sum of all percentages is 100%.

80% of experiences are on desktop, 20% on phone, and 0% on tablet.

50% of experiences are on desktop and have good LCP performance. This is Good LCP in the dashboard.

But the way the data is shown in the dashboard is to normalize the results depending on the devices that are filtered. You could opt to look at only desktop and tablet experiences, only phone experiences, all devices combined, etc. In order to handle any arbitrary grouping, we need to dynamically normalize the results.

To calculate Normalized Good LCP, we use this formula: SUM(fast_lcp) / (SUM(fast_lcp) + SUM(avg_lcp) + SUM(slow_lcp))

So if you're filtering for desktop and tablet experiences, the Normalized Good LCP is calculated as: (50+0) / ((50+0) + (20+0) + (10+0)) = 50 / 80 = 63%. In other words, 63% of experiences on desktop or tablet devices have good LCP.

For all devices combined, the Normalized Good LCP comes out to 50 / (50+20+10+10+10) or 50%. So if you haven't filtered out any devices, it should be equivalent to the non-normalized Good LCP. Try adding a device filter to the dashboard to explore how it works.

Out of curiosity, are you exploring the underlying metrics because the default dashboard is lacking features you need?

--
You received this message because you are subscribed to the Google Groups "Chrome UX Report (Discussions)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ux-repo...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ux-report/4fade184-2067-484f-9f5a-37bd2b8a418en%40chromium.org.

[Thibault Lbt]

Hi Rick,

Thanks for taking the time to explain the logic, that makes a lot more sense now! I'm working on a gTLD where local country sites are organized into subfolders, so I've to dive into the BigQuery data...

Luckily I found your query here which will be helpful to recreate the report. I understand that what is refered as pct_good_lcp is in fact "Normalized Good LCP". But now I'm wondering about the p75_ fields there: are they corresponding to "P75" or "P75 (All Devices)" in the Data Studio report?

Thanks!

[Rick Viscomi]

Ah ok, thanks for clarifying.

The p75 metrics in the country_summary table reflect the device-specific aggregations, specifically.

If you navigate to the chrome-ux-report.materialized.device_query "view" on BigQuery, on the details page you can see the exact query used to generate the device_summary table used by the CrUX Dashboard. (BigQuery is being weird and not letting me create a permalink to the view, sorry)

Hope that helps.

[Thibault Lbt]

Cool, that's great to know! Is there any reason why p75 in the default DS report doesn't use this device-specific aggregations (as it's stated here "toggling specific devices with the Device filter will not recalculate the percentile")? Just being curious...

Thanks again.

[Rick Viscomi]

Data for the CrUX Dashboard is sourced from the `chrome-ux-report.materialized.device_summary` table, which contains the origin-level p75 (used by the dashboard) as well as device-specific p75 values. The reason why we can't use the latter is because the dashboard allows you to select not only individual devices but arbitrary combinations of devices (phone+desktop, tablet+phone, etc). The summary tables are only aggregated at either the origin-level or the device-level, and we don't have aggregations for subsets of devices. Doing that is technically possible but would add complexity, and it's unclear how valuable that information would be to users.

[Thibault Lbt]

Ah got it, so you had to choose between either letting people visualize device-specific p75 values with a warning that it would not be correct for arbitrary combinations or using origin-level p75 with a warning that the device filter will not recalculate the percentile... I agree this information isn't valuable for subsets of devices (e.g. desktop+tablet...) but still valuable for desktop, tablet or mobile only. Although I understand that the first option is safer as it doesn't let people output wrong values.

I'm making some progress in replicating the DS report with country breakdown, now getting something like this in BQ:

On the Data Studio side, is it then correct to set aggregation to Average for normalized_ and p75_ metrics (whereas in the default DS report, normalized_ fields are set to Sum and p75_ to Median...)?

[Shimaa El-Srougy]

Hi All, 

      Great thread. I wanted to ask you guys is there any way to calculate the Median p50 instead of p75. I am using the API method and was trying to calculate it. Thank you in advance for the help.

Shimaa