Discrepancy in LCP between Core Web vitals (GSC) and CrUX

[Jason Lax]

Hi - I haven't found anything in the forum that explain why there can be such a huge difference between the Core Web Vital data I see in Google Search Console Vs. what appears in the CrUX Dashboard in Data Studio.

This is a challenge for us because we have a large web presence and I need accurate data to justify the investments needed to improve our page speed.

If we look at LCP only, here the data I see in both sources:

• Google Search Console > Core Web Vitals > Mobile > LCP
  • Good: 0%
  • Need Improvement: 24%
  • Poor: 76%
• Chrome UX Dashboard > Phone > LCP
  • Good: 33%
  • 23% Need Improvement
  • 44% Poor

The site isn't prone to major swings in performance. I would appreciate some guidance on why there is a discrepancy and what data I should rely on. 

My hypothesis is that Google Search Console is sampling data and weighing it based on traffic whereas the CrUX Dashboard might be looking at all URLs without weighting. 

Here are some screenshots from, first, Google Search Console:

And now the CrUX Dashboard in Data Studio:

Thank you,

Jason

[Gilberto Cocchi]

Hi Jason, nice to e-meet you.

CrUX Dashboard is showing data from Chrome UX Report BigQuery and that's definitively weighted at the whole Origin traffic, not sampled per URL.

You can find more information about how Search Console grouper per URL here https://support.google.com/webmasters/thread/104436075?hl=en.

It's completely expected to see the numbers differ as you see there as they measure different groupings.

I would suggest to use the CrUX API if you want to measure single URLs in CrUX vs Search Console, there might be difference because of URL grouping but there should be a closer match.

--
You received this message because you are subscribed to the Google Groups "Chrome UX Report (Discussions)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ux-repo...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ux-report/5f628a9e-f7ce-400c-b425-22264279604dn%40chromium.org.

--

gTech Up

Gilberto Cocchi Web Ecosystem Consultant gilb...@google.com +39 02 3661 8337

Google Italy | Via Federico Confalonieri 4 | Porta Nuova Isola | Building C | Milan 20124

Registered in Milan, Italy

This email may be confidential and privileged. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

The above terms reflect a potential business arrangement, are provided solely as a basis for further discussion, and are not intended to be and do not constitute a legally binding obligation. No legally binding obligations will be created, implied, or inferred until an agreement in final form is executed in writing by all parties involved.

[Barry Pollard]

Here's some more information that might help you (note I do not work at Google, so this is just based on my experience).

CrUX measures daily at page view level, and at origin level, and this is available daily in the CrUX API (at page level where available, and also origin level).

Google Search Console attempts to map these page views to a score for each page, for pages indexed by Google based on what 75% of your users get for that page.

The CrUX dashboard is based on the monthly dump of CrUX Origin data only to BigQuery, again based on page views.

So, in your example only 33.05% of page views on mobile get a good rating at origin level, and 23.05% get a needs improvement rating at origin level. Assuming all pages are equal and so all got these same stats (which is a BIG assumption but go with me here for a bit), then I don't think it's surprising what you see in Google Search Console. You don't have 75% of your page views above the "needs improvement" nor "good" thresholds therefore at a page level (instead of a page view level) the pages are all marked as "poor" overall.

The relatively small number of "needs improvement" pages you do have, suggests that there is some variance in your pages and not all pages are exactly equal (which is not surprising really).

And a final note is that CrUX often doesn't have enough data for many pages (and doesn't even have enough data at all for some low-traffic-ed origins!). Google Search Console seems to try to estimate the likely CrUX value of these pages based on similar pages or, if it doesn't think there are page groupings of similar pages, then at origin level. Popular "poor" pages can therefore impact the Google Search Console values for lots of other less-popular pages, even if those pages may be absolutely fine. So keep an eye on your origin level data (as shown in CrUX Dashboard or CrUX API) and try to ensure it keeps above 75% of pages being marked as good. For example, I had one page that got a lot of traffic and pulled down my origin level numbers, even though 99% of my pages were fine, but then that poor rating for my popular page was then reflected in lots of less-popular pages that didn't have enough data in CrUX. Fixing that one bad page, massively improved my origin score and suddenly all the other pages (which were not touched) jumped into the "good" category.

Hope that helps.

Thanks,

Barry

[Jason Lax]

Thank you for your reply Gilberto.

[Jason Lax]

Thanks Barry.  I'm still trying to simplify this explanation but it makes sense and is consistent with what I have been trying to change: Not all pages are equally visited so we just need to speed up the right pages to lift the overall average. For most websites (and things in life) the 80/20 rule still applies where 80% of the traffic occurs from 20% of the URLs. Well, maybe it's 90/10 in some cases. 

Real, page level stats that aren't aggregated or grouped will really help so I'm going to get started with Big Query. 

Jason

[Barry Pollard]

Unfortunately BigQuery is origin-level only so you can’t get page level stats there.

You need to implement your own page level tracking using either the Google Provided webvitals.js script or some RUM solution.

I’ve a bit more of a discussion on this whole topic here: 

https://www.smashingmagazine.com/2021/04/complete-guide-measure-core-web-vitals/

Thanks,

Barry

[Gilberto Cocchi]

+1 to what Barry said.

I would suggest in particular to try the CrUX API, that tool would allow you to daily query Origin or URL data for a URL and you can pick up what Device to inspect or also the Net Info API value.

[Jason Lax]

I have something to keep me busy now! Interesting stuff. Thank you both.