Inconsistency between CRUX API data and GSC Web Vitals

[Duru Ahanotu]

Hi,

I am not able to reconcile the FID data coming through the CRUX API and the Web Vitals reporting in GSC. They use the same data, correct? I would expect that if the 28-day API data shows a metric going over an important threshold, we would see GSC report some URLs that are causing the problem. Am I missing something in the relationship between the datasets?

Thanks.

--- Duru

[Rick Viscomi]

Hi Duru,

The tools use slightly different data methodologies. We're working to smooth out any differences and appreciate all feedback like yours that highlight points of friction.

Note that Search Console's Core Web Vitals report using URL groups to aggregate pages together and determine their group-level assessment. Note that an individual page in the group may be assessed differently from the group as a whole. Here are a couple of other related threads:

https://groups.google.com/a/chromium.org/g/chrome-ux-report/c/bQq-6YVXZEE

https://groups.google.com/a/chromium.org/g/chrome-ux-report/c/THTmCfVcGho

I'd be curious to hear more about what you're seeing in the report. How misaligned are the group-level "Agg FID" value and the URL-level FID values for the example pages listed in the report?

Rick

[Duru Ahanotu]

Thank you for the additional references.

The disconnect is severe from the data I am seeing. The FID threshold that generates GSC warnings is supposed to be 100ms (has that changed?). I have seen CRUX averages over that threshold for two consecutive months so surely some URL or group of URLs must also be over the 100ms threshold that would trigger the warning in GSC.

--- Duru

[Rick Viscomi]

Gotcha. The threshold for "good FID" is still 100ms. Have you gone through each of the 20 sample URLs and evaluated their URL-level FID in PSI? If any pages are over the threshold, those would be good places to start to get the group-level aggregate down. Or are you not seeing any URLs with poor FID?

[Duru Ahanotu]

Hi Rick,

I am not seeing any URLs reported with FID over the threshold. That's the core disconnect I am seeing with the data.

--- Duru

[Rick Viscomi]

Oh! I may have misunderstood you earlier. I had assumed Search Console was flagging a group of URLs as having issues, while CrUX API said otherwise. Are you saying that you're seeing URLs with poor or "needs improvement" FID in CrUX API, but you're surprised to see that none of those URLs are being flagged by Search Console?

If that's the case, one explanation is that Search Console is reporting the Agg FID of groups of similar URLs rather than individual URLs' FID. In other words, pages with > 100ms FID may be part of a group whose Agg FID is actually <= 100ms.

In either case, it sounds like the disconnect between the URL-level assessments and the group-level assessments is causing confusion. That's good feedback and I'll pass it along to the relevant teams. For anyone else subscribed to this thread who may be experiencing similar confusion in our tools, please let me know! It's helpful to have a sense of the number of people affected when I report your feedback.

Rick

[Duru Ahanotu]

Hi Rick,

I am not getting any URL information. The issue is the disconnect between the aggregated FID data reported through the CrUX API and the Web VItals reported in GSC. CruX says FID is above threshold for two straight months. GSC implies that FID is fine and below threshold for ALL URLs. 

My expectation is that if CrUX API shows the aggregate of any Web Vitals metric is above an important threshold over an entire month, especially for two straight months, then GSC will show at least some of those problematic URLS, where it is individually or as a group. It seems extremely unlikely that GSC's groupings will report ALL groups below the threshold in this scenario. If so, then GSC can only be relied upon to flag very extreme breaks of the threshold?

[Duru Ahanotu]

Hi Rick,

 

Have we reached a dead-end here? At the moment, I have concluded that the reporting is broken or inconsistent at best.

 

- --

Duru Ahanotu

[Rick Viscomi]

Hi Duru,

Thanks for confirming the issue. As of now both tools are reporting correct information, but at different granularities: CrUX API at the URL level and Search Console at the page group level. Sometimes a URL may have FID issues but the group it belongs to does not. I completely agree that this is inconsistent at best. I appreciate your feedback and I've passed it on to the relevant teams so we can hopefully align these tools more closely. Stay tuned to the "CrUX announce" channel for any updates on that front.

Rick