Missing metrics per originLoadingExperience in CrUX?

[Peter Hedenskog]

Hi,

we use the CrUX API to collect metric for Wikipedia domains and it seems like something changed the 30th of July. 

For at least two of our mobile domains (en.m.wikipedia.org and ca.m.wikipedia.org) we started to get chrome ux report data not found from the API since that date. Other mobile domains work. Has anyone else get the same or is this only happening to Wikipedia?

Best

Peter

[Barry Pollard]

Hi Peter,

Seeing the same as you for these two origins and we're investigating. Will let you know what we find.

Thanks,

Barry

--
You received this message because you are subscribed to the Google Groups "Chrome UX Report (Discussions)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ux-repo...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ux-report/9426e00c-534b-449d-813c-cd606116a90en%40chromium.org.

[Barry Pollard]

Hi Peter,

We have identified the cause of this missing data for those two wikipedia mobile origins (and a small number of other origins) and flagged this with the appropriate team. Will let you know when we have an update.

Thanks,

Barry

[Peter Hedenskog]

Thank you Barry looking into it!

Best

Peter

[Peter Hedenskog]

Hi Barry,

any news about this issue? I checked our metrics and it seems to still be an issue.

Best

Peter

[Barry Pollard]

Hi Peter,

We're still working on this, but don't have an ETA at present. It's basically a problem for some sites that use a redirect on their root page under a specific set of circumstances. It needs a few changes in our systems to handle this like we used to.

Will update this thread as soon as I have more information.

Thanks,

Barry

[Riccardo “Merlinox” Mares]

I found the same trouble for these sites:

m(dot)moscarossa(dot)biz

www(dot)moscarossa(dot)biz

[Barry Pollard]

Riccardo m(dot)moscarossa(dot)biz has a robots noindex in the header. I notice it was included in the October dataset so did this change recently?

www(dot)moscarossa(dot)biz does not have a noindex in the header, but is being redirected to the m(dot) site when crawled as a mobile device which I suspect mobile first indexing may do (though this is more a question for search).

[Riccardo “Merlinox” Mares]

Hi Barry,

but neither have data 

[Barry Pollard]

Correct. Due to the noindex header as detailed in our docs: https://developer.chrome.com/docs/crux/methodology/#origin-eligibility

> Your site is not discoverable if your root page fails the Page is blocked from indexing or Page has unsuccessful HTTP status code audits.

Your site is currently saying not to index, and CrUX respects that directive and so does not include this page, nor hits origin, in our data.

[Merlinox]

Thanks. I didn't know about noindex.

It's very strange: it's one of the first Italian sites (for this argument) by organic traffic.

ps: no, it isn't mine

[Riccardo “Merlinox” Mares]

The strange thing is that that site completely ignores the "separated sites" for mobile guideline, but it's a leader in its sector (organic traffic).

Returning to the previous argument:

You can verify that a page is discoverable by running a Lighthouse audit with the SEO category enabled. Your page is not discoverable if it fails the Page is blocked from indexing or Page has unsuccessful HTTP status code audits.

Does it mean that the eligibility at "Publicly discoverable" is made by Google only via a mobile agent?

Thanks

[Barry Pollard]

That really is a question for the Search team, as they handle that side of this process.

However, I would point you to this documentation of theirs:

https://developers.google.com/search/mobile-sites/mobile-first-indexing#googlebot-can-see-content

Make sure that Google can access and render your mobile page content and resources.

• Use the same meta robots tags on the mobile and desktop site. If you use a different meta robots tag on the mobile site (especially the noindex or nofollow tags), Google may fail to crawl and index your page when your site is enabled for mobile-first indexing.

[Merlinox]

Thanks.

But it seems that Google appreciates it anyway....

———————————————————————————————————

Riccardo Mares

COO @WMR Group - ASICS, Garmin, Shokz Ambassador

Linkedin - Bio, Siti e Pubblicazioni

Riflessi Di Corsa è il mio libro sul running. Esperienze ed emozioni, riflessioni sul mondo della corsa e dello sport in generale. Decine di recensioni su Amazon ne confermano la qualità: manca solo la tua!
http://bit.ly/RDCcarta

[Peter Hedenskog]

Hi Barry, any news or timeline when this could be fixed?

Thanks

Peter

[Barry Pollard]

Hi Peter, unfortunately we don't have a timeline for the fix yet as it requires a few fixes outside of our direct control. We're continuing to chase for the small number of sites affected by this!

[Bastien C.]

Hi everyone,

We also noticed that our origin www.hemea.com is not part of CrUX datasets since July, we first thought it was because of popularity but we checked all eligibility criteria and waited a few months, then we looked for an explanation and I think that it may be related to this specific issue, maybe because of our redirection to /fr path ? I understand this issue still requires investigation and progress, so we will follow this thread.

Thanks !

Bastien

[Trinidad Bean]

Hi, same happens for www.miles-and-more.com,  last data is from June 2022. Is any update on this issue? 
Thanks!

[Peter Hedenskog]

I've seen in the Search console that our Core Web Vitals on mobile FID issues only reports desktop URLs, could that be related to this or is it two different issues (or maybe mobile traffic accessing desktop URLs)?

Best

Peter

[Barry Pollard]

That would be unrelated to this issue (which basically excludes certain origins completely because it thinks the root page is set to NOINDEX because that's the last available data we had on it before the redirect was put in place).

As your mobile origin is currently excluded completely, I'm guessing you're getting some traffic accessing the desktop site, hence why you see any data until mobile.

[Kevin Farrugia]

I have encountered the same issue when querying using the CrUX API on an origin where the homepage redirects (/ -> /de or /en or /tr). There is no data for the origin ("chrome ux report data not found"), but data is returned correctly when querying by URL.

I also noticed that it isn't consistently missing. It was missing on 28/03/23 and 31/03/23, but present on 29/03/23 and 30/03/23.

curl "https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=${CRUX_API_KEY}" --header 'Content-Type: application/json' --data "{ 'origin': 'https://sports.tipico.de' }"

curl "https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=${CRUX_API_KEY}" --header 'Content-Type: application/json' --data "{ 'url': 'https://sports.tipico.de/de' }"

[Barry Pollard]

Peter,

As you may have seen in the June release note we have partially resolved the redirect issue that was preventing some domains from appearing in CrUX. We see some more additional wikipedia domains appearing this month, though unfortunately the UK mobile version is providing a little stubborn so isn't quite there. We'll continue working on that one.

The rest of you should also check your sites if they were being excluded for the same reason, as they may now be appearing. As I say not all the origins have been resolved so still some more to come.

Thanks,

Barry

[Kevin Farrugia]

Thank you, I can confirm it works well for the origins I was looking at.

[Peter Hedenskog]

Thank you Barry, no I missed that, looking forward for the fix for the English version on Wikipedia.

[Barry Pollard]

Good news Peter, we fixed the final issues we're aware of here and just tested the mobile english version of Wikipedia and it is now returning data through the API. It should also be included in next months BigQuery dataset (subject to the usual inclusion criteria).

Thank you very much for your patience.

[Peter Hedenskog]

Ah thanks, I verified that it works now.

Thank you again :)

[Abhishek Singh Bais]

Hello, 

I have noticed that the origin data for www(dot)halfords(dot)com is missing since 10th September 2024, there is no redirect in the URL and there is more than sufficient traffic on the website. Something happened last week since that there is no Origin data available. 

However all the individual URLs have real user metrics, and the less popular domain www(dot)halfords(dot)ie also has origin data but not www(dot)halfords(dot)com. 

I have tried using pagespeed API and https://pagespeed.web.dev/, but there is no Origin data in either. 
Please could someone help with this, as the origin level data is very important for reporting.