Is CC-NEWS included in common crawl index api or CC-MAIN in general?

[Max Dallabetta]

I was wondering if the news sub dataset of common crawl CC-NEWS is actually included in CC-MAIN as a real subset and therefor indexed by the index api and if not if there is any alternate indexing for CC-NEWS?

I want to query the data from CC-NEWS for specific publishers/domains and would like to avoid iterating over the hole dataset for various reasons.

Thanks

Max

[Sebastian Nagel]

Hi Max,

unfortunately, we do not yet provide any URL index for the CC-NEWS
dataset. There is some overlap between CC-NEWS and the main crawls,
if you know the domain names you could also run a search there.

Best,
Sebastian