Maintenance downtime and upgrade of the news crawl
20 views
Skip to first unread message
Sebastian Nagel
Aug 6, 2026, 6:50:13 AM (2 days ago) Aug 6
to Common Crawl
Hi everybody,
the news crawler is moved to new hardware during the next 24 hours.
It should be back again tomorrow. During this time no new WARC files
are uploaded to s3://commoncrawl/crawl-data/CC-NEWS/
In the upcoming weeks after the hardware upgrade we plan to improve
the news crawl further and extend the collection to cover more languages
and geographic regions.
Best,
Sebastian
the news crawler is moved to new hardware during the next 24 hours.
It should be back again tomorrow. During this time no new WARC files
are uploaded to s3://commoncrawl/crawl-data/CC-NEWS/
In the upcoming weeks after the hardware upgrade we plan to improve
the news crawl further and extend the collection to cover more languages
and geographic regions.
Best,
Sebastian
Luca Foppiano
Aug 7, 2026, 5:25:31 AM (yesterday) Aug 7
to Common Crawl
Dear all,
the news crawler is back live in production.
Regards
Luca
Reply all
Reply to author
Forward
0 new messages