September 2024 crawl and Web Graph release

71 views

Skip to first unread message

Thom Vaughan

unread,

Sep 26, 2024, 4:51:07 PM9/26/24

to Common Crawl

Hi all,

We're happy to announce the release of our September crawl (CC-MAIN-2024-38) and Web Graph release (cc-main-2024-jul-aug-sep).

The September crawl consists of 2.8 billion web pages (or 410 TiB of uncompressed content), and the corresponding Web Graph consists of 306.5 million nodes and 2.6 billion edges.

More information can be found on the respective blog posts below:

https://commoncrawl.org/blog/september-2024-crawl-archive-now-available

https://commoncrawl.org/blog/host--and-domain-level-web-graphs-july-august-and-september-2024