Page ranks available for the host-level web graph of Aug/Sept/Oct 2017 crawls

[Sebastian Nagel]

Hi everyone,

you may remember that we couldn't calculate the page ranks for the last host-level
web graph (Aug/Sept/Oct 2017) because of it's size (5 billion nodes). The issue
is now fixed, the ranks file contains now both harmonic centrality and page rank.

You'll find the download link on
http://commoncrawl.org/2017/11/host-and-domain-level-web-graphs-augseptoct-2017/

Thanks to Sebastiano Vigna, one of the authors of the WebGraph framework [1],
for the kind support and his advice how to port the corresponding classes
of the LAW library [2] to big graphs with more than 2^31 nodes.


Best,
Sebastian


[1] http://webgraph.di.unimi.it/
[2] http://law.di.unimi.it/software.php