Winter and Spring Crawl

[Robert Meusel]

Hi,

I was wondering weather the two crawls of 2013 are related in any way? Are the same seeds used, or is the winter crawl an extension of the spring crawl? Or are the same URLs crawled in both datasets?

Would be really interesting to know.

Thanks a lot,

Robert

[Jordan Mendelson]

Both crawls used URL lists donated by Blekko from their search engine crawler. They should contain a good number of URLs that overlap, but they were separately generated lists.

Jordan

--
You received this message because you are subscribed to the Google Groups "Common Crawl" group.
To unsubscribe from this group and stop receiving emails from it, send an email to common-crawl...@googlegroups.com.
To post to this group, send email to common...@googlegroups.com.
Visit this group at http://groups.google.com/group/common-crawl.
For more options, visit https://groups.google.com/groups/opt_out.

[Robert Meusel]

thanks a lot. and the selection strategy is also the same i guess?