Downloading URLS from common crawl

Skip to first unread message

Dr. Nisha T N

Feb 6, 2021, 5:36:04 AMFeb 6
to Common Crawl
Hi researchers,

I am a reseracher in information security and currently working on phishing attack detection.  

Can anyone guide me, how to download URL s from common crawl dataset?

Tom Alby

Feb 6, 2021, 5:42:26 AMFeb 6
to Common Crawl
I have just gone through that, and I suggest to use the Athena interface, it was the fastest way for me:
It costs a few $, I have paid like $7 for extensive querying and downloading the whole db with several additional columns.

Dr. Nisha T N

Feb 6, 2021, 5:44:25 AMFeb 6
Thank you very much  Tom. Let me try...

You received this message because you are subscribed to the Google Groups "Common Crawl" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web visit
Reply all
Reply to author
0 new messages