You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Common Crawl
Hi there
I'm a student new the common crawl data set and I was just wondering if there were any updated tutorials for beginners.
The few I've found online seem quite out of date, AWS and specifcally EMR have seemed to updated quite a bit in the last few years and navigating it to execute a sample job is a bit of a minefield right now
Any help would be much appreciated
Regards
Sebastian Nagel
unread,
Nov 14, 2016, 10:18:22 AM11/14/16
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to common...@googlegroups.com
Hi Matt,
thanks for your interest in Common Crawl. You're right the "officially" provided
examples need an overhaul and we hope to get this done soon. A list with verified
examples is already getting compiled.
> and specifcally EMR
If you have in mind
https://github.com/commoncrawl/cc-mrjob I've tested it recently: running it locally works out-of-the-box,
in distributed mode there is one issue with a work-around available.