CDN for faster download or alternative protocols - UDT,QUIC

57 views
Skip to first unread message

brano199

unread,
Oct 23, 2017, 4:48:15 AM10/23/17
to Common Crawl
Hello,

did you guys think about moving the files to CloudFront or set up a server which can speed up the download ? (UDT, QUIC). There is even a tutorial how you can set up the server where you host it for Tsunami UDP https://aws.amazon.com/blogs/big-data/moving-big-data-into-the-cloud-with-tsunami-udp/.

It would help people outside US to download files by orders of magnitude faster for their analysis.

Thank you for your answers.

Sara Crouse

unread,
Oct 23, 2017, 12:08:57 PM10/23/17
to Common Crawl
Hello, 

Thanks for your suggestion, and it is duly noted. There are no plans at this time to set up such a server, though we aim to improve file organization/compression for more efficient delivery of data. 

Best,

Sara
Reply all
Reply to author
Forward
0 new messages