License to use when republishing parts of the WDC data.

[Djellel Eddine Difallah]

Hi Everyone,

We are working on a dataset extracted from CommonCrawl (but including a subset of webpages) and partly your RDFa data. We were wondering about the licence to use, as we want to republish the data. is the following enough?

"The extracted data is provided according the same terms of use, disclaimer of warranties and limitation of liabilities that apply to the Common Crawl corpus."

We had a doubt since the CC terms of use states:

"We grant you a non-assignable, non-transferable, non-sublicensable, limited license to use our site and data in accordance with the terms of the ToU."

We are also contacting CC of their input, but we were wondering if you have already some experience with that. 

Thanks

Djellel Difallah

Exascale Infolab

University of Fribourg, CH

[Robert Meusel]

Hi,

No unfortunately we have no experience here. Best would be to settle this with CC.

Cheers,

Robert

[Tom Morris]

On Thu, Apr 28, 2016 at 8:45 AM, Djellel Eddine Difallah <difa...@gmail.com> wrote:

We are working on a dataset extracted from CommonCrawl (but including a subset of webpages) and partly your RDFa data. We were wondering about the licence to use, as we want to republish the data. is the following enough?

"The extracted data is provided according the same terms of use, disclaimer of warranties and limitation of liabilities that apply to the Common Crawl corpus."

We had a doubt since the CC terms of use states:

"We grant you a non-assignable, non-transferable, non-sublicensable, limited license to use our site and data in accordance with the terms of the ToU."

Yes, that's a little ironic. By my reading (although I'm not a lawyer), unless WDC was separately granted the right to sub-license by the Common Crawl Foundation, they're not actually following the CC Terms of Use.

The current Common Crawl Terms of Use don't seem like they're aligned with the goals of the CC Foundation since they effectively prohibit the distribution of any derivative works such as the WDC. I'd be interested in learning what you hear back from Common Crawl. The right to sub-license is key to anyone who wants to publish derivative works.

Tom

p.s. The full Terms of Use are actually here: http://commoncrawl.org/terms-of-use/full/

[Alexander Panchenko]

Hello, Tom,

Did you get any answer from CC? What is the best way to license a derivative dataset based on CC?