The relationship between the Main dataset and the News dataset.

98 views

Skip to first unread message

郑舒力

unread,

Jun 11, 2024, 5:18:56 AM6/11/24

to Common Crawl

Does the Main dataset include the News dataset, or is the News dataset completely separate from the Main dataset?

Greg Lindahl

unread,

Jun 11, 2024, 5:20:28 AM6/11/24

to common...@googlegroups.com

Hi! The two crawls are totally separate. The main crawl has 2 indexes,
the news crawl has none. The main crawl happens monthly, the news
crawl is continuous.

-- greg

On Tue, Jun 11, 2024 at 2:18 AM 郑舒力 <nezha...@gmail.com> wrote:
>
> Does the Main dataset include the News dataset, or is the News dataset completely separate from the Main dataset?
>

> --
> You received this message because you are subscribed to the Google Groups "Common Crawl" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to common-crawl...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/common-crawl/2400addd-088d-40e8-8a0a-ba9a3b5d44c3n%40googlegroups.com.

Reply all

Reply to author

Forward

0 new messages