The relationship between the Main dataset and the News dataset.
68 views
Skip to first unread message
郑舒力
Jun 11, 2024, 5:18:56 AMJun 11
to Common Crawl
Does the Main dataset include the News dataset, or is the News dataset completely separate from the Main dataset?
Greg Lindahl
Jun 11, 2024, 5:20:28 AMJun 11
to common...@googlegroups.com
Hi! The two crawls are totally separate. The main crawl has 2 indexes,
the news crawl has none. The main crawl happens monthly, the news
crawl is continuous.
-- greg
On Tue, Jun 11, 2024 at 2:18 AM 郑舒力 <nezha...@gmail.com> wrote:
>
> Does the Main dataset include the News dataset, or is the News dataset completely separate from the Main dataset?
>
> --
> You received this message because you are subscribed to the Google Groups "Common Crawl" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to common-crawl...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/common-crawl/2400addd-088d-40e8-8a0a-ba9a3b5d44c3n%40googlegroups.com.
the news crawl has none. The main crawl happens monthly, the news
crawl is continuous.
-- greg
On Tue, Jun 11, 2024 at 2:18 AM 郑舒力 <nezha...@gmail.com> wrote:
>
> Does the Main dataset include the News dataset, or is the News dataset completely separate from the Main dataset?
>
> You received this message because you are subscribed to the Google Groups "Common Crawl" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to common-crawl...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/common-crawl/2400addd-088d-40e8-8a0a-ba9a3b5d44c3n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages