CCTLD Scrap

44 views
Skip to first unread message

Bhavik Hingrajia

unread,
Aug 22, 2016, 3:32:32 AM8/22/16
to Common Crawl
Hi

There is no any source to get list of new registered and active CCTLDs. So is there any way we can scrap all the domains (CCTLDs ) which are active and which get registered at some time interval.

I don't find any good source of the same. Some are selling but that seems pricy and I need to something automated so it works itself and put it to my server.

Sebastian Nagel

unread,
Aug 22, 2016, 11:08:13 AM8/22/16
to common...@googlegroups.com
Hi,

what do you exactly need? Domain names or country code top-level domains [1]?
Or domain names registered below certain ccTLDs?

Thanks,
Sebastian

[1] https://en.wikipedia.org/wiki/List_of_Internet_top-level_domains#Country_code_top-level_domains

OneSpeedFast

unread,
Aug 23, 2016, 3:20:41 AM8/23/16
to common...@googlegroups.com
This is what you are looking for: https://publicsuffix.org/list/

I use this list for my crawler to identify hostnames found.

--
You received this message because you are subscribed to the Google Groups "Common Crawl" group.
To unsubscribe from this group and stop receiving emails from it, send an email to common-crawl+unsubscribe@googlegroups.com.
To post to this group, send email to common...@googlegroups.com.
Visit this group at https://groups.google.com/group/common-crawl.
For more options, visit https://groups.google.com/d/optout.

Bhavik Hingrajia

unread,
Sep 15, 2016, 3:29:41 AM9/15/16
to Common Crawl
Hi

it gives just list of available cctld suffix. I need the daily registered cctld domains..


On Tuesday, 23 August 2016 12:50:41 UTC+5:30, One Speed wrote:
This is what you are looking for: https://publicsuffix.org/list/

I use this list for my crawler to identify hostnames found.
On Mon, Aug 22, 2016 at 12:32 AM, Bhavik Hingrajia <bhav...@gmail.com> wrote:
Hi

There is no any source to get list of new registered and active CCTLDs. So is there any way we can scrap all the domains (CCTLDs ) which are active and which get registered at some time interval.

I don't find any good source of the same. Some are selling but that seems pricy and I need to something automated so it works itself and put it to my server.

--
You received this message because you are subscribed to the Google Groups "Common Crawl" group.
To unsubscribe from this group and stop receiving emails from it, send an email to common-crawl...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages