Web viewer for CT logs?

[Mike Hearn]

I renewed my DigiCert EV certificate today and my companies website is now auditable via CT - hurrah!

I figured I'd go look up its entry in the logs to explore and maybe do some queries to see if there were any certificates I don't recognise out there under my names. I remember encountering a website that let you query CT logs at some point. Alas, I can no longer find it.

Does anyone know the URL? Is it still active? The point of CT is to allow data mining of the logs, so are there any tools to do that yet, even if it's only basic?

thanks!

[Al Cutter]

Hi Mike,

Tom Fitzhenry has site which lets you look into log contents at ctwatch.net.

Otherwise, not web, but there are some tools in the ct github repo which will let you scrabble through the logs for entries you're interested in, the Go scanner in particular will definitely let you look for certs whose subject matches a given regex, there are Python bits which do similar there too, but I'm less familiar with those.

Cheers,
Al.

--
You received this message because you are subscribed to the Google Groups "certificate-transparency" group.
To unsubscribe from this group and stop receiving emails from it, send an email to certificate-transp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Hearn]

Thanks Al. Seems ctwatch.net is down at the moment.

I guess using the command line tools would require downloading the whole log? Or do the CT servers support regex querying as part of the protocol? Any idea how big the logs are now?

[Al Cutter]

On 29 Mar 2015 3:24 pm, "Mike Hearn" <mi...@plan99.net> wrote:
>
> Thanks Al. Seems ctwatch.net is down at the moment.

D'oh. I'll drop Tom a note tomorrow.

>
> I guess using the command line tools would require downloading the whole log? Or do the CT servers support regex querying as part of the protocol? Any idea how big the logs are now?

Yes, they'll grab the whole lot (although the Go scanner will let you specify the leaf index to start at if you happen to know you're not interested in anything which came before.) I suspect you're probably looking at around 5-6GB at the moment from Pilot, a touch less from Aviator and Rocketeer, but I'm not sure what the exact sizes are as I haven't looked for a while.

[Peter Bowen]

On Sun, Mar 29, 2015 at 7:41 AM, 'Al Cutter' via
certificate-transparency <certificate-...@googlegroups.com>
wrote:

> On 29 Mar 2015 3:24 pm, "Mike Hearn" <mi...@plan99.net> wrote:
>> I guess using the command line tools would require downloading the whole
>> log? Or do the CT servers support regex querying as part of the protocol?
>> Any idea how big the logs are now?
>
> Yes, they'll grab the whole lot (although the Go scanner will let you
> specify the leaf index to start at if you happen to know you're not
> interested in anything which came before.) I suspect you're probably looking
> at around 5-6GB at the moment from Pilot, a touch less from Aviator and
> Rocketeer, but I'm not sure what the exact sizes are as I haven't looked for
> a while.

Pilot is a little bigger than that :) My cached local copy of just
the Pilot certs is 13GB, and that is after removing the chains. The
download size is going to be somewhat bigger.

[Ben Laurie]

Yeah - about 75 GB.

[Al Cutter]

Blimey, as I said it's been a while since I looked!

[Eran Messeri]

The Python log observer/scanner populates a database SQL (and caches its fetches so it'll only download entries once):

https://github.com/google/certificate-transparency/blob/master/python/utilities/logobserver/logobserver.py

 

[Rob Stradling]

Hi. We're planning to announce one soon. Stay tuned.

--
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online

[Rob Stradling]

On 30/03/15 15:41, Rob Stradling wrote:
> On 29/03/15 15:05, Mike Hearn wrote:
>> I renewed my DigiCert EV certificate today and my companies website is
>> now auditable via CT - hurrah!
>>
>> I figured I'd go look up its entry in the logs to explore and maybe do
>> some queries to see if there were any certificates I don't recognise out
>> there under my names. I remember encountering a website that let you
>> query CT logs at some point. Alas, I can no longer find it.
>>
>> Does anyone know the URL? Is it still active? The point of CT is to
>> allow data mining of the logs, so are there any tools to do that yet,
>> even if it's only basic?
>>
>> thanks!
>
> Hi. We're planning to announce one soon. Stay tuned.

Here it is...

https://crt.sh

Pronounced "search". :-)

[Linus Nordberg]

Rob Stradling <rob.st...@comodo.com> wrote
Wed, 03 Jun 2015 13:22:30 +0100:

| https://crt.sh

Neat!

What log(s) does it seartsh (heh)?

[Rob Stradling]

On 03/06/15 13:28, Linus Nordberg wrote:
> Rob Stradling <rob.st...@comodo.com> wrote
> Wed, 03 Jun 2015 13:22:30 +0100:
>
> | https://crt.sh
>
> Neat!

Thanks Linus. :-)

> What log(s) does it seartsh (heh)?

It regularly pulls entries from all of the logs I know about...
https://ct1.digicert-ct.com/log
https://ct.akamai.com
https://ct.googleapis.com/aviator
https://ct.googleapis.com/pilot
https://ct.googleapis.com/rocketeer
https://ct.izenpe.com
https://log.certly.io

...except for the ones that are only for testing or have been switched off:
https://alpha.ctlogs.org
https://ct.googleapis.com/testtube
https://flimsy.ct.nordu.net:8080

--
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online

Office Tel: +44.(0)1274.730505
Office Fax: +44.(0)1274.730909
www.comodo.com

COMODO CA Limited, Registered in England No. 04058690
Registered Office:
3rd Floor, 26 Office Village, Exchange Quay,
Trafford Road, Salford, Manchester M5 3EQ

This e-mail and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error please notify the
sender by replying to the e-mail containing this attachment. Replies to
this email may be monitored by COMODO for operational or business
reasons. Whilst every endeavour is taken to ensure that e-mails are free
from viruses, no liability can be accepted and the recipient is
requested to use their own virus checking software.

[Linus Nordberg]

Rob Stradling <rob.st...@comodo.com> wrote
Wed, 03 Jun 2015 13:38:03 +0100:

| It regularly pulls entries from all of the logs I know about...

Feature request: Show which log(s) a chain was found in, possibly
together with index number for each log.

Publishing the list of logs monitored would be another nice thing.

Thanks for the service!

[Mike Hearn]

Thanks Rob! That's great! Super fast and simple UI, like it.

[Rob Stradling]

On 03/06/15 14:03, Linus Nordberg wrote:
> Rob Stradling <rob.st...@comodo.com> wrote
> Wed, 03 Jun 2015 13:38:03 +0100:
>
> | It regularly pulls entries from all of the logs I know about...
>
> Feature request: Show which log(s) a chain was found in, possibly
> together with index number for each log.

It already shows the log(s) and index number(s) for each entry.

For example, look at the "Certificate Transparency" section near the top
of this page: https://crt.sh/?id=5657147

> Publishing the list of logs monitored would be another nice thing.

I've just added this to the "Advanced..." search page:
https://crt.sh/?a=1

> Thanks for the service!

:-)

[Rob Stradling]

On 03/06/15 14:21, Mike Hearn wrote:
> Thanks Rob! That's great! Super fast and simple UI, like it.

Thanks Mike. :-)

[Linus Nordberg]

Rob Stradling <rob.st...@comodo.com> wrote
Wed, 03 Jun 2015 14:32:12 +0100:

| > | It regularly pulls entries from all of the logs I know about...
| >
| > Feature request: Show which log(s) a chain was found in, possibly
| > together with index number for each log.
|
| It already shows the log(s) and index number(s) for each entry.
|
| For example, look at the "Certificate Transparency" section near the
| top of this page: https://crt.sh/?id=5657147

Oh, please accept my appologies! My inferior read'n'click skills have
embarrased me yet again. Thanks for your patience.

| > Publishing the list of logs monitored would be another nice thing.
|
| I've just added this to the "Advanced..." search page:
| https://crt.sh/?a=1

Nice!

[Rob Stradling]

On 03/06/15 14:50, Linus Nordberg wrote:
> Rob Stradling <rob.st...@comodo.com> wrote
> Wed, 03 Jun 2015 14:32:12 +0100:
>
> | > | It regularly pulls entries from all of the logs I know about...
> | >
> | > Feature request: Show which log(s) a chain was found in, possibly
> | > together with index number for each log.
> |
> | It already shows the log(s) and index number(s) for each entry.
> |
> | For example, look at the "Certificate Transparency" section near the
> | top of this page: https://crt.sh/?id=5657147
>
> Oh, please accept my appologies! My inferior read'n'click skills have
> embarrased me yet again. Thanks for your patience.

That's no problem at all, Linus.

> | > Publishing the list of logs monitored would be another nice thing.
> |
> | I've just added this to the "Advanced..." search page:
> | https://crt.sh/?a=1
>
> Nice!

[Matt Palmer]

On Wed, Jun 03, 2015 at 01:22:30PM +0100, Rob Stradling wrote:
> On 30/03/15 15:41, Rob Stradling wrote:
> >On 29/03/15 15:05, Mike Hearn wrote:
> >>I renewed my DigiCert EV certificate today and my companies website is
> >>now auditable via CT - hurrah!
> >>
> >>I figured I'd go look up its entry in the logs to explore and maybe do
> >>some queries to see if there were any certificates I don't recognise out
> >>there under my names. I remember encountering a website that let you
> >>query CT logs at some point. Alas, I can no longer find it.
> >>
> >>Does anyone know the URL? Is it still active? The point of CT is to
> >>allow data mining of the logs, so are there any tools to do that yet,
> >>even if it's only basic?
> >>
> >>thanks!
> >
> >Hi. We're planning to announce one soon. Stay tuned.
>
> Here it is...
>
> https://crt.sh

DNS lookups are failing for me for that name on all of
ns[01].comododns.{com,net} on IPv4 and IPv6.

- Matt

[Matt Palmer]

On Wed, Jun 03, 2015 at 01:38:03PM +0100, Rob Stradling wrote:
> On 03/06/15 13:28, Linus Nordberg wrote:
> >What log(s) does it seartsh (heh)?
>
> It regularly pulls entries from all of the logs I know about...

Symantec asked for inclusion recently, too:

https://code.google.com/p/chromium/issues/detail?id=483625

- Matt

[Rob Stradling]

Hi Matt. Sorry about that. Somebody on our NOC team, whilst "tidying"
the BIND config last night, inadvertently moved the service to
https://www.crt.sh !

https://crt.sh is working again now.

[Rob Stradling]

Thanks.

https://crt.sh is now monitoring this log too.