I'd like to explore running a Sunlight CT server

[Hilton De Meillon]

Hi there,

I'd like to explore running a CT server. I will be upfront and say that part of my motivation is curating threat intelligence and external attack surface monitoring by accessing the CT log of my own server, so as not to put unnecessary and unfair load on public servers.

I can't provide a committed 3gbps server to start with so I plan to start with a 1gbps that has burst capacity. I'd also like to see if using tigris as the store would work. If my business case works I can then build up to 3gbps or if some philanthropic org wants to sponsor part of the solution that would work too.

I'm pumped about running a CT server - let me know your thoughts or how to proceed from here..

Hilton

[Matt Palmer]

On Tue, Jul 22, 2025 at 05:08:09PM -0700, Hilton De Meillon wrote:
> I'd like to explore running a CT server. I will be upfront and say that
> part of my motivation is curating threat intelligence and external attack
> surface monitoring by accessing the CT log of my own server, so as not to
> put unnecessary and unfair load on public servers.

I may be misunderstanding what you're envisioning, but based on what
you've written I think you might have the wrong impression of what a CT
log (server) does.

In order for the log to have entries, they need to be submitted to the
log. This can be from CAs submitting (pre-)certificates to get enough
SCTs for the certificate to be "valid" in the eyes of CT-validating user
agents, or it can be from someone submitting certificates post-issuance
"just because".

Neither of these approaches is likely to result in your log having
anywhere close to a "full" view of the certificate population at any
given time. As such, using (just) your own log as a source of data for
any purpose is unlikely to produce a useful result. You will still need
to scrape all other logs to ensure you have a complete view.

As for whether get-entries requests can be considered "unnecessary" or
"unfair", I don't see how scraping all the entries in a log could be
considered as such, since the entire point of a CT log is to make
certificates transparently available. As long as you're not repeatedly
hitting the logs for the same entries over and over, your requests are
neither "unnecessary" nor "unfair". Do what all other monitors do --
that is, scrape the log entries into a database and query away, and you
won't be causing anyone any problems.

- Matt

[Filippo Valsorda]

Hi Hilton!

Thank you for choosing to contribute back to the CT ecosystem.

Starting with 1 Gbps is going to be totally fine. Right now Tuscolo is seeing a lot less and I wouldn't be surprised if P95 never crossed that. The 2 Gbps in the (updated) post is a conservative long-term estimate.

I actually used to operate the Sunlight dev prototype on Tigris, so it would probably work.

The next step would be to pick a name and stand up the instance and report back, so the community can offer feedback if you'd like. Then when you are ready you submit the log to the browsers, which starts a 90 days qualification process.

https://github.com/FiloSottile/sunlight#operating-a-sunlight-log and https://config.sunlight.geomys.org/ should help setting up.

Let us know if you have any questions!

Cheers,

Filippo

--

You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/720fb3de-dcd5-436b-bf6e-d735ae681fe5n%40chromium.org.

[Hilton De Meillon]

Thanks Matt and Winston for your reply and for clearing up the misconception. It sounds like reading the CT log is what it is there for so that is good news for my use case although it will probably mean that I won't be running a CT server if I don't need to as the cost would be non-trivial. 

For my own interest, how do the CAs choose which CT log server to submit to?

Let me see if I can find a transit + colo sponsor here in Australia. 

H

[Winston de Greef]

Hi Hilton,

It's great to hear you're excited about running a CT log. Having more CT logs is great for the CT ecosystem. While I can't speak for others in the CT community, I'd be happy to answer any questions you have about running a CT log.

However, it seems like you have some misconceptions about the CT ecosystem. If the information you need for your threat intelligence and external attack surface monitoring is just the certificates that are logged, then there's no need to feel guilty about putting an unfair load on public servers. This is one of the explicit use cases for CT. The whole point of CT is that everyone gets access to all issued certificates.

Also, most certificates will not be submitted to your log, so if you wanted to analyze all issued certificates, you would still need to keep up with other logs.

If you want to access private data that is only available to the operator (ie what IPs are requesting inclusion proofs), I'm not sure how useful that information is, because not exposing private information to log operators is something that a lot of attention is paid to. I'd also say that that would be a practice frowned upon by the CT community.

You mention a business case in your email. I want to let you know that a CT log is a public good, and there is no way to directly extract value from running a log. The only business case for running a log is that it provides some positive marketing value. Products like SSLmate and MerkleTown that analyze what certificates are issued for your domain (to find ones that weren't supposed to be issued) are run without the company providing these services also hosting a CT log.

Sincerely,

Winston de Greef

--

[Matthew McPherrin]

> how do the CAs choose which CT log server to submit to?

Let's Encrypt generally submits to most trusted logs in Apple/Google's programs. However, as the CA with the most volume, we are a little bit cautious about which logs we use, and sometimes avoid logs if we think they are struggling.

We manually curate that list.

I know that some other CAs submit to all trusted logs in parallel.  I believe some also have a preferred set of logs they use.

--

You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/4fcc4704-4250-4578-8214-82ea7b8edb09n%40chromium.org.

[Hilton De Meillon]

Ok, I've arranged a server with the following :

Intel Xeon E3-1275V6

64GB ECC

4TB Spinning hard drive 

512 GB Nvme SSD

1Gbps pipe

The server is located at Hetzner in Finland. 

I plan on creating a ZFS raidz volume and then adding the 512GB SSD as L2ARC cache so that the read path is speedier. I'll let you know when I'm ready for the next step!

Hilton

[Hilton De Meillon]

Ok I have my server prepped and am having some issues getting sunlight and skylight running. Here are a few questions:

- Do I need two public IPs or can I just run the two services on different ports? I could setup a load balancer but that just adds to the complexity.

- Since I am running this on a shoestring, the host is behind nat - I haven't configured hairpin nat so is adding an entry to /etc/hosts sufficient for the services connecting to each other?

Hilton