DigiCert - Shutdown old 2024 and 2025 logs

[Rick Roos]

Hello All,

We have some old CT logs that are past their inclusion window and will be shut down.  All of these logs have been frozen and have had their last signed tree published for several weeks.

The following logs will be turned off on Thursday June 4th:

https://sphinx.ct.digicert.com/2024h1

https://sphinx.ct.digicert.com/2024h2

https://sphinx.ct.digicert.com/2025h1

https://sphinx.ct.digicert.com/2025h2

https://wyvern.ct.digicert.com/2024h1

https://wyvern.ct.digicert.com/2024h2

https://wyvern.ct.digicert.com/2025h1

https://wyvern.ct.digicert.com/2025h2

Thanks,

Rick

[Filippo Valsorda]

Hi Rick,

Any chance you could archive the 2025h2 shards to https://github.com/geomys/ct-archive? The rest were already captured.

Thank you!

Filippo

--

You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CACrB8x%3DRktsT8gBkJcZhitRbHmRm9fw5sEi2KJ3t7Pbf4oy9dw%40mail.gmail.com.

[Filippo Valsorda]

Hi Rick,

Have you given any consideration to archiving the 2025h2 shards before shutting them down tomorrow?

Fetching them externally seems impossible due to rate limits: it'd take 4M requests each, which based on https://groups.google.com/a/chromium.org/g/ct-policy/c/lTqtb4WHsqo/m/38t48RAyAgAJ would take a month and a half, assuming 1 req/s because you have not confirmed the actual rate limit and ideal pattern.

It's unfortunate to have CT logs in the ecosystem that are so hard to monitor that it's impossible to fetch them between the day their shut down is announced and the day they are shut down.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/b4de02ee-1fb0-4759-8d26-29e130c26d64%40app.fastmail.com.

[Rick Roos]

Hi Filippo,

We discussed this internally and will delay shutting down the 2025h2 logs to give you more time to download them (we will still move forward shutting down the other logs).  In addition, we are increasing the per Nginx server rate limit to 5 requests per second. I just responded to Rob on the other thread with the best download pattern. Can your tool download the entries in parallel?  If so that will greatly speed up the process. 

Let us know if this speeds up your download and if you can give an estimate of how long it will take to download.

Additionally, in the future we will announce the freezing of logs earlier and allow more time to download the logs before shutting them down.

Thanks,

Rick

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/5a650f37-4ca3-429d-bab4-eb1fae6ada44%40app.fastmail.com.

[Filippo Valsorda]

Thank you, I kicked off a fetch and I seem to be able to get between 5 and 15 requests per second by using a pool of parallel connections.

Should be done in a week or so, I will report back.

[Filippo Valsorda]

I have finished downloading digicert_sphinx2025h2 and digicert_wyvern2025h2. The latter is already uploaded at https://archive.org/details/ct_digicert_wyvern2025h2, the former is still uploading to https://archive.org/details/ct_digicert_sphinx2025h2.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/775cdddd-114a-4593-804d-6b3add44ccf9%40app.fastmail.com.

[Rick Roos]

Thanks Filippo!  We will go ahead and shut down the 2025h2 logs this week.

-Rick