Unable to monitor Google Argon

[Andrew Ayer]

I have been unable to retrieve entries from Argon 2025h2, Argon 2026h1, Argon 2026h2, or Argon 2027h1 since 2025-08-08T12:36:40Z. Calls to get-entries (e.g. https://ct.googleapis.com/logs/us1/argon2025h2/ct/v1/get-entries?start=1743512768&end=1743512799) return a 503 Service Unavailable response with an HTML page that says "We're sorry... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now."

I've tried sending requests from different source IP addresses, but a new source address only works for a couple minutes before the above error starts being returned. The source IP addresses are: 54.147.57.226 54.236.181.46 54.83.199.30 54.89.103.216

I send at most 4 get-entries requests in parallel, which seems reasonable given that get-entries returns only 32 entries.

Regards,
Andrew

[Luke Valenta]

We're seeing the same thing from our CT monitor starting at approximately the same time as Andrew observed.

Best,

Luke

--
You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/20250808162849.486f96251b970c1703eb7391%40andrewayer.name.

--

Luke Valenta

Systems Engineer - Research

[Martin Hutchinson]

Thanks both for the reports. The service hosting Argon logs in the US is currently under unusual load, which has triggered Google's DoS protection. The service operating these logs is running fine, but unfortunately some percentage of IPs are false positives and being blocked by DoS protection, which seems certain to be the cause of the monitoring failures you have reported.

The unusual traffic on the log is still ongoing, so DoS protections are not automatically disengaging. Some legitimate traffic is still being blocked, but some traffic is still being served. We are investigating how we can mitigate the impact.

We'll keep you posted,

Martin, on behalf of Google TrustFabric

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAAUDTJiw%2Bfe1djBDO9V8z8WCCpxhcG5ecsQByZ7pz4Kwk8Vxyw%40mail.gmail.com.

[Martin Hutchinson]

Short update:

We believe we've refined the DoS targeting to narrow down the IP addresses from this unusual load. Some IP addresses are still blocked, but will be automatically released in the coming hour if all goes normally from here. @Patrick Flynn is looking into whether we can manually release the false positive IPs early.

Cheers,

Martin

[Martin Hutchinson]

Hi all,

Optimistically calling this incident over. We believe all innocent IPs caught in DoS protection have full service access again now; traffic to the CTFE looks consistent with pre-attack.

If you are still observing problems, please update here. Also please update if you can see service is restored as that would be a reassuring data point. :-)

Cheers,

Martin

[Andrew Ayer]

Thanks Martin! It's looking good - I haven't gotten any errors in the last 40 minutes.

Regards,
Andrew

On Sat, 9 Aug 2025 00:50:12 +0100
"'Martin Hutchinson' via Certificate Transparency Policy"

<ct-p...@chromium.org> wrote:

> Hi all,
>
> Optimistically calling this incident over. We believe all innocent IPs
> caught in DoS protection have full service access again now; traffic
> to the CTFE looks consistent with pre-attack.
>
> If you are still observing problems, please update here. Also please
> update if you can see service is restored as that would be a
> reassuring data point. :-)
>
> Cheers,
> Martin
>
> On Sat, 9 Aug 2025 at 00:24, Martin Hutchinson
> <mhutc...@google.com> wrote:
>
> > Short update:
> >
> > We believe we've refined the DoS targeting to narrow down the IP
> > addresses from this unusual load. Some IP addresses are still
> > blocked, but will be automatically released in the coming hour if

> > all goes normally from here. @Patrick Flynn <paf...@google.com> is

> >>> <https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAAUDTJiw%2Bfe1djBDO9V8z8WCCpxhcG5ecsQByZ7pz4Kwk8Vxyw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>> .

> >>>
> >>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Certificate Transparency Policy" group. To unsubscribe from
> this group and stop receiving emails from it, send an email to
> ct-policy+...@chromium.org. To view this discussion visit

> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/CAJGt4a%2B7QD9GNR4%3DhaH9JgMC32AgEG%2BFmU_JXzYnbpwQbpjX_Q%40mail.gmail.com.