Incident: Terribly slow download from repo.jenkins-ci.org

[Damien Duportal]

Hello dear Jfrog support,

The Jenkins project users are reporting performance issues since the past 2 hours.

Our build system at ci.jenkins.io (public) reports downloads speed less than 1kb per second, leading to terrible performances.

We can reproduce this issue on both Azure US East, Amazon us-east1 and 2, and also on our european ISPs, which leads us to think there is an outage on JFrog’s services, even if https://status.jfrog.io/ says "all green".

For context, you can find the issue here: https://github.com/jenkins-infra/helpdesk/issues/2733 on our tracker.

Could you help us please ?

For the Jenkins Infrastructure team,

Damien DUPORTAL

[Jesse Glick]

(Not CCing JFrog support.)

Should we revisit the decision to drop our repository mirror? We used to have one running in Azure, IIRC using nginx as a simple caching proxy (saving 200 responses with a LRU policy), but at some point we got rid of it after determining that connectivity to Artifactory was fast enough.

[Damien Duportal]

That is a very good question to ask Jesse. That could help at least for the builds on the Azure agents of ci.jenkins.io (e.g. close of the proxy), not sure about other agents though.

From user point of view, do you feel that the JFrog incidents are slowing you down frequently enough (asking the question to put in balance the maintenance effort on infra part vs. the value for users).

Cheers

Damien

[Jesse Glick]

On Sat, Jan 15, 2022 at 3:52 AM Damien Duportal <damien....@gmail.com> wrote:

That could help at least for the builds on the Azure agents of ci.jenkins.io (e.g. close of the proxy)

Well of course any new proxy could reside wherever we thought it made the most sense, such as in the same K8s cluster as typical container agents.

do you feel that the JFrog incidents are slowing you down frequently

No strong opinion.