Find out which CI flow produces most of gradle load

[Denis Khabensky]

Our Gerrit instance serves a small number of developers and a large number of different CI flows. CI produce produce 99% of load which mainly consists of checkouts (git-upload-pack).
When Gerrit instance is overloaded, I need to know kind of top-consumers across CI flows, so I can terminate most expensive or least important of them to reduce overall load.

The problem is: all CI flows use robot (service) account so are basically indistinguishable for gerrit. Flows (and their number) change pretty frequently so I cannot give each flow its own account.

My idea is to use only http checkout and include some sort of flow identifiers/tags into user_agent.
So I will be able to breakdown all the load by user_agents and find ones that consume too much.

Do you have better ideas or any ready instruments for this problem? Or is it feasible for ssh checkouts somehow, so I can skip migration from ssh to http checkout? 

[Björn Pedersen]

Denis Khabensky schrieb am Donnerstag, 12. Februar 2026 um 16:05:11 UTC+1:

Our Gerrit instance serves a small number of developers and a large number of different CI flows. CI produce produce 99% of load which mainly consists of checkouts (git-upload-pack).
When Gerrit instance is overloaded, I need to know kind of top-consumers across CI flows, so I can terminate most expensive or least important of them to reduce overall load.

The problem is: all CI flows use robot (service) account so are basically indistinguishable for gerrit. Flows (and their number) change pretty frequently so I cannot give each flow its own account.

My idea is to use only http checkout and include some sort of flow identifiers/tags into user_agent.
So I will be able to breakdown all the load by user_agents and find ones that consume too much.

The load is mainly determined by repo size. 

Check e.g. https://groups.google.com/g/repo-discuss/c/iZprzMwwD9U/m/9gAoWb4cDQAJ  and  https://www.slideshare.net/slideshow/gerrit-performance-tuning-talk-at-google-summit/54962424  to see if your  config does just overload gerrit.

[Björn Pedersen]

Björn Pedersen schrieb am Donnerstag, 12. Februar 2026 um 16:35:25 UTC+1:

Denis Khabensky schrieb am Donnerstag, 12. Februar 2026 um 16:05:11 UTC+1:

Our Gerrit instance serves a small number of developers and a large number of different CI flows. CI produce produce 99% of load which mainly consists of checkouts (git-upload-pack).
When Gerrit instance is overloaded, I need to know kind of top-consumers across CI flows, so I can terminate most expensive or least important of them to reduce overall load.

The problem is: all CI flows use robot (service) account so are basically indistinguishable for gerrit. Flows (and their number) change pretty frequently so I cannot give each flow its own account.

My idea is to use only http checkout and include some sort of flow identifiers/tags into user_agent.
So I will be able to breakdown all the load by user_agents and find ones that consume too much.

The load is mainly determined by repo size. 

Check e.g. https://groups.google.com/g/repo-discuss/c/iZprzMwwD9U/m/9gAoWb4cDQAJ  and  https://www.slideshare.net/slideshow/gerrit-performance-tuning-talk-at-google-summit/54962424  to see if your  config does just overload gerrit.

And  https://groups.google.com/g/repo-discuss/c/TJ4Pdg3KHUI/m/8nH9xqgSBAAJ as well (or search for  tuning in this group...)

[Denis Khabensky]

Thanks, Björn!
Seems I already read most of the links you just provided and our Gerrit instance is technically configured good enough.
It does not die under load, the problem is that clients receive poor service (long checkout times).

We already have scaling procedures in our schedule, but here I am looking for short-term measures, more into how "fair" our clients use Gerrit, maybe there are misconfiguration on checkout side. One that was mentioned in https://groups.google.com/g/repo-discuss/c/TJ4Pdg3KHUI/m/8nH9xqgSBAAJ as "dont use clean dir, work in more incremental ways" is extremely relevant, but unfortunately it is unfeasible for our CI servers - clean environment is strictly enforced and bringing repos as a zip-archieves is slower than doing git clone.

четверг, 12 февраля 2026 г. в 18:38:50 UTC+3, Björn Pedersen:

[Björn Pedersen]

Denis Khabensky schrieb am Freitag, 13. Februar 2026 um 04:57:23 UTC+1:

Thanks, Björn!
Seems I already read most of the links you just provided and our Gerrit instance is technically configured good enough.
It does not die under load, the problem is that clients receive poor service (long checkout times).

We already have scaling procedures in our schedule, but here I am looking for short-term measures, more into how "fair" our clients use Gerrit, maybe there are misconfiguration on checkout side. One that was mentioned in https://groups.google.com/g/repo-discuss/c/TJ4Pdg3KHUI/m/8nH9xqgSBAAJ as "dont use clean dir, work in more incremental ways" is extremely relevant, but unfortunately it is unfeasible for our CI servers - clean environment is strictly enforced and bringing repos as a zip-archieves is slower than doing git clone.

Then I would look into using reference repos close to the CI services that is updated via replication. 

Check `--reference` here: https://git-scm.com/docs/git-clone or the docs for your CI systems ( e.g. jenkins has a reference option for git)