how to maximize Gerrit server performance and git clone speed?

[philipp...@gmail.com]

Hi,
when cloning from our gerrit server in our LAN, I typically see transfer rates around 1 MiB/s.

I would like to maximize this transfer rate. What settings and configurations can I adjust to increase the glone speed?

My current setup is:

- gerrit server running on a Ubuntu 12.04.5 LTS VM running on Citrix 6.2 Xenserver.
- VM allocated with 4 VCPU and 8 GB RAM
- gerrit running on Oracle Java 7 JRE 1.7.0_80
- running being nginx reverse proxy (https)

thanks
Phil McLean

[Martin Fick]

On Monday, July 13, 2015 04:10:17 PM

You ommitted the one key version number, Gerrit's?

If you are running a version with bitmap support, repack
with bitmaps and that is likely the best way to speed up
clones.

What is your limiting factor, have you debugged it? Is it
your CPU, or your IO?

-Martin

--
The Qualcomm Innovation Center, Inc. is a member of Code
Aurora Forum, hosted by The Linux Foundation

[Philippe McLean]

- Gerrit version 2.11.1

I'm  not sure the limiting factor --- when running top, CPU rarely exceeds 20%.

What tools & approach do you recommend I use now?

[Martin Fick]

On Tuesday, July 14, 2015 10:24:23 AM Philippe McLean wrote:
> - Gerrit version 2.11.1
>
> I'm not sure the limiting factor --- when running top,
> CPU rarely exceeds 20%.

Depending on how you are reading top, on a 4 core machine,
that might be close to maxing out a single core? I have no
experience with running top in a VM, to see how it behaves.
But generally, a clone is single threaded, so it can only
use one core.

> What tools & approach do you recommend I use now?

I generally use top and iotop, sometimes iftop. I use
vmstat to see if I am waiting for disk seeks (if iotop shows
little activity, and CPU is low, sometimes it is disk
seeks).

I suggest maybe you experiment with other hardware to see if
faster CPUs will boost your throughput.

It might be worth testing other programs to see if you can
quantify the output of your tools better. For example,
get/write a program that consumes tons of CPU (single core),
and see how top behaves. Do a simple netcat on your
machine, and measure the maximum throughput with that.

How were you estimating the throughput, using git's progress
bar output? Some of the things I mention above tend to
limit clone speeds, however not generally the network IO
portion of the clone. In other words, a clone does tons of
IO and CPU work up front, but once it builds a new pack in
memory, I think it just streams it out. During the
streaming out portion, disk IO should not limit you, and
neither should CPU. However, encryption could (which is
CPU), but I generally don't see that on my servers.

Are you swapping, or encountering some sort of RAM bandwidth
limitation imposed by your VM setup?

Are you cloning over ssh or https?