git gc and repo/mirror maintenance

[Garret]

I didn't find any previous posts on this topic, but apologies in
advance if I missed it.

We maintain several internal mirrors of android.git.kernel.org. They
are basically "read-only" in that we don't commit to them and only
pull in new updates continuously.

Is there any suggested maintenance for the mirrors? For example,
would running "git gc --auto" on a weekly basis with repo forall be
advisable? Are there any scenarios where this would not be safe?

thanks,
Garret

[Shawn Pearce]

Yea, probably a `repo forall -c git gc --auto` once a week, or every
couple of weeks, would be a good idea. How frequently you would need
to GC depends on how frequently you fetch, the less often you fetch,
the less small objects are created in the repository, so the less
often you need to GC. Fortunately `git gc --auto` has rules built
into it to mostly guess this correctly, and only repack if its
necessary, so doing `repo forall -c git gc --auto` about once a week
is fairly reasonable.

There really isn't any danger to repacking repositories, Git and
Gerrit are quite good about handling concurrent read access to a
repository that is actively being repacked. The only downside to
repacking is incremental filesystem backups would be larger, as there
are now new files that contain a complete copy of the repository.

FWIW, I repack android.git.kernel.org about once a month. However,
because of the way Gerrit Code Review creates objects on disk, I
repack with the -f flag, to force recomputation of all deltas. Its
rather time consuming, and easily takes an hour or more.

[johnny]

Hi,

I am also maintaining a mirror server. I have another issue with repo
sync. If currently I am using repo init -b cupcake, only the projects
in cupcake manifest will be synced. If I want to sync projects only
within donut, I have to repo init -b donut and repo sync again. I
wonder if there is a way to sync the server completely in one time.

Regards,
Johnny

[Jean-Baptiste Queru]

I think it'd make more sense to start from the output of "gerrit
ls-projects" (run it straight on gerrit's port), and work from there.

JBQ

--
Jean-Baptiste M. "JBQ" Queru
Software Engineer, Android Open-Source Project, Google.

Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.

[Garret]

In case anyone else runs into this, if you see a message saying:

"Auto packing deferred; not on AC"

when running git gc --auto, this is coming from a sample git hook
script.

According to Sean,

"We chose to distribute the sample pre-auto-gc hook as part of repo,
and automatically enable it, due to the high number of android
developers who use laptops"

The problem I have is that the hook is triggering even though I'm
running this on a server which is always on AC power. You may need to
modify or disable the hook like I did.