Relevance of jgit config: gc.auto and gc.autoPackLimit in Gerrit

198 views
Skip to first unread message

Musab Shakeel

unread,
Sep 28, 2024, 10:15:31 AM9/28/24
to Repo and Gerrit Discussion
`gc.auto` and `gc.autoPackLimit` control what happens when `git gc --auto` is run.

Some examples of commands that run `git gc --auto`:
- pull / merge / rebase / commit

On the Gerrit side, what are scenarios / commands where the JGit equivalent of  `git gc --auto` might end up getting called?

Thanks!

Musab Shakeel

unread,
Oct 8, 2024, 10:13:32 AM10/8/24
to Repo and Gerrit Discussion
Just an update - we found out that `gc.auto` and `gc.autoPackLimit` are indeed very important for the JGit config for Gerrit. 

`gc.auto` defaults to 6700. We had it set to 20 for a long time (years) and didn't experience any issues with Gerrit's JGit GC. Last week, we tried changing it to 2000 (it wasn't clear why we'd ever set it to 20 - didn't have gerrit.config checkin to SCM historically), and all of a sudden disk usage went up drastically. The routine Gerrit JGit GC for some reason stopped repacking / optimizing the packfiles. Some of the busy repos started accumulating hundreds of packfiles and Gerrit GC would not optimize them. The `gc.log` file in these busy repos during a Gerrit GC run would say:

```
Too many loose, unpruneable objects after garbage collection. Consider adjusting gc.auto or gc.pruneExpire
```

We "fixed" these repos by simply running a `git gc` in them. Not clear to me why `git gc` trivially repacked correctly, but Gerrit's JGit GC would just not do that:

[2024-10-07 16:31:35,086] INFO  : [my-repo] before: numberOfPackedRefs=264778, numberOfPackFiles=116, sizeOfLooseObjects=1571752, numberOfLooseObjects=820, numberOfPackedObjects=111730806, numberOfBitmaps=5152, numberOfLooseRefs=5, sizeOfPackedObjects=38368357392
[2024-10-07 16:38:48,106] INFO  : [my-repo] after:  numberOfPackedRefs=264782, numberOfPackFiles=121, sizeOfLooseObjects=1803370, numberOfLooseObjects=830, numberOfPackedObjects=116094265, numberOfBitmaps=5796, numberOfLooseRefs=7, sizeOfPackedObjects=39919702803

Musab Shakeel

unread,
Oct 8, 2024, 10:16:11 AM10/8/24
to Repo and Gerrit Discussion
> On the Gerrit side, what are scenarios / commands where the JGit equivalent of  `git gc --auto` might end up getting called?

Seems like the answer is - git receive-pack calls `git gc --auto` equivalent [1]. And, on gerrit server side, `merge / rebase` also get called (e.g. users can use UI to merge / rebase). So `gc.auto` and `gc.autoPackLimit` are very relevant.

Matthias Sohn

unread,
Oct 10, 2024, 6:01:46 AM10/10/24
to Musab Shakeel, Repo and Gerrit Discussion
On Tue, Oct 8, 2024 at 4:16 PM 'Musab Shakeel' via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
> On the Gerrit side, what are scenarios / commands where the JGit equivalent of  `git gc --auto` might end up getting called?

Seems like the answer is - git receive-pack calls `git gc --auto` equivalent [1]. And, on gerrit server side, `merge / rebase` also get called (e.g. users can use UI to merge / rebase). So `gc.auto` and `gc.autoPackLimit` are very relevant.



We switch off auto gc in jgit.config and schedule gc from another process (not in Gerrit's JVM process):

gc.auto = 0
receive.autogc = false

if you are using Gerrit gc then you can in addition switch off autoDetach since Gerrit runs gc already in a background thread

gc.autoDetach = false

On Tuesday, October 8, 2024 at 10:13:32 AM UTC-4 Musab Shakeel wrote:
Just an update - we found out that `gc.auto` and `gc.autoPackLimit` are indeed very important for the JGit config for Gerrit. 

`gc.auto` defaults to 6700. We had it set to 20 for a long time (years) and didn't experience any issues with Gerrit's JGit GC. Last week, we tried changing it to 2000 (it wasn't clear why we'd ever set it to 20 - didn't have gerrit.config checkin to SCM historically), and all of a sudden disk usage went up drastically. The routine Gerrit JGit GC for some reason stopped repacking / optimizing the packfiles. Some of the busy repos started accumulating hundreds of packfiles and Gerrit GC would not optimize them. The `gc.log` file in these busy repos during a Gerrit GC run would say:

```
Too many loose, unpruneable objects after garbage collection. Consider adjusting gc.auto or gc.pruneExpire
```

We "fixed" these repos by simply running a `git gc` in them. Not clear to me why `git gc` trivially repacked correctly, but Gerrit's JGit GC would just not do that:

[2024-10-07 16:31:35,086] INFO  : [my-repo] before: numberOfPackedRefs=264778, numberOfPackFiles=116, sizeOfLooseObjects=1571752, numberOfLooseObjects=820, numberOfPackedObjects=111730806, numberOfBitmaps=5152, numberOfLooseRefs=5, sizeOfPackedObjects=38368357392
[2024-10-07 16:38:48,106] INFO  : [my-repo] after:  numberOfPackedRefs=264782, numberOfPackFiles=121, sizeOfLooseObjects=1803370, numberOfLooseObjects=830, numberOfPackedObjects=116094265, numberOfBitmaps=5796, numberOfLooseRefs=7, sizeOfPackedObjects=39919702803
On Saturday, September 28, 2024 at 10:15:31 AM UTC-4 Musab Shakeel wrote:
`gc.auto` and `gc.autoPackLimit` control what happens when `git gc --auto` is run.

Some examples of commands that run `git gc --auto`:
- pull / merge / rebase / commit

On the Gerrit side, what are scenarios / commands where the JGit equivalent of  `git gc --auto` might end up getting called?

Thanks!

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/a9b6ab32-26d6-4142-8dcb-c716bb82778en%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages