Re: Reducing Gerrit size on disk
157 views
Skip to first unread message
Magnus Bäck
Aug 29, 2012, 1:15:54 AM8/29/12
to repo-d...@googlegroups.com
On Tuesday, August 28, 2012 at 19:44 EDT,
Will DeBerry <willd...@gmail.com> wrote:
> Couldn't find any answers anywhere else so figured I would turn here.
> Was wondering if there was anyway to reduce the size of gerrit at all?
I assume you mean the size of the Git repositories managed by Gerrit.
> Maybe some way to clean it up.
That's what "git gc" is for, and you should run it with sufficient
frequency (depending on how active the gits are) to avoid performance
degrades and keep the repository size down.
> Our projects have branches on them that aren't being used anymore, so
> I started removing the branches but realized that this isn't helping
> as well as I thought it would. Basically our disk space is double of
> what it should be, so just looking for some help.
Why are the gits so big? Are you checking in huge amounts of binaries?
How big is big? The "why so big" is important as it affects the options.
Deleting branches will only make a noticeable difference if the commits
on the deleted branches refer to files that aren't used elsewhere. If
you've merged branch A into branch B and delete branch A, you won't save
anything.
Assuming the objects you want to get rid of truly are unreferenced, "git
gc" will get rid of them. However, Git will keep around unreachable
objects for a while after their creation (two weeks by default, I
think), but that can be changed with the --prune option. Also, the
commits better not be referenced in the reflog -- that's another safety
mechanism to avoid data loss.
--
Magnus Bäck
ba...@google.com
Will DeBerry <willd...@gmail.com> wrote:
> Couldn't find any answers anywhere else so figured I would turn here.
> Was wondering if there was anyway to reduce the size of gerrit at all?
I assume you mean the size of the Git repositories managed by Gerrit.
> Maybe some way to clean it up.
That's what "git gc" is for, and you should run it with sufficient
frequency (depending on how active the gits are) to avoid performance
degrades and keep the repository size down.
> Our projects have branches on them that aren't being used anymore, so
> I started removing the branches but realized that this isn't helping
> as well as I thought it would. Basically our disk space is double of
> what it should be, so just looking for some help.
Why are the gits so big? Are you checking in huge amounts of binaries?
How big is big? The "why so big" is important as it affects the options.
Deleting branches will only make a noticeable difference if the commits
on the deleted branches refer to files that aren't used elsewhere. If
you've merged branch A into branch B and delete branch A, you won't save
anything.
Assuming the objects you want to get rid of truly are unreferenced, "git
gc" will get rid of them. However, Git will keep around unreachable
objects for a while after their creation (two weeks by default, I
think), but that can be changed with the --prune option. Also, the
commits better not be referenced in the reflog -- that's another safety
mechanism to avoid data loss.
--
Magnus Bäck
ba...@google.com
Lundh, Gustaf
Aug 29, 2012, 8:30:03 AM8/29/12
to Magnus Bäck, repo-d...@googlegroups.com
Not only will unreferenced objects not be pruned directly, but deleting branches can even temporary deflate the repositories and make them _huge_.
When git gc is executed after a ref has been removed, there will most likely be a lots of loose objects. And these objects are lifted out from the pack-files into separate object-files while waiting to be pruned. Until those objects actually are --prune:d, depending on your GIT, you can see a vastly increased size of the repository. I've seen repositories increase to 5-6 times their previous size, just because of this.
Cheers
G
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
When git gc is executed after a ref has been removed, there will most likely be a lots of loose objects. And these objects are lifted out from the pack-files into separate object-files while waiting to be pruned. Until those objects actually are --prune:d, depending on your GIT, you can see a vastly increased size of the repository. I've seen repositories increase to 5-6 times their previous size, just because of this.
Cheers
G
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
Magnus Bäck
Aug 29, 2012, 8:50:37 AM8/29/12
to repo-d...@googlegroups.com
On Wednesday, August 29, 2012 at 08:30 EDT,
--
Magnus Bäck
ba...@google.com
"Lundh, Gustaf" <Gustaf...@sonymobile.com> wrote:
> Not only will unreferenced objects not be pruned directly, but
> deleting branches can even temporary deflate the repositories and
> make them _huge_.
>
> When git gc is executed after a ref has been removed, there will most
> likely be a lots of loose objects. And these objects are lifted out
> from the pack-files into separate object-files while waiting to be
> pruned. Until those objects actually are --prune:d, depending on your
> GIT, you can see a vastly increased size of the repository. I've seen
> repositories increase to 5-6 times their previous size, just because
> of this.
But "git gc --prune=now" should take care of that, right?
> Not only will unreferenced objects not be pruned directly, but
> deleting branches can even temporary deflate the repositories and
> make them _huge_.
>
> When git gc is executed after a ref has been removed, there will most
> likely be a lots of loose objects. And these objects are lifted out
> from the pack-files into separate object-files while waiting to be
> pruned. Until those objects actually are --prune:d, depending on your
> GIT, you can see a vastly increased size of the repository. I've seen
> repositories increase to 5-6 times their previous size, just because
> of this.
--
Magnus Bäck
ba...@google.com
Lundh, Gustaf
Aug 29, 2012, 8:59:41 AM8/29/12
to Magnus Bäck, repo-d...@googlegroups.com
Yes. --prune will fix it.
I just thought I would mention it, since I spent some time trying to figure out why some repositories on a few of our Gerrit-slaves was very much larger compared to the original repos on our master. And surely enough, due to changed access rights and the usage of remote.authGroup, a few refs was deleted on some of the slaves and soon after a "git gc" had been executed. Perhaps someone else will be running into the same issue.
/G
-----Original Message-----
From: repo-d...@googlegroups.com [mailto:repo-d...@googlegroups.com] On Behalf Of Magnus Bäck
Sent: den 29 augusti 2012 14:51
To: repo-d...@googlegroups.com
Subject: Re: Reducing Gerrit size on disk
I just thought I would mention it, since I spent some time trying to figure out why some repositories on a few of our Gerrit-slaves was very much larger compared to the original repos on our master. And surely enough, due to changed access rights and the usage of remote.authGroup, a few refs was deleted on some of the slaves and soon after a "git gc" had been executed. Perhaps someone else will be running into the same issue.
/G
-----Original Message-----
From: repo-d...@googlegroups.com [mailto:repo-d...@googlegroups.com] On Behalf Of Magnus Bäck
Sent: den 29 augusti 2012 14:51
To: repo-d...@googlegroups.com
Subject: Re: Reducing Gerrit size on disk
Martin Fick
Aug 29, 2012, 10:32:15 AM8/29/12
to Magnus Bäck, repo-d...@googlegroups.com
You should probably never run that on a live Gerrit server since you could cause new uploads to fail with it.
>--
>To unsubscribe, email repo-discuss...@googlegroups.com
>More info at http://groups.google.com/group/repo-discuss?hl=en
Employee of Qualcomm Innovation Center,Inc. which is a member of Code Aurora Forum
>To unsubscribe, email repo-discuss...@googlegroups.com
>More info at http://groups.google.com/group/repo-discuss?hl=en
Reply all
Reply to author
Forward
0 new messages