kernel git repo pruning

[Chirantan Ekbote]

​My most recent run of repo sync is giving the following warnings:

​From https://chromium.googlesource.com/a/chromiumos/third_party/kernel

   7cb3b304a6a84..2a228ce41cc70  chromeos-3.18 -> cros/chromeos-3.18

   dde96a05aa6bb..1eebe41b56b69  release-R56-9000.B-chromeos-3.10 -> cros/release-R56-9000.B-chromeos-3.10

   b896e6bf7f7d7..96be95bbfc849  release-R56-9000.B-chromeos-3.8 -> cros/release-R56-9000.B-chromeos-3.8

Auto packing the repository in background for optimum performance.

See "git help gc" for manual housekeeping.

error: The last gc run reported the following. Please correct the root cause

and remove /media/ssd/chromiumos/.repo/projects/src/third_party/kernel/v3.10.git/gc.log.

Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

​This causes 'repo sync' to fail because of errors.  There are similar errors for each of the kernel repos.  If I remember correctly we actually disabled pruning on the kernel repo because pruning could corrupt it (crbug.com/375945) due to the fact that it's different branches of the same repo checked out in different parts of the tree.  So I'm a bit concerned about following the tool's suggestion of running 'git prune'.

What's the right way to deal with this? 

Thanks,

Chirantan

[Gabe Black]

You should definitely wait for David to say what to do, but you're right that we disable garbage collection on repositories where the object database is shared between multiple checkouts of the repository. That's because when git garbage collects in one of the checkouts, it assumes that if it can get to an object from the branches, etc., there then nobody can get to it. It's quite possible one of the other checkouts needs it though, and will be upset if it gets deleted out from under it. I would also be concerned about running git prune for that reason.

My guess is that you can just delete gc.log and it will stop complaining, but it would probably be safest to wait until David replies.

Gabe

[Mike Frysinger]

hmm, does the corruption only affect branches you've created ?  each repo should have the same of remote branches, so if there's nothing local, running gc yourself should work.

-mike

--
--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en

[Doug Anderson]

I would say that these types of problems are quite common for me.  I'm often picking things from gerrit, rebasing, fetching linuxnext, etc.  It adds lots of garbage to my kernel tree.  I need to periodically gc / prune (otherwise certain "git" operations become super slow), but when I do so it always messes things up.

Recently I've been trying things like:

* Add other kernel trees as "remote" to the one I'm going to gc/prune on, fetch them, then gc/prune.

* Clone my kernel tree elsewhere on my hard disk to make sure that even if I mess up I don't lose work.

I haven't spent much time debugging.  Usually I just start killing .git/refs/blah whenever it gets messed up and try to salvage what I can.

NOTE: this also always seems to mess up each time a new kernel branch is added, possibly because it does a gc and/or prune sometime before it finishes adding everything (so it hasn't disabled it yet).  No idea if it still does this--I'd have to tell you next time we add a new kernel if I'm in the "bad" state...

-Doug

[Sonny Rao]

On Fri, Jan 13, 2017 at 4:30 PM, Doug Anderson <dian...@chromium.org> wrote:
> I would say that these types of problems are quite common for me. I'm often
> picking things from gerrit, rebasing, fetching linuxnext, etc. It adds lots
> of garbage to my kernel tree. I need to periodically gc / prune (otherwise
> certain "git" operations become super slow), but when I do so it always
> messes things up.
>
> Recently I've been trying things like:
> * Add other kernel trees as "remote" to the one I'm going to gc/prune on,
> fetch them, then gc/prune.

^^^

I recently had to do this because doing a simple fetch from origin
cros took 5 minutes :-/
I lost a couple of local branches somehow (not sure why) but mostly
everything I cared about survived

You can also push important local branches to your sandbox which is
what I usually do anyway because what will you do if your SSD dies?

> You received this message because you are subscribed to the Google Groups
> "Chromium OS dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chromium-os-d...@chromium.org.

[Chirantan Ekbote]

So just to close the loop in case anyone in the future finds this thread, this is what I did to fix the problem:

$ cd ~/cros/src/third_party/kernel/v4.4

$ git remote add v3.8 ../v3.8

$ git remote add v3.10 ../v3.10

$ git remote add v3.14 ../v3.14

$ git remote add v3.18 ../v3.18

$ git fetch --all

$ git prune

> email to chromium-os-dev+unsubscribe@chromium.org.