Syncing just the manifest branch by default

[Brian Harring]

Hola folks-

Currently, for all repo's except chromiumos-overlay, we pull down all branches for each git-repository.  This is actually is an additional 700MB or so for the private checkouts- data that most people don't actually care about.

This is around 10%; anyone care if I suppress that, and make repo just sync the branches that the manifest specifics?

~brian

[Jonathan Kliegman]

 Overall this seems a good way to save time - I think most of the time I'm not looking at the other branches unless I'm merging a backport and then I generally have a second checkout I use for that.  We'd want an easy way to still get those branches though.

Would I still be able to run 'git branch -r' to see the branches?  And just take a hit the first time I pulled one?  Or would I not even know they existed and have to do some tricks to pull them?

Would it be possible to add a way to annotate the manifest with "also sync these branches" or to modify something in the git metadata to do the equivalent?  I feel there are times (updating to new kernel versions) where people working in the tree would care about multiple branches at once.  It'd be nice to make things simpler in those cases. 

~brian

--
--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en
 
 
 

[Sonny Rao]

On Thu, Nov 15, 2012 at 10:58 PM, Jonathan Kliegman <kli...@chromium.org> wrote:
>
>
>
> On Fri, Nov 16, 2012 at 1:07 AM, Brian Harring <ferr...@google.com> wrote:
>>
>> Hola folks-
>>
>> Currently, for all repo's except chromiumos-overlay, we pull down all
>> branches for each git-repository. This is actually is an additional 700MB
>> or so for the private checkouts- data that most people don't actually care
>> about.
>>
>> This is around 10%; anyone care if I suppress that, and make repo just
>> sync the branches that the manifest specifics?
>
> Overall this seems a good way to save time - I think most of the time I'm
> not looking at the other branches unless I'm merging a backport and then I
> generally have a second checkout I use for that. We'd want an easy way to
> still get those branches though.
>
> Would I still be able to run 'git branch -r' to see the branches? And just
> take a hit the first time I pulled one? Or would I not even know they
> existed and have to do some tricks to pull them?
>
> Would it be possible to add a way to annotate the manifest with "also sync
> these branches" or to modify something in the git metadata to do the
> equivalent? I feel there are times (updating to new kernel versions) where
> people working in the tree would care about multiple branches at once. It'd
> be nice to make things simpler in those cases.

Or just an argument to repo sync that would sync all (or some
branches) would work.

[Hung-Te Lin]

On Fri, Nov 16, 2012 at 5 :25 PM, Sonny Rao <sonn...@chromium.org> wrote:

On Thu, Nov 15, 2012 at 10:58 PM, Jonathan Kliegman <kli...@chromium.org> wrote:
> On Fri, Nov 16, 2012 at 1:07 AM, Brian Harring <ferr...@google.com> wrote:
>> Currently, for all repo's except chromiumos-overlay, we pull down all
>> branches for each git-repository.  This is actually is an additional 700MB
>> or so for the private checkouts- data that most people don't actually care
>> about.
>>
>> This is around 10%; anyone care if I suppress that, and make repo just
>> sync the branches that the manifest specifics?
>
>  Overall this seems a good way to save time - I think most of the time I'm
> not looking at the other branches unless I'm merging a backport and then I
> generally have a second checkout I use for that.  We'd want an easy way to
> still get those branches though.
>
> Would I still be able to run 'git branch -r' to see the branches?  And just
> take a hit the first time I pulled one?  Or would I not even know they
> existed and have to do some tricks to pull them?
>
> Would it be possible to add a way to annotate the manifest with "also sync
> these branches" or to modify something in the git metadata to do the
> equivalent?  I feel there are times (updating to new kernel versions) where
> people working in the tree would care about multiple branches at once.  It'd
> be nice to make things simpler in those cases.

Or just an argument to repo sync that would sync all (or some
branches) would work.

Just like the "--depth" param in repo, we should make sure there's an easy way for people who needs to cherry-pick a lot to work with.

[Brian Harring]

Our repo sync doesn't currently have such an option, although I expect one to appear soon enough.  If you wanted all branches w/in a repository, `git remote update` currently would suffice.

Just like the "--depth" param in repo, we should make sure there's an easy way for people who needs to cherry-pick a lot to work with.

I'd expect that cherry-picking is mostly limited to non ToT branches; we can easily enough have those manifests sync all branches rather than just the manifest defined one.  Really depends on what people prefer for this.

~harring

[Hung-Te Lin]

Just to double confirm, is this change already landed?

Not sure what I've done, but my recent check out of chromiumos-overlay contains only cros/master, not any other branches like release-R23-XXXX.

[Brian Harring]

Crap, this fell off my todo and wasn't landed globally.

What you're seeing in your case is chromiumos-overlay explicitly being forced to sync-c; that's been configured that way for 3-4 months now, this thread was for enabling it for all repositories by default.

Barring complaints, this will land pre christmas (and yes, I'm aware people probably won't see the email till post christmas). ;)
~brian

[Hung-Te Lin]

Where is the change that made chromiumos-overlay forced sync-c -- is it in repo program, depot_tools, or on remote gerrit/git server configuration?

And is there any document or instruction for how to disable that (force sync-c) behavior if I really want all branches downloaded?

Because I don't want to do "repo init -b" each time when I need to work on multiple branches, ex "R23, R25, stabilize, stabilize2".

[Hung-Te Lin]

I think it may probably be "git remote update" but I'm not quite sure about how that command works;

should I always do that after each "repo sync", or do I only need to do it one time and all further "repo sync" calls will get contents from all branches after I've done that?

[Brian Harring]

The change that enabled this behaviour for chromiumos-overlay is https://gerrit.chromium.org/gerrit/#/c/34419/  from 10/2/12; we control it via the manifest (it's not possible to control that from server side).

Roughly, w/out that setting, repo does a `git fetch cros '+refs/heads/*:refs/remotes/cros/*'`; w/ the setting, it limits it to +refs/heads/master:refs/remotes/cros/master (or whatever the manifest defined branch is).

So... if you want to snag branches beyond what repo grabs, you can either do `git remote update`- which grabs *all* branches for that repo, or you can do a fetch of what you want.

I'd suggest taking a look at the manpage for git config (git config --help), same for remote (git remote --help).  That'll describe how you can modify the git config to limit/control what all `git remote update` grabs if you don't want everything.

Either way, note that's all modifying git- repo however will still keep doing a `git fetch cros +refs/heads/master:refs/heads/master` regardless of how you' mangled the local configuration.  If you'd like to change that behaviour in some fashion, please grab a checkout of https://code.google.com/p/git-repo and see if you can make the change.

If you can't figure it out, then file a bug my way and I'll get it when I can.

That said, I'd really appreciate it if you or someone else wrote the patch- we really need to increase the bus factor for git-repo modifications beyond just myself. :)

Cheers-
~harring