Re: Cloning one branch with repo

[Magnus Bäck]

On Wednesday, January 02, 2013 at 20:37 EST,
Gulshan Singh <gsing...@gmail.com> wrote:

> With Git, you can clone a specific branch as seen here:
> http://stackoverflow.com/questions/1911109/git-clone-a-specific-branch
> and here:
> http://stackoverflow.com/questions/1778088/how-to-clone-a-single-branch-in-git
>
> The benefit I see with this is you don't waste space and time fetching
> branches that you're not going to use. I've never needed this when
> using Git, but I feel like it may be a more important feature with
> repo. With repo, you can manage multiple Git repositories, so the
> amount and time and space wasted fetching unused branches is actually
> significant.

Have you measured the effect?

> I thought of this while I was cloning AOSP. It takes a long time to
> do a complete repo sync and I'm running out of space on my Ubuntu
> partition as well.

Remember that the AOSP codeline is continuous, i.e. the branch of
release N+1 contains all the commits from the branch of release N.
This suggests that not downloading release N and older won't actually
reduce the data you need to fetch and store. In reality there are
maintenance branches so this statement isn't entirely true, but the
bulk of all commits are shared between subsequent branches.

> Is it possible to do this with repo? If not, does it sound like
> something useful enough to go on your todo list?

It should certainly be possible to do, but in many cases (AOSP in
particular) I doubt the effect will be significant.

What might be more useful is to use the --depth option to "repo init"
to create shallow clones.

--
Magnus Bäck
ba...@google.com

[Jean-Baptiste Queru]

repo sync -c does what you want.

That being said, do you have numbers that show how significant the
difference actually is? For all active projects, old releases are
merged into the latest release anyway, so you'll download them anyway,
and inactive projects are removed from the manifest entirely.

JBQ

On Wed, Jan 2, 2013 at 5:37 PM, Gulshan Singh <gsing...@gmail.com> wrote:
> With Git, you can clone a specific branch as seen here:
> http://stackoverflow.com/questions/1911109/git-clone-a-specific-branch and
> here:

> http://stackoverflow.com/questions/1778088/how-to-clone-a-single-branch-in-git.

>
> The benefit I see with this is you don't waste space and time fetching
> branches that you're not going to use. I've never needed this when using
> Git, but I feel like it may be a more important feature with repo. With
> repo, you can manage multiple Git repositories, so the amount and time and
> space wasted fetching unused branches is actually significant.
>

> I thought of this while I was cloning AOSP. It takes a long time to do a
> complete repo sync and I'm running out of space on my Ubuntu partition as
> well.
>

> Is it possible to do this with repo? If not, does it sound like something
> useful enough to go on your todo list?
>

> --
> To unsubscribe, email repo-discuss...@googlegroups.com
> More info at http://groups.google.com/group/repo-discuss?hl=en



--
Jean-Baptiste M. "JBQ" Queru
Technical Lead, Android Open Source Project, Google.

Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.

[Jean-Baptiste Queru]

Answering my own question, some numbers:

in master, there's no difference, as expected:
default: 7.7G
-c: 7.7G

in gingerbread, there's a decent difference:
default: 5.3G
-c: 2.3G

More information:

The Android servers are configured for a 2-step download, where a
static bundle snapshot is downloaded first over a plain https GET, and
the incremental difference is then downloaded over the regular git
https POST. Even with -c, the static bundle gets downloaded anyway,
and that contains all the branches; that's a repo bug, I think.
However, with --depth, repo correctly bypasses the static clone bundle
and goes straight over git.

Now, you can use --depth=1 in repo init. This makes things smaller,
with some significant drawbacks. You're using a non-resumable
protocol, so you need a reliable enough connection. You won't have any
change history. You won't be able to push from such a client
(including the ability to upload changes). Future syncs might
re-download everything again.

gingerbread, -c --dedpth=1: 1012M

Beyond -c and --depth, another approach is -g, to specify manifest
groups. E.g. if you don't need the darwin host tools, only build for
ARM, and don't worry about the platform but not the support files for
the flagship devices, you can use -g
all,-notdefault,-darwin,-x86,-mips,-device.

master, -c, -g, --depth=1: 2.8G

Note: all those numbers were measured against a local mirror of AOSP,
not against the live servers. While I've tried to document the
differences I know between a mirror and the live servers, I might have
missed some.

JBQ