Make gclient sync --with_branch_heads the default?

[Torne (Richard Coles)]

A lot of people appear to be confused by the fact that checking out a release tag and then running gclient sync often doesn't work, judging by the frequency with which we get emails/questions on iRC/etc about this (which are answered with "use gclient sync --with_branch_heads).

I just fetched a fresh checkout on my machine and compared the sizes with and without that option, and it turns out that currently the difference is only 0.4GB of git data: the total size of all .git directories in the checkout after "fetch chromium" is 13GB, and after doing "gclient sync --with_branch_heads" it only increases to 13.4GB.

This saving doesn't seem worth it - I suspect it was a much larger saving before third_party/WebKit was merged into src..

Should we just change the default behaviour?

We might need to consider what happens for checkouts with --no-history, though.. maybe just only do it for non-shallow checkouts?

[Torne (Richard Coles)]

Related question I just found while looking at what the impact of this would be on --no-history: is fetch --no-history entirely broken, or am I holding it wrong?

It looks like the effect of "fetch --no-history" is to clone all the repos involved with --depth=1, which is all well and good - but the fetch command then runs "git config --add remote.origin.fetch '+refs/tags/*:refs/tags/*'" in src immediately after the sync completes. This means that the very first time you run "git pull" or any similar command in the src repository, almost all of history gets downloaded in the process of downloading every single release tag that's ever existed. The only revisions that you *don't* end up downloading are the ones that are more recent than the most recent release tag, but older than the tip of tree at the time of the initial clone, which is only going to be max 1 day of commits.

The total size of all .git directories after "fetch --no-history chromium" is 3.7GB.

After running "cd src; git pull" to update it just once, it grows to 13.1GB.

So, unless I'm missing something it looks like fetch shouldn't be adding the tags to the fetch spec when passed --no-history..

[Ryan Tseng]

Shallow clones aren't well supported by the git service that we use because it bypasses a majority of the pre-computation and optimization that is done on the backend.  Shallow clones also have finicky quirks, so officially they're a use-at-your-own-risk sort of feature.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEV-rjcFgM8bBwARGedKTXp%3D6H2-Zka7yHN9v5-rM49-JVv8XA%40mail.gmail.com.

[Torne (Richard Coles)]

I'm aware of that class of issue, but this specific problem seems to be an actual bug in "fetch" that entirely defeats the purpose of making a shallow clone in the first place, rather than just a performance problem or a limitation to be aware of :)

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Ryan Tseng]

My first guess would be that it's due to the ignore/foo and ignore/bar branches that we added in to ensure that tags are reachable and included in the bitmap index.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

[Torne (Richard Coles)]

It's due to depot_tools's fetch.py explicitly configuring the local git settings to download all tags :) I've uploaded a CL to fix this here: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/909653

So, does anyone have any thoughts on changing the gclient behaviour to sync --with_branch_heads by default? I can just prepare a CL that adds a negated option and sets the default based on whether the checkout appears to be shallow.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[san...@chromium.org]

I am all for changing the default. I and several coworkers each have scripts that wrap gclient sync just so that it can pass --wtih_branch_heads by default.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

[Dirk Pranke]

I think it's fine to change the default.

-- Dirk

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/a1d12324-9f2b-4692-b258-1af69f22b452%40chromium.org.

[Michael Moss]

Agreed. Back when that was added, the extra 1/2 GB of data was a significant chunk of the checkout. Nowadays, not so much.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEoffTDG2aCuo2gNUBOPPNQDxk7VDnbZFSGSQUz5qFo139xB7A%40mail.gmail.com.

[Torne (Richard Coles)]

No, sorry - I have had this on my todo list for a while but it wasn't quite as straightforward as I hoped and I haven't gotten around to continuing with it. If you'd like to do it, feel free :)

On Thu, 25 Oct 2018 at 13:33 <xiao...@chromium.org> wrote:

Did we actually change the default? 

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[xiao...@chromium.org]

Can you elaborate a little bit why "it wasn't quite as straightforward" before I make any commitments :)

[xiao...@chromium.org]

Did we actually change the default? 

On Thursday, February 8, 2018 at 2:33:00 PM UTC-8, Michael Moss wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEV-rjcFgM8bBwARGedKTXp%3D6H2-Zka7yHN9v5-rM49-JVv8XA%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/a1d12324-9f2b-4692-b258-1af69f22b452%40chromium.org.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

[Torne (Richard Coles)]

I don't remember offhand, sorry; it was a long while ago that I had a look at it. I assume it just ended up not being something I could get done and test in an afternoon and so I didn't get around to it.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/ca74903e-1997-4e9d-9654-5389fdcb353e%40chromium.org.