details of gclient sync - are DEPS repos stored locally with their branches/history?

[Duaa Abdelgadir]

When you initally fetch chromium and gclient sync, does the history for these third party repos in DEPS get stored?
Ive run gclient sync with -verbose option to get a closer look, and it looks like its accessing internet for each checkout, even when the revision has not changed. It looks like this fetch is coming from _fetch function within git-cache.py in depot_tools. Is this access for bringing in the appropriate revision or something else?
(I also ran with --nohooks if that helps)

When I comment out the _fetch function, it looks like its still retrieving the appropriate revisions, but using information locally instead of going to the remote repo.

Ex: From https://chromium.googlesource.com/chromium/buildtools vs.  From /home/user/chromium/src/testdir/chromium.googlesource.com-chromium-buildtools

with _fetch


________ running 'git -c core.deltaBaseCacheLimit=2g fetch origin --verbose' in '/home/user/chromium/src/buildtools'
[0:00:00] From https://chromium.googlesource.com/chromium/buildtools
[0:00:00]  = [up to date]      master     -> origin/master
[0:00:00] Up-to-date; skipping checkout.
Checked out revision 39b1db2ab4aa4b2ccaa263c29bdf63e7c1ee28aa
[0:00:00] Finished.

without _fetch


________ running 'git -c core.deltaBaseCacheLimit=2g fetch origin --verbose' in '/home/user/chromium/src/src/buildtools'
[0:00:09] From /home/user/chromium/src/testdir/chromium.googlesource.com-chromium-buildtools
[0:00:09]  = [up to date]      master     -> origin/master
[0:00:09] Up-to-date; skipping checkout.
Checked out revision adb8bf4e8fc92aa1717bf151b862d58e6f27c4f2
[0:00:09] Finished.

Let me know if you need more clarification. Thanks alot

[Primiano Tucci]

-chromium-dev +infra-dev

> does the history for these third party repos in DEPS get stored

Do you mean the *git* history? In that case yes, unless you do gclient sync --no-history

> It looks like this fetch is coming from _fetch function within git-cache.py in depot_tools. Is this access for bringing in the appropriate revision or something else? 

Maybe gclient always doing a remote-update/fetch on the remotes speculating that the DEPS have been rolled to something newer.

Probably it could be smarter and avoid fetching if the commit pinned by DEPS is already there?

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Duaa Abdelgadir]

Any idea where the current revision is stored and reset to the new one? It seems that at the start of update function in gclient_scm, the new revision info is already set. Where can I compare the pinned DEPS to the new one to see if there have been any changes?

Thanks!

[Aaron Gable]

In general, gclient isn't invested in making sure that the filesystem is in the most recent state it was "supposed" to be in. In particular, such functionality would break on a lot of bots which do crazy things to their filesystem. It does store the most recently sync'd state in the ".gclient_entries" file, adjacent to the ".gclient" file at the root, but it rarely references that file.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.
To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/8b3b6847-ac0d-4957-b239-9fe79f43d88e%40chromium.org.

[Stefan Zager]

On Fri, Oct 21, 2016 at 7:06 AM, Duaa Abdelgadir <duaa.ab...@gmail.com> wrote:

When you initally fetch chromium and gclient sync, does the history for these third party repos in DEPS get stored?
Ive run gclient sync with -verbose option to get a closer look, and it looks like its accessing internet for each checkout, even when the revision has not changed. It looks like this fetch is coming from _fetch function within git-cache.py in depot_tools. Is this access for bringing in the appropriate revision or something else?
(I also ran with --nohooks if that helps)

There used to be logic in gclient to avoid hitting the network if the pinned revision was already present, but that may have been later disabled.  And I'm not sure whether that logic ever worked properly when you're using a local git cache.

[Aaron Gable]

Yeah, that logic has gotten messier. My CL (still not yet uploaded) is fixing a lot of it. Unfortunately, none of it applies if you have a git cache. Both before and after my CL (to avoid changing too many things at once), the behavior is "if you have a cache, tell it to update itself; then we'll see if I need to pull anything from it.".