[PSA] Git Submodules in chromium/src and significant git workflow changes

[Josip Sokcevic]

Tl;dr - We are switching chromium/src to use git submodules and this change will impact developer workflows. Always run gclient sync after checking out a new commit (e.g. after git pull) or you will need to explicitly add files that you intend to change.

We are adding git submodules to chromium/src. With that, git submodule diffs will show up in repos that have git dependencies. Not running gclient sync after git pull or git checkout, may result in outdated git submodule checkouts and git showing unstaged submodule commits.

What does this mean for me?

Always run gclient sync after updating your chromium checkout (e.g. git pull, git checkout), and you shouldn't see any changes in your developer workflow. Rely on git status to see if there are any unstaged submodule changes, especially if you rely on git add [*|.|-A|-u]. If you rely on git commit -a, check the "Changes to be committed" section that shows up in the edit commit message.

If you accidentally stage or commit a git submodule, read our git submodule documentation which will guide you how to undo it.

What if I accidentally staged git submodule (not yet committed)?

If you accidentally stage a git submodule, you can unstage it by running git restore --staged <path to submodule>.

What if I accidentally committed a git submodule?

We will need to create either a commit that sets it back to old value, or amend the commit that added it. You can try to run gclient sync to bring the commit back to what is expected. If that doesn't work, you can use gclient setdep -r <path>@{old hash}, run gclient gitmodules to sync all submodules commits back to what is in DEPS, or check detailed instructions in Managing dependencies.

How do I update Chromium dependencies?

If you have a Skia autoroller set, we got you covered - no action is needed. For manual rolls, please follow our official documentation for managing dependencies.

Note that we continue to store git dependencies in DEPS files, and we expect it to be in sync with git submodules until we can safely remove the git DEPS entries. This will be enforced with presubmit checks.

When will this change take effect?

We will be adding git submodules to chromium/src within a week. For all projects that currently use git submodules, this change takes effect immediately. 

Why are we doing this?

As outlined in this doc, we want to bring Chromium development workflow closer to conventional git use and this means using git submodules.

Can I opt out?

Unfortunately, you cannot opt-out. Submodules will be part of the chromium/src repo. 

If you wish, you can instruct git to assume individual git submodules are unchanged by running: git update-index --assume-unchanged <path to submodule>. To undo, run git update-index --no-assume-unchanged <path to submodule>.

How can I provide feedback?

Please file bugs for any issues you find. These are big changes and we encourage you to reach out when you encounter any problems or confusion. Even if you assume some new behavior is expected / associated with git submodules it is still good to reach out so we and the git team can be aware of workflows that may be improved.

Useful links

Guide to git submodules

Managing dependencies

Provide Feedback

Previous announcement

[Josip Sokcevic]

Hi everyone,

As of 15 minute ago, chromium/src contains git submodules. The change is available on the main branch: https://crrev.com/ef72622958e9358800d88739155030deaab314f3. The first release branch that will contain git submodules will be M118.

As requested in the original email, please do provide feedback by filing bugs under the Infra>SDK component. 

--

Josip Sokcevic

[Avi Drissman]

Even if I do "gclient sync" I'm seeing submodules show up as uncommitted changes. I filed https://crbug.com/1475339; please help.

Avi

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOijJGEjZJYGfaOm3h-%2BvyJVM4e5RxC5DT8mmk-YkCT3mnA%40mail.gmail.com.

[Sven Zheng]

I also filed crbug.com/1475324. Currently stuck and can't upload any changes.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CACWgwAZxJEgpH_4%3DsBO_QxC%2B2RL22qFw9_OJZYbRQyB2W-rpaw%40mail.gmail.com.

[Josip Sokcevic]

Until we fix that, please avoid using git commit -a / git add *|. If you stage or commit gitlink, here are instructions how to unstage / amend commit: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/git_submodules.md

I'm setting all those bugs to P0/P1.

--

Josip Sokcevic

[Olivier Robin]

I don't know if this is related to other issues, but "git cl patch" does not seem to work.

It reports a lot of dirty files that are not displayed in in git status.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOiinmYu3iocqM_ymyp8SZgYmM%2By%3Dk3ebTAcQVUZ-2SxXJA%40mail.gmail.com.

[Josip Sokcevic]

Tomasz filed a bug and we were able to resolve that issue (https://crbug.com/1475324). It appears that gclient can leave git dependencies completely untracked. This is not supposed to happen - user should be always warned to run gclient sync -D until those untracked dependencies are actually removed. We were not aware of this bug until we added git submodules - now git tells us that we are pointing to an old commit. We are tracking that in https://crbug.com/1475448.

Those untracked repositories from gclient perspective are contributing to not updating git dependencies to the state where DEPS points to. If you'd like to run a one-off script to move all unmanaged git dependencies outside of chromium workspace, you can run the following

# Inside chromium/src checkout:

# This ensures that all managed dependencies are in sync:

gclient sync -D 

# This moves all unused dependencies to ../unused directory in gclient root (just outside of src directory). It then tells git to restore gitlink.

for f in $( git status | grep '(new commits)' | awk '{print $2}' ); do mkdir -p "../unused/`dirname $f`" && mv $f "../unused/$f" && git checkout -- $f; done

# inspect ../unused/ if you'd like, and remove it there's nothing useful there, e.g. no non-uploaded commits.

I will update https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/git_submodules.md with instructions too.

Please continue to report any issues that you encounter; as well as problems with the new workflow. We might be able to simplify workflow and provide scripts, but we need your feedback.

p.s. we didn't mention, but please don't use git-submodule CLI since it doesn't work well with gclient (i.e. it has no conditional support) and may cause unique problems that we have yet to address. Instead, always rely on gclient sync to get dependencies for you.

On Wed, Aug 23, 2023 at 2:11 PM Tomasz Wiszkowski <en...@google.com> wrote:

I am seeing the same.

essentially, after invoking `gclient sync` and `git status` i see:

        modified:   chrome/chrome_cleaner/internal (new commits)
        modified:   chrome/installer/mac/third_party/xz/xz (new commits)
        modified:   components/omnibox/browser/autocomplete_grouper_sections.cc
        ... some 30 more ...
        modified:   third_party/material_design_icons/src (new commits)
        modified:   third_party/nacl_sdk_binaries (new commits)

`git reset --hard` won't revert this, `git commit -A` will put this in a commit, and my tree comes up dirty.

is there a way we can reset all these submodules to "clean" state?

--

Josip Sokcevic

[Olivier Robin]

For team working with some submodules (e.g. any _internal sub repo), the submodule is marked as dirty as soon as a branch is created in it.

This is both confusing and can probably lead to some errors, specially it you make a change that touch both src and the internal repo

Can the tools be adapted to avoid this?

[Avi Drissman]

This breaks git fsmonitor (https://crbug.com/1475739). This is incredibly bad; fsmonitor is the only way git is an acceptable speed.

[Haiyang Pan]

+1 for the dirty submodule issue. To add more details, here is what "git status" shows for chromium/src when I have a branch in the submodule "third_party/catapult":

$ git status
HEAD detached at origin/main
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   third_party/catapult (new commits)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAYr5oMA07uduqoze04PEeX8qen%3D3Zt-U0-zsaWSHyh2fmqtsg%40mail.gmail.com.

--

Best,

Haiyang

[Yngve N. Pettersen]

Hi,

Disclaimer: I have not really looked at the current upstream submodule system or worked with it, this is based on my ordinary use of submodules in Vivaldi (which I will have to integrate with this new system).

It might be an idea for those of you who are having the dirty submodule issue after branching to also include the commands you used to create the branch.

A normal branch at HEAD of a submodule checkout should not cause the parent module to show "new commits". It might be an idea to run git diff.

So, in the below case, my suspicion is that the submodule branchpoint (in catapult) was not at the current checkout indicated by the parent module.

My suggestion is to first run a forced submodule update from the parent module, the Git way for doing that is:

   git submodule update -f third_party/catapult

The depot_tools way may be different.

the cd to third_party/catapult and run

  git checkout -b <my_new_branch>

This will create the new branch, starting from the currently checked out commit. It should NOT change the checked out commit.

(I generally do not use git branch, except for very specific scripted operations)

Specifying a different branch or commit that is not currently pointing at HEAD on the line after the new branch name WILL change the checked out commit ID.

Operationally, if you have not done so already, the git server (or Gerrit) should run a hook script that limits submodule commit updates to the main branch to specific branches (main, and possibly some others, all access-controlled) in the submodule (that is, e.g the commit ID pushed to chromium src main branch would have to be on main in the submodule). One thing to be careful about is accidental reverts to an older commit. In the Chromium system, the Gerrit apply scripts might manage that instead. Such a system should prevent accidental changes of the submodule pointer into non-production branches. 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CA%2B1sxgZHdf_wDBPA-JW-j_1tFMBjOxa0f1x8RBuHgGhKiDt5mw%40mail.gmail.com.

-- 
Sincerely,
Yngve N. Pettersen
Vivaldi Technologies AS

[Tomasz Wiszkowski]

I am seeing the same.

essentially, after invoking `gclient sync` and `git status` i see:

        modified:   chrome/chrome_cleaner/internal (new commits)
        modified:   chrome/installer/mac/third_party/xz/xz (new commits)
        modified:   components/omnibox/browser/autocomplete_grouper_sections.cc
        ... some 30 more ...
        modified:   third_party/material_design_icons/src (new commits)
        modified:   third_party/nacl_sdk_binaries (new commits)

`git reset --hard` won't revert this, `git commit -A` will put this in a commit, and my tree comes up dirty.

is there a way we can reset all these submodules to "clean" state?

On Wednesday, August 23, 2023 at 12:42:13 PM UTC-7 Olivier Robin wrote:

[Yngve N. Pettersen]

Hi,

The Git command to reset submodules to the parent module git commit reference is

  git submodule update -f

(-f means "force") and you may need to add a   --recursive flag, too, to handle the sub-submodules 

Of course, the depot_tools system should be fixed to handle this.

Beware, though: One possibility is that the commit IDs in DEPS is out of sync with the submodule references in the repo, in which case the above command will treat the git repo references as the governing reference, while the current RULE is that the reference in DEPS governs. That can only be resolved by ensuring that DEPS and the references in the repo on main cannot differ.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/bf34d83c-4efa-4b76-94a2-a596d81ad49cn%40chromium.org.

[Josip Sokcevic]

On Thu, Aug 24, 2023 at 11:26 AM Haiyang Pan <hy...@google.com> wrote:

+1 for the dirty submodule issue. To add more details, here is what "git status" shows for chromium/src when I have a branch in the submodule "third_party/catapult":

$ git status
HEAD detached at origin/main
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   third_party/catapult (new commits)

This is expected behavior - you checked out to a different commit in third_party/catapult. With git submodules, git is now aware of that state and reports it. You can ignore the state, but it won't work well with `git commit -a` / `git add .` since it will add them to the commit.

If you'd like to instruct git to simply ignore all changes in that submodule, you can run:

git update-index --assume-unchanged third_party/catapult

be aware, if you do any manual rolls, you will need to undo the change using git update-index --no-assume-unchanged third_party/catapult so its gitlink entry can be committed. 

Given this is the second or third time I see a similar question / comment so I filed the following feature request: https://crbug.com/1475769. Please star it  if you'd like to have this functionality; and if there's enough interest, I'll prioritize that work.

Best,

--

Josip Sokcevic

[Olivier Robin]

I think it would be great, notably because instructions (https://chromium.googlesource.com/chromium/src/+/main/docs/contributing.md#creating-a-change) recommend to use git commit -a, which will cause some issues with submodules.

[Chris Harrelson]

@Mason Freed 

Data point: multiple people from my team have had their checkouts hosed/broken by this update.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAYr5oMRmjUxM1pESckOdxEXm%2BKH320bp5LH_CZJ6yMENGy_Vg%40mail.gmail.com.

[Daniel Cheng]

It feels like it should be easier to revert accidental changes in submodules, whether staged or already committed. I think this is a common mistake; I'm not sure if this is true of other people, but I previously had the habit of using `git commit -a` a lot, because it was simple and easy.

For testing purposes, I intentionally committed a change including a submodule diff:

```
$ git diff HEAD^..HEAD
diff --git a/third_party/swiftshader b/third_party/swiftshader
index 97bdc453fbede..87018302eca37 160000
--- a/third_party/swiftshader
+++ b/third_party/swiftshader
@@ -1 +1 @@
-Subproject commit 97bdc453fbedee92ed0fc7687544098efc821622
+Subproject commit 87018302eca37f1e4775af2f2c248359db6622b7
```

If I want to revert this later, per "What if I accidentally committed a git submodule?", it seems like this ought to work:

```
gclient setdep -r third_party/swiftshader@97bdc453fbedee92ed0fc7687544098efc821622
gclient gitmodules
```

But this doesn't work: `gclient setup` gives the error message "KeyError: 'Could not find any dependency called third_party/swiftshader.'".

What *does* seem to work is:
```
cd third_party/swiftshader
git reset --hard 97bdc453fbedee92ed0fc7687544098efc821622
cd ../..
git add third_party/swiftshader
git commit --amend
```

But this doesn't seem to be mentioned anywhere.

One other surprising thing. I currently have 30d5b1ee39d0143e20ca71ac9d0cf77f9816e149 checked out. This is a random commit from yesterday, with no local changes. However, if I run `gclient gitmodules`, it is *not* a no-op. After running it, `git diff` shows this:

```

diff --git a/components/test/data/autofill/heuristics-json/internal b/components/test/data/autofill/heuristics-json/internal
deleted file mode 160000
index fc9323141d970..0000000000000
--- a/components/test/data/autofill/heuristics-json/internal
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit fc9323141d970c474cb466c32c9be6cfcf20e549
```

Which makes no sense to me either. Maybe I don't understand what `gclient gitmodules` is supposed to do, but I would have expected it to be a no-op if I'm running it against a commit from chromium/src/main.

Daniel

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAOMQ%2Bw9ONKT-47tss9s4iA0BVHFBpQS556fVNHtqBGUM6pXJYA%40mail.gmail.com.

[Josip Sokcevic]

On Fri, Aug 25, 2023 at 2:07 PM Daniel Cheng <dch...@chromium.org> wrote:

It feels like it should be easier to revert accidental changes in submodules, whether staged or already committed. I think this is a common mistake; I'm not sure if this is true of other people, but I previously had the habit of using `git commit -a` a lot, because it was simple and easy.

This is indeed a fairly common operation. From analysis we did prior to the submodule rollout, about 40% of commits are created either using git commit -a or git add *. That's why we suggested running gclient sync after each git pull / checkout, and included instructions on how to ignore submodule changes via --assume-unchanged.

I think we'll need to do more, and one possible solution https://crbug.com/1475769.

 

For testing purposes, I intentionally committed a change including a submodule diff:

```
$ git diff HEAD^..HEAD
diff --git a/third_party/swiftshader b/third_party/swiftshader
index 97bdc453fbede..87018302eca37 160000
--- a/third_party/swiftshader
+++ b/third_party/swiftshader
@@ -1 +1 @@
-Subproject commit 97bdc453fbedee92ed0fc7687544098efc821622
+Subproject commit 87018302eca37f1e4775af2f2c248359db6622b7
```

If I want to revert this later, per "What if I accidentally committed a git submodule?", it seems like this ought to work:

```
gclient setdep -r third_party/swiftshader@97bdc453fbedee92ed0fc7687544098efc821622
gclient gitmodules
```

But this doesn't work: `gclient setup` gives the error message "KeyError: 'Could not find any dependency called third_party/swiftshader.'".

Since the chromium/src DEPS file doesn't use relative_paths, all dependencies are prefixed with src. If you run:

gclient setdep -r third_party/swiftshader@97bdc453fbedee92ed0fc7687544098efc821622

then it works okay.

I'll update the documentation to make that clear. This only affects chromium/src since all other projects use relative paths.

 

What *does* seem to work is:
```
cd third_party/swiftshader
git reset --hard 97bdc453fbedee92ed0fc7687544098efc821622
cd ../..
git add third_party/swiftshader
git commit --amend
```

But this doesn't seem to be mentioned anywhere.

That works too, but do we really want to expose devs to those workflows? I'm happy to add more git, step-by-step, specific instructions, but I hope we can avoid those.

One other surprising thing. I currently have 30d5b1ee39d0143e20ca71ac9d0cf77f9816e149 checked out. This is a random commit from yesterday, with no local changes. However, if I run `gclient gitmodules`, it is *not* a no-op. After running it, `git diff` shows this:

```

diff --git a/components/test/data/autofill/heuristics-json/internal b/components/test/data/autofill/heuristics-json/internal
deleted file mode 160000
index fc9323141d970..0000000000000
--- a/components/test/data/autofill/heuristics-json/internal
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit fc9323141d970c474cb466c32c9be6cfcf20e549
```

Which makes no sense to me either. Maybe I don't understand what `gclient gitmodules` is supposed to do, but I would have expected it to be a no-op if I'm running it against a commit from chromium/src/main.

For about 5h, we had a git dependency in DEPS file without gitlink. The commit you linked was created in that window, resulting in such behavior. I filed a bug to fix that in https://crbug.com/1476115.

Site note: gclient gitmodules stages changes, so using git diff --cached would be more accurate to see the staged diff. git diff shows changes that are not staged.

 

Daniel

On Thu, 24 Aug 2023 at 16:25, Chris Harrelson <chri...@chromium.org> wrote:

@Mason Freed 

Data point: multiple people from my team have had their checkouts hosed/broken by this update.

Chris, thanks for the data point. I would appreciate it if they could provide more details by filing bugs if they haven't already. I suspect it's related to fsmonitor being broken, and we just added a warning in gclient sync and git cl upload that fsmonitor should be disabled until we get it fixed.

--

Josip Sokcevic

[Aaron Leventhal]

What's the tl;dr at this point?

I've been holding off on doing a git pull && gclient sync until I hear everything works as intended.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOij18qEvYjE9YqZpQ%2BTqnKSQ7vhQX6eiBy3w8qAWgEP3_g%40mail.gmail.com.

[Stefan Zager]

What's the status of support for depot_tools/git-cache with submodules? I've been using git-cache on my developer workstation since approximately forever and I'm very fond of it.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOii%3DK12mgodxO9%2BkqbDuOCxEa%2BeX%2Bd5pSC0A4%2BzwuKT%2B0w%40mail.gmail.com.

[Josip Sokcevic]

Hi all,

I'd like to hear from others if they disagree, but from my perspective, it's fine to pull from the latest. To summarize changes since the launch:

* we fixed gclient gitmodules on Win machines

* we updated https://chromium.googlesource.com/chromium/src/+/main/docs/git_submodules.md to cover more cases. We discovered that gclient could've lost track of some submodules over time so a manual cleanup may be necessary (see Git status shows modified dependencies). We are working on fixing that bug, but the fix won't clean up old workspaces that were already affected.

*  we recommend disabling fsmonitor for MacOS due to a bug in fsmonitor. Instructions are printed each time gclient sync or git cl upload is executed.

Known issues:

* fsmonitor, still being investigated (git mailing list thread: https://lore.kernel.org/git/20230829005606.136...@google.com/). Tracking in: https://crbug.com/1475405 

* a git dependency can be added to DEPS, but with no git submodule entry. While DEPS and git submodules are out of sync, gclient gitmodules will generate that entry and likely cause confusion. https://crbug.com/1476115

* gclient losing track of dependencies https://crbug.com/1475448 

Feature requests:

* unmanaged dependencies will be shown in "Changes not staged for commit" with "(new commit)".  We are looking into ways to simplify the workflow (https://crbug.com/1475769,  https://crbug.com/1475324). I created the following internal doc: http://doc/1sSTscELkYIg-pKy3nnp-k0X2i5Kr0RCGxi-RYo-DTyA - comments welcome.

As for git_cache, it continues to work just fine (from technical perspective, gclient is still managing dependencies, but uses git submodules information, ie .gitmodules + gitlinks in addition to DEPS).

Best,

--

Josip Sokcevic

[Bruce Dawson]

Can we confirm/consistentify where fsmonitor is broken? Previous notices just said fsmonitor was broken, and crbug.com/1475405 isn't tagged as being Mac specific, but this email says that the fsmonitor bug is MacOS specific. I disabled fsmonitor on my gWindows machines and now I'm wondering if I should re-enable it.

You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAJiyOij0Sx9_2v%2BVyi3fkYakrD-8o-%2BJaK3r6AdhEvJL4k7HsQ%40mail.gmail.com.

--

Bruce Dawson, he/him

[Josip Sokcevic]

It should have clarified we only received reports for MacOS so far, but we recommend to disable fsmonitor on all OS as it may block CL uploads. At this point, I don't know if Windows is affected. Git folks are leading the investigation, and we should know what OS are affected as they make progress.

--

Josip Sokcevic

[Dirk Pranke]

As I just replied to the other thread, I was not able to reproduce any issues on Windows, and I don't think fsmonitor does anything at all on Linux. Take that for what it's worth, of course; I'm not actually on the Git team :).

In response to Aaron's question, my personal $0.02 is that developing at HEAD is fine; I haven't seen any other serious issues that would make me reluctant to work on this side of the transition.

In response to Stefan's question, I have no idea :).

-- Dirk

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAJiyOijhvLWjyzxx477y641f_mD4ntfqzwZB7k_rKdF1gA3CaA%40mail.gmail.com.

[Torne (Richard Coles)]

Yeah fsmonitor is not supported on linux yet, enabling the option does nothing, and trying to launch the daemon manually just exits saying that it's not supported on this platform.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEoffTA-Nqq1ku-FB_9R0dEbWp1gy%2BO8CAYMD1EwdnTy57SfcQ%40mail.gmail.com.

[Andrew Grieve]

Sorry if this has been answered already.

I keep //clank at tip-of-tree rather than what's in DEPS, and I have this in my .gclient custom_deps:

'src/clank': 'https://chrome-internal.googlesource.com/clank/internal/apps.git@unmanaged',

I've tried both:

git config submodule.clank.ignore all

git config submodule.clank.active false

The "ignore" line removes it from "git status", but it still is added when I do "git commit -a".

Is there a way to have git not add submodules when using "git commit -a"? I think we'd never want this with our setup, right?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEV-rjcnSG4hFPN8-0oA9maejhM2aOxfGkoe6VrD5akZTh9nTQ%40mail.gmail.com.

[Josip Sokcevic]

The following will do the trick for now:

git update-index --assume-unchanged clank

If you need to roll clank ever, you will need to undo it (git update-index --no-assume-unchanged clank)

I'm exploring ways we can automate this in gclient. I think two competing ideas are a global switch via .gclient property or gclient command, and another one is to ignore all custom_deps. Given we don't have any telemetry, I think I'll share an anonymous survey so we can decide by a popular vote.

--

Josip Sokcevic

[Ian Kilpatrick]

The fsmonitor bug is a pretty severe performance hit for those impacted, and a fix doesn't appear imminent. Is there a possibility that we can rollback submodules support until that issue is fixed? 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOigGJ8Uj%2BUt4JgHQrCEefE-EzHvNEgEFmF11QYgkDTVSkw%40mail.gmail.com.

[Mason Freed]

On Wednesday, August 30, 2023 at 11:09:27 AM UTC-7 Ian Kilpatrick wrote:

The fsmonitor bug is a pretty severe performance hit for those impacted, and a fix doesn't appear imminent. Is there a possibility that we can rollback submodules support until that issue is fixed? 

+1. Or at least is it possible to try to commit to a timetable to fix it? The estimate I heard was "next year" which isn't great.

 

* fsmonitor, still being investigated (git mailing list thread: https://lore.kernel.org/git/20230829005606.136615-1-jonat...@google.com/). Tracking in: https://crbug.com/1475405 

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOii%3DK12mgodxO9%2BkqbDuOCxEa%2BeX%2Bd5pSC0A4%2BzwuKT%2B0w%40mail.gmail.com.

--

Josip Sokcevic

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAJiyOij0Sx9_2v%2BVyi3fkYakrD-8o-%2BJaK3r6AdhEvJL4k7HsQ%40mail.gmail.com.

--

Bruce Dawson, he/him

--

Josip Sokcevic

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAJiyOijhvLWjyzxx477y641f_mD4ntfqzwZB7k_rKdF1gA3CaA%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEoffTA-Nqq1ku-FB_9R0dEbWp1gy%2BO8CAYMD1EwdnTy57SfcQ%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEV-rjcnSG4hFPN8-0oA9maejhM2aOxfGkoe6VrD5akZTh9nTQ%40mail.gmail.com.

--

Josip Sokcevic

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

[Sunny Sachanandani]

Re: skipping submodules in "git commit -a", "git status", etc.

> I'm exploring ways we can automate this in gclient. I think two competing ideas are a global switch via .gclient property or gclient command, and another one is to ignore all custom_deps. Given we don't have any telemetry, I think I'll share an anonymous survey so we can decide by a popular vote.

Should we consider doing this for *all* submodules? That would match the legacy gclient DEPS behavior of not showing submodules/DEPS in various git operations, right?

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/0a38d673-2742-4307-9f45-51e69433e374n%40chromium.org.

[Josip Sokcevic]

We chatted about rollback and we determined it would come with its own set of challenges. Moreover, we don't know if anyone started to rely on submodules, and that would be extremely disruptive for them. With that, we agreed to keep the submodules. As for the fsmonitor bug, we will prioritize the root-causing and bug fixing. I should have more information by the end of next week once I'm able to get my hands on a mac machine.

On Fri, Sep 1, 2023 at 10:01 AM Sunny Sachanandani <sun...@chromium.org> wrote:

Re: skipping submodules in "git commit -a", "git status", etc.

> I'm exploring ways we can automate this in gclient. I think two competing ideas are a global switch via .gclient property or gclient command, and another one is to ignore all custom_deps. Given we don't have any telemetry, I think I'll share an anonymous survey so we can decide by a popular vote.

Should we consider doing this for *all* submodules? That would match the legacy gclient DEPS behavior of not showing submodules/DEPS in various git operations, right?

Yes, we are considering that - having a way to hide all submodules, like gclient used to do. There are, of course, trade-offs, so we are considering alternatives - pre-commit hooks, using custom_deps var. I hope to have a decision and implementation in place by the end of next week too.

--

Josip Sokcevic

[Fergal Daly]

The doc here suggests adding a hook to run `gclient sync` on every checkout. This means switching branches switched from instant to ~60 seconds. Anything that makes me to run `gcient sync` more often than I currently have to is going to be a bad time. Currently I run it once, after I pull from origin/main and that's all,

F

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOihEKkuy0gdR7X8MZcb%3D-HAe5%3DZr3ev5AC6tT62nVxAwhg%40mail.gmail.com.

[Colin Blundell]

(from correct address) 

On Mon, Sep 4, 2023 at 8:18 AM 'Fergal Daly' via Chromium-dev <chromi...@chromium.org> wrote:

The doc here suggests adding a hook to run `gclient sync` on every checkout. This means switching branches switched from instant to ~60 seconds. Anything that makes me to run `gcient sync` more often than I currently have to is going to be a bad time. Currently I run it once, after I pull from origin/main and that's all,

F

+1. In my naive opinion it should be almost non-negotiable to keep the following invariants from the flows that Chromium developers are used to (let me know if I'm misunderstanding some of the nuances here):

• Running "git commit -a" does not impact submodules
• Running "gclient sync" is required only after either (1) pulling from origin/main or (2) switching to a branch with DEPS at a different version

Changing the first is going to result in continual developer pain over a protracted period of time, and changing the second is going to result not only in continual developer pain over a protracted period of time but also a slowdown in engineering velocity forever.

Best,

Colin

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAozHLmMpsNKDgoW_HB-oaAMTV558LKn3goQZzcK6oaRJn5pow%40mail.gmail.com.

[Daniel Cheng]

If you change branches but DEPS is the same version, you still don't need to gclient sync; nothing has changed from before the submodules switch.

However, in the past, you might have been able to get away with skipping `gclient sync` more, since `git commit -a` now picks up changes in submodules.

Daniel

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAMGE5NHupPp-2Cr%2BUiq5DrGZGmjQMkVC%2B7CKv-q7kcQJB%2BzOoA%40mail.gmail.com.

[Colin Blundell]

On Tue, Sep 5, 2023 at 7:27 PM Daniel Cheng <dch...@chromium.org> wrote:

If you change branches but DEPS is the same version, you still don't need to gclient sync; nothing has changed from before the submodules switch.

However, in the past, you might have been able to get away with skipping `gclient sync` more, since `git commit -a` now picks up changes in submodules.

Gotcha, thanks. To make sure that I'm understanding correctly, a change in a submodule corresponds to what would have been a change in a third-party repo pulled in via DEPS pre-submodules?

[Daniel Cheng]

That's correct. I think the long-term plan is to get rid of DEPS, but that's something the infra team would be better positioned to comment on than me.

Daniel

[Fergal Daly]

On Wed, 6 Sept 2023 at 02:27, Daniel Cheng <dch...@chromium.org> wrote:

If you change branches but DEPS is the same version, you still don't need to gclient sync; nothing has changed from before the submodules switch.

However, in the past, you might have been able to get away with skipping `gclient sync` more, since `git commit -a` now picks up changes in submodules.

The presence of "diffs" breaks some tools. E.g. `git rebase-update` which does a pull and tries to rebase all your branches gets upset by these diffs. So in terms of what files are checked out, nothing has changed. In terms of workflows involving git, things are broken.

Yesterday I got into a situation where even running `g4 client sync --force` was not fixing things. I ended up deleting `.git/index` and doing a reset. I don't recall doing anything unusual or interesting to get into that situation.

Is there a git command to quickly set all the submodles to their expected state (without the rest of g4 client sync?). I've seen

  git submodule update -f

but when I run that I get a flood of errors (about missing credentials for internal repos),

F

[Hayato Ito]

On Fri, Sep 8, 2023 at 8:10 AM 'Fergal Daly' via Chromium-dev <chromi...@chromium.org> wrote:

On Wed, 6 Sept 2023 at 02:27, Daniel Cheng <dch...@chromium.org> wrote:

If you change branches but DEPS is the same version, you still don't need to gclient sync; nothing has changed from before the submodules switch.

However, in the past, you might have been able to get away with skipping `gclient sync` more, since `git commit -a` now picks up changes in submodules.

The presence of "diffs" breaks some tools. E.g. `git rebase-update` which does a pull and tries to rebase all your branches gets upset by these diffs. So in terms of what files are checked out, nothing has changed. In terms of workflows involving git, things are broken.

Regarding 'git rebase-update', I got into the same situation a few days ago.

Currently, I use the following workflow, instead of 'git rebase-update':

% git fetch
% git switch --detach origin/main
% gclient sync
% git rebase-update --no_fetch

However, this probably doesn't work if there is any local branch which doesn't track origin/main.
Updating submodules would be required for each local branch before a rebase, I guess.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAozHLm%3DngkF-JAasafrPWC2rtLFa3xGEF%2BbCN1_qRqaMDqMPA%40mail.gmail.com.

--

Hayato

[Dirk Pranke]

I assume there are some typos below? `gclient` instead of `g4 client` and such? :)

Josip can say whether he'd expect rebase-update to be broken or not, but note that rebase-update is a script in depot_tools, not a built-in git command. So it may need to be updated for submodules.

-- Dirk

[Dirk Pranke]

Oh, I missed a question, answer below ...

On Thu, Sep 7, 2023 at 6:28 PM Dirk Pranke <dpr...@google.com> wrote:

I assume there are some typos below? `gclient` instead of `g4 client` and such? :)

Josip can say whether he'd expect rebase-update to be broken or not, but note that rebase-update is a script in depot_tools, not a built-in git command. So it may need to be updated for submodules.

-- Dirk

On Thu, Sep 7, 2023 at 4:10 PM Fergal Daly <fer...@google.com> wrote:

On Wed, 6 Sept 2023 at 02:27, Daniel Cheng <dch...@chromium.org> wrote:

If you change branches but DEPS is the same version, you still don't need to gclient sync; nothing has changed from before the submodules switch.

However, in the past, you might have been able to get away with skipping `gclient sync` more, since `git commit -a` now picks up changes in submodules.

The presence of "diffs" breaks some tools. E.g. `git rebase-update` which does a pull and tries to rebase all your branches gets upset by these diffs. So in terms of what files are checked out, nothing has changed. In terms of workflows involving git, things are broken.

Yesterday I got into a situation where even running `g4 client sync --force` was not fixing things. I ended up deleting `.git/index` and doing a reset. I don't recall doing anything unusual or interesting to get into that situation.

Is there a git command to quickly set all the submodles to their expected state (without the rest of g4 client sync?). I've seen

  git submodule update -f

but when I run that I get a flood of errors (about missing credentials for internal repos),

I have no idea what `git submodule update -f` does :). Running submodule commands directly is black magic to me :). 

Did you try the things suggested in the page Josip linked to? Just to be clear, in general, `gclient sync` is the normal way to make sure your submodules are in the right state, and you shouldn't need to be running `submodule` commands directly.

 If you are sure you know what you're doing, you can usually skip running the hooks, so, e.g., run `gclient sync --nohooks`, but if you do that, you should then run `gclient runhooks` to ensure that they've run and your checkout is in sync.

I'll also mention that from a correctness standpoint, every time you need to run `gclient sync` now is a time you needed to run `gclient sync` before, and vice versa. Nothing has changed in that regard. It may just be that you were unaware of your checkouts being out of sync before because we were ignoring the submodules. And often you can get away with things being out of date, but it's not something I would recommend you do as part of normal practice.

I would guess that it's possible we can add some flag to something to turn ignoring submodules back on, but I don't know what the effect of that would be now that we're trying to use submodules in some ways when we weren't before.

-- Dirk

[Bruce Dawson]

I use "git rebase-update" frequently. I can imagine a scenario where it would be broken because it tries to resolve branches before I run "gclient sync", which might be invalid in the context of git submodules. I haven't seen any problems yet but I might have just been lucky.

The command I run is:

git rebase-update && gclient sync -D

This needs to work. So far it has.

--

Bruce Dawson, he/him

[Fergal Daly]

On Fri, 8 Sept 2023 at 10:41, Dirk Pranke <dpr...@google.com> wrote:

Oh, I missed a question, answer below ...

On Thu, Sep 7, 2023 at 6:28 PM Dirk Pranke <dpr...@google.com> wrote:

I assume there are some typos below? `gclient` instead of `g4 client` and such? :)

Josip can say whether he'd expect rebase-update to be broken or not, but note that rebase-update is a script in depot_tools, not a built-in git command. So it may need to be updated for submodules.

-- Dirk

On Thu, Sep 7, 2023 at 4:10 PM Fergal Daly <fer...@google.com> wrote:

On Wed, 6 Sept 2023 at 02:27, Daniel Cheng <dch...@chromium.org> wrote:

If you change branches but DEPS is the same version, you still don't need to gclient sync; nothing has changed from before the submodules switch.

However, in the past, you might have been able to get away with skipping `gclient sync` more, since `git commit -a` now picks up changes in submodules.

The presence of "diffs" breaks some tools. E.g. `git rebase-update` which does a pull and tries to rebase all your branches gets upset by these diffs. So in terms of what files are checked out, nothing has changed. In terms of workflows involving git, things are broken.

Yesterday I got into a situation where even running `g4 client sync --force` was not fixing things. I ended up deleting `.git/index` and doing a reset. I don't recall doing anything unusual or interesting to get into that situation.

Is there a git command to quickly set all the submodles to their expected state (without the rest of g4 client sync?). I've seen

  git submodule update -f

but when I run that I get a flood of errors (about missing credentials for internal repos),

I have no idea what `git submodule update -f` does :). Running submodule commands directly is black magic to me :). 

It was suggested earlier in the thread. I also don't know.

 

Did you try the things suggested in the page Josip linked to? Just to be clear, in general, `gclient sync` is the normal way to make sure your submodules are in the right state, and you shouldn't need to be running `submodule` commands directly.

I'm not sure what on that page I should have done. I don't do any work in submodules and I was just trying to get my client to have no diffs vs origin/main. I expected `gclient sync --force` to get me into a good state but it didn't. If I get into this state again, I will try to preserve things,

F

[Josip Sokcevic]

On Thu, Sep 7, 2023 at 6:57 PM Fergal Daly <fer...@google.com> wrote:

On Fri, 8 Sept 2023 at 10:41, Dirk Pranke <dpr...@google.com> wrote:

Oh, I missed a question, answer below ...

On Thu, Sep 7, 2023 at 6:28 PM Dirk Pranke <dpr...@google.com> wrote:

I assume there are some typos below? `gclient` instead of `g4 client` and such? :)

Josip can say whether he'd expect rebase-update to be broken or not, but note that rebase-update is a script in depot_tools, not a built-in git command. So it may need to be updated for submodules.

-- Dirk

On Thu, Sep 7, 2023 at 4:10 PM Fergal Daly <fer...@google.com> wrote:

On Wed, 6 Sept 2023 at 02:27, Daniel Cheng <dch...@chromium.org> wrote:

If you change branches but DEPS is the same version, you still don't need to gclient sync; nothing has changed from before the submodules switch.

However, in the past, you might have been able to get away with skipping `gclient sync` more, since `git commit -a` now picks up changes in submodules.

The presence of "diffs" breaks some tools. E.g. `git rebase-update` which does a pull and tries to rebase all your branches gets upset by these diffs. So in terms of what files are checked out, nothing has changed. In terms of workflows involving git, things are broken.

Yesterday I got into a situation where even running `g4 client sync --force` was not fixing things. I ended up deleting `.git/index` and doing a reset. I don't recall doing anything unusual or interesting to get into that situation.

Is there a git command to quickly set all the submodles to their expected state (without the rest of g4 client sync?). I've seen

  git submodule update -f

but when I run that I get a flood of errors (about missing credentials for internal repos),

I have no idea what `git submodule update -f` does :). Running submodule commands directly is black magic to me :). 

It was suggested earlier in the thread. I also don't know.

In a nutshell, `gclient sync` is the only command you should need to work with git dependencies and git submodules. If there's an accidental git submodule command, we provide `gclient gitmodules` which will reset submodules to the appropriate state.

git submodule commands should not be used. It will create a lot more problems, so we all should continue to use gclient to manage git dependencies exclusively.

To provide a bit more technical context: git has multiple forms for submodules (docs). We are using the old form, which is exactly the same as how gclient manages git dependencies. If a `git submodule <subcmd>` command pulls a new repository, it will use the new form which is not supported with gclient (or, maybe I should say tested).  Moreover, git doesn't understand gclient-condition, so it will operate on all git dependencies.

I intentionally didn't want to include any information about git submodules as it will be noise for most folks (since we expect gclient to work), and it could create additional confusion. I'm happy to document it.

To address your particular problem, I see you filed a bug under https://crbug.com/1478668. I'll assign it to me and will post a comment shortly.

 

Did you try the things suggested in the page Josip linked to? Just to be clear, in general, `gclient sync` is the normal way to make sure your submodules are in the right state, and you shouldn't need to be running `submodule` commands directly.

I'm not sure what on that page I should have done. I don't do any work in submodules and I was just trying to get my client to have no diffs vs origin/main. I expected `gclient sync --force` to get me into a good state but it didn't. If I get into this state again, I will try to preserve things,

F

--

Josip Sokcevic

[Hayato Ito]

FWIW, https://crbug.com/1478668 is an issue for 'git rebase-update'.
I am about to file an issue, but there was.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAozHLkW8MevAH%3DMnEzgMNeXUack3KYPREr9b6_U2mH4RGLoxA%40mail.gmail.com.

--

Hayato

[Shezan Baig]

Hi Josip,

Is this change in the m118 branch?  I just fetched 118.0.5993.0 and did "gclient sync -D --reset --with_branch_heads --nohooks --deps=win --jobs 16", but I don't see any submodules being initialized.

To be clear, I do see the entries in `src/.gitmodules`, but I don't see anything in `src/.git/modules` (which is typically where git puts the .git directories for initialized submodules).

Thanks,

-shez-

On Wed, Aug 23, 2023 at 1:26 PM 'Josip Sokcevic' via Chromium-dev <chromi...@chromium.org> wrote:

Hi everyone,

As of 15 minute ago, chromium/src contains git submodules. The change is available on the main branch: https://crrev.com/ef72622958e9358800d88739155030deaab314f3. The first release branch that will contain git submodules will be M118.

As requested in the original email, please do provide feedback by filing bugs under the Infra>SDK component. 

Josip Sokcevic

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOijJGEjZJYGfaOm3h-%2BvyJVM4e5RxC5DT8mmk-YkCT3mnA%40mail.gmail.com.

[Josip Sokcevic]

Hi Shez,

That change is on m118 branch. gclient should still be used to manage git dependencies, and we are using "old form" submodules (docs). I provided more details earlier in this thread (see https://groups.google.com/a/chromium.org/g/chromium-dev/c/LDFlTtY8k3A/m/CsdtWg8LBAAJ). If you, or anyone else, encounter any problems, please file bugs under Infra>SDK.

I'll also use this time to provide updates on what's in works:

* fsmonitor bug has been identified, and we are iterating on the patch.

* we are working on optional (pre-)commit hooks that will drop accidental git submodule (gitlink) updates. There will be a way to bypass the hook. Tracked in https://crbug.com/1481266 (P1)

* after consulting with git experts, we concluded that git update-index --assume-unchanged is not the best solution for hiding submodule changes. We need an alternative, and work is tracked in https://crbug.com/1481271  (P2 for now, unlikely to get worked on this year unless there's strong demand). Please star or comment on that ticket so we can adjust priority if it is off.

In the meantime, if you are using git commit -a in your commit workflow, and you are annoyed with accidental git submodule updates, you can use:

    git -c diff.ignoreSubmodules=all commit -a

Since it's not a friendly syntax, I recommend creating an alias. We will engage with the git community and see if we can make a flag (or config) that can be used for git commit, and git add.

Best,

--

Josip Sokcevic

[Nico Weber]

How do I modify roll CLs before they land now? As far as I can tell, https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/dependencies.md doesn't answer this.

I want to patch in https://chromium-review.googlesource.com/c/chromium/src/+/4860859/3 , modify it, and then upload the changes. What I did:

1. git cl patch -b mybranch https://chromium-review.googlesource.com/c/chromium/src/+/4860859

2. `gclient sync`

3. Make changes

4. Commit changes

5. `git cl upload`

In 2, gclient updated the local dep based on the change in the DEPS file I patched in. In step 4, I originally didn't add the change to the gitlink, but then 5 gave me

** Presubmit ERRORS: 1 **
DEPS file indicates git submodule migration is in progress,
but the commit objects do not match DEPS entries.

dependencies.md "Manual roll / low level roll" kind of suggests that it's desired to commit the gitlink change. So I added that too and uploaded it (https://chromium-review.googlesource.com/c/chromium/src/+/4860859/5) and things seem to work (from a submodules PoV at least), but now the change has a gitlink change in it, while it didn't have that when I originally patched in the unmodified roll.

Is it expected that there's no change to gitlink in the autorolled CL, but that one appears when I patch in, edit, and re-upload?

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAJiyOii%3DK12mgodxO9%2BkqbDuOCxEa%2BeX%2Bd5pSC0A4%2BzwuKT%2B0w%40mail.gmail.com.

[Josip Sokcevic]

On Tue, Sep 12, 2023 at 2:14 PM Nico Weber <tha...@chromium.org> wrote:

How do I modify roll CLs before they land now? As far as I can tell, https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/dependencies.md doesn't answer this.

I want to patch in https://chromium-review.googlesource.com/c/chromium/src/+/4860859/3 , modify it, and then upload the changes. What I did:

1. git cl patch -b mybranch https://chromium-review.googlesource.com/c/chromium/src/+/4860859

2. `gclient sync`

3. Make changes

4. Commit changes

5. `git cl upload`

In 2, gclient updated the local dep based on the change in the DEPS file I patched in. In step 4, I originally didn't add the change to the gitlink, but then 5 gave me

** Presubmit ERRORS: 1 **
DEPS file indicates git submodule migration is in progress,
but the commit objects do not match DEPS entries.

dependencies.md "Manual roll / low level roll" kind of suggests that it's desired to commit the gitlink change. So I added that too and uploaded it (https://chromium-review.googlesource.com/c/chromium/src/+/4860859/5) and things seem to work (from a submodules PoV at least), but now the change has a gitlink change in it, while it didn't have that when I originally patched in the unmodified roll.

PS5 looks good from a submodule perspective. 

Is it expected that there's no change to gitlink in the autorolled CL, but that one appears when I patch in, edit, and re-upload?

No, this is not expected - when DEPS is updated, there should be a corresponding gitlink entry. libcxx-chromium autoroller needs to be patched, filed a bug in: https://issues.skia.org/issues/300153470 . I also filed https://crbug.com/1481903 to audit all autorollers.

Thanks for catching that, and sorry for the trouble.

--

Josip Sokcevic

[yx ma]

I am using the gclient sync command on a windows system, after the execution is complete I continue to execute the command gclient gitmodules, at this time the .gitmodules file is generated, at this time I execute the git submodule update command, at this time an error occurs, it will clone https:// chrome-internal.googlesource.com/chrome/src-internal.git, I see that the DEPS file contains
[submodule "internal"]
path = internal
url = https://chrome-internal.googlesource.com/chrome/src-internal.git
gclient-condition = checkout_src_internal
But in my DEPS file there is a
'checkout_src_internal': False,
Why does my execution of glient gitmodules include checkout_src_internal?
I don't have https://chrome-internal.googlesource.com/chrome/src-internal.git访问权限 reporting errors:
fatal: unable to access 'https://chrome-internal.googlesource.com/fuchsia/build.git/': The requested URL returned error: 403

Translated with DeepL.com (free version).

thanks