repo 1.12.24, "cannot overwrite a local work tree"

[James Muir]

after upgrading to the 1.12.24 revision of repo, our automated builds all failed when syncing their trees:

$ repo sync -d -c

<snip>

Fetching projects:  98% (517/527)  Fetching project platform/hardware/intel/common/bd_prov

Fetching project platform/frameworks/opt/timezonepicker

Fetching project platform/external/libyuv

Fetching project platform/packages/apps/Tag

Fetching project platform/frameworks/webview

Fetching projects: 100% (527/527), done.  

info: A new version of repo is available

project .repo/repo/

Updating 97836cf..4e16c24

Fast-forward

 project.py | 82 +++++++++++++++++++++++++++++++++++++++++++-------------------

 1 file changed, 57 insertions(+), 25 deletions(-)

info: Restarting repo with latest version

Fetching project platform/hardware/qcom/gps

Fetching project platform/external/blktrace

Fetching project platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.6

Fetching project platform/external/fsck_msdos

Fetching project platform/external/dhcpcd

<snip>

Fetching project platform/external/jhead

Fetching project platform/external/libpcap

Fetching project platform/external/gcc-demangle

Fetching project platform/packages/apps/Gallery

Exception in thread Thread-117:

Traceback (most recent call last):

  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner

    self.run()

  File "/usr/lib/python2.7/threading.py", line 763, in run

    self.__target(*self.__args, **self.__kwargs)

  File "/opt/git/android-5.1-platform/.repo/repo/subcmds/sync.py", line 244, in _FetchProjectList

    success = self._FetchHelper(opt, project, *args, **kwargs)

  File "/opt/git/android-5.1-platform/.repo/repo/subcmds/sync.py", line 286, in _FetchHelper

    optimized_fetch=opt.optimized_fetch)

  File "/opt/git/android-5.1-platform/.repo/repo/project.py", line 1143, in Sync_NetworkHalf

    self._InitGitDir()

  File "/opt/git/android-5.1-platform/.repo/repo/project.py", line 2183, in _InitGitDir

    self._CheckDirReference(self.objdir, self.gitdir, share_refs=False)

  File "/opt/git/android-5.1-platform/.repo/repo/project.py", line 2304, in _CheckDirReference

    raise GitError('cannot overwrite a local work tree')

GitError: cannot overwrite a local work tree

error: Exited sync due to fetch errors

Anyone know what is going?

If I keep running "repo sync -d -c" manually, then after 4 or 5 failures like this, the sync operation eventually succeeds (and the error seems to go away).

Maybe this is related to the object pruning mentioned in the 1.12.23 thread (Conley Owens, 16 July 2015).

-James M

 

[Conley Owens]

We recently introduced this to prevent a sync from unintentionally
discarding data.
384b3c594831e79348a2eb98809ef9f3e34f6aa2

Here's the message:

Fail if gitdir does not point to objdir during sync

There are a set of cases that can cause the git directory in
.repo/projects to point to a directory in .repo/project-objects that
is not the one specified in the manifest. This results in a tree that
is not sane, and so should cause a failure.

In order to reproduce the failure case:
1) Sync to any manifest
2) Change the 'name' of a project to a different repository. Leave the
'path' the same.
3) Resync the modified project. The project-objects directory will not
be created, and the projects directory will remain pointed at the old
project-objects.

Does this sound similar to your automated tools' behavior?

> --
> --
> To unsubscribe, email repo-discuss...@googlegroups.com
> More info at http://groups.google.com/group/repo-discuss?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Repo and Gerrit Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to repo-discuss...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Degi, Kevin]

This is a result of the ‘Fail if gitdir does not point to objdir during sync’ (384b3c594831e79348a2eb98809ef9f3e34f6aa2) commit that exposes an error that results whentrying to do an incremental sync for a manifest that uses a different project in the same path as the existing tree. Does that sound like something that may have happened in your case?

Kevin Degi

Employee of Qualcomm Innovation Center, Inc.

Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

[James Muir]

Thanks for your quick reply :-)

In our trees, we start with an aosp manifest and then adjust it using a local-manifest.  We add new projects (under new paths), and also use "remove-project" to replace an aosp project with our own version (and we put it under the existing path).  Perhaps this is why we are hitting this new error path.

But, it seems strange to me that the error goes away after 4 or 5 times.

-James M

[Conley Owens]

> But, it seems strange to me that the error goes away after 4 or 5 times.

Yes, I'm getting reports of this as well. Running sync on the same
project multiple times is causing the error to go away.

On Mon, Jul 27, 2015 at 10:09 AM, James Muir

[Degi, Kevin]

It looks like the failure goes away because repo no longer considers the project ‘new’ after it fails the reference check once (because both the source and dest directories exist). The project is still in a ‘bad’ state regarding the shared objects, but the check is not re-run.

Maybe _InitGitDir() needs to clean up after itself if it fails if we want the failure behavior to be consistent?

Kevin Degi
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

[Dean Wheatley]

We follow a similar approach to James M and are hitting this issue.

[Ryuinferno XDA]

I can confirm this, the project I was involved in made changes to the "name" attribute of our manifest since we swapped remote, so syncing now spits the "cannot overwrite a local..." error. Just wondering if it is possible to revert that commit? Since changing the "name" atrribute does not neccesarily mean we have local changes.

[pelya]

I've also encountered this - after making some changes to AOSP, I've run 'repo sync' and got 'cannot overwrite a local work tree'. So I did 'repo forall -c git checkout -f' but that did not help.
I ended up working around this bug:

 while ! repo sync ; do echo "Google, please stop breaking things" ; done

[James Muir]

I think you can also work around it by using the flag "--force-broken"; e.g.:

$ repo sync --force-broken --current-branch --detach

Instead of just exiting after the first failure, repo will continue and you will see failures for various projects.  However, repo will give up before it updates your tree.

If you run "repo sync" again, then it should work.

I generally like the idea of better error checking, but 384b3c594831e79348a2eb98809ef9f3e34f6aa2 doesn't really fit the work-flow we use in-house.

Conley:  if the new error checking is important to keep, then perhaps a flag could be added that lets people skip that check.

-James M

[Degi, Kevin]

Ieddda8ad54e264a1eb4a9d54881dd6ebc8a03833 adds the option to force repo to correct the broken data layout, at the risk of losing any local modifications to the project. Is that something that could address your in-house use case James?

Kevin Degi

Employee of Qualcomm Innovation Center, Inc.

Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

[Conley Owens]

Kevin's changed has been pushed at 1.12.25. The error will now point
users to a flag to use on sync to force sync.

[James Muir]

Kevin:  thanks, for the pointer to the patch.  Yes, I think that flag will work for us.  I will add it to our automatic builds.

Conley:  thanks for the 1.12.25 update!

One thing that I found confusing about the "cannot overwrite a local work tree" message is that it wasn't obvious what project triggered the error.  Perhaps the project name/path could be added the error message.

-James M

[Conley Owens]

> Perhaps the project name/path could be added the error message.

That would be useful. I'm also thinking it'd be nice to automatically
"force sync" every time and back up the project, giving the user
instructions at the end to restore in the event that it did lose local
data.

[s...@tycho.nsa.gov]

All of my repo syncs in existing clones have been broken ever since the original change, and I was not doing anything special other than using local manifest files.
Even after deleting the local manifest files, the errors about overwriting a local work tree persist and do not clear up no matter how many times I run repo sync.
I reverted the breaking change locally in .repo/repo to permit working.  When this new change came in, I reset .repo/repo to origin/stable.
With this new --force-sync option, I now get this error instead:
Exception in thread Thread-114:

Traceback (most recent call last):

  File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.7/threading.py", line 764, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/sds/aosp/.repo/repo/subcmds/sync.py", line 254, in _FetchProjectList

    success = self._FetchHelper(opt, project, *args, **kwargs)

  File "/home/sds/aosp/.repo/repo/subcmds/sync.py", line 297, in _FetchHelper
    optimized_fetch=opt.optimized_fetch)
  File "/home/sds/aosp/.repo/repo/project.py", line 1144, in Sync_NetworkHalf
    self._InitGitDir(force_sync=force_sync)
  File "/home/sds/aosp/.repo/repo/project.py", line 2184, in _InitGitDir
    copy_all=True)
  File "/home/sds/aosp/.repo/repo/project.py", line 2371, in _ReferenceGitDir
    os.symlink(os.path.relpath(src, os.path.dirname(dst)), dst)
OSError: [Errno 17] File exists

[Conley Owens]

Thanks for the report. Dan has a change up to fix it:
https://gerrit-review.googlesource.com/#/c/69984/

[Dan Willemsen]

That change has now been pushed as part of v1.12.26 -- a repo selfupdate should solve the OSError problem (and allow you to use --force-sync again)

[Andreas Wolf]

Hi,
  I have the same issues over the last week or so. We are running CI builds using the AOSP base and a 'local manifest' XML that replaces some projects. Those builds were broken many days, now.
  What you seem to say is: After a normal "repo sync" step, if that step fails I should use the "--force-sync" option to "fix" the broken repo area? After it is fixed, a normal "repo sync" should work again?

  Also, is the use of a 'local_manifest' XML part of a "smoke test" before these new Python files are released? If it is not it should be!

[Conley Owens]

> if that step fails I should use the "--force-sync" option to "fix" the broken repo area?

Correct.

> Also, is the use of a 'local_manifest' XML part of a "smoke test" before these new Python files are released?

No, they are just for additional projects you want to sync.

[James Muir]

> Also, is the use of a 'local_manifest' XML part of a "smoke test" before these new Python files are released?

No, they are just for additional projects you want to sync.

I think Andreas was asking if repo updates are tested in trees that use local_manifests before they are released :-)

We use local_manifests in-house, but would move away from it if we knew it wasn't something on the radar of the repo devs.

-James M

[Conley Owens]

We use them in house as well, so they aren't something off our radar.

[James Muir]

Use "repo sync --force-sync".

This will delete the projects that fail the consistency check and then recreate them.

Beware that any local branches or remotes in those directories will be deleted.

-James M

On Aug 8, 2015 4:16 PM, "winnie" <lvwe...@gmail.com> wrote:

Hello, I try to use flag but I still get the error:

Fetching projects:  97% (353/363)  

error: Exited sync due to fetch errors

 Then I try to use : repo sync, I get

GitError: --force-sync not enabled; cannot overwrite a local work tree

Fetching projects:   4% (15/363)  

error: Exited sync due to fetch errors

Can anyone help me as soon as possible?

[winnie]

Hello, I try to use flag but I still get the error:

Fetching projects:  97% (353/363)  

error: Exited sync due to fetch errors

 Then I try to use : repo sync, I get

GitError: --force-sync not enabled; cannot overwrite a local work tree

Fetching projects:   4% (15/363)  

error: Exited sync due to fetch errors

Can anyone help me as soon as possible?

[winnie]

Hello, I try to use flag( repo sync --force-broken --current-branch --detach ) but it done at 97%:

Fetching project platform/external/eigen

Fetching project platform/frameworks/opt/mms

Fetching projects:  97% (353/363)  

error: Exited sync due to fetch errors

Then I try to type : repo sync, it still get the same error:

tdroid-4.3_r1/.repo/repo/project.py", line 2198, in _InitGitDir

    raise e

GitError: --force-sync not enabled; cannot overwrite a local work tree

Fetching project platform/packages/apps/PackageInstaller

Fetching projects:   4% (15/363)  

error: Exited sync due to fetch errors

Can anybody help me fix it soon?

[winnie]

Thanks! It works well.

[z7z8th]

Are you kiding me?
old version of git-repo stores all git object in .repo/projects,
after upgrade to new git-repo, you want to store them in .repo/project-objects, I can't sync any project now!!!
What do you want me to to? delete all projects and sync all of the 400 projects of android again?