repo init and repo sync silently fail to get any files besides .repo directory

[Thomas Thorne]

I am attempting to use repo to setup a Yocto work area.  The intention is to clone a handful of repositories into a single directory in an automated manner rather than doing so by hand.  My problem is that repo sync seems to silently fail, resulting in a directory that contains only the .repo sub-directory.  

I have authored a manifest XML file based on the documentation and a few examples.  I am sure this must be where the problem originates from.  Please could someone cast an eye over the file and let me know if they spot some problems?  

my-device.xml

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE manifest SYSTEM "manifest.dtd">

<manifest>

  <!-- Define Remotes -->

  <!-- define common Yocto remotes -->

  <remote fetch="git://git.yoctoproject.org/poky" name="poky"/>

  <remote fetch="git://github.com/openembedded" name="oe"/>

  <!-- define other remotes -->

  <remote fetch="git://github.com/Quectel-Community/" name="qc"/>

  <remote fetch="git://github.com/meta-qt5/" name="qt5"/>

  <remote fetch="git://git.yoctoproject.org/meta-raspberrypi" name="raspberrypi"/>

  <remote fetch="git://github.com/jumpnow/" name="jumpnow"/>

  <!-- Define data for this manifest -->

  <default sync-j="4" revision="krogoth"/>

  <!-- Define Projects -->

  <!-- Poky project layer -->

  <project remote="poky" revision="krogoth" name="poky" path="meta"/>

  <project remote="poky" revision="krogoth" name="poky" path="meta-poky"/>

  <project remote="poky" revision="krogoth" name="meta-yocto" path="meta-yocto"/>

  <project remote="poky" revision="krogoth" name="meta-yocto-bsp" path="meta-yocto-bsp"/>

  <!-- Open Embedded layer -->

  <project remote="oe" revision="krogoth" name="meta-openembedded" path="meta-openembedded"/>

  <!-- Raspberry Pi 3 related -->

  <project remote="qt5" revision="krogoth" name="meta-qt5" path="meta-qt5"/>

  <project remote="raspberrypi" revision="krogoth" name="meta-raspberrypi" path="meta-raspberrypi"/>

  <project remote="jumpnow" revision="krogoth" name="meta-rpi" path="meta-rpi"/>

  <!-- Quectel Driver Specific -->

  <project remote="qc" revision="krogoth" name="meta-quectel-community" path="meta-quectel-community"/>

</manifest>

As I was having some issues I setup a manifest.dtd based on the Google documentation.  I used this with xmllint which helped spot a couple of spelling errors.  The Document Type Definition file is nothing remarkable but I include it below for completeness.  

manifest.dtd

<!--

repo manifest Document Type Definition (DTD)

This document describes the scheme for a repo manifest.  It is based on

documentation provided by Google in the repo Manifest Format.  You can read the

full document on the Google Source Gerrir server:

https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.txt

The full document also includes some explanation of the meaning of each element.

The same document is provided within an initialised repo work area under

.repo/repo/docs/manifest-format.txt

With this DTD document you can validate manifest XML files.  xmllint is one

such tool which can validate XML against a DTD.  e.g.

-->

<!ELEMENT manifest (notice?,

                    remote*,

                    default?,

                    manifest-server?,

                    remove-project*,

                    project*,

                    extend-project*,

                    repo-hooks?,

                    include*)>

<!ELEMENT notice (#PCDATA)>

<!ELEMENT remote EMPTY>

    <!ATTLIST remote name         ID    #REQUIRED>

    <!ATTLIST remote alias        CDATA #IMPLIED>

    <!ATTLIST remote fetch        CDATA #REQUIRED>

    <!ATTLIST remote pushurl      CDATA #IMPLIED>

    <!ATTLIST remote review       CDATA #IMPLIED>

    <!ATTLIST remote revision     CDATA #IMPLIED>

<!ELEMENT default EMPTY>

    <!ATTLIST default remote      IDREF #IMPLIED>

    <!ATTLIST default revision    CDATA #IMPLIED>

    <!ATTLIST default dest-branch CDATA #IMPLIED>

    <!ATTLIST default sync-j      CDATA #IMPLIED>

    <!ATTLIST default sync-c      CDATA #IMPLIED>

    <!ATTLIST default sync-s      CDATA #IMPLIED>

<!ELEMENT manifest-server EMPTY>

    <!ATTLIST manifest-server url CDATA #REQUIRED>

<!ELEMENT project (annotation*,

                   project*,

                   copyfile*,

                   linkfile*)>

    <!ATTLIST project name        CDATA #REQUIRED>

    <!ATTLIST project path        CDATA #IMPLIED>

    <!ATTLIST project remote      IDREF #IMPLIED>

    <!ATTLIST project revision    CDATA #IMPLIED>

    <!ATTLIST project dest-branch CDATA #IMPLIED>

    <!ATTLIST project groups      CDATA #IMPLIED>

    <!ATTLIST project sync-c      CDATA #IMPLIED>

    <!ATTLIST project sync-s      CDATA #IMPLIED>

    <!ATTLIST project upstream    CDATA #IMPLIED>

    <!ATTLIST project clone-depth CDATA #IMPLIED>

    <!ATTLIST project force-path  CDATA #IMPLIED>

<!ELEMENT annotation EMPTY>

    <!ATTLIST annotation name  CDATA #REQUIRED>

    <!ATTLIST annotation value CDATA #REQUIRED>

    <!ATTLIST annotation keep  CDATA "true">

<!ELEMENT copyfile EMPTY>

    <!ATTLIST copyfile src  CDATA #REQUIRED>

    <!ATTLIST copyfile dest CDATA #REQUIRED>

<!ELEMENT linkfile EMPTY>

    <!ATTLIST linkfile src  CDATA #REQUIRED>

    <!ATTLIST linkfile dest CDATA #REQUIRED>

<!ELEMENT extend-project EMPTY>

    <!ATTLIST extend-project name   CDATA #REQUIRED>

    <!ATTLIST extend-project path   CDATA #IMPLIED>

    <!ATTLIST extend-project groups CDATA #IMPLIED>

<!ELEMENT remove-project EMPTY>

    <!ATTLIST remove-project name  CDATA #REQUIRED>

<!ELEMENT repo-hooks EMPTY>

    <!ATTLIST repo-hooks in-project   CDATA #REQUIRED>

    <!ATTLIST repo-hooks enabled-list CDATA #REQUIRED>

<!ELEMENT include EMPTY>

    <!ATTLIST include name CDATA #REQUIRED>

When I try to execute the repo init and sync scripts I get the following output:

$ repo init -u "../dev_utils" -m "repo_manifests/my-device.xml"

Get https://gerrit.googlesource.com/git-repo/clone.bundle

Get https://gerrit.googlesource.com/git-repo

remote: Finding sources: 100% (69/69)

remote: Total 69 (delta 22), reused 69 (delta 22)

Unpacking objects: 100% (69/69), done.

From https://gerrit.googlesource.com/git-repo

   ffb4b89..35d2221  master     -> origin/master

Get ../dev_utils

remote: Counting objects: 96, done.        

remote: Compressing objects: 100% (86/86), done.        

remote: Total 96 (delta 30), reused 0 (delta 0)        

From ../dev_utils

 * [new branch]      master     -> origin/master

Your identity is: Thomas Thorne <obscured_by_hand>

If you want to change this, please re-run 'repo init' with --config-name

repo has been initialized in /home/thomasthorne/work/yocto

$ ls -l

total 0

$ ls -la

total 12

drwxrwxr-x  3 thomasthorne thomasthorne 4096 Jun 13 11:38 .

drwxrwxr-x 76 thomasthorne thomasthorne 4096 Jun 13 09:43 ..

drwxrwxr-x  5 thomasthorne thomasthorne 4096 Jun 13 11:38 .repo

$ repo --trace sync -l -f --force-sync

: git config --file /home/thomasthorne/work/yocto/.repo/manifests.git/config --null --list 1>| 2>|

: parsing /home/thomasthorne/.gitconfig

: git config --file /home/thomasthorne/work/yocto/.repo/repo/.git/config --null --list 1>| 2>|

: load refs /home/thomasthorne/work/yocto/.repo/manifests.git

$ ls -la

total 12

drwxrwxr-x  3 thomasthorne thomasthorne 4096 Jun 13 11:38 .

drwxrwxr-x 76 thomasthorne thomasthorne 4096 Jun 13 09:43 ..

drwxrwxr-x  5 thomasthorne thomasthorne 4096 Jun 13 11:38 .repo

I had read a suggestion about using the trace and -f and --force-sync options while looking for a solution to my problem.  It did not appear to change the behaviour.  

I wondered if repo might be unhappy with sourcing the manifest from the local filesystem but I can see a copy of it symlinked inside .repo/ so I do not think that is the issue.  

[David Pursehouse]

I was able to get it working by creating a local git repository for the manifest as follows:

temp $ cd repotest/

repotest $ git init manifest

Initialized empty Git repository in /Users/dpursehouse/temp/repotest/manifest/.git/

repotest $ cd manifest/

manifest $ git commit -m "initial" --allow-empty

[master (root-commit) f8d6d82] initial

manifest $ nano default.xml

[[paste in the content of the manifest, omitting the DOCTYPE line]]

manifest $ git add default.xml

manifest $ git commit -m "Add default.xml"

[master f5d0cce] Add default.xml

 1 file changed, 38 insertions(+)

 create mode 100644 default.xml

manifest $ cd ../

repotest $ mkdir workspace

repotest $ cd workspace/

workspace $ repo init -u ../manifest/

... A new repo command ( 1.24) is available.

... You should upgrade soon:

    cp /Users/dpursehouse/temp/repotest/.repo/repo/repo /Users/dpursehouse/bin/repo

warn: .repo/manifests: Not replacing locally modified commit-msg hook

Your identity is: David Pursehouse <redacted>

If you want to change this, please re-run 'repo init' with --config-name

repo has been initialized in /Users/dpursehouse/temp/repotest

If this is not the directory in which you want to initialize repo, please run:

   rm -r /Users/dpursehouse/temp/repotest/.repo

and try again.

However it then fails to sync due to access denied:

workspace $ repo sync -j1

... A new repo command ( 1.24) is available.

... You should upgrade soon:

    cp /Users/dpursehouse/temp/repotest/.repo/repo/repo /Users/dpursehouse/bin/repo

warn: .repo/manifests: Not replacing locally modified commit-msg hook

Fetching project meta-raspberrypi

fatal: remote error: access denied or repository not exported: /meta-raspberrypi/meta-raspberrypi

fatal: remote error: access denied or repository not exported: /meta-raspberrypi/meta-raspberrypi

error: Cannot fetch meta-raspberrypi from git://git.yoctoproject.org/meta-raspberrypi/meta-raspberrypi

error: Exited sync due to fetch errors

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Thorne]

I think I have spotted the key difference that lets it work or not work (well up to the point of the error in my config).  

On Tuesday, June 13, 2017 at 12:23:44 PM UTC+1, David Pursehouse wrote:

[[paste in the content of the manifest, omitting the DOCTYPE line]]

As soon as I omit the DOCTYPE line I no longer see a problem with repo.  Is it possible that the allowable syntax forbids the inclusion of a  <!DOCTYPE manifest SYSTEM "manifest.dtd"> line?