kas wont apply patches

[Daniel Wagner]

Hi,

FYI, I am using the patch apply feature of kas. It usually works just
fine. That is, I do 'kas build foo.yml' many times in a row.
But sometimes kas detects that poky is dirty and doesn't checkout out
the base version to apply the patches. I don't touch poky at all,


$ kas build meta-cpu1518mfp/kas-cpu1518mfp-devel.yml
2018-06-26 09:50:45 - INFO - kas 0.18.0 started
2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-cpu1518mfp$ git rev-parse --show-toplevel
2018-06-26 09:50:45 - INFO - Using /home/wagi/work/plc1500/meta-cpu1518mfp as root for repository meta-cpu1518mfp
2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-cpu1518mfp$ git rev-parse --show-toplevel
2018-06-26 09:50:45 - INFO - Using /home/wagi/work/plc1500/meta-cpu1518mfp as root for repository meta-cpu1518mfp
2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git cat-file -t fe5c83312de11e80b85680ef237f8acb04b4b26e
2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/poky$ git cat-file -t 13c465663d0d1835773fb30baba345396cf0906e
2018-06-26 09:50:45 - INFO - Repository meta-openembedded already contains fe5c83312de11e80b85680ef237f8acb04b4b26e as commit
2018-06-26 09:50:45 - INFO - Repository poky already contains 13c465663d0d1835773fb30baba345396cf0906e as commit
2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git diff-index HEAD --shortstat
2018-06-26 09:50:46 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git rev-parse --verify HEAD
2018-06-26 09:50:46 - INFO - fe5c83312de11e80b85680ef237f8acb04b4b26e
2018-06-26 09:50:46 - INFO - Repo meta-openembedded has already checkout out correct refspec. nothing to do
2018-06-26 09:50:46 - INFO - /home/wagi/work/plc1500/poky$ git diff-index HEAD --shortstat
2018-06-26 09:50:46 - INFO - 3 files changed, 0 insertions(+), 0 deletions(-)
2018-06-26 09:50:46 - WARNING - Repo poky is dirty. no checkout
2018-06-26 09:50:46 - INFO - /home/wagi/work/plc1500/poky$ /tmp/tmpzy75p5z9 /home/wagi/work/plc1500/build
2018-06-26 09:50:46 - INFO - /home/wagi/work/plc1500/poky$ git quiltimport --author kas <k...@example.com> --patches /home/wagi/work/plc1500/meta-cpu1518mfp/patches
2018-06-26 09:50:46 - INFO - 0001-bitbake-fetch-support-multiple-mirror-tarball-filena.patch
2018-06-26 09:50:46 - ERROR - error: patch failed: bitbake/lib/bb/fetch2/__init__.py:425
2018-06-26 09:50:46 - ERROR - error: bitbake/lib/bb/fetch2/__init__.py: patch does not apply
2018-06-26 09:50:46 - ERROR - error: patch failed: bitbake/lib/bb/fetch2/git.py:206
2018-06-26 09:50:46 - ERROR - error: bitbake/lib/bb/fetch2/git.py: patch does not apply
2018-06-26 09:50:46 - ERROR - error: patch failed: bitbake/lib/bb/fetch2/hg.py:77
2018-06-26 09:50:46 - ERROR - error: bitbake/lib/bb/fetch2/hg.py: patch does not apply
2018-06-26 09:50:46 - ERROR - error: patch failed: bitbake/lib/bb/fetch2/npm.py:91
2018-06-26 09:50:46 - ERROR - error: bitbake/lib/bb/fetch2/npm.py: patch does not apply
2018-06-26 09:50:46 - ERROR - Could not apply patch. Please fix repos and patches. (patch path: /home/wagi/work/plc1500/meta-cpu1518mfp/patches, repo: poky, patch entry: shallow-tar, vcs output: 0001-bitbake-fetch-support-multiple-mirror-tarball-filena.patch
)


/me keeps debugging

Thanks,
Daniel

[Claudius Heine]

Hi Daniel,

On Tue, 2018-06-26 at 09:58 +0200, Daniel Wagner wrote:
> Hi,
>
> FYI, I am using the patch apply feature of kas. It usually works just
> fine. That is, I do 'kas build foo.yml' many times in a row.
> But sometimes kas detects that poky is dirty and doesn't checkout out
> the base version to apply the patches. I don't touch poky at all,
>
>
> $ kas build meta-cpu1518mfp/kas-cpu1518mfp-devel.yml
> 2018-06-26 09:50:45 - INFO - kas 0.18.0 started
> 2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-
> cpu1518mfp$ git rev-parse --show-toplevel
> 2018-06-26 09:50:45 - INFO - Using /home/wagi/work/plc1500/meta-
> cpu1518mfp as root for repository meta-cpu1518mfp
> 2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-
> cpu1518mfp$ git rev-parse --show-toplevel
> 2018-06-26 09:50:45 - INFO - Using /home/wagi/work/plc1500/meta-
> cpu1518mfp as root for repository meta-cpu1518mfp
> 2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-

> openembedded$ git cat-file -t
> fe5c83312de11e80b85680ef237f8acb04b4b26e
> 2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/poky$ git
> cat-file -t 13c465663d0d1835773fb30baba345396cf0906e
> 2018-06-26 09:50:45 - INFO - Repository meta-openembedded already
> contains fe5c83312de11e80b85680ef237f8acb04b4b26e as commit
> 2018-06-26 09:50:45 - INFO - Repository poky already contains
> 13c465663d0d1835773fb30baba345396cf0906e as commit
> 2018-06-26 09:50:45 - INFO - /home/wagi/work/plc1500/meta-
> openembedded$ git diff-index HEAD --shortstat
> 2018-06-26 09:50:46 - INFO - /home/wagi/work/plc1500/meta-
> openembedded$ git rev-parse --verify HEAD
> 2018-06-26 09:50:46 - INFO -
> fe5c83312de11e80b85680ef237f8acb04b4b26e
> 2018-06-26 09:50:46 - INFO - Repo meta-openembedded has already
> checkout out correct refspec. nothing to do
> 2018-06-26 09:50:46 - INFO - /home/wagi/work/plc1500/poky$ git
> diff-index HEAD --shortstat
> 2018-06-26 09:50:46 - INFO - 3 files changed, 0 insertions(+), 0
> deletions(-)
> 2018-06-26 09:50:46 - WARNING - Repo poky is dirty. no checkout

Maybe add a debug print that gives further information as to why this
repo is dirty and what changes sullied it.

Since all those patches should be commits, that should not happen.

Cheers,
Claudius

--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: c...@denx.de

PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153
Keyserver: hkp://pool.sks-keyservers.net

[Daniel Wagner]

On 06/26/2018 10:13 AM, Claudius Heine wrote:
>> 2018-06-26 09:50:46 - WARNING - Repo poky is dirty. no checkout
>
> Maybe add a debug print that gives further information as to why this
> repo is dirty and what changes sullied it.

I've added the dirty check to the repos_apply_patches() function. It
should not try to apply patches if the repo is dirty. So that is
something we should add anyway.

> Since all those patches should be commits, that should not happen.

Agreed. Let's see what I get after my dirty check hits.

[Daniel Wagner]

On 06/26/2018 10:21 AM, [ext] Daniel Wagner wrote:
> On 06/26/2018 10:13 AM, Claudius Heine wrote:
>>> 2018-06-26 09:50:46 - WARNING - Repo poky is dirty. no checkout
>>
>> Maybe add a debug print that gives further information as to why this
>> repo is dirty and what changes sullied it.

$ ~/work/kas/run-kas build meta-cpu1518mfp/kas-cpu1518mfp-devel.yml

2018-06-26 14:41:09 - INFO - run-kas 0.18.0 started
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/meta-cpu1518mfp$ git rev-parse --show-toplevel
2018-06-26 14:41:09 - INFO - Using /home/wagi/work/plc1500/meta-cpu1518mfp as root for repository meta-cpu1518mfp
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/meta-cpu1518mfp$ git rev-parse --show-toplevel
2018-06-26 14:41:09 - INFO - Using /home/wagi/work/plc1500/meta-cpu1518mfp as root for repository meta-cpu1518mfp
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/poky$ git cat-file -t 13c465663d0d1835773fb30baba345396cf0906e
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git cat-file -t fe5c83312de11e80b85680ef237f8acb04b4b26e
2018-06-26 14:41:09 - INFO - Repository poky already contains 13c465663d0d1835773fb30baba345396cf0906e as commit
2018-06-26 14:41:09 - INFO - Repository meta-openembedded already contains fe5c83312de11e80b85680ef237f8acb04b4b26e as commit
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/poky$ git diff-index HEAD --shortstat
2018-06-26 14:41:09 - INFO - 9 files changed, 0 insertions(+), 0 deletions(-)
2018-06-26 14:41:09 - WARNING - Repo poky is dirty. no checkout
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git diff-index HEAD --shortstat
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git rev-parse --verify HEAD
2018-06-26 14:41:09 - INFO - fe5c83312de11e80b85680ef237f8acb04b4b26e
2018-06-26 14:41:09 - INFO - Repo meta-openembedded has already checkout out correct refspec. nothing to do
2018-06-26 14:41:09 - INFO - /home/wagi/work/plc1500/poky$ /tmp/tmpf5m291ns /home/wagi/work/plc1500/build
2018-06-26 14:41:09 - WARNING - Repo poky is dirty. no patches will be applied

$ git diff-index HEAD --shortstat
$ echo $?
0

$ ~/work/kas/run-kas build meta-cpu1518mfp/kas-cpu1518mfp-devel.yml
2018-06-26 14:44:41 - INFO - run-kas 0.18.0 started
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/meta-cpu1518mfp$ git rev-parse --show-toplevel
2018-06-26 14:44:41 - INFO - Using /home/wagi/work/plc1500/meta-cpu1518mfp as root for repository meta-cpu1518mfp
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/meta-cpu1518mfp$ git rev-parse --show-toplevel
2018-06-26 14:44:41 - INFO - Using /home/wagi/work/plc1500/meta-cpu1518mfp as root for repository meta-cpu1518mfp
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/poky$ git cat-file -t 13c465663d0d1835773fb30baba345396cf0906e
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git cat-file -t fe5c83312de11e80b85680ef237f8acb04b4b26e
2018-06-26 14:44:41 - INFO - Repository poky already contains 13c465663d0d1835773fb30baba345396cf0906e as commit
2018-06-26 14:44:41 - INFO - Repository meta-openembedded already contains fe5c83312de11e80b85680ef237f8acb04b4b26e as commit
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/poky$ git diff-index HEAD --shortstat
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/poky$ git rev-parse --verify HEAD
2018-06-26 14:44:41 - INFO - b6880fa9d85c6157d555c9bef5096d5959870cb3
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/poky$ git checkout -q 13c465663d0d1835773fb30baba345396cf0906e
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git diff-index HEAD --shortstat
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/meta-openembedded$ git rev-parse --verify HEAD
2018-06-26 14:44:41 - INFO - fe5c83312de11e80b85680ef237f8acb04b4b26e
2018-06-26 14:44:41 - INFO - Repo meta-openembedded has already checkout out correct refspec. nothing to do
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/poky$ /tmp/tmp0vcs_00_ /home/wagi/work/plc1500/build
2018-06-26 14:44:41 - INFO - /home/wagi/work/plc1500/poky$ git quiltimport --author kas <k...@example.com> --patches /home/wagi/work/plc1500/meta-cpu1518mfp/patches
2018-06-26 14:44:41 - INFO - 0001-bitbake-fetch-support-multiple-mirror-tarball-filena.patch
2018-06-26 14:44:41 - INFO - 0002-bitbake-git-make-shallow-add-script-to-make-a-git-re.patch
2018-06-26 14:44:41 - INFO - 0003-bitbake-fetch-git-add-support-for-shallow-mirror-tar.patch
2018-06-26 14:44:41 - INFO - 0004-bitbake-fetch-git-support-per-branch-per-url-depths-.patch
2018-06-26 14:44:41 - INFO - 0005-bitbake-fetch-git-add-support-for-keeping-extra-refs.patch
2018-06-26 14:44:41 - INFO - 0006-bitbake-fetch-gitsm-add-support-for-shallow-mirror-t.patch
2018-06-26 14:44:41 - INFO - 0007-bitbake-fetch-gitannex-add-support-for-shallow-mirro.patch
2018-06-26 14:44:41 - INFO - 0008-bitbake-fetch-git-add-support-for-removing-arbitrary.patch
2018-06-26 14:44:41 - INFO - Patch applied. (patch path: /home/wagi/work/plc1500/meta-cpu1518mfp/patches, repo: poky, patch entry:


In other words 'git diff-index HEAD --shortstat' is not giving
a consistent answer. Jan told me offline, that he remembers something
similar about 'git diff-index' giving wrong results.

Maybe we should use a different git command to identify if the
repo is dirty unless someone know how to fix diff-index :)

Thanks,
Daniel