[PATCH] styhead migration fixes

[Dan Walkes]

Based on the guidance from [1], resolve build errors on styhead.

This resolves 2 issues with styhead as found with the integration at [2]
and testing of branch at [3], as referenced at [4], [5], and [6]:

```
WARNING: swupdate-image-tegra-1.0-r0 do_unpack: swupdate-image-tegra: the directory ${WORKDIR}/${PN} (/home/dan/trellis/oe4t/tegra-demo-distro/build-master/tmp/work/jetson_orin_nano_devkit_nvme-oe4t-linux/swupdate-image-tegra/1.0/swupdate-image-tegra) pointed to by the S variable doesn't exist - please set S within the recipe to point to where the source has been unpacked to
```
This warning is printed due to the setting of S based on WORKDIR
in swupdate.bbclass and the fact that the S directory is no longer
automatically created as discussed in [1]. To resolve, use the guidance
in [1] to set S and UNPACKDIR for recipes which only use file:// sources.

```
ERROR: swupdate-image-tegra-1.0-r0 do_swuimage: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_swuimage(d)
0003:
File: '/home/dan/trellis/oe4t/tegra-demo-distro/build-master/../layers/meta-swupdate/classes-recipe/swupdate-common.bbclass', lineno: 325, function: do_swuimage
0321: list_for_cpio = ["sw-description"]
0322: workdir = d.getVar('WORKDIR')
0323: s = d.getVar('S')
0324: imgdeploydir = d.getVar('SWUDEPLOYDIR')
*** 0325: shutil.copyfile(os.path.join(workdir, "sw-description"), os.path.join(s, "sw-description"))
0326:
...
File: '/usr/lib/python3.10/shutil.py', lineno: 254, function: copyfile
0250:
0253: else:
*** 0254: with open(src, 'rb') as fsrc:
```

This error occurs due to the use of WORKDIR as the location of files instead
of UNPACKDIR. When using the guidance in [1] to set UNPACKDIR there is no
longer a need to copy from WORKDIR to UNPACKDIR since S is UNPACKDIR, so both
the workdir reference and shutil copy can be removed.

1: https://docs.yoctoproject.org/next/migration-guides/migration-5.1.html
2: https://github.com/OE4T/tegra-demo-distro/blob/master/layers/meta-tegrademo/dynamic-layers/meta-swupdate/README.md
3: https://github.com/Trellis-Logic/tegra-demo-distro/tree/master%2Bswupdate-kas
4: https://groups.google.com/g/swupdate/c/E2h_qgHvJuk
5: https://groups.google.com/g/swupdate/c/x4jxRPbtXa0/m/QQkYqgILAQAJ
6: https://mail.google.com/mail/u/0/#search/swupdate+/FMfcgzQVzNxnTXkCCgksdnXCHPqsJLzq

Signed-off-by: Dan Walkes <danw...@trellis-logic.com>
---
classes-recipe/swupdate-common.bbclass | 2 --
classes-recipe/swupdate-image.bbclass | 3 ++-
classes-recipe/swupdate.bbclass | 3 ++-
3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/classes-recipe/swupdate-common.bbclass b/classes-recipe/swupdate-common.bbclass
index 15b45c3..79b79c3 100644
--- a/classes-recipe/swupdate-common.bbclass
+++ b/classes-recipe/swupdate-common.bbclass
@@ -322,10 +322,8 @@ python do_swuimage () {
import shutil

list_for_cpio = ["sw-description"]
- unpackdir = d.getVar('UNPACKDIR')
s = d.getVar('S')
imgdeploydir = d.getVar('SWUDEPLOYDIR')
- shutil.copyfile(os.path.join(unpackdir, "sw-description"), os.path.join(s, "sw-description"))

if d.getVar('SWUPDATE_SIGNING'):
list_for_cpio.append('sw-description.sig')
diff --git a/classes-recipe/swupdate-image.bbclass b/classes-recipe/swupdate-image.bbclass
index 175fc4e..62ef19d 100644
--- a/classes-recipe/swupdate-image.bbclass
+++ b/classes-recipe/swupdate-image.bbclass
@@ -13,7 +13,8 @@
inherit swupdate-common
inherit image-artifact-names

-S = "${WORKDIR}/${PN}"
+S = "${WORKDIR}/sources"
+UNPACKDIR = "${S}"

SRC_URI += "file://sw-description"
SWUPDATE_IMAGES += "${IMAGE_LINK_NAME}"
diff --git a/classes-recipe/swupdate.bbclass b/classes-recipe/swupdate.bbclass
index 3656b1c..ae00085 100644
--- a/classes-recipe/swupdate.bbclass
+++ b/classes-recipe/swupdate.bbclass
@@ -36,7 +36,8 @@
inherit swupdate-common
inherit image-artifact-names

-S = "${WORKDIR}/${PN}"
+S = "${WORKDIR}/sources"
+UNPACKDIR = "${S}"

IMAGE_DEPENDS ?= ""

--
2.34.1

[Khem Raj]

why do we need to delete this ? sw-description is brought in via SRC_URI
so the correct way to refer to it is via UNPACKDIR where it will land
after do_unpack.

>
> if d.getVar('SWUPDATE_SIGNING'):
> list_for_cpio.append('sw-description.sig')
> diff --git a/classes-recipe/swupdate-image.bbclass b/classes-recipe/swupdate-image.bbclass
> index 175fc4e..62ef19d 100644
> --- a/classes-recipe/swupdate-image.bbclass
> +++ b/classes-recipe/swupdate-image.bbclass
> @@ -13,7 +13,8 @@
> inherit swupdate-common
> inherit image-artifact-names
>
> -S = "${WORKDIR}/${PN}"
> +S = "${WORKDIR}/sources"
> +UNPACKDIR = "${S}"

This looks ok however, since both swupdate and swupdate-image classes inherit
swupdate-common class, it might make sense to move it to swupdate-common class
and do it once.

>
> SRC_URI += "file://sw-description"
> SWUPDATE_IMAGES += "${IMAGE_LINK_NAME}"
> diff --git a/classes-recipe/swupdate.bbclass b/classes-recipe/swupdate.bbclass
> index 3656b1c..ae00085 100644
> --- a/classes-recipe/swupdate.bbclass
> +++ b/classes-recipe/swupdate.bbclass
> @@ -36,7 +36,8 @@
> inherit swupdate-common
> inherit image-artifact-names
>
> -S = "${WORKDIR}/${PN}"
> +S = "${WORKDIR}/sources"
> +UNPACKDIR = "${S}"
>

This too.

> IMAGE_DEPENDS ?= ""
>
> --
> 2.34.1
>
> --
> You received this message because you are subscribed to the Google Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to swupdate+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/swupdate/20240923020027.18595-1-danwalkes%40trellis-logic.com.

[Dan Walkes]

I think the problem is in  os.path.join(s, "sw-description"), which doesn't exist unless you add this yourself in the recipe.  See https://groups.google.com/g/swupdate/c/x4jxRPbtXa0/m/2I2EXoBfBAAJ

I need to update the commit with the correct error message after rebase, it's

```

Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/dan/trellis/oe4t/tegra-demo-distro/build-master/tmp/work/jetson_orin_nano_devkit_nvme-oe4t-linux/swupdate-image-tegra/1.0/swupdate-image-tegra/sw-description'

```

>
>      if d.getVar('SWUPDATE_SIGNING'):
>          list_for_cpio.append('sw-description.sig')
> diff --git a/classes-recipe/swupdate-image.bbclass b/classes-recipe/swupdate-image.bbclass
> index 175fc4e..62ef19d 100644
> --- a/classes-recipe/swupdate-image.bbclass
> +++ b/classes-recipe/swupdate-image.bbclass
> @@ -13,7 +13,8 @@
>  inherit swupdate-common
>  inherit image-artifact-names
>
> -S = "${WORKDIR}/${PN}"
> +S = "${WORKDIR}/sources"
> +UNPACKDIR = "${S}"

This looks ok however, since both swupdate and swupdate-image classes inherit
swupdate-common class, it might make sense to move it to swupdate-common class
and do it once.

Sure, will do.

 

>
>  SRC_URI += "file://sw-description"
>  SWUPDATE_IMAGES += "${IMAGE_LINK_NAME}"
> diff --git a/classes-recipe/swupdate.bbclass b/classes-recipe/swupdate.bbclass
> index 3656b1c..ae00085 100644
> --- a/classes-recipe/swupdate.bbclass
> +++ b/classes-recipe/swupdate.bbclass
> @@ -36,7 +36,8 @@
>  inherit swupdate-common
>  inherit image-artifact-names
>
> -S = "${WORKDIR}/${PN}"
> +S = "${WORKDIR}/sources"
> +UNPACKDIR = "${S}"
>

This too.

>  IMAGE_DEPENDS ?= ""
>
> --
> 2.34.1
>
> --
> You received this message because you are subscribed to the Google Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to swupdate+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/swupdate/20240923020027.18595-1-danwalkes%40trellis-logic.com.

--

Dan Walkes

Trellis-Logic

cell: 720-239-2553

[Khem Raj]

UNPACKDIR is what seems to be correct here, since WORKDIR can change files.
so unless these files are edited in subsequent tasks after do_unpack then they
should be brought into WORKDIR and edited. So this copy operation maybe useless
if there is no editing going on. It should just be removed in that case.

[Dan Walkes]

I think if S = UNPACKDIR as suggested by the migration guide and in this patch then sw-description will be brought into ${S} which will be ${WORKDIR}/sources

 

So this copy operation maybe useless
if there is no editing going on. It should just be removed in that case

I think all the editing will be happening in ${S} which will also be ${UNPACKDIR} as suggested by the migration guide, then the rest of the editing logic in swupdate-common.bbclass just automagically work when referencing ${S}, no copy is necessary, and this line can be removed as proposed here, and the do_swuimage[dirs] += "${S}" is no longer required to get past the build error.

In other words I just need to fix my commit message to reference the correct rebased error message and fix the other issue you pointed out with duplicate logic and we should be good to merge.

Agree?

[Khem Raj]

yes that seems plausible.

However, if we expect to edit the file then its best to copy it to
WORKDIR and do the editing. unpackdir was introduced exactly to avoid
situations where workdir was getting inconsistent. When you set
UNPACKADIR
and S to be same, you are telling the system that we do not intend to
change anything in S because UNPACKADIR should stay unmodified.

> In other words I just need to fix my commit message to reference the correct rebased error message and fix the other issue you pointed out with duplicate logic and we should be good to merge.
>
> Agree?

Mostly, yes. I am just worried that we may be violating some
assumptions about unpackdir. If thats cleared I am good with just
editing the commit msg.

[Dan Walkes]

OK I'm not seeing this limitation mentioned in the migration notes at https://docs.yoctoproject.org/next/migration-guides/migration-5.1.html#migration-notes-for-5-1-styhead

 

> In other words I just need to fix my commit message to reference the correct rebased error message and fix the other issue you pointed out with duplicate logic and we should be good to merge.
>
> Agree?

Mostly, yes. I am just worried that we may be violating some
assumptions about unpackdir.

I just tried posting a message to the matrix chat at https://matrix.to/#/!HcmmNcyDJayRxYGYMm:matrix.org/$1LHA-6eYzN2BQ4LMlucrXQDlgoSl5pyR5uXHzYsZ9bY?via=matrix.org&via=gitter.im&via=geanix.com to see if anyone could help clarify.

[Dan Walkes]

Based on the guidance from [1], resolve build errors on styhead.

This resolves 2 issues with styhead as found with the integration at [2]

and testing of branch at [3], as referenced at [4], [5], [6], and [7]:

```

WARNING: swupdate-image-tegra-1.0-r0 do_unpack: swupdate-image-tegra: the directory ${WORKDIR}/${PN} (/home/dan/trellis/oe4t/tegra-demo-distro/build-master/tmp/work/jetson_orin_nano_devkit_nvme-oe4t-linux/swupdate-image-tegra/1.0/swupdate-image-tegra) pointed to by the S variable doesn't exist - please set S within the recipe to point to where the source has been unpacked to
```
This warning is printed due to the setting of S based on WORKDIR
in swupdate.bbclass and the fact that the S directory is no longer
automatically created as discussed in [1]. To resolve, use the guidance
in [1] to set S and UNPACKDIR for recipes which only use file:// sources.

In addition, as suggested in [7], move to swupdate-common to share between
swupdate-image and swupdate.

```

Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/dan/trellis/oe4t/tegra-demo-distro/build-master/tmp/work/jetson_orin_nano_devkit_nvme-oe4t-linux/swupdate-image-tegra/1.0/swupdate-image-tegra/sw-description'
```

This error occurs due to a missing S directory which is no longer created
automatically after styhead

When using the guidance in [1] to set UNPACKDIR there is no
longer a need to copy from WORKDIR to UNPACKDIR since S is UNPACKDIR, so both
the workdir reference and shutil copy can be removed.

1: https://docs.yoctoproject.org/next/migration-guides/migration-5.1.html
2: https://github.com/OE4T/tegra-demo-distro/blob/master/layers/meta-tegrademo/dynamic-layers/meta-swupdate/README.md
3: https://github.com/Trellis-Logic/tegra-demo-distro/tree/master%2Bswupdate-kas
4: https://groups.google.com/g/swupdate/c/E2h_qgHvJuk
5: https://groups.google.com/g/swupdate/c/x4jxRPbtXa0/m/QQkYqgILAQAJ

6: https://groups.google.com/g/swupdate/c/x4jxRPbtXa0
7: https://groups.google.com/g/swupdate/c/8K-9H7C9o5E

Signed-off-by: Dan Walkes <danw...@trellis-logic.com>
---

classes-recipe/swupdate-common.bbclass | 5 +++--
classes-recipe/swupdate-image.bbclass | 2 --
classes-recipe/swupdate.bbclass | 2 --
3 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/classes-recipe/swupdate-common.bbclass b/classes-recipe/swupdate-common.bbclass
index 15b45c3..7b49561 100644
--- a/classes-recipe/swupdate-common.bbclass
+++ b/classes-recipe/swupdate-common.bbclass
@@ -4,6 +4,9 @@

inherit swupdate-lib

+S = "${WORKDIR}/sources"
+UNPACKDIR = "${S}"

+
DEPENDS += "\
cpio-native \
${@ 'openssl-native' if d.getVar('SWUPDATE_SIGNING') or d.getVar('SWUPDATE_ENCRYPT_SWDESC') or d.getVarFlags('SWUPDATE_IMAGES_ENCRYPTED') else ''} \
@@ -322,10 +325,8 @@ python do_swuimage () {

import shutil

list_for_cpio = ["sw-description"]
- unpackdir = d.getVar('UNPACKDIR')
s = d.getVar('S')
imgdeploydir = d.getVar('SWUDEPLOYDIR')
- shutil.copyfile(os.path.join(unpackdir, "sw-description"), os.path.join(s, "sw-description"))

if d.getVar('SWUPDATE_SIGNING'):
list_for_cpio.append('sw-description.sig')
diff --git a/classes-recipe/swupdate-image.bbclass b/classes-recipe/swupdate-image.bbclass

index 175fc4e..679e622 100644
--- a/classes-recipe/swupdate-image.bbclass
+++ b/classes-recipe/swupdate-image.bbclass
@@ -13,8 +13,6 @@

inherit swupdate-common
inherit image-artifact-names

-S = "${WORKDIR}/${PN}"

-

SRC_URI += "file://sw-description"
SWUPDATE_IMAGES += "${IMAGE_LINK_NAME}"

diff --git a/classes-recipe/swupdate.bbclass b/classes-recipe/swupdate.bbclass

index 3656b1c..5390c9b 100644
--- a/classes-recipe/swupdate.bbclass
+++ b/classes-recipe/swupdate.bbclass
@@ -36,8 +36,6 @@

inherit swupdate-common
inherit image-artifact-names

-S = "${WORKDIR}/${PN}"

-
IMAGE_DEPENDS ?= ""

do_configure[noexec] = "1"
--
2.34.1

[Dan Walkes]

Hi Khem,

As discussed today, let me know if you'd like any help testing this beyond what I've done with tegra-demo-distro.

Alternatively if this patch won't be upstreamed I'll implement the same workaround you did in yoe distro so we can get swupdate working again on master.

Thanks!

Dan

[Khem Raj]

Hi Stefano
Thanks for following up Dan, I re-reviewed it today again and I am ok
to apply this patch

> --
> You received this message because you are subscribed to the Google Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to swupdate+u...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/swupdate/deedc6f3-e3cf-497f-8842-f6400121abcfn%40googlegroups.com.

[Stefano Babic]

On 10.10.24 18:51, Khem Raj wrote:
> Hi Stefano
> Thanks for following up Dan, I re-reviewed it today again and I am ok
> to apply this patch

Thanks for your review ! I will apply it then.

Best regards,
Stefano