[PATCH] meta/classes/sdk.bbclass: add --transform in TAR_OPTIONS

[Srinuvasan Arjunan]

From: Srinuvasan A <srinuv...@mentor.com>

When we untar the SDK rootfs archive file, all the contents are extracted
into the current directory, not a folder, this looks ugly hence fixed
it with the file name transformations option.

Signed-off-by: Srinuvasan A <srinuv...@mentor.com>
---
meta/classes/sdk.bbclass | 2 ++
1 file changed, 2 insertions(+)

diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
index 06cae02..56b650a 100644
--- a/meta/classes/sdk.bbclass
+++ b/meta/classes/sdk.bbclass
@@ -11,6 +11,8 @@ inherit crossvars
BBCLASSEXTEND = "sdk"
BPN = "${PN}"

+TAR_OPTIONS = "--transform="s|.|$(basename ${PP_DEPLOY}/${IMAGE_FULLNAME})|""
+
python sdk_virtclass_handler() {
pn = e.data.getVar('PN')
if pn.endswith('-sdk'):
--
2.34.1

[Schmidt, Adriaan]

Srinuvasan Arjunan, Freitag, 31. März 2023 10:50:

> From: Srinuvasan A <srinuv...@mentor.com>
>
> When we untar the SDK rootfs archive file, all the contents are extracted
> into the current directory, not a folder, this looks ugly hence fixed
> it with the file name transformations option.
>
> Signed-off-by: Srinuvasan A <srinuv...@mentor.com>
> ---
> meta/classes/sdk.bbclass | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
> index 06cae02..56b650a 100644
> --- a/meta/classes/sdk.bbclass
> +++ b/meta/classes/sdk.bbclass
> @@ -11,6 +11,8 @@ inherit crossvars
> BBCLASSEXTEND = "sdk"
> BPN = "${PN}"
>
> +TAR_OPTIONS = "--transform="s|.|$(basename ${PP_DEPLOY}/${IMAGE_FULLNAME})|""
> +

Hi Arjunan,

this will change the tar options globally for all images, not only the SDKs.
You probably want to:
TAR_OPTIONS:class-sdk = "..."

Or even:
TAR_OPTIONS:append:class-sdk = " ..."
^- important space here

The latter would allow other options to be set via configs.

Regards,
Adriaan

> python sdk_virtclass_handler() {
> pn = e.data.getVar('PN')
> if pn.endswith('-sdk'):
> --
> 2.34.1
>

> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/20230331084952.414690-1-
> srinuvasan_a%40mentor.com.

[Srinuvasan Arjunan]

From: Srinuvasan A <srinuv...@mentor.com>

When we untar the SDK rootfs archive file, all the contents are extracted
into the current directory, not a folder, this looks ugly hence fixed
it with the file name transformations option.

Signed-off-by: Srinuvasan A <srinuv...@mentor.com>
---

meta/classes/sdk.bbclass | 1 +
1 file changed, 1 insertion(+)

diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
index 06cae02..50f718c 100644
--- a/meta/classes/sdk.bbclass
+++ b/meta/classes/sdk.bbclass
@@ -53,6 +53,7 @@ ROOTFS_MANIFEST_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
ROOTFS_DPKGSTATUS_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"

IMAGE_FSTYPES:class-sdk = "${SDK_FORMATS}"
+TAR_OPTIONS:append:class-sdk = " --transform="s|.|$(basename ${PP_DEPLOY}/${IMAGE_FULLNAME})|""

# bitbake dependencies
SDKDEPENDS += "sdk-files ${SDK_INSTALL}"
--
2.34.1

[Srinuvasan Arjunan]

   Hi adriaan,

                    Can you please have a look, addressed your comments.

  Thanks,

  Srinuvasan.A

[Schmidt, Adriaan]

Hi,

Yes, looks better now. This limits the change to the SDK tarball only.

However, I did not test the result, as I don’t really use the Isar SDK Feature.

Maybe someone else can comment whether this change in the SDK tarball could have impact somewhere else.

Thanks,

Adriaan

 

--

You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/06e04599-5749-418f-a162-2e3a9b019150n%40googlegroups.com.

[Jan Kiszka]

To my understanding, this will move the content of the SDK tarball into
a subfolder. What will that name be? ${IMAGE_FULLNAME}? And isn't
$(basename ${PP_DEPLOY}/${IMAGE_FULLNAME}) the same as just
${IMAGE_FULLNAME}?

I'm fine with the relocation per se, I'm just wondering if we documented
or otherwise referred to the structure of an unpacked SDK somewhere, in
the user manual eg., and if that then also needs updating.

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

[Srinuvasan Arjunan]

On Wed, Apr 5, 2023 at 4:27 PM 'Jan Kiszka' via isar-users <isar-...@googlegroups.com> wrote:

On 31.03.23 12:33, Srinuvasan Arjunan wrote:
> From: Srinuvasan A <srinuv...@mentor.com>
>
> When we untar the SDK rootfs archive file, all the contents are extracted
> into the current directory, not a folder, this looks ugly hence fixed
> it with the file name transformations option.
>
> Signed-off-by: Srinuvasan A <srinuv...@mentor.com>
> ---
>  meta/classes/sdk.bbclass | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
> index 06cae02..50f718c 100644
> --- a/meta/classes/sdk.bbclass
> +++ b/meta/classes/sdk.bbclass
> @@ -53,6 +53,7 @@ ROOTFS_MANIFEST_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
>  ROOTFS_DPKGSTATUS_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
> 
>  IMAGE_FSTYPES:class-sdk = "${SDK_FORMATS}"
> +TAR_OPTIONS:append:class-sdk = " --transform="s|.|$(basename ${PP_DEPLOY}/${IMAGE_FULLNAME})|""
> 
>  # bitbake dependencies
>  SDKDEPENDS += "sdk-files ${SDK_INSTALL}"

To my understanding, this will move the content of the SDK tarball into
a subfolder. What will that name be? ${IMAGE_FULLNAME}? And isn't
$(basename ${PP_DEPLOY}/${IMAGE_FULLNAME}) the same as just
${IMAGE_FULLNAME}?

    Tested, yes ${IMAGE_FULLNAME} works here instead of $(basename ${PP_DEPLOY}/${IMAGE_FULLNAME}) 

I'm fine with the relocation per se, I'm just wondering if we documented
or otherwise referred to the structure of an unpacked SDK somewhere, in
the user manual eg., and if that then also needs updating.

     Yes we have gap in the document, will update and send next version 

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/b460e006-136e-9c4f-311a-edfa724c32e2%40siemens.com.

[Srinuvasan Arjunan]

From: Srinuvasan A <srinuv...@mentor.com>

When we untar the SDK rootfs archive file, all the contents are extracted
into the current directory, not a folder, this looks ugly hence fixed
it with the file name transformations option.

But the below transform pattern not works, specifically with the symlink
TAR_OPTIONS:append:class-sdk = " --transform="s|.|${IMAGE_FULLNAME}|""

instead of changedir -C with {PP_ROOTFS} and choose all files (.),
use {PP} and choose rootfs dir, now by default extracted into a rootfs
sub folder, in this case --transform works fine with the filename.

Updated the document as well.

Signed-off-by: Srinuvasan A <srinuv...@mentor.com>
---

doc/user_manual.md | 20 ++++++++++----------
meta/classes/imagetypes.bbclass | 2 +-
meta/classes/sdk.bbclass | 1 +
3 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 92075a8..60b7ca9 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -1063,8 +1063,8 @@ User manually triggers creation of SDK root filesystem for his target platform b
`bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).

-The resulting SDK rootfs is archived into `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz`.
-It is additionally available for direct use under `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}/`.
+The resulting SDK rootfs is archived into `tmp/deploy/images/${MACHINE}/${IMAGE_FULLNAME}.tar.xz`.
+Once you untar the compressed file, the content will be extracted into the ${IMAGE_FULLNAME} sub folder.
The SDK rootfs directory `/isar-apt` contains a copy of isar-apt repo with locally prebuilt target debian packages (for <HOST_DISTRO>).
One may chroot into the SDK and install required target packages with the help of `apt-get install <package_name>:<DISTRO_ARCH>` command.

@@ -1073,7 +1073,7 @@ One may chroot into the SDK and install required target packages with the help o
- Trigger creation of SDK root filesystem

```
-bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base
+bitbake -c do_populate_sdk mc:qemuarm-bullseye:isar-image-base
```

- Mount the following directories in chroot by passing resulting rootfs as an argument to the script `mount_chroot.sh`:
@@ -1091,14 +1091,14 @@ mount devtmpfs $1/dev -t devtmpfs -o mode=0755,nosuid
mount devpts $1/dev/pts -t devpts -o gid=5,mode=620
mount tmpfs $1/dev/shm -t tmpfs -o rw,seclabel,nosuid,nodev

-$ sudo scripts/mount_chroot.sh ../build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf
+$ sudo scripts/mount_chroot.sh ../build/tmp/deploy/images/qemuarm/isar-image-base-sdk-debian-bullseye-qemuarm

```

- chroot to isar SDK rootfs:

```
-$ sudo chroot build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf
+$ sudo chroot build/tmp/deploy/images/qemuarm/isar-image-base-sdk-debian-bullseye-qemuarm
```
- Check that cross toolchains are installed

@@ -1190,7 +1190,7 @@ Daemon socket) accessible in the container.
What can endanger the stability and security of the host.

The resulting SDK formats are archived into
-`tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_format}.tar.xz`
+`tmp/deploy/images/${MACHINE}/isar-image-base-sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_format}.tar.xz`
(being `sdk_format` each one of the formats specified in `SDK_FORMATS`).
The SDK container directory `/isar-apt` contains a copy of isar-apt repo with
locally prebuilt target debian packages (for <HOST_DISTRO>).
@@ -1207,27 +1207,27 @@ to edit files in the host with an IDE and build in the container.
For one-shot builds (use `local.conf` otherwise):

```
-export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMATS"
+export BB_ENV_PASSTHROUGH_ADDITIONS="$BB_ENV_EXTRAWHITE SDK_FORMATS"
export SDK_FORMATS="docker-archive"
```

- Trigger creation of SDK root filesystem

```
-bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base
+bitbake -c do_populate_sdk mc:qemuarm-bullseye:isar-image-base
```

- Load the SDK container image into the Docker Daemon

```
-docker load -i build/tmp/deploy/images/qemuarm/sdk-isar-image-base-debian-buster-armhf-1.0-r0-docker-archive.tar.xz
+docker load -i build/tmp/deploy/images/qemuarm/isar-image-base-sdk-debian-bullseye-armhf-1.0-r0-docker-archive.tar.xz
```

- Run a container using the SDK container image (following commands starting
with `#~:` are to be run in the container)

```
-docker run --rm -ti --volume "$(pwd):/build" sdk-isar-image-base-debian-buster-armhf:1.0-r0
+docker run --rm -ti --volume "$(pwd):/build" isar-image-base-sdk-debian-bullseye-armhf:1.0-r0
```

- Check that cross toolchains are installed
diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass
index 2ee3240..e476778 100644
--- a/meta/classes/imagetypes.bbclass
+++ b/meta/classes/imagetypes.bbclass
@@ -9,7 +9,7 @@ TAR_OPTIONS ?= ""

IMAGE_CMD:tar() {
${SUDO_CHROOT} tar ${TAR_OPTIONS} -cvSf \
- ${IMAGE_FILE_CHROOT} --one-file-system -C ${PP_ROOTFS} .
+ ${IMAGE_FILE_CHROOT} --one-file-system -C ${PP} rootfs
}

# image type: ext4
diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
index 06cae02..c6dc689 100644

--- a/meta/classes/sdk.bbclass
+++ b/meta/classes/sdk.bbclass
@@ -53,6 +53,7 @@ ROOTFS_MANIFEST_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
ROOTFS_DPKGSTATUS_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"

IMAGE_FSTYPES:class-sdk = "${SDK_FORMATS}"

+TAR_OPTIONS:append:class-sdk = " --transform="s|rootfs|${IMAGE_FULLNAME}|""

# bitbake dependencies
SDKDEPENDS += "sdk-files ${SDK_INSTALL}"

--
2.34.1

[Srinuvasan Arjunan]

Any comments or improvements?

--
You received this message because you are subscribed to a topic in the Google Groups "isar-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/isar-users/4-CFUZ66nmo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to isar-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20230419140940.1268690-1-srinuvasan_a%40mentor.com.

[Anton Mikanovich]

19/04/2023 17:09, Srinuvasan Arjunan wrote:
> From: Srinuvasan A <srinuv...@mentor.com>
>
> When we untar the SDK rootfs archive file, all the contents are extracted
> into the current directory, not a folder, this looks ugly hence fixed
> it with the file name transformations option.
>
> But the below transform pattern not works, specifically with the symlink
> TAR_OPTIONS:append:class-sdk = " --transform="s|.|${IMAGE_FULLNAME}|""
>
> instead of changedir -C with {PP_ROOTFS} and choose all files (.),
> use {PP} and choose rootfs dir, now by default extracted into a rootfs
> sub folder, in this case --transform works fine with the filename.
>
> Updated the document as well.
>
> Signed-off-by: Srinuvasan A <srinuv...@mentor.com>

Applied to next, thanks.

[Jan Kiszka]

This not only affected the sdk, it broke all tarball image we generate
by adding rootfs/ to the extraction path. I have no good suggestion how
to limit the effect on sdk only, so I would vote for now reverting this.

Better suggestions?

Jan

> }
>
> # image type: ext4
> diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
> index 06cae02..c6dc689 100644
> --- a/meta/classes/sdk.bbclass
> +++ b/meta/classes/sdk.bbclass
> @@ -53,6 +53,7 @@ ROOTFS_MANIFEST_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
> ROOTFS_DPKGSTATUS_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
>
> IMAGE_FSTYPES:class-sdk = "${SDK_FORMATS}"
> +TAR_OPTIONS:append:class-sdk = " --transform="s|rootfs|${IMAGE_FULLNAME}|""
>
> # bitbake dependencies
> SDKDEPENDS += "sdk-files ${SDK_INSTALL}"

--

Siemens AG, Technology
Linux Expert Center

[Gylstorff Quirin]

I posted a possible fix for this in
https://groups.google.com/g/isar-users/c/a1s7UqXI4kY

Quirin