[PATCH 0/6] Update the meta-efibootguard layer for latest Yocto layers

[Farhat]

From: Farhat-Abbas <Farhat...@mentor.com>

These patches are meant to update meta-efibootguard layer so that it works
with the latest Yocto layers i.e. latest poky(rocko) layers. There are changes
in openembedded-layer related to WIC Image Creation which have broken current
meta-efibootguard layer's WIC Image Creation scripts. I have fixed prepared these
patches to align this layer with the upstream Yocto layers.

Farhat-Abbas (6):
DEPLOY_DIR_IMAGE-Issue-Resolved
WIC Error Handling is broken, no module named 'os_exit()' is
available.
Remove Extra Imports, that are not being used at all.
INITRD file must be copied to BOOT partition when specified.
Copy only the efibootguard bootloader to the EFI partition properly.
Updated README file, tested with updated Poky(rocko)

README.md | 6 +++---
.../lib/wic/plugins/source/efibootguard-boot.py | 21 +++++++++++++++------
scripts/lib/wic/plugins/source/efibootguard-efi.py | 22 ++++++++++++++--------
3 files changed, 32 insertions(+), 17 deletions(-)

--
2.7.4

[Farhat]

From: Farhat-Abbas <Farhat...@mentor.com>

Required artifacts, to be copied, are available in
'DEPLOY_DIR_IMAGE' directory pointed to by 'kernel_dir'. The recipe
efibootguard.bb stages efibootguard bootloader to 'DEPLOY_DIR_IMAGE'.
So, efibootguard should be copied from 'kernel_dir'. OE-Core's script
'oe-core/scripts/lib/wic/plugins/source/bootimg-efi.py' is used as reference.
Also, 'HDDDIR' has been removed from 'WICVARS' & won't be accessible in scripts.
'HDDDIR' was populated ONLY if 'NOHDD=1' but images can be built w/o it, so
it was optional but 'DEPLOY_DIR_IMAGE' is always populated with the required artifacts.

See following commits in 'oe-core' for reference.
https://github.com/openembedded/openembedded-core/commit/afc486b6316f1118c8dcc74a5e217bb217dd1f85
https://github.com/openembedded/openembedded-core/commit/2457ea5338f7309316b474562b4723e8cb09286d
https://github.com/openembedded/openembedded-core/commit/48a5d67d7cafdeac821e6f35bea380521ef017d5

Signed-off-by: Farhat-Abbas <Farhat...@mentor.com>
---
scripts/lib/wic/plugins/source/efibootguard-boot.py | 8 ++++----
scripts/lib/wic/plugins/source/efibootguard-efi.py | 10 +++++-----
2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/efibootguard-boot.py b/scripts/lib/wic/plugins/source/efibootguard-boot.py
index bad7c27..f4326ae 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-boot.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-boot.py
@@ -82,10 +82,10 @@ class EfibootguardBootPlugin(SourcePlugin):
'prepares' the partition to be incorporated into the image.
In this case, prepare content for an EFI (grub) boot partition.
"""
- if not bootimg_dir:
- bootimg_dir = get_bitbake_var("HDDDIR")
- if not bootimg_dir:
- msger.error("HDDDIR not set, exiting\n")
+ if not kernel_dir:
+ kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+ if not kernel_dir:
+ msger.error("DEPLOY_DIR_IMAGE not set, exiting\n")
os.exit(1)

staging_kernel_dir = kernel_dir
diff --git a/scripts/lib/wic/plugins/source/efibootguard-efi.py b/scripts/lib/wic/plugins/source/efibootguard-efi.py
index 8a33412..62e283a 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-efi.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-efi.py
@@ -55,10 +55,10 @@ class EfibootguardEFIPlugin(SourcePlugin):
'prepares' the partition to be incorporated into the image.
In this case, prepare content for an EFI (grub) boot partition.
"""
- if not bootimg_dir:
- bootimg_dir = get_bitbake_var("HDDDIR")
- if not bootimg_dir:
- msger.error("HDDDIR not set, exiting\n")
+ if not kernel_dir:
+ kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+ if not kernel_dir:
+ msger.error("DEPLOY_DIR_IMAGE not set, exiting\n")

staging_kernel_dir = kernel_dir

@@ -67,7 +67,7 @@ class EfibootguardEFIPlugin(SourcePlugin):
install_cmd = "install -d %s/EFI/BOOT" % hdddir
exec_cmd(install_cmd)

- cp_cmd = "cp %s/EFI/BOOT/* %s/EFI/BOOT" % (bootimg_dir, hdddir)
+ cp_cmd = "cp %s/EFI/BOOT/* %s/EFI/BOOT" % (kernel_dir, hdddir)
exec_cmd(cp_cmd, True)

# Calculate the number of extra blocks to be sure that the
--
2.7.4

[Farhat]

From: Farhat-Abbas <Farhat...@mentor.com>

Signed-off-by: Farhat-Abbas <Farhat...@mentor.com>
---
scripts/lib/wic/plugins/source/efibootguard-boot.py | 4 ++--
scripts/lib/wic/plugins/source/efibootguard-efi.py | 3 ++-
2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/efibootguard-boot.py b/scripts/lib/wic/plugins/source/efibootguard-boot.py
index f4326ae..6b1760f 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-boot.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-boot.py
@@ -30,6 +30,7 @@ import os
import shutil
import logging

+from wic import WicError
from wic.pluginbase import SourcePlugin
from wic.engine import get_custom_config
from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var, \
@@ -85,8 +86,7 @@ class EfibootguardBootPlugin(SourcePlugin):
if not kernel_dir:

kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")

if not kernel_dir:
- msger.error("DEPLOY_DIR_IMAGE not set, exiting\n")
- os.exit(1)
+ raise WicError("DEPLOY_DIR_IMAGE not set, exiting\n")

staging_kernel_dir = kernel_dir

diff --git a/scripts/lib/wic/plugins/source/efibootguard-efi.py b/scripts/lib/wic/plugins/source/efibootguard-efi.py
index 62e283a..7a3aef2 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-efi.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-efi.py
@@ -31,6 +31,7 @@ import os.path
import shutil
import logging

+from wic import WicError
from wic.pluginbase import SourcePlugin
from wic.engine import get_custom_config
from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var, \
@@ -58,7 +59,7 @@ class EfibootguardEFIPlugin(SourcePlugin):
if not kernel_dir:

kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")

if not kernel_dir:
- msger.error("DEPLOY_DIR_IMAGE not set, exiting\n")
+ raise WicError("DEPLOY_DIR_IMAGE not set, exiting\n")

staging_kernel_dir = kernel_dir

--
2.7.4

[Farhat]

From: Farhat-Abbas <Farhat...@mentor.com>

Signed-off-by: Farhat-Abbas <Farhat...@mentor.com>
---

scripts/lib/wic/plugins/source/efibootguard-boot.py | 1 -
scripts/lib/wic/plugins/source/efibootguard-efi.py | 2 --
2 files changed, 3 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/efibootguard-boot.py b/scripts/lib/wic/plugins/source/efibootguard-boot.py
index 6b1760f..abccfb2 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-boot.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-boot.py
@@ -27,7 +27,6 @@
# Andreas Reichel <andreas.reichel (at] tngtech.com>

import os
-import shutil
import logging

from wic import WicError
diff --git a/scripts/lib/wic/plugins/source/efibootguard-efi.py b/scripts/lib/wic/plugins/source/efibootguard-efi.py
index 7a3aef2..ab3bbfd 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-efi.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-efi.py
@@ -27,8 +27,6 @@
# Andreas Reichel <andreas.reichel (at] tngtech.com>

import os
-import os.path
-import shutil
import logging

from wic import WicError
--
2.7.4

[Farhat]

From: Farhat-Abbas <Farhat...@mentor.com>

INITRD files are available in 'DEPLOY_DIR_IMAGE' directory pointed to by 'kernel_dir'.
It can be copied to BOOT partition, when specified in 'source-params' by WKS file.

OE-Core's script 'oe-core/scripts/lib/wic/plugins/source/bootimg-efi.py' is used as reference.

Signed-off-by: Farhat-Abbas <Farhat...@mentor.com>
---
scripts/lib/wic/plugins/source/efibootguard-boot.py | 10 ++++++++++
1 file changed, 10 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/efibootguard-boot.py b/scripts/lib/wic/plugins/source/efibootguard-boot.py
index abccfb2..7b59928 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-boot.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-boot.py
@@ -100,6 +100,16 @@ class EfibootguardBootPlugin(SourcePlugin):
fd.write(part.label.upper().encode("utf-16le"))
fd.close()

+ # Copy the specified initrd to the BOOT partition
+ initrd = source_params.get('initrd')
+
+ if initrd:
+ cp_cmd = "cp %s/%s %s" % (kernel_dir, initrd, hdddir)
+ exec_cmd(cp_cmd, True)
+
+ else:
+ msger.debug("Ignoring missing initrd")
+
du_cmd = "du -bks %s" % hdddir
out = exec_cmd(du_cmd)
blocks = int(out.split()[0])
--
2.7.4

[Farhat]

From: Farhat-Abbas <Farhat...@mentor.com>

There isn't any EFI/BOOT directory in the 'DEPLOY_DIR_IMAGE'
directory & efibootguard is present at top level in it, possibly alongwith
other *efi bootloaders like grub-efi. Only efibootguard bootloader should be
copied to the image from here, in the required architecture specific naming
convention. This patch updates the script to the latest OE-Core tree with
file 'oe-core/scripts/lib/wic/plugins/source/bootimg-efi.py' as reference.

Signed-off-by: Farhat-Abbas <Farhat...@mentor.com>
---
scripts/lib/wic/plugins/source/efibootguard-efi.py | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/efibootguard-efi.py b/scripts/lib/wic/plugins/source/efibootguard-efi.py
index ab3bbfd..e626f4f 100644
--- a/scripts/lib/wic/plugins/source/efibootguard-efi.py
+++ b/scripts/lib/wic/plugins/source/efibootguard-efi.py
@@ -66,8 +66,15 @@ class EfibootguardEFIPlugin(SourcePlugin):

install_cmd = "install -d %s/EFI/BOOT" % hdddir
exec_cmd(install_cmd)

- cp_cmd = "cp %s/EFI/BOOT/* %s/EFI/BOOT" % (kernel_dir, hdddir)
- exec_cmd(cp_cmd, True)
+ # Locate & install the efibootguard bootloader properly to the EFI partition
+ # i586 ARCH: efibootguardia32.efi ---> bootia32.efi
+ # x86_64 ARCH: efibootguardx64.efi ---> bootx64.efi
+ for mod in [x for x in os.listdir(kernel_dir) if x.startswith("efibootguard")]:
+ # mod = efibootguardia32.efi or efibootguardx64.efi
+ # efi_image = bootia32.efi or bootx64.efi
+ efi_image = mod.replace('efibootguard', 'boot')
+ cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, efi_image)
+ exec_cmd(cp_cmd, True)

# Calculate the number of extra blocks to be sure that the

# resulting partition image is of the wanted size
--
2.7.4

[Farhat]

From: Farhat-Abbas <Farhat...@mentor.com>

Signed-off-by: Farhat-Abbas <Farhat...@mentor.com>
---

README.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/README.md b/README.md
index 639bd78..b7d3190 100644
--- a/README.md
+++ b/README.md
@@ -14,13 +14,13 @@ Archive:

Tested with:

-poky (rocko): 16e22f3e37788afb83044f5089d24187d70094bd
+poky (rocko): 342fbd6a3e57021c8e28b124b3adb241936f3d9d
https://git.yoctoproject.org/git/poky

-meta-intel: aa246ba9855d35d454ddcf041578310cbaa90250
+meta-intel: 7bf03f7612834142355189c46ccdb4b309acc58a
https://git.yoctoproject.org/git/meta-intel

-meta-openembedded: 29a4983d5a4462d8e7b9abcd55bfb3013f578fdd
+meta-openembedded: 8cef6b38e62e6c79ec857510df454cefc868b0df
http://git.openembedded.org/meta-openembedded

## project integration ##
--
2.7.4

[Jan Kiszka]

On 2018-02-21 13:27, Farhat wrote:
> From: Farhat-Abbas <Farhat...@mentor.com>
>
> These patches are meant to update meta-efibootguard layer so that it works
> with the latest Yocto layers i.e. latest poky(rocko) layers. There are changes
> in openembedded-layer related to WIC Image Creation which have broken current
> meta-efibootguard layer's WIC Image Creation scripts. I have fixed prepared these
> patches to align this layer with the upstream Yocto layers.

Given that Andreas was recently able to generate a working IOT2000
update demo on top of an upstream rocko state, I'm now confused if these
updates are really targeting rocko or actually only master, ie. the next
Yocto release. Can someone explain this to me?

Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[Jan Kiszka]

On 2018-02-21 18:41, [ext] Jan Kiszka wrote:
> On 2018-02-21 13:27, Farhat wrote:
>> From: Farhat-Abbas <Farhat...@mentor.com>
>>
>> These patches are meant to update meta-efibootguard layer so that it works
>> with the latest Yocto layers i.e. latest poky(rocko) layers. There are changes
>> in openembedded-layer related to WIC Image Creation which have broken current
>> meta-efibootguard layer's WIC Image Creation scripts. I have fixed prepared these
>> patches to align this layer with the upstream Yocto layers.
>
> Given that Andreas was recently able to generate a working IOT2000
> update demo on top of an upstream rocko state, I'm now confused if these
> updates are really targeting rocko or actually only master, ie. the next
> Yocto release. Can someone explain this to me?

IOW, did Yocto actually manage to break APIs inside a rocko stable series??

[Abbas, Farhat]

Apparently looks like they did! See following commits. These changes are in rocko branch alongwith master.
These commits are breaking your WIC scripts.

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=rocko&id=3ea735d390799012b16505f845c5d7e2d14d4a9b
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=rocko&id=627952a234a844783a6d4ee9aa3f1608a0d537d0

Regards,
Farhat Abbas  | Software Development Engineer
Mentor Embedded™ | 6th Floor, Ali Tower, 105 B2 M. M Alam Road, Gulberg III, Lahore, Pakistan, 54660
P +92-42-3609-9254 | M +92-343-7750571

[Andreas Reichel]

On Thu, Feb 22, 2018 at 04:12:54AM +0000, Abbas, Farhat wrote:

Had a quick lock over the patches, they seem good to me.
At the moment no time for thorough testing, but if it builds,
it should be fine. Thanks!

> --
> You received this message because you are subscribed to the Google Groups "EFI Boot Guard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to efibootguard-d...@googlegroups.com.
> To post to this group, send email to efibootg...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/efibootguard-dev/fabf767c0a244b5a9dfb4b5cb08afd2b%40svr-ies-mbx-02.mgc.mentorg.com.
> For more options, visit https://groups.google.com/d/optout.

--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant

Andreas...@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082

[Abbas, Farhat]

That'll do!

[Jan Kiszka]

On 2018-02-21 13:27, Farhat wrote:

Just a quick note that this series is not forgotten, only queued locally
until my own test setup is fully in place.

[Jan Kiszka]

On 2018-02-22 05:12, Abbas, Farhat wrote:
> Apparently looks like they did! See following commits. These changes are in rocko branch alongwith master.
> These commits are breaking your WIC scripts.
>
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=rocko&id=3ea735d390799012b16505f845c5d7e2d14d4a9b
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=rocko&id=627952a234a844783a6d4ee9aa3f1608a0d537d0

Now I'm confused again: I just built latest poky/rocko with current
meta-efibootguard using Andreas' patch queue for meta-iot2000, and all
worked fine, the build as well as the update (except for unrelated issues).

meta-efibootguard = "master:0d8218a7693b80c4672abde206ac89b941c74737"
meta-intel = "HEAD:92d66730fba1fdc71d2548332c8ff57160e31b05"
meta-nodejs = "HEAD:7e90d506488036cf1c490d1ac1086675eb2805b6"
meta-oe = "HEAD:a65c1acb1822966c3553de9fc98d8bb6be705c4e"
meta-swupdate = "HEAD:145736d31a557a4ee0ec1746ab14fec5eeb0a667"
meta
meta-poky
meta-yocto-bsp = "HEAD:5ba69a97ab5faa8f3866aaeab1d6eaa3cb8149ed"
meta-iot2000-bsp
meta-iot2000-example =
"swupdate-testing:3c38e072bbcad15421100e76e9cf63682f29d4cd"

Andreas, are your changes not triggering the issues that Farhat fixes?

Will now try what happens when combining this series with Andreas'
meta-iot2000...

Thanks,

[Jan Kiszka]

On 2018-02-21 13:27, Farhat wrote:

Unless I'm missing something, this should work also for arm and aa64 -
just that the comments do not cover that. At least aa64 is a foreseeable
use case (arm possibly as well, thanks to Alex Graf's efi enhancements
of u-boot).

[Jan Kiszka]

On 2018-02-21 13:27, Farhat wrote:

Changes work fine here, I've merged them to next.

Thanks,

[Abbas, Farhat]

Hi Jan,
Have you merged/verified all of them ?

Regards,
Farhat Abbas  | Software Development Engineer
Mentor Embedded™ | 6th Floor, Ali Tower, 105 B2 M. M Alam Road, Gulberg III, Lahore, Pakistan, 54660
P +92-42-3609-9254 | M +92-343-7750571



-----Original Message-----
From: Jan Kiszka [mailto:jan.k...@siemens.com]
Sent: Tuesday, March 06, 2018 1:12 PM
To: Abbas, Farhat <Farhat...@mentor.com>; efibootg...@googlegroups.com
Subject: Re: [PATCH 0/6] Update the meta-efibootguard layer for latest Yocto layers

[Jan Kiszka]

On 2018-03-06 09:22, Abbas, Farhat wrote:
> Hi Jan,
> Have you merged/verified all of them ?

Yes, all 6 merged to next and pushed. I don't have test cases for all of
them at hand, specifically the initrd topic requires some fixes in
Andreas' queue, so I did manual review only. I assume you validated that
in your setup.

[Abbas, Farhat]

Hi Jan,
Yeah I had validated all the cases, with/without an initrd image file.

Regards,
Farhat Abbas  | Software Development Engineer
Mentor Embedded™ | 6th Floor, Ali Tower, 105 B2 M. M Alam Road, Gulberg III, Lahore, Pakistan, 54660
P +92-42-3609-9254 | M +92-343-7750571



-----Original Message-----
From: Jan Kiszka [mailto:jan.k...@siemens.com]
Sent: Tuesday, March 06, 2018 1:59 PM
To: Abbas, Farhat <Farhat...@mentor.com>
Cc: efibootg...@googlegroups.com
Subject: Re: [PATCH 0/6] Update the meta-efibootguard layer for latest Yocto layers

[Abbas, Farhat]

Hi Jan,
When these changes will make it to the "master" branch ?

Regards,
Farhat Abbas  | Software Development Engineer
Mentor Embedded™ | 6th Floor, Ali Tower, 105 B2 M. M Alam Road, Gulberg III, Lahore, Pakistan, 54660
P +92-42-3609-9254 | M +92-343-7750571



-----Original Message-----
From: Jan Kiszka [mailto:jan.k...@siemens.com]
Sent: Tuesday, March 06, 2018 1:59 PM
To: Abbas, Farhat <Farhat...@mentor.com>
Cc: efibootg...@googlegroups.com
Subject: Re: [PATCH 0/6] Update the meta-efibootguard layer for latest Yocto layers

[Jan Kiszka]

On 2018-03-06 10:04, Abbas, Farhat wrote:
> Hi Jan,

> When these changes will make it to the "master" branch ?

I'm counting on Andreas to give them a second try when he finds time to
update his queue. If that should take longer, I will merge into master
earlier.

Jan

PS: Please avoid top-posting on mailing lists.

[Jan Kiszka]

On 2018-03-06 11:45, Jan Kiszka wrote:
> On 2018-03-06 10:04, Abbas, Farhat wrote:
>> Hi Jan,
>> When these changes will make it to the "master" branch ?
>
> I'm counting on Andreas to give them a second try when he finds time to
> update his queue. If that should take longer, I will merge into master
> earlier.

...and into rocko, of course.

Jan

[Jan Kiszka]

On 2018-03-06 11:49, [ext] Jan Kiszka wrote:
> On 2018-03-06 11:45, Jan Kiszka wrote:
>> On 2018-03-06 10:04, Abbas, Farhat wrote:
>>> Hi Jan,
>>> When these changes will make it to the "master" branch ?
>>
>> I'm counting on Andreas to give them a second try when he finds time to
>> update his queue. If that should take longer, I will merge into master
>> earlier.
>
> ...and into rocko, of course.
>

Both done.

Thanks again,
Jan

[Abbas, Farhat]

Thanks Jan.

Regards,
Farhat Abbas  | Software Development Engineer
Mentor Embedded™ | 6th Floor, Ali Tower, 105 B2 M. M Alam Road, Gulberg III, Lahore, Pakistan, 54660
P +92-42-3609-9254 | M +92-343-7750571


-----Original Message-----
From: Jan Kiszka [mailto:jan.k...@siemens.com]
Sent: Wednesday, March 07, 2018 12:16 PM
To: Abbas, Farhat <Farhat...@mentor.com>; Andreas Reichel <Andreas...@tngtech.com>
Cc: efibootg...@googlegroups.com
Subject: Re: [PATCH 0/6] Update the meta-efibootguard layer for latest Yocto layers