[PATCH 1/1] add support for debian build profiles

[Felix Moessbauer]

This patch adds the bitbake variables DEB_BUILD_PROFILES and
DEB_BUILD_PROFILES_CROSS.
The values of these variables are used to define the DEB_BUILD_PROFILES
environment variable.

When cross-compiling, the DEB_BUILD_PROFILES_CROSS variable is defaulted
to "cross", to be consistent with upstream debian.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
RECIPE-API-CHANGELOG.md | 8 ++++++++
meta/classes/dpkg-base.bbclass | 12 ++++++++++++
meta/classes/dpkg.bbclass | 5 ++---
3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 55836258..f95f7db5 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -306,3 +306,11 @@ When using the plugins it is advised to name the partition "/boot" and to exclud

The variable is renamed to get closer to OE/Poky variables naming. The old naming
will still also work, but with deprecation warning shown.
+
+### Introduce debian build profiles
+
+All recipes that inherit from dpkg and dpkg-base can utilize the variables DEB_BUILD_PROFILES and DEB_BUILD_PROFILES_CROSS.
+These variables define the DEB_BUILD_PROFILES environment variable which is available in do_install_builddeps and do_dpkg_build.
+The DEB_BUILD_PROFILES_CROSS bitbake variable is set to "cross" when cross compiling.
+
+
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 6704385b..c1588528 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -13,6 +13,8 @@ inherit deb-dl-dir
DEPENDS ?= ""

DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
+DEB_BUILD_PROFILES ?= ""
+DEB_BUILD_PROFILES_CROSS ?= "cross"

python do_adjust_git() {
import subprocess
@@ -201,7 +203,16 @@ dpkg_runbuild() {
die "This should never be called, overwrite it in your derived class"
}

+def isar_export_build_profiles(d):
+ import os
+ deb_build_profiles = os.environ['DEB_BUILD_PROFILES'] if 'DEB_BUILD_PROFILES' in os.environ else ''
+ deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES', True)
+ if d.getVar("ISAR_CROSS_COMPILE") == "1":
+ deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES_CROSS', True)
+ os.environ['DEB_BUILD_PROFILES'] = deb_build_profiles.strip()
+
python do_dpkg_build() {
+ isar_export_build_profiles(d)
lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
shared=True)
bb.build.exec_func("dpkg_do_mounts", d)
@@ -278,6 +289,7 @@ python do_devshell() {
bb.build.exec_func('dpkg_do_mounts', d)

isar_export_proxies(d)
+ isar_export_build_profiles(d)

buildchroot = d.getVar('BUILDCHROOT_DIR')
pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS'))
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 7da73341..8e5626e8 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -9,10 +9,9 @@ PACKAGE_ARCH ?= "${DISTRO_ARCH}"
do_install_builddeps() {
dpkg_do_mounts
E="${@ isar_export_proxies(d)}"
+ E="${@ isar_export_build_profiles(d)}"
distro="${DISTRO}"
- if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
- distro="${HOST_DISTRO}"
- fi
+
deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
${PP}/${PPS} ${PACKAGE_ARCH} --download-only
--
2.30.2

[vijai kumar]

Hi Felix,

Thank you for the patch.

We could modify the u-boot recipe to have it as an example.

Also, It might be useful if we can say something about custom profile
names. Debian has a standard[1] for those.
pkg.$sourcepackage.$anything

Better to recommend users to follow it, to make sure they don't
accidentally reuse one of the registered profile names when creating
custom profiles.

[1] https://wiki.debian.org/BuildProfileSpec

> +
> +
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 6704385b..c1588528 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -13,6 +13,8 @@ inherit deb-dl-dir
> DEPENDS ?= ""
>
> DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
> +DEB_BUILD_PROFILES ?= ""
> +DEB_BUILD_PROFILES_CROSS ?= "cross"

Do we really need two variables? Is the DEB_BUILD_PROFILES not sufficient?

>
> python do_adjust_git() {
> import subprocess
> @@ -201,7 +203,16 @@ dpkg_runbuild() {
> die "This should never be called, overwrite it in your derived class"
> }
>
> +def isar_export_build_profiles(d):
> + import os
> + deb_build_profiles = os.environ['DEB_BUILD_PROFILES'] if 'DEB_BUILD_PROFILES' in os.environ else ''

Ideally the user should set DEB_BUILD_PROFILES in the recipe and the
below line would be sufficient.

> + deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES', True)
> + if d.getVar("ISAR_CROSS_COMPILE") == "1":
> + deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES_CROSS', True)

Probably we can directly add "cross" and drop DEB_BUILD_PROFILES_CROSS

Probably a mistake dropping the above lines?

Thanks,
Vijai Kumar K

> +
> deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
> sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
> ${PP}/${PPS} ${PACKAGE_ARCH} --download-only
> --
> 2.30.2
>

> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20211221190653.1142246-1-felix.moessbauer%40siemens.com.

[Moessbauer, Felix]

Hi Vijai,

Good idea, I'll send a v2 out next week.

>
> Also, It might be useful if we can say something about custom profile names.
> Debian has a standard[1] for those.
> pkg.$sourcepackage.$anything

That's true, but is it really the responsibility of ISAR to ensure that developers follow the Debian best practices?
I doubt that. I would better only document ISAR specific stuff but avoid re-documenting Debian policies / best practices.

>
> Better to recommend users to follow it, to make sure they don't accidentally
> reuse one of the registered profile names when creating custom profiles.
>
> [1]

> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.debi
> an.org%2FBuildProfileSpec&amp;data=04%7C01%7Cfelix.moessbauer%40sieme
> ns.com%7C5bf5db77529f46681e4108d9c4c4297a%7C38ae3bcd95794fd4addab
> 42e1495d55a%7C1%7C0%7C637757170088619029%7CUnknown%7CTWFpbGZs
> b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0
> %3D%7C3000&amp;sdata=aExYnPwm%2F2hvZ%2BkhJQA4Y%2BWfKRGfxet%2B
> T1na%2FdpIgSQ%3D&amp;reserved=0

>
> > +
> > +
> > diff --git a/meta/classes/dpkg-base.bbclass
> > b/meta/classes/dpkg-base.bbclass index 6704385b..c1588528 100644
> > --- a/meta/classes/dpkg-base.bbclass
> > +++ b/meta/classes/dpkg-base.bbclass
> > @@ -13,6 +13,8 @@ inherit deb-dl-dir
> > DEPENDS ?= ""
> >
> > DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if
> d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-
> essential-riscv64' else ''}"
> > +DEB_BUILD_PROFILES ?= ""
> > +DEB_BUILD_PROFILES_CROSS ?= "cross"
>
> Do we really need two variables? Is the DEB_BUILD_PROFILES not sufficient?

Well... I implemented this patch series while working on a layer that has to support both arm64 and x64.
With just one variable the inline python functions became a pattern because (depending on the package), various profiles had to be activated only for arm64 (e.g. like "nocheck", "nodoc").
That's why I prefer to have both variables.
Is there any strong reason against it (apart from having two variables to maintain)?

>
> >
> > python do_adjust_git() {
> > import subprocess
> > @@ -201,7 +203,16 @@ dpkg_runbuild() {
> > die "This should never be called, overwrite it in your derived class"
> > }
> >
> > +def isar_export_build_profiles(d):
> > + import os
> > + deb_build_profiles = os.environ['DEB_BUILD_PROFILES'] if
> 'DEB_BUILD_PROFILES' in os.environ else ''
>
> Ideally the user should set DEB_BUILD_PROFILES in the recipe and the below line
> would be sufficient.

That's for backward compatibility. We already have a ton of recipes where the profiles are manually activated by exporting the DEB_BUILD_PROFILES in the do_install_builddeps and do_dpkg_build step.
IMO it's not good to "magically" overwrite environment variables, but your points are also valid.

>
> > + deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES', True)
> > + if d.getVar("ISAR_CROSS_COMPILE") == "1":
> > + deb_build_profiles += ' ' +
> > + d.getVar('DEB_BUILD_PROFILES_CROSS', True)
>
> Probably we can directly add "cross" and drop DEB_BUILD_PROFILES_CROSS

See my comment above.

Good catch! Yes, that's a mistake. Will be fixed in v2.

Felix!

>
> Thanks,
> Vijai Kumar K
>
> > +
> > deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
> > sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
> > ${PP}/${PPS} ${PACKAGE_ARCH} --download-only
> > --
> > 2.30.2
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+...@googlegroups.com.
> > To view this discussion on the web visit

> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> oogle.com%2Fd%2Fmsgid%2Fisar-users%2F20211221190653.1142246-1-
> felix.moessbauer%2540siemens.com&amp;data=04%7C01%7Cfelix.moessbauer
> %40siemens.com%7C5bf5db77529f46681e4108d9c4c4297a%7C38ae3bcd95794
> fd4addab42e1495d55a%7C1%7C0%7C637757170088619029%7CUnknown%7CT
> WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV
> CI6Mn0%3D%7C3000&amp;sdata=eCOTmdSafQe7QH5a%2BG6CmdvqmUXyse9
> K06QPIZHVXMM%3D&amp;reserved=0.

>
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+...@googlegroups.com.
> To view this discussion on the web visit

> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> oogle.com%2Fd%2Fmsgid%2Fisar-
> users%2FCALLGG_J19GiSTcmyOgOeUftatj2kkbZMaVZReJYRon_YAmJBmA%2540
> mail.gmail.com&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C
> 5bf5db77529f46681e4108d9c4c4297a%7C38ae3bcd95794fd4addab42e1495d55
> a%7C1%7C0%7C637757170088619029%7CUnknown%7CTWFpbGZsb3d8eyJWIj
> oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C300
> 0&amp;sdata=KXiRwZVA0MUjQCThhD5SZ5r4DmvkqiAj5692jiNAIfw%3D&amp;re
> served=0.

[Jan Kiszka]

On 31.12.21 12:23, Moessbauer, Felix (T RDA IOT SES-DE) wrote:
>>> diff --git a/meta/classes/dpkg-base.bbclass
>>> b/meta/classes/dpkg-base.bbclass index 6704385b..c1588528 100644
>>> --- a/meta/classes/dpkg-base.bbclass
>>> +++ b/meta/classes/dpkg-base.bbclass
>>> @@ -13,6 +13,8 @@ inherit deb-dl-dir
>>> DEPENDS ?= ""
>>>
>>> DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if
>> d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-
>> essential-riscv64' else ''}"
>>> +DEB_BUILD_PROFILES ?= ""
>>> +DEB_BUILD_PROFILES_CROSS ?= "cross"
>>
>> Do we really need two variables? Is the DEB_BUILD_PROFILES not sufficient?
>
> Well... I implemented this patch series while working on a layer that has to support both arm64 and x64.
> With just one variable the inline python functions became a pattern because (depending on the package), various profiles had to be activated only for arm64 (e.g. like "nocheck", "nodoc").
> That's why I prefer to have both variables.
> Is there any strong reason against it (apart from having two variables to maintain)?
>

The central pattern to append "cross" to DEB_BUILD_PROFILES if
cross-compiling makes sense. I just wonder if making that "cross"
appendix configurable via a variable is actually necessary. Also, that
var name is confusing, may suggest that DEB_BUILD_PROFILES_CROSS is used
alternatively to DEB_BUILD_PROFILES if cross-compiling.

Jan

--
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

[Felix Moessbauer]

Changes since v1:

- fix erronous code removal reported by Vijai
- only use value of DEB_BUILD_PROFILES_CROSS when cross-compiling (no combination of values)
- improve API changelog docs
- rebased onto next
- use DEB_BUILD_PROFILES infrastructure in linux-custom.inc


Felix Moessbauer (2):

add support for debian build profiles

refactor linux-custom.inc to use ISAR's DEB_BUILD_PROFILES support

RECIPE-API-CHANGELOG.md | 9 +++++++++
meta/classes/dpkg-base.bbclass | 13 +++++++++++++
meta/classes/dpkg.bbclass | 4 +++-
meta/recipes-kernel/linux/linux-custom.inc | 14 +++++++++-----
4 files changed, 34 insertions(+), 6 deletions(-)

--
2.30.2

[Felix Moessbauer]

This patch adds the bitbake variables DEB_BUILD_PROFILES and
DEB_BUILD_PROFILES_CROSS.
The values of these variables are used to define the DEB_BUILD_PROFILES
environment variable.

When cross-compiling, the DEB_BUILD_PROFILES_CROSS variable is defaulted
to "cross", to be consistent with upstream debian.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

RECIPE-API-CHANGELOG.md | 9 +++++++++
meta/classes/dpkg-base.bbclass | 13 +++++++++++++
meta/classes/dpkg.bbclass | 4 +++-

3 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 3bbb42a9..1f3a19b4 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -314,3 +314,12 @@ The "NAME" used to be rather static and the TAG was always "latest", now the val
### Renamed variable CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS

The meaning remains the same, just the name changed.

+### Introduce debian build profiles
+
+All recipes that inherit from dpkg and dpkg-base can utilize the variables DEB_BUILD_PROFILES and DEB_BUILD_PROFILES_CROSS.
+These variables define the DEB_BUILD_PROFILES environment variable which is available in do_install_builddeps and do_dpkg_build.

+When cross compiling, the values in DEB_BUILD_PROFILES_CROSS (default: "cross") are used to define the environment variable.
+
+For a list of well-known Debian build profiles and common practices, we refer to Debian's BuildProfileSpec.
+
+
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index cb5ce4a9..05c1cbd3 100644

--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -13,6 +13,8 @@ inherit deb-dl-dir
DEPENDS ?= ""

DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"

+DEB_BUILD_PROFILES ?= ""
+DEB_BUILD_PROFILES_CROSS ?= "cross"

python do_adjust_git() {
import subprocess
@@ -201,7 +203,17 @@ dpkg_runbuild() {

die "This should never be called, overwrite it in your derived class"
}

+def isar_export_build_profiles(d):
+ import os
+ deb_build_profiles = os.environ['DEB_BUILD_PROFILES'] if 'DEB_BUILD_PROFILES' in os.environ else ''

+ if d.getVar("ISAR_CROSS_COMPILE") == "1":

+ deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES_CROSS', True)

+ else:

+ deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES', True)

+ os.environ['DEB_BUILD_PROFILES'] = deb_build_profiles.strip()
+
python do_dpkg_build() {
+ isar_export_build_profiles(d)
lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
shared=True)
bb.build.exec_func("dpkg_do_mounts", d)

@@ -279,6 +291,7 @@ python do_devshell() {

isar_export_proxies(d)
isar_export_ccache(d)

+ isar_export_build_profiles(d)

buildchroot = d.getVar('BUILDCHROOT_DIR')
pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS'))
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass

index 27fe84f4..68f63f93 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -9,10 +9,12 @@ PACKAGE_ARCH ?= "${DISTRO_ARCH}"

do_install_builddeps() {
dpkg_do_mounts
E="${@ isar_export_proxies(d)}"
+ E="${@ isar_export_build_profiles(d)}"
distro="${DISTRO}"

if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
- distro="${HOST_DISTRO}"

+ distro="${HOST_DISTRO}"
fi

[Felix Moessbauer]

This patch replaces the manual setup of the DEB_BUILD_PROFILES
environment variable in the linux-custom.inc recipe.
Instead, the recently introduced DEB_BUILD_PROFILES infrastructure
of ISAR is used.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

meta/recipes-kernel/linux/linux-custom.inc | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index ed89aa09..5ac08487 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -117,6 +117,15 @@ def config_fragments(d):
fragments.append(local)
return fragments

+def get_additional_build_profiles(d):
+ profiles = d.getVar('BASE_DISTRO', True)
+ if d.getVar('KERNEL_LIBC_DEV_DEPLOY', True) != '1':
+ profiles += ' nolibcdev'
+ return profiles
+
+DEB_BUILD_PROFILES += "${@get_additional_build_profiles(d)}"
+DEB_BUILD_PROFILES_CROSS += "${@get_additional_build_profiles(d)}"
+
do_prepare_build_prepend() {
# copy meta-data over to source tree
rm -rf ${S}/debian
@@ -176,10 +185,5 @@ dpkg_configure_kernel() {
}

dpkg_runbuild_prepend() {
- profiles="${BASE_DISTRO}"
- if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then
- profiles="${profiles} nolibcdev"
- fi
- export DEB_BUILD_PROFILES="${profiles}"
dpkg_configure_kernel
}
--
2.30.2

[Jan Kiszka]

On 04.01.22 11:02, Felix Moessbauer wrote:
> This patch adds the bitbake variables DEB_BUILD_PROFILES and
> DEB_BUILD_PROFILES_CROSS.
> The values of these variables are used to define the DEB_BUILD_PROFILES
> environment variable.
>
> When cross-compiling, the DEB_BUILD_PROFILES_CROSS variable is defaulted
> to "cross", to be consistent with upstream debian.

This interface is clearly a step backward compared to v1.

I still don't see the use case why someone would want to avoid that Isar
auto-injects "cross" into the build profile list when doing cross
compilation, thus the use case for a configurable variable.

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

[Jan Kiszka]

On 04.01.22 11:02, Felix Moessbauer wrote:

As said on patch 1: Having to set two vars in recipes is needless
boilerplate logic.

[vijai kumar]

I was worried more about accidentally creating a custom profile name
that is predefined. for example nocheck and probably using it for a
different purpose thereby creating a possible conflict.
We could give a pointer to the predefined list if that helps.

I might need to do a PoC to confirm, but it seems very much possible.

Can you give me an example? Does DEB_BUILD_PROFILES not give an option
to activate profile based on arch?

>
> >
> > >
> > > python do_adjust_git() {
> > > import subprocess
> > > @@ -201,7 +203,16 @@ dpkg_runbuild() {
> > > die "This should never be called, overwrite it in your derived class"
> > > }
> > >
> > > +def isar_export_build_profiles(d):
> > > + import os
> > > + deb_build_profiles = os.environ['DEB_BUILD_PROFILES'] if
> > 'DEB_BUILD_PROFILES' in os.environ else ''
> >
> > Ideally the user should set DEB_BUILD_PROFILES in the recipe and the below line
> > would be sufficient.
>
> That's for backward compatibility. We already have a ton of recipes where the profiles are manually activated by exporting the DEB_BUILD_PROFILES in the do_install_builddeps and do_dpkg_build step.
> IMO it's not good to "magically" overwrite environment variables, but your points are also valid.

Ok. In case you have a lot of recipes in such a way, and it needs time
to migrate. But we might need to have a deprecation notice(warning).

[vijai kumar]

Still trying to understand the need for the extra variable. Maybe an
example will help me understand better.
It could just be DEB_BUILD_PROFILES += "cross"

Probably an unwanted indent change.

Thanks,
Vijai Kumar K

[Felix Moessbauer]

Changes since v2:

- only provide DEB_BUILD_PROFILES variable (no _CROSS variable)
- add "cross" directly to env variable when cross-compiling
- add DEB_BUILD_OPTIONS bitbake variable
(profiles like "nocheck" must also add "nocheck" to DEB_BUILD_OPTIONS)
- update API changelog
- use DEB_BUILD_OPTIONS in hello.bb example

Changes since v1:

- fix erronous code removal reported by Vijai
- only use value of DEB_BUILD_PROFILES_CROSS when cross-compiling (no combination of values)
- improve API changelog docs
- rebased onto next
- use DEB_BUILD_PROFILES infrastructure in linux-custom.inc

Felix Moessbauer (3):
add support for debian build profiles and options

refactor linux-custom.inc to use ISAR's DEB_BUILD_PROFILES support

Use DEB_BUILD_OPTIONS bb variable in hello.bb example

RECIPE-API-CHANGELOG.md | 9 +++++++++
meta-isar/recipes-app/hello/hello.bb | 8 ++------
meta/classes/dpkg-base.bbclass | 16 ++++++++++++++++
meta/classes/dpkg.bbclass | 4 +++-
.../recipes-bsp/libubootenv/libubootenv_0.3-3.bb | 5 +----
meta/recipes-kernel/linux/linux-custom.inc | 13 ++++++++-----
6 files changed, 39 insertions(+), 16 deletions(-)

--
2.30.2

[Felix Moessbauer]

This patch adds the bitbake variables DEB_BUILD_PROFILES and DEB_BUILD_OPTIONS.
These are used to define the respective environment variables.

When cross-compiling, "cross" is added to the DEB_BUILD_PROFILES environment variable.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
RECIPE-API-CHANGELOG.md | 9 +++++++++

meta/classes/dpkg-base.bbclass | 16 ++++++++++++++++
meta/classes/dpkg.bbclass | 4 +++-

3 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 3bbb42a9..65a1d9e4 100644

--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -314,3 +314,12 @@ The "NAME" used to be rather static and the TAG was always "latest", now the val
### Renamed variable CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS

The meaning remains the same, just the name changed.
+### Introduce debian build profiles
+

+All recipes that inherit from dpkg and dpkg-base can utilize the variables DEB_BUILD_PROFILES and DEB_BUILD_OPTIONS.
+The bitbake variable defines the respective environment variable which is available in `do_install_builddeps` and `do_dpkg_build`.
+When cross compiling, `cross` is added to the DEB_BUILD_PROFILES environment variable.

+
+For a list of well-known Debian build profiles and common practices, we refer to Debian's BuildProfileSpec.
+
+
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass

index cb5ce4a9..ebf67ebc 100644

--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -13,6 +13,8 @@ inherit deb-dl-dir
DEPENDS ?= ""

DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
+DEB_BUILD_PROFILES ?= ""

+DEB_BUILD_OPTIONS ?= ""

python do_adjust_git() {
import subprocess
@@ -201,7 +203,20 @@ dpkg_runbuild() {

die "This should never be called, overwrite it in your derived class"
}

+def isar_export_build_settings(d):

+ import os
+ deb_build_profiles = os.environ['DEB_BUILD_PROFILES'] if 'DEB_BUILD_PROFILES' in os.environ else ''

+ deb_build_profiles += ' ' + d.getVar('DEB_BUILD_PROFILES', True)

+ if d.getVar("ISAR_CROSS_COMPILE") == "1":

+ deb_build_profiles += ' cross'

+ os.environ['DEB_BUILD_PROFILES'] = deb_build_profiles.strip()
+

+ deb_build_options = os.environ['DEB_BUILD_OPTIONS'] if 'DEB_BUILD_OPTIONS' in os.environ else ''
+ deb_build_options += ' ' + d.getVar('DEB_BUILD_OPTIONS', True)
+ os.environ['DEB_BUILD_OPTIONS'] = deb_build_options.strip()
+
python do_dpkg_build() {
+ isar_export_build_settings(d)

lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
shared=True)
bb.build.exec_func("dpkg_do_mounts", d)

@@ -279,6 +294,7 @@ python do_devshell() {

isar_export_proxies(d)
isar_export_ccache(d)
+ isar_export_build_settings(d)

buildchroot = d.getVar('BUILDCHROOT_DIR')
pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS'))
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass

index 27fe84f4..67d63813 100644

--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -9,10 +9,12 @@ PACKAGE_ARCH ?= "${DISTRO_ARCH}"
do_install_builddeps() {
dpkg_do_mounts
E="${@ isar_export_proxies(d)}"

+ E="${@ isar_export_build_settings(d)}"

distro="${DISTRO}"
if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
- distro="${HOST_DISTRO}"
+ distro="${HOST_DISTRO}"

[Felix Moessbauer]

This patch replaces the manual setup of the DEB_BUILD_PROFILES
environment variable in the linux-custom.inc recipe.
Instead, the recently introduced DEB_BUILD_PROFILES infrastructure
of ISAR is used.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

meta/recipes-kernel/linux/linux-custom.inc | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index ed89aa09..59d42c84 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -117,6 +117,14 @@ def config_fragments(d):

fragments.append(local)
return fragments

+def get_additional_build_profiles(d):
+ profiles = d.getVar('BASE_DISTRO', True)
+ if d.getVar('KERNEL_LIBC_DEV_DEPLOY', True) != '1':
+ profiles += ' nolibcdev'
+ return profiles
+
+DEB_BUILD_PROFILES += "${@get_additional_build_profiles(d)}"

+
do_prepare_build_prepend() {
# copy meta-data over to source tree
rm -rf ${S}/debian

@@ -176,10 +184,5 @@ dpkg_configure_kernel() {

[Felix Moessbauer]

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

meta-isar/recipes-app/hello/hello.bb | 8 ++------
meta/recipes-bsp/libubootenv/libubootenv_0.3-3.bb | 5 +----
2 files changed, 3 insertions(+), 10 deletions(-)

diff --git a/meta-isar/recipes-app/hello/hello.bb b/meta-isar/recipes-app/hello/hello.bb
index d6bdf9bb..acf8ed73 100644
--- a/meta-isar/recipes-app/hello/hello.bb
+++ b/meta-isar/recipes-app/hello/hello.bb
@@ -15,6 +15,8 @@ SRC_URI = "apt://${PN}"
MAINTAINER = "isar-users <isar-...@googlegroups.com>"
CHANGELOG_V = "<orig-version>+isar"

+DEB_BUILD_OPTIONS += "${@ 'nocheck' if d.getVar('ISAR_CROSS_COMPILE') == '1' else '' }"
+
do_prepare_build() {
deb_add_changelog
# this seems to be a build dep missing in the upstream control file
@@ -22,9 +24,3 @@ do_prepare_build() {
sed -i -e 's/Build-Depends:/Build-Depends: texinfo,/g' ${S}/debian/control
fi
}
-
-dpkg_runbuild_prepend() {
- if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
- export DEB_BUILD_OPTIONS="nocheck"
- fi
-}
diff --git a/meta/recipes-bsp/libubootenv/libubootenv_0.3-3.bb b/meta/recipes-bsp/libubootenv/libubootenv_0.3-3.bb
index 68a55003..9af01aa7 100644
--- a/meta/recipes-bsp/libubootenv/libubootenv_0.3-3.bb
+++ b/meta/recipes-bsp/libubootenv/libubootenv_0.3-3.bb
@@ -19,7 +19,4 @@ SRCREV = "a1a3504e5cda1883928a8747a0bedc56afff6910"

S = "${WORKDIR}/git"

-
-dpkg_runbuild_prepend() {
- export DEB_BUILD_OPTIONS="nocheck"
-}
+DEB_BUILD_OPTIONS += "nocheck"
--
2.30.2

[vijai kumar]

Like said in the previous thread. If this is eventually going to be
dropped, a warning to the user would be needed so that he could notice
and migrate to the DEB_BUILD_PROFILES/OPTIONS variable instead of a
manual export.

Thanks,
Vijai Kumar K

[Felix Moessbauer]

Changes since v3:

Set variables via export shell directive instead of setting via os.environ.
This change is required as os.environ sets the environment of the whole bitbake run,
hence affects other tasks as well.
As we have to use export, there is no way for us to check if the variable is already exported.
Also checking the current env-vars in isar_deb_build_profiles does not help as the bb
function is called prior to the invocation of the shell function that does the compiling.
I added a statement to the RECIPE-API-CHANGELOG about that.

Changes since v2:

- only provide DEB_BUILD_PROFILES variable (no _CROSS variable)
- add "cross" directly to env variable when cross-compiling
- add DEB_BUILD_OPTIONS bitbake variable
(profiles like "nocheck" must also add "nocheck" to DEB_BUILD_OPTIONS)
- update API changelog
- use DEB_BUILD_OPTIONS in hello.bb example

Changes since v1:

- fix erronous code removal reported by Vijai
- only use value of DEB_BUILD_PROFILES_CROSS when cross-compiling (no combination of values)
- improve API changelog docs
- rebased onto next
- use DEB_BUILD_PROFILES infrastructure in linux-custom.inc

Felix Moessbauer (3):
add support for debian build profiles and options
refactor linux-custom.inc to use ISAR's DEB_BUILD_PROFILES support
Use DEB_BUILD_OPTIONS bb variable in hello.bb example

RECIPE-API-CHANGELOG.md | 8 ++++++++
meta-isar/recipes-app/hello/hello.bb | 8 ++------
meta/classes/dpkg-base.bbclass | 19 +++++++++++++++++++
meta/classes/dpkg.bbclass | 7 ++++++-
.../libubootenv/libubootenv_0.3-3.bb | 5 +----
meta/recipes-kernel/linux/linux-custom.inc | 13 ++++++++-----
6 files changed, 44 insertions(+), 16 deletions(-)

--
2.30.2

[Felix Moessbauer]

This patch adds the bitbake variables DEB_BUILD_PROFILES and DEB_BUILD_OPTIONS.
These are used to define the respective environment variables.

When cross-compiling, "cross" is added to the DEB_BUILD_PROFILES environment variable.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

RECIPE-API-CHANGELOG.md | 8 ++++++++

meta/classes/dpkg-base.bbclass | 19 +++++++++++++++++++
meta/classes/dpkg.bbclass | 7 ++++++-

3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 3bbb42a9..39a1616b 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -314,3 +314,11 @@ The "NAME" used to be rather static and the TAG was always "latest", now the val

### Renamed variable CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS

The meaning remains the same, just the name changed.
+### Introduce debian build profiles
+

+All recipes that inherit from dpkg and dpkg-base can utilize the variables `DEB_BUILD_PROFILES` and `DEB_BUILD_OPTIONS`.

+The bitbake variable defines the respective environment variable which is available in `do_install_builddeps` and `do_dpkg_build`.
+When cross compiling, `cross` is added to the `DEB_BUILD_PROFILES` environment variable.

+Please note, that manually exported versions of the variables are overwritten.

+
+For a list of well-known Debian build profiles and common practices, we refer to Debian's BuildProfileSpec.

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index cb5ce4a9..de35bcd5 100644

--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -13,6 +13,8 @@ inherit deb-dl-dir
DEPENDS ?= ""

DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
+DEB_BUILD_PROFILES ?= ""
+DEB_BUILD_OPTIONS ?= ""

python do_adjust_git() {
import subprocess

@@ -201,6 +203,22 @@ dpkg_runbuild() {

die "This should never be called, overwrite it in your derived class"
}

+def isar_deb_build_profiles(d):
+ deb_build_profiles = d.getVar('DEB_BUILD_PROFILES', True)

+ if d.getVar("ISAR_CROSS_COMPILE") == "1":
+ deb_build_profiles += ' cross'

+ return deb_build_profiles.strip()
+
+def isar_deb_build_options(d):
+ deb_build_options = d.getVar('DEB_BUILD_OPTIONS', True)
+ return deb_build_options.strip()
+
+# use with caution: might contaminate multiple tasks

+def isar_export_build_settings(d):
+ import os

+ os['DEB_BUILD_OPTIONS'] = isar_deb_build_options(d)
+ os['DEB_BUILD_PROFILES'] = isar_deb_build_profiles(d)
+
python do_dpkg_build() {

lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
shared=True)

@@ -279,6 +297,7 @@ python do_devshell() {

isar_export_proxies(d)
isar_export_ccache(d)
+ isar_export_build_settings(d)

buildchroot = d.getVar('BUILDCHROOT_DIR')
pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS'))
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass

index 27fe84f4..320102ba 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -9,10 +9,13 @@ PACKAGE_ARCH ?= "${DISTRO_ARCH}"

do_install_builddeps() {
dpkg_do_mounts
E="${@ isar_export_proxies(d)}"

+ export DEB_BUILD_OPTIONS="${@ isar_deb_build_options(d)}"
+ export DEB_BUILD_PROFILES="${@ isar_deb_build_profiles(d)}"

distro="${DISTRO}"
if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
- distro="${HOST_DISTRO}"
+ distro="${HOST_DISTRO}"
fi
+
deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
${PP}/${PPS} ${PACKAGE_ARCH} --download-only

@@ -33,6 +36,8 @@ addtask devshell after do_install_builddeps
dpkg_runbuild() {
E="${@ isar_export_proxies(d)}"
E="${@ isar_export_ccache(d)}"
+ export DEB_BUILD_OPTIONS="${@ isar_deb_build_options(d)}"
+ export DEB_BUILD_PROFILES="${@ isar_deb_build_profiles(d)}"
export PARALLEL_MAKE="${PARALLEL_MAKE}"
sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
/isar/build.sh ${PP}/${PPS} ${PACKAGE_ARCH}
--
2.30.2

[Felix Moessbauer]

This patch replaces the manual setup of the DEB_BUILD_PROFILES
environment variable in the linux-custom.inc recipe.
Instead, the recently introduced DEB_BUILD_PROFILES infrastructure
of ISAR is used.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

[Felix Moessbauer]

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

[Felix Moessbauer]

Changes since v4:

- fix devshell (glitch introduced in v3)

meta/classes/dpkg-base.bbclass | 19 +++++++++++++++++++
meta/classes/dpkg.bbclass | 7 ++++++-

[Felix Moessbauer]

This patch adds the bitbake variables DEB_BUILD_PROFILES and DEB_BUILD_OPTIONS.
These are used to define the respective environment variables.

When cross-compiling, "cross" is added to the DEB_BUILD_PROFILES environment variable.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

RECIPE-API-CHANGELOG.md | 8 ++++++++

meta/classes/dpkg-base.bbclass | 19 +++++++++++++++++++
meta/classes/dpkg.bbclass | 7 ++++++-

3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 3bbb42a9..39a1616b 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -314,3 +314,11 @@ The "NAME" used to be rather static and the TAG was always "latest", now the val
### Renamed variable CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS

The meaning remains the same, just the name changed.
+### Introduce debian build profiles
+
+All recipes that inherit from dpkg and dpkg-base can utilize the variables `DEB_BUILD_PROFILES` and `DEB_BUILD_OPTIONS`.
+The bitbake variable defines the respective environment variable which is available in `do_install_builddeps` and `do_dpkg_build`.
+When cross compiling, `cross` is added to the `DEB_BUILD_PROFILES` environment variable.
+Please note, that manually exported versions of the variables are overwritten.
+
+For a list of well-known Debian build profiles and common practices, we refer to Debian's BuildProfileSpec.
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass

index cb5ce4a9..202cc115 100644

--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -13,6 +13,8 @@ inherit deb-dl-dir
DEPENDS ?= ""

DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
+DEB_BUILD_PROFILES ?= ""
+DEB_BUILD_OPTIONS ?= ""

python do_adjust_git() {
import subprocess
@@ -201,6 +203,22 @@ dpkg_runbuild() {
die "This should never be called, overwrite it in your derived class"
}

+def isar_deb_build_profiles(d):
+ deb_build_profiles = d.getVar('DEB_BUILD_PROFILES', True)
+ if d.getVar("ISAR_CROSS_COMPILE") == "1":
+ deb_build_profiles += ' cross'
+ return deb_build_profiles.strip()
+
+def isar_deb_build_options(d):
+ deb_build_options = d.getVar('DEB_BUILD_OPTIONS', True)
+ return deb_build_options.strip()
+
+# use with caution: might contaminate multiple tasks
+def isar_export_build_settings(d):
+ import os

+ os.environ['DEB_BUILD_OPTIONS'] = isar_deb_build_options(d)
+ os.environ['DEB_BUILD_PROFILES'] = isar_deb_build_profiles(d)

+
python do_dpkg_build() {
lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
shared=True)
@@ -279,6 +297,7 @@ python do_devshell() {

isar_export_proxies(d)
isar_export_ccache(d)
+ isar_export_build_settings(d)

buildchroot = d.getVar('BUILDCHROOT_DIR')
pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS'))
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 27fe84f4..320102ba 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -9,10 +9,13 @@ PACKAGE_ARCH ?= "${DISTRO_ARCH}"
do_install_builddeps() {
dpkg_do_mounts
E="${@ isar_export_proxies(d)}"
+ export DEB_BUILD_OPTIONS="${@ isar_deb_build_options(d)}"
+ export DEB_BUILD_PROFILES="${@ isar_deb_build_profiles(d)}"
distro="${DISTRO}"

if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then

[Felix Moessbauer]

This patch replaces the manual setup of the DEB_BUILD_PROFILES
environment variable in the linux-custom.inc recipe.
Instead, the recently introduced DEB_BUILD_PROFILES infrastructure
of ISAR is used.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

[Felix Moessbauer]

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

[Jan Kiszka]

After local experiments, it looks like this patch must be folded into
patch 1 in order to avoid that linux-libc-dev gets built accidentally
when it shouldn't.

[Moessbauer, Felix]

> -----Original Message-----
> From: Kiszka, Jan (T CED) <jan.k...@siemens.com>
> Sent: Thursday, January 27, 2022 10:29 AM
> To: Moessbauer, Felix (T CED SES-DE) <felix.mo...@siemens.com>; isar-
> us...@googlegroups.com
> Cc: vijaikumar....@gmail.com
> Subject: Re: [PATCH v5 2/3] refactor linux-custom.inc to use ISAR's
> DEB_BUILD_PROFILES support
>

I just sent out a v6 with all commits squashed.

While we are at it:
Without sbuilder, removing packages due to conflicting build dependencies is a recipe for disaster.
The do_install_builddeps and do_dpkg_build steps are not atomic, so another package might remove build deps in between the two tasks.

We should at least scan the do_install_builddeps log and search for "[1-9]+ to remove" and issue a warning if we find such a line.

But don't know when sbuilder will be ready, and if we should put time into this check.
Opinions?

Felix

[Jan Kiszka]

They are atomic (do_install_builddeps[lockfiles] vs. read-lock in
do_dpkg_build), don't worry about that.

But, yes, this can cause troubles if undesired packages are being pushed
into the buildchroot and/or dependencies were not properly expressed.

>
> We should at least scan the do_install_builddeps log and search for "[1-9]+ to remove" and issue a warning if we find such a line.

On partial rebuild, you do want to see those messages.

Jan