[PATCH] linux-custom: support vendor directories via HEADERS_INSTALL_EXTRA

[Cedric Hombourger]

Some vendor kernels come with additional directories to be included in the
linux-headers package in order to support building of their out-of-tree
drivers. HEADERS_INSTALL_EXTRA may be set to a list of directories relative
to ${S} in any kernel recipes that includes linux-custom.inc

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---
RECIPE-API-CHANGELOG.md | 12 ++++++++++++
.../linux/files/debian/isar/install.tmpl | 6 ++++++
meta/recipes-kernel/linux/linux-custom.inc | 12 ++++++++++++
3 files changed, 30 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 65cd98b3..21c558d2 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -634,3 +634,15 @@ into kernel kbuild package.

When downloading from debian snapshot mirrors, rate limits might apply.
To limit the amount of parallel fetching to n kB / s, you can set `ISAR_APT_DL_LIMIT="<n>`.
+
+### Custom directories in vendor kernels
+
+Some vendor kernels come with additional directories to be included in the
+linux-headers package in order to support building of their out-of-tree
+drivers. `HEADERS_INSTALL_EXTRA` may be set to a list of directories relative
+to ${S} in any kernel recipes that includes `linux-custom.inc`. A l4t kernel
+recipe would use the following setting:
+
+```
+HEADERS_INSTALL_EXTRA += "nvidia"
+```
diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
index 00011517..0a7bdd66 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
@@ -193,6 +193,12 @@ kernel_headers() {
(cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files}
(cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files}

+ if [ -n "${HEADERS_INSTALL_EXTRA}" ]; then
+ for d in ${HEADERS_INSTALL_EXTRA}; do
+ (cd ${S}; find ${d} -type f) >>${src_hdr_files}
+ done
+ fi
+
if [ -n "${CONFIG_MODULES}" ]; then
echo Module.symvers >> ${obj_hdr_files}
fi
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index 6aa70fd3..7e312b75 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -48,6 +48,8 @@ LINUX_VERSION_EXTENSION ?= ""

KERNEL_DEFCONFIG ??= ""

+HEADERS_INSTALL_EXTRA ??= ""
+
# Add our template meta-data to the sources
FILESPATH:append := ":${FILE_DIRNAME}/files"
SRC_URI += "file://debian"
@@ -84,6 +86,7 @@ TEMPLATE_VARS += " \
KAFLAGS \
DISTRIBUTOR \
KERNEL_EXTRA_BUILDARGS \
+ HEADERS_INSTALL_EXTRA \
"

inherit dpkg
@@ -224,6 +227,15 @@ do_prepare_build:prepend() {

# produce a changelog for our kernel build
deb_add_changelog
+
+ # make sure user-specified directories do exist in ${S}
+ if [ -n "${HEADERS_INSTALL_EXTRA}" ]; then
+ for d in ${HEADERS_INSTALL_EXTRA}; do
+ [ -d ${S}/${d} ] || {
+ bbfatal "HEADERS_INSTALL_EXTRA: '${d}' not found in \${S}!"
+ }
+ done
+ fi
}

# build directory for our "full" kernel build
--
2.39.2

[Jan Kiszka]

On 10.08.24 23:58, 'Cedric Hombourger' via isar-users wrote:
> Some vendor kernels come with additional directories to be included in the
> linux-headers package in order to support building of their out-of-tree
> drivers. HEADERS_INSTALL_EXTRA may be set to a list of directories relative
> to ${S} in any kernel recipes that includes linux-custom.inc
>

Does this affect more than the legacy Nvidia Jetpack 5 kernel? I worked
with a lot of vendor kernels but didn't have the need elsewhere so far.

Note that we also didn't merge (or even propose?) the ccache
optimization for that horrible and now (JP6) removed kernel overlay of
Nvidia. And for the header topic, we resolve it in the downstream recipe
- a 3-liner by now.

Jan

Siemens AG, Technology
Linux Expert Center

[cedric.h...@siemens.com]

On Sun, 2024-08-11 at 10:10 +0200, Jan Kiszka wrote:
> On 10.08.24 23:58, 'Cedric Hombourger' via isar-users wrote:
> > Some vendor kernels come with additional directories to be included
> > in the
> > linux-headers package in order to support building of their out-of-
> > tree
> > drivers. HEADERS_INSTALL_EXTRA may be set to a list of directories
> > relative
> > to ${S} in any kernel recipes that includes linux-custom.inc
> >
>
> Does this affect more than the legacy Nvidia Jetpack 5 kernel? I
> worked
> with a lot of vendor kernels but didn't have the need elsewhere so
> far.

Same story with QCOM kernel. While for Nvidia, Claudius had chosen to
ship a copy of Isar's install.tmpl, for QCOM I had patched debian/rules
to ship additional files into linux-headers. Neither approaches are IMO
satisfactory.

Are there concerns about adding a new user-visible variable in linux-
custom? Or was the question about this being proposed to address a
single corner-case or multiple vendor kernels?

Cedric Hombourger
Siemens AG
www.siemens.com

[Jan Kiszka]

On 11.08.24 10:35, Hombourger, Cedric (DI CTO FDS CES LX) wrote:
> On Sun, 2024-08-11 at 10:10 +0200, Jan Kiszka wrote:
>> On 10.08.24 23:58, 'Cedric Hombourger' via isar-users wrote:
>>> Some vendor kernels come with additional directories to be included
>>> in the
>>> linux-headers package in order to support building of their out-of-
>>> tree
>>> drivers. HEADERS_INSTALL_EXTRA may be set to a list of directories
>>> relative
>>> to ${S} in any kernel recipes that includes linux-custom.inc
>>>
>>
>> Does this affect more than the legacy Nvidia Jetpack 5 kernel? I
>> worked
>> with a lot of vendor kernels but didn't have the need elsewhere so
>> far.
>
> Same story with QCOM kernel. While for Nvidia, Claudius had chosen to
> ship a copy of Isar's install.tmpl, for QCOM I had patched debian/rules
> to ship additional files into linux-headers. Neither approaches are IMO
> satisfactory.
>
> Are there concerns about adding a new user-visible variable in linux-
> custom? Or was the question about this being proposed to address a
> single corner-case or multiple vendor kernels?

The latter, because the Nvidia case is in phase-out now. If Qualcomm
needs longer to understand their mistakes, we apparently need this.

Is ccache fine with the QCOM kernel?

Jan

[MOESSBAUER, Felix]

On Sun, 2024-08-11 at 10:10 +0200, 'Jan Kiszka' via isar-users wrote:
> On 10.08.24 23:58, 'Cedric Hombourger' via isar-users wrote:
> > Some vendor kernels come with additional directories to be included
> > in the
> > linux-headers package in order to support building of their out-of-
> > tree
> > drivers. HEADERS_INSTALL_EXTRA may be set to a list of directories
> > relative
> > to ${S} in any kernel recipes that includes linux-custom.inc
> >
>
> Does this affect more than the legacy Nvidia Jetpack 5 kernel? I
> worked
> with a lot of vendor kernels but didn't have the need elsewhere so
> far.
>
> Note that we also didn't merge (or even propose?) the ccache
> optimization for that horrible and now (JP6) removed kernel overlay
> of

I proposed this patch in [1], but due to a lack of downstream consumers
(except for JP5) and the questionable issue it is solving, it was never
merged.

[1] https://groups.google.com/g/isar-users/c/dQqxKdA7d-4/m/xuFhLj7HEgAJ

Felix

[cedric.h...@siemens.com]

On Mon, 2024-08-12 at 11:41 +0000, Moessbauer, Felix (T CED OES-DE)
wrote:

> On Sun, 2024-08-11 at 10:10 +0200, 'Jan Kiszka' via isar-users wrote:
> > On 10.08.24 23:58, 'Cedric Hombourger' via isar-users wrote:
> > > Some vendor kernels come with additional directories to be
> > > included
> > > in the
> > > linux-headers package in order to support building of their out-
> > > of-
> > > tree
> > > drivers. HEADERS_INSTALL_EXTRA may be set to a list of
> > > directories
> > > relative
> > > to ${S} in any kernel recipes that includes linux-custom.inc
> > >
> >
> > Does this affect more than the legacy Nvidia Jetpack 5 kernel? I
> > worked
> > with a lot of vendor kernels but didn't have the need elsewhere so
> > far.
> >
> > Note that we also didn't merge (or even propose?) the ccache
> > optimization for that horrible and now (JP6) removed kernel overlay
> > of
>
> I proposed this patch in [1], but due to a lack of downstream
> consumers
> (except for JP5) and the questionable issue it is solving, it was
> never
> merged.

Thanks for sharing. I had a chance to look at our JP5 layer earlier
today (was away for the last two weeks) and I now see that it uses sed
to insert the "nvidia/include" directory to the list of directories to
ship with linux-headers.

No doubt that it works and is simple to maintain.

My concern is that it introduces a hidden contract between our kernel
recipe and linux-custom. That said chances are that changes to linux-
custom will cause a build failure for out-of-tree kernel modules if sed
did not manage to augment the list of directories to ship. We are
probably fine with that approach but I generally do not like such
hidden contracts. The other downside is that other layers may opt to go
with a different approach and I was hoping that we could get Isar to
offer a unified way to address this use-case even if the number of
users may be limited (and will hopefully remain limited as we would not
want vendor kernels to "pollute" kernel trees with extra folders.

Cedric Hombourger
Siemens AG
http://www.siemens.com/

[Jan Kiszka]

I think Felix was referring to the ccache-related changes. Those are
orthogonal to the header installation.

Again my question: does the QCOM kernel have ccache issues as well?
Should we reconsider Felix' patch?

Jan

[Uladzimir Bely]

On Sat, 2024-08-10 at 23:58 +0200, 'Cedric Hombourger' via isar-users
wrote:

Hello all.

The patch is checked in CI and it passes OK, but it's still not clear
from a discussion if it's worth applying. Or should this new feature
leave on a downstrem side?

--
Best regards,
Uladzimir.

[Jan Kiszka]

I said in my reply that "we apparently need this", now that we have two
downstream kernels with this pain.

We then had some unfinished (IIRC) follow-up discussion around ccache,
but that should not delay this change.

[Uladzimir Bely]

OK, got you. Thanks for the answer.

--
Best regards,
Uladzimir.

[Uladzimir Bely]

On Sat, 2024-08-10 at 23:58 +0200, 'Cedric Hombourger' via isar-users
wrote:

Applied to next, thanks.

--
Best regards,
Uladzimir.