[PATCH] Introduce SCRIPTSDIR variable

[Vijai Kumar K]

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is mostly used with wic implemetation for the scripts
directory. Introduce SCRIPTSDIR to satisfy wic.

When ISARROOT equivalent is needed, derive it from SCRIPTSDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>
---
isar-init-build-env | 1 +
meta/classes/image-sdk-extension.bbclass | 2 +-
meta/classes/wic-img.bbclass | 9 +++++----
scripts/ci_build.sh | 12 +++++++-----
scripts/isar-buildenv-internal | 5 ++++-
5 files changed, 18 insertions(+), 11 deletions(-)

diff --git a/isar-init-build-env b/isar-init-build-env
index b08bb59..85fb82f 100755
--- a/isar-init-build-env
+++ b/isar-init-build-env
@@ -55,5 +55,6 @@ export ISARROOT
unset ISARROOT
return 1
}
+unset ISARROOT

[ -z "$BUILDDIR" ] || cd "$BUILDDIR"
diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index b9f2cf4..b91fdd6 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -19,7 +19,7 @@ do_populate_sdk() {
sudo rm -f ${SDKCHROOT_DIR}/chroot-setup.sh ${SDKCHROOT_DIR}/configscript.sh

# Copy mount_chroot.sh for convenience
- sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
+ sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}

# Create SDK archive
cd -P ${SDKCHROOT_DIR}/..
diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 7d30e45..e715f56 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -86,7 +86,7 @@ STAGING_DATADIR ?= "/usr/lib/"
STAGING_LIBDIR ?= "/usr/lib/"
STAGING_DIR ?= "${TMPDIR}"
IMAGE_BASENAME ?= "${PN}-${DISTRO}"
-FAKEROOTCMD ?= "${ISARROOT}/scripts/wic_fakeroot"
+FAKEROOTCMD ?= "${SCRIPTSDIR}/wic_fakeroot"
RECIPE_SYSROOT_NATIVE ?= "/"
BUILDCHROOT_DIR = "${BUILDCHROOT_TARGET_DIR}"

@@ -132,7 +132,7 @@ do_wic_image() {
buildchroot_do_mounts
sudo -s <<'EOSUDO'
( flock 9
- for dir in ${BBLAYERS} ${STAGING_DIR} ${ISARROOT}/scripts; do
+ for dir in ${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR}; do
mkdir -p ${BUILDCHROOT_DIR}/$dir
if ! mountpoint ${BUILDCHROOT_DIR}/$dir >/dev/null 2>&1; then
mount --bind --make-private $dir ${BUILDCHROOT_DIR}/$dir
@@ -148,12 +148,13 @@ EOSUDO
WICTMP=$(cd ${BUILDCHROOT_DIR}; mktemp -d -p tmp)

sudo -E chroot ${BUILDCHROOT_DIR} \
- ${ISARROOT}/scripts/wic create ${WKS_FULL_PATH} \
+ ${SCRIPTSDIR}/wic create ${WKS_FULL_PATH} \
--vars "${STAGING_DIR}/${MACHINE}/imgdata/" \
-o /$WICTMP/${IMAGE_FULLNAME}.wic/ \
--bmap \
-e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
- sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
+ ROOTDIR=`dirname ${SCRIPTSDIR}`
+ sudo chown -R $(stat -c "%U" ${ROOTDIR}) ${ROOTDIR}/meta ${ROOTDIR}/meta-isar ${SCRIPTSDIR} || true
WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 7cd5a2f..727b4a9 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -133,13 +133,15 @@ if [ ! -d "$BUILD_DIR" ]; then
fi
source isar-init-build-env "$BUILD_DIR"

+ROOTDIR=`dirname ${SCRIPTSDIR}`
+
if [ -n "$CROSS_BUILD" ]; then
sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf
fi

if [ -n "$REPRO_BUILD" ]; then
- ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
- ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
+ ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ROOTDIR/testsuite/base-apt/test_pub.key"
+ ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ROOTDIR/testsuite/base-apt/test_priv.key"
export GNUPGHOME=$(mktemp -d)
gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE

@@ -188,9 +190,9 @@ if [ -z "$FAST_BUILD" ]; then
bitbake $BB_ARGS $TARGETS_SET
fi

-cp -a "${ISARROOT}/meta/classes/dpkg-base.bbclass" "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup"
-echo -e "do_fetch_append() {\n\n}" >> "${ISARROOT}/meta/classes/dpkg-base.bbclass"
+cp -a "${ROOTDIR}/meta/classes/dpkg-base.bbclass" "${ROOTDIR}/meta/classes/dpkg-base.bbclass.ci-backup"
+echo -e "do_fetch_append() {\n\n}" >> "${ROOTDIR}/meta/classes/dpkg-base.bbclass"

bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base

-mv "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup" "${ISARROOT}/meta/classes/dpkg-base.bbclass"
+mv "${ROOTDIR}/meta/classes/dpkg-base.bbclass.ci-backup" "${ROOTDIR}/meta/classes/dpkg-base.bbclass"
diff --git a/scripts/isar-buildenv-internal b/scripts/isar-buildenv-internal
index 72a91ab..a91ad7a 100755
--- a/scripts/isar-buildenv-internal
+++ b/scripts/isar-buildenv-internal
@@ -74,5 +74,8 @@ export PATH
BBPATH="${BUILDDIR}"
export BBPATH

-BB_ENV_EXTRAWHITE="BUILDDIR ISARROOT http_proxy https_proxy ftp_proxy no_proxy GNUPGHOME"
+SCRIPTSDIR="${ISARROOT}/scripts"
+export SCRIPTSDIR
+
+BB_ENV_EXTRAWHITE="BUILDDIR SCRIPTSDIR http_proxy https_proxy ftp_proxy no_proxy GNUPGHOME"
export BB_ENV_EXTRAWHITE
--
2.17.1

[Henning Schild]

The unspoken rule (not yet in Isar) would be to use kas and kas-docker
to get an always reproducible build with a known environment.

Looking at the patch you do replace the env variable ISARROOT with
another "global" SCRIPTSDIR. Could you go into detail who unsets that
to solve the issue with global env variables?
I did not see an "unset" so i assume it must be in contrib (oe or
bitbake) code.

Henning

[Vijai Kumar K]

On Tue, Jan 28, 2020 at 10:43:47PM +0100, Henning Schild wrote:
> The unspoken rule (not yet in Isar) would be to use kas and kas-docker
> to get an always reproducible build with a known environment.
>

Yes. I agree that kas effectively handles these scenarios. But, we
should take into consideration the non-kas based builds, as we do now,
and its repercussions. Atlast kas is just another tool outside ISAR and
people might use it or skip it :)

> Looking at the patch you do replace the env variable ISARROOT with
> another "global" SCRIPTSDIR. Could you go into detail who unsets that
> to solve the issue with global env variables?

Sorry. I am not sure what is the issue you are referring to. I don't
think it needs any unset. It is derived from ISARROOT and set each time
the setup script is sourced. Somewhat like your BUILDDIR. If you see,
the problem with ISARROOT is it is set only when it is empty. This
behaviour derived from OE, as pointed out by Jan in [1], is to have
provision to have a caller/wrapper script to pass in ISARROOT to the
isar-init-build-env script.

Considering all these it made sense either to duplicate ISARROOT
(something like ISARHOME) and unset the former. But, on going through
the history, the ISARROOT was orignally exported to locate the most
commonly used scripts directory, for wic and other purposes. So it made
sense to address that and derive anything which might need the actual
ISARROOT equivalent from it.

[1] https://groups.google.com/forum/#!msg/isar-users/jklvMqAT_uA/L5BQY8xXBwAJ

Thanks,
Vijai Kumar K

[vijai kumar]

Are there any more review comments on this?

Thanks,

Vijai Kumar K

> > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>

[Baurzhan Ismagulov]

Hello Vijai Kumar,

On Wed, Jan 29, 2020 at 01:05:20AM +0530, Vijai Kumar K wrote:
> ISARROOT is mostly used with wic implemetation for the scripts
> directory. Introduce SCRIPTSDIR to satisfy wic.
>
> When ISARROOT equivalent is needed, derive it from SCRIPTSDIR.

Thanks for pursuing this annoying issue. I like the concept of having a special
variable for the directory. The advantage is that the code doesn't rely on a
specific directory structure.

Now the patch fails to apply upon d90c2ab "wic-img: allow wic to find bitbake
binary", submitted shortly before yours. Looking at both together, I realize
that deriving the ISARROOT equivalent from SCRIPTSDIR makes it rely on the
directory structure again.

One possibilty would be to introduce both SCRIPTSDIR and the ISARROOT
equivalent (calling it e.g. ISARDIR). What do you think?

With kind regards,
Baurzhan.

[vijai kumar]

On Sun, Apr 5, 2020 at 10:19 PM Baurzhan Ismagulov <i...@radix50.net> wrote:
>
> Hello Vijai Kumar,
>
> On Wed, Jan 29, 2020 at 01:05:20AM +0530, Vijai Kumar K wrote:
> > ISARROOT is mostly used with wic implemetation for the scripts
> > directory. Introduce SCRIPTSDIR to satisfy wic.
> >
> > When ISARROOT equivalent is needed, derive it from SCRIPTSDIR.
>
> Thanks for pursuing this annoying issue. I like the concept of having a special
> variable for the directory. The advantage is that the code doesn't rely on a
> specific directory structure.
>
> Now the patch fails to apply upon d90c2ab "wic-img: allow wic to find bitbake
> binary", submitted shortly before yours. Looking at both together, I realize
> that deriving the ISARROOT equivalent from SCRIPTSDIR makes it rely on the
> directory structure again.

Yes. Indeed. Please see comments below.

>
> One possibilty would be to introduce both SCRIPTSDIR and the ISARROOT
> equivalent (calling it e.g. ISARDIR). What do you think?

Hi Baurzhan,

From the current ISAR next, there are atleast 4 different uses cases
for ISARROOT,

1. ISARROOT to find scripts directory for wic.
2. ISARROOT to find bitbake/bin directory for wic.
3. ISARROOT used as part of chown brought in by
f13b2bf71dd841eaebbbcd04f14a7fbcb2649572 to fix __pycache__
permission.
4. ISARROOT in ci-build.sh

The first one could be addressed by a dedicated SCRIPTSDIR variable.
Second one could also be a dedicated variable? like BITBAKEDIR??
Third one I am not sure if we could use any other location for stat.
Since in my build __pycache_ is created only in scripts/lib/wic/
folder. In that case maybe we could stat LAYERDIR_core instead of
ISARROOT.
Fourth one can be replaced with a combination of getting
LAYERDIR_{core/isar} from bitbake -e and probably a new variable for
testsuite directory or any other way to get the gpg-keys path.

I am just thinking if we should straight away adapt an ISARROOT
equivalent (ISARDIR) and have developers use those to to derive any
future
paths, or should we have dedicated variables to limit developers to
certain paths which are absolutely needed(SCRIPTSDIR, TESTSUITEDIR,
BITBAKEDIR etc).

I would opt the latter, as it would avoid unnecessary duplicate
definitions for the same path. For ex. path to meta can be
${ISARROOT}/meta or could just be ${LAYERDIR_core}. Also this means
that we would establish the directory relationship at setup time,
easier to change or override in future without affecting large part of
the codebase.

Thanks,
Vijai Kumar K


>
> With kind regards,
> Baurzhan.
>
> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20200405164943.mccmlrje526qxyw3%40yssyq.m.ilbers.de.

[vijai kumar]

I will send a V2 with this approach for review. It can be further

discussed on top of that I guess.

 

Thanks,

Vijai Kumar K

I would opt the latter, as it would avoid unnecessary duplicate
definitions for the same path. For ex. path to meta can be
${ISARROOT}/meta or could just be ${LAYERDIR_core}. Also this means
that we would establish the directory relationship at setup time,
easier to change or override in future without affecting large part of
the codebase. 

Thanks,
Vijai Kumar K


>
> With kind regards,
> Baurzhan.
>
> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.

[Vijai Kumar K]

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is used mostly to derive the location of bitbake, scripts,
testsuite, meta and meta-isar directories.

Layers like meta, meta-isar already have their paths in LAYERDIR_core
and LAYERDIR_isar which can be used. For other directories introduce
dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>

---
isar-init-build-env | 1 +
meta/classes/image-sdk-extension.bbclass | 2 +-

meta/classes/wic-img.bbclass | 31 ++++++++++++------------
scripts/ci_build.sh | 12 +++++----
scripts/isar-buildenv-internal | 7 ++++--
5 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/isar-init-build-env b/isar-init-build-env
index b08bb59..85fb82f 100755
--- a/isar-init-build-env
+++ b/isar-init-build-env
@@ -55,5 +55,6 @@ export ISARROOT
unset ISARROOT
return 1
}
+unset ISARROOT

[ -z "$BUILDDIR" ] || cd "$BUILDDIR"
diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index b9f2cf4..b91fdd6 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -19,7 +19,7 @@ do_populate_sdk() {
sudo rm -f ${SDKCHROOT_DIR}/chroot-setup.sh ${SDKCHROOT_DIR}/configscript.sh

# Copy mount_chroot.sh for convenience
- sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
+ sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}

# Create SDK archive
cd -P ${SDKCHROOT_DIR}/..
diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass

index afab394..1bb1021 100644

--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -86,7 +86,7 @@ STAGING_DATADIR ?= "/usr/lib/"
STAGING_LIBDIR ?= "/usr/lib/"
STAGING_DIR ?= "${TMPDIR}"
IMAGE_BASENAME ?= "${PN}-${DISTRO}"
-FAKEROOTCMD ?= "${ISARROOT}/scripts/wic_fakeroot"
+FAKEROOTCMD ?= "${SCRIPTSDIR}/wic_fakeroot"
RECIPE_SYSROOT_NATIVE ?= "/"
BUILDCHROOT_DIR = "${BUILDCHROOT_TARGET_DIR}"

@@ -132,7 +132,7 @@ do_wic_image() {
buildchroot_do_mounts
sudo -s <<'EOSUDO'
( flock 9

- for dir in ${BBLAYERS} ${STAGING_DIR} ${ISARROOT}/scripts ${ISARROOT}/bitbake; do
+ for dir in ${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}; do

mkdir -p ${BUILDCHROOT_DIR}/$dir
if ! mountpoint ${BUILDCHROOT_DIR}/$dir >/dev/null 2>&1; then
mount --bind --make-private $dir ${BUILDCHROOT_DIR}/$dir

@@ -149,25 +149,26 @@ EOSUDO

sudo -E chroot ${BUILDCHROOT_DIR} \

sh -c ' \
- ISARROOT="$1"
- WKS_FULL_PATH="$2"
- STAGING_DIR="$3"
- MACHINE="$4"
- WICTMP="$5"
- IMAGE_FULLNAME="$6"
- IMAGE_BASENAME="$7"
- shift 7
-
- export PATH="$ISARROOT/bitbake/bin:$PATH"
- "$ISARROOT"/scripts/wic create "$WKS_FULL_PATH" \
+ BITBAKEDIR="$1"
+ SCRIPTSDIR="$2"
+ WKS_FULL_PATH="$3"
+ STAGING_DIR="$4"
+ MACHINE="$5"
+ WICTMP="$6"
+ IMAGE_FULLNAME="$7"
+ IMAGE_BASENAME="$8"
+ shift 8
+
+ export PATH="$BITBAKEDIR/bin:$PATH"
+ "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
--vars "$STAGING_DIR/$MACHINE/imgdata/" \

-o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
--bmap \

-e "$IMAGE_BASENAME" $@' \
- my_script "${ISARROOT}" "${WKS_FULL_PATH}" "${STAGING_DIR}" \
+ my_script "${BITBAKEDIR}" "${SCRIPTSDIR}" "${WKS_FULL_PATH}" "${STAGING_DIR}" \
"${MACHINE}" "${WICTMP}" "${IMAGE_FULLNAME}" "${IMAGE_BASENAME}" \

${WIC_CREATE_EXTRA_ARGS}
- sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true

+ sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true

WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh

index adc403b..d15aba9 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -138,8 +138,8 @@ if [ -n "$CROSS_BUILD" ]; then

fi

if [ -n "$REPRO_BUILD" ]; then
- ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
- ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"

+ ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key"
+ ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key"

export GNUPGHOME=$(mktemp -d)
gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE

@@ -187,9 +187,11 @@ if [ -z "$FAST_BUILD" ]; then

bitbake $BB_ARGS $TARGETS_SET
fi

-cp -a "${ISARROOT}/meta/classes/dpkg-base.bbclass" "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup"
-echo -e "do_fetch_append() {\n\n}" >> "${ISARROOT}/meta/classes/dpkg-base.bbclass"

+eval $(bitbake -e | grep "^LAYERDIR_core=")
+
+cp -a "${LAYERDIR_core}/classes/dpkg-base.bbclass" "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup"
+echo -e "do_fetch_append() {\n\n}" >> "${LAYERDIR_core}/classes/dpkg-base.bbclass"

bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base

-mv "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup" "${ISARROOT}/meta/classes/dpkg-base.bbclass"

+mv "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" "${LAYERDIR_core}/classes/dpkg-base.bbclass"
diff --git a/scripts/isar-buildenv-internal b/scripts/isar-buildenv-internal
index 72a91ab..fe1e65f 100755
--- a/scripts/isar-buildenv-internal
+++ b/scripts/isar-buildenv-internal
@@ -59,8 +59,11 @@ fi
unset BDIR

export BUILDDIR
+export BITBAKEDIR="${ISARROOT}/bitbake"
+export SCRIPTSDIR="${ISARROOT}/scripts"
+export TESTSUITEDIR="${ISARROOT}/testsuite"

-for newpath in "$ISARROOT/bitbake/bin" "$ISARROOT/scripts"; do
+for newpath in "$BITBAKEDIR/bin" "$SCRIPTSDIR"; do
# Remove any existences of $newpath from $PATH
PATH=$(echo $PATH | sed -re "s#(^|:)$newpath(:|$)#\2#g;s#^:##")

@@ -74,5 +77,5 @@ export PATH

BBPATH="${BUILDDIR}"
export BBPATH

-BB_ENV_EXTRAWHITE="BUILDDIR ISARROOT http_proxy https_proxy ftp_proxy no_proxy GNUPGHOME"

+BB_ENV_EXTRAWHITE="BUILDDIR BITBAKEDIR SCRIPTSDIR TESTSUITEDIR http_proxy https_proxy ftp_proxy no_proxy GNUPGHOME"
export BB_ENV_EXTRAWHITE
--
2.17.1

[vijai kumar]

Somewhere somehow the  additional "From:" field from git send email is stripped.

All my configurations are right and I tested with some dummy mailing list as well, this info is not there.

But, while sending I do see that the git-send email clearly says 

From: Vijai Kumar K <Vijaikumar_...@mentor.com>.

I believe I would have to manually enter this information in message body still I figure it out.

Will send a v3.

Thanks,

Vijai Kumar K

On Monday, April 27, 2020 at 2:42:16 PM UTC+5:30, vijai kumar wrote:

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is used mostly to derive the location of bitbake, scripts,
testsuite, meta and meta-isar directories.

Layers like meta, meta-isar already have their paths in LAYERDIR_core
and LAYERDIR_isar which can be used. For other directories introduce
dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>

[Vijai Kumar K]

From: Vijai Kumar K <Vijaikumar_...@mentor.com>

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is used mostly to derive the location of bitbake, scripts,
testsuite, meta and meta-isar directories.

Layers like meta, meta-isar already have their paths in LAYERDIR_core
and LAYERDIR_isar which can be used. For other directories introduce
dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>
---
Changes in v3:
- Send patch using proper send.
Changes in v2:
- Approach using dedicated variables for each commonly used
directories.

[vijai kumar]

Any review comments on this?

Thanks,

Vijai Kumar K

On Monday, April 27, 2020 at 3:59:07 PM UTC+5:30, vijai kumar wrote:

From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is used mostly to derive the location of bitbake, scripts,
testsuite, meta and meta-isar directories.

Layers like meta, meta-isar already have their paths in LAYERDIR_core
and LAYERDIR_isar which can be used. For other directories introduce
dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>

[Baurzhan Ismagulov]

Hello Vijai Kumar,

On Mon, May 25, 2020 at 07:54:57PM -0700, vijai kumar wrote:
> Any review comments on this?

This looks good to me, sorry for the delay. I had held this because it happens
to conflict with the hardlinking series which arrived earlier. However, we've
newly discovered that the latter seems to have problems with its test case, so
I tend to pick the one that could be applied first.

With kind regards,
Baurzhan.

> On Monday, April 27, 2020 at 3:59:07 PM UTC+5:30, vijai kumar wrote:
> >
> > From: Vijai Kumar K <Vijaikumar_...@mentor.com>
> >
> > When switching between two ISAR workspaces in the same shell
> > session, the ISARROOT setting of the previous workspace would be
> > picked up for the new workspace resulting in an incorrect
> > configuration. The user had to manually unset ISARROOT to avoid
> > any issues.
> >
> > As like OEROOT in OE, let us unset ISARROOT at the end of the
> > initialization script. This helps us to avoid the above issue
> > and also preserves support for passing ISARROOT from a caller
> > script as in OE.
> >
> > ISARROOT is used mostly to derive the location of bitbake, scripts,
> > testsuite, meta and meta-isar directories.
> >
> > Layers like meta, meta-isar already have their paths in LAYERDIR_core
> > and LAYERDIR_isar which can be used. For other directories introduce
> > dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.
> >

> > Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>

[vijai kumar]

On Tuesday, May 26, 2020 at 1:21:05 PM UTC+5:30, Baurzhan Ismagulov wrote:

Hello Vijai Kumar,

On Mon, May 25, 2020 at 07:54:57PM -0700, vijai kumar wrote:
> Any review comments on this?

This looks good to me, sorry for the delay. I had held this because it happens
to conflict with the hardlinking series which arrived earlier. However, we've
newly discovered that the latter seems to have problems with its test case, so
I tend to pick the one that could be applied first.

Thanks for the update Baurzhan. Good to know its in pipeline.

BR,

Vijai Kumar K

[Baurzhan Ismagulov]

Hello Vijai Kumar,

thanks for updating the wic series. Could you please also rebase this one?

With kind regards,
Baurzhan.

[vijai kumar]

On Thu, 17 Sep 2020 at 12:42 AM, Baurzhan Ismagulov <i...@radix50.net> wrote:

Hello Vijai Kumar,



thanks for updating the wic series. Could you please also rebase this one?

Sure Baurzhan. I will work on this and send it today.

Thanks,

Vijai Kumar K

--

You received this message because you are subscribed to the Google Groups "isar-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20200916191157.GG16317%40yssyq.m.ilbers.de.

[Vijai Kumar K]

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is used mostly to derive the location of bitbake, scripts,
testsuite, meta and meta-isar directories.

Layers like meta, meta-isar already have their paths in LAYERDIR_core
and LAYERDIR_isar which can be used. For other directories introduce
dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>
---

Hi Baurzhan,

I have rebased against the latest next branch. Once merged. I will rebase
and send the WIC series. I have also added an entry in RECIPE-API-CHANGELOG
to notify that ISARROOT is no longer available in bitbake environment.

Thanks,
Vijai Kumar K

Changes in v4:
- Rebase against latest next
- Added a entry in RECIPE-API-CHANGELOG.

Changes in v3:
- Send patch using proper send.
Changes in v2:
- Approach using dedicated variables for each commonly used
directories.

RECIPE-API-CHANGELOG.md | 6 +++++

isar-init-build-env | 1 +
meta/classes/image-sdk-extension.bbclass | 2 +-
meta/classes/wic-img.bbclass | 31 ++++++++++++------------

scripts/ci_build.sh | 17 +++++++------
scripts/isar-buildenv-internal | 7 ++++--
6 files changed, 38 insertions(+), 26 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 1839770..ac9d8d6 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -246,3 +246,9 @@ by setting DEBIAN_BUILD_DEPENDS.

${S} can now be used for checking out sources without being linked implicitly
with ${D} which needs to be filled explicitly in do_install as before.
+
+### Remove ISARROOT from bitbake environment
+
+ISARROOT variable is now removed from the bitbake environment. It is unset
+after the initial setup. It is replaced with dedicated variables like
+BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

index af996d1..82136e3 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -144,8 +144,8 @@ if [ -n "$CROSS_BUILD" ]; then

fi

if [ -n "$REPRO_BUILD" ]; then
- ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
- ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
+ ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key"
+ ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key"
export GNUPGHOME=$(mktemp -d)
gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE

@@ -206,21 +206,22 @@ if [ -z "$FAST_BUILD" ]; then
while [ -e bitbake.sock ]; do sleep 1; done

fi

-cp -a "${ISARROOT}/meta/classes/dpkg-base.bbclass" "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup"
-echo -e "do_fetch_append() {\n\n}" >> "${ISARROOT}/meta/classes/dpkg-base.bbclass"
+eval $(bitbake -e | grep "^LAYERDIR_core=")

+cp -a "${LAYERDIR_core}/meta/classes/dpkg-base.bbclass" "${LAYERDIR_core}/meta/classes/dpkg-base.bbclass.ci-backup"
+echo -e "do_fetch_append() {\n\n}" >> "${LAYERDIR_core}/meta/classes/dpkg-base.bbclass"

bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base

-mv "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup" "${ISARROOT}/meta/classes/dpkg-base.bbclass"

+mv "${LAYERDIR_core}/meta/classes/dpkg-base.bbclass.ci-backup" "${LAYERDIR_core}/meta/classes/dpkg-base.bbclass"

# Test wic --exclude-path
-cp -a "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"
+cp -a "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"
mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img \
${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup
-sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"
+sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"

bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base

-mv "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"
+mv "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"
mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \
${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img

[vijaikumar....@gmail.com]

Ah. Looks like I broke the implementation in this series. It should be LAYERDIR_isar.

Sorry about that. Will send v5.

Thanks,

Vijai Kumar K 

mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img \
${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup
-sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"
+sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"

bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base

-mv "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"
+mv "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_core}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"

Here too.

[Vijai Kumar K]

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is used mostly to derive the location of bitbake, scripts,
testsuite, meta and meta-isar directories.

Layers like meta, meta-isar already have their paths in LAYERDIR_core
and LAYERDIR_isar which can be used. For other directories introduce
dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>
---

Changes in v5:
Fixed a bug introduced by me in the CI build script in last series.
CI build is successful now.
http://ci.isar-build.org:8080/job/isar_vkk_devel/82/

Changes in v4:
- Rebase against latest next
- Added a entry in RECIPE-API-CHANGELOG.
Changes in v3:
- Send patch using proper send.
Changes in v2:
- Approach using dedicated variables for each commonly used
directories.

RECIPE-API-CHANGELOG.md | 6 +++++
isar-init-build-env | 1 +
meta/classes/image-sdk-extension.bbclass | 2 +-
meta/classes/wic-img.bbclass | 31 ++++++++++++------------

scripts/ci_build.sh | 18 ++++++++------
scripts/isar-buildenv-internal | 7 ++++--
6 files changed, 39 insertions(+), 26 deletions(-)

index af996d1..bf224a6 100755

--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -144,8 +144,8 @@ if [ -n "$CROSS_BUILD" ]; then
fi

if [ -n "$REPRO_BUILD" ]; then
- ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
- ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
+ ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key"
+ ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key"
export GNUPGHOME=$(mktemp -d)
gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE

@@ -206,21 +206,23 @@ if [ -z "$FAST_BUILD" ]; then

while [ -e bitbake.sock ]; do sleep 1; done
fi

-cp -a "${ISARROOT}/meta/classes/dpkg-base.bbclass" "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup"
-echo -e "do_fetch_append() {\n\n}" >> "${ISARROOT}/meta/classes/dpkg-base.bbclass"
+eval $(bitbake -e | grep "^LAYERDIR_core=")

+eval $(bitbake -e | grep "^LAYERDIR_isar=")
+cp -a "${LAYERDIR_core}/classes/dpkg-base.bbclass" "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup"
+echo -e "do_fetch_append() {\n\n}" >> "${LAYERDIR_core}/classes/dpkg-base.bbclass"

bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base

-mv "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup" "${ISARROOT}/meta/classes/dpkg-base.bbclass"

+mv "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" "${LAYERDIR_core}/classes/dpkg-base.bbclass"

# Test wic --exclude-path
-cp -a "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"

+cp -a "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"

mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img \
${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup
-sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"

+sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"

bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base

-mv "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"

+mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"

[vijaikumar....@gmail.com]

Can this be merged?

Thanks,

Vijai Kumar K

[Baurzhan Ismagulov]

Hello Vijai Kumar,

On Tue, Sep 22, 2020 at 05:39:08PM +0530, Vijai Kumar K wrote:
> Changes in v5:
> Fixed a bug introduced by me in the CI build script in last series.
> CI build is successful now.
> http://ci.isar-build.org:8080/job/isar_vkk_devel/82/

Thanks for looking at this. I had tried to fix that, but encountered issues and
gave up after two steps. Applied to next.

With kind regards,
Baurzhan.

[vijaikumar....@gmail.com]

Thanks Baurzhan. Now this is in, I will rebase the wic series and send it out. 

Best,

Vijai Kumar K