State scripts not executed from /var/lib/mender

[Ari-Pekka Sihvonen]

Hi,

The scripts that are located in /var/lib/mender are not executed. Syslog however, shows that the trasition has been made,

Syslog:Aug 17 11:49:21 raspberrypi3 mender[1162]: level=info msg="State transition: update-store [Download] -> update-install [ArtifactInstall]" module=mender

Aug 17 11:49:23 raspberrypi3 mender[1162]: level=info msg="State transition: update-install [ArtifactInstall] -> reboot [ArtifactReboot_Enter]" module=menderMender is not running script from /var/lib/mender/scripts for any states changes.
In example above for states [ArtifactInstall] and [ArtifactReboot_Enter].# ll /var/lib/mender/scripts/*Install*
-rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactArtifactInstall_Enter_01
-rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactArtifactInstall_Error_01
-rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactArtifactInstall_Leave_01
-rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactInstall_Enter_01

-rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactInstall_Error_01

-rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactInstall_Leave_01Btw, what is the right name "ArtifactArtifactInstall_Enter_01" or "ArtifactInstall_Enter_01" ?

regards

AP Sihvonen

[Mirza Krak]

On Mon, Aug 20, 2018 at 8:29 AM, Ari-Pekka Sihvonen
<ap.si...@gmail.com> wrote:
> Hi,

HI,

> The scripts that are located in /var/lib/mender are not executed. Syslog
> however, shows that the trasition has been made,
>
> Syslog:Aug 17 11:49:21 raspberrypi3 mender[1162]: level=info msg="State
> transition: update-store [Download] -> update-install [ArtifactInstall]"
> module=mender
> Aug 17 11:49:23 raspberrypi3 mender[1162]: level=info msg="State transition:
> update-install [ArtifactInstall] -> reboot [ArtifactReboot_Enter]"
> module=menderMender is not running script from /var/lib/mender/scripts for
> any states changes.

Do you install the files to /var/lib/mender/scripts? Or has the Mender
client un-packed them here?

> In example above for states [ArtifactInstall] and [ArtifactReboot_Enter].#
> ll /var/lib/mender/scripts/*Install*
> -rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactArtifactInstall_Enter_01
> -rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactArtifactInstall_Error_01
> -rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactArtifactInstall_Leave_01
> -rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactInstall_Enter_01
> -rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactInstall_Error_01
> -rwxr-xr-x 1 root root 597 Aug 17 07:55 ArtifactInstall_Leave_01Btw, what is
> the right name "ArtifactArtifactInstall_Enter_01" or
> "ArtifactInstall_Enter_01" ?

Correct would be, ArtifactInstall_Enter_01.

--
Mirza Krak | Embedded Solutions Architect | https://mender.io

Northern.tech AS | @northerntechHQ

[Ari-Pekka Sihvonen]

/var/lib/mender/scripts directory is populated by YOCTO,

br

AP Sihvonen

--
You received this message because you are subscribed to the Google Groups "Mender List mender.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mender+un...@lists.mender.io.
To post to this group, send email to men...@lists.mender.io.
Visit this group at https://groups.google.com/a/lists.mender.io/group/mender/.

[Mirza Krak]

On Mon, Aug 20, 2018 at 9:33 AM, Ari-Pekka Sihvonen <asi...@iki.fi> wrote:
> /var/lib/mender/scripts directory is populated by YOCTO,

Is it a custom recipe that does this?

The correct way of deploying state-scripts would be using the
"mender-state-scripts" class and installing to
MENDER_STATE_SCRIPTS_DIR variable, check [1]. And note that any
state-script beginning with Artifact should not reside on the
file-system but instead should be embedded in the .mender artifact
file and the Mender client will later unpack them. This is because
Artifact states spans across reboots during updates. If you use
"mender-state-scripts" class and MENDER_STATE_SCRIPTS_DIR it will sort
that out for you.

[1]. https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-demo/recipes-mender/example-state-scripts/example-state-scripts_1.0.bb

[Ari-Pekka Sihvonen]

So instead of this:

inherit mender-state-scripts

FILES_${PN}+="/etc/mender/scripts/*"
FILES_${PN}+="/var/lib/mender/scripts/*"

RDEPENDS_${PN}+="python"

do_install() {
    install -d ${D}/etc/mender/scripts/
    install -d ${D}/var/lib/mender/scripts/
    install -m 755 ${DEV}../scripts/* ${D}/etc/mender/scripts/
    install -m 755 ${DEV}../scripts/* ${D}/var/lib/mender/scripts/
}

  I should:

inherit mender-state-scripts

FILES_${PN}+="/etc/mender/scripts/*"
FILES_${PN}+="/var/lib/mender/scripts/*"

RDEPENDS_${PN}+="python"

do_install() {
    install -d ${D}/etc/mender/scripts/
    install -d ${D}/var/lib/mender/scripts/
    install -m 755 ${DEV}../scripts/* ${MENDER_STATE_SCRIPTS_DIR}

}

[Mirza Krak]

On Mon, Aug 20, 2018 at 9:50 AM, Ari-Pekka Sihvonen <asi...@iki.fi> wrote:
> So instead of this:
>
> inherit mender-state-scripts
>
> FILES_${PN}+="/etc/mender/scripts/*"
> FILES_${PN}+="/var/lib/mender/scripts/*"
>
> RDEPENDS_${PN}+="python"
>
> do_install() {
> install -d ${D}/etc/mender/scripts/
> install -d ${D}/var/lib/mender/scripts/
> install -m 755 ${DEV}../scripts/* ${D}/etc/mender/scripts/
> install -m 755 ${DEV}../scripts/* ${D}/var/lib/mender/scripts/
> }
>
> I should:
> inherit mender-state-scripts
>
> FILES_${PN}+="/etc/mender/scripts/*"
> FILES_${PN}+="/var/lib/mender/scripts/*"
>
> RDEPENDS_${PN}+="python"
>
> do_install() {
> install -d ${D}/etc/mender/scripts/
> install -d ${D}/var/lib/mender/scripts/
> install -m 755 ${DEV}../scripts/* ${MENDER_STATE_SCRIPTS_DIR}
>
> }

You are not providing the full-context, but yeah something like that.

You can even reduce it to:

inherit mender-state-scripts

do_install() {

install -m 755 ${DEV}../scripts/* ${MENDER_STATE_SCRIPTS_DIR}
}

The "mender-state-script" class will put it at the correct location
based on the script name, e.g if the state scripts starts with
Artifact it will put it in the artifact (-s option to mender-artifact
tool) otherwise it will install them to /etc/mender/scripts.

[Ari-Pekka Sihvonen]

Using that in 'bf-state-scripts.bb recipe, the yocto build fails, what could make it fail.

....

Reading package lists...
Building dependency tree...
Package bf-state-scripts is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'bf-state-scripts' has no installation candidate

...

building only the 'bf-state-scripts is successful but incorporating that to the image fails.

br

AP Sihvonen

[Drew Moseley]

Hi AP,

I’m not sure what would cause this particular failure.  It’s possible you need something like

ALLOW_EMPTY_${PN}=“1”

in your recipe but I could not reproduce it locally. How are you including it into your image?

Drew

[Ari-Pekka Sihvonen]

Using that produces following error

ERROR: ParseError at <path>/bf-state-scripts_0.2.5.bb:10: unparsed line: 'ALLOW_EMPTY_${PN}=“1”'

I tried to make clean build by emptying the build directroy but that did not help.

The scripts are in bf-state-scripts_<version>.bb recipe and included into the build normally in the main image

IMAGE_INSTALL = " \

.

.

bf-state-scripts \

.

.

"

[Kristian Amlie]

On 21/08/18 08:21, Ari-Pekka Sihvonen wrote:
> Using that produces following error
> ERROR: ParseError at <path>/bf-state-scripts_0.2.5.bb:10

> <http://bf-state-scripts_0.2.5.bb:10>: unparsed line:

> 'ALLOW_EMPTY_${PN}=“1”'
>
> I tried to make clean build by emptying the build directroy but that did
> not help.

Are those quotes copied literally? I think those are advanced UTF
quotes. bitbake will probably understand only simple ASCII ones, like
this one:

ALLOW_EMPTY_${PN} = "1"

--
Kristian

[Ari-Pekka Sihvonen]

It was some kind a YOCTO glitch, I managed to bitbake the recipe but it did not populate the directories.

First it complained that unknow state ArtifactDownload, Renaiming ArtifactDownload_* scripts to Download_* fixed that and they were copied to /etc/mender/scripts/ directory and that was it. Nothing else was copied.

I have following scripts in that directory

ArtifactCommit_Enter_01   ArtifactInstall_Error_01   ArtifactRollback_Error_01        Download_Enter_10
ArtifactCommit_Error_01   ArtifactInstall_Leave_01   ArtifactRollback_Leave_01        Download_Enter_99
ArtifactCommit_Leave_01   ArtifactReboot_Enter_01    ArtifactRollbackReboot_Enter_01  Download_Error_01
ArtifactFailure_Enter_01  ArtifactReboot_Error_01    ArtifactRollbackReboot_Error_01  Download_Leave_01
ArtifactFailure_Leave_01  ArtifactReboot_Leave_01    ArtifactRollbackReboot_Leave_01
ArtifactInstall_Enter_01  ArtifactRollback_Enter_01  Download_Enter_01

What is proper naming of the scripts, I thought that they have to start with Artifact.

br

AP Sihvonen

[Kristian Amlie]

On 21/08/18 11:10, Ari-Pekka Sihvonen wrote:
> It was some kind a YOCTO glitch, I managed to bitbake the recipe but it
> did not populate the directories.
> First it complained that unknow state ArtifactDownload, Renaiming
> ArtifactDownload_* scripts to Download_* fixed that and they were copied
> to /etc/mender/scripts/ directory and that was it. Nothing else was copied.
> I have following scripts in that directory
> ArtifactCommit_Enter_01   ArtifactInstall_Error_01  
> ArtifactRollback_Error_01        Download_Enter_10
> ArtifactCommit_Error_01   ArtifactInstall_Leave_01  
> ArtifactRollback_Leave_01        Download_Enter_99
> ArtifactCommit_Leave_01   ArtifactReboot_Enter_01   
> ArtifactRollbackReboot_Enter_01  Download_Error_01
> ArtifactFailure_Enter_01  ArtifactReboot_Error_01   
> ArtifactRollbackReboot_Error_01  Download_Leave_01
> ArtifactFailure_Leave_01  ArtifactReboot_Leave_01   
> ArtifactRollbackReboot_Leave_01
> ArtifactInstall_Enter_01  ArtifactRollback_Enter_01  Download_Enter_01
>
> What is proper naming of the scripts, I thought that they have to start
> with Artifact.

Everything looks correct. Scripts that start with Artifact and scripts
that don't are two different types of scripts. Download scripts end up
on the filesystem, but Artifact* scripts end up inside the artifact,
*not* on the filesystem. So what you see is indeed correct.

It's all explained here:
https://docs.mender.io/1.5/artifacts/state-scripts#root-file-system-and-artifact-scripts

--
Kristian

[Ari-Pekka Sihvonen]

"but Artifact* scripts end up inside the artifact" Where exactly. Are you saying that they have life of their own in mender file. Is it so that when signing the artifact one must also include all of the Artifact scripts at the same time and there is no way the YOCTO can incorporate the scripts into the image.

br.

AP Sihvonen

[Kristian Amlie]

On 21/08/18 11:38, Ari-Pekka Sihvonen wrote:
> "but Artifact* scripts end up inside the artifact" Where exactly. Are
> you saying that they have life of their own in mender file. Is it so
> that when signing the artifact one must also include all of the Artifact
> scripts at the same time and there is no way the YOCTO can incorporate
> the scripts into the image.

Yes, the Artifact scripts are also signed when you request signing via
Yocto or the mender-artifact standalone tool. Artifact scripts are only
relevant when installing a new update, hence they are part of the mender
artifact, not part of the filesystem. They will be temporarily installed
in /data/mender while running.

--
Kristian

[Ari-Pekka Sihvonen]

I think I was a bit unclear. How do I incorporate the Artifact scripts to the mender artifact. Can it be done automatically with YOCTO or do I have to write them to the artifact by hand using mender-artifact tool at the same time when I generate signed artifact.

[Mirza Krak]

On Tue, Aug 21, 2018 at 12:04 PM, Ari-Pekka Sihvonen <asi...@iki.fi> wrote:
> I think I was a bit unclear. How do I incorporate the Artifact scripts to
> the mender artifact. Can it be done automatically with YOCTO or do I have to
> write them to the artifact by hand using mender-artifact tool at the same
> time when I generate signed artifact.

If you sign the artifacts outside of Yocto then you must also provide
the state-scripts at that stage.

But if you use Yocto to sign the artifacts [1] then state-scripts can
be integrated automatically using the "mender-state-script" bbclass
which has been linked in previous mails.

[1]. https://docs.mender.io/1.5/artifacts/building-for-production#artifact-signing-and-verification-keys

[Ari-Pekka Sihvonen]

From documentation "The private key used for signing the Mender Artifact should be protected and kept outside of the build system, thus there are no extra steps needed to add it to any part of the build system, Mender Client nor Server." If I understand it correctly you cannot sign artifact with YOCTO you have to use mender-artifact for that with -k private.key switch.

[Mirza Krak]

On Wed, Aug 22, 2018 at 6:57 AM, Ari-Pekka Sihvonen <asi...@iki.fi> wrote:
> From documentation "The private key used for signing the Mender Artifact
> should be protected and kept outside of the build system, thus there are no
> extra steps needed to add it to any part of the build system, Mender Client
> nor Server." If I understand it correctly you cannot sign artifact with
> YOCTO you have to use mender-artifact for that with -k private.key switch.

Yeah I see the confusion. Confused me for a second as well.

But actually you can use MENDER_ARTIFACT_SIGNING_KEY [1], to set a
path to the private key and that will be converted to an "-k
${MENDER_ARTIFACT_SIGNING_KEY}" argument to mender-artifact. See [2]

[1]. https://docs.mender.io/1.5/artifacts/variables#mender_artifact_signing_key
[2]. https://github.com/mendersoftware/meta-mender/blob/d38fa1548b2de25c71462761c7d69cba157c714a/meta-mender-core/classes/mender-artifactimg.bbclass#L39-L52

[Ari-Pekka Sihvonen]

Works like a charm with latest advice. Thank you for your patience

regards

AP Sihvonen