execution order, scripts, and version comparisons

[Dominique MARTINET]

Hi,

sorry for the subject I'm still messing around with our install setup
and have been struggling a bit with the design to go with.
Please advise or tell me if there are things that can be
changed/documented somewhere.



First is execution order; from what I understand it goes like this:
- sw-description signature is verified (nothing runs if that fails)
- embedded lua hooks (executed during parsing)
- preupdatecmd
- actual handlers if installed-directly is set
- preinstall scripts (either shell or lua)
- other handlers (sw->images so image, file or partition)
- postinstall scripts
- postupdatecmd if requested (swupdate-client -p or network update
only)


So first that means if I want to run before installed-directly images I
need to use either preupdatecmd or lua hooks; while I can work with that
it's not strictly obvious and might be simpler if installed-directly
images were run in a separate step with other normal handlers.


With that in mind I'll probably be using a preupdatecmd, so I wanted to
use postupdatecmd along with it but I'm not sure if it's intended that
it doens't run when I do e.g. `swupdate -p /post.sh -i img.swu -v`
manually?
(for now swupdate-client -p does the trick on local files, and network
updates seem to work fine; I haven't tested hawkbit but I think it'll
probably work too)


And finally, if any step fails none of the following steps run (we
discussed that very briefly with a fail command - I'm still interested
for cleanup)



The last point of the subject is slightly unrelated; but I plan on
having images where all components are versioned, so assuming devices
poll an url for update regularly once they updated all versions would be
identical to what is running.

In that case, swupdate fails rather loudly:

[ERROR] : SWUPDATE failed [0] ERROR : Found nothing to install
[ERROR] : SWUPDATE failed [0] ERROR : JSON File corrupted
[ERROR] : SWUPDATE failed [0] ERROR : no parser available to parse sw-description!
[ERROR] : SWUPDATE failed [0] ERROR : Compatible SW not found
[ERROR] : SWUPDATE failed [0] ERROR : Writing into SWUpdate IPC stream failed.
[ERROR] : SWUPDATE failed [0] ERROR : Channel operation returned error (23): 'Failed writing received data to disk/application'
[ERROR] : SWUPDATE failed [1] Image invalid or corrupted. Not installing...


But the file is perfectly valid so I don't think that's an error.
(I agree execution should stop as it does now and not run even
precommand, that is the correct behaviour; although it will have run lua
hooks these can check versions themselves)

Would it be possible to fail a bit more gracefully here?


(speaking of error messages I see everytime, I also get an error on ubi
partition scan:
[ERROR] : SWUPDATE failed [0] ERROR : cannot attach mtd0 - maybe not a NAND or raw device

since nothing in the sw-description requires it it's a bit annoying as
well, but I'll just remove CONFIG_UBIATTACH from our builds as a
workaround)


Thanks,
--
Dominique

[Stefano Babic]

Hi Dominique,

On 02.04.21 08:04, Dominique MARTINET wrote:
> Hi,
>
> sorry for the subject I'm still messing around with our install setup
> and have been struggling a bit with the design to go with.
> Please advise or tell me if there are things that can be
> changed/documented somewhere.
>
>
>
> First is execution order; from what I understand it goes like this:
> - sw-description signature is verified (nothing runs if that fails)

Right

> - embedded lua hooks (executed during parsing)

Right

> - preupdatecmd
> - actual handlers if installed-directly is set

Right, order is not fixed. Frst image streamed, first image served (of
course).

> - preinstall scripts (either shell or lua)

Right

> - other handlers (sw->images so image, file or partition)
> - postinstall scripts
> - postupdatecmd if requested (swupdate-client -p or network update
> only)

Or swupdate started with -p or postinstall command defined in
configuration file.

>
>
> So first that means if I want to run before installed-directly images I
> need to use either preupdatecmd or lua hooks;

Use Lua Hooks. This was already discussed on the ML, and it is the safer
place to do something before installing.

> while I can work with that
> it's not strictly obvious and might be simpler if installed-directly
> images were run in a separate step with other normal handlers.

I do not understand this.

If this is nmot strictly obvious, it could be better documented.

>
>
> With that in mind I'll probably be using a preupdatecmd, > so I wanted to
> use postupdatecmd along with it but I'm not sure if it's intended that
> it doens't run when I do e.g. `swupdate -p /post.sh -i img.swu -v`
> manually?

I do not get the point - a postupdate command is called when it is its
turn. Of course, it is not called if SWUpdate stops before doing to an
error.

> (for now swupdate-client -p does the trick on local files, and network
> updates seem to work fine; I haven't tested hawkbit but I think it'll
> probably work too)

There is no difference between local or OTA update. All interfaces
communicate with the installer via IPC, and SWUpdate itself (the
installer) does not bother where a SWU is coming.

>
>
> And finally, if any step fails none of the following steps run

Right.

(we
> discussed that very briefly with a fail command - I'm still interested
> for cleanup)
>
>
>
> The last point of the subject is slightly unrelated; but I plan on
> having images where all components are versioned, so assuming devices
> poll an url for update regularly once they updated all versions would be
> identical to what is running.
>
> In that case, swupdate fails rather loudly:
>
> [ERROR] : SWUPDATE failed [0] ERROR : Found nothing to install
> [ERROR] : SWUPDATE failed [0] ERROR : JSON File corrupted
> [ERROR] : SWUPDATE failed [0] ERROR : no parser available to parse sw-description!
> [ERROR] : SWUPDATE failed [0] ERROR : Compatible SW not found
> [ERROR] : SWUPDATE failed [0] ERROR : Writing into SWUpdate IPC stream failed.
> [ERROR] : SWUPDATE failed [0] ERROR : Channel operation returned error (23): 'Failed writing received data to disk/application'
> [ERROR] : SWUPDATE failed [1] Image invalid or corrupted. Not installing...
>
>
> But the file is perfectly valid so I don't think that's an error.

This was a neverending discussion in the Mailing list and it is more a
philosophical as technical discussion. If there is nothing to install,
we can say that SWUpdate has successfully installed "nothing". This was
also the old behavior. Anyway, if SWUpdate is successful, one can
interprete this as a successful update and a toglle / reboot is
triggered. The failure (even if it is not a strict failure) stops any
attempt to toggle (if done externally) and reboot.

> (I agree execution should stop as it does now and not run even
> precommand, that is the correct behaviour; although it will have run lua
> hooks these can check versions themselves)
>
> Would it be possible to fail a bit more gracefully here?

Failure is failure, SWUpdate reports at the end a boolean as result.
There is not a weak failure.

>
>
> (speaking of error messages I see everytime, I also get an error on ubi
> partition scan:
> [ERROR] : SWUPDATE failed [0] ERROR : cannot attach mtd0 - maybe not a NAND or raw device
>
> since nothing in the sw-description requires it it's a bit annoying as
> well, but I'll just remove CONFIG_UBIATTACH from our builds as a
> workaround)

No, you have to add you own UBIWHITELIST and mtd0 should not be part of
it. This is normal when UBI is used, and not all MTD devices must be
scanned.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

[Dominique MARTINET]

Thanks for the quick reply

Stefano Babic wrote on Fri, Apr 02, 2021 at 12:08:28PM +0200:
> > - other handlers (sw->images so image, file or partition)
> > - postinstall scripts
> > - postupdatecmd if requested (swupdate-client -p or network update
> > only)
>
> Or swupdate started with -p or postinstall command defined in configuration
> file.

That doesn't always work in my tests.
With a postinstall command defined in configuration config (or -p
somescript, I had similar behaviour), this triggered the post script:

swupdate -d '-u http://someurl/test.swu'

But this did not:

swupdate -i test.swu

I don't have the code open right now but I think this comes from the
fact that the precommand runs from stream extractor so always runs, but
postupdate runs from an explicit rpc request in either swupdate-client
(so swupdate-client -p test.swu works) or the downloader part of the
code, but not local install.

> > So first that means if I want to run before installed-directly images I
> > need to use either preupdatecmd or lua hooks;
>
> Use Lua Hooks. This was already discussed on the ML, and it is the safer
> place to do something before installing.

If lua hooks could call external lua scripts I wouldn't mind so much,
but managing code in sw-description is really not something I want to
do right now.
I also don't like the dissymmetry that would come from "pre" stuff being
in embed script while the post part would be separate; I'd rather have
things simple for others to pick up and having a single logic is better
for that.

Anyway, once I finally figured how the post command was triggered that
works for me.

> > while I can work with that
> > it's not strictly obvious and might be simpler if installed-directly
> > images were run in a separate step with other normal handlers.
>
> I do not understand this.
>
> If this is nmot strictly obvious, it could be better documented.

for example swupdate.rst states "- Support for preinstall scripts. They
run before updating the images" but if there are items with
installed-directly that is no longer true.

Looking further below there is a note about using embedded script:
---
- runs pre-install scripts, if any. Please note: if artifacts are
streamed, a preinstall script cannot be executed because it depends on
the order the artifact (including the script) are packed into the CPIO
archive. In this case, please use the "embedded-script" feature in
sw-description to execute functions before any installation takes place.
---

but the whole section is still before the extract/installed-directly
step in the list so it wasn't really clear to me what order would be
expected from pre-install scripts.

Maybe the two could be rephrased a bit, I don't know.

> > [ERROR] : SWUPDATE failed [0] ERROR : Found nothing to install
> > [ERROR] : SWUPDATE failed [0] ERROR : JSON File corrupted
> > [ERROR] : SWUPDATE failed [0] ERROR : no parser available to parse sw-description!
> > [ERROR] : SWUPDATE failed [0] ERROR : Compatible SW not found
> > [ERROR] : SWUPDATE failed [0] ERROR : Writing into SWUpdate IPC stream failed.
> > [ERROR] : SWUPDATE failed [0] ERROR : Channel operation returned error (23): 'Failed writing received data to disk/application'
> > [ERROR] : SWUPDATE failed [1] Image invalid or corrupted. Not installing...
> >
> >
> > But the file is perfectly valid so I don't think that's an error.
>
> This was a neverending discussion in the Mailing list and it is more a
> philosophical as technical discussion. If there is nothing to install, we
> can say that SWUpdate has successfully installed "nothing". This was also
> the old behavior. Anyway, if SWUpdate is successful, one can interprete this
> as a successful update and a toglle / reboot is triggered. The failure (even
> if it is not a strict failure) stops any attempt to toggle (if done
> externally) and reboot.

Ok, I wasn't aware of the past discussion as I subscribed just recently,
sorry for that. I think both point of views are valid here.

My problem is that error log level will be written in a rather small
syslog so if possible I don't want to repeat these error lines everytime
we check if there was a new update.
I guess I can wrap the swupdate execution around a separate poll that
checks for the http date but I'm not sure how reliable that would really
be compared to just reading the sw-description part of the cpio and
checking versions there... So I'll be more inclined towards adding an
empty shell script just to make the update non-empty.
Well, I'll think about it a bit more.

> > (speaking of error messages I see everytime, I also get an error on ubi
> > partition scan:
> > [ERROR] : SWUPDATE failed [0] ERROR : cannot attach mtd0 - maybe not a NAND or raw device
> >
> > since nothing in the sw-description requires it it's a bit annoying as
> > well, but I'll just remove CONFIG_UBIATTACH from our builds as a
> > workaround)
>
> No, you have to add you own UBIWHITELIST and mtd0 should not be part of it.
> This is normal when UBI is used, and not all MTD devices must be scanned.

Hm, these are still config options and I won't use the handler at all,
so it's probably best to just disable. In general security practice
would be to disable everything we don't use anyway, I just had created a
swupdate package with all handlers enabled to test things (and
potentially package it for alpine linux like debian did), but I think
it's best if people compile something for their own needs ultimately.


Thanks,
--
Dominique

[Stefano Babic]

Hi Dominique,

On 02.04.21 16:55, Dominique MARTINET wrote:
> Thanks for the quick reply
>
> Stefano Babic wrote on Fri, Apr 02, 2021 at 12:08:28PM +0200:
>>> - other handlers (sw->images so image, file or partition)
>>> - postinstall scripts
>>> - postupdatecmd if requested (swupdate-client -p or network update
>>> only)
>>
>> Or swupdate started with -p or postinstall command defined in configuration
>> file.
>
> That doesn't always work in my tests.
> With a postinstall command defined in configuration config (or -p
> somescript, I had similar behaviour), this triggered the post script:
>
> swupdate -d '-u http://someurl/test.swu'
>
> But this did not:
>
> swupdate -i test.swu
>
> I don't have the code open right now but I think this comes from the
> fact that the precommand runs from stream extractor so always runs, but
> postupdate runs from an explicit rpc request in either swupdate-client
> (so swupdate-client -p test.swu works) or the downloader part of the
> code, but not local install.

I think you're right. This is like a regression after dropping the
internal path for local install and using network streaming for local
install, too. As local installer, the function install_from_file() is
called and this acts like a swupdate-client, but it does not call at the
end the IPC for postupdate.

>
>>> So first that means if I want to run before installed-directly images I
>>> need to use either preupdatecmd or lua hooks;
>>
>> Use Lua Hooks. This was already discussed on the ML, and it is the safer
>> place to do something before installing.
>
> If lua hooks could call external lua scripts I wouldn't mind so much,
> but managing code in sw-description is really not something I want to
> do right now.
> I also don't like the dissymmetry that would come from "pre" stuff being
> in embed script while the post part would be separate; I'd rather have
> things simple for others to pick up and having a single logic is better
> for that.

The "pre" in embedded script guarantees that it is always called before
anything is done, and this cannot be guaranteed if a preinstall script
is part of the SWU.

On the other end, there are several differnt use cases for post install,
for example SWUpdate does not do anything and a post install is done by
swupdate-progress or by an external application.

>
> Anyway, once I finally figured how the post command was triggered that
> works for me.
>
>
>>> while I can work with that
>>> it's not strictly obvious and might be simpler if installed-directly
>>> images were run in a separate step with other normal handlers.
>>
>> I do not understand this.
>>
>> If this is nmot strictly obvious, it could be better documented.
>
> for example swupdate.rst states "- Support for preinstall scripts. They
> run before updating the images" but if there are items with
> installed-directly that is no longer true.

This is a very old statement before I implemented the streaming feature,
and it is currently wrong.

>
> Looking further below there is a note about using embedded script:
> ---
> - runs pre-install scripts, if any. Please note: if artifacts are
> streamed, a preinstall script cannot be executed because it depends on
> the order the artifact (including the script) are packed into the CPIO
> archive. In this case, please use the "embedded-script" feature in
> sw-description to execute functions before any installation takes place.
> ---
>
> but the whole section is still before the extract/installed-directly
> step in the list so it wasn't really clear to me what order would be
> expected from pre-install scripts.
>
> Maybe the two could be rephrased a bit, I don't know.

Yes, the second one was added, the whole context is misleading. It
should be rewritten.

So just disable it..

> so it's probably best to just disable. In general security practice
> would be to disable everything we don't use anyway,

Right, that is the reason you can configure a lot of thinng in SWUpdate
and you should disable everything you do not need.

> I just had created a
> swupdate package with all handlers enabled to test things (and
> potentially package it for alpine linux like debian did), but I think
> it's best if people compile something for their own needs ultimately.

Right. This is how it works in OE - each project provides its own
specific defconfig for SWUpdate.

[Dominique MARTINET]

Stefano Babic wrote on Fri, Apr 02, 2021 at 12:08:28PM +0200:

> > With that in mind I'll probably be using a preupdatecmd, > so I wanted to
> > use postupdatecmd along with it but I'm not sure if it's intended that
> > it doens't run when I do e.g. `swupdate -p /post.sh -i img.swu -v`
> > manually?
>
> I do not get the point - a postupdate command is called when it is its turn.
> Of course, it is not called if SWUpdate stops before doing to an error.

Actually that's not consistent either, running with post command defined
in config file:
# swupdate -d '-u http://10.1.1.1/out.swu' -v

If the error is that the swu has nothing to install, the post command is
not called.
If the error is a handler failure (I made a typo in sw-description with
archive handler having `dest = "path"` instead of `path = "path"`, it runs:


[TRACE] : SWUPDATE running : [install_single_image] : Found installer for stream rootfs.tar.gz archive
[TRACE] : SWUPDATE running : [install_archive_image] : Missing path attribute
[TRACE] : SWUPDATE running : [install_single_image] : Installer for archive not successful !
Environment OK, copy 0
[ERROR] : SWUPDATE failed [1] Installation failed !
Environment OK, copy 0
[TRACE] : SWUPDATE running : [network_initializer] : Main thread sleep again !
[INFO ] : No SWUPDATE running : Waiting for requests...
[TRACE] : SWUPDATE running : [network_initializer] : Main loop Daemon
[ERROR] : SWUPDATE failed [0] ERROR : /root/test_post.sh: line 4: stderr from post script

(while looking at this the "Missing path attribute" should probably be
bumped up from TRACE to ERROR; I can send a patch for that if you agree)


We probably want to make that consistent as well.

> > But the file is perfectly valid so I don't think that's an error.
>
> This was a neverending discussion in the Mailing list and it is more a
> philosophical as technical discussion. If there is nothing to install, we
> can say that SWUpdate has successfully installed "nothing". This was also
> the old behavior. Anyway, if SWUpdate is successful, one can interprete this
> as a successful update and a toglle / reboot is triggered. The failure (even
> if it is not a strict failure) stops any attempt to toggle (if done
> externally) and reboot.

Ok, so I finally noticed just now that if there is anything to update
the whole cpio is downloaded, even skipped parts.

I for some reason always had thought that skipped files of the cpio were
not downloaded at all (that should be possible with libcurl as we know
the size of the cpio header and the size of each file inside); not being
able to arbitrarily download each file also clears up to me why
installed directly was done that way.

Was the inability to skip downloading parts of the archive a concious
decision, or is it just a byproduct of how communication currently works
with the downloader process? e.g. if I were to spend time making it work
so that the main swupdate process requests ranges from downloader and
downloader abides by it with HTTP Range requests, would you be
interested?

Stefano Babic wrote on Mon, Apr 05, 2021 at 10:39:21AM +0200:
> > I don't have the code open right now but I think this comes from the
> > fact that the precommand runs from stream extractor so always runs, but
> > postupdate runs from an explicit rpc request in either swupdate-client
> > (so swupdate-client -p test.swu works) or the downloader part of the
> > code, but not local install.
>
> I think you're right. This is like a regression after dropping the internal
> path for local install and using network streaming for local install, too.
> As local installer, the function install_from_file() is called and this acts
> like a swupdate-client, but it does not call at the end the IPC for
> postupdate.

Ok, I can fix that if you say it's a bug.
Will send a patch later this week to call the post update IPC request.

> > for example swupdate.rst states "- Support for preinstall scripts. They
> > run before updating the images" but if there are items with
> > installed-directly that is no longer true.
>
> This is a very old statement before I implemented the streaming feature, and
> it is currently wrong.

> > Looking further below there is a note about using embedded script:
> > ---
> > - runs pre-install scripts, if any. Please note: if artifacts are
> > streamed, a preinstall script cannot be executed because it depends on
> > the order the artifact (including the script) are packed into the CPIO
> > archive. In this case, please use the "embedded-script" feature in
> > sw-description to execute functions before any installation takes place.
> > ---
> >
> > but the whole section is still before the extract/installed-directly
> > step in the list so it wasn't really clear to me what order would be
> > expected from pre-install scripts.
> >
> > Maybe the two could be rephrased a bit, I don't know.
>
> Yes, the second one was added, the whole context is misleading. It should be
> rewritten.

I don't personally care much anymore at this point, but I guess since I
brought it up I should send a rewrite suggestion...

> > I just had created a
> > swupdate package with all handlers enabled to test things (and
> > potentially package it for alpine linux like debian did), but I think
> > it's best if people compile something for their own needs ultimately.
>
> Right. This is how it works in OE - each project provides its own specific
> defconfig for SWUpdate.

I think having swupdate as part of distros could also make sense if this
code could be disabled in swupdate.cfg rather than build time for
example.

I will disable MTD here but atmark provides bare boards with some
pre-integration for our customers to develop real usecases on, and they
might want to use things I didn't plan for while still using the base OS
we provide, so some runtime flexibility is always appreciable - For
example I will leave lua handlers configured for them.

But that is beyond the time I can spend on this now, so just disabling
is good enough for me.

--
Dominique

[Stefano Babic]

Hi Dominique,

On 06.04.21 07:00, Dominique MARTINET wrote:
> Stefano Babic wrote on Fri, Apr 02, 2021 at 12:08:28PM +0200:
>>> With that in mind I'll probably be using a preupdatecmd, > so I wanted to
>>> use postupdatecmd along with it but I'm not sure if it's intended that
>>> it doens't run when I do e.g. `swupdate -p /post.sh -i img.swu -v`
>>> manually?
>>
>> I do not get the point - a postupdate command is called when it is its turn.
>> Of course, it is not called if SWUpdate stops before doing to an error.
>
> Actually that's not consistent either, running with post command defined
> in config file:
> # swupdate -d '-u http://10.1.1.1/out.swu' -v
>
> If the error is that the swu has nothing to install, the post command is
> not called.

Sorry, buit it is consistent. There is an ERROR: SWUpdate stops
immediately. Then a post update command should not be called.

> If the error is a handler failure (I made a typo in sw-description with
> archive handler having `dest = "path"` instead of `path = "path"`, it runs:
>
>
> [TRACE] : SWUPDATE running : [install_single_image] : Found installer for stream rootfs.tar.gz archive
> [TRACE] : SWUPDATE running : [install_archive_image] : Missing path attribute
> [TRACE] : SWUPDATE running : [install_single_image] : Installer for archive not successful !
> Environment OK, copy 0
> [ERROR] : SWUPDATE failed [1] Installation failed !
> Environment OK, copy 0
> [TRACE] : SWUPDATE running : [network_initializer] : Main thread sleep again !
> [INFO ] : No SWUPDATE running : Waiting for requests...
> [TRACE] : SWUPDATE running : [network_initializer] : Main loop Daemon
> [ERROR] : SWUPDATE failed [0] ERROR : /root/test_post.sh: line 4: stderr from post script

Is it called ? Mmhhhh, it should not be.

>
> (while looking at this the "Missing path attribute" should probably be
> bumped up from TRACE to ERROR; I can send a patch for that if you agree)

+1

>
>
> We probably want to make that consistent as well.
>

Right

>
>>> But the file is perfectly valid so I don't think that's an error.
>>
>> This was a neverending discussion in the Mailing list and it is more a
>> philosophical as technical discussion. If there is nothing to install, we
>> can say that SWUpdate has successfully installed "nothing". This was also
>> the old behavior. Anyway, if SWUpdate is successful, one can interprete this
>> as a successful update and a toglle / reboot is triggered. The failure (even
>> if it is not a strict failure) stops any attempt to toggle (if done
>> externally) and reboot.
>
> Ok, so I finally noticed just now that if there is anything to update
> the whole cpio is downloaded, even skipped parts.
>

Right

> I for some reason always had thought that skipped files of the cpio were
> not downloaded at all (that should be possible with libcurl as we know
> the size of the cpio header and the size of each file inside);

It is possible if the server supports this feature, most servers do.

> not being
> able to arbitrarily download each file also clears up to me why
> installed directly was done that way.
>
> Was the inability to skip downloading parts of the archive a concious
> decision, or is it just a byproduct of how communication currently works
> with the downloader process?

There were several reasons, and also that the updater itself should be
unaware about the incoming data and this should be just a stream.

> e.g. if I were to spend time making it work
> so that the main swupdate process requests ranges from downloader and
> downloader abides by it with HTTP Range requests, would you be
> interested?

Please read the following thread:

https://groups.google.com/g/swupdate/c/yCDBnWip5xw/m/BA77xKWtCQAJ

This contains the discussion about the topic, a short design from me how
to implement and the comments from community to change this and to do in
a different way.

This is currently on the desk of a customer of mine, and I will go on as
soon as the customer will finance it.

>
> Stefano Babic wrote on Mon, Apr 05, 2021 at 10:39:21AM +0200:
>>> I don't have the code open right now but I think this comes from the
>>> fact that the precommand runs from stream extractor so always runs, but
>>> postupdate runs from an explicit rpc request in either swupdate-client
>>> (so swupdate-client -p test.swu works) or the downloader part of the
>>> code, but not local install.
>>
>> I think you're right. This is like a regression after dropping the internal
>> path for local install and using network streaming for local install, too.
>> As local installer, the function install_from_file() is called and this acts
>> like a swupdate-client, but it does not call at the end the IPC for
>> postupdate.
>
> Ok, I can fix that if you say it's a bug.

Yes, it seems to me a bug.

> Will send a patch later this week to call the post update IPC request.

Ok

>
>>> for example swupdate.rst states "- Support for preinstall scripts. They
>>> run before updating the images" but if there are items with
>>> installed-directly that is no longer true.
>>
>> This is a very old statement before I implemented the streaming feature, and
>> it is currently wrong.
>
>>> Looking further below there is a note about using embedded script:
>>> ---
>>> - runs pre-install scripts, if any. Please note: if artifacts are
>>> streamed, a preinstall script cannot be executed because it depends on
>>> the order the artifact (including the script) are packed into the CPIO
>>> archive. In this case, please use the "embedded-script" feature in
>>> sw-description to execute functions before any installation takes place.
>>> ---
>>>
>>> but the whole section is still before the extract/installed-directly
>>> step in the list so it wasn't really clear to me what order would be
>>> expected from pre-install scripts.
>>>
>>> Maybe the two could be rephrased a bit, I don't know.
>>
>> Yes, the second one was added, the whole context is misleading. It should be
>> rewritten.
>
> I don't personally care much anymore at this point, but I guess since I
> brought it up I should send a rewrite suggestion...

Yes, you are saying it is not clear, it will be nice if you rewrite this
so that becomes more understandable.

>
>>> I just had created a
>>> swupdate package with all handlers enabled to test things (and
>>> potentially package it for alpine linux like debian did), but I think
>>> it's best if people compile something for their own needs ultimately.
>>
>> Right. This is how it works in OE - each project provides its own specific
>> defconfig for SWUpdate.
>
> I think having swupdate as part of distros could also make sense if this
> code could be disabled in swupdate.cfg rather than build time for
> example.

...but runtime disable means to use the UBI lists, or dropping the scan
for some MTD passing the -b parameter. For example, to avoid scan for
mtd0 and mtd1

swupdate -b "0 1"

>
> I will disable MTD here but atmark provides bare boards with some
> pre-integration for our customers to develop real usecases on, and they
> might want to use things I didn't plan for while still using the base OS
> we provide, so some runtime flexibility is always appreciable

There is already with the -b parameter. This is the same runtime
behavior as CONFIG_UBIBLACKLIST. There is currently no paramter to
mirror at runtime CONFIG_UBIWHITELIST, but I have nothing to say against
such as patch.

> - For
> example I will leave lua handlers configured for them.
>
> But that is beyond the time I can spend on this now, so just disabling
> is good enough for me.
>

[Dominique MARTINET]

Stefano Babic wrote on Tue, Apr 06, 2021 at 09:50:41AM +0200:
> > If the error is a handler failure (I made a typo in sw-description with
> > archive handler having `dest = "path"` instead of `path = "path"`, it runs:
> >
> >
> > [TRACE] : SWUPDATE running : [install_single_image] : Found installer for stream rootfs.tar.gz archive
> > [TRACE] : SWUPDATE running : [install_archive_image] : Missing path attribute
> > [TRACE] : SWUPDATE running : [install_single_image] : Installer for archive not successful !
> > Environment OK, copy 0
> > [ERROR] : SWUPDATE failed [1] Installation failed !
> > Environment OK, copy 0
> > [TRACE] : SWUPDATE running : [network_initializer] : Main thread sleep again !
> > [INFO ] : No SWUPDATE running : Waiting for requests...
> > [TRACE] : SWUPDATE running : [network_initializer] : Main loop Daemon
> > [ERROR] : SWUPDATE failed [0] ERROR : /root/test_post.sh: line 4: stderr from post script
>
> Is it called ? Mmhhhh, it should not be.

> > We probably want to make that consistent as well.
>
> Right

Ok, I will look into that as well...

Might be a bit of time before I have done everything I promised in this
thread, will post things as they are ready.

> > e.g. if I were to spend time making it work
> > so that the main swupdate process requests ranges from downloader and
> > downloader abides by it with HTTP Range requests, would you be
> > interested?
>
> Please read the following thread:
>
> https://groups.google.com/g/swupdate/c/yCDBnWip5xw/m/BA77xKWtCQAJ
>
> This contains the discussion about the topic, a short design from me how to
> implement and the comments from community to change this and to do in a
> different way.
>
> This is currently on the desk of a customer of mine, and I will go on as
> soon as the customer will finance it.

Thanks for the link, it looks like I had missed some previous discussion
again, sorry for that.

That pretty much corresponds to what I had in mind.
I guess we will start with what we have and hope for this feature for
now, please do notify me/the list if the funding does not come and
others could implement it.

--
Dominique