Use SWUpdate with BIOS motherboard

279 views
Skip to first unread message

Joao Mendes

unread,
Feb 9, 2017, 2:05:13 PM2/9/17
to swup...@googlegroups.com
Hello,

First of all I apologize for my English.

As part of a project for school (bachelor level), I need to realize a NAS and I need an upgrade system.

My teacher told me to take a look at SWUpdate. But I use an Asrock C2550D4I motherboard (with BIOS) wich is not an embedded system. My linux is a fedora-server distribution.

Basically I must have 4 partitions:

- recovery (system never change use only in last solution to upgrade nas image 1) (ext4 partition)
- nas-image-1 (system) (ext4 partition)
- nas-image-2 (alternative system) (ext4 partition)
- data (ext4 partition)

At the begin nas-image-1 and nas-image-2 are the same and the NAS run on nas-image-1.

When the user wants upgrade the system I need to load the new image and install it on one of the nas-image-1 or nas-image-2 partition.
 
If the NAS runs on nas-image-1, I install it on nas-image-2 partition and if the NAS runs on nas-image-2 I install it on nas-image-1.

That way in case of problem I still have an unchanged partition and I still can boot. It look like the Double copy with fall-back system of SWUpdate.

My question is can I do it with SWUpdate? And how can I do it? Because I have read the documentation and I feel like I do not have enough knowledge to do it.

My first intuition would be to create a script to load an image file and use dd command to write the image on the partition.

Thanks in advance for your help.

Joao Mendes

okt...@gmail.com

unread,
Feb 10, 2017, 4:13:21 AM2/10/17
to swupdate, joaomen...@gmail.com
> My question is can I do it with SWUpdate? And how can I do it?

Certainly you can. I see a couple of ways:

- Have your rootfs as ext4 file (with path pointing to a location with enough free space on target) within SWUpdate CPIO archive, accompanied with a shellscript containing pre- and post-install tasks.
In pre-install task, create the target file as empty: this is required due to how 'file' items are handled by SWUpdate.
Let SWUpdate extract the ext4 file into the location, and in post-install task do whatever is necessary to place the ext4 rootfs into the unused partition.

- Have your rootfs as ext4 image within SWUpdate CPIO archive, and use the "remote handler" to pass the image data (via ZeroMQ) to a program you'll have written. That program is responsible for placing the ext4 rootfs into the unused partition.

You could use squashfs instead of ext4 to make the update archive smaller (but then you cannot use 'dd').

(I'm using a custom handler, written in-house, that streams the data to a back-end installer, which is a command-line utility; it's kinda like the "remote handler" but way simpler imho.)

--sami

Stefano Babic

unread,
Feb 10, 2017, 5:01:36 AM2/10/17
to Joao Mendes, swup...@googlegroups.com
Hi Joao,

On 09/02/2017 20:05, Joao Mendes wrote:
> Hello,
>
> First of all I apologize for my English.
>

No worry - if I had to apologize any time for my english, I had no time
to work !

> As part of a project for school (bachelor level), I need to realize a
> NAS and I need an upgrade system.
>

ok

> My teacher told me to take a look at SWUpdate. But I use an Asrock
> C2550D4I motherboard (with BIOS) wich is not an embedded system. My
> linux is a fedora-server distribution.
>

It is not an issue which distro you use on your host. Maybe can you
revive the meta-baryon layer ?

https://wiki.yoctoproject.org/wiki/Baryon

I can just suggest to take a look how to build the NAS software with
Yocto. This will be surely well seen by your teacher :-)

The main issue you have to check is regarding the bootloader. You will
have Grub, and you have to switch between the partitions setting the
Grub environment. The item was already discussed on the ML, you can
check in archive.

> Basically I must have 4 partitions:
>
> - recovery (system never change use only in last solution to upgrade nas
> image 1) (ext4 partition)
> - nas-image-1 (system) (ext4 partition)
> - nas-image-2 (alternative system) (ext4 partition)
> - data (ext4 partition)
>
> At the begin nas-image-1 and nas-image-2 are the same and the NAS run on
> nas-image-1.
>
> When the user wants upgrade the system I need to load the new image and
> install it on one of the nas-image-1 or nas-image-2 partition.
>
> If the NAS runs on nas-image-1, I install it on nas-image-2 partition
> and if the NAS runs on nas-image-2 I install it on nas-image-1.
>

This is just the description for a dual-copy architecture.

> That way in case of problem I still have an unchanged partition and I
> still can boot. It look like the Double copy with fall-back system of
> SWUpdate.

A safe system is much more. You have to be sure that a failing system
(due to kernel panic, corruption ,whatever..) will be recognize by the
bootloader that should start the secondary partition.

>
> My question is can I do it with SWUpdate? And how can I do it? Because I
> have read the documentation and I feel like I do not have enough
> knowledge to do it.
>
> My first intuition would be to create a script to load an image file and
> use dd command to write the image on the partition.

I do not think it is a good idea. If you have external tools, you missed
most of the functionalities and specially what happens in case of failure.

You should use SWUpdate in dual-copy mode, and prepare your rootfs as
.ext4.gz image, that SWUpdate can install on your media without any
strange pre- or prost- install script.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

Joao Mendes

unread,
Feb 11, 2017, 10:26:43 AM2/11/17
to Stefano Babic, swup...@googlegroups.com
Thanks for your help. It helped me to see more clearly what to do.

But I steel have some questions.

In my NAS I use ZFS file system to store users data and ext4 for system.

I have an SSD to the system partitions and 4 WD RED to store users datas and I have realized a command line interpreter for it to be simpler for users.
I put the files for the cli execution in /opt. And use the data partition to store the config (which I had to restore after an upgrade).

When I build with Yocto will the cli files be had to my image or only packages are added?

Also just to be sure I have understand the order of thing I have to do.

1 - Set up my config

2 - Use yocto to buil with meta-SWUpdate

3 - Write swdescription file

4 - Make the .swu file

5 - Make the script to change grub2 config in post-install

6 - run swupdate

Did I forget something?

Finally actually I have done my entire config in virtual with VirtualBox should I build my virtual config or reproduce it in my hardware config and after do the build,
because i steel need to do an from scratch install on the hardware or can  I directly install the build image?

Are there any other?


Thanks again for your help 





--
Joao Mendes
Chemin des Semailles 9C
1212 Grand-Lancy
+41 79 707 95 22
joaomen...@gmail.com

Stefano Babic

unread,
Feb 11, 2017, 10:57:33 AM2/11/17
to Joao Mendes, Stefano Babic, swup...@googlegroups.com
Hi Joao,

On 11/02/2017 16:26, Joao Mendes wrote:
> Thanks for your help. It helped me to see more clearly what to do.
>
> But I steel have some questions.
>
> In my NAS I use ZFS file system to store users data and ext4 for system.
>
> I have an SSD to the system partitions and 4 WD RED to store users datas
> and I have realized a command line interpreter for it to be simpler for
> users.
> I put the files for the cli execution in /opt. And use the data
> partition to store the config (which I had to restore after an upgrade).
>
> When I build with Yocto will the cli files be had to my image or only
> packages are added?

This is your decision - you can put the cli files where you want.

>
> Also just to be sure I have understand the order of thing I have to do.
>
> 1 - Set up my config
>
> 2 - Use yocto to buil with meta-SWUpdate
>

With Yocto you will get the whole rootfs or everything you are planning
to build.

> 3 - Write swdescription file
>
> 4 - Make the .swu file
>
> 5 - Make the script to change grub2 config in post-install
>

You can integrate the post install script in the swu image

> 6 - run swupdate
>
> Did I forget something?
>
> Finally actually I have done my entire config in virtual with VirtualBox
> should I build my virtual config or reproduce it in my hardware config
> and after do the build,
> because i steel need to do an from scratch install on the hardware or
> can I directly install the build image?

It depends - but generally, you will get with Yocto the images to be
installed on the target. How to install them, it is project specific and
depends on the hardware. In your case, you plan to have two copies of
the software. You have first to copy the built images in one of the two
copies.

>
> I almost forgot in the ML the only discussion about grub I found are
> these 2
>
> https://groups.google.com/forum/#!msg/swupdate/OYuT-ZKeVUc/S3seJ7BwEwAJ
>
> https://groups.google.com/forum/m/#!msg/swupdate/nc38u9crxy4/WogTddmdCQAJ
>
> Are there any other?
>

They are.

Best regards,
Stefano Babic

>
> Thanks again for your help
>
>
>
>
>
>
> 2017-02-10 11:01 GMT+01:00 Stefano Babic <sba...@denx.de
> <mailto:sba...@denx.de>>:
> Phone: +49-8142-66989-53 <tel:%2B49-8142-66989-53> Fax:
> +49-8142-66989-80 <tel:%2B49-8142-66989-80> Email: sba...@denx.de
> <mailto:sba...@denx.de>
> =====================================================================
>
>
>
>
> --
> Joao Mendes
> Chemin des Semailles 9C
> 1212 Grand-Lancy
> +41 79 707 95 22
> joaomen...@gmail.com <mailto:joaomen...@gmail.com>
>
> --
> You received this message because you are subscribed to the Google
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swupdate+u...@googlegroups.com
> <mailto:swupdate+u...@googlegroups.com>.
> To post to this group, send email to swup...@googlegroups.com
> <mailto:swup...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

Joao Mendes

unread,
Feb 16, 2017, 9:57:16 AM2/16/17
to Stefano Babic, swup...@googlegroups.com
First, sorry to bother you again with questions.

I encounter problems in the build with yocto.


When I run bitbake swupdate-image I get an error Nothing RPROVIDES 'libconfig' (but /../meta-swupdate/recipes-extended/images/swupdate-image.bb RDEPENDS on or otherwise requires it).


I have download libconfig-1.5.tar.gz found on www.hyperrealm.com/libconfig because the version on github have errors when I run make. I have run make and make install without problem.


Also when I try to make swupdate I get mtd/libmtd.h: No such file or directory. I have installed all requirements listed on the documentation.

I have installed the packages from the repositories, is that ok?

I have tried to fix these problems but I didn't manage to do it.

Thank you again for your help.



>
> --
> You received this message because you are subscribed to the Google
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send

> To post to this group, send email to swup...@googlegroups.com

> For more options, visit https://groups.google.com/d/optout.

Stefano Babic

unread,
Feb 16, 2017, 10:02:50 AM2/16/17
to Joao Mendes, Stefano Babic, swup...@googlegroups.com
Hi Joao,

On 16/02/2017 15:57, Joao Mendes wrote:
> First, sorry to bother you again with questions.
>
> I encounter problems in the build with yocto.
>
>
> When I run *bitbake swupdate-image *I get an error *Nothing RPROVIDES
> 'libconfig' (but
> /../meta-swupdate/recipes-extended/images/swupdate-image.bb
> <http://swupdate-image.bb> RDEPENDS on or otherwise requires it).
>
>
> *
> I have download *libconfig-1.5.tar.gz* found on

No, wrong.

Some packages are supplied by other layers. libconfig is in
meta-openembedded. You should add meta-openembedded/meta-oe to your
bblayers.conf.


> *www.hyperrealm.com/libconfig <http://www.hyperrealm.com/libconfig>*
> because the version on github have errors when I run make. I have run
> make and make install without problem.
>
>
> Also when I try to make swupdate I get *mtd/libmtd.h: No such file or
> directory. *I have installed all requirements listed on the documentation.

Have you disabled MTD on your configuration ?

This files are installed here:
recipes-devtools/mtd/mtd-utils_%.bbappend

>
> I have installed the packages from the repositories, is that ok?

I do not know what you mind, but I think it is ok :-D

>
> I have tried to fix these problems but I didn't manage to do it.
>
> Thank you again for your help.
>

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

Joao Mendes

unread,
Feb 17, 2017, 12:01:47 PM2/17/17
to Stefano Babic, swup...@googlegroups.com
Thank you for the attention you give to my problems

I got another issue.

This time I got Nothing PROVIDES 'u-boot-fw-utils' (../recipes-support/swupdate/swupdate_2016.10.bb DEPENDS).

I tried to find a solution in the ML but I didn't found it.

I have this meta-layers: meta, meta-poky, meta-yocto, meta-yocto-bsp, meta-intel, meta-oe, meta-swupdate

Did I need others? Or install other packages?


It's really blurry in my mind because I will use grub so do I need u-boot?


When I build with yocto and meta-swupdate this will produce directly an .swu file or I still have to make the CPIO archive?


Also my hardware use an Intel Atom C2550 so I use MACHINE= "genericx86_64" wich is the closest but in the documentation
this was not mentioned has supported device could it be a problem?


Can I make my own layer and build it an then make a swu file with it? This will work or not?


For the mtd/libmtd.h: No such file or directory is in my recovery system when I try to have a swupdate executable to make the install

of my image. I have install all requirements but I still have this issue. In my configuration I have CONFIG_MTD=y.

When I mean I have installed all requirements from repositories I mean I have done the installation like

sudo apt install mtd-utils, I say that because I have found on github vamanea/mtd-utils and I wasn't sure if I have got the good version.


Thank you for helping me this may seem easy for you but for me it is really complex and I think I still have questions after so Thanks again.

Stefano Babic

unread,
Feb 17, 2017, 12:08:37 PM2/17/17
to Joao Mendes, Stefano Babic, swup...@googlegroups.com
On 17/02/2017 18:01, Joao Mendes wrote:
> Thank you for the attention you give to my problems
>
> I got another issue.
>
> This time I got *Nothing PROVIDES 'u-boot-fw-utils'
> (../recipes-support/swupdate/swupdate_2016.10.bb
> <http://swupdate_2016.10.bb> DEPENDS)*.
>
> I tried to find a solution in the ML but I didn't found it.
>
> I have this meta-layers: meta, meta-poky, meta-yocto, meta-yocto-bsp,
> meta-intel, meta-oe, meta-swupdate
>
> Did I need others? Or install other packages?
>
>
> It's really blurry in my mind because I will use grub so do I need u-boot?

Go into meta-swupdate and drop u-boot-fw-utils from DEPENDS in swupdate
recipe.

>
>
> When I build with yocto and meta-swupdate this will produce directly an
> .swu file or I still have to make the CPIO archive?

A .swu is generated, but you have to write a recipe inheriting the
swupdate class and providing your sw-description file.

>
>
> Also my hardware use an Intel Atom C2550 so I use MACHINE=
> "genericx86_64" wich is the closest but in the documentation
> this was not mentioned has supported device could it be a problem?

You should write your own.

>
>
> Can I make my own layer and build it an then make a swu file with it?
> This will work or not?

Yes, it will work

>
>
> For the *mtd/libmtd.h: No such file or directory* is in my recovery
> system when I try to have a swupdate executable to make the install
>
> of my image. I have install all requirements but I still have this
> issue. In my configuration I have CONFIG_MTD=y.
>

As you have an Intel board like a PC, you can disable MTD at all

> When I mean I have installed all requirements from repositories I mean I
> have done the installation like
>
> sudo apt install mtd-utils,

Sorry, this has nothing to do. Yocto builds without using distro file.

> I say that because I have found on github
> vamanea/mtd-utils and I wasn't sure if I have got the good version.
>

Do not waste your time scanning github. Yocto is self contained and it
downloads from the right repos. In meta/ you will find the recipe for
mtd and other stuff.

>
> Thank you for helping me this may seem easy for you but for me it is
> really complex and I think I still have questions after so Thanks again.

You're welcome

Best regards,
Stefano

>
>
> 2017-02-16 16:02 GMT+01:00 Stefano Babic <sba...@denx.de
> <mailto:sba...@denx.de>>:
>
> Hi Joao,
>
> On 16/02/2017 15:57, Joao Mendes wrote:
> > First, sorry to bother you again with questions.
> >
> > I encounter problems in the build with yocto.
> >
> >
> > When I run *bitbake swupdate-image *I get an error *Nothing RPROVIDES
> > 'libconfig' (but
> > /../meta-swupdate/recipes-extended/images/swupdate-image.bb
> <http://swupdate-image.bb>
> > <http://swupdate-image.bb> RDEPENDS on or otherwise requires it).
> >
> >
> > *
> > I have download *libconfig-1.5.tar.gz* found on
>
> No, wrong.
>
> Some packages are supplied by other layers. libconfig is in
> meta-openembedded. You should add meta-openembedded/meta-oe to your
> bblayers.conf.
>
>
> > *www.hyperrealm.com/libconfig
> <http://www.hyperrealm.com/libconfig>
> <http://www.hyperrealm.com/libconfig
> <http://www.hyperrealm.com/libconfig>>*
> > because the version on github have errors when I run make. I have run
> > make and make install without problem.
> >
> >
> > Also when I try to make swupdate I get *mtd/libmtd.h: No such file or
> > directory. *I have installed all requirements listed on the
> documentation.
>
> Have you disabled MTD on your configuration ?
>
> This files are installed here:
> recipes-devtools/mtd/mtd-utils_%.bbappend
>
> >
> > I have installed the packages from the repositories, is that ok?
>
> I do not know what you mind, but I think it is ok :-D
>
> >
> > I have tried to fix these problems but I didn't manage to do it.
> >
> > Thank you again for your help.
> >
>
> Best regards,
> Stefano Babic
>
> --
> =====================================================================
> DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-53 <tel:%2B49-8142-66989-53> Fax:
> +49-8142-66989-80 <tel:%2B49-8142-66989-80> Email: sba...@denx.de
> <mailto:sba...@denx.de>
> =====================================================================
>
>
>
>
> --
> Joao Mendes
> Chemin des Semailles 9C
> 1212 Grand-Lancy
> +41 79 707 95 22
> joaomen...@gmail.com <mailto:joaomen...@gmail.com>
>
> --
> You received this message because you are subscribed to the Google
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swupdate+u...@googlegroups.com
> <mailto:swupdate+u...@googlegroups.com>.
> To post to this group, send email to swup...@googlegroups.com
> <mailto:swup...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


--
Meet DENX at the Embedded World Trade Show
14 Mar - 16 Mar 2017, Nuremberg Trade Fair Centre, Hall 4, Booth 581

Joao Mendes

unread,
Feb 17, 2017, 12:45:31 PM2/17/17
to Stefano Babic, swup...@googlegroups.com
> Go into meta-swupdate and drop u-boot-fw-utils from DEPENDS in swupdate recipe.

I have tried to remove this lines from recipes-support/swupdate/swupdate.inc

if 'CONFIG_UBOOT=y' in features: depends = ...

but then I got an error do_compile failed with exit code '1'

Is that the wrong file to edit or just another issue?



>
> --
> You received this message because you are subscribed to the Google
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send

> To post to this group, send email to swup...@googlegroups.com

> For more options, visit https://groups.google.com/d/optout.


--
Meet DENX at the Embedded World Trade Show
14 Mar - 16 Mar 2017, Nuremberg Trade Fair Centre, Hall 4, Booth 581
--
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

Stefano Babic

unread,
Feb 18, 2017, 1:57:59 PM2/18/17
to Joao Mendes, Stefano Babic, swup...@googlegroups.com
On 17/02/2017 18:45, Joao Mendes wrote:
>> Go into meta-swupdate and drop u-boot-fw-utils from DEPENDS in
> swupdate recipe.
>
> I have tried to remove this lines from recipes-support/swupdate/swupdate.inc
>
> if 'CONFIG_UBOOT=y' in features: depends = ...
>
> but then I got an error*do_compile failed with exit code '1'
>
> *
> Is that the wrong file to edit or just another issue?
>

It is telling you that you are changing in the wrong place. The line
checks if UBOOT is needed, and u-boot-fw-utils is automatically removed
if CONFIG_UBOOT is not set. So run "bitbake -c menuconfig swupdate",
drop UBOTT support (from the handler), and try again to build.

Best regards,
Stefano Babic

--
Reply all
Reply to author
Forward
0 new messages