tar.gz of rootfs

[Brendan Simon (eTRIX)]

Hi Stefano,

The documentation says:

    "support for compressed images, using the zlib library. tarball (tgz file) are supported."

What is meant by compressed "images" ?

Is it a disk image (e.g. from dd) that is tar-gzipped, or is it a directory containing the rootfs that is tar-gzipped ??

I'm hoping that latter.  that it gets un-gzipped and un-tarred into a mounted directory.  Yes/No ??

Thanks, Brendan.

[Stefano Babic]

Hi Brendan,

Both - there is support for disk images in compressed format. They are
decompressed on the fly by swupdate and written into a partition / disk.

What you are asking, it is to install a tarball - this is supported by
the "archive" handler, that takes a tarball (compressed or not) and
install it into a device that swupdate will mount.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

[Brendan Simon (eTRIX)]

Hi Stefano

On 4/12/2015 7:36 PM, Stefano Babic wrote:

Hi Brendan,

Both - there is support for disk images in compressed format. They are
decompressed on the fly by swupdate and written into a partition / disk.

What you are asking, it is to install a tarball - this is supported by
the "archive" handler, that takes a tarball (compressed or not) and
install it into a device that swupdate will mount.

I didn't find too much info on the archive handler, but I did notice that invoking swupdate with no parameters listed handlers, which included "tar" and "archive".

My next big issue is fw_env.config.  I don't know much about it and it seems difficult to get a working one.
fw_printenv continually gives me CRC error and uses the defaults.

I'm using a Digilent ZedBoard with a Xilinx Zynq 7020 SoC.  The kernel is from Analog Devices and is a fork of 4.0.0 (according to the Makefile).

Here is info from /proc/mtd and my /etc/fw_env.config.

root@bjs:~# cat /proc/mtd                                                                                                 
dev:    size   erasesize  name
mtd0: 00500000 00010000 "boot"
mtd1: 00020000 00010000 "bootenv"
mtd2: 00020000 00010000 "config"
mtd3: 00a80000 00010000 "image"
mtd4: 01040000 00010000 "spare"

root@bjs:~# cat /etc/fw_env.config                                                                                                 
# Configuration file for fw_(printenv/setenv) utility.
# Up to two entries are valid, in this case the redundant
# environment sector is assumed present.
# MTD device name       Device offset   Env. size       Flash sector size
#         /dev/mtd1               0x0000          0x20000         0x40000
#         /dev/mtd1               0x0000          0x20000         0x20000
#         /dev/mtd1               0x0000          0x20000         0x10000
#         /dev/mtd2               0x0000          0x20000         0x40000
#         /dev/mtd2               0x0000          0x20000         0x20000
         /dev/mtd2               0x0000          0x20000         0x10000

Have you got any insight into what the correct fw_env.config settings should be from the above info?

Thanks,
Brendan.

[Stefano Babic]

Hi Brendan,

On 05/12/2015 22:22, Brendan Simon (eTRIX) wrote:
> Hi Stefano
>
> On 4/12/2015 7:36 PM, Stefano Babic wrote:
>> Hi Brendan,
>>
>> Both - there is support for disk images in compressed format. They are
>> decompressed on the fly by swupdate and written into a partition / disk.
>>
>> What you are asking, it is to install a tarball - this is supported by
>> the "archive" handler, that takes a tarball (compressed or not) and
>> install it into a device that swupdate will mount.
>
> I didn't find too much info on the archive handler, but I did notice
> that invoking swupdate with no parameters listed handlers, which
> included "tar" and "archive".
>
> My next big issue is fw_env.config. I don't know much about it and it
> seems difficult to get a working one.
> fw_printenv continually gives me CRC error and uses the defaults.

This is an issue related to U-Boot, not swupdate. You get a better
support asking to U-Boot ML.

You have to check into U-Boot sources, not kernel. You have to find
where U-Boot and the environment is, check if it is mapped to a mtd, and
set the fw_env.config. You cannot find the right values just looking
into the kernel.

I guess that there is not a mtd for the environmnent, because mtd1 is
very large and U-Boot cannot be so big. I can then just guess that the
environment is in the mtd1, maybe at the same offset as most Zync board
(0xE0000). You have to check this, and better rearrange the MTD setup to
assign an own mtd to the environment.

[Brendan Simon (eTRIX)]

Hi Stefano

On 7/12/2015 7:11 PM, Stefano Babic wrote:

On 05/12/2015 22:22, Brendan Simon (eTRIX) wrote:

My next big issue is fw_env.config.  I don't know much about it and it
seems difficult to get a working one.
fw_printenv continually gives me CRC error and uses the defaults.

This is an issue related to U-Boot, not swupdate. You get a better
support asking to U-Boot ML.

I did email the u-boot ML.  I finally got some feedback -- probably as people were resting over the weekend :)
Wow, there is a lot of noise on the u-boot list with all the patches.  Hard to actually filter out questions and discussion emails.

I will look into the u-boot sources to find out info.

BTW, might be nice to put a small section in the swupdate documents re /etc/fw_env.config and to search u-boot sources to the correct information.

Thanks,
Brendan :)

[Stefano Babic]

Hi Brendan,

On 07/12/2015 11:34, Brendan Simon (eTRIX) wrote:
> Hi Stefano
>
> On 7/12/2015 7:11 PM, Stefano Babic wrote:
>> On 05/12/2015 22:22, Brendan Simon (eTRIX) wrote:
>>> My next big issue is fw_env.config. I don't know much about it and it
>>> seems difficult to get a working one.
>>> fw_printenv continually gives me CRC error and uses the defaults.
>> This is an issue related to U-Boot, not swupdate. You get a better
>> support asking to U-Boot ML.
>
> I did email the u-boot ML. I finally got some feedback -- probably as
> people were resting over the weekend :)
> Wow, there is a lot of noise on the u-boot list with all the patches.
> Hard to actually filter out questions and discussion emails.

swupdate is not so used and it has not a large community as U-Boot. If
you find difficult with U-Boot, try with LKML.

>
> I will look into the u-boot sources to find out info.
>
> BTW, might be nice to put a small section in the swupdate documents re
> /etc/fw_env.config and to search u-boot sources to the correct information.

It can be difficult for me because it looks to me obvious. swupdate is a
FOSS project, so feel free to send a patch to this ML fixing or better
explaining what was not so clear for you.

[Brendan Simon (eTRIX)]

On 4/12/2015 7:36 PM, Stefano Babic wrote:

On 04/12/2015 04:22, Brendan Simon (eTRIX) wrote:

Hi Stefano,

The documentation says:

    "support for compressed images, using the zlib library. tarball (tgz
file) are supported."

What is meant by compressed "images" ?

Is it a disk image (e.g. from dd) that is tar-gzipped, or is it a
directory containing the rootfs that is tar-gzipped ??

I'm hoping that latter.  that it gets un-gzipped and un-tarred into a
mounted directory.  Yes/No ??

Both - there is support for disk images in compressed format. They are
decompressed on the fly by swupdate and written into a partition / disk.

What you are asking, it is to install a tarball - this is supported by
the "archive" handler, that takes a tarball (compressed or not) and
install it into a device that swupdate will mount.

I'm trying the archive handler and getting part of the way (I think), but nothing is being stored in my unused partition.  sw_update gives me some failed notifications and exits.
I'm specifying a device.  e.g. /dev/mmcblk0p3.  Is that ok, or is that only for raw mode disk images?

Here is my sw_description file:

software =
{
        version = "0.1";

        ## differentiate running image modes/sets.
        stable:
        {
                sd_rootfs1:
                {
                        images: (
                        {
                                filename = "rootfs.tar.gz";
                                device = "/dev/mmcblk0p2";
                                type = "archive";
                                #compressed = true;
                        }
                        );
                };

                sd_rootfs2:
                {
                        images: (
                        {
                                filename = "rootfs.tar.gz";
                                device = "/dev/mmcblk0p3";
                                type = "archive";
                                #compressed = true;
                        }
                        );
                };
        };
}

Here is contents of cpio file:

Contents of SWU file: IND-software_0.1.swu
-rw-r--r--   1 brendan  brendan      1458 Dec  7 23:24 sw-description
-rw-r--r--   2 brendan  brendan  92519408 Dec  4 00:25 rootfs.tar.gz
180706 blocks

Here is the output from:

$ sudo /usr/local/sbin/swupdate -v -w "-document_root ./www" -e stable,sd_rootfs2

X_FILENAME: IND-software_0.1.swu lenght: 92522496
[NOTIFY] : SWUPDATE running :  [network_thread] : Incoming network request: proc
essing...
[NOTIFY] : SWUPDATE started :  Software Update started !
[NOTIFY] : SWUPDATE running :  [extract_files] : Found file:
        filename sw-description
        size 2696
Version 0.1
[NOTIFY] : SWUPDATE running :  [parse_images] : Found Image: rootfs.tar.gz in de
vice : /dev/mmcblk0p3 for handler archive

[NOTIFY] : SWUPDATE running :  [extract_files] : Found file:
        filename rootfs.tar.gz
        size 92519408 required
[NOTIFY] : SWUPDATE running :  [network_initializer] : Valid image found: copyin
g to FLASH
[NOTIFY] : SWUPDATE running :  Installation in progress
[NOTIFY] : SWUPDATE running :  [install_images] : Found installer for stream roo
tfs.tar.gz archive
[NOTIFY] : SWUPDATE running :  [install_archive_image] : install_archive_image /
dev/mmcblk0p3 

[NOTIFY] : SWUPDATE running :  [install_images] : Installer for archive not succ
essful !
[NOTIFY] : SWUPDATE failed [1] Installation failed !
[NOTIFY] : SWUPDATE running :  [network_initializer] : Main thread sleep again !
[NOTIFY] : No SWUPDATE running :  Waiting for requests...

I presume my sw_description is not quite right.  Any ideas?

Thanks, Brendan.

[Stefano Babic]

Path is missing. You have to inform swupdate where the tarball is
extracted after mounting the device. See reference with git log:

commit ddabca7e2f673e570d5466b4f465b556c3164309
Author: Stefano Babic <sba...@denx.de>
Date: Sun Jul 19 19:25:26 2015 +0200

handler: add "archive" handler

Add a handler to extract tarballs using the libarchive
library.

Example in sw-description:

files: (
{
filename = "archive.tar.gz";
type = "archive";
/* device is mounted automatically by swupdate */
device = "/dev/mmcblk0p2";
filesystem = "ext3";
/* path where the tarball is extracted */
path = "home/root";
}
);

:

[Brendan Simon (eTRIX)]

On 8/12/2015 12:02 AM, Stefano Babic wrote:

On 07/12/2015 13:26, Brendan Simon (eTRIX) wrote:

On 4/12/2015 7:36 PM, Stefano Babic wrote:

I'm trying the archive handler and getting part of the way (I think), but nothing is being stored in my unused partition. sw_update gives me some failed notifications and exits. I'm specifying a device. e.g. /dev/mmcblk0p3. Is that ok, or is that only for raw mode disk images? Here is my sw_description file: software = { version = "0.1"; ## differentiate running image modes/sets. stable: { sd_rootfs1: { images: ( { filename = "rootfs.tar.gz"; device = "/dev/mmcblk0p2"; type = "archive"; #compressed = true; }

Path is missing. You have to inform swupdate where the tarball is
extracted after mounting the device. See reference with git log:

commit ddabca7e2f673e570d5466b4f465b556c3164309
Author: Stefano Babic <sba...@denx.de>
Date:   Sun Jul 19 19:25:26 2015 +0200

    handler: add "archive" handler

    Add a handler to extract tarballs using the libarchive
    library.

    Example in sw-description:

    files: (
        {
                filename = "archive.tar.gz";
                type = "archive";
                /* device is mounted automatically by swupdate */
                device = "/dev/mmcblk0p2";
                filesystem = "ext3";
                /* path where the tarball is extracted */
                path = "home/root";
        }
    );

OK.  Didn't think to look it git logs for documentation :-/
I was looking here: https://github.com/sbabic/swupdate/blob/master/doc/source/sw-description.rst

Got it unpacking now :)
Is there a setting so that the path can be removed first.  The update just unpacks over the top of what is there?
i.e. rm -rf ${path}/

If not, then I could do it in a preinst script.
I had a quick play and the destination directory is not known.  Is there an env var set for destination directory?

If not, I just have to fix it to /tmp/datadst which would work as it is fixed in swupdate.
It would be better if the dest dir was passed via an env var or some other mechanism, in case it changes in the future, or maybe if two upgrades were happening, etc.

Cheers,
Brendan.

[Stefano Babic]

Hi Brendan,

sw-description must be updated to reflec the new handlers, yes.

>
> Got it unpacking now :)
> Is there a setting so that the path can be removed first. The update
> just unpacks over the top of what is there?
> i.e. rm -rf ${path}/

No. This is specific to a project. Each handler must have no knowledge
about this, else it can work for someone and not for someone else.

>
> If not, then I could do it in a preinst script.
> I had a quick play and the destination directory is not known. Is there
> an env var set for destination directory?

There is not.

>
> If not, I just have to fix it to /tmp/datadst which would work as it is
> fixed in swupdate.
> It would be better if the dest dir was passed via an env var or some
> other mechanism, in case it changes in the future, or maybe if two
> upgrades were happening, etc.

Not sure - everything that is temporary is removed at the end of the
update. There is no conflict when the second update runs.