Swupdate NAND and SD-card update

[Isak Lichtenstein]

Hi Stefano,

Sorry for bugging you again today.

Our custom board is based on a BeagleBone, but does have a NAND flash device. Additionally our system has also a SD-card. For the moment, my system runs from this flash and swupdate is installed as a component on this system.

As mentioned previously, I'm trying to update let's assume u-boot on this nand. When I start swupdate, it shows that it contains a flashhandler (see output below ) and the code looks like it supports NOR and NAND. But in the documentation only NOR is mentioned.

- What is the reality?
- Could you please give me an example how to define such an image update in the sw-description file for NOR and NAND?
- If the previous points respond with true for NAND support, would it be enough for me to add a type description in sw-description and remove the automatic mapping for image to ubi, as you mentioned in the previous post?
> The rule in the libconfig-parser is to associate a handler with the main tag: images --> ubi, files --> raw, uboot --> uboot handler. Maybe this can be changed in future.image maps


In the meantime, I tried to update a file on our SD-card, but still without success. Here the different files and log outputs.

Here is the console output showing the existence of my NAND-flash (deleted output of most devices. Actually mtd0-4 belong to a not configured SPI-Flash, while mtd5-12 are NAND-flash) and SD-card
root@myboard-evb:~# fdisk -l

Disk /dev/mtdblock0: 0 MB, 65536 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/mtdblock0 doesn't contain a valid partition table

...

Disk /dev/mmcblk0: 2002 MB, 2002780160 bytes
255 heads, 63 sectors/track, 243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/mmcblk0p1 * 1 9 72261 c Win95 FAT32 (LBA)
/dev/mmcblk0p2 11 125 923737+ 83 Linux
/dev/mmcblk0p3 127 243 939802+ 83 Linux


Here is part of dmesg from mounting our NAND:
[ 0.929443] Creating 5 MTD partitions on "spi_flash":
[ 0.934722] 0x000000000000-0x000000020000 : "SPL"
[ 0.939636] mtd: partition "SPL" extends beyond the end of device "spi_flash" -- size truncated to 0x10000
[ 0.950775] 0x000000010000-0x000000050000 : "U-Boot"
[ 0.955993] mtd: partition "U-Boot" is out of reach -- disabled
[ 0.963134] 0x000000000000-0x000000002000 : "U-Boot Env"
[ 0.968688] mtd: partition "U-Boot Env" doesn't end on an erase block -- force read-only
[ 0.978027] 0x000000002000-0x000000382000 : "Kernel"
[ 0.983215] mtd: partition "Kernel" extends beyond the end of device "spi_flash" -- size truncated to 0xe000
[ 0.993499] mtd: partition "Kernel" doesn't start on an erase block boundary -- force read-only
[ 1.003417] 0x000000010000-0x000000010000 : "File System"
[ 1.009063] mtd: partition "File System" is out of reach -- disabled
[ 1.016876] omap2-nand driver initializing
[ 1.021423] ONFI flash detected
[ 1.024871] ONFI param page 0 valid
[ 1.028503] NAND device: Manufacturer ID: 0x2c, Chip ID: 0xda (Micron MT29F2G08ABAEAWP)
[ 1.036865] Creating 8 MTD partitions on "omap2-nand.0":
[ 1.042419] 0x000000000000-0x000000020000 : "SPL"
[ 1.048339] 0x000000020000-0x000000040000 : "SPL.backup1"
[ 1.054931] 0x000000040000-0x000000060000 : "SPL.backup2"
[ 1.061492] 0x000000060000-0x000000080000 : "SPL.backup3"
[ 1.068115] 0x000000080000-0x000000260000 : "U-Boot"
[ 1.075012] 0x000000260000-0x000000280000 : "U-Boot Env"
[ 1.081542] 0x000000280000-0x000000780000 : "Kernel"
[ 1.089691] 0x000000780000-0x000010000000 : "File System"
[ 1.199340] UBI: attaching mtd12 to ubi0
[ 1.203430] UBI: physical eraseblock size: 131072 bytes (128 KiB)
[ 1.210021] UBI: logical eraseblock size: 126976 bytes
[ 1.215637] UBI: smallest flash I/O unit: 2048
[ 1.220550] UBI: sub-page size: 512
[ 1.225372] UBI: VID header offset: 2048 (aligned 2048)
[ 1.231628] UBI: data offset: 4096
[ 2.634216] UBI: max. sequence number: 2
[ 2.653045] UBI: attached mtd12 to ubi0
[ 2.657104] UBI: MTD device name: "File System"
[ 2.662841] UBI: MTD device size: 248 MiB
[ 2.668029] UBI: number of good PEBs: 1988
[ 2.672912] UBI: number of bad PEBs: 0
[ 2.677581] UBI: number of corrupted PEBs: 0
[ 2.682220] UBI: max. allowed volumes: 128
[ 2.687042] UBI: wear-leveling threshold: 4096
[ 2.691925] UBI: number of internal volumes: 1
[ 2.696563] UBI: number of user volumes: 1
[ 2.701202] UBI: available PEBs: 0
[ 2.705841] UBI: total number of reserved PEBs: 1988
[ 2.710998] UBI: number of PEBs reserved for bad PEB handling: 19
[ 2.717376] UBI: max/mean erase counter: 1/0
[ 2.721832] UBI: image sequence number: 541244709

This is my sw-description file for updating ( in this case only adding ) a kernel image to the SD-card:
//sw-description
software =
{
version = "0.1.0";

hardware-compatibility: [ "1.0", "1.2", "1.3"];

files: (
{
filename = "uImage-myboard-evb.bin";
path = "/";
device = "/dev/mmcblk0p1";
filesystem = "vfat";
}
);
}

I created my .swu file with this script:
// createSWU script
#! /bin/bash

CONTAINER_VER="0.2"
PRODUCT_NAME="my-software"
FILES="sw-description uImage-myboard-evb.bin"
for i in $FILES;do
echo $i;done | cpio -ov -H crc > ${PRODUCT_NAME}_${CONTAINER_VER}.swu

and here is the output fron running the previous script:
// generating swu file
lic@Dev:~/swupdate$ ./createSWU
sw-description
uImage-myboard-evb.bin
6912 blocks

Now when running sw-update and uploading the generated files, this is the log ( I inserted my questions with --> )

root@myboard-evb:~# swupdate -v -w "-document_root /www/pages/swupdate"
Swupdate v0.1.0.git
Licensed under GPLv2. See source distribution for detailed copyright notices.

Registered handlers:
flash
raw
rawfile
ubivol
ubipartition
lua
uboot
Mongoose web server v. 3.8 started on port(s) 8080 with web root [/www/pages/swupdate]
X_FILENAME: my-software_0.2.swu lenght: 3538944
[NOTIFY] : Recovery running : [network_thread] : Incoming network request: processing...
[NOTIFY] : Recovery started : Software Update started !

Cannot parse config file: No such file or directory
Error: environment not initialized

--> What are these errors?

[NOTIFY] : Recovery running : [extract_files] : Found file:
filename sw-description
size 356
Version 0.1.0
[NOTIFY] : Recovery running : [parse_hw_compatibility] : Accepted Hw Revision : 1.0

[NOTIFY] : Recovery running : [parse_hw_compatibility] : Accepted Hw Revision : 1.2

[NOTIFY] : Recovery running : [parse_hw_compatibility] : Accepted Hw Revision : 1.3

[NOTIFY] : Recovery running : [parse_files] : Found File: uImage-myboard-evb.bin --> / (/dev/mmcblk0p1)

[NOTIFY] : Recovery running : [extract_files] : Found file:
filename uImage-myboard-evb.bin
size 3537808 required

[NOTIFY] : Recovery running : [[15068.581298] UBI: attaching mtd0 to ubi1
network_initiali[15068.586730] UBI: physical eraseblock size: 32768 bytes (32 KiB)
zer] : Valid ima[15068.594482] UBI: logical eraseblock size: 32640 bytes
ge found: copyin[15068.601409] UBI: smallest flash I/O unit: 1
g to FLASH
[NOT[15068.607452] UBI: VID header offset: 64 (aligned 64)
IFY] : Recovery [15068.614715] UBI: data offset: 128
running : Installation in progr[15068.623260] UBI: max. sequence number: 0
ess
[15068.628417] UBI error: ubi_read_volume_table: the layout volume was not found
[NOTIFY] : Recovery running : [scan_ubi_partitions] : cannot attach mtd0 - maybe not a NAND or raw device
Floating point exception
--> Why is swupdate trying to mount mtd0 at all? And if so, why as UBI? I'm trying to write a file to /dev/mmcblk0p1?

Best regards

Isak


Isak Lichtenstein
Entwicklungsingenieur Hardware & SW

Kistler Instrumente AG
Eulachstrasse 22, 8408 Winterthur, Switzerland
Direct+41 52 2241 963, Main Office+41 52 224 11 11
isak.lic...@kistler.com, www.kistler.com





Confidentiality Notice:
This e-mail is privileged and confidential and for the use of the addressee only. Should you have received this e-mail in error please notify us by replying directly to the sender or by sending a message to in...@kistler.com. Unauthorised dissemination, disclosure or copying of the contents of this e-mail, or any similar action, is prohibited.



-----Ursprüngliche Nachricht-----
Von: swup...@googlegroups.com [mailto:swup...@googlegroups.com] Im Auftrag von Stefano Babic
Gesendet: 19 March 2014 13:05
An: Lichtenstein Isak; 'Stefano Babic'
Cc: Detlev Zundel; 'swup...@googlegroups.com'
Betreff: Re: AW: [swupdate] General questions

Hi Isak,

On 19/03/2014 12:35, Isak Lichtenstein wrote:
> Hi Stefano,
>
> Thank you very much for your quick reply.
>
> I'm trying to get swupdate running on our system based on a TI
> beaglebone using yocto 1.5.
>

Nice.

> For the moment I'm simply trying out swupdate to see if this could be
> a solution for our firmware update.

I think it is the best solution - having a fast prototype, and checking what it is possible and if it fills your requirement.

> Therefore I've simply added
> swupdate to our image and I'm not trying to generate a minimal image.

I agree with your approach. It is enough for an evaluation.

>
> I've got now the following questions:

> - In the documentation it's
> mentioned that U-boot should define CONFIG_ENV_OFFSET_REDUND. Is this
> a recommendation or a must?

It is a strong recommendation if you update U-Boot variables.

> As much as I understood this is simply to make the use of swupdate
> safer, as there exists a backup environment if something happens
> during a write of the u-boot environment. Is this correct?

Exactly. It is not needed if you do not plan to update the environment.
However, allowing to update the environment is a nice feature. I have also in plan to implement a sort of "black list" for U-Boot environemnt, avoiding that swupdate tries to change variables that must be fixed as planned in the factory. But anyway, this is now only on a "wish list", thare is current no code to support it.

- Same question for CONFIG_BOOTCOUNTER_LIMIT ?

Yes. I wanted to explain how I am using it he project.
CONFIG_BOOTCOUNTER_LIMIT is in case an image is correctly installed, but this new software has in a bug, that was not discovered in test, and happens in field. The system does not complete the whole initialization.

CONFIG_BOOTCOUNTER_LIMIT has only sense if the kernel support it, that means it supports a bootcounter. This feature is not mainline, I know.

> - In
> order to evaluate Swupdate, I'm trying to imitate the firmware
> "update" after production, which would install U-boot, Swupdate-image,
> kernel and root-FS on our NAND. When I upload a .swu file containing
> the updates, swupdate always tries to mount the partition as UBI.

On Beaglebone you have only eMMC, I think. You have to use the raw handler, thought to copy directly into a device without any intermediate layer. I use it generally for updating SD-Card, but it should be the right one in your case.

> When checking the code, I saw that for images the type is
> automatically set to "ubi".

Generally this is what I wanted and I used it. You should use "files"
instead of "images", and then the raw handler is taken.

Anyway, I appreciate your point. The rule in the libconfig-parser is to associate a handler with the main tag: images --> ubi, files --> raw, uboot --> uboot handler. Maybe this can be changed in future.

> But in your external parser
> example, the xml sets the uboot binary type as "uboot". Would my test
> work if I enabled the external parser?

No, the external parser is thought to let the user able to set his own rules instead of relying on JSON-like libconfig. For this reason, the user can write the external parser in LUA, having full rights to set the rules he wants. Of course, what you think it is possible, but I think it should be enough using "files".

> Or if I modified the code the
> set the type to "uboot" instead of "ubi"?

Type "uboot" is only for environment. swupdate does not know and *must* not know what is stored in a specific location -else we lose flexibility. It is not important *what* must be installed, but *how*.
That means that swupdate must only have the right handler, and it does not care if the source is u-boot, kernel, rootfs, or whatever.

>
> - If we decide to use Swupdate for our firmware update, could we
> outsource it to you (or Denx ) to implement it for us?

Of course we as Denx can help you. You can contact Detlev (in CC) for more information.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de =====================================================================

--
You received this message because you are subscribed to the Google Groups "swupdate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swupdate+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Stefano Babic]

Hi Isak,

On 19/03/2014 15:52, Isak Lichtenstein wrote:
>
> Hi Stefano,
>

> Sorry for bugging you again today.
>
> Our custom board is based on a BeagleBone, but does have a NAND flash
> device. Additionally our system has also a SD-card. For the moment,
> my system runs from this flash and swupdate is installed as a
> component on this system.
>
> As mentioned previously, I'm trying to update let's assume u-boot on
> this nand. When I start swupdate, it shows that it contains a
> flashhandler (see output below ) and the code looks like it supports
> NOR and NAND. But in the documentation only NOR is mentioned.
>

I reused my code from a previous project for NOR flashes. Code is
partially taken from mtd-utils, with handling for NOR flashes. I only
mentioned NOR flashes because I never used NAND flashes in raw mode
without UBI. It should be tested if it works on raw NAND (another item
added to the TO-DO list !).

However, I am reading again the code in the flash handler. Ther eis no
management of bad blocks. For NAND, I must at least check during the
erase/write procedure if a sector is bad, that is if the NAND controller
reported unrecoverable ECC errors. This is not done in the current code.
IMHO, if we need support for NAND in raw mode, we should add the same
mechanism as we see in U-Boot for nandwrite, skipping the bad blocks. I
will also reuse part the libmtd library, because most of the routines
are already implemented and the library is linked to swupdate.

> - What is the reality?

Simply that I never used NAND flash in raw mode. Quite all my recent
projects have NANDs with UBI volumes.

> - Could you please give me an example how to
> define such an image update in the sw-description file for NOR and
> NAND?
- If the previous points respond with true for NAND support,
> would it be enough for me to add a type description in sw-description
> and remove the automatic mapping for image to ubi, as you mentioned
> in the previous post?

This is not true - I do not see code to write into the NAND in raw mode.
In my personal view (it is good that I can share different views),
writing NAND in raw mode is only important if I need to update U-Boot. I
have not this requirement in the past, and it is not yet supported.

>
> This is my sw-description file for updating ( in this case only
> adding ) a kernel image to the SD-card: //sw-description software =
> { version = "0.1.0";
>
> hardware-compatibility: [ "1.0", "1.2", "1.3"];
>
> files: ( { filename = "uImage-myboard-evb.bin"; path = "/"; device =
> "/dev/mmcblk0p1"; filesystem = "vfat"; } ); }

The path must be complete, not only the directory name.

Something like:

files: (
{
filename = "uImage-myboard-evb.bin";

path = "/uImage-myboard-evb.bin";

device = "/dev/mmcblk0p1";
filesystem = "vfat"
}


>

> Now when running sw-update and uploading the generated files, this is
> the log ( I inserted my questions with --> )
>
> root@myboard-evb:~# swupdate -v -w "-document_root
> /www/pages/swupdate" Swupdate v0.1.0.git Licensed under GPLv2. See
> source distribution for detailed copyright notices.
>
> Registered handlers: flash raw rawfile ubivol ubipartition lua uboot
> Mongoose web server v. 3.8 started on port(s) 8080 with web root
> [/www/pages/swupdate] X_FILENAME: my-software_0.2.swu lenght:
> 3538944 [NOTIFY] : Recovery running : [network_thread] : Incoming
> network request: processing... [NOTIFY] : Recovery started :
> Software Update started !
>
> Cannot parse config file: No such file or directory Error:
> environment not initialized
>
> --> What are these errors?

They are both for the U-Boot environment. To update U-Boot environment,
swupdate must know where it is stored and at which offset. It uses the
same routines as fw_setenv/fw_printenv in U-Boot.

Exactly as the tools in U-Boot, a file /tmp/fw_env.cfg is parsed. You
can consider these output as warnings if you do not want to update the
environemnt. However, if sw-description contains a U-Boot script for
variables, swupdate stops with an error when it tries to run the U-Boot
handler.

>
> [NOTIFY] : Recovery running : [extract_files] : Found file: filename
> sw-description size 356 Version 0.1.0 [NOTIFY] : Recovery running :
> [parse_hw_compatibility] : Accepted Hw Revision : 1.0
>
> [NOTIFY] : Recovery running : [parse_hw_compatibility] : Accepted Hw
> Revision : 1.2
>
> [NOTIFY] : Recovery running : [parse_hw_compatibility] : Accepted Hw
> Revision : 1.3
>
> [NOTIFY] : Recovery running : [parse_files] : Found File:
> uImage-myboard-evb.bin --> / (/dev/mmcblk0p1)
>
> [NOTIFY] : Recovery running : [extract_files] : Found file: filename
> uImage-myboard-evb.bin size 3537808 required
>
> [NOTIFY] : Recovery running : [[15068.581298] UBI: attaching mtd0 to
> ubi1 network_initiali[15068.586730] UBI: physical eraseblock size:
> 32768 bytes (32 KiB) zer] : Valid ima[15068.594482] UBI: logical
> eraseblock size: 32640 bytes ge found: copyin[15068.601409] UBI:
> smallest flash I/O unit: 1 g to FLASH [NOT[15068.607452] UBI: VID
> header offset: 64 (aligned 64) IFY] : Recovery
> [15068.614715] UBI: data offset: 128 running :
> Installation in progr[15068.623260] UBI: max. sequence number:
> 0 ess [15068.628417] UBI error: ubi_read_volume_table: the layout
> volume was not found [NOTIFY] : Recovery running :
> [scan_ubi_partitions] : cannot attach mtd0 - maybe not a NAND or raw
> device Floating point exception --> Why is swupdate trying to mount
> mtd0 at all?

swupdate finds the layout reading directly from the flashes. It checks
for all flashes and search for UBI volumes. If they are not found, it
should not matter and the internal map should be empty. However, I see a
"device Floating point exception". This looks like a bug to be fixed.

> And if so, why as UBI? I'm trying to write a file to
> /dev/mmcblk0p1?

UBI is scanned at the beginning to get the layout. We could consider to
add a CONFIG_ to disable it if UBI is not wanted at all.

Best regards,
Stefano