Allwinner pack tools adventures

[Ivan Kozic]

Hi all,

It seems that no one really cares about AW pack tools enough to document at least a part of it. By Allwinner pack tools I mean various Linux/Windows proprietary SW coming from AW, like "dragon", "update_mbr", "update_23" etc.

Since I'm on a good way to figure a part of this out, I'd like to add it to the linux-sunxi.org if possible, so that everyone can use it, and not lose days and days finding various SDKs and BSPs only to figure that half of it is not working...

Actually the biggest problem is that none of this is documented and these programs usually do not give you any version or help. And there are at least 5 different "dragon" versions I've tried - some of them work with image.cfg + sys_partition.fex, while some of them only seem to use more advanced image.cfg format and not really needing sys_partition.fex. I've also noticed that there are different versions of update_mbr for A10 and A20 - one seems to only put softw311 marker, while the other one works with softw411 marker. Also some of them can pack by only following symbolic links to rootfs, while others actually need rootfs.fex in the same folder. Mixes in SDKs can include anything - I've found combos like 'new dragon + old update_mbr', which mess up always and produce unreadable images... sunxi-bsp set of tools on the other hand looks very broken to me, as there is no A20 support per say - there are script files for A20 boards, but allwinner-tools used by sunxi-bsp only produce images suited for A10 (softw311) and maybe A13 (these are the only two in the eFex configuration files)...

I would also like to point out that most info regarding MBR structure is now well known (for A20 - boot1 source code contains wboot_mbr.h header which has the packed MBR layout), so I think update_mbr can already be deprecated and changed to a better suited open-source version. If I had any +time,  I'd happily write this out and try it...

As I said, wiki pages on linux-sunxi are just making things worse by not really explaining any of this and yet somehow making everything even more complicated (just look at Software section under http://linux-sunxi.org/LiveSuit_images - I have no idea what is the point of that section).
So I'd just like to know how I can (or if I may in fact) change/append to linux-sunxi.org?

Cheers!

[Olliver Schinagl]

Hey Ivan,

Just create a wiki account, ask for it to be promoted from robot to human (via mail or IRC) and start editing :)

Cheers!

--
You received this message because you are subscribed to the Google Groups "linux-sunxi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Piotr Król]

On Thu, Sep 03, 2015 at 02:01:10AM -0700, Ivan Kozic wrote:
> Hi all,
>
> It seems that no one really cares about AW pack tools enough to document at
> least a part of it. By Allwinner pack tools I mean various Linux/Windows
> proprietary SW coming from AW, like "dragon", "update_mbr", "update_23" etc.
>
> Since I'm on a good way to figure a part of this out, I'd like to add it to
> the linux-sunxi.org if possible, so that everyone can use it, and not lose
> days and days finding various SDKs and BSPs only to figure that half of it
> is not working...
>

Hi Ivan,
Last week I went through exactly that path to create working LiveSuit image
for my Cubietruck. I asked about some things that are not clear to me
here:

https://groups.google.com/d/msg/linux-sunxi/N301MZXe6AM/zLUgZgwkCQAJ

and here:

http://www.cubieforums.com/index.php/topic,3887.msg24181.html#msg24181

but not much interest in this topic.

> Actually the biggest problem is that none of this is documented and these
> programs usually do not give you any version or help. And there are at
> least 5 different "dragon" versions I've tried - some of them work with
> image.cfg + sys_partition.fex, while some of them only seem to use more
> advanced image.cfg format and not really needing sys_partition.fex. I've
> also noticed that there are different versions of update_mbr for A10 and
> A20 - one seems to only put softw311 marker, while the other one works with
> softw411 marker. Also some of them can pack by only following symbolic
> links to rootfs, while others actually need rootfs.fex in the same folder.
> Mixes in SDKs can include anything - I've found combos like 'new dragon +
> old update_mbr', which mess up always and produce unreadable images...
> sunxi-bsp set of tools on the other hand looks very broken to me, as there
> is no A20 support per say - there are script files for A20 boards, but
> allwinner-tools used by sunxi-bsp only produce images suited for A10
> (softw311) and maybe A13 (these are the only two in the eFex configuration
> files)...
>

Agree. sunxi-bsp doesn't work for A20. I extracted some parts from
dl.cubieboard.org image but this cause:

update mbr: partcount = 4
update mbr file ok
fsbuild error. The size if filesystem must more than 855k now is 128
fsbuild failed 338

> I would also like to point out that most info regarding MBR structure is
> now well known (for A20 - boot1 source code contains wboot_mbr.h header
> which has the packed MBR layout), so I think update_mbr can already be
> deprecated and changed to a better suited open-source version. If I had any
> +time, I'd happily write this out and try it...
>
> As I said, wiki pages on linux-sunxi are just making things worse by not
> really explaining any of this and yet somehow making everything even more
> complicated (just look at Software section under
> http://linux-sunxi.org/LiveSuit_images - I have no idea what is the point
> of that section).
> So I'd just like to know how I can (or if I may in fact) change/append to
> linux-sunxi.org?
>

I'm also interested in supporting this effort. Can offer testing on
Cubietruck and writing some articles.

Thanks,
--
Piotr Król
3mdeb - Embedded Systems Consulting
Burgaska 9D/10
80-287 Gdansk POLAND
tel: +48880673344
http://3mdeb.com | @3mdeb_com

[Piotr Król]

On Thu, Sep 03, 2015 at 02:01:10AM -0700, Ivan Kozic wrote:

> Hi all,
>
> It seems that no one really cares about AW pack tools enough to document at
> least a part of it. By Allwinner pack tools I mean various Linux/Windows
> proprietary SW coming from AW, like "dragon", "update_mbr", "update_23" etc.
>
> Since I'm on a good way to figure a part of this out, I'd like to add it to
> the linux-sunxi.org if possible, so that everyone can use it, and not lose
> days and days finding various SDKs and BSPs only to figure that half of it
> is not working...

Hi Ivan,
Last week I went through call of duty to create working LiveSuit image

for my Cubietruck. I asked about some things that are not clear to me
here:

https://groups.google.com/d/msg/linux-sunxi/N301MZXe6AM/zLUgZgwkCQAJ

and here:

http://www.cubieforums.com/index.php/topic,3887.msg24181.html#msg24181

but not much interest in this topic.

>

> Actually the biggest problem is that none of this is documented and these
> programs usually do not give you any version or help. And there are at
> least 5 different "dragon" versions I've tried - some of them work with
> image.cfg + sys_partition.fex, while some of them only seem to use more
> advanced image.cfg format and not really needing sys_partition.fex. I've
> also noticed that there are different versions of update_mbr for A10 and
> A20 - one seems to only put softw311 marker, while the other one works with
> softw411 marker. Also some of them can pack by only following symbolic
> links to rootfs, while others actually need rootfs.fex in the same folder.
> Mixes in SDKs can include anything - I've found combos like 'new dragon +
> old update_mbr', which mess up always and produce unreadable images...
> sunxi-bsp set of tools on the other hand looks very broken to me, as there
> is no A20 support per say - there are script files for A20 boards, but
> allwinner-tools used by sunxi-bsp only produce images suited for A10
> (softw311) and maybe A13 (these are the only two in the eFex configuration
> files)...
>

Agree. sunxi-bsp doesn't work for A20. I extracted some parts from
dl.cubieboard.org image but this cause:

update mbr: partcount = 4
update mbr file ok
fsbuild error. The size if filesystem must more than 855k now is 128
fsbuild failed 338

> I would also like to point out that most info regarding MBR structure is
> now well known (for A20 - boot1 source code contains wboot_mbr.h header
> which has the packed MBR layout), so I think update_mbr can already be
> deprecated and changed to a better suited open-source version. If I had any
> +time, I'd happily write this out and try it...
>
> As I said, wiki pages on linux-sunxi are just making things worse by not
> really explaining any of this and yet somehow making everything even more
> complicated (just look at Software section under
> http://linux-sunxi.org/LiveSuit_images - I have no idea what is the point
> of that section).
> So I'd just like to know how I can (or if I may in fact) change/append to
> linux-sunxi.org?
>

[Ivan Kozic]

Hi Piotr,

I've just registered as Oliver suggested (thanks Oliver), although it seems I will need some time before I can manage to write something useful. I'm currently deep into this and it's not so complicated - problem is that everything is somehow crazy half-redundant (code, comments, files...) and it just takes one question from the side to get you off-track...

Anyway, I've made a "bootable" image. I can't say for sure, but I think I'm using modified version of the A20 SDK from dl.cubieboard.org. I checked it in quite some time ago to our SVN - in fact I think I wrote here about it as well... Nevermind, I'll post more when it's booting correctly.

I think I'll have some more infos later today, or at least I hope. Off to play with *.AXF files...

Cheers!

[Ivan Kozic]

Quick status update:

Ok, so code and scripts and tools are a mess... Complete mess, there are 3x bootfs folders and some builds go to one, some to the other. Some don't go where they're supposed to go at all, and pack scripts don't seem to be aware of anything. So let's start. Your best bet is actually A20_SDK_20130319.tar.gz. Don't ask me why - I found it to be the friendliest to build (even though it ain't friendly at all). It's missing some stuff, mainly the build toolchain which should be under
/lichee/boot/config/gcc-linaro but this whole directory is missing. Just take it from another SDK like I did, or make some symlinks to what you have already.

From this SDK, I'm using pack script found under /lichee/boot/pack/ - there's also one more at /lichee/tools/pack, but that one is a bit less friendly as far as I remember. Due to various issues (dragon version, missing rootfs links...), these scripts will fail building the image - but I have almost made it work by making some very unorthodox mods :) Anyway, I'll explain what I did once I'm finished and when it's all polished, but I guess it's not a bad idea for everyone to download this SDK and get to know it a bit before I make the official how-to. It's now very dirty with some ugly absolute paths and really not useable.

What's quite important about this is that Boot0/1, *.AXF files and configs all need to be rebuilt and updated with whatever board config. The pristine files coming with this SDK are not useable (original boot.axf binary is booting to math_test(), as this was built from Boot1's BurnQC app which doesn't really look like it's doing anything - don't really know what this QC is used for...). Anyway boot.axf and sprite.axf both need to be rebuilt from sources. I've also modified and rebuilt the drv_de driver, since I had different GPIOs for LCD than the original file...

If you're wondering what the heck I'm talking about - boot0 loads boot1 which loads its "application". There are several different applications depending on where you're booting from, what do you want to do etc (not really clear for me either). For normal NAND boot flow, boot1 will load boot.axf (called also Boot_Android in sources) which will in turn load all the drivers and start u-boot. If you're booting from PhoenixCard, it will load sprite.axf (Card_Android in sources) and start with the firmware update.

So... maybe now everything is a bit clearer - I'll clear the scripts/code and then post back, but it could take a while...

Cheers!

[Luc Verhaegen]

On Thu, Sep 03, 2015 at 05:47:58AM -0700, Ivan Kozic wrote:
> Quick status update:
>
> Ok, so code and scripts and tools are a mess... Complete mess, there are 3x
> bootfs folders and some builds go to one, some to the other. Some don't go
> where they're supposed to go at all, and pack scripts don't seem to be
> aware of anything. So let's start. Your best bet is actually
> A20_SDK_20130319.tar.gz

> <http://dl.linux-sunxi.org/SDK/A20/A20_SDK_20130319.tar.gz>. Don't ask me

> why - I found it to be the friendliest to build (even though it ain't
> friendly at all). It's missing some stuff, mainly the build toolchain which
> should be under

> */lichee/boot/config/gcc-linaro
> <http://dl.linux-sunxi.org/SDK/A20/A20_SDK_20130319.tar.gz>*but this whole

> directory is missing. Just take it from another SDK like I did, or make
> some symlinks to what you have already.

> <http://dl.linux-sunxi.org/SDK/A20/A20_SDK_20130319.tar.gz>
>
> >From this SDK, I'm using pack script found under */lichee/boot/pack/ *-

Our wiki is just that, a wiki. Meant to be edited easily and often.

Instead of writing your insights into emails, you can also dump them
straight into the wiki, as you go along, and massage those into
something more useful and structured as you go along and afterwards, as
you see fit.

Luc Verhaegen.

[Ivan Kozic]

Hi Luc,

Ok, that would be one of the questions actually - I didn't know if anyone would complain about it being not so structured, as I've never updated any public wiki pages, only company and private... If it can hang there a bit dirty and unorganized then I'd actually prefer to do it directly, as this indeed seems to be only piling up.

So I'm understanding this as you've just given me permit to do so :)
Thanks Luc and cheers!

[Thomas Kaiser]

Ivan Kozic wrote:

I didn't know if anyone would complain about it being not so structured, as I've never updated any public wiki pages

You could start summarizing/editing the stuff below http://linux-sunxi.org/index.php?title=User_talk:Ikozic&action=edit&redlink=1 as long as it's early "work in progress" and then move it to where it belongs to if you feel you might want feedback/corrections from others.

[Ivan Kozic]

Hi Thomas,

This is a very nice feature - however, I've already started editing :( I've added some stuff to LiveSuit Images page, although it's just a start, still no useful info.
Tomorrow I'm going to vacation, so it will probably stay untouched until the next Monday (maybe some smaller additions today), but then I'll fill it in further.

Regarding progress:
I was talking to Piotr - he will try out some SDKs as well, so hopefully we'll have something useful which others can use in 2-3 weeks. I have several issues which maybe others can help with by some moderate brainstorming:

1. LiveSuit (3.06, x64 Linux) bad image issue (but PhoenixCard working like a charm, no issues whatsoever) - I'm kind of suspecting dragon tool; maybe the new Livesuit expects something that this relatively old dragon doesn't do... Could it be that?
2. PhoenixSuit (1.10, Windows) - working, but flashing SD-Card instead of NAND. If there's no SD-Card it will fail. This is quite funny - there's probably a script in my image which tells PhoenixSuit to do so (cardscript.fex? cardtool.fex?), but I have no idea which - I'm pretty sure it has nothing to do with sys_partition.fex, but maybe it could have something to do with image.cfg (directly or indirectly). Either that or cardscript.fex but I cannot find anything meaningful there.

So if anyone's got any pointers about this, please share!
Cheers!

[Ivan Kozic]

Hi all,

Quick progress update:

LiveSuit:
I have fixed the error regarding Linux LiveSuit, there were some fex tool files missing in the image, now it behaves the same like PhoenixSuit in Windows (trying to update SD-Card instead of NAND), but at least images are not invalid anymore.

Second issue stays for now - the way I see it, most of download/flashing stuff is actually in these card folders where fex tools are. So far I have tried several different versions of these tools:
- original SDK versions (both from /boot/pack/chips/sun7i/eFex and /tools/pack/chips/sun7i/eFex; yes, they are different (?!?)) - these are Fes version 097.5,
- extracted from Olimex Android image for Olinuxino Micro A20 - these are Fes version 098,
- "new" SDK A20  versions (not really sure if really new, but newer than original) - these are again the same 097.5.

I have also tried cardscript and cardtool from both versions and all of these attempts were unsuccessful - LiveSuit/PhoenixSuit is always trying to update SD Card instead of NAND... ("Hello SDCARD register" instead of "Hello NAND register" in FES UART output). Getting a bit frustrating honestly...

I'd like to try out images that do work (extract and use their tools), so:
@Piotr:
Can you give me a link to a known working cubie image that you were using (original one)? The one which worked for NAND out of the box. Also, please tell me how exactly did you extract it (image_repacker?). Thanks!

Cheers!

[Piotr Król]

Hi Ivan,
image that works for me is:
http://dl.cubieboard.org/software/a20-cubietruck/debian/debieez/ct-debian-nand.img.gz

To change it I used:
http://forum.xda-developers.com/showpost.php?p=28329544&postcount=1

Simply:
./imgrepacker ct-debian-nand.img
to extract and:
./imgrepacker ct-debian-nand.img.dump
to pack.

This image contain a lot of fex files. I mainly changed
bootloader.fex.iso and rootfs.fex.iso.

[Ivan Kozic]

Hi Piotr,

Thanks, I'll try it out straight away.

--
You received this message because you are subscribed to a topic in the Google Groups "linux-sunxi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/linux-sunxi/IoOEbf-X11E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to linux-sunxi...@googlegroups.com.

[Ivan Kozic]

Hi Piotr,

Well, I'm officially ashamed :) Of course, culprit was something stupid that I took for granted from the start - I've accidentally took sys_config.fex for SD-Card... So basically, these binary utilities for download/burn (all those fel/fes fex files) actually first read script.bin (or sys_config.fex in binary format) - [target] section has the following item:

storage_type = 0

This was set to '1' in my sys_config (0=NAND, 1=SD)... Now my image works as it should in LiveSuit :) I've just flashed it by using LiveSuit in Linux. But...

Since the first boot failed, I also remembered how badly I want to remove 'boot' partition with ANDROID! magic (actually a RAW bImage of Android kernel, no fs whatsoever). This one has to go since I do not want to download that partition. It is also a bit dangerous, since if something fails with the bootloader FAT partition, I've seen several times that it boots this ancient kernel (I've just copied /dev/nandb from Olimex like a year ago and been using it in this format ever since - maybe not the best idea I've had ;) ).

In fact I only want to have bootloader and rootfs, maybe one more user partition. If anyone's interested in removing 'boot' - just comment out the check in SPL (it's in boot.axf 'app'):

boot/boot1/apps/Boot_Android/BootOS/BootOS.c:
- if (memcmp(fb_hdr->magic, FASTBOOT_BOOT_MAGIC, 8)) {
-         __inf("boot1: bad boot image magic, maybe not a boot.img?\n");
-         return -1;
-     }

That's it. Say byebye to the 'boot' magic.

I still have to prep some stuff before I leave, but I'll try to update the wiki page or at least give you pointers on how to do it afterwards. This whole thing along with 'boot' partition removal should go there, since I've heard tons of complaints about this, no one using Linux really wants it there.

So, let's continue afterwards - I'll let you know of my progress.

Cheers!

[Dragan Gregurec]

Hi Ivan,

did you find any solutions in meantime, because I have the same problem with Allwinner A20 pack/unpack tools.

Best regards,

Dragan Gregurec

[Dragan Gregurec]

Hi Ivan,

I have the same problem as you with packager/unpackager tool for Allwinner A20, and did you find right software.

Best regards,

Dragan Gregurec

Dana četvrtak, 3. rujna 2015. u 11:01:10 UTC+2, korisnik Ivan Kozic napisao je:

[Sergey Lapin]

Hi all!

have anybody managed to make converter from allwinner images to dd-able images?

Thanks!

--

You received this message because you are subscribed to the Google Groups "linux-sunxi" group.

To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi+unsubscribe@googlegroups.com.