json parser

[Barbier, Renaud]

In the past, I have tested a software collection using libconfig with no issue.

I now use a JSON version of this file generated from a template through a python script.

The sw-description file is read back correctly through the python command line.

However, swupdate sees an error

software set: stable mode: live
/tmp/sw-description:1 - syntax error
/tmp/sw-description 1 syntax error [NOTIFY] : SWUPDATE failed [0] ERROR : ..exi
ting


n = rm927rc
n = hardware-compatibility
[NOTIFY] : SWUPDATE failed [0] ERROR : HW compatibility not found


Using the function json_object_to_json_string_ext, the json object is turned back (json_tokener_parse) into a json string to the console.
Using this ouput and the website https://jsonformatter.curiousconcept.com/, the json string is verified correct.

Any ideas of what I could investigate next?

Cheers,
Renaud

[Stefano Babic]

Hi Renaud,

On 30/01/2017 19:56, Barbier, Renaud wrote:
> In the past, I have tested a software collection using libconfig with no issue.
>
> I now use a JSON version of this file generated from a template through a python script.
>
> The sw-description file is read back correctly through the python command line.
>
> However, swupdate sees an error
>
> software set: stable mode: live
> /tmp/sw-description:1 - syntax error
> /tmp/sw-description 1 syntax error [NOTIFY] : SWUPDATE failed [0] ERROR : ..exi
> ting

Really SWUpdate iterates through the parsers until one of them reports
no error. I guess this error is reported by the libconfig parser, that
cannot understand JSON syntax.

Anyway, after the first parser fails, it goes through to the next one
(JSON).

It is questionable if this behaviour is correct. It has never disturb
me, but I confess, I use in 95% of my projects libconfig.

>
>
> n = rm927rc
> n = hardware-compatibility
> [NOTIFY] : SWUPDATE failed [0] ERROR : HW compatibility not found

This is the real error: it is not syntax error, but hardware/software
compatibility is not verified.

>
>
> Using the function json_object_to_json_string_ext, the json object is turned back (json_tokener_parse) into a json string to the console.
> Using this ouput and the website https://jsonformatter.curiousconcept.com/, the json string is verified correct.
>
> Any ideas of what I could investigate next?
>

I cannot say more without your sw-description. You could also run
swupdate -c -i <your swu> just to check the correctness of the
description. Try to disable the hw-sw compatibility check to verify if
this, as I suppose, the cause of the error.

Best regards,
Stefano Babic


--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

[Barbier, Renaud]

The json output below is obtained with "str = json_object_to_json_string_ext(cfg,JSON_C_TO_STRING_PRETTY);"


The issue seems to be that I have too many nodes in json while this work with the libconfig version.
I updated json_object *find_json_recursive_node with:

const char *first = "software";

if (json_object_object_get_ex(node, first, &onode)) {
printf("GOT FIRST NODE: %s\n", first);
node = onode;
}

And could skip over the software node to start the search. All errors have gone.

So after parsing the json string should the first node be skipped or is my json file not has you intended?

json string=
{
"software":{
"version":"ow_mimas-6_0_1-RC3",
"rm927rc":{
"hardware-compatibility":[
"0"
],
"stable":{
"live":{
"files":[
{
"path":"\/tmp\/mnt\/primary\/kernel.bin",
"sha256":"ffe2c8ab5cc94fc6a3c347b23986cc1249e81b811cb39c4742cc9e0a173b4292",
"name":"kernel live",
"filename":"kernel.bin"
},
{
"path":"\/tmp\/mnt\/primary\/dtbx.dtb",
"sha256":"67565d41f8ac7a07128e11a6bfc44e24df5b75c68b235cf47042daa59224ebce",
"name":"dtb live",
"filename":"dtbx.dtb"
}
],
"images":[
{
"volume":"fs-active",
"sha256":"d60d7435790e215e995d330b2e7e5418c2009ec817e462fb2c3715b61ca074b9",
"type":"ubivol",
"name":"nand",
"filename":"nand.ubifs"
}
],
"scripts":[
{
"sha256":"e6f253116d0bea2234f6ea2e1eca2734cb5001ff6ec8f6106e5b8fd8d895b78a",
"type":"preinstall",
"filename":"preinstall_live"
},
{
"sha256":"f2cc785adefcac8be5d4a13a9e74c41c0e411270cd6cbd1becfd353502a1957b",
"type":"postinstall",
"filename":"postinstall_live"
}
],
"partitions":[
{
"mtdname":"nand",
"name":"fs-active",
"size":"134217728"
}
]
},
"factory":{
"files":[
{
"path":"\/tmp\/mnt\/secondary\/kernel.bin",
"sha256":"ffe2c8ab5cc94fc6a3c347b23986cc1249e81b811cb39c4742cc9e0a173b4292",
"name":"kernel factory",
"filename":"kernel.bin"
},
{
"path":"\/tmp\/mnt\/secondary\/dtbx_fac.dtb",
"sha256":"67565d41f8ac7a07128e11a6bfc44e24df5b75c68b235cf47042daa59224ebce",
"name":"dtb factory",
"filename":"dtbx_fac.dtb"
}
],
"images":[
{
"volume":"fs-redundant",
"sha256":"d60d7435790e215e995d330b2e7e5418c2009ec817e462fb2c3715b61ca074b9",
"type":"ubivol",
"name":"nand",
"filename":"nand.ubifs"
}
],
"scripts":[
{
"sha256":"e63c3370df5bcc79f5503e3041ffae151d6ceb08792cbab22a5af1388f69ca2b",
"type":"preinstall",
"filename":"preinstall_fac"
},
{
"sha256":"b1a96f95222366d6c23c34d67088af54024f8f49cf6a21937766b2f5
e789bc6e",
"type":"postinstall",
"filename":"postinstall_fac"
}
],
"partitions":[
{
"mtdname":"nand",
"name":"fs-redundant",
"size":"134217728"
}
]
}
}
},
"name":"Openware Upgrade Package"
}
}
...
[NOTIFY] : SWUPDATE failed [0] ERROR : Found nothing to install

[NOTIFY] : SWUPDATE failed [0] ERROR : no parser available to parse sw-description!

[NOTIFY] : SWUPDATE failed [0] ERROR : failed to parse sw-description!

[Stefano Babic]

Hi Renaud,

On 31/01/2017 15:57, Barbier, Renaud wrote:
> The json output below is obtained with "str = json_object_to_json_string_ext(cfg,JSON_C_TO_STRING_PRETTY);"
>
>
> The issue seems to be that I have too many nodes in json while this work with the libconfig version.
> I updated json_object *find_json_recursive_node with:
>
> const char *first = "software";
>
> if (json_object_object_get_ex(node, first, &onode)) {
> printf("GOT FIRST NODE: %s\n", first);
> node = onode;
> }
>

Quite - just the "software" entry is not proofed by the JSOIN parser.

I can parse without error this slightly modified file:

{

"hardware-compatibility":[
"0"
],

"files":[
{
"path":"\/tmp\/mnt\/secondary\/kernel.bin",

"sha256":"ffe2c8ab5cc94fc6a3c347b23986cc1249e81b811cb39c4742cc9e0a173b4292",
"name":"kernel factory",
"filename":"kernel.bin"
},
{
"path":"\/tmp\/mnt\/secondary\/dtbx_fac.dtb",

"sha256":"67565d41f8ac7a07128e11a6bfc44e24df5b75c68b235cf47042daa59224ebce",
"name":"dtb factory",
"filename":"dtbx_fac.dtb"
}
],
"images":[
{
"volume":"fs-redundant",

"sha256":"d60d7435790e215e995d330b2e7e5418c2009ec817e462fb2c3715b61ca074b9",
"type":"ubivol",
"name":"nand",
"filename":"nand.ubifs"
}
],
"scripts":[
{

"sha256":"e63c3370df5bcc79f5503e3041ffae151d6ceb08792cbab22a5af1388f69ca2b",
"type":"preinstall",
"filename":"preinstall_fac"
},
{

"sha256":"b1a96f95222366d6c23c34d67088af54024f8f49cf6a21937766b2f5e789bc6e",

"type":"postinstall",
"filename":"postinstall_fac"
}
],
"partitions":[
{
"mtdname":"nand",
"name":"fs-redundant",
"size":"134217728"
}
]
}
}
}
}

The output I get from a .swu with just the sw-description is:

./swupdate -v -c -Hrm927rc:0 -e stable,factory -i tests/ML/software_1.0.swu
Swupdate v2017.1.0-rc1

Licensed under GPLv2. See source distribution for detailed copyright
notices.

Searching for custom LUA handlers :[string "require
("swupdate_handlers")"]:1: module 'swupdate_handlers' not found:
no field package.preload['swupdate_handlers']
no file '/usr/local/share/lua/5.2/swupdate_handlers.lua'
no file '/usr/local/share/lua/5.2/swupdate_handlers/init.lua'
no file '/usr/local/lib/lua/5.2/swupdate_handlers.lua'
no file '/usr/local/lib/lua/5.2/swupdate_handlers/init.lua'
no file '/usr/share/lua/5.2/swupdate_handlers.lua'
no file '/usr/share/lua/5.2/swupdate_handlers/init.lua'
no file './swupdate_handlers.lua'
no file '/usr/local/lib/lua/5.2/swupdate_handlers.so'
no file '/usr/lib/x86_64-linux-gnu/lua/5.2/swupdate_handlers.so'
no file '/usr/lib/lua/5.2/swupdate_handlers.so'
no file '/usr/local/lib/lua/5.2/loadall.so'
no file './swupdate_handlers.so'
Custom handlers not found, no error, skipping...

Running on rm927rc Revision 0
Registered handlers:
archive
tar
flash
flash-hamming1
raw
rawfile
ubivol
ubipartition
lua
shellscript
preinstall
postinstall
remote
uboot
software set: stable mode: factory
[NOTIFY] : SWUPDATE running : [extract_sw_description] : Found file:
filename sw-description
size 3059
checksum 0x2c3c9 VERIFIED

[NOTIFY] : SWUPDATE running : [parse_images] : Found Image nand :
nand.ubifs in volume : fs-redundant for handler ubivol

[NOTIFY] : SWUPDATE running : [parse_scripts] : Found Script:
postinstall_fac

[NOTIFY] : SWUPDATE running : [parse_scripts] : Found Script:
preinstall_fac

[NOTIFY] : SWUPDATE running : [parse_files] : Found File kernel factory
: kernel.bin --> /tmp/mnt/secondary/kernel.bin (ROOTFS)

[NOTIFY] : SWUPDATE running : [parse_files] : Found File dtb factory :
dtbx_fac.dtb --> /tmp/mnt/secondary/dtbx_fac.dtb (ROOTFS)

[NOTIFY] : SWUPDATE failed [0] ERROR parser/parser.c : parse_partitions
: 237 : Partition incompleted in description file
[NOTIFY] : SWUPDATE running : [check_hw_compatibility] : Hardware
rm927rc Revision: 0
[NOTIFY] : SWUPDATE failed [0] ERROR core/swupdate.c : install_from_file
: 337 : SW not compatible with hardware


This time the file is parsed, no error and all entries are parsed.

[Barbier, Renaud]

> -----Original Message-----
> : [check_hw_compatibility] : Hardware rm927rc Revision: 0 [NOTIFY] :
> SWUPDATE failed [0] ERROR core/swupdate.c : install_from_file
> : 337 : SW not compatible with hardware
>
>
> This time the file is parsed, no error and all entries are parsed.
>
> Best regards,
> Stefano Babic

What is the way forward?
Should find_node_json be modified and consider the root node "software"?
I have done a modification of this function and all the nodes are found.
I could submit a patch later.


Unhappily, GET_FIELD_STRING(p, hw, NULL, s); in parse_hw_compatibility fails to retrieve the value "0"
even though " count = get_array_length(p, setting);" returns a count of 1
The length is found to be 0(strlen(s)).

Which explains the error at the end (that we both see in our test)

"SW not compatible with hardware "

Will have a further look on this tomorrow.

[Stefano Babic]

On 31/01/2017 18:45, Barbier, Renaud wrote:
>
>
>> -----Original Message-----
>> : [check_hw_compatibility] : Hardware rm927rc Revision: 0 [NOTIFY] :
>> SWUPDATE failed [0] ERROR core/swupdate.c : install_from_file
>> : 337 : SW not compatible with hardware
>>
>>
>> This time the file is parsed, no error and all entries are parsed.
>>
>> Best regards,
>> Stefano Babic
>
> What is the way forward?
> Should find_node_json be modified and consider the root node "software"?
> I have done a modification of this function and all the nodes are found.
> I could submit a patch later.

Then the "root" node should be configurable as for libconfig. And
LIBCONFIGROOT should be unified, having just one for both parsers.

Or documentation should be updated telling that JSON does not have such
root nofe as libconfig.

>
>
> Unhappily, GET_FIELD_STRING(p, hw, NULL, s); in parse_hw_compatibility fails to retrieve the value "0"
> even though " count = get_array_length(p, setting);" returns a count of 1
> The length is found to be 0(strlen(s)).

>
> Which explains the error at the end (that we both see in our test)
> "SW not compatible with hardware "
>
> Will have a further look on this tomorrow.
>