Looking for a test suite to test my implementation; and a few comments

[Benito van der Zander]

Hi,
is there a JSONiq test suite available somewhere, like the XQTS?

Since I have implemented some of JSONiq in my Xidel / Internet Tools and a test suite would be really useful to see,
if there are any missing corner cases...


And some comments to the specification:

jn:decode-from-roundtrip(jn:encode-for-roundtrip( X )) does not return X, if X already has the format of a roundtrip encoding.
E.g. jn:decode-from-roundtrip(jn:encode-for-roundtrip( jn:encode-for-roundtrip( X ) ))  is X, but should be jn:encode-for-roundtrip( X )
    
That makes it kind of useless for actual roundtripping. Might even become a security issue, if there are user defined strings involved.
    
    
    
Similarly xs:integer(2), xs:decimal(2) and xs:double(2) all serialize to 2 in the normal, not-roundtrip serialization, wouldn't it be better to serialize them to 2, 2.0 and 2E0, to keep the type information intact?



is-null(()) is a type error, Is that intended? Then you cannot even write is-null(null), with disabled literals...
    

And I think it would be nice to have a members-like function that behaves like members for arrays and like the identity function for everything else. Makes it easier to handle formats, where multiple values are stored as {"foo": [1,2,3]} and single values as {"foo": 4} ...
(i.e  like: typeswitch ($x) case array() return members($x) default return $x
       

Although, why is jn:members even needed?
An array could be automatically cast to a sequence, whereever a sequence is needed.
After all a sequence is already auto converted to an array (when used as property in a serialized object),
and an array cannot contain a sequence (cf. [1 to 10] ), so it would not cause ambiguity.

        
There also quite a few actual errors:

In Chapter 5. Navigation in JSON content
    If the base expression $s is a sequence of objects, or a sequence of arrays, then $sequence($params) is ...
    =>  That does not seem to allow a mixed sequence of (objects and arrays).    

In 5.2. Array selectors:
    "thursday"" ]
    
In 6.1. fn:boolean (aka Effective Boolean Value):
    EBV ( [1], jn:null() )  
    => EBV is supposed to be an 1-arity function

In Example 6.9. Retrieving the size of an array
    returns
    => that has the wrong formatting (should not be part of the code)
    
In 6.10. jn:keys:
    for $key in json:keys($map)
    => that namespace prefix is not defined
    
In 6.18. Changes to general comparison semantics:
    Example 6.11. Addition
    => that example has nothing to do with addition
   
 
In 6.13. jn:object:
    return jn:object($object1, $object2)
    => this calls a arity-2 object function, which does not exist
    
    
In 8.1. libjn:accumulate :
    for $distinct-key := distinct-values($all-keys)
    
In 8.2. libjn:descendant-objects :
    for $v in libjn:members($i)
    => that should be a jn: function
    
In 8.3. libjn:descendant-pairs :
    jn:descendant-pairs($o)("first")   
    => that should be an libjn: function
    Result: { "first" : 1 }, { "first" : "a" }         
    => No, the accessor only returns the value, so the result is "1 a"
    
In 8.4. libjn:flatten:
    for $value in libjn:values($a)
    => values is not defined for arrays. Should probably be jn:members
    
In 8.6. libjn:project:
    "First Officer" : "Spock",
    => there should not be a comma
    Result: ()
    => actually returns {}, since it always calls jn:object
     
    
Benito
   

[Matthias Brantner]

Benito

It's great to hear that you are implementing JSONiq. We really appreciate your

feedback.

Unfortunately, there is no JSONiq test suite, yet. However, the are tests in Zorba which we

created during the implementation. They are at least a good start. You can

find 

- the queries at http://bazaar.launchpad.net/~zorba-coders/zorba/trunk/files/head:/test/rbkt/Queries/zorba/jsoniq/ and

- the expected results for the queries that are not expecting an error at http://bazaar.launchpad.net/~zorba-coders/zorba/trunk/files/head:/test/rbkt/ExpQueryResults/zorba/jsoniq/

We will send you a more detailed answer regarding your comments later. Also,

we will fix the errors that you have found.

Best regards

Matthias

--
You received this message because you are subscribed to the Google Groups "JSONiq" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jsoniq+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Benito van der Zander]

Hi,

However, the are tests in Zorba which we

created during the implementation. They are at least a good start. You can

find 

Cool, they even helped to find some bugs

But unfortunately they are really Zorba specific ("http://www.zorba-xquery.com/modules/store/dynamic/collections/ddl", jn:flatten, jn:member (without s), variable without declare, jsoniq-strip-top-level-array, {} as empty sequence ...).

We will send you a more detailed answer regarding your comments later.

And have you decided on the issues?

Especially for encode/decode-for-roundtrip, as I'm not sure if it makes sense to implement that function in its current state at all.


Also the casting behaviour is not really consistent, an array/object casted as string throws an exception, but null becomes "null".

I would prefer it to become the respective JSON serialization as string. (use case: have a function that performs a http request and accepts the post data as string. Autoserialization makes it easier to use it with JSON apis)


Benito

[Ghislain]

Hi Benito,

Again thanks for your useful comments. Most typos should be fixed now (in the JSONiq extension to XQuery spec). I commented below, please let me know if I have missed anything.

Many thanks and kind regards,
Ghislain


On roundtripping: I will take it to the team.

> is-null(()) is a type error, Is that intended? Then you cannot even write is-null(null), with disabled literals...

The signature could be changed to is-null(item()?) as xs:boolean?. I will take it to the team.

> And I think it would be nice to have a members-like function that behaves like members for arrays and like the identity function for everything else. Makes it easier to handle formats, where multiple values are stored as {"foo": [1,2,3]} and single values as {"foo": 4} ...
> (i.e like: typeswitch ($x) case array() return members($x) default return $x

I will take it to the team. If you have a more elaborate use case, it could help motivating the introduction of such a function.

> Although, why is jn:members even needed?
> An array could be automatically cast to a sequence, whereever a sequence is needed.
> After all a sequence is already auto converted to an array (when used as property in a serialized object),
> and an array cannot contain a sequence (cf. [1 to 10] ), so it would not cause ambiguity.

This is a good question, and this was discussed, as well as other solutions. Every time, for various reasons (too complicated or unintuitive, singleton sequence singularities, ...) we fell back to the current solution (arrays are items, no auto-unboxing).

> There also quite a few actual errors:
>
> In Chapter 5. Navigation in JSON content
> If the base expression $s is a sequence of objects, or a sequence of arrays, then $sequence($params) is ...
> => That does not seem to allow a mixed sequence of (objects and arrays).

I am not sure this is an error. Are there use cases for an object+array lookup (which would involve casting for either the formers or the latters)? I will take it to the team.

>
>
> In 5.2. Array selectors:
> "thursday"" ]

Fixed.

>
> In 6.1. fn:boolean (aka Effective Boolean Value):
> EBV ( [1], jn:null() )
> => EBV is supposed to be an 1-arity function

Fixed.

>
> In Example 6.9. Retrieving the size of an array
> returns
> => that has the wrong formatting (should not be part of the code)

Fixed.

>
> In 6.10. jn:keys:
> for $key in json:keys($map)
> => that namespace prefix is not defined

Fixed.

>
> In 6.18. Changes to general comparison semantics:
> Example 6.11. Addition
> => that example has nothing to do with addition

Fixed.

>
>
> In 6.13. jn:object:
> return jn:object($object1, $object2)
> => this calls a arity-2 object function, which does not exist

Fixed.

>
>
> In 8.1. libjn:accumulate :
> for $distinct-key := distinct-values($all-keys)

Fixed.

>
> In 8.2. libjn:descendant-objects :
> for $v in libjn:members($i)
> => that should be a jn: function

Fixed.

>
> In 8.3. libjn:descendant-pairs :
> jn:descendant-pairs($o)("first")
> => that should be an libjn: function

Fixed.

> Result: { "first" : 1 }, { "first" : "a" }
> => No, the accessor only returns the value, so the result is "1 a"

Fixed.

>
> In 8.4. libjn:flatten:
> for $value in libjn:values($a)
> => values is not defined for arrays. Should probably be jn:members

Fixed.

>
> In 8.6. libjn:project:
> "First Officer" : "Spock",
> => there should not be a comma

Fixed.

> Result: ()
> => actually returns {}, since it always calls jn:object

Fixed.

--
Dr. Ghislain Fourny
Software Architect

28msec Inc.

http://www.28msec.com
http://twitter.com/28msec

[Benito van der Zander]

Hi,

> The signature could be changed to is-null(item()?) as xs:boolean?. I will take it to the team.
>

Or alternatively: is-null(item()?) as xs:boolean
with is-null(()) = false ?

>> And I think it would be nice to have a members-like function that behaves like members for arrays and like the identity function for everything else. Makes it easier to handle formats, where multiple values are stored as {"foo": [1,2,3]} and single values as {"foo": 4} ...
>> (i.e like: typeswitch ($x) case array() return members($x) default return $x
> I will take it to the team. If you have a more elaborate use case, it could help motivating the introduction of such a function.
>

The idea was that you need to access a webservice which returns json
data, and only uses arrays of at least size 2.
I have heard some systems behave like that, because they pass their data
to an serialization library, which serializes it as single value for
single values, and array for multiple values.
But I do not have a specific example.

Although JSONiq itself might create such data, if someone writes
{"result": (... query ... ) }.
Then it can either becomes {"result": "foobar"} or {"result": [1,2,3]},
and whoever processes the data, needs to unwrap the array first,
depending on its type

> In Chapter 5. Navigation in JSON content
> If the base expression $s is a sequence of objects, or a sequence of arrays, then $sequence($params) is ...
> => That does not seem to allow a mixed sequence of (objects and arrays).
> I am not sure this is an error. Are there use cases for an object+array lookup (which would involve casting for either the formers or the latters)? I will take it to the team.

I think I tried it in the Zorba live demo, where it allowed mixing
objects and arrays back then.

Although now it throws a casting exception





How relevant is the JSOniq extension anyways now that the is the JSONiq
standalone version?

Will the new features in the standalone version (like the dot object
property selector) be ported to the extension version?

Can you write

let $obj := {"a": 123} return $obj.a

(without spaces at the side of the dot? or only with spaces? on which
side? I have implemented something like $obj.a without spaces a long
time ago)


Benito

[Ghislain]

Hi Benito,

Thanks for your answer. We will take a look at your suggestions.

Regarding JSONiq core vs. JSONiq extension to XQuery : both are important. The semantics and data model are the same, only the syntax (i.e., the parser code) differs. The bottom line is that there are syntactic collisions between XQuery and JSONiq, i.e., between XML support and JSON support.

If you are working primarily with JSON data (nice syntax), the JSONiq syntax is the way to go.
If you are working primarily with XML (nice syntax), but also need to process some JSON (a slightly less nice syntax), then XQuery (+ the JSONiq extension to XQuery) is the way to go.

Both syntaxes are relatively stable. In particular, the dot lookup will not be ported to the "JSONiq extension to XQuery" syntax, because it collides with the fact that dots are allowed in XML names.

JSONiq core syntax:

jsoniq version "1.0";

let $obj := { a : 123} return $obj.a

XQuery + JSONiq extension to XQuery syntax:

xquery version "3.0";
let $obj := { "a" : 123 } return $obj("a") (: $obj.a would be a variable named "obj.a" :)

I hope this makes sense?

Kind regards,
Ghislain