JSON Schema: ordered properties

[Richard Kennard]

Hi guys,

I am the author of Metawidget (http://metawidget.org), an Open Source tool that generates UIs from JSON files. I'm always interested in additional metadata I can gather to refine my UIs, so JSON Schema is of great interest to me.

I would like to understand more about the decision to use 'objects with named properties' for JSON Schema...

[ "prop1": { "length": 30, "required": true }, "prop2": { "length": 50 } ]

...as opposed to 'an array of objects'. For example:

[ { "name": "prop1", "length": 30, "required": true }, { "name": "prop2", "length": 50 } ]

Clearly the latter is not as easy to access (you can't use the dot notation to lookup by name), but a fundamental feature I require for Metawidget is to be able to specify ordering within the metadata.

When adopting the 'objects with named properties' approach, how does JSON Schema convey ordering of properties?

Regards,

Richard.

[Francis Galiegue]

Hello,

On Fri, Mar 29, 2013 at 12:58 AM, Richard Kennard
<richard.kenna...@gmail.com> wrote:
> Hi guys,
>
> I am the author of Metawidget (http://metawidget.org), an Open Source tool
> that generates UIs from JSON files. I'm always interested in additional
> metadata I can gather to refine my UIs, so JSON Schema is of great interest
> to me.
>
> I would like to understand more about the decision to use 'objects with
> named properties' for JSON Schema...
>
> [ "prop1": { "length": 30, "required": true }, "prop2": { "length": 50 } ]
>

This is not valid JSON. A JSON array is: [ x1, x2, x3 ] where x1, x2,
x3 are JSON values -- any value, which means it can be an array, an
object, etc.

> ...as opposed to 'an array of objects'. For example:
>
> [ { "name": "prop1", "length": 30, "required": true }, { "name": "prop2",
> "length": 50 } ]
>
> Clearly the latter is not as easy to access (you can't use the dot notation
> to lookup by name), but a fundamental feature I require for Metawidget is to
> be able to specify ordering within the metadata.
>
> When adopting the 'objects with named properties' approach, how does JSON
> Schema convey ordering of properties?
>

The short answer:

* JSON Schema does not define any more ordering than what JSON (RFC
4627) defines;
* the only situation where order matters is in arrays.

Let us take two simple examples. This array:

[ 1, 2 ]

contains the same elements as:

[ 2, 1 ]

but they are _different_ arrays.

On the other hand:

{ "a": 1, "b", 2 }

and:

{ "b": 2, "a": 1 }

are the _same_ JSON values, since JSON objects have no notion of property order.

But there is no notion, in JSON, of "complex object ordering". There
is no way, in JSON, to say that for two objects o1 and o2, o1 is less
than o2 if o1's "name" is lexically less than o2's "name".

Relying on ordering of properties or whatever in any JSON value over
the wire would require a preliminary agreement between the _producer_
of this JSON value and its consumer. As I do Java and use Jackson, I
can testify that you _can_ bind Jackson so as to tell it to emit
property members, or even array elements, in a given order.

But in the general case? Don't count on it.

More broadly, JSON Schema describes JSON, and JSON only. Anything
beyond JSON, and this includes ordering, is beyond the scope of JSON
Schema altogether.

[note: there is also the problem with numeric value equality, but this
is beyond the scope of this discussion]

Hope this helps,
--
Francis Galiegue, fgal...@gmail.com
JSON Schema in Java: http://json-schema-validator.herokuapp.com

[Austin Wright]

Strictly speaking objects in JSON are unordered, so such a property in JSON Schema wouldn't be of much use. Generally if you need an order, then you use an array, in the form of [{key: "x", value:"..."}].

But there can still be an informative property for serialization purposes, to say "serialize the properties of this object in this order". I haven't yet implemented it in my own application, but I do have a "propertyOrder" property for this purpose, that gives an ordered array of property names or property patterns (like ["prop1", "prop2"]). No such property exists in JSON Schema though, and I think it would be out of the scope to include anyways.

[Richard Kennard]

Thank you both for your detailed responses. I am confused by this statement:

"JSON Schema describes JSON, and JSON only. Anything beyond JSON... is beyond the scope of JSON Schema altogether".

Surely JSON Schema describes JSON metadata, which by definition is "beyond the scope of JSON". For example there is no concept in JSON for 'required', 'maxLength' or 'minimum'. So that is what JSON Schema is for?

"I do have a "propertyOrder" property for this purpose... no such property exists in JSON Schema though, and I think it would be out of the scope to include anyways"

How is, say, 'propertyOrder' different from 'required'? It would solve my requirement for ordered properties, so that I could use JSON Schema natively without having to rewrite it into an array. Surely ordering is useful metadata?

[Francis Galiegue]

On Fri, Mar 29, 2013 at 2:36 AM, Richard Kennard
<richard.kenna...@gmail.com> wrote:
> Thank you both for your detailed responses. I am confused by this statement:
>
> "JSON Schema describes JSON, and JSON only. Anything beyond JSON... is
> beyond the scope of JSON Schema altogether".
>
> Surely JSON Schema describes JSON metadata, which by definition is "beyond
> the scope of JSON". For example there is no concept in JSON for 'required',
> 'maxLength' or 'minimum'. So that is what JSON Schema is for?
>

No, JSON Schema describes the _structure_ of JSON data:

* if the JSON value is an object, then the "required" keyword defines
a constraint on what members MUST appear in this object value in order
to validate;
* if the JSON value is a string, then the "maxLength" keyword defines
a constraint on the maximum number of Unicode code points which MUST
be present in order for this string to validate;
* if the JSON value is a number, then the "minimum" keyword defines a
constraint on what mathematical quantity the number MUST be greater
than, or equal to, in order to validate.

All these constraints are "superimposed" over JSON alone, as it is
defined by RFC 4627. Save for numeric constraints (since JSON Schema
considers the mathematical value of numeric instances, not their JSON
representation), all constraints describe possible JSON values
_without other expectations that a JSON value be a JSON value_. It
does not impose any _semantic_ constraints over JSON (save for
"format", which is a particular beast).

> "I do have a "propertyOrder" property for this purpose... no such property
> exists in JSON Schema though, and I think it would be out of the scope to
> include anyways"
>
> How is, say, 'propertyOrder' different from 'required'? It would solve my
> requirement for ordered properties, so that I could use JSON Schema natively
> without having to rewrite it into an array. Surely ordering is useful
> metadata?
>

"propertyOrder" is different in the sense that it requires, for instance:

{ "a": 1, "b": 2 }

to be interpreted differently than:

{ "b": 2, "a": 1 }

even though they are the _same JSON values_. It imposes ordering which
JSON does not define.

[Richard Kennard]

Thank you for your patience listening to me. As a potential early adopter of this draft specification, I appreciate the opportunity to have my use case considered.

> No, JSON Schema describes the _structure_ of JSON data:

Clearly, I would argue that property ordering is related to the _structure_ of the data?

> "propertyOrder" is different in the sense that it requires, for instance { "a": 1, "b": 2 } to be interpreted
> differently than { "b": 2, "a": 1 }

I appreciate that. However:

{ "a": { "propertyOrder": 1, "required": true }, "b": { "propertyOrder": 2, "maxLength": 30 } }

Would seem to be quite a benign addition? As would:

"properties": { "a": { "required": true }, "b": { "maxLength": 30 } }, "propertiesOrder": [ 'a', 'b' ] }

I'm not sure which syntax Austin is alluding to, but either (or something else) would work fine for me.

Regards,

Richard.

[Francis Galiegue]

On Fri, Mar 29, 2013 at 4:01 AM, Richard Kennard
<richard.kenna...@gmail.com> wrote:
> Thank you for your patience listening to me. As a potential early adopter of
> this draft specification, I appreciate the opportunity to have my use case
> considered.
>
>
>> No, JSON Schema describes the _structure_ of JSON data:
>
> Clearly, I would argue that property ordering is related to the _structure_
> of the data?
>

No it isn't, since JSON does not define a property order. If JSON
Schema adds such a constraint, this isn't JSON anymore ;)

[Richard Kennard]

I think I'm confusing you. I am *not* saying that JSON Schema properties should be ordered. That would, as you say, break the JSON specification.

All I'm asking for is that 'propertyOrder' be added as an extra little piece of metadata, alongside 'required', 'maxLength' etc. So it would look:

{ properties: {
   "street": { "maxLength": 30, "propertyOrder": 1 },
   "city": { "maxLength": 20, "propertyOrder": 2 },
   "zip": { "propertyOrder": 3 }
}

I believe such metadata would be useful for many purposes.

[Geraint]

On Friday, March 29, 2013 9:13:15 AM UTC, fge wrote:

On Fri, Mar 29, 2013 at 4:01 AM, Richard Kennard
<richard.kenna...@gmail.com> wrote:
> Thank you for your patience listening to me. As a potential early adopter of
> this draft specification, I appreciate the opportunity to have my use case
> considered.
>
>
>> No, JSON Schema describes the _structure_ of JSON data:
>
> Clearly, I would argue that property ordering is related to the _structure_
> of the data?
>

No it isn't, since JSON does not define a property order. If JSON
Schema adds such a constraint, this isn't JSON anymore ;)

I disagree.  The data would still be JSON.

However, the issue is that the keywords would describe something that is unobservable in the JSON data - that is the source of the reluctance, I think.

When it comes to *display* order of object properties (which I think is what your original requirement is based on?), this has been raised before - one of the points in this Google Groups thread: https://groups.google.com/d/msg/json-schema/zxeCUle4wj0/qdboT0VIaF4J, and raised on the GitHub repo for my project: https://github.com/geraintluff/jsonary/issues/24#issuecomment-15169083

For my project (Jsonary), I decided the use the (completely made up and non-standard) "displayOrder" property, which works more or less like "z-order" in CSS (https://github.com/geraintluff/jsonary/issues/46).

I have for some time been thinking about some kind of "JSON Stylesheet" standard - it would probably be an extension to JSON Schema, and would concern itself with issues like this.

[Francis Galiegue]

On Fri, Mar 29, 2013 at 10:27 AM, Geraint <gerai...@gmail.com> wrote:
[...]

>> >
>> > Clearly, I would argue that property ordering is related to the
>> > _structure_
>> > of the data?
>> >
>>
>> No it isn't, since JSON does not define a property order. If JSON
>> Schema adds such a constraint, this isn't JSON anymore ;)
>
>
> I disagree. The data would still be JSON.
>

No. If you make a difference between { "a": 1, "b": 2 } and { "b": 2,
"a": 1 } then you don't do JSON.

[Richard Kennard]

Perhaps we are talking at cross-purposes, for which I apologise.

I am asking for some metadata, but I am *not* asking for a constraint. So if your concern is that you couldn't validate a piece of JSON against such a JSON Schema (like you can with 'required', 'maxLength', 'maximum' etc) then I agree.

However, not all metadata needs to be a validate-able constraint. For example JSON Schema defines 'description' as a piece of metadata:

"id": {
            "type": "number",
            "description": "Product identifier",
            "required": true
}

Presumably 'description' is not validated against, and neither should 'propertyOrder' (or 'displayOrder') be.

[Geraint]

The keyword you describe is completely possible - in fact, as I mentioned before, I am using the (experimental, non-validation, meta-data) property "displayOrder" in a branch of my project, the world has somehow not imploded. :)

It is not a use-case that is currently covered by JSON Schema, though - and I think there's a good reason.  JSON Schema has two concerns: the validation part concerns itself with the structure of the data, and the hyper-schema part concerns itself with interaction.  However, ordering of JSON properties is a display issue.

The "description" and "title" properties could be used for display - but usually, they are used as a form of documentation within the schema.  "displayOrder"/"propertyOrder" is part of a new concern for the display of JSON data.

However, JSON Schema has always been designed so that you can extend it, so there is nothing stopping you from using a custom property.  In fact, as this display ordering is a concern for me as well, we should consider collaborating to try and get as much interoperability as possible.  Are you on GitHub?

[Richard Kennard]

Austin,

Geraint has started a GitHub project to collect ideas about 'propertyOrder' and similar. I would value your contribution:

https://github.com/json-stylesheet/json-stylesheet
https://github.com/json-stylesheet/json-stylesheet/issues/1

Regards,

Richard

[Geraint]

OK - we have discussed this on GitHub, and I have changed my view.

Basically, although this keyword would have UI benefits (as does *every* keyword, including "required" or "maxItems"), it is actually not primarily a UI-concern.  It is a more general processing hint, and I think it's part of a larger category of keywords, for which there is already precedent in the standard.

Let's look at "format" - if a number has a format of "utc-millisec" (defined in v3), then it represents a millisecond offset from the Unix epoch.  However, this information places absolutely no constraints on the numeric value of the data - it is not validatable, whatever library you are using.  However, it still describes something important about the format.

I think a good term for this is an "informative" keyword.  It is not validated (validators are allowed to completely ignore "format"), but it still gives you information about the format.

I think that "propertyOrder" is in many ways similar - it's not quite the documentation of "title"/"description", but it would be widely useful as a processing hint.  ORMs generating database tables would use it for column order, as would CSV export.  Format translators (e.g. JSON -> XML) could use it slightly differently.  I also still believe that an "unordered" keyword for arrays would be similarly useful - say you're processing schemas to produce Java classes, it can help you decide when a Set<?> is more appropriate than a List<?>.  It could tell processors whether they can sort the array for efficiency, or whatever.

So I think that we should consider grouping "format" with these new properties and including them in the standard.  I have raised an issue on the GitHub repo: https://github.com/json-schema/json-schema/issues/87