Understanding the "name" property and combining schemas

[Sam Fentress]

Hi all, I have two questions.

First, I'm trying to understand whether "name" is a valid property of
a schema, and what it is used for. The name property is used in the
example schema at the top of page 4 in the draft v.3,
("name":"Product", "properties":{...}). The draft, however, doesn't
mentioned the "name" property anywhere, and the self-describing schema
for schemas linked in the draft also does not include the "name"
property. (That sample schema is actually invalid against the meta-
schema with a whole bunch of errors, as seen when validated with
Kris's Dojo extension, but that may be an "optional/require"
difference between the extension and the specification.)

Second, and this may be related, I'm trying to figure out how to DRY
up my schema, and it sounds like I can do this by referring to and
extending multiple schemas, but I can't find any examples or clear
explanations of this.

Say I have an object with two properties, "topPane" and "bottomPane."
Both properties expect objects with the same attributes (e.g. "width",
"height"...). It seems like, instead of repeating myself, there's some
way to define a new schema with the name(?) "paneSchema", and then say
that the object in both "topPane" and "bottomPane" have to validate
against this. Is that possible?

Thank you,
Sam

[Francis Galiegue]

Yes. You can do, for instance:

----
{
"rootSchema": {
"patternProperties": {
"Pane": { "$ref": "#/paneSchema" }
}
},
"paneSchema": {
"properties": {
//define a pane object here
}
}
}
----

and then validate using this schema starting at #/rootSchema.
#/rootSchema and #/paneSchema are JSON paths.

I'm not sure about the "name" attribute either.

--
Francis Galiegue, fgal...@gmail.com
"It seems obvious [...] that at least some 'business intelligence'
tools invest so much intelligence on the business side that they have
nothing left for generating SQL queries" (Stéphane Faroult, in "The
Art of SQL", ISBN 0-596-00894-5)

[Gary Court]

On Thu, Nov 10, 2011 at 5:38 PM, Sam Fentress <sfen...@concord.org> wrote:
> Hi all, I have two questions.
>
> First, I'm trying to understand whether "name" is a valid property of
> a schema, and what it is used for. The name property is used in the
> example schema at the top of page 4 in the draft v.3,
> ("name":"Product", "properties":{...}). The draft, however, doesn't
> mentioned the "name" property anywhere, and the self-describing schema
> for schemas linked in the draft also does not include the "name"
> property. (That sample schema is actually invalid against the meta-
> schema with a whole bunch of errors, as seen when validated with
> Kris's Dojo extension, but that may be an "optional/require"
> difference between the extension and the specification.)
>

This property in this example should probably be "title" and not
"name". But in any case, it is an attribute that has no definition as
per the JSON Schema schema (which is allowed).

This is not the only example in draft 3 that has errors. These will be
fixed come draft 4.

--Gary

[Francis Galiegue]

On Fri, Nov 11, 2011 at 02:58, Gary Court <gary....@gmail.com> wrote:
[...]

>
> This is not the only example in draft 3 that has errors. These will be
> fixed come draft 4.
>

Any idea on the date?

[Sam Fentress]

On Nov 10, 7:48 pm, Francis Galiegue <fgalie...@gmail.com> wrote:
>
> Yes. You can do, for instance:
>
> ----
> {
>     "rootSchema": {
>         "patternProperties": {
>             "Pane": { "$ref": "#/paneSchema" }
>         }
>     },
>     "paneSchema": {
>         "properties": {
>             //define a pane object here
>         }
>     }}
>
> ----
>
> and then validate using this schema starting at #/rootSchema.
> #/rootSchema and #/paneSchema are JSON paths.

Thanks for the reply.

I am trying to make the following example fail and have been unable to
do so, I assume because I haven't understood the JSON path format:

----
schema = {
"rootSchema": {
"properties": {
"pane": {

"$ref": "#/paneSchema"
}
}
},
"paneSchema": {
"properties": {

"height": {
"type": "number"
}
}
}
}

doc = {
"pane": {
"height": "not-a-number"
}
}

results = dojox.json.schema.validate(doc, schema.rootSchema); //
results in "valid"
----

I have tried this both with the Dojo validator and with JSV, and
haven't succeeded with either of them -- the Dojo version fails to
fail while JSV reports "Unknown schema reference" and fails in all
cases. Clearly I need to understand the JSON path reference more, but
most of the examples I've seen start with a URI to an external JSON
file, not to another reference in the same JSON object.

Thanks,
Sam

[Francis Galiegue]

You have written "Pane" in uppercase in the schema, this is why!

Anyway, with my implementation
(https://github.com/fge/json-schema-validator), after fixing that, it
gives what is expected:

----
public final class Test
{
public static void main(final String... args)
throws IOException
{
final JsonNode schema = JsonLoader.fromResource("/t/schema.json");
final JsonNode data = JsonLoader.fromResource("/t/data.json");

final JsonValidator validator = new JsonValidator(schema);

final ValidationReport report = validator.validate("#/rootSchema",
data);

System.out.println("Success: " + report.isSuccess());

for (final String msg: report.getMessages())
System.out.println(msg);

System.exit(0);
}
}
----

The output:

----
Success: false
#/pane/height: instance is of type string, which is none of the
allowed primitive types ([integer, number])
----

[Sam Fentress]

On Nov 14, 4:03 am, Francis Galiegue <fgalie...@gmail.com> wrote:
>
> You have written "Pane" in uppercase in the schema, this is why!
>

I didn't... In the code in my last message the word "pane" is never
capitalized. (You may have been looking at your own code that was
quoted in my reply, where it is capitalized).

I have created a Gist with the exact code that I am running, and a
bl.ocks that runs it:

http://gist.github.com/1364126
http://bl.ocks.org/1364126

If you run that second link, you should see that the validator claims
that the json object is valid.

The way you are running it is different from the way I am running it.
Maybe the pure-javascript Dojox version doesn't support local
references?

Sam

[Francis Galiegue]

On Mon, Nov 14, 2011 at 16:17, Sam Fentress <sfen...@concord.org> wrote:
>
>
> On Nov 14, 4:03 am, Francis Galiegue <fgalie...@gmail.com> wrote:
>>
>> You have written "Pane" in uppercase in the schema, this is why!
>>
>
>
> I didn't... In the code in my last message the word "pane" is never
> capitalized. (You may have been looking at your own code that was
> quoted in my reply, where it is capitalized).
>

Yes, indeed, sorry for that :/

> I have created a Gist with the exact code that I am running, and a
> bl.ocks that runs it:
>
> http://gist.github.com/1364126
> http://bl.ocks.org/1364126
>
> If you run that second link, you should see that the validator claims
> that the json object is valid.
>

Yes indeed, and it is indeed wrong about this. But as I don't do
javascript, I won't be able to help...

If you try to inline the schema, ie:

----
{
"properties": {
"pane": {
"properties": {
"height": {
"type": "number"
}
}
}
}
}
----

and change your call to "dojox.json.schema.validate(doc, schema)", does it work?

> The way you are running it is different from the way I am running it.

The paths I use are written following the JSON Pointer draft, that is why.

> Maybe the pure-javascript Dojox version doesn't support local
> references?
>

That would be strange, since it is a base feature of json schema...
But maybe it does not support JSON Pointer? Can you also try and write
"#.paneSchema" instead of "#/paneSchema"? If yes then there is a
problem in this implementation.

[Gary Court]

Two issues from your gist:

1. You can not do referencing that way. References must point to a
subschema, or an already registered schema. This is what you should be
doing:

schema = {
"patternProperties": {

"pane": {
"properties": {
"height": {
"type": "number"
}
}
}
}
}

2. I don't know how up-to-date Dojo's JSON Schema implementation is. I
would recommend taking a look at JSV
(https://github.com/garycourt/JSV).

--Gary

[Francis Galiegue]

On Mon, Nov 14, 2011 at 20:04, Gary Court <gary....@gmail.com> wrote:
> Two issues from your gist:
>
> 1. You can not do referencing that way. References must point to a
> subschema, or an already registered schema.

Hmm, can you please define "registered schema"?

Personally, and my implementation does, I consider the schema valid
because it is what the validator is told to eat -- with a "$schema"
identifier (or is that "id"?) of "#". How is this interpretation
flawed? (and I am the culprit, I am the one who suggested the
solution)

[Sam Fentress]

On Nov 14, 2:04 pm, Gary Court <gary.co...@gmail.com> wrote:
> Two issues from your gist:
>
> 1. You can not do referencing that way. References must point to a
> subschema, or an already registered schema. This is what you should be
> doing:
>
>         schema = {
>           "patternProperties": {
>             "pane": {
>               "properties": {
>                 "height": {
>                   "type": "number"
>                 }
>               }
>             }
>           }
>         }
>

Hi Gary,

Thanks for your reply. You say that we can point to a "subschema" -- I
believe this is exactly what I'm trying to do. The issue is that I
don't know how to define or refer to the subschema "paneSchema."

You ask why I'm not using your simpler example. The original problem
was that I wanted to DRY up my schema -- I had a lot of repetition.
For example, if I had three panes, a topPane, middlePane and
bottomPane, I did not want to have to specify the redundant and error-
prone schema below:

----
schema = {
  "patternProperties": {
    "topPane": {
      "properties": {
        "height": {
          "type": "number"
        }
      }
    },
"middlePane": {
      "properties": {
        "height": {
          "type": "number"
        }
      }
    },
"bottomPane": {

      "properties": {
        "height": {
          "type": "number"
        }
      }
    }
  }
}
----

(obviously in the actual code there is more to a "pane" than just one
property.)

Instead I wanted to be able to define a "pane" with a local subschema
and refer to it directly. If the issue is that the main schema and the
subschema can't share the same JSON object, I can separate it out into
two. However, I would greatly prefer not to have to put them at
different URLS -- the point here is to avoid redundancy, not to share
different subschemas across multiple applications, so different URLs
seems like overkill...

> 2. I don't know how up-to-date Dojo's JSON Schema implementation is. I
> would recommend taking a look at JSV

Indeed, I am using JSV in my current tests. JSV gives a better result
than the dojo implementation, giving the error message that it can't
find the schema reference, instead of just passing like the Dojo
implementation.

Thank you,
Sam

[Francis Galiegue]

On Mon, Nov 14, 2011 at 20:44, Sam Fentress <sfen...@concord.org> wrote:

Beware with patternProperties, regexes are not anchored by default:
/topPane/ will match property "NOTtopPaneITellYou".

Maybe you could then write:

----
{
"patternProperties": {
"Pane$": {
// etc
----

I'm at a loss here. What happens if a schema does NOT have a "$schema"
or "id" field in it? Is it considered invalid by the current draft?
Can't we use "anonymous" schemas? Plenty of people do that... And
examples already exist which address subschemas using JSON Pointers
from a same root document. My example looked perfectly valid to me.
So, what is what?

[Gary Court]

Here's what your example should look like with references:

schema = {
"patternProperties": {
"topPane": {
"properties": {
"height": {
"type": "number"
}
}
},

"middlePane": {"$ref" : "#/patternProperties/topPane"},
"bottomPane": {"$ref" : "#/patternProperties/topPane"}
}
}

The problem with your previous gist was that you were calling the dojo
validator with "schema.rootSchema". In JavaScript, that cuts off any
reference to "schema.paneSchema". The above example alleviates that
problem.

--Gary

[Gary Court]

On Mon, Nov 14, 2011 at 12:18 PM, Francis Galiegue <fgal...@gmail.com> wrote:
> On Mon, Nov 14, 2011 at 20:04, Gary Court <gary....@gmail.com> wrote:
>> Two issues from your gist:
>>
>> 1. You can not do referencing that way. References must point to a
>> subschema, or an already registered schema.
>
> Hmm, can you please define "registered schema"?

A registered schema is a schema that is already known by the
validator. For example, "http://json-schema.org/schema" is a
registered schema in almost all validators.

--Gary

[Sam Fentress]

On Nov 14, 3:57 pm, Gary Court <gary.co...@gmail.com> wrote:
>
> Here's what your example should look like with references:
>
>  schema = {
>    "patternProperties": {
>      "topPane": {
>        "properties": {
>          "height": {
>            "type": "number"
>          }
>        }
>      },
>     "middlePane": {"$ref" : "#/patternProperties/topPane"},
>     "bottomPane": {"$ref" : "#/patternProperties/topPane"}
>    }
>  }
>
> The problem with your previous gist was that you were calling the dojo
> validator with "schema.rootSchema". In JavaScript, that cuts off any
> reference to "schema.paneSchema". The above example alleviates that
> problem.
>
> --Gary

Ok, this is great, and is much closer to what I was looking for
before. This new example runs as expected.

However, it still seems to me that it should be possible to define the
subschemas in their own sections, instead of later properties randomly
referring to subschemas from earlier properties. Conceptually, it's
not clear from your example that topPane, middlePane and bottomPane
are all instances of the same thing, and if we change things around --
say remove topPane, it's going to require refactoring.

It seems that it should be possible to do this:

----
schema = {
"properties": {
"topPane": { "$ref": "#/paneSchema" },
"middlePane": { "$ref": "#/paneSchema" },
"bottomPane": { "$ref": "#/paneSchema" }
},
"paneSchema": {
"type": "object",
"properties": {
"height": "number"
}
}
}

doc = {
"topPane": { "height": 10 },
"middlePane": { "height": 10 }
"bottomPane": { "height": "not-a-number" }
}

results = jsvEnv.validate(doc, schema);
----

This schema is much clearer (to me). Now we are loading in the top-
level schema, and the references are local and the whole schema,
including the subschemas, should have been registered with the
validator. However, neither JSV nor the dojo validator accept this.
Why is that?

I've update my blocks example with your working code and my not-
working code: http://bl.ocks.org/1364126

Sam

[Gary Court]

Yes, you can do that. JSV will accept this schema if you move the
"paneSchema" property to the top of the object (JSV 3.x only supports
backward references, will be fixed in the next release).

--Gary

[Sam Fentress]

On Nov 14, 8:41 pm, Gary Court <gary.co...@gmail.com> wrote:

That would be great, but it doesn't seem to work for me. Maybe the
version of JSV that's published on NPM doesn't support that? It claims
to be version 3.5.

The code I tried is here: https://gist.github.com/1367826

Sam

[Gary Court]

>> Yes, you can do that. JSV will accept this schema if you move the
>> "paneSchema" property to the top of the object (JSV 3.x only supports
>> backward references, will be fixed in the next release).
>>
>> --Gary
>
>
> That would be great, but it doesn't seem to work for me. Maybe the
> version of JSV that's published on NPM doesn't support that? It claims
> to be version 3.5.
>
> The code I tried is here: https://gist.github.com/1367826
>

Oopps... Sorry, you can't actually do that because JSV doesn't know
that "paneSchema" is a schema and not just an object. My bad. You will
have to go back to using my previous example instead.

--Gary

[Sam Fentress]

On Nov 15, 1:52 pm, Gary Court <gary.co...@gmail.com> wrote:>>
Oopps... Sorry, you can't actually do that because JSV doesn't know>
that "paneSchema" is a schema and not just an object. My bad. You
will> have to go back to using my previous example instead.> > --Gary

Ah, but I *can* register the subschema with the environment, and that
does seem to work:
----schema = {  "properties": {    "topPane": {      "$ref":
"paneSchema"     // referring directly by a new id, not #/paneSchema 
  },    "bottomPane": {      "$ref": "paneSchema"    }  }, 
"paneSchema": {    "properties": {      "height": {        "type":
"number"      }    }  }};
doc = {  "topPane": {    "height": 10  },  "bottomPane": {   
"height": "not-a-number"  }};
env = JSV.createEnvironment();
env.createSchema(schema.paneSchema, schema, "paneSchema");   // this
is the key
report = env.validate(doc, schema);----
This works correctly.
It's not perfect, because you can't take any arbitrary schema that's
been defined as a JSON file and expect it to work: you have to know
about the existence of the other subschemas in advance. It seems as if
it would be fairly straight-forward(?) to add a function to the JSV
Environment that would run through the passed-in schema, find all the
subschemas (i.e., all properties of the main schema that aren't
defined in the meta-schama and are themselves valid schemas), and
register them before validation. That way the schema could refer to
its own subschemas without the validator needing to know about them in
advance.
Anyway, for my purposes this is fine.Thank you for your help,Sam

[Sam Fentress]

My last email seems to have been garbled. Trying again with a Gist.

On Nov 15, 1:52 pm, Gary Court <gary.co...@gmail.com> wrote:>>
Oopps... Sorry, you can't actually do that because JSV doesn't know>
that "paneSchema" is a schema and not just an object. My bad. You
will> have to go back to using my previous example instead.> > --Gary

Ah, but I *can* register subschema with the environment, and that does
seem to work:
https://gist.github.com/1369981

[Francis Galiegue]

On Wed, Nov 16, 2011 at 13:43, Sam Fentress <sfen...@concord.org> wrote:
[...]

> https://gist.github.com/1369981
>
> This works correctly.

This normally shouldn't: $ref just doesn't point anywhere, and to my
eyes, it works only by chance (because JSV is written in Javascript
maybe?). "paneSchema" is not a valid JSON Pointer.

Don't forget that $ref is a URI, and such a thing as "paneSchema" is a
URI, yes, but relative to... What? This isn't defined in the draft at
the moment, but consider this, which is legal in $ref:

some/thing/here#/some/path

This has to be clarified. For me, "paneSchema" as a value in $ref is
just illegal, unless you consider a schema itself to be a base for
relative URIs without a JSON Pointer. But then what of JSON Pointer in
such a case?

[Sam Fentress]

On Nov 16, 8:06 am, Francis Galiegue <fgalie...@gmail.com> wrote:

> On Wed, Nov 16, 2011 at 13:43, Sam Fentress <sfentr...@concord.org> wrote:
>
> [...]
>
> >https://gist.github.com/1369981
>
> > This works correctly.
>
> This normally shouldn't: $ref just doesn't point anywhere, and to my
> eyes, it works only by chance (because JSV is written in Javascript
> maybe?). "paneSchema" is not a valid JSON Pointer.

It works because I've registered the subschema's JSON object with the
URI "paneSchema" in the JSV environment in line 30 of the gist. The
fact that the variable also happens to be called "paneSchema" is not
what's being referred to here.

Note that if I set $ref to be "paneSchema#", i.e. the root of the
schema with the URI "paneSchema", it still works. I assume that the
$ref is smart enough to add the # if there isn't one, or else the #
isn't required if you're pointing directly to the root.

At least, I think this is how it works. I would test it with the Dojo
implementation as well, but as far as I can tell, it doesn't have the
ability to register new schemas.

Sam