The problem with extends

[Andi]

I have defined a schema with id "MyClass", than I link to this class
inside another schema, but I want this object to be optional. So I
write:

...
"my_property": {
type: "object",
optional: true,
extends: {$ref:"MyClass"}
},
...

Of course, this does not work, because the standard says that the
schema pointed to by extends will be validated first, and then the
extending schema. But now I will get an error, because "MyClass" is
not optional.

So we need a way to truly extend a schema and overwrite values instad
of only being able to narrow the extended schema. I hope you
understand what I am trying to explain ;)

Andi

[Gary Court]

The problem is not so much with the "extends" attribute, but the
default value of the "optional" attribute. I have brought this up in
another post:

http://groups.google.com/group/json-schema/browse_thread/thread/b193f4b349c075c9/9db043aed508cfc5

Since I have brought this issue up, several solutions have been
discussed but no census has been reached.

-Gary

[Andi]

This is not enough. There can be other boolean properties that make
trouble.

I think there is no real alternative to correctly extending a schema
internally and validating once instead of validating several schemas
that contradict each other!

On 19 Aug., 06:14, Gary Court <gary.co...@gmail.com> wrote:
> The problem is not so much with the "extends" attribute, but the
> default value of the "optional" attribute. I have brought this up in
> another post:
>

> http://groups.google.com/group/json-schema/browse_thread/thread/b193f...

[Andi]

Next example: additionalProperties, which is true by default! I came
across the same problem here.

JSON SCHEMA COMMUNITY: We need a discussion!!! ;)

[Gary Court]

When I was writing JSV, I was unable to come up with a solution where
I could guarantee that I only needed to validate against the extended
schema. And frankly, as the spec is currently written, it would be
very difficult. There was project called jsev ( http://code.google.com/p/jsev/
) that tried to create a set of restrictions on how extensions are
done to ensure this, but I think it's been abandoned and is now
outdated with the current spec.

Basically, to get extensions to work the way the spec intended, I had
to validate against both the newly extended/child schema, and the
parent schema.

Now, I'm not sure what issues with additionalProperties you're
referring to. Can you elaborate?

Being that you are bringing this up, I would like to see the next
revision of JSON Schema to have a more indepth explanation on how
"extends" should work, and include several examples. The one glaring
issue I found left out is what happens when you extend a schema with
"properties" with another schema with "properties". What happens when
they both have the same property defined? JSV uses the approach where
it tries to mix objects together the best it can, while other non-
object attributes are just replaced/overwritten.

-Gary

[Andi]

Take a look at this document: http://groups.google.com/group/json-schema/web/json-schema-proposal-working-draft
(paragraph "Extending and Referencing"). This is the only place, where
I found a clear definition of how "extends" should be treated. At
first, I thought, that this is the easiest - and best - way to
validate extends, but after creating a real world case I found that it
is bad idea. It is about constraints. So the parent must be more
constrained than the extended schema to make this way of validating
useful. For the most cases it's OK.

Back to my problem - I have two schemas:

{
id:'A',
type:'object',
properties: {
a: {...},
b: {...}
},
additionalProperties: false,
extends: {$ref:'B'}
}

and

{
id:'B',
type:'object',
properties: {
a: {...},
c: {...}
}
}

So at first I validate an instance to schema B, additional properties
are allowed, then while validating schema A, property c is not allowed
any more. In reality, properties a, b and c should by allowed in
schema A, but because of additionalProperties: false in A, c will not
be allowed anymore.

Conclusion: It happens, that schemas contradict each other, if they
are used separately. If you overwrite (deeply extend) schema B by
schema A internally, you have a schema that has no perils. You can
still define "extends", but in the new proposal, it will be treated as
a special case. The validator is forced to build one compact schema
internally.

My main motivation is to make JSON Schema able to handle real world
cases. Take a look at my JavaScript validator: http://github.com/akidee/schema.js
I have added features like fallbacks (for tolerant value adaptation)
and adapters. The "official" JSON Schema validators validate only in
strict mode, and I don't see any sense in this, because data from a
client is always an untrusted source and must be adapted to your
schema. Examples: 'false' => false, '1.234' => 1.234, '0' => false

My framework is the only framework that is able to handle the real
world cases, and you should use it.

It is still in development. Another problem that arises with the
current handling of "extends" is that you have less performance
because you revalidate some properties and that the error list is not
unique, when a validation fails in such a case.

It's really tricky - but challenging - to implement a JSON Schema
validator that makes sense and has got the features that are really
important. Then you can use it for any case: Another interesting
example is secure function calls - you simply proxy calls to a
function (in your API) with a JSON Schema validation and that's it.
Ready for production ;)

Final point: The current proposal and the core schema contradict to
each other. It seems that JSON Schema is not sophisticated in the
moment. But I want to make it better, because JSON is _the_ standard
for data.

On 24 Aug., 04:45, Gary Court <gary.co...@gmail.com> wrote:
> When I was writing JSV, I was unable to come up with a solution where
> I could guarantee that I only needed to validate against the extended
> schema. And frankly, as the spec is currently written, it would be

> very difficult. There was project called jsev (http://code.google.com/p/jsev/

[Kris Zyp]

On Aug 24, 2:42 am, Andi <andreaskal...@gmx.de> wrote:
> Take a look at this document:http://groups.google.com/group/json-schema/web/json-schema-proposal-w...

It is definitely my intent and belief that a, b, and c should be
allowed in instances of schema A. Setting additionalProperties: false
should mean that no properties may be included that are not defined in
the schema and its super schemas. If it a property is defined in super
schema, additionalProperties: false does not prohibit it.

As far as the "optional" attribute, as Gary mentioned, there was no
final consensus on the other thread, but adding "undefined" as a type
seemed like the most plausible solution. We could continue that thread
to nail that down.

Thanks,
Kris

[Andi]

I know, my interpretation was very strict. But my point is still that
this problem not only arises with extends and additionalProperties but
with other boolean values as well. So which values are considered in
which way when you extend? We need to have CLEAR AND SIMPLE
CONVENTION. My new proposal for the "extends" property is:

On the implementation level there must be validated exactly one schema
that properly extends all data from the extended schema. Example:

{
id:'A'

type:'object',
properties: {
a: {

type: 'integer',
minimum: 5,
optional: true
}
},
extends: {$ref:'B'}

}

{
id:'B',
type:'object',

additionalProperties:false,
properties: {
a: {
type: 'integer',
maximum: 100
},
b: {
type: 'string'
}
}
}

Your Implementation parses the schema A, saves the original schema (in
JavaScript, toJSON() can return it to reserialize it), but will use an
extended version for validation that looks like that:

{ // deep extend was made
id:'A'
type:'object',
additionalProperties:false, // from B
properties: {
a: {
type: 'integer',
minimum: 5,
maximum: 100,
optional: true
},
b: { // from B
type: 'string'
}
},
extends: { // resolved reference

id:'B',
type:'object',

additionalProperties:false,
properties: {
a: {
type: 'integer',
maximum: 100
},
b: {
type: 'string'
}
}
}
}

I just want to keep it as simple as possible and not to define rules
for every property that say how extends works. A function in JS that
does deep extend: http://github.com/akidee/extensions.js/blob/master/index.js#L81

Unique properties like "id" are not going to be copied. To reset a
property in an extending schema, you can set it to undefined. To
remove property b in the A schema:

{
id:'A'

type:'object',
properties: {
a: {

type: 'integer',
minimum: 5,
optional: true
},
b: undefined
},
extends: {$ref:'B'}

[Andi]

I hope we can continue this discussion to conclude with a clear new
draft ;)

[Iain Duncan]

Did anything ever happen with this?  I have just hit exactly the same issue and agree with Andi that when validating against an extended schema a deep extends should be performed rather than validating each individually.

Cheers,

Iain

[Gary Court]

As far as I am aware of, all "extends" issues were resolved in
revision 3 of the JSON Schema. A proper JSON Schema validator should
perform a deep extends (assuming this means a schema that has been
extended multiple times successively).

-Gary

> --
> You received this message because you are subscribed to the Google Groups
> "JSON Schema" group.
> To post to this group, send email to json-...@googlegroups.com.
> To unsubscribe from this group, send email to
> json-schema...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/json-schema?hl=en.
>