linking to self as a means of versioning a schema
ps
Xample
in the future. You have associated data (the document) which is
supposed to know who is it's schema validator, right ?
At first, there is no official way to force a document to be validated
by a specific schema. The reason is that there is no constraints on
documents but the schema you actually want to apply itself.
But you can still do what you want. Here is 3 way to do so, the third
being probably the answer you are looking for.
- Either reuse the $schema property within the document itself. You
will still have to parse it before to then know which schema to apply
- Or Add your own document "version" tag. In the real case, we usually
get a document (imagine for example a photoshop document), watch for
it's version (f.e. version 5) and then apply the right schema on it.
(i.e. open it with the right parser)
- Make a schema which will responds to a version constraint (using
their values through the enum property):
documentV1 = { "version":1 , "message":"I'm a document version 1"};
documentV2 = { "version":2 , "message":"I'm a document version 2",
"info":"something which was not in version 1"};
schema =
{
"type":[
{"properties":{"version":{"enum":[1]}}, "extends":{"$ref":"http://
your_schema_for_version_1"}},
{"properties":{"version":{"enum":[2]}}, "extends":{"$ref":"http://
your_schema_for_version_2"}}
]
}
It's then up to you to either provide the right schema at the url
(http://your_schema_for_version_1 / http://your_schema_for_version_2)
or to directly replace the {"$ref":"http://your_schema_for_version_X"}
objects with your own schema.
Best regards
ps
I think your option #1 ---- re-using the $schema property in the
document ----- is best for my purposes.
This provides the full URL to the schema, so someone can discover it
by manual inspection
and figure out what to do with it. This is more self-documenting.
I have three questions:
1) What does "$" mean in JSON Schemas? My understanding is that
the use of $ is purely a stylistic flourish in JavaScript, so what is
the meaning in this context?
2) My primary schema (called stream-item) relies on a couple other
schemas, so should I be using the "links" attribute in my schemas? Is
the purpose of "links" to help a JSON schema validator?
3) I would very much appreciate a critical review of these two
short schemas. Their purpose is in the description fields, and they
are pasted below.
http://trec-kba.org/schemas/v1.0/content-item
http://trec-kba.org/schemas/v1.0/stream-item
Any opinions, changes, feedback, etc would be very welcome!
Thanks.
-John
http://trec-kba.org/schemas/v1.0/content-item
{
"description": "Raw data, such as text, used by stream-item for
'title', 'body', and 'anchor'.",
"type": "object",
"properties": {
"$schema": {
"description": "URI of this JSON schema document.",
"type": "string",
"enum": ["http://trec-kba.org/schemas/v1.0/content-item"],
"required": true,
"default": "http://trec-kba.org/schemas/v1.0/content-item"
},
"raw_data": {
"description": "Raw data downloaded from remote source.",
"type": "string",
"required": true
},
"encoding": {
"description": "Encoding used to serialize raw data,
implemented by python string.encode(encoding)",
"type": "string",
"enum": ["utf-8", "unicode-escape", "string-escape"],
"required": true
},
"cleansed": {
"description": "Transformed raw_data to remove non-
informative substrings. Encoded as utf-8 however artifacts of the
raw_data's encoding may appear.",
"type": "string",
"optional": true
}
}
}
}
http://trec-kba.org/schemas/v1.0/stream-item
{
"description": "An item in a stream corpus. Typically has one or
more properties of type content-item.",
"type": "object",
"properties": {
"$schema": {
"description": "URI of this JSON schema document.",
"type": "string",
"enum": ["http://trec-kba.org/schemas/v1.0/stream-item"],
"required": true,
"default": "http://trec-kba.org/schemas/v1.0/stream-item"
},
"doc_id": {
"description": "md5 hash of abs_url",
"title": "Document ID",
"type": "string",
"pattern": "^[a-f0-9]{32}$",
"required": true
},
"abs_url": {
"description": "Normalized absolute URL derived from
original_url.",
"title": "Normalized URL",
"type": "string",
"required": true
},
"schost": {
"description": "scheme://netloc from abs_url",
"title": "scheme://netloc",
"type": "string",
"required": true
},
"original_url": {
"description": "URL provided by stream_source, only non-
null if differs from abs_url.",
"title": "Original URL",
"type": ["string", null],
"required": true
},
"stream_source": {
"description": "Name of source providing original_url and
stream_time.",
"title": "Stream Source",
"type": "string",
"enum": ["news", "social", "linking"],
"required": true
},
"stream_time": {
"description": "Time at which content entered data
stream. Derived differently for each stream source.",
"type": "object",
"properties": {
"epoch_ticks": {
"description": "Integer seconds since the epoch
January 1, 1970.",
"type": "integer",
"required": true
}
"zulu_timestamp": {
"description": "Calculated from epoch_ticks in
format %Y-%m-%dT%H:%M:%SZ",
"type": "string",
"pattern": "^\d{4}-\d{2}-\d{2}(T|\s)\d{2}:\d{2}:\d{2}\.\d{6}(Z|
\s|)$",
"required": true
}
}
},
"title": "http://trec-kba.org/schemas/v1.0/content-item",
"body": "http://trec-kba.org/schemas/v1.0/content-item",
"anchor": "http://trec-kba.org/schemas/v1.0/content-item",
"source_metadata": {
"description": "Info provided by stream source.",
"type": ["http://trec-kba.org/schemas/v1.0/news-metadata",
"http://trec-kba.org/schemas/v1.0/social-
metadata",
"http://trec-kba.org/schemas/v1.0/linking-
metadata"],
"optional": true
},
"extracted_metadata": {
"description": "Info generated by automated content
analytics.",
"type": "http://trec-kba.org/schemas/v1.0/extracted-metadata",
"optional": true
},
},
"additionalProperties": true
}
Francis Galiegue
> Xample, thanks for describing these three options.
>
> I think your option #1 ---- re-using the $schema property in the
> document ----- is best for my purposes.
>
> This provides the full URL to the schema, so someone can discover it
> by manual inspection
> and figure out what to do with it. This is more self-documenting.
>
> I have three questions:
>
> 1) What does "$" mean in JSON Schemas? My understanding is that
> the use of $ is purely a stylistic flourish in JavaScript, so what is
> the meaning in this context?
>
We are talking JSON here. The $ has no special meaning in any case. If
you are talking about "$ref", it is a special keyword which points to
another JSON document, which should be a schema.
Remind that the JSON spec allows _anything_ as a key value in object
instances. For instance, this is valid JSON:
{
"": true,
"/a/b": 1,
" ": "foo",
"$^@\~#{\¹fzuih&": "yep"
}
It has nothing to do with JavaScript at all.
> 2) My primary schema (called stream-item) relies on a couple other
> schemas, so should I be using the "links" attribute in my schemas? Is
> the purpose of "links" to help a JSON schema validator?
>
No, $ref serves such a purpose.
> 3) I would very much appreciate a critical review of these two
> short schemas. Their purpose is in the description fields, and they
> are pasted below.
>
> http://trec-kba.org/schemas/v1.0/content-item
>
> http://trec-kba.org/schemas/v1.0/stream-item
>
> Any opinions, changes, feedback, etc would be very welcome!
> Thanks.
>
> -John
>
>
> http://trec-kba.org/schemas/v1.0/content-item
> {
> "description": "Raw data, such as text, used by stream-item for
> 'title', 'body', and 'anchor'.",
> "type": "object",
> "properties": {
> "$schema": {
> "description": "URI of this JSON schema document.",
> "type": "string",
> "enum": ["http://trec-kba.org/schemas/v1.0/content-item"],
> "required": true,
> "default": "http://trec-kba.org/schemas/v1.0/content-item"
> },
No, "$schema" is not an object. It helps telling the validator which
version of the JSON Schema draft you use, and its values are URIs. For
instance:
"$schema": "http://json-schema.org/schema/draft-v3/schema#"
[...]
> "original_url": {
> "description": "URL provided by stream_source, only non-
> null if differs from abs_url.",
> "title": "Original URL",
> "type": ["string", null],
Should be "null" (with quotes), not null, which is the JSON null value
but does not represent the null type.
[...]
> "type": ["http://trec-kba.org/schemas/v1.0/news-metadata",
> "http://trec-kba.org/schemas/v1.0/social-
> metadata",
> "http://trec-kba.org/schemas/v1.0/linking-
> metadata"],
This should be:
"type": [
{ "$ref": "http://trec-kba.org/schemas/v1.0/news-metadata" },
{ "$ref": "http://trec-kba.org/schemas/v1.0/social-metadata" },
{ "$ref": "http://trec-kba.org/schemas/v1.0/linking-metadata" }
]
[...]
> "type": "http://trec-kba.org/schemas/v1.0/extracted-metadata",
This should be:
"type": [ { "$ref":
"http://trec-kba.org/schemas/v1.0/extracted-metadata" } ]
> "optional": true
"optional" does not exist anymore (it did in draft v2) -- in draft v3,
it has been replaced with "required" which means the exact opposite.
And false is the default for "required", so you can drop this line
altogether.
> },
> },
> "additionalProperties": true
You might as well remove this line since by default
"additionalProperties" has an empty schema as a value -- meaning any
additional property will have to match the empty schema ({}), which
matches all JSON documents.
Hope this helps,
--
Francis Galiegue, fgal...@gmail.com
"It seems obvious [...] that at least some 'business intelligence'
tools invest so much intelligence on the business side that they have
nothing left for generating SQL queries" (Stéphane Faroult, in "The
Art of SQL", ISBN 0-596-00894-5)
ps
> >http://trec-kba.org/schemas/v1.0/content-item
> > {
> > "description": "Raw data, such as text, used by stream-item for
> > 'title', 'body', and 'anchor'.",
> > "type": "object",
> > "properties": {
> > "$schema": {
> > "description": "URI of this JSON schema document.",
> > "type": "string",
> > "enum": ["http://trec-kba.org/schemas/v1.0/content-item"],
> > "required": true,
> > "default": "http://trec-kba.org/schemas/v1.0/content-item"
> > },
my instance documents, then doesn't my schema need to look like
above? I realize this is an overloading of the string "$schema" to
use it as a property name in instance documents. My goal is to
provide a pointer to the appropriate version of *my* schema in every
instance doc.
Perhaps there is a better way to do this?
> No, "$schema" is not an object. It helps telling the validator which
> version of the JSON Schema draft you use, and its values are URIs. For
> instance:
>
> "$schema": "http://json-schema.org/schema/draft-v3/schema#"
document, this line would go at the top level and not inside of the
"properties" object, right?
Thanks!
John
Francis Galiegue
Oops! I was indeed misreading that one. So, yes, you are totally
right, here "$schema" is a key of your object, and the provided schema
is correct.
However, you restrict it to only one value with your "enum" directive
-- why "default" at all? Especially since its value is the same as
your "enum".
ps
> >> > {
> >> > "description": "Raw data, such as text, used by stream-item for
> >> > 'title', 'body', and 'anchor'.",
> >> > "type": "object",
> >> > "properties": {
> >> > "$schema": {
> >> > "description": "URI of this JSON schema document.",
> >> > "type": "string",
> >> > "enum": ["http://trec-kba.org/schemas/v1.0/content-item"],
> >> > "required": true,
> >> > "default": "http://trec-kba.org/schemas/v1.0/content-item"
> >> > },
> -- why "default" at all? Especially since its value is the same as
> your "enum".
When there are more versions in the future, the enum will list all of
them. This allows the schema to communicate the existence of previous
versions.
The "default" indicates the current working version, analogous to a
symlink from "latest" to a tarball of the most recent version of a
software package.
Any reactions to this? Good idea? Dirty idea?
-John
Francis Galiegue
> [...]
>> However, you restrict it to only one value with your "enum" directive
>> -- why "default" at all? Especially since its value is the same as
>> your "enum".
>
>
> More overloading for the purpose of making it all self-documenting.
>
> When there are more versions in the future, the enum will list all of
> them. This allows the schema to communicate the existence of previous
> versions.
>
> The "default" indicates the current working version, analogous to a
> symlink from "latest" to a tarball of the most recent version of a
> software package.
>
> Any reactions to this? Good idea? Dirty idea?
>
Well, with this further explanation, that makes sense ;)
ps
of this URI?
"$schema": "http://json-schema.org/schema/draft-v3/schema#",
Should I be putting such an empty fragment hashmark in references to
schemas that I write?
jrf
Francis Galiegue
It is a JSON Pointer reference, and '#' means the root of the schema.
For instance, consider:
{
"id": "http://my.site/myschema#",
"sub": {
"type": "integer"
}
}
If you were to reference the schema proper, and provided that you can
resolve "http://my.site/myschema", then you would use:
"$ref": "http://my.site/myschema#"
If you wanted to reference "sub", you would then do:
"$ref": "http://my.site/myschema#/sub"
In short, '#' always refers to the current schema. One nice thing
about this way of doing is that you can refer to other parts of the
schema itself. For instance:
{
"even-numbers": {
"type": "integer",
"divisibleBy": 2
},
"type": "array",
"additionalItems": {
"$ref": "#/even-numbers"
}
}
fully defines a schema which describes an array of even integers.
Granted, this example is convoluted, but you see the point.
You may want to have a look at the JSON Pointer specification here:
http://tools.ietf.org/html/draft-pbryan-zyp-json-pointer-02
[side note: where is the warning about possible loops via pointers? It
seems to have been gone from the draft...]
ps
Cool.
Related: is there an online validator that respects draft v3?
jrf
ps
Who runs json-schema.org?
It seems to me that its a bit out of date, or perhaps I'm confused.
Does it have the latest draft somewhere on that site? Why isn't json-
schema.org/schema a symlink to the latest draft?
I'm trying to use a validator, like JSV, which seems to carry around
its own private copy of the latest draft spec.
In trying to use the dojo validator, it seems to fail to validate an
empty object {}:
http://trec-kba.org/schemas/v1.0/validate.html
You have to view source and watch the javascript console to see how
that simple example is failing. (sorry)
I'll try to make a more fully featured online validator --- but
shouldn't such a thing live on json-schema.org?
jrf
Xample
Armishev, Sergey
Xample,
I am new to JSON schema but I know that I need exactly the functionality that you mentioned: unserializing documents, initing class with default values,… Unfortunately I can’t find examples of such code on the Internet. JSV provides examples for validating but not much those that you are using. Do you know any of such code examples exists (based on JSV code)? Or you have your own code examples published somewhere?
-Sergey
--
You received this message because you are subscribed to the Google Groups "JSON Schema" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/json-schema/-/kt9hZ3a3NmgJ.
To post to this group, send email to json-...@googlegroups.com.
To unsubscribe from this group, send email to json-schema...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/json-schema?hl=en.
_____________________________________________________
This electronic message and any files transmitted with it contains
information from iDirect, which may be privileged, proprietary
and/or confidential. It is intended solely for the use of the individual
or entity to whom they are addressed. If you are not the original
recipient or the person responsible for delivering the email to the
intended recipient, be advised that you have received this email
in error, and that any use, dissemination, forwarding, printing, or
copying of this email is strictly prohibited. If you received this email
in error, please delete it and immediately notify the sender.
_____________________________________________________
Xample
To unsubscribe from this group, send email to json-schema+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/json-schema?hl=en.
Armishev, Sergey
Xample,
Thank you so much for sharing design ideas! I like this approach. I have another question regarding default values that you mentioned. My understanding is that I can use
default values in schema to provide such default values for real instance. It means if this is schema:
{
"title": "default integers",
"type": "object",
"properties": {
"myinteger":{
"required":true,
"description": "nteger with default value",
"type": "integer",
"default":5
}
}
}
And this is instance:
{
}
Then such empty instance is absolutely valid! The implementor should use default value from schema. But JSV validator gives an error “Property is required”. Then what the point to use defaults if you should provide the instance data anyway? The only usage is in enumeration to choose initial value and all other type
I tested also
{
"myinteger":""
}
And got error "Instance is not a required type"
I want actually to use default values from schema to control default values for all json files based on it. This way merely changing schema default I change values in all jsons! It is kind of “json css”
I really appreciate if you, Xample and community can share your thoughts about this
-Sergey
To unsubscribe from this group, send email to json-schema...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/json-schema?hl=en.
_____________________________________________________
This electronic message and any files transmitted with it contains
information from iDirect, which may be privileged, proprietary
and/or confidential. It is intended solely for the use of the individual
or entity to whom they are addressed. If you are not the original
recipient or the person responsible for delivering the email to the
intended recipient, be advised that you have received this email
in error, and that any use, dissemination, forwarding, printing, or
copying of this email is strictly prohibited. If you received this email
in error, please delete it and immediately notify the sender.
_____________________________________________________
--
You received this message because you are subscribed to the Google Groups "JSON Schema" group.
To view this discussion on the web visit https://groups.google.com/d/msg/json-schema/-/lxXG41qzJ3wJ.
To post to this group, send email to json-...@googlegroups.com.
To unsubscribe from this group, send email to json-schema...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/json-schema?hl=en.
Francis Galiegue
> Xample,
> Thank you so much for sharing design ideas! I like this approach. I have
> another question regarding default values that you mentioned. My
> understanding is that I can use default values in schema to provide such
> default values for real instance. It means if this is schema:
>
> {
>
> "title": "default integers",
> "type": "object",
> "properties": {
> "myinteger":{
> "required":true,
> "description": "nteger with default value",
> "type": "integer",
> "default":5
> }
> }
> }
>
> And this is instance:
>
> {
> }
>
> Then such empty instance is absolutely valid!
No it isn't. The "myinteger" property is required, you don't provide
it -- as such the JSON document is NOT valid against the schema. Any
validator which tells otherwise is wrong.
> The implementor should use
> default value from schema. But JSV validator gives an error “Property is
> required”.
Which it is perfectly right about.
> Then what the point to use defaults if you should provide the
> instance data anyway?
As I see it, "default" does not even belong to JSON Schema -- but to
JSON Patch. The spec of which is still a work in progress.
Also, there is a gap in the specification. Consider this:
{
"type": "integer",
"divisibleBy": 2,
"default": 3
}
According to the current spec, _this is a valid schema_. But there is
an obvious contradiction: the default value does not obey the
schema...
[...]
>
>
> I tested also
>
> {
> "myinteger":""
> }
>
> And got error "Instance is not a required type"
>
Which is normal: "" is a string value.
Hope this helps,
Armishev, Sergey
But I kindly disagree that "default" doesn't belong to JSON schema
That is from the spec http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.20
5.20. default
This attribute defines the default value of the instance when the
instance is undefined.
Francis, could you or somebody explain this? I am actually more interested in full support for such "default" , e.i. that I can automatically merge json instance with schema default values. That gives json schema a huge power. And validation happened only after such merge happened. Like when validator doesn't see element in instance it tries to substitute it with default value (according to above spec). If default value not found then real error like
"Property is required but missing. Default value not defined". And then allow to query for a full json instance where missing default values substituted with one from schema
Again, that is about powerful functionality that allows to control set of instances
-Sergey
-----Original Message-----
From: json-...@googlegroups.com [mailto:json-...@googlegroups.com] On Behalf Of Francis Galiegue
Sent: Monday, March 05, 2012 2:52 PM
To: json-...@googlegroups.com
Subject: Re: [json-schema] Re: linking to self as a means of versioning a schema
--
You received this message because you are subscribed to the Google Groups "JSON Schema" group.
Armishev, Sergey
Var schema =
{
"title": "default integers",
"type": "object",
"properties": {
"myinteger":{
"description": "nteger with default value",
"type": "integer",
"default":5
}
}
}
{}
Again, what I would like to have is not just validation but merge of schema with default values so that after validation Report has this new merged value so I can just query it
Report report = env.validate(empty_data,schema);
if(report.errors.length === 0) {
var merged_data = report.mergedData();
console.log(JSON.stringify(merged_data, undefined, 4));
}
Console log:
{
"myinteger":5
}
Francis and community, what do you all think?
Gary Court, what do you think?
Gary Court
that uses JSV to do this. You would just need to recursively iterate
through a provided schema and instance (value).
--Gary
Xample
In my case I have a function:
- initWithDefaultValues()For a given schema:- I look at the "default" value if there is one and that one comply with the specified type, I use its type as default.- If no "default" is specified or if that one does not comply with the allowed "type"- If the type is a string, it will take it as the default type- If the type is an array, I will take it's first item as the default type- If there is nothing… (no default value specified neither a given type (or "any" as the type)) I decided to use "null" as the default value.-> Now we have a type for the object to create:For a number :- If a default value is present, I simply use it otherwise I use 0 as the default value- If the minimum property is present, clip the value with it's minimum possible- If the maximum property is present, clip the value with it's maximum possibleFor a string- If a default value is present, I use it- else, if minimumLength is defined, and create a string with this length- else use "" as the default value- if maximumLength is defined, truncate the string to this maximum length.For an array- If a default array is present, I use it then- watch for the minItems and extend or create a default array with N dummy elements, watch for their associated schema (items, additionalItems) and call the initWithDefaultValues(itemInArray[i], schemaForItem(i));For an object- If a default object is present, I use it then- If there is required properties, I create them all and call initWithDefaultValues(object["aRequiredProperty"],schemaForProperty["aRequiredProperty"]);For a boolean- If a default object is present, I use it otherwise false
Francis Galiegue
> Thank you , Francis
> But I kindly disagree that "default" doesn't belong to JSON schema
It doesn't belong to validation per se. The question arises: should
JSON Schema be about validation only?
At some point there was the proposal of separating validation keywords
from metadata keywords in different sections of the specification. I
think this should be done.
> That is from the spec http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.20
> 5.20. default
>
>
> This attribute defines the default value of the instance when the
> instance is undefined.
>
> Francis, could you or somebody explain this?
Well, I am not the best person to answer that -- as I mentioned
previously, I only handle (and take care of) validation currently.
> I am actually more interested in full support for such "default" , e.i. that I can automatically merge json instance with schema default values. That gives json schema a huge power. And validation happened only after such merge happened. Like when validator doesn't see element in instance it tries to substitute it with default value (according to above spec). If default value not found then real error like
> "Property is required but missing. Default value not defined". And then allow to query for a full json instance where missing default values substituted with one from schema
> Again, that is about powerful functionality that allows to control set of instances
>
I disagree with the behaviour you mention. If a property is required
and is not present in the instance to be validated, then the instance
should, imo, be considered invalid. Otherwise you contradict the very
definition of the "required" keyword...
Armishev, Sergey
I was thinking about this and think that you as an originator for JSV can definitely do it much better and then keep it under your control. I can add test cases. What do you think?
Francis Galiegue
[...]
>
> Unfortuantely, this doesn't work for me at all. And I don't know why. And
> I need it deliberately to work, because it's so simple and obvious.
>
> Trying to copy & paste this example, similarly as other ones from other
> discussion https://gist.github.com/1367826, as well as my own using INTERNAL
> references, into http://jsonschemalint.com/ , leads to the same
> un-understable error:
>
> /additionalItemsUnknown schema
> referenceurn:uuid:b959f6e1-9a61-4ffc-8375-3d716b4c07fe#/even-numbers
>
That definitely looks like a bug, especially since in the gist, the
path into the schema is properly reported...
You should probably open a bug at https://github.com/garycourt/JSV
since it looks like this is the implementation used here.