I was wondering if you would consider adding link support, making it a hypermedia format.
Erland,
I was wondering if you would consider adding link support, making it a hypermedia format.
Absolutely!In fact, it would be wonderful if JSON-stat-based APIs were 100% REST APIs (including the HATEOAS constraint): imagine connecting to a single endpoint of the Open Data Initiative of a certain statistical office and discovering from there all the possible actions and resources available!Our current focus though is on the type of response that is able to express data in table (cube) form. At some point, JSON-stat will need to support other types different from this core "datasets' response" (for these types, probably some Atom flavor format could do the job).The reason I avoided URIs in the first version of the documentation (there were actually links in previous drafts) was: URIs are so powerful that can play many roles. I wasn't sure these roles were compatible, or at least I tried to avoid possible conflict by avoiding URIs altogether. I also was aware that my first proposals were probably too naif but unfortunately the current state of the art (JSON hypermedia) seamed too shaky ground at the time.In particular, I think URIs can be useful in JSON-stat for:1) Discovery: Hypermedia. In an early draft there was a simple "uri" element (child of the "dataset" element) that was a natural companion of the "source" element.
2) Semantics: the use of URIs to identify the resources used in a JSON-stat response, to make the format more semantic and interoperable.3) Support for partial responses (this is probably the same as 2): see for example: Shared vocabularies as partial responses.
I didn't know collection+json (I've been a member for several years of HAL, though, and now I've just discovered Siren).My questions then are:A) collection+json vs. HAL vs. Siren: What standard should we adopt for this?
B) Links and Identity (hypermedia and semantics/partial responses): Should these two issues concerning URIs be faced at the same time? Or should they be addressed separately?
C) Levels: What nodes of the JSON-stat tree should support links? In your proposal only the "dataset" element supports them, but this seems too poor and, besides, partial responses need URIs deeper in the tree.
So, to summarize, yes, please, lets make JSON-stat a hypermedia format. But before doing so I think we need to try to answer at least these three questions. What do you think?X.
"links": [
{"href": "http://json-stat.org/samples/cj.json", "rel": "self"},{"href": "http://json-stat.org/samples/cj.csv", "rel": "alternate", "type": "text/csv"}{"href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP", "rel": "alternate", "type": "text/html"}
],
"label" : "US States by GSP and population",
...
"links": [
{"href": "http://json-stat.org/samples/cj.json?dimension=concept", "rel": "self"}
],
"label" : "Concepts",
...
"links": [
{"href": "http://json-stat.org/samples/cj.json?dimension=year", "rel": "self"}
],
"label" : "Year",
...
"links": [
{"href": "http://json-stat.org/samples/cj.json?dimension=state", "rel": "self"},
{"href": "http://en.wikipedia.org/wiki/List_of_U.S._state_abbreviations", "rel": "alternate", "type": "text/html"}
],
"label" : "State",
...
"_links": {
"self": { "href": "/samples/hal.json" },
"alternate": { "href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP" }
},
Same as Collection+JSON but "rel" is an array of strings instead of an array (["self"] instead of "self").
Does the HAL version allow for multiple links with the same relation?
{ "href": "http://json-stat.org/samples/hal.csv", "type": "text/csv" },
{ "href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP", "type": "text/html" }
]
},
ds1: {...},
ds2: {...},
_links: {
"self": { "href": "http://provider/api/v1/bundle.json?ds=ds1+ds2" }
}
How does a client discover requests similar toor even more radical (include only the value array; or return everything but the value array)? This problem should probably be addressed through some sort of template solution that allows you to expose the JSON-stat Query Language (JSTQL), or (worse) defining some JSON-stat specific rels for common queries like in this example:
application/jsonstat+json;type=dataset
application/dataset+json
"_links": {"self": { ... },"schema": { href: "http://json-stat.org/json-schema/dataset" }
}
For practical reasons, it's interesting to include the type in the response body. This could be achieved using a new special JSON-stat string property ("type") allowing three possible values ("bundle" -default-, "dataset", "dimension") or a more general web linking mechanism (in HAL, cj or siren syntax):"_links": {"self": { ... },"schema": { href: "http://json-stat.org/json-schema/dataset" }}The second approach has the benefit of avoiding creating new root properties but seems too complicated for our case (closed list of only 3 types). I have updated all the HAL samples using the first approach ("type" property added to responses of type "dataset" and "dimension").
"_links": {"self": { "href": "http://...", "type": "application/dimension+json" }
}
Thank you for relentless work! I have been following this thread, but haven't had the time to post a reply.
To cut to the chase; I am also between 2) and 3), but leaning towards HAL. It's correct that HAL is not particularly wide spread, but it has more and more traction, and there are allready tools out like the HAL browser (https://github.com/mikekelly/hal-browser)
I like the simplicity of 3) though!
Anyway: I vote for 2)
Your vote has been counted!
Let's hear Erlend's opinion on this issue.
BTW, what I called the "href" proposal (3)
http://json-stat.org/samples/href/dataset.json
{
"href": "http://json-stat.org/samples/href/dataset.json",
"links": {
"alternate": [
{ "href": "http://json-stat.org/samples/href/us-gsp.csv", "type": "text/csv" },
{ "href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP", "type": "text/html" }
]
},
...
}
can be rewritten in Collection+JSON style:
{
"href": "http://json-stat.org/samples/href/dataset.json",
"links": [
{ "rel": "alternate", "href": "http://json-stat.org/samples/href/us-gsp.csv", "type": "text/csv" },
{ "rel": "alternate", "href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP", "type": "text/html" }
],
...
}
In fact, I think this is valid C+J (and I like the idea of keeping "self" apart, like in the "href" proposal). This can be considered a 4) possibility (or a particular expression of 1).
X.
What i don't like about HAL is the need for detection parsing, meaning the link MAY be an
array of strings or a single string.
"_links": {"self": {},"alternate": [{"type": "text/html"}]}
How does a partial response look like? I can't seem to find it.
{"dataset" : {"value" : [174400, 1.2, 4.8, 36333, 45600, 0.31, 0.7, 65143, 261300, 1.8, 6.4, 40828, 105800, 0.73, 2.9, 36483, 2080600, 13.34, 37.3, 51914, 259700, 1.79, 5, 51940, 233400, 1.61, 3.6, 64833, 62700, 0.43, 0.9, 69667, 104700, 0.72, 0.6, 174500, 754000, 5.2, 18.8, 40106, 403100, 2.79, 9.7, 41711, 68900, 0.47, 1.4, 49214, 54800, 0.38, 1.6, 34250, 644200, 4.44, 12.8, 50328, 267600, 1.84, 6.5, 41169, 147200, 1.01, 3, 49067, 128500, 0.89, 2.9, 44310, 161400, 1.11, 4.3, 37535, 213600, 1.47, 4.5, 47467, 53200, 0.37, 1.3, 40923, 300000, 2.07, 5.8, 51724, 377700, 2.6, 6.5, 58108, 372400, 2.57, 9.9, 37616, 267100, 1.84, 5.3, 50396, 98900, 0.68, 3, 32967, 246700, 1.7, 6, 41117, 37200, 0.26, 1, 37200, 89600, 0.62, 1.8, 49778, 127500, 0.88, 2.7, 47222, 61600, 0.42, 1.3, 47385, 497000, 3.42, 8.8, 56477, 75500, 0.52, 2.1, 35952, 1156500, 7.68, 19.4, 57423, 407400, 2.81, 9.5, 42884, 33400, 0.23, 0.7, 47714, 483400, 3.33, 11.5, 42035, 160500, 1.11, 3.8, 42237, 168900, 1.16, 3.8, 44447, 575600, 3.97, 12.7, 45323, 49500, 0.34, 1.1, 45000, 164300, 1.13, 4.6, 35717, 39900, 0.27, 0.8, 49875, 250300, 1.72, 6.3, 39730, 1458300, 8.92, 25.1, 58099, 116900, 0.81, 2.8, 41750, 26400, 0.18, 0.6, 44000, 427700, 2.95, 8, 53463, 351100, 2.42, 6.7, 52403, 66600, 0.46, 1.9, 35053, 251400, 1.73, 5.7, 44105, 38200, 0.26, 0.6, 63667]}}
1) On HALWhat i don't like about HAL is the need for detection parsing, meaning the link MAY be an
array of strings or a single string.I don't like it either (it's true that JSON-stat has also elements that can have several types, but that's because the producer is free to choose between intrinsically different ways of specifying things, not because of numerality). Every time I said HAL I meant the simplified (stricter) version introduced in this post:In this simplified version, the "self" element is always an object (with a single property: "href") and the other relations are always an array of objects (unless we decide it's even simpler to use an array for all relations, "self" included, even if it doesn't make too much sense for "self").Example:"_links": {"self": {},"alternate": [{"type": "text/html"}]}Check for example these HAL-compliant samples:
2) On partial responsesHow does a partial response look like? I can't seem to find it.I haven't uploaded any partial response sample (in the meaning of this post:) as a partial response can be any JSON-stat response with arbitrary missing elements ("Arbitrary" = freely discarded by the client): I have only uploaded full native responses.Here you have an example of a partial response:The client requests a bundle-type response that has only one dataset (named "dataset") and asks to receive the "value" element only. This is what the client gets:{"dataset" : {"value" : [174400, 1.2, 4.8, 36333, 45600, 0.31, 0.7, 65143, 261300, 1.8, 6.4, 40828, 105800, 0.73, 2.9, 36483, 2080600, 13.34, 37.3, 51914, 259700, 1.79, 5, 51940, 233400, 1.61, 3.6, 64833, 62700, 0.43, 0.9, 69667, 104700, 0.72, 0.6, 174500, 754000, 5.2, 18.8, 40106, 403100, 2.79, 9.7, 41711, 68900, 0.47, 1.4, 49214, 54800, 0.38, 1.6, 34250, 644200, 4.44, 12.8, 50328, 267600, 1.84, 6.5, 41169, 147200, 1.01, 3, 49067, 128500, 0.89, 2.9, 44310, 161400, 1.11, 4.3, 37535, 213600, 1.47, 4.5, 47467, 53200, 0.37, 1.3, 40923, 300000, 2.07, 5.8, 51724, 377700, 2.6, 6.5, 58108, 372400, 2.57, 9.9, 37616, 267100, 1.84, 5.3, 50396, 98900, 0.68, 3, 32967, 246700, 1.7, 6, 41117, 37200, 0.26, 1, 37200, 89600, 0.62, 1.8, 49778, 127500, 0.88, 2.7, 47222, 61600, 0.42, 1.3, 47385, 497000, 3.42, 8.8, 56477, 75500, 0.52, 2.1, 35952, 1156500, 7.68, 19.4, 57423, 407400, 2.81, 9.5, 42884, 33400, 0.23, 0.7, 47714, 483400, 3.33, 11.5, 42035, 160500, 1.11, 3.8, 42237, 168900, 1.16, 3.8, 44447, 575600, 3.97, 12.7, 45323, 49500, 0.34, 1.1, 45000, 164300, 1.13, 4.6, 35717, 39900, 0.27, 0.8, 49875, 250300, 1.72, 6.3, 39730, 1458300, 8.92, 25.1, 58099, 116900, 0.81, 2.8, 41750, 26400, 0.18, 0.6, 44000, 427700, 2.95, 8, 53463, 351100, 2.42, 6.7, 52403, 66600, 0.46, 1.9, 35053, 251400, 1.73, 5.7, 44105, 38200, 0.26, 0.6, 63667]}}This an only-data response (apparently the client is already in possession of all the necessary metadata to make sense of these data, maybe from previous requests).Partial responses are supported in several Google APIs. See for example:I think Facebook supports something very similar. Also LinkedIn:If you need more info, let me know,X.
Yes. This is good.
{ds1: {...},ds2: {...}}
{
_links: {...},
ds1: {...},ds2: {...}}
{
_links: {...},
dataset: {
ds1: {...},ds2: {...}}
}