hypermedia-format

90 views
Skip to first unread message

Erlend Hamnaberg

unread,
Nov 11, 2013, 6:19:39 AM11/11/13
to json...@googlegroups.com
Hi all.

I have been looking quite a bit at this format, and there are some good ideas here.

I was wondering if you would consider adding link support, making it a hypermedia format.

Why you ask?

Adding links would enable us to utilize the Web as intended. 
Like linking to alternate representations of the same data: HTML table, CSV etc.
It could also be possible to link to other types of information which are somewhat related to the dataset.

I would suggest that a new property links is added to the dataset object.

It could look something like this:

...
"links": [
  {"href": "uri-to-some-site", "rel": "related", "type": "text/html"},
  {"href": "uri-to-where-you-found-the-data", "rel": "self"},
  {"href": "uri-to-alternate-representation", "rel": "alternate", "type": "application/vnd.collection+json", "title": "Alternate in Collection+json"},
  {"href": "uri-to-alternate-representation", "rel": "alternate", "type": "text/csv",  "title": "Alternate in CSV"}
]


What do you think?

This has other advantages, like for instance adding profile support, so you could for instance include GEOJSON where it would be useful, or other things.


Syntax is inspired by collection+json

--
Erlend

Xavier Badosa

unread,
Dec 1, 2013, 4:28:34 AM12/1/13
to
Erland,

I was wondering if you would consider adding link support, making it a hypermedia format.


Absolutely! 

In fact, it would be wonderful if JSON-stat-based APIs were 100% REST APIs (including the HATEOAS constraint): imagine connecting to a single endpoint of the Open Data Initiative of a certain statistical office and discovering from there all the possible actions and resources available!

Our current focus though is on the type of response that is able to express data in table (cube) form. At some point, JSON-stat will need to support other types different from this core "datasets' response" (for these types, probably some Atom flavor format could do the job).

The reason I avoided URIs in the first version of the documentation (there were actually links in previous drafts) was: URIs are so powerful that can play many roles. I wasn't sure these roles were compatible, or at least I tried to avoid possible conflict by avoiding URIs altogether. I also was aware that my first proposals were probably too naif but unfortunately the current state of the art (JSON hypermedia) seamed too shaky ground at the time.

In particular, I think URIs can be useful in JSON-stat for:

1) Discovery: Hypermedia. In an early draft there was a simple "uri" element (child of the "dataset" element) that was a natural companion of the "source" element.

2) Semantics: the use of URIs to identify the resources used in a JSON-stat response, to make the format more semantic and interoperable.

3) Support for partial responses (this is probably the same as 2): see for example: Shared vocabularies as partial responses.

I didn't know collection+json (I've been a member for several years of HAL, though, and now I've just discovered Siren).

My questions then are:

A) collection+json vs. HAL vs. Siren: What standard should we adopt for this?

B) Links and Identity (hypermedia and semantics/partial responses): Should these two issues concerning URIs be faced at the same time? Or should they be addressed separately?

C) Levels: What nodes of the JSON-stat tree should support links? In your proposal only the "dataset" element supports them, but this seems too poor and, besides, partial responses need URIs deeper in the tree.

So, to summarize, yes, please, lets make JSON-stat a hypermedia format. But before doing so I think we need to try to answer at least these three questions. What do you think?

X.

Erlend Hamnaberg

unread,
Dec 1, 2013, 4:28:07 AM12/1/13
to
Hello Xavier.

On Mon, Nov 11, 2013 at 6:25 PM, Xavier Badosa wrote:
Erland,

I was wondering if you would consider adding link support, making it a hypermedia format.


Absolutely! 

In fact, it would be wonderful if JSON-stat-based APIs were 100% REST APIs (including the HATEOAS constraint): imagine connecting to a single endpoint of the Open Data Initiative of a certain statistical office and discovering from there all the possible actions and resources available!

Our current focus though is on the type of response that is able to express data in table (cube) form. At some point, JSON-stat will need to support other types different from this core "datasets' response" (for these types, probably some Atom flavor format could do the job).

The reason I avoided URIs in the first version of the documentation (there were actually links in previous drafts) was: URIs are so powerful that can play many roles. I wasn't sure these roles were compatible, or at least I tried to avoid possible conflict by avoiding URIs altogether. I also was aware that my first proposals were probably too naif but unfortunately the current state of the art (JSON hypermedia) seamed too shaky ground at the time.

In particular, I think URIs can be useful in JSON-stat for:

1) Discovery: Hypermedia. In an early draft there was a simple "uri" element (child of the "dataset" element) that was a natural companion of the "source" element.

I think this should be called "href" to be more in line with other formats.

2) Semantics: the use of URIs to identify the resources used in a JSON-stat response, to make the format more semantic and interoperable.

3) Support for partial responses (this is probably the same as 2): see for example: Shared vocabularies as partial responses.

How do you know that this is a partial response? What are the processing rules?
Is a partial (category) response, a separate media type?
 

I didn't know collection+json (I've been a member for several years of HAL, though, and now I've just discovered Siren).

My questions then are:

A) collection+json vs. HAL vs. Siren: What standard should we adopt for this?

I favour c+j (with additions) as the lists make more sense for me. 

The HAL version allows uri-templates, however, which c+j does not. 
C+J has "queries" which enables this feature.

Does the HAL version allow for multiple links with the same relation?
The HAL version seems much more complicated than it needs to be.

 
B) Links and Identity (hypermedia and semantics/partial responses): Should these two issues concerning URIs be faced at the same time? Or should they be addressed separately?

They can be either or, it does not really matter.
 

C) Levels: What nodes of the JSON-stat tree should support links? In your proposal only the "dataset" element supports them, but this seems too poor and, besides, partial responses need URIs deeper in the tree.

The dataset was where I found it natural to put at least one block of links. I did not know about the partial response thingy.
I suggest enumerating all positions where it might be natural to include links, and we'll see how to proceed from there.
 

So, to summarize, yes, please, lets make JSON-stat a hypermedia format. But before doing so I think we need to try to answer at least these three questions. What do you think?

X.


-- Erlend 

Xavier Badosa

unread,
Dec 1, 2013, 4:27:39 AM12/1/13
to
> I suggest enumerating all positions where it might be natural to include links, and we'll see how to proceed from there.

The JSON-stat tree is this:

  dataset
     value
     dimension
        category

Links seem to make sense in all but the value node. In all these nodes different types of links (different "rel"s) make sense: alternative content types, IDs (rel="self"), etc.

X.

Xavier Badosa

unread,
Dec 1, 2013, 4:26:46 AM12/1/13
to
Probably, in only two elements of the JSON-stat tree links seem natural: dataset and dimension.

Taking as an example this JSON-stat response (http://json-stat.org/samples/us-gsp.json) which contains one dataset and three dimensions (year, state, concept), this is how links should be probably expressed according to HAL-, Collection+JSON- and Siren-inspired formats.

Collection+JSON

"dataset" : {
"links": [
{"href": "http://json-stat.org/samples/cj.json", "rel": "self"},
{"href": "http://json-stat.org/samples/cj.csv", "rel": "alternate", "type": "text/csv"}
{"href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP", "rel": "alternate", "type": "text/html"}
],
"label" : "US States by GSP and population",
...


"concept" : {
"links": [
],
"label" : "Concepts",
...

"year" : {
"links": [
],
"label" : "Year",
 ...

"state" : {
"links": [
{"href": "http://en.wikipedia.org/wiki/List_of_U.S._state_abbreviations", "rel": "alternate", "type": "text/html"}
],
"label" : "State",
 ...

Siren

Same as Collection+JSON but "rel" is an array of strings instead of an array (["self"] instead of "self").

HAL

HAL accepts similar semantics but uses a different structure, more in line with JSON-stat as "rel" is actually the key:

"dataset" : {
"_links": {
"self": { "href": "/samples/hal.json" },
},

X.

Xavier Badosa

unread,
Dec 1, 2013, 4:26:17 AM12/1/13
to json...@googlegroups.com
Same as Collection+JSON but "rel" is an array of strings instead of an array (["self"] instead of "self").

Typo: I meant to say:  "rel" is an array of strings instead of a string

Answering Erlend:

Does the HAL version allow for multiple links with the same relation?

Yes, even though HAL seems to assume that the common case is one link per rel (an object instead of an array of objects). The full dataset example in HAL:

"dataset" : {
"_links": {
"self": { "href": "http://json-stat.org/samples/hal.json" },
"alternate": [
{ "href": "http://json-stat.org/samples/hal.csv", "type": "text/csv" },
]
},

(In this sample code, it is assumed that this response was the result of the following request:


I plan to implement a working example when a syntax has been chosen.)

So the main differences (Erlend, correct me if I'm wrong) between the 3 approaches are, from a JSON-stat perspective:

1) Siren seems to have special care of links with several relations. Even if there is a use case for this, it doesn't seem the Siren structure is particularly easy to parse, specially when you only want to get a URI (the "self" rel) to check a partial response.

2) C+J seems similar but simpler than Siren, but you still have to loop through links to retrieve a particular rel. JSON-stat has generally tried to avoid looping as much as possible for regular use cases: a C+J structure doesn't seem to fit well in a JSON-stat structure. For partial response support, retrieving the "self" rel URI should be easy. In JSON-stat style, it should something like dataset.links.self.href.

3) HAL seems to have added support for rel with several links as a second thought which might make links difficult to parse as they can be objects or arrays (side note: this is also a feature odf JSON-stat). It uses the _ prefix (_links) that no one else uses (do we really need a prefix acting as a name space?). But HAL "fits well" inside JSON-stat. Particularly if we simplify it: a) "self" will always be an object; b) the rest of rels (particularly "alternate") will always be arrays of objects.

So my bet is 3) (simplification included). What's yours?

By the way, because JSON-stat responses can include more than one dataset, it make sense to support links at the JSON-stat root. For example (ds1 and ds2 are datasets):

{
ds1: {...},
ds2: {...},
_links: {
} 
}

X.

Xavier Badosa

unread,
Dec 3, 2013, 2:50:26 PM12/3/13
to json...@googlegroups.com

Xavier Badosa

unread,
Dec 4, 2013, 12:02:13 PM12/4/13
to json...@googlegroups.com
Some notes on the previous sample responses:

Because a full JSON-stat response is a collection of datasets, the following requests return slightly different things:

Collection containing a single dataset (full response)

Dataset partial response

Because all dimensions can be retrieved on their own using dimension partial responses

there should be a way to retrieve this same single dataset bundle without including the dimensions that don't usually change over time (concept and state, for example). Besides, some classifications are standard and are shared between several datasets (for example, the US State nomenclature).

This is how a partial response that does not include the concept and state dimensions might look:


Thanks to dataset.dimension.state._links.self.href it is possible, not only (ID function) to verify that the state classification hasn't changed since we last retrieved it (the URI would probably include a version number), but also to actually update it or retrieve it for the first time (hyperlink function: it could point to a standards organization serving classifications as JSON-stat partial responses).

How does a client discover requests similar to


or even more radical (include only the value array; or return everything but the value array)? This problem should probably be addressed through some sort of template solution that allows you to expose the JSON-stat Query Language (JSTQL), or (worse) defining some JSON-stat specific rels for common queries like in this example:

"dataset" : {
"_links": {
"dataonly": [ { "href": "http://json-stat.org/samples/hal/us-gsp.json?keep=value" } ], //don't try it: this call is not implemented
"nodata": [ { "href": "http://json-stat.org/samples/hal/us-gsp.json?remove=value" } ], //don't try it: this call is not implemented
"alternate": [
{ "href": "http://json-stat.org/samples/hal/us-gsp.csv", "type": "text/csv" },
]
}, ...

I personally don't like this last solution as it is not flexible enough (it reminds me of the old SDMX approach) and it is not really consistent with a powerful JSTQL.

Again, feedback is welcome,

X.

Xavier Badosa

unread,
Dec 6, 2013, 11:48:25 AM12/6/13
to
How does a client discover requests similar to


or even more radical (include only the value array; or return everything but the value array)? This problem should probably be addressed through some sort of template solution that allows you to expose the JSON-stat Query Language (JSTQL), or (worse) defining some JSON-stat specific rels for common queries like in this example:

To answer this question, maybe we should distinguish between (real) partial responses and (full and partial) native responses. 

Native responses

A native response is any response of type "bundle" (default), "dataset" and "dimension".

The type should be stated in the Content-Type header somehow. Some possibilities:

application/jsonstat+json;type=dataset
application/dataset+json

For practical reasons, it's interesting to include the type in the response body. This could be achieved using a new special JSON-stat string property ("type") allowing three possible values ("bundle" -default-, "dataset", "dimension") or a more general web linking mechanism (in HAL, cj or siren syntax):

"_links": {
   "self": { ... },
   "schema": { href: "http://json-stat.org/json-schema/dataset" }
 }

The second approach has the benefit of avoiding creating new root properties but seems too complicated for our case (closed list of only 3 types). I have updated all the HAL samples using the first approach ("type" property added to responses of type "dataset" and "dimension").

Partial responses

A partial response is any arbitrary fragment of a native response. Responses that are partial keep their original type.

The fragment is defined by a JSTQL query (for example, do not include certain properties in the response).

Defined like this, partial responses shouldn't be, in my opinion, considered for web linking ("_links", "links"): no need for "dataonly" or even templates. The JSTQL is in a different layer and should be described there, not in the response. 

It also means that "self" always points to a native response of some type.

X.

Xavier Badosa

unread,
Dec 6, 2013, 11:55:15 AM12/6/13
to json...@googlegroups.com
For practical reasons, it's interesting to include the type in the response body. This could be achieved using a new special JSON-stat string property ("type") allowing three possible values ("bundle" -default-, "dataset", "dimension") or a more general web linking mechanism (in HAL, cj or siren syntax):

"_links": {
   "self": { ... },
   "schema": { href: "http://json-stat.org/json-schema/dataset" }
 }

The second approach has the benefit of avoiding creating new root properties but seems too complicated for our case (closed list of only 3 types). I have updated all the HAL samples using the first approach ("type" property added to responses of type "dataset" and "dimension").

 A third approach: using the "self" object to specify "type":

"_links": {
   "self": { "href": "http://...", "type": "application/dimension+json" }
 }

That would be consistent with the header.

Xavier Badosa

unread,
Dec 6, 2013, 12:47:46 PM12/6/13
to json...@googlegroups.com
The common approach of all the competing JSON hypermedia standards is to group all links under a property ("links", "_links"). It is obvious though that the "self" link is really different from the other links: it is always unique and required, its main goal is to ID and the only child it really needs is "href"; on the other hand, the other links are optional, can be multiple and their "type" is usually important.

So I've been thinking if there could be a simpler approach... and then I discovered the Stormpath API


They basically avoid the "links" property altogether and attach "href" directly to the affected element. And then they support something call "link expansion"


that is very similar to what we need.

I have tried a mixed approach: keep the optional "links" object but move the "self" property outside of it. The result can be seen here:


This bundle contains mainly a reference to a single dataset:


The dataset contains alternate links but also references to three dimensions:


The expanded JSON-stat bundle response can be retrieved like this:


Observe that the "type" element is not needed in the "bundle" response (expanded or not).

This terse response is very similar to early proposals in this group ("href" was called "uri" then) 

https://groups.google.com/d/msg/json-stat/Y91AN71QkaY/EKL7i8zwrPEJ (Warning: old meaning of "partial response")

but has the addition of the "links" elements to allow discovery of related content.

X.

Xavier Badosa

unread,
Dec 7, 2013, 12:46:16 PM12/7/13
to json...@googlegroups.com
1) A Collection+JSON- or Siren- inspired approach in JSON-stat

Both formats have a broader scope ("collections", "entities"). Consequence: using any of them won't mean strictly adopting them but taking inspiration from them.

2) HAL

HAL is older than C+J or Siren, is less verbose than these and simpler (also less powerful: query templates, form actions...). It's consistent with JSON-stat.

3) "href"

That is, the simple approach than the current competing formats described in my previous message and illustrated at


This third option is simpler than 1) and 2) but is not based on any of the existing formats. Using it would sound crazy if 1) or 2) had a broad adoption (but that doesn't seem to be the case, does it?). Fortunately, like 1) and 2), 3) is based on more fundamental standards (Web Linking:


).

I'm between 2) and 3), but with a preference for 3) unless someone proves to me there's a big adoption of HAL.

Users of JSON-stat, what do you vote for? 1, 2 or 3?

X.

Trygve Falch

unread,
Dec 7, 2013, 2:46:50 PM12/7/13
to json...@googlegroups.com
Hi Xavier!

Thank you for relentless work! I have been following this thread, but haven't had the time to post a reply.

To cut to the chase; I am also between 2) and 3), but leaning towards HAL. It's correct that HAL is not particularly wide spread, but it has more and more traction, and there are allready tools out like the HAL browser (https://github.com/mikekelly/hal-browser)

I like the simplicity of 3) though!

Anyway: I vote for 2)

Xavier Badosa

unread,
Dec 10, 2013, 2:17:52 PM12/10/13
to json...@googlegroups.com
Trygve,

Your vote has been counted!

Let's hear Erlend's opinion on this issue.

BTW, what I called the "href" proposal (3)

http://json-stat.org/samples/href/dataset.json

{
"href": "http://json-stat.org/samples/href/dataset.json",
"links": {
"alternate": [
{ "href": "http://json-stat.org/samples/href/us-gsp.csv", "type": "text/csv" },


{ "href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP", "type": "text/html" }
]
},
...
}

can be rewritten in Collection+JSON style:

{
"href": "http://json-stat.org/samples/href/dataset.json",
"links": [
{ "rel": "alternate", "href": "http://json-stat.org/samples/href/us-gsp.csv", "type": "text/csv" },
{ "rel": "alternate", "href": "http://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP", "type": "text/html" }
],
...
}

In fact, I think this is valid C+J (and I like the idea of keeping "self" apart, like in the "href" proposal). This can be considered a 4) possibility (or a particular expression of 1).

X.

Xavier Badosa

unread,
Jan 4, 2014, 7:29:04 AM1/4/14
to json...@googlegroups.com, Erlend Hamnaberg
After the holidays, I guess it's time to close this issue (option 2 vs. option 3 in 


) that was brought back to the table by Erlend


My bet for option 2 was based on a certain distaste for the use of the own URI as a link relationship as 

a) this URI has a different function (ID, not discovery), 
b) this URI has different existence conditions (it might not be specified but there's always an ID while the rest of the links can exist or not),
c) considering this URI as a link makes things complicated in elements that can't/don't have discovery links.

I've recently seen people sharing my view in the Siren group from a developer perspective. For example, David Hansen


or Matt Bishop (from Elastic Path)


That said, "self" as a link relation is deep-rooted in web standards


and it is used in all current hypermedia standards.

Finally, HAL has recently adopted by Amazon in its AppStream API


All things considered, it's probably safer to follow the conservative route (2 = HAL) as Lars suggested.

So unless there's someone expressing a strong opinion to the contrary, I'll work on a new draft of the specs to include HAL.

X.

PS: Implications for existing JSON-stat libraries in a later message.

Erlend Hamnaberg

unread,
Jan 7, 2014, 10:52:55 AM1/7/14
to Xavier Badosa, json...@googlegroups.com
Hello guys.

I have been offline for a month now, and not been able to reply.

HAL seems to be the most in spirit with json-stat, and seems like a good choice.

What i don't like about HAL is the need for detection parsing, meaning the link MAY be an
array of strings or a single string. 

Similar approaches can be found in json-stat, so I think it probably doesnt matter.

How does a partial response look like? I can't seem to find it.

--
Erlend

Xavier Badosa

unread,
Jan 7, 2014, 2:24:30 PM1/7/14
to json...@googlegroups.com, Xavier Badosa
1) On HAL
 
What i don't like about HAL is the need for detection parsing, meaning the link MAY be an
array of strings or a single string. 

I don't like it either (it's true that JSON-stat has also elements that can have several types, but that's because the producer is free to choose between intrinsically different ways of specifying things, not because of numerality). Every time I said HAL I meant the simplified (stricter) version introduced in this post:


In this simplified version, the "self" element is always an object (with a single property: "href") and the other relations are always an array of objects (unless we decide it's even simpler to use an array for all relations, "self" included, even if it doesn't make too much sense for "self").

Example:

"_links": {
  "self": {
  },
  "alternate": [
    { 
      "type": "text/html" 
    }
  ]
}

Check for example these HAL-compliant samples:


2) On partial responses

How does a partial response look like? I can't seem to find it.

I haven't uploaded any partial response sample (in the meaning of this post:


) as a partial response can be any JSON-stat response with arbitrary missing elements ("Arbitrary" = freely discarded by the client): I have only uploaded full native responses.

Here you have an example of a partial response: 

The client requests a bundle-type response that has only one dataset (named "dataset") and asks to receive the "value" element only. This is what the client gets:

{
   "dataset" : {
      "value" : [
174400, 1.2, 4.8, 36333, 45600, 0.31, 0.7, 65143, 261300, 1.8, 6.4, 40828, 105800, 0.73, 2.9, 36483, 2080600, 13.34, 37.3, 51914, 259700, 1.79, 5, 51940,  233400, 1.61, 3.6, 64833, 62700, 0.43, 0.9, 69667, 104700, 0.72, 0.6, 174500, 754000, 5.2, 18.8, 40106, 403100, 2.79, 9.7, 41711, 68900, 0.47, 1.4, 49214, 54800, 0.38, 1.6, 34250, 644200, 4.44, 12.8, 50328, 267600, 1.84, 6.5, 41169, 147200, 1.01, 3, 49067, 128500, 0.89, 2.9, 44310, 161400, 1.11, 4.3, 37535, 213600, 1.47, 4.5, 47467, 53200, 0.37, 1.3, 40923, 300000, 2.07, 5.8, 51724, 377700, 2.6, 6.5, 58108, 372400, 2.57, 9.9, 37616, 267100, 1.84, 5.3, 50396, 98900, 0.68, 3, 32967, 246700, 1.7, 6, 41117, 37200, 0.26, 1, 37200, 89600, 0.62, 1.8, 49778, 127500, 0.88, 2.7, 47222, 61600, 0.42, 1.3, 47385, 497000, 3.42, 8.8, 56477, 75500, 0.52, 2.1, 35952, 1156500, 7.68, 19.4, 57423, 407400, 2.81, 9.5, 42884, 33400, 0.23, 0.7, 47714, 483400, 3.33, 11.5, 42035, 160500, 1.11, 3.8, 42237, 168900, 1.16, 3.8, 44447, 575600, 3.97, 12.7, 45323, 49500, 0.34, 1.1, 45000, 164300, 1.13, 4.6, 35717, 39900, 0.27, 0.8, 49875, 250300, 1.72, 6.3, 39730, 1458300, 8.92, 25.1, 58099, 116900, 0.81, 2.8, 41750, 26400, 0.18, 0.6, 44000, 427700, 2.95, 8, 53463, 351100, 2.42, 6.7, 52403, 66600, 0.46, 1.9, 35053, 251400, 1.73, 5.7, 44105, 38200, 0.26, 0.6, 63667
      ]
   }
}

This an only-data response (apparently the client is already in possession of all the necessary metadata to make sense of these data, maybe from previous requests).

Partial responses are supported in several Google APIs. See for example:


I think Facebook supports something very similar. Also LinkedIn:


If you need more info, let me know,

X.

Erlend Hamnaberg

unread,
Jan 13, 2014, 4:14:41 AM1/13/14
to json...@googlegroups.com, Xavier Badosa
Hello.


On Tue, Jan 7, 2014 at 8:24 PM, Xavier Badosa <xba...@gmail.com> wrote:
1) On HAL
 
What i don't like about HAL is the need for detection parsing, meaning the link MAY be an
array of strings or a single string. 

I don't like it either (it's true that JSON-stat has also elements that can have several types, but that's because the producer is free to choose between intrinsically different ways of specifying things, not because of numerality). Every time I said HAL I meant the simplified (stricter) version introduced in this post:


In this simplified version, the "self" element is always an object (with a single property: "href") and the other relations are always an array of objects (unless we decide it's even simpler to use an array for all relations, "self" included, even if it doesn't make too much sense for "self").

Example:

"_links": {
  "self": {
  },
  "alternate": [
    { 
      "type": "text/html" 
    }
  ]
}

Check for example these HAL-compliant samples:


Yes. This is good.
 

2) On partial responses

How does a partial response look like? I can't seem to find it.

I haven't uploaded any partial response sample (in the meaning of this post:


) as a partial response can be any JSON-stat response with arbitrary missing elements ("Arbitrary" = freely discarded by the client): I have only uploaded full native responses.

Here you have an example of a partial response: 

The client requests a bundle-type response that has only one dataset (named "dataset") and asks to receive the "value" element only. This is what the client gets:

{
   "dataset" : {
      "value" : [
174400, 1.2, 4.8, 36333, 45600, 0.31, 0.7, 65143, 261300, 1.8, 6.4, 40828, 105800, 0.73, 2.9, 36483, 2080600, 13.34, 37.3, 51914, 259700, 1.79, 5, 51940,  233400, 1.61, 3.6, 64833, 62700, 0.43, 0.9, 69667, 104700, 0.72, 0.6, 174500, 754000, 5.2, 18.8, 40106, 403100, 2.79, 9.7, 41711, 68900, 0.47, 1.4, 49214, 54800, 0.38, 1.6, 34250, 644200, 4.44, 12.8, 50328, 267600, 1.84, 6.5, 41169, 147200, 1.01, 3, 49067, 128500, 0.89, 2.9, 44310, 161400, 1.11, 4.3, 37535, 213600, 1.47, 4.5, 47467, 53200, 0.37, 1.3, 40923, 300000, 2.07, 5.8, 51724, 377700, 2.6, 6.5, 58108, 372400, 2.57, 9.9, 37616, 267100, 1.84, 5.3, 50396, 98900, 0.68, 3, 32967, 246700, 1.7, 6, 41117, 37200, 0.26, 1, 37200, 89600, 0.62, 1.8, 49778, 127500, 0.88, 2.7, 47222, 61600, 0.42, 1.3, 47385, 497000, 3.42, 8.8, 56477, 75500, 0.52, 2.1, 35952, 1156500, 7.68, 19.4, 57423, 407400, 2.81, 9.5, 42884, 33400, 0.23, 0.7, 47714, 483400, 3.33, 11.5, 42035, 160500, 1.11, 3.8, 42237, 168900, 1.16, 3.8, 44447, 575600, 3.97, 12.7, 45323, 49500, 0.34, 1.1, 45000, 164300, 1.13, 4.6, 35717, 39900, 0.27, 0.8, 49875, 250300, 1.72, 6.3, 39730, 1458300, 8.92, 25.1, 58099, 116900, 0.81, 2.8, 41750, 26400, 0.18, 0.6, 44000, 427700, 2.95, 8, 53463, 351100, 2.42, 6.7, 52403, 66600, 0.46, 1.9, 35053, 251400, 1.73, 5.7, 44105, 38200, 0.26, 0.6, 63667
      ]
   }
}

This an only-data response (apparently the client is already in possession of all the necessary metadata to make sense of these data, maybe from previous requests).

Partial responses are supported in several Google APIs. See for example:


I think Facebook supports something very similar. Also LinkedIn:


If you need more info, let me know,

X.


Not sure if following google or facebook is a good idea here. as they advocate lock-in.
I think a partial response should be well-defined and specified, not just random fields.
HTTP advocates course grained media types, meaning that most uses of "Partial responses" are 
working against HTTP. Caching solves a lot of these usecases. 

I see, however, the problem of defining re-usable dimensions. And I guess most of that could be defined with
the "describedBy" relation and some addressing scheme. Using the fragment identifier of a URI seems reasonable to identify a
single category within a dimension.

Utilizing the Hypermedia Factors [1] may be a good idea for creating something which may help us
designing something good here.


[1]: http://amundsen.com/hypermedia/hfactor/


--
Erlend

Xavier Badosa

unread,
Jan 15, 2014, 2:43:54 PM1/15/14
to json...@googlegroups.com
1) On HAL

Yes. This is good.

Perfect. I'll update the specs to include this when I have a moment.

In my examples, _links were supported at the bundle level, at the dataset level and at the dimension level. Supporting them at the bundle level is problematic because datasets in a bundle don't have a container. Ex.;

{
   ds1: {...},
   ds2: {...}
}

Options:

a) _links is not a valid dataset name: 

{
   _links: {...},
   ds1: {...},
   ds2: {...}
}

This is not very elegant and it's a little messy for parsers.

b) datasets are inside a container:

{
   _links: {...},
   dataset: {
      ds1: {...},
      ds2: {...}
   }
}
 
This adds a new layer to the structure and it's not backward-compatible with the current specs.

c) _links is not supported at the bundle level

This doesn't seem to be a very good idea. Bundles won't always be arbitrary collections of datasets (the oecd-canada.json is an arbitrary collection for demo-only purposes).

Change "bundle" for "tabulation" and "dataset" for "table" and this becomes clearer: imagine you need to express the full tabulation of a survey in JSON-stat. You could send a bundle with a single big table ("dataset") simultaneously crossing all the variables ("dimensions") in the survey. Or, to avoid empty cells, you could choose to send a bundle with many smaller tables ("datasets"). In this second case, _links is a useful property to link to the webpage of such tabulation or to the PDF version if available, or to reference the methodology of the survey.

On the other hand, expressing full tabulations of surveys in JSON-stat is not a very likely use case. That's probably also why JSON-stat does not even support labels or other metadata for bundles: a bundle is currently a convenient way to pack in a single response several tables. From this perspective, not supporting links at the bundle level fits well in current JSON-stat.

a), b) or c) ? Not an easy choice but after considering all the pros and cons, my vote goes to c).

Where goes yours?

2) On partial responses

I see re-usable dimensions as a problem of supporting several native responses (bundle, dataset, dimension), not about partial responses (which actually are somehow out of the scope of the response format and are about the query language).

I don't fully see the need for the "describedBy" relation in this context. I understand that you see the dimension link at the dataset level (the dataset is "described by" the dimensions). I think it's a simpler and more general solution as a "self" relation at the proper level ("dimension").

The idea of using the fragment identifier of a URI to identify a single category within a dimension is interesting. The State of New York would look like this


right?

X.

Xavier Badosa

unread,
Feb 2, 2014, 2:18:38 PM2/2/14
to json...@googlegroups.com
I've been busy in another project and I've been unable to move JSON-stat forward in the last weeks. But I'll try now. Of course, all inputs are still welcome.

I'm studying a little deeper the hypermedia formats before closing this issue. At the same time I'm working on the JSON-stat Javascript Toolkit v. 0.7.0. It's a big leap from current version 0.6.2 as it will accept all kinds of native responses (not only "bundles") and should be able to combine them in a single complete response. This will help me decide in practical terms what's the best approach.

X.

Xavier Badosa

unread,
Feb 8, 2014, 5:04:42 AM2/8/14
to json...@googlegroups.com
In the previous discussions in this thread, there are probably three distinct issues that, even though they are connected, probably deserve to be treated separately. In my opinion, these issues are: 

1) JSON-stat and the collection pattern

2) JSON-stat hypermedia enrichment

3) Response types in JSON-stat (aka native responses)

I'll begin with the first one in a new thread (JSON-stat as a collection pattern format).

X.
Reply all
Reply to author
Forward
0 new messages