Domain specific media types?

172 views
Skip to first unread message

Jørn Wildt

unread,
Apr 10, 2012, 5:24:49 AM4/10/12
to hyperme...@googlegroups.com
I have always been puzzled about the scope of media types as a concept. In Fielding's now famous rant [http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven] he states that "A REST API should never have “typed” resources that are significant to the client.". Further down in the comments he clarifies "It [REST] has value because it is far easier to standardize representation and relation types than it is to standardize objects and object-specific interfaces."

Stuart Charlton [http://www.stucharlton.com/blog/archives/000551.html] puts it this way (my emphasis):

The main problem when you jump into the enterprise with the Web Architecture is the semantic interoperability problem. This is the same problem with other architectures, but with REST's benefits of easier interoperability, it becomes more important. We might be able to specify media types for infrastructure-level topics, like identity, authentication, or syndication, or content publishing, but we have much less success with, say, "Job Applications" or "Order Management", because those domains change faster than a media type should, and the semantics themselves are subject to disagreement across trust boundaries.

And Mike Kelly has codified it in HAL that completely removes the notion of data types.

None the less there's plenty of examples and discussions of domain specific media types. One of these is Mike's Maze+XML. My question is now: why did you choose a domain specific media type for this purpose - as opposed to reusing an existing domain agnostic media type that supports linking, especially when considering a statement like "A REST API should never have 'typed' resources"?

I am asking since Maze+XML serves as an educational example. But should developers go for domain specific media types or domain agnostic media types? What should their choice be guided by? What are the benefits and downsides of the two categories?

And, yes, I know I have been asking this before but, alas, I still don't have a clear understanding. Sorry. Right now I am mainly on the domain agnostic side, but then I see examples like Maze+XML and get in doubt.

/Jørn

mca

unread,
Apr 10, 2012, 12:20:16 PM4/10/12
to hyperme...@googlegroups.com
<snip>

> I am asking since Maze+XML serves as an educational example. But should
> developers go for domain specific media types or domain agnostic media
> types? What should their choice be guided by? What are the benefits and
> downsides of the two categories?
</snip>

It's a great question; one i am asked often.

First, I selected a domain-specific type to prove that such a thing
can be done; to assert the design itself. IOW, this was an experiment.

Second, "domain-specific" designs can be very handy; esp. for one-off
apps or apps where the domain itself is unlikely to change much over
time and/or where the data model in the domain is very limited in
scope. The Maze domain fits this bill.

Third, I created this design as a very limited one in order to explore
the details of extending media types. IOW, this particular iteration
is missing several key features I want in a Maze (think concepts like
3-d mazes, the ability to find and collect items in the maze, the
ability to recognize and interact w/ others, etc.). So I started w/ a
limited design to help expose strengths/weakenesses of the model
itself.

Finally, as for whether others should design "domain-specific" types,
my answer is "yes." I think a great deal can be accomplished w/ a
clear, simple hypermedia type and there is nothing inherently "bad" in
creating a limited type.

BTW - I think it's important not to conflate Fielding's warning about
"typed" resources (i.e. a "customer resource") w/ domain-specific
media types. A media type designed to support a single problem domain
(i.e. Search [see OpenSearch]) does not need to fall into the pit of
object serialization.

I hope this helps.

mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

> --
> You received this message because you are subscribed to the Google Groups
> "Hypermedia Web" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/hypermedia-web/-/9hYtBpUZ-n8J.
> To post to this group, send email to hyperme...@googlegroups.com.
> To unsubscribe from this group, send email to
> hypermedia-we...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/hypermedia-web?hl=en.

Mike Kelly

unread,
Apr 10, 2012, 12:43:59 PM4/10/12
to hyperme...@googlegroups.com

Hey Mike, what are the specific reasons you think others should design
domain-specific media types vs reusing affordances from a
domain-agnostic type?

mca

unread,
Apr 10, 2012, 12:59:09 PM4/10/12
to hyperme...@googlegroups.com
one of the key "attractors" to designing a domain-specific type is
that it can reduce the level of "abstraction" devs must deal w/

<customer href="..." />
vs.
<atom:link href="..." rel="customer" />

Also, I think there are aesthetic appeals that are worth considering.
IOW, it looks/feels "good" using a type that is geared directly at the
problem.

None of these are _quantitative_ reasons, however; just qualitative ones.

> --
> You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.

Mike Kelly

unread,
Apr 10, 2012, 3:32:42 PM4/10/12
to hyperme...@googlegroups.com
Does this constitute a qualitative benefit of a generic media type:
"The number of open libraries available for a ubiquitous generic type
are greater than one that is domain-specific." ?

mca

unread,
Apr 10, 2012, 4:21:40 PM4/10/12
to hyperme...@googlegroups.com
<snip>

> Does this constitute a qualitative benefit of a generic media type:
> "The number of open libraries available for a ubiquitous generic type
> are greater than one that is domain-specific." ?
</snip>
it's qualitative until there's a clear count for a specific media
type; then it's quantitative, eh?

Mike Kelly

unread,
Apr 10, 2012, 5:01:06 PM4/10/12
to hyperme...@googlegroups.com
On Tue, Apr 10, 2012 at 9:21 PM, mca <m...@amundsen.com> wrote:
> <snip>
>> Does this constitute a qualitative benefit of a generic media type:
>> "The number of open libraries available for a ubiquitous generic type
>> are greater than one that is domain-specific." ?
> </snip>
> it's qualitative until there's a clear count for a specific media
> type; then it's quantitative, eh?
>

Ok, take HTML as the example, I'm guessing its count is relatively
high compared to a domain-specific media type like Maze+XML? Is it
also a fair qualitative statement to say that tooling for generic
media types have a greater potential for re-use since they can be used
across many domains?

Cheers,
Mike

Jørn Wildt

unread,
Apr 10, 2012, 5:18:21 PM4/10/12
to hyperme...@googlegroups.com
All this fits quite well with my observations so far: there are many good
quantitative/objective reasons for going with a domain agnostic media type,
but yet most people have that gut-feeling of "wait, I cannot represent
something without specifying WHAT it is" - it simply feels right to use a
domain specific media type.

That is my very reason for keep pushing on media types - why is it that so
many brilliant people ignore good advice and insist on doing what is said to
be less than optimal? What is the context that defines what "optimal" means?

I am trying to collect a list of benefits and drawbacks of either approach
to write down a set of guidelines for media type design/choice.

So far I have (+/- => benefit/drawback):

Domain Specific Media Types
+ Easy to understand by humans.
+ Can easily be discovered (like Open Search and Atom which has special
handling in a browser).
- The domain may change faster than a media type should (thus eventually
breaking clients).
- Has to re-invent hyper media controls (thus forcing clients to not only
learn the domain specific stuff but also the standard REST stuff).
- Couples the questions of "HOW to work with the data" and "WHAT data is it"
into one media type(obviously making it impossible to reuse it in other
domains)
- Leads to a pletora of different media types (instead of focusing on a
small set of standards that are easy to learn by clients).

Domain Agnostic Media Types
- Difficult to read for humans.
- Has less visibility than domain specific media types (cannot be processed
specially in a browser).
+ Change seldomly, if ever (and changes are less likely to break clients).
+ Can reuse existing libraries to read/write data and locate hyper media
controls (less coding).
+ Decouples HOW/WHAT (improving reuse).
+ Keeps the set of media types a client needs to learn small-ish (thus
enabling the client to focus on the domain specific part of the API).

Many of these are variations of each other, I know.

By "breaking clients" I mean "clients do not crash while parsing data into
some internal representation - after that it can look for domain specific
profile and versioning information as needed".

> BTW - I think it's important not to conflate Fielding's warning about
> "typed" resources (i.e. a "customer resource") w/ domain-specific
> media types. A media type designed to support a single problem domain
> (i.e. Search [see OpenSearch]) does not need to fall into the pit of
> object serialization.

I think this is the crux of understanding media types - why is it that an
Atom item/feed (with its title + description), an Open search description,
and a Maze (with its directions) is better than a serialized "customer
resource"? I think some of the answer is:

- A media type should represent a "problem domain" as Mike puts it, not a
specific entity. Once the problem domain has been identified it is possible
to dispatch handling to a suitable processor.

Clarification: A Person/Customer is not a problem domain but "Contact
management" is - and if the "Contact management" media type further more
avoids talking about "Persons" altogether, but uses the more generic
"Contact" plus an extensible set of properties then it improves chances of
reuse and yet it is still a very visible/discoverable media type, making it
possible to open up the user's contact book (vcard).

- Atom + Open Search covers many different domains with very few structural
elements, thus making it easy to reuse in many different scenarios.

- Atom is extensible: a generic Atom reader can read the generic values
while it is still possible to to have more specialized readers.

- I am not sure if Open Search is extensible?

- The Customer serialization is hard to standardize across different
organizations and thus not suitable for a media type. It also very likely
changes often.

- Maze is, well, hmmm, an educational example :-)

Another point worth noticing is that many of the web APIs on the internet do
not require special "processing models" - they are all about representing
business data flat and simple, thus not requiring any special media type.
All that is needed is a hyper-media enabled media type. This is unlike Atom
which benefits from browsers knowing the specific media type (and starting
up feed readers).

/J�rn

mca

unread,
Apr 11, 2012, 5:11:48 AM4/11/12
to hyperme...@googlegroups.com
to start, one quote that I think applies here:
"Every act of communication is an act of translation." -- Gregory Rabassa[1]

For me, media types are the medium for "shared understanding" between
parties; the language. Each party has to "translate" their information
into this shared language for each request/response.

The "design" of that language is interesting to me. I do not approach
it just as an academic or technical activity, but also as a _design_
activity similar to industrial design or even (dare i say it?) "art."
It is interesting to me to explore the design space, not just push
bits about.

Also, in considering benefits and drawbacks, I think there is a wider
spectrum to consider than Specific & Agnostic. I see it as a
continuum. I cover my own POV on this continuum, it's benefits and
drawbacks in the first chapter of the "Building Hypermedia APIs"
book[2].


[1] http://en.wikipedia.org/wiki/Gregory_Rabassa
[2] http://shop.oreilly.com/product/0636920020530.do

> /Jørn

Jørn Wildt

unread,
Apr 11, 2012, 5:50:46 AM4/11/12
to hyperme...@googlegroups.com
> The "design" of that language is interesting to me. I do not approach
> it just as an academic or technical activity, but also as a _design_
> activity similar to industrial design or even (dare i say it?) "art."
>  It is interesting to me to explore the design space, not just push
> bits about.

Thanks for the clarification. That is another goal than mine - which is figuring out the best possible choice for a developer that creates a new REST service.

/Jørn

> To post to this group, send email to hypermedia-web@googlegroups.com.


> To unsubscribe from this group, send email to

> hypermedia-web+unsubscribe@googlegroups.com.

mca

unread,
Apr 11, 2012, 6:01:06 AM4/11/12
to hyperme...@googlegroups.com
FWIW, on the industrial design side, I am influenced by the thinking
and methodologies of:
Massimo Vignelli[1]
Charles & Ray Eames [2]

[1] http://en.wikipedia.org/wiki/Massimo_Vignelli
[2] http://en.wikipedia.org/wiki/Charles_and_Ray_Eames

>> > To post to this group, send email to hyperme...@googlegroups.com.


>> > To unsubscribe from this group, send email to

>> > hypermedia-we...@googlegroups.com.


>> > For more options, visit this group at
>> > http://groups.google.com/group/hypermedia-web?hl=en.
>> >
>

> --
> You received this message because you are subscribed to the Google Groups
> "Hypermedia Web" group.

> To view this discussion on the web visit

> https://groups.google.com/d/msg/hypermedia-web/-/K6Qdcga2owUJ.
>
> To post to this group, send email to hyperme...@googlegroups.com.


> To unsubscribe from this group, send email to

> hypermedia-we...@googlegroups.com.

Jørn Wildt

unread,
Apr 13, 2012, 1:51:39 AM4/13/12
to hyperme...@googlegroups.com
I am starting to think that my difficulties with media type selection for my
purpose is the lack of any concrete use cases. We have a lot of data in our
system and all we want is to expose it to our partners for systems
integration. That means we don't have any specific use case in mind other
than "here is the data - do what you want, mash it up, read it into your
systems or push you data into our system.".

Had there been any specific requirement like 1) sales orders must be
printable, or 2) new orders must be available as a feed, or 3) show graphs
of the reporting data, or 4) data is used by browsers - then I would have no
trouble selecting media types like PDF, Atom, PNG/Excel and JSON. But as it
happens, we only have a generic "make data available" use case - and as such
any choice of media type could be as good as any other.

This means we end up thinking mostly in terms of (hyper linked) resources of
the entities of the system with a few query resources and insert/update
semantical link relations thrown in (with post-once-exactly
implementations).

Perhaps what is needed is a generic "systems integration" media type?
Something that supports the use cases of, well, systems integration. If I
look at our few generic requirements I would get a list like this:

- Media type should be domain agnostic.
- The format itself must be extendable such that it can evolve, but it
should also provide "best practice" for extendable domain specific data.
- Formats for numbers, dates and such like must be well defined for
interoperability.
- Resources are linked using traditional link relations. The concrete system
must specific domain specific link relation types.
- URL templates should be supported for quick look up of data based on
external keys (could also be done with traditional HTML GET forms).
- Some kind of service document of the whole service should be available at
the root URL.
- Some kind of simplified forms must exist for modifying data.
- Data should be easily serialized into most general programming languages.
- There should be a well defined convention for post-once-exactly semantics
such that inserting new stuff can be repeated safely in the face of network
failures. Maybe as a property on the forms descriptors?
- It would be nice with a standard for inline documentation and description.

Would such a thing make sense? If it became a success people could start
writing "data browsers" (instead of HTML browsers) for surfing the APIs (or
creating XML stylesheets for transforming into HTML). Such a generic data
browser would be just as useful as the database administration tools people
are used to work with.

Perhaps this is also the goal of HAL? Are there other existing approaches to
this "problem space"?

/J�rn

Mike Kelly

unread,
Apr 13, 2012, 2:41:11 AM4/13/12
to hyperme...@googlegroups.com

On 13 Apr 2012, at 06:51 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

>
> Perhaps this is also the goal of HAL?

Yes that's one way of looking at it I suppose.

Some of those points you've listed were not built into HAL because they aren't really necessary and add complexity. I'm working on another type that extends HAL with fancy stuff like forms

Cheers,
Mike

Jørn Wildt

unread,
Apr 13, 2012, 2:52:48 AM4/13/12
to hyperme...@googlegroups.com
> Some of those points you've listed were  not built into HAL because they aren't really necessary and add complexity

Would you mind telling which (and why)?

 Looking forward to see the forms stuff.

/Jørn

mca

unread,
Apr 13, 2012, 3:49:24 AM4/13/12
to hyperme...@googlegroups.com
couple things:

1) T-SQL is a "media type" (although not at all designed for
distributed network apps)
2) "general data management" is, IMO, a domain-specific use case
3) "here is the data - do what you want, mash it up, read it into your
systems or push you data into our system." sounds like a specific user
case (if vaguely described).

Finally, my point is: for most cases, given a sufficiently described
problem domain, is it relatively trivial to define a media-type that
enables successful domain interactions over a distributed network.

FWIW, I suspect your challenge is not the "design part" but the
sufficient description of the problem domain. after that is done, the
rest of the work is usually quite mundane<g>.

Darrel Miller

unread,
Apr 13, 2012, 10:55:08 AM4/13/12
to hyperme...@googlegroups.com
My belief is that the best kinds of media types are ones that are
horizontally applicable. Most people when they think about domain
specific media types they are often thinking of vertically partitioned
domain semantics. i.e. semantics specific to a particular application
or business domain.

I'd like to see media types for:
error messages
validation rules
data entry forms
questions and Answers
poll surveys
resource status
lists of things
permissions
UI widget layout
data visualization
event subscription
calendar of events
event timeline
resource usage / distribution / consumption
Input events

I can imagine building a client that understands these different
concepts and knows how to render them and interact with them and then
building applications in many different domains by composing resources
that produce the appropriate representations.

Darrel

Mike Kelly

unread,
Apr 13, 2012, 7:15:13 AM4/13/12
to hyperme...@googlegroups.com

On 13 Apr 2012, at 08:49 AM, mca <m...@amundsen.com> wrote:

>
> Finally, my point is: for most cases, given a sufficiently described
> problem domain, is it relatively trivial to define a media-type that
> enables successful domain interactions over a distributed network.

I think Jorn was interested in more than just enabling successful interactions.
Specific media types, by their very nature, fly against the idea of "browsable" apps he was talking about and HAL actually enables:

Mike Kelly

unread,
Apr 13, 2012, 7:08:52 AM4/13/12
to hyperme...@googlegroups.com

On 13 Apr 2012, at 06:51 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

> - Formats for numbers, dates and such like must be well defined for interoperability.

Is this necessary for a generic type?

> - Some kind of service document of the whole service should be available at the root URL.

This is unnecessary, could be done with a link relation if it needed to be standardised

> - Some kind of simplified forms must exist for modifying data.

This will add complexity to the type making it more difficult to consume. These generic media types (such as HAL) battle a chicken and egg problem in the beginning, the more difficult it is to consume the less likely it is to get adopted and overcome the chicken and egg problem. It's hard enough convincing people to expose their app with basic links - if you raise the bar even higher with forms I don't think you will see a lot of traction.

This is why the form capable version of Hal will be a separate type - as a second phase once we have more people happy using 'basic' link rel hypermedia with standard HAL.

> - There should be a well defined convention for post-once-exactly semantics such that inserting new stuff can be repeated safely in the face of network failures. Maybe as a property on the forms descriptors?

Why this interaction specifically over all others? Is there a reason this shouldn't be established on a per-application basis?

> - It would be nice with a standard for inline documentation and description.

Again this can be established as a standard link relation - whether that's 'profile' or 'documentation' or whatever. If you literally wanted the documentation inline in the doc you could embed it using HAL.

>
> Would such a thing make sense? If it became a success people could start writing "data browsers" (instead of HTML browsers) for surfing the APIs (or creating XML stylesheets for transforming into HTML). Such a generic data browser would be just as useful as the database administration tools

Someone has already done this for HAL - they use it for on-boarding new developers by getting them to browse around their app. Unfortunately that isn't publicly available, but I'm going to be writing one myself that is open source

Cheers,
Mike

mca

unread,
Apr 13, 2012, 11:13:25 AM4/13/12
to hyperme...@googlegroups.com
<snip>

> Specific media types, by their very nature, fly against the idea of "browsable" apps he was talking about and HAL actually enables:
</snip>

CRM is a domain

an XML-based (hyper)media type that includes an XSLT link in
representations can be "browsable" in the world's most-common web
client.

Mike Kelly

unread,
Apr 13, 2012, 11:22:19 AM4/13/12
to hyperme...@googlegroups.com
kind of.. It's capable of being tans transformed into something browseable , it's not actually browsable in and of itself. How does this work for JSON?

mca

unread,
Apr 13, 2012, 11:36:00 AM4/13/12
to hyperme...@googlegroups.com
<snip>> kind of.. It's capable of being tans transformed into
something browseable ,</snip>
meh.

Jørn Wildt

unread,
Apr 13, 2012, 2:55:09 PM4/13/12
to hyperme...@googlegroups.com
Thanks for the feedback Mike. First it got me thinking; well, okay, we are
aiming for different targets - maybe my problem domain ain't so generic as I
first thought.

For systems integration I think it is necessary for both parties to agree on
how dates and numbers and such like are formated ... and btw so does HAL
also since it uses JSON which AFAIK is quite specific on these formats.

I said "some kind of service document", and, sure, a root HAL document with
a bunch of links would work perfectly for that. I certainly also agree on
the use of links for documentation.

Regarding forms vs. simplicity; interesting chicken and egg analysis - I get
your point.

Then we get to the POE semantics:

> Why this interaction specifically over all others?
> Is there a reason this shouldn't be established on a per-application
> basis?

First the reasons, remember my focus for this media type is systems
intergration:

1) Systems integration needs to be "safe" (in lack of better word) - it
won't be good if one entry in one system suddenly gets multiplied in another
system as a consequence of network failures and retries.

2) I like interoperability. POE can be done in many ways - lets focus on
exactly one way of doing it.

3) I would say the same about other often used interactions (if I could find
an example).

Then the consequences: while thinking this I realised that this is pointing
more and more in the direction of an "enterprise" solution that tries to
solve lots of the stuff that has already been solved in WS-* but from a
RESTful point of view - using hyper media, uniform interface, etc. and
(relatively) simple safety precausions like POE instead of transactions,
built on top of that.

I not sure if this is good or bad. People moved away from SOAP/WS-* to make
web API's simpler to consume - simply ignoring problems like stability; I
mean, who cares if a human operated Twitter client ends up posting two
tweets instead of one? The success of all the open web APIs (I will not call
them RESTful APIs!) is probably due to this percieved simplicity of "REST" -
which is only simple because it ignores some of the problematic issues with
a distributed solution.

One nice thing about using a media type to document/enforce all of these
properties of the API is that it is negotiable; if you don't like one style
of integration, well, then use another (if supported).

Enterprise systems integration is a different kind of beast than an open
web/RESTful API - and maybe REST is not the answer for that? Okay, I'll
answer that myself since I believe in REST - lots of the systems
integrations I have seen are between two absolutely independent system that
really has nothing to do with each other - and enforcing advanced
integration solutions like for instance MSMQ over NServiceBus (which I
absolutely love) is seldom a possibility - "simple" APIs over HTTP is much
more generally available - but since adoption is not a big issue we can
allow ourself to raise the entry barrier a bit, requiring developers to
understand things like POE.

So where did this get me? Well, I do not have enough experience with WS-*
and what ever trouble it brings - and I do not have enough experience with
RESTful APIs to really understand/appreciate/internalize the benefits and
make sure a new media type actually brings these benefits to the kind of
system I work with. Which leaves me in some sort of decision vacum ... Need.
More. Data :-)

Thanks for listening and feedback.

/J�rn

Jan Algermissen

unread,
Apr 13, 2012, 4:06:08 PM4/13/12
to hyperme...@googlegroups.com
[Resending this - a little late, but maybe it's of use to anyone]

On Apr 10, 2012, at 6:20 PM, mca wrote:

> Finally, as for whether others should design "domain-specific" types,
> my answer is "yes." I think a great deal can be accomplished w/ a
> clear, simple hypermedia type and there is nothing inherently "bad" in
> creating a limited type.

++++1

Besides ... you need specific semantics anyhow to get stuff done. The media type (or link relation) is the place to put them.


>
> BTW - I think it's important not to conflate Fielding's warning about
> "typed" resources (i.e. a "customer resource") w/ domain-specific
> media types. A media type designed to support a single problem domain
> (i.e. Search [see OpenSearch]) does not need to fall into the pit of
> object serialization.

Yep! Exactly.

<img src="/foo"/> does not mean that /foo is a resource of type image (nor does it constrain the nature of /foo in any way). It is just another resource and the server tells the client something usefull (here "you might want to fetch /foo and render it in place")

Jan


Jørn Wildt

unread,
Apr 13, 2012, 4:44:55 PM4/13/12
to hyperme...@googlegroups.com
> > BTW - I think it's important not to conflate Fielding's warning about
> > "typed" resources (i.e. a "customer resource") w/ domain-specific
> > media types.

> Yep! Exactly.


> <img src="/foo"/> does not mean that /foo is a resource of type image (nor
> does it constrain the nature of /foo in any way).

Okay. Lets say that was an image of our customer "/foo". I could also get a
PDF with the customer's information from the same URL. Or I could get a JSON
object describing the customer details - also from the same URL. All three
representations (JPG/PDF/JSON) is of a customer and to work with the JSON
object I must know how a customer is serialized ... wouldn't that JSON
representation be of type "customer"?

Same goes for the XML representation (or RDF) - to work with the data my
client must understand that it represents a customer, thus the type is,
well, "customer"? Otherwise it wouldn't know how to pick the customer
name/birthday etc. out of the data?

But, hmmm, that leaves the question of "Exactly what XML representation are
you refering to J�rn?" - it's late, so I start talking to myself ...

What if "the" XML representation was a V(X)Card
(http://tools.ietf.org/html/rfc6351); then my client would know (because it
followed a well defined link) that 1) this is a customer representation, but
2) the name can be found according to the XCard specs - completely unrelated
to the type "customer". That kind of explains it to me :-)

The problem is then that many domains don't have useful existing media
types. How should for instance a case file (my business domain), a
biological molecule, or a set of radio components be represented? This
either calls for a generic media type like HAL or HTML - or a domain
specific media type ... and when defining a new domain specific media type
it is very difficult not to define it in terms of your own internal business
data, which is when you run into problems and won't be able to distinguish
type from representation. Right?

/J�rn

++++1

Yep! Exactly.

Jan


--
You received this message because you are subscribed to the Google Groups
"Hypermedia Web" group.

Jan Algermissen

unread,
Apr 13, 2012, 5:14:39 PM4/13/12
to hyperme...@googlegroups.com

On Apr 13, 2012, at 10:44 PM, Jørn Wildt wrote:

>> > BTW - I think it's important not to conflate Fielding's warning about
>> > "typed" resources (i.e. a "customer resource") w/ domain-specific
>> > media types.
>
>> Yep! Exactly.
>> <img src="/foo"/> does not mean that /foo is a resource of type image (nor does it constrain the nature of /foo in any way).
>
> Okay. Lets say that was an image of our customer "/foo". I could also get a PDF with the customer's information from the same URL. Or I could get a JSON object describing the customer details - also from the same URL.
> All three representations (JPG/PDF/JSON) is of a customer and to work with the JSON object I must know how a customer is serialized ... wouldn't that JSON representation be of type "customer"?

The representation would be of a media type, yes. Otherwise, you could not process it.

>
> Same goes for the XML representation (or RDF) - to work with the data my client must understand that it represents a customer,

NO! NO! You got it backwards.

There is some user with some use case and the user controls a user agent. The user agent is used (even if the user is not involved in any action steps of the use case) to create an application (composed of itself and some server components) that realizes the use case.

Hence, the client issues any particular request with an intention (to satisfy the next step in the use case). The target of the request is picked based upon hypermedia semantics and maybe there is an expectation there, that /foo 'is' a customer. But that expectation is entirely held by the client. The same resource might 'have' a totally different type to another client following another use case.

The client owns the use case and creates an application. The server has nothing to do with it - it just needs to comply with HTTP.

The *trick* is that the client does not need to know what the resource 'is'. It just needs to follow its own intent and (and this is what REST is all about) prepare for expectations to fail. REST merely emphasizes the fact that you have no control over the server in a networked system since the cost of having control is prohibitive. And....REST provides the best 'framework' so far for preparing to deal with such failing expectations.

RPC style architectures (e.g. SOAP) do the contrary: they pretend you can control the server and simply hide the cost. The trick you into building a coupled mess .... until you realize the cost of control (e.g. bringing the whole system down for upgrades, development-time dependencies of source code across otherwise independent systems, thousands of customers with legacy API versions you endlessly need to support, and what not).

> thus the type is, well, "customer"? Otherwise it wouldn't know how to pick the customer name/birthday etc. out of the data?

The type is a property of the payload, allowing software to extract information from it. That is something different. The next client might well just index the payload based on the title (and give a damn about whether the thing is a customer or a kitchen sink)

>
> But, hmmm, that leaves the question of "Exactly what XML representation are you refering to Jørn?" - it's late, so I start talking to myself ...


>
> What if "the" XML representation was a V(X)Card (http://tools.ietf.org/html/rfc6351); then my client would know (because it followed a well defined link) that 1) this is a customer representation,

No, it is simply a vcard payload. no more no less. The intent of the client is up to the client.

> but 2) the name can be found according to the XCard specs - completely unrelated to the type "customer". That kind of explains it to me :-)
>
> The problem is then that many domains don't have useful existing media types.

Ha! Right. And maybe we will never see that outside 'closed doors' because it is difficult to agree on what constitutes a customer's data (reference data models come to mind). UBL is an attempt to do that for procurement, but its flexibility is symptomatic for the problem :-)

Not a problem, though. If a big player was to go ahead, we would probably see useful media types all over the place. Unfortunately, all energy is still spend in designing RPC APIs. I am sure there are opportunities for common types in the area of social networks, billing, dating, shopping, etc. It is just that nobody steps forward.

> How should for instance a case file (my business domain), a biological molecule, or a set of radio components be represented? This either calls for a generic media type like HAL or HTML - or a domain specific media type ... and when defining a new domain specific media type it is very difficult not to define it in terms of your own internal business data,

Yes, right. Though .... one has to start with some model in mind. Maybe it is your model today and, slightly modified, our model tomorrow?

> which is when you run into problems and won't be able to distinguish type from representation. Right?

???

Jan

>
> /Jørn

Mike Kelly

unread,
Apr 14, 2012, 1:43:24 AM4/14/12
to hyperme...@googlegroups.com

On 13 Apr 2012, at 04:36 PM, mca <m...@amundsen.com> wrote:

> <snip>> kind of.. It's capable of being tans transformed into
> something browseable ,</snip>
> meh.

Thats not just pedantry though.. If you produced an app using a custom domain specific media type that must be browsable via XSLT into a web browser - you've now picked up two extra debts:

- maintaining the custom domain specific media type
- maintaining XSLT doc(s)

Why would you want to do that, just for the sake of

<customer href="..." />

over

<link rel="customer" href="..." />

Your point about communication breaks down here too since I can generate the same HTML representation of the link from either.

Cheers,
Mike

Jørn Wildt

unread,
Apr 14, 2012, 3:14:36 AM4/14/12
to hyperme...@googlegroups.com
That was a very useful description Jan. Thanks. Let me try an example to
verify that I understand what you are saying:

1) The homepage/resource of Copenhagen airport is http://www.cph.dk. Lets
call it {Cph}.

2) Another completely unrelated resource {ShopOrder} exposes a sales order
with Copenhagen airport as customer. This sales order contains a link like
this: <link href="{Cph}" rel="customer"/>.

3) Yet another resource {Flight} thinks of {Cph} as being the destination of
a flight using the link <link href="{Cph}" rel="destination"/>

A client comming from the shop (2) could try to get in contact with the
customer by sending a request for a XCard to {Cph}.

A client comming from the flight description (3) could try to get more
information about the destination, sending a request for something like
application/location+xml to {Cph}.

None of the clients has any idea of the actual type behind {Cph} - they
simply ask for various representations from {Cph} through the use of
different media types.

Is this what you are trying to tell me? That makes a lot of sense to me and
explains Fieldings "A REST API should never have "typed" resources that are

significant to the client".

But this point of view doesn't fit very well with generic media types, does
it? Sending a request for application/hal+xml for {Cph} won't solve the two
use cases above - the {Cph} resource would not be able to return either
XCard or Location. But, as you have stated earlier, a media type could
simply be an alias for an existing generic media type, thereby reducing the
amount of work needed to create a new media type.

Thanks, J�rn

Jan Algermissen

unread,
Apr 14, 2012, 3:30:59 AM4/14/12
to hyperme...@googlegroups.com

On Apr 14, 2012, at 9:14 AM, Jørn Wildt wrote:

> That was a very useful description Jan. Thanks. Let me try an example to verify that I understand what you are saying:
>
> 1) The homepage/resource of Copenhagen airport is http://www.cph.dk. Lets call it {Cph}.
>
> 2) Another completely unrelated resource {ShopOrder} exposes a sales order with Copenhagen airport as customer. This sales order contains a link like this: <link href="{Cph}" rel="customer"/>.
>
> 3) Yet another resource {Flight} thinks of {Cph} as being the destination of a flight using the link <link href="{Cph}" rel="destination"/>
>
> A client comming from the shop (2) could try to get in contact with the customer by sending a request for a XCard to {Cph}.

Yup - the issue here is: On what grounds does the client(-developer) decide what to put in the Accept header. Essentially the accept header means: If you send your response in foo/bar I am able to work through my use case from there. So, yes. If your client is coded to proceed through the use case if the GET returns a xcard then the client tells that to the server using Accept.

>
> A client comming from the flight description (3) could try to get more information about the destination, sending a request for something like application/location+xml to {Cph}.

Yes. Essentially saying: I can proceed if I get a response in application/location+xml.


>
> None of the clients has any idea of the actual type behind {Cph} - they simply ask for various representations from {Cph} through the use of different media types.

Yes, but the various media types are not the issue here. Could well be the same type. What matters is that the client expects the resource at the end of the customer link to 'be' something but it does not matter what the resource really is (it is just a resource anyhow).

>
> Is this what you are trying to tell me? That makes a lot of sense to me and explains Fieldings "A REST API should never have "typed" resources that are significant to the client".

Yes, I think so. Glad if I can help. Otherwise, keep asking :-)


>
> But this point of view doesn't fit very well with generic media types, does it?

Generic media types are an anti pattern (sorry folks :-). You need specific semantics to get stuff done. Otherwise you just shove the issue up one layer, thereby creating out of band coupling.

> Sending a request for application/hal+xml for {Cph} won't solve the two use cases above - the {Cph} resource would not be able to return either XCard or Location. But, as you have stated earlier, a media type could simply be an alias for an existing generic media type, thereby reducing the amount of work needed to create a new media type.

Yup - you can easily express a subset of XHTML with, say RelaxNG, specify the semantics, name that thing application/customer and be done with your generic type. Parsers come for free as well as sending text/html to HTML-user agents based on conneg.

We need agreement (contract) to enable communication to happen. The artifact where this contract lives in REST is the media type (and only the media type) [well, plus link rels of course] and this is where all design work must happen when creating RESTful systems. There is simply no other artifact in REST where you can put any contract information.


>
> Thanks, Jørn

Jan Algermissen

unread,
Apr 14, 2012, 3:32:27 AM4/14/12
to hyperme...@googlegroups.com

On Apr 14, 2012, at 9:30 AM, Jan Algermissen wrote:

> and be done with your generic type

s/generic/specific/

Doh!

Jan

Jørn Wildt

unread,
Apr 14, 2012, 3:41:13 AM4/14/12
to hyperme...@googlegroups.com
> > But this point of view doesn't fit very well with generic media types, does it?
> Generic media types are an anti pattern (sorry folks :-).
 
 
... This situation reminds me of my earliest days as a programmer (back when IBM was inventing punch cards :)  ) - I would often be facing "six of one/half-dozen of the other" type decisions.  So I would ask one senior person who would convince me to go with the "6 of one" and then two days later I would talk to another senior person who would convince me that clearly "half dozen of the other" was the way to go.  If this happens to anyone out there enough so that you feel like a ping-pong ball - you can follow what I did - I went into management :)
At least now I believe I have enough understanding to make a rational choice.
 
/Jørn
 
----- Original Message -----
From: "Jan Algermissen" <algermi...@mac.com>
Sent: Saturday, April 14, 2012 9:30 AM
Subject: Re: Domain specific media types?


Jørn Wildt

unread,
Apr 14, 2012, 3:52:49 AM4/14/12
to hyperme...@googlegroups.com
A consequence of this must be that we get quite a lot of domain specific
media types that explains how to interact with the resources of a service.
Right?

That also fits very well with Fielding's other quote "A REST API should
spend almost all of its descriptive effort in defining the media type(s)
used for representing resources and driving application state"

I also love this one
https://twitter.com/#!/mamund/status/181927406561533952: "@dret: new #REST
rule: no, you cannot play with your URIs until you have designed all of your
media and link relation types." +1.

/J�rn

----- Original Message -----
From: "Jan Algermissen" <algermi...@mac.com>
To: <hyperme...@googlegroups.com>

Sent: Saturday, April 14, 2012 9:30 AM

Subject: Re: Domain specific media types?


>
> Thanks, J�rn

Jan Algermissen

unread,
Apr 14, 2012, 4:09:55 AM4/14/12
to hyperme...@googlegroups.com

On Apr 14, 2012, at 9:52 AM, Jørn Wildt wrote:

> A consequence of this must be that we get quite a lot of domain specific media types that explains how to interact with the resources of a service. Right?

Yes. Though we obviously should work (hard!) to prevent media type bloat. That is, for example, why we have central registries for media types and link relations and a couple of guys that keep an eye on them.

Even if you are behind the firewall and need more specific types than on the public web and have not desire to go about registering them with IANA you can put up a company-global registry to limit bloat.

>
> That also fits very well with Fielding's other quote "A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state"

Yup. That is the quote.

>
> I also love this one https://twitter.com/#!/mamund/status/181927406561533952: "@dret: new #REST rule: no, you cannot play with your URIs until you have designed all of your media and link relation types." +1.
>

:-)

Jan

> /Jørn


>
> ----- Original Message ----- From: "Jan Algermissen" <algermi...@mac.com>
> To: <hyperme...@googlegroups.com>
> Sent: Saturday, April 14, 2012 9:30 AM
> Subject: Re: Domain specific media types?
>
>
>
> On Apr 14, 2012, at 9:14 AM, Jørn Wildt wrote:
>
>> That was a very useful description Jan. Thanks. Let me try an example to verify that I understand what you are saying:
>>
>> 1) The homepage/resource of Copenhagen airport is http://www.cph.dk. Lets call it {Cph}.
>>
>> 2) Another completely unrelated resource {ShopOrder} exposes a sales order with Copenhagen airport as customer. This sales order contains a link like this: <link href="{Cph}" rel="customer"/>.
>>
>> 3) Yet another resource {Flight} thinks of {Cph} as being the destination of a flight using the link <link href="{Cph}" rel="destination"/>
>>
>> A client comming from the shop (2) could try to get in contact with the customer by sending a request for a XCard to {Cph}.
>
> Yup - the issue here is: On what grounds does the client(-developer) decide what to put in the Accept header. Essentially the accept header means: If you send your response in foo/bar I am able to work through my use case from there. So, yes. If your client is coded to proceed through the use case if the GET returns a xcard then the client tells that to the server using Accept.
>
>>
>> A client comming from the flight description (3) could try to get more information about the destination, sending a request for something like application/location+xml to {Cph}.
>
> Yes. Essentially saying: I can proceed if I get a response in application/location+xml.
>
>
>>
>> None of the clients has any idea of the actual type behind {Cph} - they simply ask for various representations from {Cph} through the use of different media types.
>
> Yes, but the various media types are not the issue here. Could well be the same type. What matters is that the client expects the resource at the end of the customer link to 'be' something but it does not matter what the resource really is (it is just a resource anyhow).
>
>>
>> Is this what you are trying to tell me? That makes a lot of sense to me and explains Fieldings "A REST API should never have "typed" resources that are significant to the client".
>
> Yes, I think so. Glad if I can help. Otherwise, keep asking :-)
>
>
>>
>> But this point of view doesn't fit very well with generic media types, does it?
>
> Generic media types are an anti pattern (sorry folks :-). You need specific semantics to get stuff done. Otherwise you just shove the issue up one layer, thereby creating out of band coupling.
>
>> Sending a request for application/hal+xml for {Cph} won't solve the two use cases above - the {Cph} resource would not be able to return either XCard or Location. But, as you have stated earlier, a media type could simply be an alias for an existing generic media type, thereby reducing the amount of work needed to create a new media type.
>
> Yup - you can easily express a subset of XHTML with, say RelaxNG, specify the semantics, name that thing application/customer and be done with your generic type. Parsers come for free as well as sending text/html to HTML-user agents based on conneg.
>
> We need agreement (contract) to enable communication to happen. The artifact where this contract lives in REST is the media type (and only the media type) [well, plus link rels of course] and this is where all design work must happen when creating RESTful systems. There is simply no other artifact in REST where you can put any contract information.
>
>
>>

>> Thanks, Jørn

Mike Kelly

unread,
Apr 14, 2012, 4:11:59 AM4/14/12
to hyperme...@googlegroups.com


Sent from my iPhone

On 14 Apr 2012, at 08:30 AM, Jan Algermissen <algermi...@mac.com> wrote:





But this point of view doesn't fit very well with generic media types, does it?

Generic media types are an anti pattern (sorry folks :-). You need specific semantics to get stuff done. Otherwise you just shove the issue up one layer, thereby creating out of band coupling.

With all due respect, that is utter bollocks. Media type specifications are actually *more* out of band than URIs used for link relations. You can't dereferrence a media type identifier.. at least a link rel URI is discoverable. Link relations also happen to be much more easily reused and changed in a granular fashion.

Your interpretation of what-a-media-type-must-be is completely subjective, generic media types are not an anti-pattern.

Mike Kelly

unread,
Apr 14, 2012, 4:24:20 AM4/14/12
to hyperme...@googlegroups.com

On 14 Apr 2012, at 08:14 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

> That was a very useful description Jan. Thanks. Let me try an example to verify that I understand what you are saying:
>
> 1) The homepage/resource of Copenhagen airport is http://www.cph.dk. Lets call it {Cph}.
>
> 2) Another completely unrelated resource {ShopOrder} exposes a sales order with Copenhagen airport as customer. This sales order contains a link like this: <link href="{Cph}" rel="customer"/>.
>
> 3) Yet another resource {Flight} thinks of {Cph} as being the destination of a flight using the link <link href="{Cph}" rel="destination"/>
>
> A client comming from the shop (2) could try to get in contact with the customer by sending a request for a XCard to {Cph}.
>
> A client comming from the flight description (3) could try to get more information about the destination, sending a request for something like application/location+xml to {Cph}.
>
> None of the clients has any idea of the actual type behind {Cph} - they simply ask for various representations from {Cph} through the use of different media types.
>
> Is this what you are trying to tell me? That makes a lot of sense to me and explains Fieldings "A REST API should never have "typed" resources that are significant to the client".

The equivalent of this with a generic type is separate application paths with different preceding rels.

>
> But this point of view doesn't fit very well with generic media types, does it? Sending a request for application/hal+xml for {Cph} won't solve the two use cases above - the {Cph} resource would not be able to return either XCard or Location. But, as you have stated earlier, a media type could simply be an alias for an existing generic media type, thereby reducing the amount of work needed to create a new media type.

These should be separate resources. The most sensible way of using conneg is for negotiating variants that are different only 'mechanically' (iirc fielding used this term) - ie different renderings of the same data eg XML + JSON

Duncan Cragg

unread,
Apr 15, 2012, 3:21:50 PM4/15/12
to Hypermedia Web
Hi Jørn

> Perhaps what is needed is a generic "systems integration" media type?

A number of us are active in this area. I listed those I'm aware of in
a similar discussion on API Craft:

https://groups.google.com/group/api-craft/browse_thread/thread/40a75b1cad705d8e/#3f7abab27486c944

Anyway, here's the list:

- JSON-LD
- HAL
- Collection+JSON
- OData/JSON
- JSON-{Schema,Reference,Pointer,Patch,Query}
- "A Convention for HTTP Access to JSON Resources"
- The Object Network -- my own approach

Let me go through your requirements wearing 'Object Network glasses':

> - Media type should be domain agnostic.

Well, once I have 'rough consensus and running code', I'll create an
Internet Draft and hopefully an RFC for 'object-net+json' or
something.

But yes, the Media Type will be domain-independent, and already is, in
the form of JSON. You don't mint your own Media Types in the Object
Net!

> - The format itself must be extendable such that it can evolve, but it should also provide "best practice" for extendable domain specific data.

The Object Network has two layers that are like the way Microformats
or Microdata work on top of HTML.

There is a basic set of conventions for the way we use JSON; a 'JSON+
+' interpretation of things, such as lists and links, which is like
the HTML layer: it's the Media Type level.

Then there are formats or types on top, which are basically what you
get with Microformats, Microdata, schema.org, Semantic Web: contacts,
events, media metadata, lists, queries, articles, messages, reviews,
etc., etc.

You can mix'n'match and extend within or on top of those formats.

> - Formats for numbers, dates and such like must be well defined for interoperability.

The Object Net is pretty chilled out about this - it uses regular-
expression-level syntax to determine if something in a string can be
seen as matching some type - such as one of several date formats,
hyperlinks, email addresses, etc. Also strings can be seen as other
types: "3.33" is the same as 3.33, "true" the same as true, etc.

> - Resources are linked using traditional link relations. The concrete system must specify domain-specific link relation types.

The set of truly useful link relations in the (RFC5988) registry is
actually pretty small. Go look, and tell me what you think you might
actually use! Maybe 'license'... :-)

Anyway, in the Object Network, we don't have anything special called
'link relations' - you just use a shared, common, known, standard tag
or key before the link. Like this:

{ .. "license": "http://.. " .. }

Pretty simple - low-stress hypermedia! Simple means happy developers.

> - URL templates should be supported for quick look up of data based on external keys (could also be done with traditional HTML GET forms).

All URLs are considered to be opaque strings in the Object Net. There
are no URL or query string templates.

Queries have their own JSON format, which you POST to collection or
list resources.

> - Some kind of service document of the whole service should be available at the root URL.

There's a 'site' format, which can be used for entry points. It can be
mixed in with a collection or list type.

> - Some kind of simplified forms must exist for modifying data.

There are no forms, as that's not needed, and thus would add
unnecessary complexity.

When you recognise a returned type or JSON format (e.g. calendar
event), you know you can POST something back to it in some other known
type (e.g. RSVP).

> - Data should be easily serialized into most general programming languages.

The Object Network 'hyperdata' only uses JSON. Plus images, Web links,
etc, of course.

> - There should be a well defined convention for post-once-exactly semantics such that inserting new stuff can be repeated safely in the face of network failures...

The Object Net sees POST as a kind of 'reverse GET' - a proactive push
of some JSON object state to another object.

As such, it already comes with idempotency on POST through having
unique ids on all objects, in either direction.

> - It would be nice with a standard for inline documentation and description.

Nah. :-)

> .. If it became a success people could start writing "data browsers" (instead of HTML browsers) for surfing the APIs.

Exactly!

My online Javascript data browser is called 'Object-Mash'.

My Android data browser is called 'NetMash'.

I can give more info about them if you're interested. Or go look them
up on GitHub.

- - - - - - - - - - - - - - -

I have a couple of articles up about the Object Network on InfoQ and
on ProgrammableWeb:

http://www.infoq.com/news/2012/02/the-object-network

http://blog.programmableweb.com/2012/02/13/the-object-network-making-mashups-easy/

Main site:

http://the-object.net

Its underlying architecture is FOREST - described in a paper here:

http://rd.springer.com/chapter/10.1007/978-1-4419-8303-9_7

Google Group for the Object Network:

https://groups.google.com/group/the-object-net

Blog intro:

http://duncan-cragg.org/blog/post/introduction-object-network/

Basics of the HTTP and JSON patterns:

http://duncan-cragg.org/blog/post/basics-object-network/

- - - - - - - - - - - - - - -

Oh - nearly forgot, the Object Network comes with a declarative JSON
transformation language or rule notation. It's currently called
'Fjord' (but I'm looking for an alternative name). In Fjord, you match
patterns in JSON objects, then rewrite them. I have a version in
Node.js, but it's out of date, and I'm currently re-implementing it in
NetMash. It takes the place of a JSON path traversal and/or query
language in the Object Network, or the query language for a JSON
database.

Cheers!

Duncan

Erik Wilde

unread,
Apr 15, 2012, 4:03:01 PM4/15/12
to hyperme...@googlegroups.com, REST Discuss
hello.

On 2012-04-14 10:11 , Mike Kelly wrote:
> On 14 Apr 2012, at 08:30 AM, Jan Algermissen <algermi...@mac.com

> <mailto:algermi...@mac.com>> wrote:
>> Generic media types are an anti pattern (sorry folks :-). You need
>> specific semantics to get stuff done. Otherwise you just shove the
>> issue up one layer, thereby creating out of band coupling.
> With all due respect, that is utter bollocks. Media type specifications
> are actually *more* out of band than URIs used for link relations. You
> can't dereferrence a media type identifier.. at least a link rel URI is
> discoverable. Link relations also happen to be much more easily reused
> and changed in a granular fashion.

it seems to me that at that level, discussing "generic" versus "domain"
is very binary, and very subjective. for example, to me, atom is both
generic (it says nothing about the payload) and specific (it operates in
the domain of collections exposing a fairly specific set of metadata).
the usefulness of atom is based on the fact that it is generic enough to
be reusable, and specific enough to build generic clients that still can
do useful things. my personal viewpoint is that generic is good if it is
still a domain with useful behavior. if it is more generic than that
(just "objects" that have "actions"), then it indeed is pretty useless.
even though you could still provide patterns at that level, only that
this level itself would not make sense to be made available as a RESTful
service.

cheers,

dret.

--
erik wilde | mailto:dr...@berkeley.edu - tel:+1-510-2061079 |
| UC Berkeley - School of Information (ISchool) |
| http://dret.net/netdret http://twitter.com/dret |

Erik Wilde

unread,
Apr 15, 2012, 4:09:51 PM4/15/12
to hyperme...@googlegroups.com, REST Discuss
hello duncan.

On 2012-04-15 21:21 , Duncan Cragg wrote:
> There is a basic set of conventions for the way we use JSON; a 'JSON+
> +' interpretation of things, such as lists and links, which is like
> the HTML layer: it's the Media Type level.
> Then there are formats or types on top, which are basically what you
> get with Microformats, Microdata, schema.org, Semantic Web: contacts,
> events, media metadata, lists, queries, articles, messages, reviews,
> etc., etc.

sounds a lot like "profiles" to me ;-) apart from that, i am wondering
how you deal with the formats/types. because it's exactly what we are
doing, and my thinking is to provide the foundation as a media type
(robust and stable over time, and with a well-defined extension model),
and provide the formats/types as profiles with well-defined descriptions
which can be interpreted at run time by clients wondering about the
format/type. generic clients can still happily live with the media type,
and since the media type also defines how format/type data has to be
represented, even generic clients can do some basic stuff (rendering in
a basic UI, for example, but without knowing what that field means),
while more sophisticated ones can learn about the profile at runtime. is
that a design that's similar to yours, or how are you dealing with user
formats/types? to me, it seems that expressing those as media types
would not have the stability that you would expect from a media type, so
they should use something more dynamic (like profiles) instead.

Duncan Cragg

unread,
Apr 15, 2012, 4:35:04 PM4/15/12
to hyperme...@googlegroups.com

> sounds a lot like "profiles" to me ;-) apart from that, i am wondering
> how you deal with the formats/types. because it's exactly what we are
> doing, and my thinking is to provide the foundation as a media type
> (robust and stable over time, and with a well-defined extension
> model), and provide the formats/types as profiles with well-defined
> descriptions which can be interpreted at run time by clients wondering
> about the format/type. generic clients can still happily live with the
> media type, and since the media type also defines how format/type data
> has to be represented, even generic clients can do some basic stuff
> (rendering in a basic UI, for example, but without knowing what that
> field means), while more sophisticated ones can learn about the
> profile at runtime. is that a design that's similar to yours, or how
> are you dealing with user formats/types? to me, it seems that
> expressing those as media types would not have the stability that you
> would expect from a media type, so they should use something more
> dynamic (like profiles) instead.
>

I'll have to have a look at this 'profiles' thing of which you speak. I
wasn't aware of it as being something related to all this, so haven't
read about it yet.

Sounds like similar things are going on here, though... Take this
fragment of an example:

200 OK
Content-Type: application/json OR application/object-net+json

{ "is": "event", .. }

The Media Type is JSON, or 'JSON+', which gives some basic understanding
(you know about JSON, or you know about the Object Network's patterns of
usage of JSON),

Then on top of that is a layer of these formats or types or
Microformat-like schemas. The above is a calendar event. So generic
client can hopefully understand that, and allow it to be rendered and
interacted with (RSVP perhaps).

On top of that, if you really need it - if the standard types aren't
enough, either singly or in combination or in sub-parts like 'location'
or 'title' - then you can extend in a backwards-compatible way, by
adding new fields, documenting them so we can hopefully all agree that
they're a good idea and can blend them in to a standard.

Extension can either be through such addition, or through using a base
format like 'contact' or 'event' or 'article' to carry your payload,
like in Atom.

Cheers!

duncan


Jørn Wildt

unread,
Apr 16, 2012, 2:07:28 AM4/16/12
to hyperme...@googlegroups.com
Okay, so we have two different ways of doing RESTful APIs/services: using domain agnostic media types (DAG) or domain specific media types (DSP). DAG's handle domain variety through link-rels whereas DSP's handle variety through media types. Both ways seems to solve the problem at hand - but what do we gain or loose by using either?

What I like about DAG's is the ability to easily browse them - it is "machine HTML" for API browsers. That's neat. I like tooling :-)

Versioning through link-rels in DAG's has the nice effect of being able to incrementally change the API by adding new link-rels - compared to DSP's where the whole media type definition must be bumped. A DSP can although choose to use <link rel="customer"/> instead of <customer href="..."/> and thus make the format extendable by adding new link-rels and document them outside the media type specification - thus gaining the same benefits that DAG's got.

But if DAG's uses link-rels for versioning then these "new versions of the same resource" must have new URLs, meaning that if "customer" links to /foo then "customer.1" must point to /foo.1. But if all I wanted was to indicate that /foo was available in a new resource representation (for instance using a new vocabulary for data) then I would find it strange that my customer suddenly got a new location!

Yet another consequence of DAG's is that we must work with resources as if they had a specific type - the client cannot negotiate what representation it wants from the server and thus a specific resource must forever and always have the same type (or "shape" or "vocabulary") - forcing the server to freeze the resource and never be able to evolve.

There is another problem with DAG's that I find disturbing; DAG's cannot handle "untyped" link-rels like "likes". Why? Because "likes" says absolutely nothing about the link target - it can be a dog, food, a color, the weather - anything, so there is no way for the client to decode the returned generic data - at least not when using HAL which frowns upon type specifications in the representation. Object Network seems to embrace typing of resources, so I guess N.O. does not have this problem. The problem with O.N. is though that it enforces a global understanding of a resource's type - a client cannot ask an O.N. resource for different views (representations) of the same resource?

But does a DSP handle "untyped" link-rels any better? I would say yes since a client can inform the server of the DSP format(s) that the client understands thus ensuring it understands what is returned.

The use of DAG's is also restricting the ways a certain resource can be reused in a more distributed system; a resource "X" can only present itself as app/hal+json or app/objectnet+json in one particular way. This may be perfect in a not-so-distributed system where all other resources linking to "X" has the same interpretation of "X". But then another system cannot point to "X" and get another app/hal|on+json representation back. The use of a DAG seems to force one globally shared understanding of "X" to all parties that interact with "X".

From these observations it seems like DAG breaks some of what we expect to gain from REST:

- A resource has a fixed type/shape that all clients must understand in order to interact with it. This adds *strong coupling*.

- A resource cannot be *reused* in a distributed network where different clients may have different needs.

- A resource cannot *evolve* over time since clients will break if the app/hal|mo+json format changes significantly.

It also interesting to see people claiming "simplicity" as an important aspect of the DAG media types. REST is "programming for the decades" and many of the constraints are directly opposed to short-term efficiency (quoting http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven). This is not exactly a proof of anything but it does make me stop and wonder if simplicity is an important goal of REST.

But then DSP's have problems too; what if a specific domain changes way faster than a media type is supposed to do? Well, a lot of things can be done with extendable formats and some sort of profile information embedded inside the representation (as in the <head> section of HTML). But doing so moves us a step closer to a DAG where it is impossible for the client to ask for a specific profile of a specific media type. But, in the end, if the domain really changes fast - what's wrong with adding new media types then? That should solve the problem - and avoid the above mentioned problem of having to add a new resource /foo.2 for version 2 of the format a service makes available.

In the end I would like to ask Jan for some example media types that illustrates how he is using them? It does not have to be proprietary "secret" media types, but perhaps you could change them ever so slightly so others can gain some insight from them?

Thanks for listening, Jørn

Erik Wilde

unread,
Apr 16, 2012, 3:35:12 AM4/16/12
to hyperme...@googlegroups.com, REST Discuss, Jørn Wildt
hello joern.

On 2012-04-16 08:07 , Jørn Wildt wrote:
> Okay, so we have two different ways of doing RESTful APIs/services:
> using domain agnostic media types (DAG) or domain specific media types
> (DSP). DAG's handle domain variety through link-rels whereas DSP's
> handle variety through media types. Both ways seems to solve the problem
> at hand - but what do we gain or loose by using either?

you're saying this as if there was agreement on what DAG and DSP mean. i
don't think that's the case. and i don't think that it is a binary
decision, or just two things. you often have multiple layers, just look
at XML, atom, and podcasts. what are DAGs, what are DSPs? or did you
intend to give a definition of these two concepts, saying that a DAG is
defined by using link rels for variety, and a DSP by using media types?

for example, i had a brief exchange with mike who was saying that HTML
clearly is domain-agnostic, whereas my view of HTML is that it is
specific for human-oriented hypertext documents with a certain set of
features (but of course it is less specific that some other media
types). my guess is you could argue about this endlessly, so before we
start referring to these two things, we should at least know what we are
referring to.

Jørn Wildt

unread,
Apr 16, 2012, 4:45:51 AM4/16/12
to hyperme...@googlegroups.com
Eric, you are right, I have deliberately avoided talking about atom, webdav and similar not-so-generic media types. The problem I am trying to solve is that of a business that wants to open up its data for partners. If its a biochemical business it would need to represent information about molecules and what not in that domain, if its a e-procurement business it needs to expose sales orders and customers and such like.

By "domain specific media type" I mean a media type that is explicit about all the business data available - not just title+description+category (and slightly more) that Atom exposes, but also price, customer information, order numbers and so on - all the business domain specific details.

By "domain agnostic media type" I mean something like JSON, HAL or (X)HTML that are domain agnostic with respect to the business at hand. As you say, HTML is domain specific w.r.t. to rendering and user input - but not w.r.t. biochemistry or e-procurement.

Maybe its better to look at it from a client's point of view - the client has a certain use case it needs to solve:

- It may be that it has a human user that drives it, so it asks for HTML which is good for a human operator.

- It could be that it is collecting molecular data from various disparate sources and therefore it asks for a chemistry domain specific representation of data using either application/chemistry+xml or application/periodictable+xml (fictive domain specific media types). If all it could get was HAL or XML then it had no idea of what shape/vocabulary the returned data had. Mike will answer this with "it knows the shape/vocabulary because it followed a specific link" - but that means we must have different URLs for the same resource (and prohibit each of the referred resources to evolve over time).

- It could also be that it was focusing on keeping track of events in the system and thus it would use Atom.

So, I am looking for something that allows the client to get business domain specific data from a resource. Either using a media type which is business domain agnostic (XML, JSON, HTML, HAL) or using a business domain specific media type (which obviously depends on the chosen domain, so no example here). My latest post was an attempt to compare these two different approaches.

Did that clarify anything?

/Jørn

Mike Kelly

unread,
Apr 16, 2012, 5:17:45 AM4/16/12
to hyperme...@googlegroups.com
On Mon, Apr 16, 2012 at 8:35 AM, Erik Wilde <dr...@berkeley.edu> wrote:

for example, i had a brief exchange with mike who was saying that HTML
clearly is domain-agnostic, whereas my view of HTML is that it is
specific for human-oriented hypertext documents with a certain set of
features (but of course it is less specific that some other media
types). my guess is you could argue about this endlessly, so before we
start referring to these two things, we should at least know what we are
referring to.


That isn't an 'application domain' of HTML, it just reflects the type of interface it's designed to expose. Every hypermedia type is going to have a 'specific' interface to expose of some kind, so I can't see the point in that categorisation/distinction. For me, an application domain would be 'email client' or 'personal banking' - HTML is agnostic about application domains, it's generic in this sense, and that's why it's so become so ubiquitous and effective.

Cheers,
Mike

Mike Kelly

unread,
Apr 16, 2012, 5:24:58 AM4/16/12
to hyperme...@googlegroups.com
On Mon, Apr 16, 2012 at 9:45 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:
>
> - It could be that it is collecting molecular data from various disparate
> sources and therefore it asks for a chemistry domain specific representation
> of data using either application/chemistry+xml or
> application/periodictable+xml (fictive domain specific media types). If all
> it could get was HAL or XML then it had no idea of what shape/vocabulary the
> returned data had. Mike will answer this with "it knows the shape/vocabulary
> because it followed a specific link" - but that means we must have different
> URLs for the same resource (and prohibit each of the referred resources to
> evolve over time).

Why do they need to share the same URL? I don't see how this presents a problem

Cheers,
Mike

Jørn Wildt

unread,
Apr 16, 2012, 5:35:54 AM4/16/12
to hyperme...@googlegroups.com
Maybe talking about domain specific/agnostic media types is a wrong approach ... perhaps we should talk about use case specific media types instead? Then I could say:

- text/html is for the use case of a client that has a human operator. It is then the operators responsibility to decipher the meaning of the HTML.

- application/atom+xml is for clients that wants to keep track of stuff but are ignorant of business domain specific stuff.

- application/gml+xml is for clients that needs geo-stuff to solve its use case.

- application/chemistry+json is for clients that needs chemistry information.

- application/xcard+xml is for a client looking for the address of something.

... and so on, deliberately avoiding to talk about the actual type of a resource.

Jørn Wildt

unread,
Apr 16, 2012, 5:49:51 AM4/16/12
to hyperme...@googlegroups.com
> Why do they need to share the same URL? I don't see how this presents a problem

I assume you mean "I don't see how having different URLs represent a problem"?

Because we then fix the format of the content of a specific URL always and forever - meaning the resource is not allowed to change/evolve over time (by being able to return never and richer media types).

Because we loose the ability to have an authoritative URL for a specific thing. Lets assume I put myself on the web as http://jorn.elfisk.dk - this is "me". That would not be possible, because "me" would have to be http://jorn.elfisk.dk/version1 and http://jorn.elfisk.dk/version2 or http://jorn.elfisk.dk/simulated-media-type-1 and http://jorn.elfisk.dk/simulated-media-type-derivation-2 (all of these URLs returning only app/hal+xml).

/Jørn

Jørn Wildt

unread,
Apr 16, 2012, 2:31:38 PM4/16/12
to hyperme...@googlegroups.com
> > Why do they need to share the same URL? I don't see how this presents a
> > problem
> I assume you mean "I don't see how having different URLs represent a
> problem"?

One more ...

Because a client cannot store a URL to a resource and then later on be
upgraded to work with a better and improved version of that resource
(representation) - it would have no knowledge of where to look for that new
version. Compare this to media type content negotion where the client can
simply request a new format from the same URL using "Accept".

/J�rn

----- Original Message -----
From: J�rn Wildt
To: hyperme...@googlegroups.com
Sent: Monday, April 16, 2012 11:49 AM
Subject: Re: Domain specific media types?

> Why do they need to share the same URL? I don't see how this presents a
> problem

I assume you mean "I don't see how having different URLs represent a
problem"?

Because we then fix the format of the content of a specific URL always and
forever - meaning the resource is not allowed to change/evolve over time (by
being able to return never and richer media types).

Because we loose the ability to have an authoritative URL for a specific
thing. Lets assume I put myself on the web as http://jorn.elfisk.dk - this
is "me". That would not be possible, because "me" would have to be
http://jorn.elfisk.dk/version1 and http://jorn.elfisk.dk/version2 or
http://jorn.elfisk.dk/simulated-media-type-1 and
http://jorn.elfisk.dk/simulated-media-type-derivation-2 (all of these URLs
returning only app/hal+xml).

/J�rn

Den mandag den 16. april 2012 11.24.58 UTC+2 skrev Mike Kelly:

On Mon, Apr 16, 2012 at 9:45 AM, J�rn Wildt <j...@fjeldgruppen.dk> wrote:
>
> - It could be that it is collecting molecular data from various disparate
> sources and therefore it asks for a chemistry domain specific
> representation
> of data using either application/chemistry+xml or
> application/periodictable+xml (fictive domain specific media types). If
> all
> it could get was HAL or XML then it had no idea of what shape/vocabulary
> the
> returned data had. Mike will answer this with "it knows the
> shape/vocabulary
> because it followed a specific link" - but that means we must have
> different
> URLs for the same resource (and prohibit each of the referred resources to
> evolve over time).
Why do they need to share the same URL? I don't see how this presents a
problem
Cheers,
Mike

--

You received this message because you are subscribed to the Google Groups
"Hypermedia Web" group.

To view this discussion on the web visit
https://groups.google.com/d/msg/hypermedia-web/-/HnwVqJMoHf0J.

Darrel Miller

unread,
Apr 16, 2012, 5:05:36 PM4/16/12
to hyperme...@googlegroups.com
I agree with Erik. HTML was designed for the application domain of
sharing human readable text documents. The fact that we have been
able to shove email clients and personal banking into a format
designed to allow researchers to share papers is a testament to human
ingenuity and HTMLs flexibility.

Darrel

> --

Mike Kelly

unread,
Apr 17, 2012, 9:22:33 AM4/17/12
to hyperme...@googlegroups.com
On Mon, Apr 16, 2012 at 10:05 PM, Darrel Miller <darrel...@gmail.com> wrote:
> I agree with Erik.  HTML was designed for the application domain of
> sharing human readable text documents.  The fact that we have been
> able to shove email clients and personal banking into a format
> designed to allow researchers to share papers is a testament to human
> ingenuity and HTMLs flexibility.

Errr, no. HTML is way more than sharing text documents... what do
forms have to do with that?

Regardless of what the original intended design was, what we have
ended up with is a generic media type for exposing a resource as a
graphical user (hypertext) interface - and it is incredibly effective
because it is generic and because that affords the existence of 'The
Web Browser' which everyone on the planet can use to go about their
daily business surfing through documents, applications, and
pornography.

There are no examples of graphical user (hypertext) interfaces built
for specific application domains coming anywhere near HTML in terms of
their contribution to the web. In fact, what was the poster child for
REST and hypertext and domain-specific media types (AtomPub) ended up
being a colossal failure and didn't get anywhere near the adoption
everyone thought it would. Go figure.

Cheers,
Mike

Rushforth, Peter

unread,
Apr 17, 2012, 9:48:24 AM4/17/12
to hyperme...@googlegroups.com
Mike,

> In fact, what was the poster child for REST and hypertext and
> domain-specific media types (AtomPub) ended up being a
> colossal failure and didn't get anywhere near the adoption
> everyone thought it would. Go figure.

Atompub has got some traction. The problem is that it is very easy to
un-follow the constraints of REST if you don't have them as an
objective. What is needed is more guidance applying atompub without
breaking the constraints of REST, while still achieving the goals that
one's "API" set out.

Hypermedia, natch!

Cheers,
Peter


Mike Kelly

unread,
Apr 17, 2012, 11:57:17 AM4/17/12
to hyperme...@googlegroups.com
On Mon, Apr 16, 2012 at 10:49 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:
>> Why do they need to share the same URL? I don't see how this presents a
>> problem
>
> I assume you mean "I don't see how having different URLs represent a
> problem"?
>
> Because we then fix the format of the content of a specific URL always and
> forever - meaning the resource is not allowed to change/evolve over time (by
> being able to return never and richer media types).

You can change your URLs to whatever you want, the link relations are
what is being versioned. Which means you can keep the 'canonical URL'
with the most up-to-date representation and offload the old, legacy
one to some other URL.

e.g. you would go from having this:

<link rel="foo" href="/canonical" />

to then having:

<link rel="foo-v2" href="/canonical" />
+
<link rel="foo" href="/suckyoldthing" />

Cheers,
Mike

Mike Kelly

unread,
Apr 17, 2012, 12:03:15 PM4/17/12
to hyperme...@googlegroups.com
On Mon, Apr 16, 2012 at 7:31 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:
>> > Why do they need to share the same URL? I don't see how this presents a
>> > > problem
>> I assume you mean "I don't see how having different URLs represent a
>> problem"?
>
>
> One more ...
>
> Because a client cannot store a URL to a resource and then later on be
> upgraded to work with a better and improved version of that resource
> (representation) - it would have no knowledge of where to look for that new
> version. Compare this to media type content negotion where the client can
> simply request a new format from the same URL using "Accept".
>

I don't get what is different with media types.. in that case, how
does a client know that there's a new media type it should ask for?

Darrel Miller

unread,
Apr 17, 2012, 12:42:34 PM4/17/12
to hyperme...@googlegroups.com
"Regardless of what the original intended design was,..."

Feel free to re-purpose HTML however you see fit, apparently that's
what others did,

"In April 1993 ... Mosaic extended the features specified by Tim
Berners-Lee; for example, it added images, nested lists and fill-out
forms. Academics and software engineers later would argue that many of
these extensions were very much ad hoc and not properly designed."
http://www.w3.org/People/Raggett/book4/ch02.html

:-)

Darrel

Jørn Wildt

unread,
Apr 17, 2012, 2:20:00 PM4/17/12
to hyperme...@googlegroups.com
> Which means you can keep the 'canonical URL'
> with the most up-to-date representation and offload the old, legacy
> one to some other URL.
> e.g. you would go from having this:
> <link rel="foo" href="/canonical" />
> to then having:
> <link rel="foo-v2" href="/canonical" />
> +
> <link rel="foo" href="/suckyoldthing" />

That would work for new and shiny clients that understand the new format
that suddenly appeared, out of no where, at /canonical. Unfortunately it
would break older clients that would not be prepared for the new format at
/canonical.

And, no, the older clients would *not* just have followed a link. 1) Older
clients could have stored it in their own local storage for later use, and
2) Other servers, out of your control, could have linked to /canonical as
rel="foo" - there would be now way for you to inform those servers of the
changes, thus breaking older clients following those links.

This still makes it impossible for clients and server to evolve
independently of each other.

/J�rn

Jørn Wildt

unread,
Apr 17, 2012, 2:24:56 PM4/17/12
to hyperme...@googlegroups.com
> > Compare this to media type content negotion where the client can
> > simply request a new format from the same URL using "Accept".

> I don't get what is different with media types.. in that case, how
> does a client know that there's a new media type it should ask for?

An old client would not know there was a new media type to ask for. So it
continuous to send "Acccept: application/old-media-type" (as it should
always have done).

A new client would be aware of the new media type so it would send "Accept:
application/new-media-type".

It could even be that the new client could solve it's current task using
both the old media type and the new one. In this case it sends "Accept:
application/new-media-type, application/old-media-type" (probably with a
quality parameter in there too).

/J�rn

----- Original Message -----
From: "Mike Kelly" <mi...@mykanjo.co.uk>
To: <hyperme...@googlegroups.com>
Sent: Tuesday, April 17, 2012 6:03 PM
Subject: Re: Domain specific media types?

--

You received this message because you are subscribed to the Google Groups
"Hypermedia Web" group.

Mike Kelly

unread,
Apr 17, 2012, 3:21:33 PM4/17/12
to hyperme...@googlegroups.com

Agreed, it's not bulletproof form that pov.. to be honest I've reread
what you wrote, and I think I was responding to a point you didn't
actually make! Sorry about that :)

So I think I now understand your points which were:

1) Resources can't "evolve" beyond non-breaking changes if you don't use conneg
2) You're worried that using not using conneg will prevent you from
having a 'canonical URL'

re: point 1 - I'm not sure how/why this is a problem. The objective
should be to evolve your application(s), not your individual
resources. You can evolve your application by creating new resources
for new variants that introduce a breaking change, the old resources
can live amongst the new ones just fine; the costs of maintaining the
legacy resources is the same as with media type; arguably lower
actually since caching variants that share the same URL is generally a
pita, whereas if each version has its own URL it is very simple to
deal with. Oh, being able to share a link that explicitly refers to a
specific version and/or format is a nice thing you don't get if you
hide away this stuff with conneg.

re: point 2 - I think establishing a link relation called 'canonical'
or 'represents' would solve this problem for you?

Cheers,
Mike

Mike Kelly

unread,
Apr 17, 2012, 3:24:23 PM4/17/12
to hyperme...@googlegroups.com
On Tue, Apr 17, 2012 at 7:24 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:
>> > Compare this to media type content negotion where the client can
>> > simply request a new format from the same URL using "Accept".
>
>
>> I don't get what is different with media types.. in that case, how
>> does a client know that there's a new media type it should ask for?
>
>
> An old client would not know there was a new media type to ask for. So it
> continuous to send "Acccept: application/old-media-type" (as it should
> always have done).
>
> A new client would be aware of the new media type so it would send "Accept:
> application/new-media-type".

Right, but how is that any different from offering a new link relation
in parallel to the old one?

Cheers,
Mike

Jan Algermissen

unread,
Apr 17, 2012, 4:00:32 PM4/17/12
to hyperme...@googlegroups.com

On Apr 17, 2012, at 8:24 PM, Jørn Wildt wrote:

>> > Compare this to media type content negotion where the client can
>> > simply request a new format from the same URL using "Accept".
>
>> I don't get what is different with media types.. in that case, how
>> does a client know that there's a new media type it should ask for?
>
> An old client would not know there was a new media type to ask for. So it continuous to send "Acccept: application/old-media-type" (as it should always have done).
>
> A new client would be aware of the new media type so it would send "Accept: application/new-media-type".
>
> It could even be that the new client could solve it's current task using both the old media type and the new one. In this case it sends "Accept: application/new-media-type, application/old-media-type" (probably with a quality parameter in there too).
>

Exactly.

Jan

> /Jørn

Jørn Wildt

unread,
Apr 17, 2012, 4:07:46 PM4/17/12
to hyperme...@googlegroups.com
> 1) Resources can't "evolve" beyond non-breaking changes if you don't use
> conneg
> re: point 1 - I'm not sure how/why this is a problem.

Because:

> 1) Older clients could have stored it in their own local storage for later
> use, and
> 2) Other servers, out of your control, could have linked to /canonical as
> rel="foo" - there would be now way for you to inform those servers of the
> changes, thus breaking older clients following those links.

To which you answered:

> Agreed, it's not bulletproof form that pov..

This may not be a problem in an environment where clients and servers can be
expected to upgrade more or less in lock step. It could also work if the
organisation behind the server doesn't care for their clients and simply
ignore the problem of older clients.

> Oh, being able to share a link that explicitly refers to a
> specific version and/or format is a nice thing you don't get if you
> hide away this stuff with conneg.

Yes, that is a nice to have feature of different formats => different URLs.
So is the ability to have standard tooling with generic media types. But it
is not a necessity - things won't break if you don't have this feature.

What I am trying to, is to compare the different approaches against the
benefits promissed by REST and see if they break anything. I don't want to
spend hours and days on creating a RESTful API - just to discover that,
duh!, it did not support "independently evolving servers and clients" at the
time when I really need it - in production, when a bunch of
disparate-out-of-my-control desktop clients and system integrations *will*
break and cost my company dearly.

So, sorry, for being a stuborn argumenting word splitting old chap. But
these discussions *does* help me tremendously in understanding what I am
working with.

Thanks, J�rn

Jan Algermissen

unread,
Apr 17, 2012, 4:08:32 PM4/17/12
to hyperme...@googlegroups.com

Right. Breaks the 'Cool URIs don't change' principle and I guess also the 'Identification of Resources' constraint (because the same URI is used to reference different resources) [Though I'd need to check back with the dis for details]

What strikes me as really problematic here is that a rather stable design artifact (link rels) is used to address the most unstable system aspect. (Syntax/data model evolution).

Media types and conneg are the things in REST that primarily address evolution. Why tweak that?

Jan

>
> This still makes it impossible for clients and server to evolve independently of each other.
>

> /Jørn

Mike Kelly

unread,
Apr 17, 2012, 4:16:08 PM4/17/12
to hyperme...@googlegroups.com

On 17 Apr 2012, at 09:08 PM, Jan Algermissen <algermi...@mac.com> wrote:

>
> On Apr 17, 2012, at 8:20 PM, Jørn Wildt wrote:
>
>>> Which means you can keep the 'canonical URL'
>>> with the most up-to-date representation and offload the old, legacy
>>> one to some other URL.
>>> e.g. you would go from having this:
>>> <link rel="foo" href="/canonical" />
>>> to then having:
>>> <link rel="foo-v2" href="/canonical" />
>>> +
>>> <link rel="foo" href="/suckyoldthing" />
>>
>> That would work for new and shiny clients that understand the new format that suddenly appeared, out of no where, at /canonical. Unfortunately it would break older clients that would not be prepared for the new format at /canonical.
>>
>> And, no, the older clients would *not* just have followed a link. 1) Older clients could have stored it in their own local storage for later use, and 2) Other servers, out of your control, could have linked to /canonical as rel="foo" - there would be now way for you to inform those servers of the changes, thus breaking older clients following those links.
>
> Right. Breaks the 'Cool URIs don't change' principle and I guess also the 'Identification of Resources' constraint (because the same URI is used to reference different resources) [Though I'd need to check back with the dis for details]
>
> What strikes me as really problematic here is that a rather stable design artifact (link rels) is used to address the most unstable system aspect. (Syntax/data model evolution).
>
> Media types and conneg are the things in REST that primarily address evolution. Why tweak that?

The web's evolved a lot with text/html.

Jan Algermissen

unread,
Apr 18, 2012, 3:16:36 AM4/18/12
to hyperme...@googlegroups.com

On Apr 17, 2012, at 10:07 PM, Jørn Wildt wrote:

>
> Yes, that is a nice to have feature of different formats => different URLs. So is the ability to have standard tooling with generic media types. But it is not a necessity - things won't break if you don't have this feature.
>
>

You can still have separate URIs for the different formats (that is a best practice anyhow). There are various mechanism in conneg to redirect the client or even inform it about all the variant URIs.

E.g.

http://algermissen.blogspot.de/2010/09/spotted-alternates-header-in-wild.html

Jan


Jørn Wildt

unread,
Apr 18, 2012, 3:22:25 AM4/18/12
to hyperme...@googlegroups.com
I have tried to wrap up this way of thinking in a blog post: http://soabits.blogspot.com/2012/04/restful-resources-are-not-typed.html. It doesn't cover the later dicsussion on multiple resources vs. multiple media types - just tries to explain how one can avoid thinking of a resource as having a specific "type" - something which has had me puzzled for a long time.

/Jørn

Den lørdag den 14. april 2012 09.30.59 UTC+2 skrev Jan Algermissen:

On Apr 14, 2012, at 9:14 AM, Jørn Wildt wrote:

> That was a very useful description Jan. Thanks. Let me try an example to verify that I understand what you are saying:
>
> 1) The homepage/resource of Copenhagen airport is http://www.cph.dk. Lets call it {Cph}.
>
> 2) Another completely unrelated resource {ShopOrder} exposes a sales order with Copenhagen airport as customer. This sales order contains a link like this: <link href="{Cph}" rel="customer"/>.
>
> 3) Yet another resource {Flight} thinks of {Cph} as being the destination of a flight using the link <link href="{Cph}" rel="destination"/>
>
> A client comming from the shop (2) could try to get in contact with the customer by sending a request for a XCard to {Cph}.

Yup - the issue here is: On what grounds does the client(-developer) decide what to put in the Accept header. Essentially the accept header means: If you send your response in foo/bar I am able to work through my use case from there. So, yes. If your client is coded to proceed through the use case if the GET returns a xcard then the client tells that to the server using Accept.

>
> A client comming from the flight description (3) could try to get more information about the destination, sending a request for something like application/location+xml to {Cph}.

Yes. Essentially saying: I can proceed if I get a response in application/location+xml.


>
> None of the clients has any idea of the actual type behind {Cph} - they simply ask for various representations from {Cph} through the use of different media types.

Yes, but the various media types are not the issue here. Could well be the same type. What matters is that the client expects the resource at the end of the customer link to 'be' something but it does not matter what the resource really is (it is just a resource anyhow).

>
> Is this what you are trying to tell me? That makes a lot of sense to me and explains Fieldings "A REST API should never have "typed" resources that are significant to the client".

Yes, I think so. Glad if I can help. Otherwise, keep asking :-)


>
> But this point of view doesn't fit very well with generic media types, does it?

Generic media types are an anti pattern (sorry folks :-). You need specific semantics to get stuff done. Otherwise you just shove the issue up one layer, thereby creating out of band coupling.

> Sending a request for application/hal+xml for {Cph} won't solve the two use cases above - the {Cph} resource would not be able to return either XCard or Location. But, as you have stated earlier, a media type could simply be an alias for an existing generic media type, thereby reducing the amount of work needed to create a new media type.

Yup - you can easily express a subset of XHTML with, say RelaxNG, specify the semantics, name that thing application/customer and be done with your generic type. Parsers come for free as well as sending text/html to HTML-user agents based on conneg.

We need agreement (contract) to enable communication to happen. The artifact where this contract lives in REST is the media type (and only the media type) [well, plus link rels of course] and this is where all design work must happen when creating RESTful systems. There is simply no other artifact in REST where you can put any contract information.


>
> Thanks, Jørn


>
> --
> You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.

> To post to this group, send email to hypermedia-web@googlegroups.com.
> To unsubscribe from this group, send email to hypermedia-web+unsubscribe@googlegroups.com.

Mike Kelly

unread,
Apr 18, 2012, 5:02:19 AM4/18/12
to hyperme...@googlegroups.com
On Tue, Apr 17, 2012 at 9:07 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:
>> 1) Resources can't "evolve" beyond non-breaking changes if you don't use
>> conneg
>> re: point 1 - I'm not sure how/why this is a problem.
>
>
> Because:
>
>
>> 1) Older clients could have stored it in their own local storage for later
>> use, and
>> 2) Other servers, out of your control, could have linked to /canonical as
>> rel="foo" - there would be now way for you to inform those servers of the
>> changes, thus breaking older clients following those links.
>
>
> To which you answered:
>
>
>> Agreed, it's not bulletproof form that pov..

No, my revised answer having figured out the point you were making was:

Rushforth, Peter

unread,
Apr 18, 2012, 8:07:15 AM4/18/12
to hyperme...@googlegroups.com
Jan,

That is an interesting way to provide alternates. I don't see how it provides
a different url for the alternate. Somewhere I think
I saw you had said that one reason to not provide URLs to alternate formats
was because of the possibility of future 404s where the resource still exists
but the url to the representation has disappeared (because you decided to
no longer support that format, or more likely because you decided to change the
name or value of the parameter under which you provided that url.

Peter

> -----Original Message-----
> From: hyperme...@googlegroups.com
> [mailto:hyperme...@googlegroups.com] On Behalf Of Jan Algermissen
> Sent: April 18, 2012 03:17
> To: hyperme...@googlegroups.com
> Subject: Re: Domain specific media types?
>
>

> --
> You received this message because you are subscribed to the
> Google Groups "Hypermedia Web" group.

> To post to this group, send email to hyperme...@googlegroups.com.


> To unsubscribe from this group, send email to

> hypermedia-we...@googlegroups.com.

Rushforth, Peter

unread,
Apr 18, 2012, 8:08:01 AM4/18/12
to hyperme...@googlegroups.com
oops sorry I see where the file name is presented.
Cheers,

Mike Kelly

unread,
Apr 18, 2012, 8:28:51 AM4/18/12
to hyperme...@googlegroups.com
On Wed, Apr 18, 2012 at 8:16 AM, Jan Algermissen
<algermi...@mac.com> wrote:
>
> On Apr 17, 2012, at 10:07 PM, Jørn Wildt wrote:
>
>>
>> Yes, that is a nice to have feature of different formats => different URLs. So is the ability to have standard tooling with generic media types. But it is not a necessity - things won't break if you don't have this feature.
>>
>>
>
> You can still have separate URIs for the different formats (that is a best practice anyhow). There are various mechanism in conneg to redirect the client or even inform it about all the variant URIs.

Right, in which case you will have a URI per version just as with the
link relation approach.

Peter Williams

unread,
Apr 18, 2012, 12:40:55 PM4/18/12
to hyperme...@googlegroups.com
On Wed, Apr 18, 2012 at 6:28 AM, Mike Kelly <mi...@mykanjo.co.uk> wrote:
>> You can still have separate URIs for the different formats (that is a best practice anyhow). There are various mechanism in conneg to redirect the client or even inform it about all the variant URIs.
>
> Right, in which case you will have a URI per version just as with the
> link relation approach.

So you suggest having a separate resource for each incompatible
representation of single logical entity? It seems to me if you follow
this to its logical conclusion, content negotiation could be dispensed
with altogether. Is that your belief? If not where did my logic break
down?

Peter
barelyenough.org

Mike Kelly

unread,
Apr 19, 2012, 8:49:09 AM4/19/12
to hyperme...@googlegroups.com
On Wed, Apr 18, 2012 at 5:40 PM, Peter Williams <pe...@barelyenough.org> wrote:
> On Wed, Apr 18, 2012 at 6:28 AM, Mike Kelly <mi...@mykanjo.co.uk> wrote:
>>> You can still have separate URIs for the different formats (that is a best practice anyhow). There are various mechanism in conneg to redirect the client or even inform it about all the variant URIs.
>>
>> Right, in which case you will have a URI per version just as with the
>> link relation approach.
>
> So you suggest having a separate resource for each incompatible
> representation of single logical entity?

Yeah, if it was necessary to expose that single logical entity as a
resource then I would make the relation between the representation
resource(s) and that canonical resource explicit with a link in the
representation e.g.:

<resource href="/foo.hal">
<link rel="represents" href="/foo" />
</resource>


> It seems to me if you follow
> this to its logical conclusion, content negotiation could be dispensed
> with altogether. Is that your belief?

I haven't thought much about the hypothetical scenario in which this
would be feasible (as it would break the web) but I guess that is the
logical conclusion if that were the case (which it's not).

But - to be clear - I definitely would _not_ use conneg as the means
to evolve my application.. countless web sites & apps have established
and evolved their applications happily using HTML without the need to
change media type identifiers. Media types are not a good vehicle for
managing change, they are extremely coarse, and because of the way
MIME and HTTP operate they are not particularly discoverable either.

If applications state is driven by hypertext, then it makes sense to
manage and direct change with hypertext - which means introducing new
link relations and additional resources as targets for those links.
Mark Nottingham wrote this out in black and white last year:
http://www.mnot.net/blog/2011/10/25/web_api_versioning_smackdown

Cheers,
Mike

http://www.mnot.net/blog/2011/10/25/web_api_versioning_smackdown

Jørn Wildt

unread,
Apr 19, 2012, 11:39:53 AM4/19/12
to hyperme...@googlegroups.com
Mike, lets back track a bit ...

This all started with a discussion about generic vs. specialized media
types. May I ask about your view on existing specialized media types like
application/atom+xml and application/svg+xml which are XML formats?

According to your view, these media types are not needed on the web - the
generic application/xml could as well have been used ... since the client
that requests these must, per your argumentation, have followed a link that
states "there is an Atom feed here" or "embed this SVG image here". No need
for specialized media types - right?

If you do not agree with this what is it then that makes these media types
special?

/J�rn

----- Original Message -----
From: "Mike Kelly" <mi...@mykanjo.co.uk>
To: <hyperme...@googlegroups.com>

Sent: Thursday, April 19, 2012 2:49 PM
Subject: Re: Domain specific media types?

Mike Kelly

unread,
Apr 19, 2012, 12:54:52 PM4/19/12
to hyperme...@googlegroups.com
On Thu, Apr 19, 2012 at 4:39 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:
> Mike, lets back track a bit ...
>
> This all started with a discussion about generic vs. specialized media
> types. May I ask about your view on existing specialized media types like
> application/atom+xml and application/svg+xml which are XML formats?
>
> According to your view, these media types are not needed on the web - the
> generic application/xml could as well have been used ... since the client
> that requests these must, per your argumentation, have followed a link that
> states "there is an Atom feed here" or "embed this SVG image here". No need
> for specialized media types - right?

application/xml doesn't have any links so that wouldn't be very
appropriate as a generic media type for hypertext. But yes, you could
produce standard link relations like 'feed' or 'svg' or 'image'

> If you do not agree with this what is it then that makes these media types
> special?

I'm promoting the idea that having a generic type for a hypertext
interface (a la HTML) is a good thing, and that media types are not
for varying/negotiating/establishing application semantics. We still
need different media types for different *processing models*, i.e.
JSON vs XML vs HAL vs JPEG vs WebP vs WebM.

i.e. you don't need a new media type if all you are doing is
representing links, embedded resources, and resource state - that is
what HAL or Collection+JSON's processing models are targeted at
already.

Cheers,
Mike

Jørn Wildt

unread,
Apr 19, 2012, 3:16:39 PM4/19/12
to hyperme...@googlegroups.com
> > According to your view, these media types are not needed on the web -
> > the
> > generic application/xml could as well have been used

> application/xml doesn't have any links so that wouldn't be very


> appropriate as a generic media type for hypertext

That hardly matters since the link relations are typed. This means the
client will know how to look for links based on the fact that "feed" means
"At this URL you will find an XML document structured according to the Atom
definition" (and thus include <atom:link/> elements). The link would
although have to be named something like "feed-atom" to avoid confusion with
"feed-rss" and other feed formats.

> i.e. you don't need a new media type if all you are doing is
> representing links, embedded resources, and resource state

Okay, that could be Atom or RSS. So we can throw those away and stick to
using plain XML or HAL which again means that we don't need "feed-atom" and
"feed-rss" relations but can go back to "feed" since everyone should be
using HAL or XML for simply representing links and embedded resource.

> We still
> need different media types for different *processing models*, i.e.
> JSON vs XML vs HAL vs JPEG vs WebP vs WebM.

But XML and HAL has the same processing model - don't they? Both are XML and
can be parsed into a XML DOM with most tools - and there is a link leading
to them so the client will know that even if the server returned
application/xml, the client could trust the link and decode the XML as HAL
content.

I assume you don't agree with this, otherwise you wouldn't invent HAL. So
XML and HAL are somehow different and thus requires "different processing" -
what ever that means?

I assume you would say the same about XCard vs. XCal vs. HAL vs. XML => all
are XML derivations but are application specific enough to mandate their own
media types.

But why doesn't an application for cooking recipies or e-procurement mandate
its own media type?

/J�rn


----- Original Message -----
From: "Mike Kelly" <mi...@mykanjo.co.uk>
To: <hyperme...@googlegroups.com>
Sent: Thursday, April 19, 2012 6:54 PM
Subject: Re: Domain specific media types?

Cheers,
Mike

--

Jan Algermissen

unread,
Apr 19, 2012, 3:42:46 PM4/19/12
to hyperme...@googlegroups.com

On Apr 19, 2012, at 9:16 PM, Jørn Wildt wrote:

> But why doesn't an application for cooking recipies or e-procurement mandate its own media type?

Yep, exactly.

Or, turning that around: *If* generic media types are suitable for all cases, why was e.g. HTML (being a specific media type) created in the first place?

Jan


Reply all
Reply to author
Forward
0 new messages