Representing resources with multiple addressing paths
Don Day
to address in multiple ways:
* The application supports multiple workspaces that I've chosen to
call groups--like projects or domains of knowledge. So there is a
first path segment of:
groups/ (collection)
groups/{groupname}/ (collection/element)
* A group contains several kinds of resources: topics, lists, job
files, images, etc. The next natural keyword is the plural name for
each of these kinds:
- groups/{groupname}/topics (collection/element/collection)
- groups/{groupname}/topics/{topicname} (collection/element/collection/
element)
This is fine so far--by using the plural noun for each resource
(normally sorted by extension in a file system), I can access any such
resource individually.
However, another form of collection is a named list of topics, where
the normal form of addressing is:
- {groupname}/{listname} (equivalent to an aggregated document--a
table of contents, if you will)
- {groupname}/{listname}/{topicname} (equivalent to opening to a
chapter or section in a book context, for example)
Furthermore, imported resources may have been structured with folders
for file-based organization, so a topic may be in a physical folder
path of concepts/{topicname}. My chief headache is when users choose
to use folder names that are also required collection names in the
RESTful scheme (name collision), such as topics/{topicname} (where
'topics' is a folder name that qualifies the topic's full pathname,
not a collection keyword). I can solve this by artificially flattening
the resource name to 'topics.{topicname}" but this imposes a
conceptual burden for users who expect to see the / that they know is
in the system filepath where they uploaded the resource from.
I think the RESTful solution is to introduce a keyword to represent
contextual sets:
- sets/{groupname}/{listname}/{topicname}
This satisfies the syntactic sugar for making {groupname} appear in
the same path position using either form of addressing (and
consistency for routing). It just seems like the 'sets' keyword is
excessive baggage for the simple case of pointing directly to a topic
that opens up in a hierarchical context. Furthermore, sets are
themselves possibly real resources in a group--note that a list of
bookmarks in a browser is effectively a list of pointers to different
topics in different contexts, and we manage sets of bookmarks all the
time as a way to manage knowledge in meaningful categories of our own
choosing.
It's the nature of knowledge to have richness in classification. I'm
vexed by how to represent those alternate routes to the same resource
in different contexts in a consistent RESTful scheme! Can anyone
recommend any other examples of RESTful API designs that deal with
addressing resources with multiple/deep schemes? (more than just the
'category/GUID' search result representation that we see most
commonly)
FWIW, the content type I'm trying to address in this RESTful manner is
XML conforming to the OASIS DITA spec (Darwin Information Typing
Architecture).
Thanks! I hope this is not a hopeless newbie kind of question.
--
Don
Jørn Wildt
care less about the URLs. Pick whatever fits your day ... and change it
again tomorrow. That should not be a problem for a proper REST service.
The point is, you should not publish those URL structures as part of your
API. Instead you should publish one URL that points to some kind of service
document that presents all these URL schemes to the client as URL templates
identified by some abstract name.
It is then up to the client to read the sevice document, look for the
template which is named "xxx" and then substitute parameter values into in.
What you then need to document is: template names and related template
parameters.
A template could be as simple as an HTML document with a list of anchors
with human readable descriptions:
<ul>
<li><a href="{service-1-template}" rel="service-1">... description of link
"Service-1" ...</a></li>
</ul>
When this is in place you can start discussing URL schemes - but only with
your fellow devs, never to the public :-)
That doesn't change the fact that you need to come up with *some* URL
scheme - but the exact choice is irrelevant. It neither has to be pretty,
understandable or anything else other than discoverable.
/J�rn
Mike Schinkel
The point is, you should not publish those URL structures as part of your API. Instead you should publish one URL that points to some kind of service document that presents all these URL schemes to the client as URL templates identified by some abstract name.
Point of note, there are some of us who don't believe that approach is ideal for all use-cases. Maybe we are wrong, but most publicly published APIs to-date do publish URL structures and do not use HATEOAS.
I think one reason for this state of affairs is it's much easier for the client to call concrete URLs via construction than to work with URLs that are abstracted. So if reducing friction to API adoption is the most important goal I argue that "doing it the right way" might lead to less success in achieving that most important goal. FWIW.
-Mike
Jørn Wildt
document adds an extra layer of abstraction.
> Maybe we are wrong, but most publicly published APIs to-date do publish
> URL structures and do not use HATEOAS.
> I think one reason for this state of affairs is it's much easier for the
> client to call concrete URLs via construction than to work with URLs that
> are abstracted
I was actually suggesting URL construction - but by letting the client fetch
the URL template from the service instead of hardwiring the templates into
the client. But, yes, it is one extra indirection.
> So if reducing friction to API adoption is the most important goal I argue
> that "doing it the right way" might lead to less success in achieving that
> most important goal.
Right! So what can be done to reduce friction of adapting hyper media APIs?
1) Spreading the word, 2) examples "in the wild", and 3) Tooling.
Personally, I am working on (1) and (3 - see
https://github.com/JornWildt/Ramone).
Besides that, no one else offered Don a solution :-)
/J�rn
Jørn Wildt
https://groups.google.com/d/msg/hypermedia-web/dVlxE1wJOZY/ImBAEsZl0QgJ
/J�rn
Mike Schinkel
I was actually suggesting URL construction - but by letting the client fetch the URL template from the service instead of hardwiring the templates into the client. But, yes, it is one extra indirection.
Unfortunately those indirections can be expensive in too many use-cases, such as for mobile devices.
Right! So what can be done to reduce friction of adapting hyper media APIs? 1) Spreading the word, 2) examples "in the wild", and 3) Tooling. Personally, I am working on (1) and (3 - see https://github.com/JornWildt/Ramone).
Yes. I'd argue that #3 is critical but that there is a #4 that is more important than all others.#4 would be "standard" JSON and XML formats for hypertext-based interaction, one generally agree to be the standard by all participants as HTML is agreed to be standard for presentation. And this "standard" needs to emerge as a recommendation or at least a generally agreed defacto-standard.Maybe what is needed for #4 to emerge are really good tools (#3) for a given approach?
Besides that, no one else offered Don a solution :-)
My bad, I got confused and thought you were replying to another query. Oops. :)
URI template idea was emerging it seemed to me that Roy envisioned HATEOAS prior to recognizing the need for templates. Back then the idea of URL construction via templates was considered heresy by REST purists. Or at least that's how it seemed to me.
Mike Kelly
URI Template is an RFC http://tools.ietf.org/html/rfc6570
You don't need to understand the URI template spec to use them, you
just need access to a compliant library in your language and to be
told what tokens to apply to a given template.
Cheers,
Mike
Jørn Wildt
> >fetch the URL template from the service instead of hardwiring the
> >templates into the client. But, yes, it is one extra indirection.
> Unfortunately those indirections can be expensive in too many use-cases,
> such as for mobile devices.
Good point. The problem can, to some extend, be mitigated by caching and use
of ETAG for conditional requests. Then you only pay the penalty once -
probably at the time of installing the mobile app in which case you should
have plenty of bandwith.
> #4 would be "standard" JSON and XML formats for hypertext-based
> interaction
Yes. That is something HAL is working on. To be included in Ramone when time
comes :-)
/J�rn
Mike Schinkel
URI Template is an RFC http://tools.ietf.org/html/rfc6570
My bad. It had taken so long I missed that.Thanks for correcting me. Well, at least there finally is a standard.
You don't need to understand the URI template spec to use them, you
just need access to a compliant library in your language and to be
told what tokens to apply to a given template.
-MikeAllow me to be skeptical.Whenever I've heard "You don't need to understand it, the tools will take care of it" I've never seen it end well. Ever heard of SOAP? :)
Jørn Wildt
Mike Kelly
That comparison doesn't make any sense, it's just a standard syntax
for expressing variable URIs - it's perfect for using libraries
against. The reason it has taken "so long" to get out of the door is
because it has been rigorously defined specifically for that purpose.
Here's how you can deal with the a templated URI using the JavaScript library:
result = $.uritemplate(templated_uri).expand({ a: "foo", b: "bar" })
As you can see, you are not exposed to the URI template syntax at all,
you simply deal with it as a string and chuck it at the library.
The JS library is available here, btw:
https://github.com/marc-portier/uri-templates
Cheers,
Mike
Mike Schinkel
>I was actually suggesting URL construction - but by letting the client >fetch the URL template from the service instead of hardwiring the >templates into the client. But, yes, it is one extra indirection.Unfortunately those indirections can be expensive in too many use-cases, such as for mobile devices.
Good point. The problem can, to some extend, be mitigated by caching and use of ETAG for conditional requests. Then you only pay the penalty once - probably at the time of installing the mobile app in which case you should have plenty of bandwith.
Here's a thought that I wonder if anyone is considering?Rather than encoding the link templates into each response, what about offering them via an API-wide resource that could by-convention be retrieved from the API root:
{"_signature": "Web API Meta v1.0","_self": "http://api.example.com","_meta": "http://api.example.com","root": "http://api.example.com","vars":[{name:"event_id"}],"uris":[{id:"events", "uri":"/events"},{id:"event", "uri":"/events/{event_id}"},{id:"event_photos", "uri":"/events/{event_id}/photos"}]}That would mean the client only needs to request the meta once. This approach could of course be extended to include external references in the meta response for those huge APIs where it's too big to force mobile devices to download, much how conceptually a sitemap.xml can be partitioned into multiple files.
Further, the root could indicate another resource for meta if need be,
{"_signature": "Web API Meta v1.0","_meta": "http://api.example.com/meta","_self": "http://api.example.com","other": "stuff","goes": "here."}
Individual responses could provide a link back to their meta data, but it would be *optional*:
{"_meta": "http://api.example.com/meta","event_id": abc123,
"event_name": "Easter Egg Roll",
"event_location": "South Lawn, The White House, Washington DC"}
Another benefit of this approach is that it would layer over any existing API, i.e. Twitter's. And that would allow someone to create a meta file for any existing API that could be made available to the client even of the API provider does not support HATEOAS. So if I want my client to be hypertext driven I can even hardcode into my client the meta for a given API that is not HATEOAS and/or I could code my client to retrieve the meta from my own servers in the case I want my client to be more resilient to change.
This approach would allow good tools to emerge because it's an incremental approach; it doesn't try to boil the ocean.
Thoughts?
Mike Schinkel
Here's how you can deal with the a templated URI using the JavaScript library:
result = $.uritemplate(templated_uri).expand({ a: "foo", b: "bar" })
As you can see, you are not exposed to the URI template syntax at all,
you simply deal with it as a string and chuck it at the library.
The JS library is available here, btw:
https://github.com/marc-portier/uri-templates
I appreciate that. But if someone calls an API from their browser that has URI templates they'll see URI templates them may not fully understand. And that's my point.But it's a standard now (again, thanks for noting that), so my point is probably moot.
Jack Repenning
> URI Template is an RFC http://tools.ietf.org/html/rfc6570
Yes, but it's still in "PROPOSED STANDARD" status, which RFC2026 describes as:
> Implementors should treat Proposed Standards as immature
> specifications. It is desirable to implement them in order to gain
> experience and to validate, test, and clarify the specification.
> However, since the content of Proposed Standards may be changed if
> problems are found or better solutions are identified, deploying
> implementations of such standards into a disruption-sensitive
> environment is not recommended.
That language, and some painful personal experience with WebDAV/Delta-V when it was still PROPOSED (and even into the "Draft" phase), make me hesitant to use this quite yet.
Of course, "URI Templates" is a couple of light-years less complex than Delta-V. And, so far as I can tell, "URI Templates" does not have the level of controversy Delta-V still had at that stage. So it's surely a safer bet. But still, it seems to me worthwhile to take the IETF at its word, and wait for "Draft Standard" status before deploying:
> A Draft Standard is normally considered to be a final specification,
> and changes are likely to be made only to solve specific problems
> encountered. In most circumstances, it is reasonable for vendors to
> deploy implementations of Draft Standards into a disruption sensitive
> environment.
Jack Repenning
Endless invention, endless experiment,
Brings knowledge of motion, but not of stillness;
Knowledge of speech, but not of silence;
Knowledge of words, and ignorance of the Word...
Where is the Life we have lost in living?
Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?
-- T. S. Eliot, Choruses from 'The Rock'
Arlo Belshee
This is exactly the approach we took with OData. Worked reasonably well.
We found there were good reasons to not put the metadata at root. Similarly, we didn’t want to corrupt the API surface (since we were making an API for arbitrary services). So we named the metadata `/$metadata` (by default convention – a link from the root provides the real value).
There are still a whole bunch of details to work out from there (such as what does $metadata contain?). Most of the design at this point was in-house, so I’m not sure how much you’ll be able to glean from the old blog posts (which don’t go back before 2010 anyway).
However, most of the designers who were thinking this stuff through prior to 2010 are still with us. They’d be happy to share some lessons learned if you wanted.
I am definitely in favor of using metadata to drive a web API. It can make the system open for client extension, while still leaving the server in control of what’s allowed and how that is done.
There are just a ton of details involved in getting it right – many of which we learned by getting them wrong and then having to fix it while maintaining backwards compatibility even on the metadata document.
mca
IMO, a better approach if too use "meta" affordance. Clients can recognize the control and servers are free to use any URL they wish including ones in other namespaces (domains) and URLs that match languages, etc.
Mike Amundsen
Arlo Belshee
That was one of many lessons we learned with v2. Now we have a way to state the canonical URL for the metadata doc, but originally we just used it as the one well-known URL (we left the behavior of the service root undefined, figuring that services would want to do things with that).
Turns out we were wrong, and people wanted `/` to give a standardized service doc that has a ref to the metadata doc. So now we have both (for back compat).
Mike Schinkel
We found there were good reasons to not put the metadata at root. Similarly, we didn’t want to corrupt the API surface (since we were making an API for arbitrary services). So we named the metadata `/$metadata` (by default convention – a link from the root provides the real value).
How does using a fixed location square with Joe Gregorio's concerns in his essay about "No Fishing?"
OData is pretty heavy, and seems to be promoted by a single vendor (Microsoft), right?If yes, not sure I could envision it becoming the standard we all finally agree on?
I am definitely in favor of using metadata to drive a web API. It can make the system open for client extension, while still leaving the server in control of what’s allowed and how that is done.
Cool. Would you be in favor of something that was not OData?
There are just a ton of details involved in getting it right – many of which we learned by getting them wrong and then having to fix it while maintaining backwards compatibility even on the metadata document.
I can surely appreciate that.I wonder if the starting bar isn't higher for Microsoft then it would be in general simply because of the expectation that Microsite much adhere to?OTOH, HTML launched the web in part because it was so simple but it's first version certainly didn't address every need. I wonder if we couldn't get most everyone to agree with a small starting point, and then work from there?
Mike Schinkel
> On Apr 9, 2012, at 6:29 PM, mca wrote:
> IMO, a better approach if too use "meta" affordance.
Not familiar with that. Can you define "meta" affordance for the uninitiated?
-Mike
Jørn Wildt
> offering them via an API-wide resource that could by-convention be
> retrieved from the API root:
Seems like one of us missed something. You are describing the same thing as
I was trying to: download a complete service document for all globally
available URL templates at the beginning of service interaction.
I did although not say "by convention" and it also leaves room for link
relations in responses for links that are relative to some resource.
From my POV a service document contains templates/forms for
searching/look-up of resources that cannot be discovered otherwise. It also
contains links to forms the describe how to create new "root" resources that
"stands by themself" (as opposed to be added as sub-resources to some
existing resource). The rest of the URLs are discovered from there on.
Eran Hammer has already done a lot of work on the area of discovery/service
documents:
- http://hueniverse.com/discovery/
- In includes XRD: http://docs.oasis-open.org/xri/xrd/v1.0/xrd-1.0.html
Then there is Yadis: http://en.wikipedia.org/wiki/Yadis and .well-known
http://hueniverse.com/2009/11/host-meta-aka-site-meta-and-well-known-uris/
/J�rn
----- Original Message -----
From: Mike Schinkel
To: api-...@googlegroups.com
Sent: Monday, April 09, 2012 11:14 PM
Subject: One Meta URL to Rule Them All?
Changed the subject to recognize the change in topic.
Mike Schinkel
Rather than encoding the link templates into each response, what about offering them via an API-wide resource that could by-convention be retrieved from the API root:
Seems like one of us missed something. You are describing the same thing as I was trying to: download a complete service document for all globally available URL templates at the beginning of service interaction.
I will take blame. Seems a lot more has evolved since I was paid lots of attention to REST than I realized.Thanks for clarifying. But honestly, it's nice to learn of these things than to have to wait for them to be developed.
Don Day
Still, my question was about best practice for the case of objects
that can have multiple address schemes, where currently I have:
As a typed member of a collection:
- {groupName}/topics/{resourcename}
- {groupName}/lists/{resourcename}
- {groupName}/jobs/{resourcename}
- {groupName}/images/{resourcename}
And as the "member of a list in a group":
- {groupname}/{listname}/{resourcename}
In a way, a search query that uniquely locates a particular resource
is yet another address scheme, so I guess the problem is not all that
unusual. It's just that I never see it discussed much.
What I think I'm logically missing is a required first keyword of
"groups" or "sets" since those are apparent collections as well. I've
also decided to revise any singular keywords to the plural form for
both collections and singletons.
My other concern is in trying to use this RESTful URL scheme for the
wiki-like application as well. My scheme is bookmarkable, but just not
as short as most blog URLs because of the need to distinguish the
multiple address paths. Users of web apps will never see the URLs, but
users of the wiki application will see them. I'm trying to do what is
right for both cases.
--
Don
mca
These are all examples of hypermedia controls (affordances) that might
represent the API metadata link for a service implementation.
<api-metadata href=".,," />
or
{"api-metadata" {href:"..."}}
or
<atom:link rel="api-metadata" href="..." />
etc.
Whatever design is used for the messages, documentation for that
design would include text that reads something like:
"The "api-metadata" element MUST appear in every response
representation. The "href" attribute points to the API metadata
resource for returned representation. Clients SHOULD use HTTP.GET to
request a representation of the API metadata resource."
This allows clients to always recognize, parse, and activate the
control (the metadata affordance) whenever needed. It also allows
server implementations to select their own URL for returning the API
metadata instead of forcing all implementations (in all languages for
all time) using the exact same URL.
There are many possible variations, this is just an example.
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me