embedded resources

199 views
Skip to first unread message

nick.p...@gmail.com

unread,
Feb 13, 2015, 5:55:19 PM2/13/15
to hal-d...@googlegroups.com
I'm struggling a bit with embedded resources.  At first I thought I had a handle on them.  Then I posted on the Mason discussion group and asked whether Mason supported embedded resources.  I was told that the spec didn't defined embedded resources but you could simply embed the resource in the representation.  I asked how I can distinguish between the resource I requested and any embedded resources that were returned.  I was sure the next question I would be asked is why I need to differentiate between a resource and an embedded resource to which I don't have an answer yet.

Since HAL supports embedded resources in the spec I'm hoping someone can give me some insight into why that's needed?  Why couldn't you just embed the embedded resource in the representation itself?

For instance, we could have an order resource which has a relation to a customer resource.  The order could be represented as (please ignore any typos or minor syntax errors):

{
   "number" : 5,
   "date" : "2014-01-03 14:23:20",
   "amount" : 145.65,
   "_links" : {
      "customer" : {"href" : "http://foo.com/customers/56"}
   }
}

or

{
   "number" : 5,
   "date" : "2014-01-03 14:23:20",
   "amount" : 145.65,
   "customer" : {
        "name" : "Acme",
        "id" : 56,
        "address" : "...",
        .
        .
        .
   }
}

Are the two representation roughly the same?  If not, what are the differences?  Why do we need to have embeddings defined in the specification?

Thanks,
Nick

Pete Johanson

unread,
Feb 13, 2015, 6:18:34 PM2/13/15
to hal-d...@googlegroups.com

Embedded resources allow you to establish a clear semantic relationship between two resources and, when using a "smart" client library, have linked vs. embedded be an implementation detail. By doing so, the server is free to change that detail to meet needs for caching, client performance, etc.

-pete

--
You received this message because you are subscribed to the Google Groups "HAL Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hal-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Kijana Woodard

unread,
Feb 13, 2015, 6:20:31 PM2/13/15
to hal-d...@googlegroups.com
See section 8.3 on the hypertext cache pattern.

I find it rather elegant.

Pete Johanson

unread,
Feb 13, 2015, 6:27:47 PM2/13/15
to hal-d...@googlegroups.com

Indeed. Lazy of me not to link to that...

You can see an example smart client demo here: https://angular-hy-res-example.herokuapp.com/

Its my AngularJS hypermedia library allowing the exact same controller code whether the items in the collection are embedded, or just linked.

-pete

nick.p...@gmail.com

unread,
Feb 13, 2015, 6:54:10 PM2/13/15
to hal-d...@googlegroups.com
Thanks for the replies, but it's still a bit unclear.  I've read the section of the HAL spec and while I think I understand it it doesn't seem to help answer my question.  Maybe I'm just too thick.

The only bit of text which I'm not sure is key (from Pete) is: "Embedded resources allow you to establish a clear semantic relationship between two resources"

Does this mean that separate resources which are related should not be embedded (directly) into another resource, like I have done with customer in my second example?  There is some semantics there right?  The object is named "customer".

It sounds like what you're saying is that all related resources should be via links and if you want to embed those resources they should only go in a section for embedded resource and not "in-lined".  I can kind of see that, but was looking for more of a why answer.

Thanks,
Nick

Kijana Woodard

unread,
Feb 13, 2015, 7:17:13 PM2/13/15
to hal-d...@googlegroups.com
Mainly because embedded resources are not part of the resource you requested. They are related.

Take:

   {
     "_links": {
       "self": { "href": "/books/the-way-of-zen" },
       "author": { "href": "/people/alan-watts" }
     },
     "_embedded": {
       "author": {
         "_links": { "self": { "href": "/people/alan-watts" } },
         "name": "Alan Watts",
         "born": "January 6, 1915",
         "died": "November 16, 1973"
       }
     }
   }

You requested the "book resource".
Now you want to display the Author's name.

Without embedded, you have to traverse the link. With embedded, you can grab the name and avoid the network call.

Let's say you wanted the Author's favorite color. That's not in embedded, so you have to traverse anyway. 
Now, the API admins notice a high percentage of consumers traversing for author after getting a book. They research and see that lots of people need favorite color. They add it to the embedded representation and voila, those extra author requests cease without changing the clients.

If you in-line author, it becomes a part of the blog post representation, which may or may not be what you want.
Imagine if you have a Publisher related resource. How much do you want to muddle the book with it's relations?

p.s. 
Is it possible to send a pull request for https://tools.ietf.org/html/draft-kelly-json-hal-06?
It appears that the "self" link in section 8.3 changes from "/books/the-way-of-zen" to "/blog-post" by mistake.

--

Pete Johanson

unread,
Feb 13, 2015, 7:30:04 PM2/13/15
to hal-d...@googlegroups.com

Your customer property implies a semantic relationship, yes. The design of embedding in HAL allows you to reuse the *mechanism* of semantic relationships already used for web linking, which, IMHO, is the real power of the approach.

-pete

nick.p...@gmail.com

unread,
Feb 13, 2015, 9:27:46 PM2/13/15
to hal-d...@googlegroups.com
Thanks for the additional replies.  You know, after I posted I was thinking about it some more and it started to make more sense to me.  I was thinking that if the resource is documented it indicates that it has a related customer resource.  A client would therefore be looking in the links section, I guess, since it's a related resource and wouldn't be looking at the properties of the resource itself.

Which brings me to another question.  What's documented in this hypermedia API.  I know we have the entry point and media types and link relations.  I assume the resources are also documented?


Thanks,
Nick

On Friday, February 13, 2015 at 3:55:19 PM UTC-7, nick.p...@gmail.com wrote:

Chris DaMour

unread,
Feb 13, 2015, 9:50:03 PM2/13/15
to hal-d...@googlegroups.com
yeah resources are often documented in the link rel docs...but I prefer to use the profile link relationship to link to documentation.  every resource has a profile link with an href that identifies what the json represents.  Following the href also gives some docs about what fields you can expect, types, and what relationships may be present.

FYI: a smart HAL client looks in _embedded before it looks in _links.  That's what embedded is all about.

nick.p...@gmail.com

unread,
Feb 13, 2015, 10:52:19 PM2/13/15
to hal-d...@googlegroups.com
Ok, sounds good.  So what are the rules then?  If I have a link in the links section, if I want to embed that resource the embedded resource name must match the link name?  If so, what about the example in the hal specification on github:

{
    "_links": {
        "self": { "href": "/orders" },
        "curies": [{ "name": "ea", "href": "http://example.com/docs/rels/{rel}", "templated": true }],
        "next": { "href": "/orders?page=2" },
        "ea:find": {
            "href": "/orders{?id}",
            "templated": true
        },
        "ea:admin": [{
            "href": "/admins/2",
            "title": "Fred"
        }, {
            "href": "/admins/5",
            "title": "Kate"
        }]
    },
    "currentlyProcessing": 14,
    "shippedToday": 20,
    "_embedded": {
        "ea:order": [{
            "_links": {
                "self": { "href": "/orders/123" },
                "ea:basket": { "href": "/baskets/98712" },
                "ea:customer": { "href": "/customers/7809" }
            },
            "total": 30.00,
            "currency": "USD",
            "status": "shipped"
        }, {
            "_links": {
                "self": { "href": "/orders/124" },
                "ea:basket": { "href": "/baskets/97213" },
                "ea:customer": { "href": "/customers/12369" }
            },
            "total": 20.00,
            "currency": "USD",
            "status": "processing"
        }]
    }
}

I don't see a link with the name "ea:order".

Thanks,
Nick

Chris DaMour

unread,
Feb 13, 2015, 10:56:54 PM2/13/15
to hal-d...@googlegroups.com
by convention, if you always will embed, there's no reason to have the entry in _links.

Also technically if you want to have both you can.  This is very useful if say you didn't embed at first, and clients are written expecting there to be a link (application/json clients..not true hal+json clients) and you decide to start embedding.  Then at the next major version you can break backwards compatibility and just stop including the link.

You see no link very commonly with "page" resources as a page by itself without the items it contains embedded isn't very useful in most cases.  (an example of it being useful as a collection of link is when the linked resources have no hal+json representation and thus can't be embedded..think images).

A really cool effect of embedded just being "pre-fetched" links is that if you profile your clients behaviours your API can actually start embedding stuff after the client as launched and if they are a compliant hal+json client they'll see a performance bump.

--
You received this message because you are subscribed to a topic in the Google Groups "HAL Discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hal-discuss/n-YZ2lr_m_I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hal-discuss...@googlegroups.com.

Matt Clark

unread,
Feb 14, 2015, 1:30:26 PM2/14/15
to hal-d...@googlegroups.com
Except _embedded resources aren't just pre-fetched, the spec allows them to differ from the actual resources.  I think HAL would work better if _embedded resources had to be identical to their linked representations.  

darrel...@gmail.com

unread,
Feb 14, 2015, 2:39:10 PM2/14/15
to hal-d...@googlegroups.com
If embedded representations were required to be equal to complete representations that would defeat the purpose of embedding for me.  But that’s just my opinion.

Darrel

Sent from Surface

--
You received this message because you are subscribed to the Google Groups "HAL Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hal-discuss...@googlegroups.com.

Chris DaMour

unread,
Feb 15, 2015, 12:23:08 PM2/15/15
to hal-d...@googlegroups.com

i thought this originally...but then I ram into cases where a response was coming from a system that didn't have everything needed to fill the full resource...but it had enough to cover the use case of the ux-ui.

I could have hydrated the object by making a db call to the system with the info...but that would slow down everyone just so I could feel good about my responses as an architect.

if I had not read HAL can have partial resources I would have just gone the hydration route and things would have worked but been slower.

but luckily I did...so I thought about it and all I needed was a way to let the client know they should not expect everything to be in the response AND tell them a way to get the full response if they really need it (say the ux changes).

the presence of an x:full link did BOTH of those things really simply. 

I advise clients to check if they have a partial and proceed with caution by probing for value presence before using.  this way if in v2 of the client they need a field only available in the full representation they look for it in the partial first and get the full as a last resort.  this way when I release a new version of the api that has the field in the partial rep they need they get a performance bump without changing their code.  its just like hypertext cache pattern but at the field level

Matt Clark

unread,
Feb 15, 2015, 4:38:43 PM2/15/15
to hal-d...@googlegroups.com
Thats' a great example of a good use of _embedded, but it's not 'caching' in any HTTP-relevant sense of the term.  I'd be all for a _partial section or for a _partial special property of embedded resources to be put into the spec.  Either that or the spec should mandate that _embedded resources must never be treated as the full resource representation.  The language as it stands means ambiguity on all sides.

Mike Kelly

unread,
Feb 15, 2015, 4:45:04 PM2/15/15
to hal-d...@googlegroups.com


On 15 Feb 2015 21:38, "Matt Clark" <matt.w...@gmail.com> wrote:
>
>  Either that or the spec should mandate that _embedded resources must never be treated as the full resource representation.

It already does :)

" Embedded Resources MAY be a full, partial, or inconsistent version of the representation served from the target URI."

Chris DaMour

unread,
Feb 15, 2015, 5:11:41 PM2/15/15
to hal-d...@googlegroups.com

not quite following or how we got on caching embedded resources.

I'm guessing you are alluding to caching embedded resources individually as cache entries to be retrieved by the key of a url from cache.  if you wrote a client that put embedded resources in a cache so they could be retrieved from cache by their self url in this way..well then your client is already caching a layer above http cache..so just have it know about the semantics of not caching a resource that claims to be a  partial resource. maybe you're suggesting a cross implementation way to identify a partial resource.  that sounds like a good candidate for a spec on top of hal.  maybe even better would be some semantics that flagged a resource (embedded or not) as non-cacheable

the hypertext cache pattern isn't designed for caching by a url key...its designed for caching by a relationship key and only in the context of a single response. 

because of this when a service offers up a resource with embedded resources..partial or not...it must take into consideration all resources in the response when determining http cache headers as the http caching happens at the whole response level.

if you want to control headers on individual resources the prefetch link relationship would get you closer.  personally I like seeing everything I need for my UI in one big JSON clob.  easier to just paste it's self href in a browser

fyi I only use http as a protocol example here.  nothing discussed requires http...other protocols could have other cache semantics that are comparable.

--

Chris DaMour

unread,
Feb 15, 2015, 5:18:23 PM2/15/15
to hal-d...@googlegroups.com
mike, ever seen a use case for inconsistent?  is there even a definition of inconsistent?

--

Mike Kelly

unread,
Feb 15, 2015, 5:30:17 PM2/15/15
to hal-d...@googlegroups.com

No, that wording was just supposed to reaffirm that HAL itself doesn't give you any kind of integrity guarantees for embedded info

You received this message because you are subscribed to the Google Groups "HAL Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hal-discuss...@googlegroups.com.

Andrew Kuklewicz

unread,
Feb 15, 2015, 5:43:55 PM2/15/15
to hal-d...@googlegroups.com
An example of inconsistent that comes to mind is when the embedded resource has been updated since being embedded.

- Andrew

Chris DaMour

unread,
Feb 15, 2015, 5:45:59 PM2/15/15
to hal-d...@googlegroups.com
by that definition EVERY resource ever is probably inconsistent! :)

Mike Kelly

unread,
Feb 15, 2015, 6:02:27 PM2/15/15
to hal-d...@googlegroups.com

Yes, that probability is provided in each response by the TTL of the Cache-Control header

Merlyn Morgan-Graham

unread,
Feb 16, 2015, 12:11:35 AM2/16/15
to hal-d...@googlegroups.com
I've seen this asked in a different thread also - can "inconsistent" also mean "it may include things you won't find when you traverse the link"? I think this could be useful for computed, display-only, or meta properties (like "why this link is here").

A sample use case is summary information in a search result that might not appear in the exact same form (or might not appear at all) when the expanded resource is fetched. Count of items in a returned "collection", or display data for why a result was returned, or what type of result it is ("Video", "Person", etc).

If this can't be done in _embedded then it is hard to do at all. Either you have to match up properties in the main JSON message with entries in _links, or you have to create a "search result" resource to represent those properties (which might be fine, except that search result entries tend not to be individually stored on the server).

On the other hand, I've already seen where this might confuse some clients. At least a few HAL wrapper libraries assume _embedded will contain quite similar or full objects, and assume you don't need to fetch a document if it appears in embedded.

Matt Clark

unread,
Feb 16, 2015, 3:13:47 AM2/16/15
to hal-d...@googlegroups.com
But... the spec wording means there's no way to tell if an embedded resource is partial or inconsistent or full, so in practice there are as many ways of deciding this as there are HAL API designers.  And the HTTP headers aren't available for embedded resources so we can't use that mechanism either.  I've seen (and done myself) the adding of an _http property to a resource to deal with this, but it's not exactly elegant!

I'm certainly enjoying the very interesting set of use cases for embedded resources that are being mentioned, but I can see why most clients either punt on the problem completely and leave it up to the users, or are very highly opinionated.  I'll see if I can create a decent list of patterns and some possible implementations later.


On Sunday, 15 February 2015 23:02:27 UTC, Mike Kelly wrote:

Yes, that probability is provided in each response by the TTL of the Cache-Control header

...


On 15 Feb 2015 21:38, "Matt Clark" <matt.w...@gmail.com> wrote:
>
>  Either that or the spec should mandate that _embedded resources must never be treated as the full resource representation.

Matt Clark

unread,
Feb 16, 2015, 3:26:07 AM2/16/15
to hal-d...@googlegroups.com
That's certainly logical, but I'm not sure it's any more logical than the exact opposite position, that the HTTP cache control headers should refer _only_ to the requested resource, not any embedded ones.  They're definitely not the same resources, and HAL is pretty strong on 'resource' having roughly the same meaning in HAL as it does in HTTP/HTML.  One for Mike to give an opinion on probably.

On Sunday, 15 February 2015 22:11:41 UTC, Chris DaMour wrote:

... 

nick.p...@gmail.com

unread,
Feb 16, 2015, 7:16:11 PM2/16/15
to hal-d...@googlegroups.com
Even though I have done zero in terms of implementation I would agree with you.  If embedded resources had to be the full representation it would seem they'd be much less useful.  Allowing them to be partial lets the server decide what to send.  If the server thinks it can satisfy some large percentage of the clients by only providing some subset of the representation that would seem like a good thing to do.  If the rest of the clients need properties which are not part of the representation they can ask for the full representation.

Thanks,
Nick

darrel...@gmail.com

unread,
Feb 18, 2015, 10:35:01 AM2/18/15
to hal-d...@googlegroups.com
Matt,

A media type cannot redefine HTTP.  Cache control headers, as defined by HTTP apply to the representation, in its entirety. 

This issue is mitigated by the fact that it is rarely a good idea to embed a short lived resource into a long lived one.  When embedding long lived resources into a short lived one, the cache control header will expire for the main resource before the embedded ones, limiting the chance that clients may act on stale data.

In the rare case where a short lived resource is embedded into a long lived resource, then I would recommend the cache control header reflecting the life of that short lived embedded resource.

Darrel

Sent from Surface

--
You received this message because you are subscribed to the Google Groups "HAL Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hal-discuss...@googlegroups.com.

Mike Kelly

unread,
Feb 18, 2015, 10:56:56 AM2/18/15
to hal-d...@googlegroups.com
Yeah, this is a very common issue with composite resources - you have to pick the lowest common denominator if you want to ensure an appropriate cache freshness

Pete Johanson

unread,
Feb 18, 2015, 2:11:17 PM2/18/15
to hal-d...@googlegroups.com

For an internal API project at work, we used a combination of embedded resources + Linked Cache Invalidation (LCI) - https://www.mnot.net/blog/2011/05/27/lci

Very effectively. LCI isn't fully standardized or implemented, so the client side we had a bit more to implement ourselves, but it ended up working nicely. I wish LCI was adopted more generally.

-pete

Mike Kelly

unread,
Feb 18, 2015, 3:36:52 PM2/18/15
to hal-d...@googlegroups.com
Awesome! Good to hear about more LCI implementations!

Matt Clark

unread,
Feb 18, 2015, 5:13:47 PM2/18/15
to hal-d...@googlegroups.com
Well that's true, but they apply to the representation of the requested resource, not any other resources that may be returned.  Any embedded resources aren't part of the resource requested (as testified by their self hrefs which clearly distinguish them from the requested resource), so shouldn't factor into any caching in the first place.  The fact that HTTP cache's don't generally understand HAL should be dealt with by servers setting appropriate weak entity refs.  They need to do this anyway to handle property ordering issues, but they clearly must exclude any embedded resources from eref calculations.

darrel...@gmail.com

unread,
Feb 18, 2015, 7:16:52 PM2/18/15
to hal-d...@googlegroups.com
Matt,

Let me preface this with the disclaimer that I'm not sure it is worth debating this unless it is over beer. 😊  However, just in case this different perspective helps someone else, I’m going state my opinion.

When the decision is made to embed a full/partial/inconsistent representation of resource B into the HAL representation of resource A, the information from resource B that is now included in the representation of A means that it is now part of resource A.  It also exists in resource B.

Let me make an assertion:  You can't retrieve a representation of a resource that contains information that does not belong to that resource.  By definition, a representation is simply a physical manifestation of the resource concept. A media type cannot change that architectural characteristic.

HAL has a convention that allows you to identify what parts of a resource representation _also_ exist as their own independent resource.  It does not redefine the concept of resource and representation to allow parts of the representation to not belong to the resource.

In case my ability to be precise with abstract language has failed me, let me give a concrete example

GET /order
{
  “OrderNo” : “1001”,
  “Description” : “Piece of pecan pie”,
  _embedded : [
                              “customer”: {
  _links : {
“self” : { href=”/customer/75”} 
               }
  “name” : “Bob Brown”
}
     ]
}

In this example the property “name” is part of the resource “/Order”.  It is probably also part of the resource “/customer/75”. 


For the majority of discussions, the distinction between what you are saying and what I am saying does not matter.  However when we start talking about caching and Etags, where identity really does matter, then this distinction does make a difference.

Darrel



Chris DaMour

unread,
Feb 18, 2015, 10:56:41 PM2/18/15
to hal-d...@googlegroups.com

that is a rather interesting way of thinking about it.  embedded meaning its also a resource you can retrieve.  I like that quite bit!  gonna steal that one for my next explanation during hyper onboarding.

You received this message because you are subscribed to a topic in the Google Groups "HAL Discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hal-discuss/n-YZ2lr_m_I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hal-discuss...@googlegroups.com.

Matt Clark

unread,
Feb 19, 2015, 3:43:33 AM2/19/15
to hal-d...@googlegroups.com
Beer sounds like a good plan!  I have the feeling that this should be a question that has an authoritative answer.  I can totally see your reasoning, but in the case of weak etags I'd still feel that two hal docs with the same properties, one with an embedded resource and the other with only a link to the same resource, are semantically equivalent and should therefore have the same weak etag.  Googling around doesn't help much, and your way is arguably safer.
Reply all
Reply to author
Forward
0 new messages