user-agent vs server content negotiation

[Rémi Arnaud]

This is a continuation of the discussion about GET /texture/A vs GET /texture/A.png
I have been doing some more research on this subject based on the information Chad and Ewen provided.

As pointed out /texture/A and /texture/A.png, even though they may return the same thing based on the protocol we are currently specifying, are not referencing the same resource/asset. /texture/A is referenced as generic resource, and its protocol referenced as server content negotiation (using Accept http header http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html, and returning Vary header for caching), and /texture/A.png is referenced as specific resource, and its protocol referenced as user-agent content negotiation.

The main idea behind server content negotiation is that the client uses "accept:" header listing a set of types associated with a q=[0..1] floating point coefficient that the server should use to prioritize which format to send back to the client. The server need to send back the specific resource with the "vary:" parameter so caching can be enabled - the same (generic) URI represents different resources to be cached separately. (Note: I am a bit worried about all the issues/bug that I read about this, and how complex it is for a client to use this parameter. Caching is fundamental for performance in a 3D application, and it is not clear to me how the various browsers or libraries implement correctly this)

"published content (and its various representations) needs to be discoverable on the Web" - the recommendations are for both the generic URI and specific URI to exists, and for the generic URI to provide meta information about the resource and its alternate representations (list the specific URI).

The fundamental assumption from generic URI is that the returned content is equivalent (although at different level of quality). Ewen proposed the models to point to generic resource instead of specific resource, allowing the client to decide which formats the resource will be requested using the 'accept:' header.
"A simpler way to think about it is: an "object" refers to a "texture" rather than the "object" referring to a "jpeg".  The main win is that it's easy for clients to select the representation that best meets their needs."

Note: AFAICT, requesting for a specific resource /texture/A.png, or using 'accept: image/jpeg;q=1' are equivalent, except for the name of the resource returned, and the caching issues.

The main problem I see is that the fundamental assumption that image formats are equivalent is not true. For example, using a jpeg encoding for a Z-map would not work at all, but .png does work. Another example is game developer artists are very particular about the end result, and will spend a good amount of time tweaking the encoder and its parameters, and are in general disastified with automatic content format conversion.

So, I have a hard time to see how the model can simply refer to a generic resource, and let the client decide what format to use. In order for the model to refer to generic resources, the model need to also store the 'accept:' or other header specific information that we could/would design in order for the client to be able to compose the right http headers the the server. On the other hand if the model is referencing the specific resource (i.e. A.png), the client could deduct what format to ask for based on the extension and the specifics of the hardware/infrastructure it is running on. The exact protocol that the client is yet to be defined. Requesting /texture/A.png, or requesting /texture/A with 'accept: png' or even requesting /texture/A.jpg with 'accept: png' would produce the same result with 3 different resource names - and only the REST-3D specification can tell which of those the client should use, and what the expected results are from the server.

The specific URI need to exists regardless. This is the unique URI referencing a specific resource, so we can provide PUT/POST API to those resources directly, and enable content uploading through the REST-3D API. I have not seen recommendations/examples on how to use 'generic URI' to post content, and IMHO it does not seem feasible as A.png and A.jpg are two different resources that require their own unique URI.

The use of generic URI is to enable alternate content representation based on content formatting, quality but also versioning, timestamps and any other parameters. In fact, there could/should be several representation of the same image, all encoded in jpeg, with different sizes or compression parameters. So 'accept: jpg' is not enough to specify which of the representation the client want access to. Whether to use additional X-Accept-* type headers, URL 'path' encoding (/texture/1024x1020/A.png), URL query parameters (/texture/A.png?size=1024x1024) or naming conventions (/texture/A_1024x1024.png) or other means is what the REST-3D API need to define.

My current thinking is to use/specify URL 'path' encoding and avoid X-Accept-* headers as both are equivalent effort in the client code, but having specific URI simplify caching and avoid all the complex caching implementation (bugs?) reported everywhere. Personally I do not feel comfortable specifying or relying on client/server caching behavior on X-Accept-*, Vary: header, since this is not a code we can control, but specifying how to the client can compose specific URI is something that we can control.

Generic URI also should provide a list of alternate content. The generic URI would normally return a set of meta-data about the resource and its alternate representation. I understand that we should provide shortcut for a class of applications (webGL viewer..) to avoid the back and forth with the server negotiating which exact specific resource to return, using either the right set of 'Accept' and 'X-Accept' header, or/and a specific URI. In other words, asking for /texture/A with 'accept: image/png X-accept: size=1024x1024' or asking for /texture/1024x1024/A.png would enable the client to get exactly what it need without going through a discovery dialog.

But, asking for the Generic URI with 'xml','html','json' should return information about the resource, including the links to its alternate content representations. One way to do that that seem to be quite popular is to use the Atom protocol. From my understanding this protocol is recognized by many web crawlers and other applications, has well defined semantic, and used by CMS (Content Magement Systems) already. RFC 4387 [dec 2005] is AFAICS the current specification (http://www.ietf.org/rfc/rfc4287)

So, asking for /texture/A (with accept: text/xml) would return something like that:

<atom:entry xmlns:atom="http://www.w3.org/2005/Atom">
  <atom:title>A</atom:title>
  <atom:updated>2012-04-31....</atom:updated>
  <atom:content type="image/png" src="http://.../texture/A.png" />
  <atom:link rel="alternate" type="image/jpeg" href="http://..texture/A.jpg"/>
..


and so forth. other parameters should be specified such as versioning information with links to previous versions, what transforms are possible (resize..).

On thing that seems important is to know which of the content representation is the 'source' for all the others. It is really important in a content pipeline to remember what was the source content, and if possible how the source got converted into the target specific representation. For example, the source could be a photoshop .psd file, which would not be useful to a 'webgl viewer' but very important for an artist that want to edit/update the image.


Apologies for the long post, but somehow I feel that this issue of generic vs specific resource is at the heart of the REST-3D design, as it implies what the API/protocol will look like, how resources will be manged/updated, and what is specific about 3D content that generic REST API design need to be adapted to.

Best Regards,
-- Remi

[Chad Austin]

On Sun, Jun 10, 2012 at 1:38 PM, Rémi Arnaud <re...@acm.org> wrote:

[snip]

The main idea behind server content negotiation is that the client uses "accept:" header listing a set of types associated with a q=[0..1] floating point coefficient that the server should use to prioritize which format to send back to the client. The server need to send back the specific resource with the "vary:" parameter so caching can be enabled - the same (generic) URI represents different resources to be cached separately. (Note: I am a bit worried about all the issues/bug that I read about this, and how complex it is for a client to use this parameter. Caching is fundamental for performance in a 3D application, and it is not clear to me how the various browsers or libraries implement correctly this)

I don't know how well browsers implement Vary.  I do know that Varnish implements support for it https://www.varnish-cache.org/trac/wiki/ArchitectureVary and that nginx does not http://wiki.nginx.org/HttpProxyModule if you're using it as a caching reverse proxy.

nginx does not handle "Vary" headers when caching.

As a side note, does anyone know if clients ever actually use quality values other than 1.0 or 0.0?

"published content (and its various representations) needs to be discoverable on the Web" - the recommendations are for both the generic URI and specific URI to exists, and for the generic URI to provide meta information about the resource and its alternate representations (list the specific URI).

The fundamental assumption from generic URI is that the returned content is equivalent (although at different level of quality). Ewen proposed the models to point to generic resource instead of specific resource, allowing the client to decide which formats the resource will be requested using the 'accept:' header.
"A simpler way to think about it is: an "object" refers to a "texture" rather than the "object" referring to a "jpeg".  The main win is that it's easy for clients to select the representation that best meets their needs."

Note: AFAICT, requesting for a specific resource /texture/A.png, or using 'accept: image/jpeg;q=1' are equivalent, except for the name of the resource returned, and the caching issues.

The name of the resource is a key difference because it means that links to a texture can be identical on all platforms but the data served for that texture could differ for each platform.

The main problem I see is that the fundamental assumption that image formats are equivalent is not true. For example, using a jpeg encoding for a Z-map would not work at all, but .png does work. Another example is game developer artists are very particular about the end result, and will spend a good amount of time tweaking the encoder and its parameters, and are in general disastified with automatic content format conversion.

I see.  Isn't that an art pipeline issue, however?  I would suggest that content type negotiation (HTTP Accept) and automatic format conversion are orthogonal concepts.  Say that clients pass:

Accept: image/x-dxtc;q=1.0,image/png;q=0.8,image/jpeg;q=0.4

The server doesn't have to automatically encode the texture to DXTC just because it's preferred.  It could only return DXTC if the artist has manually compressed, verified, and uploaded the DXTC version of the texture.

The specific URI need to exists regardless. This is the unique URI referencing a specific resource, so we can provide PUT/POST API to those resources directly, and enable content uploading through the REST-3D API. I have not seen recommendations/examples on how to use 'generic URI' to post content, and IMHO it does not seem feasible as A.png and A.jpg are two different resources that require their own unique URI.

Agreed.  I am okay with assigning specific representations their own URLs.  Your argument regarding uploading is sound.  (Though it does not apply to our case because our RESTful 3D URLs are fed from a rather separate user-generated content system.)

The use of generic URI is to enable alternate content representation based on content formatting, quality but also versioning, timestamps and any other parameters. In fact, there could/should be several representation of the same image, all encoded in jpeg, with different sizes or compression parameters. So 'accept: jpg' is not enough to specify which of the representation the client want access to. Whether to use additional X-Accept-* type headers, URL 'path' encoding (/texture/1024x1020/A.png), URL query parameters (/texture/A.png?size=1024x1024) or naming conventions (/texture/A_1024x1024.png) or other means is what the REST-3D API need to define.

My current thinking is to use/specify URL 'path' encoding and avoid X-Accept-* headers as both are equivalent effort in the client code, but having specific URI simplify caching and avoid all the complex caching implementation (bugs?) reported everywhere. Personally I do not feel comfortable specifying or relying on client/server caching behavior on X-Accept-*, Vary: header, since this is not a code we can control, but specifying how to the client can compose specific URI is something that we can control.

Are you suggesting that clients, when fetching texture data, should append, for example, ".png" to the referenced texture URL?  That seems at odds with the goal of automatic discovery.

One benefit of using content negotiation is that it gives the server some flexibility.  Say that you have a WebGL viewer and an iOS viewer, and millions of users use each.  WebGL prefers DXTC but can handle PNG and JPEG.  iOS prefers PVRTC but can also handle PNG and JPEG.  Say that source data is always PNG or JPEG.

If the server is under heavy load, it could choose to avoid transcoding costs by temporarily delivering source data to clients.  This would result in a slightly degraded but usable customer experience.  When the server's load issues have been resolved, it would resume delivery of DXTC and PVRTC-compressed textures.

 

Generic URI also should provide a list of alternate content. The generic URI would normally return a set of meta-data about the resource and its alternate representation. I understand that we should provide shortcut for a class of applications (webGL viewer..) to avoid the back and forth with the server negotiating which exact specific resource to return, using either the right set of 'Accept' and 'X-Accept' header, or/and a specific URI. In other words, asking for /texture/A with 'accept: image/png X-accept: size=1024x1024' or asking for /texture/1024x1024/A.png would enable the client to get exactly what it need without going through a discovery dialog.

But, asking for the Generic URI with 'xml','html','json' should return information about the resource, including the links to its alternate content representations. One way to do that that seem to be quite popular is to use the Atom protocol. From my understanding this protocol is recognized by many web crawlers and other applications, has well defined semantic, and used by CMS (Content Magement Systems) already. RFC 4387 [dec 2005] is AFAICS the current specification (http://www.ietf.org/rfc/rfc4287)

[snip examples] 

I have no opinion regarding enumeration of supported content types.

On thing that seems important is to know which of the content representation is the 'source' for all the others. It is really important in a content pipeline to remember what was the source content, and if possible how the source got converted into the target specific representation. For example, the source could be a photoshop .psd file, which would not be useful to a 'webgl viewer' but very important for an artist that want to edit/update the image.

That seems reasonable to me.  I have no opinion as of yet.

Apologies for the long post, but somehow I feel that this issue of generic vs specific resource is at the heart of the REST-3D design, as it implies what the API/protocol will look like, how resources will be manged/updated, and what is specific about 3D content that generic REST API design need to be adapted to.

Indeed.  Thanks for your thinking!

Cheers,

Chad