Forms, headers and discoverablity

[Jørn Wildt]

How does forms (in HTML and in general) handle headers like for instance ETag? As I see it they simply don't - is that right? Should they?

In HTTP we can use the ETag header to avoid concurrent updates of the same resource. To do so clients should recognize ETag headers returned from a specific resource - and supply the same ETag header later on when PUTing or POSTing to the same resource.

What if we want to use ETag (and similar headers) in HTML forms? As I can see it there is no way to do so in general since an HTML form may refer to other resource URLs that handle the forms request.

Example: consider a system for managing events. At the URL /some-event we get back an HTML form for cancelling the event. But the the cancellation URL is /some-resource/cancellations. In this case the client (browser) has no way to know that an ETag returned from /some-event is to be supplied at /some-event/cancellations.

Usually this is not an issue since the equivalent to an ETag value is supplied as a hidden value in the form.

But does this mean that the whole ETag concept is broken (or simply unusable) with respect to forms? And if we assume that "hypermedia is the engine of application state" in REST - is ETags then broken/unusable in HTML and similar formats since we cannot describe HTTP headers in forms?

The exact same problem is valid for APIs too when using hypermedia formats that supports "forms" or "actions" rather than only links.

If one accepts the idea of using forms/actions in hypermedia APIs ... should the underlying hypermedia formats then support some sort of header management? It could for instance be possible to describe headers+values that should be passed back to the server in the same way as some formats allow hidden values?

/Jørn

[mca]

essentially, you're asking how a FORMAT handles a PROTOCOL. an interesting Q, but one that assumes that the format should be protocol-aware, if not protocol-specific.

there are lots of aspects of HTTP that HTML doesn't handle (PUT/DELETE/PATCH, conneg, lang-neg, caching, etc.). none of those concepts are "broken" just because a format doesn't handle them inline.

"should the underlying hypermedia formats then support some sort of header mgmt?" -- this is a DESIGN question and the answer is "sure, if that's what you want..."

for example UBER exposes a number of protocol-level items. some of them directly (accepts and sends properties, for example) and some of them indirectly (actions property). again, these are design issues.

IMO, it doesn't make sense to *duplicate* all protocol-level items in a format. but, since this is about design, i suspect my POV is not universally shared.

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[sgoto]

Oh Jorn, you completely beat me to asking this question: I've been meaning to ask this question for some time but never took the time!

I'm glad you are asking this question, I'm super interested in it too.

Just a bit of context: I'm looking at APIs from the perspective of a crawler (here is how other crawlers look at this problem too). I need hypermedia not because I think it is a cute idea, but because I don't have any way out (we *literally* tried everything else -- starting from "manually reading" one-off APIs to giving up and having to come up with something more scalable).

Here are some of the problems I ran into while crawling traditional (*and* RMM L3 APIs), more specifically:

- the discovery of APIs problem

- the off-band API management problem

We don't know what's the solution yet, hence I'm glad you kick off the conversation :)

However, here is one interesting direction we took a year ago that is looking pretty interesting: 

https://developers.google.com/structured-data/slsb-overview

We started crawling HTML5 <forms> that are "semantically applicable" to crawlers, in this specific case http://schema.org/SearchAction.

For example, a "regular HTML5 form" isn't very useful to us, BUT, if you add SEMANTICS to it, it starts lighting up:

<div itemscope itemtype="http://schema.org/WebSite">
  <meta itemprop="url" content="https://www.example.com/"/>
  <form itemprop="potentialAction" itemscope itemtype="http://schema.org/SearchAction">
    <meta itemprop="target" content="https://query.example.com/search?q={search_term_string}"/>
    <input itemprop="query-input" type="text" name="search_term_string" required/>
    <input type="submit"/>
  </form>
</div>

Here is why I like HTML and <forms>:

- with microdata, you can express both the state of the resource (e.g. in the example above it is a WebSite with a url) as well as transitions (e.g. one can SearchAction via submitting a form)

- they de-couple the structural semantics with the application semantics  (e.g. one could use microformats or opengraph on top of HTML forms rather than schema.org Actions)

- they are discoverable from the human-web (i.e. one can find them eventually automatically via following links, they get all of the page-rank kind of discoverability/rankability)

- there is no off-band API KEY set up: <FORM>s are PUBLICALLY accessible and there is NO need to register for API-KEYS (see API management challenge here)

There are PLENTY of challenges crawling <forms>, but let me not go over these before we agree on a more basic ground:

Is there anyone else producing or consuming APIs in HTML + microdata?

/Jørn

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

--

f u cn rd ths u cn b a gd prgmr !

[sgoto]

On Tue, Jul 21, 2015 at 8:04 AM, mca <m...@amundsen.com> wrote:

essentially, you're asking how a FORMAT handles a PROTOCOL. an interesting Q, but one that assumes that the format should be protocol-aware, if not protocol-specific.

Why are forms protocol-aware? Wouldn't something like the following possibly existing?

<form action="mailto:go...@google.com"></form>

<form action="ftp://ftp.sgo.to" method="UPLOAD"></form>

<form action="tel://650-443-3432"></form>

? Doesn't it seem like a natural extension of HTML forms?

 

there are lots of aspects of HTTP that HTML doesn't handle (PUT/DELETE/PATCH, conneg, lang-neg, caching, etc.). none of those concepts are "broken" just because a format doesn't handle them inline.

Right, but HTML can change/evolve right?

Or, one could write a client (new browser?) for APIs that uses this "new form of HTML that allows for things like PUT/DELETE/PATCH"?

 

"should the underlying hypermedia formats then support some sort of header mgmt?" -- this is a DESIGN question and the answer is "sure, if that's what you want..."

I'm not going to get to the specific merit of ETags (because I don't think I fully understand it).

However, I think we could borrow a few interesting ideas from things like UBER and headers like "accepting".

 

<form accepting="application/vnd.amundsen-uber+json"></form>

To give forms the ability to do further content negotiation.

[mca]

"Why are forms protocol-aware?" 

my comment was about FORMAT, not FORM. 

but to follow your line...

the protocol-aware design of HTML forms is what you show (that you can use different schemes in HTML forms).

that shows that HTML forms are not protocol-specific.

to go another way....

- Collection+JSON is protocol *specific* (the docs explain all actions via HTTP)

- UBER is protocol *aware* (the docs explain all actions as abstract keywords that are mapped to multiple protocols)

and, yes, theoretically, HTML can evolve and nothing i have stated says that HTML should not evolve.

- adding methods would be one option (there needs to be a backward-compatible solution)

- adding conneg info (like UBER does) is an example (note: HTML.FORM@enctype does this for write operations now)

finally, none of this is about ETag which was the point of Jorn's question. my regrets for hijacking the thread ;)

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Jørn Wildt]

> finally, none of this is about ETag which was the point of Jorn's question. my regrets for hijacking the thread ;)

I don't think it was hijacking - my point was not specifically about ETag but more general about the use of headers in forms.

> essentially, you're asking how a FORMAT handles a PROTOCOL. an interesting Q, but one that assumes that the format should be protocol-aware, if not protocol-specific.

Yep, I was (am) assuming HTTP based formats such as HTML and the various hypermedia API formats.

> there are lots of aspects of HTTP that HTML doesn't handle (PUT/DELETE/PATCH, conneg, lang-neg, caching, etc.). none of those concepts are "broken" just because a format doesn't handle them inline

Yep, broken is a bit too harsh wording here. Its a design choice :-) Good point, thanks.

> IMO, it doesn't make sense to *duplicate* all protocol-level items in a format but, since this is about design, i suspect my POV is not universally shared.

And, yes, this is probably the essence of my thoughts - design - should general hypermedia aware formats, such as HTML, HAL, Mason, Siren, JS-Collection etc. support headers just like they support "payload" (sending data back as JSON/URL-encoded or similar)? Why don't they? I see some valid use cases but apparently that is not enough for people to request it. Is ETag actually used outside text books :-)

/Jørn

[Jørn Wildt]

Samuel, thanks for the links, that was some interesting reads. But I don't see a direct connection between a discussion on headers in forms versus semantic web crawling ... maybe it was the "discoverability" that triggered it? But I am rather impressed by how such embedded information sneaks into GMail as various interactive widgets. First time I experienced it was when GMail showed live flight information for me inside a flight reservation e-mail.

/Jørn

[sgoto]

On Tue, Jul 21, 2015 at 2:09 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

Samuel, thanks for the links, that was some interesting reads. But I don't see a direct connection between a discussion on headers in forms versus semantic web crawling ... maybe it was the "discoverability" that triggered it?

Possibly.

I'm more generally interested in understand if anyone at all is using HTML (and forms more specifically) as the hypermedia type for their APIs (or an HTML-based). If your thread was specifically about a discussion on headers, I'd be more than happy to fork this thread into another one that suits me better.

[Mike Kelly]

On 21 Jul 2015 18:53, "sgoto" <samue...@gmail.com> wrote:
>
> On Tue, Jul 21, 2015 at 8:04 AM, mca <m...@amundsen.com> wrote:
>>
>> there are lots of aspects of HTTP that HTML doesn't handle (PUT/DELETE/PATCH, conneg, lang-neg, caching, etc.). none of those concepts are "broken" just because a format doesn't handle them inline.
>
>
> Right, but HTML can change/evolve right?
>

iirc, the community asked for this and was offered the opportunity to propose additions to the HTML5 spec adding PUT and DELETE to forms but nobody delivered.

Cheers,
M