phtal media type

[jmon...@ms3-inc.com]

Thanks for the add @mike.

I just wanted to bring this hypermedia type that I've been working on to any related groups and get some opinions/feedback, and this is probably the most appropriate group. Started work on this about a year ago and just recently gave myself the time to start working on an internet draft. I think it's at a point where it actually makes sense so I wanted to share. Jeff actually suggested I posted here a few months back.

The work in progress draft as html is here https://phtal-org.github.io/spec/draft-ietf-phtal.html and the repository for it is here https://github.com/phtal-org/spec

Let me know if you have any questions or comments!

[mca]

Jose:

thanks for posting to the list.

phtal def looks interesting. do you have any running code (or pseudo-code) available for phtal yet? 

you say phtal is meant to "extend generic hypermedia types" -- not quite sure what that means. do you have any examples?

also, i am guessing that phtal is a response format (like HTML), but not meant to be a request format (like form-urlencoded), is that right? 

finally, do i understand correctly that your "out-of-band" elements are for designtime description/definition formats (e.g. OpenAPI, ???) and your "in-band" elements are for runtime responses?

looking forward to your replies and to seeing on phtal.

cheers.

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund

--
You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-we...@googlegroups.com.
To post to this group, send email to hyperme...@googlegroups.com.
Visit this group at https://groups.google.com/group/hypermedia-web.
For more options, visit https://groups.google.com/d/optout.

[jmon...@ms3-inc.com]

Hey Mike, glad to have your feedback.

No code or pseudo code at the moment. Interestingly enough I started by working on some proof of concept implementations before having the formats fleshed out, started running into a lot of questions marks so I decided to work on the definitions first. I think now I more or less have a clear image of what it looks like and how it should behave so I might get back to a POC soon.

"extend generic hypermedia types"  So the wording there is now evidently off, initially phtal was sort of an extension to hal+json, that's why the "extends " but eventually turned into something else. So it's more of "takes some ideas from hypermedia types like hal or cj but adds some stuff" Throughout the I-D I provide examples of what would be pthal+json and phtal+xml, but here is a composed phtal+json example.

{
  "name": [
    {
      "use": "official",
      "family": "Chalmers",
      "given": [
        "Peter",
        "James"
      ]
    }
  ],
  "_links": {
    "http://registry.phtal.org/fhir/rel/bookAppointmentService": {
      "href": "http://fhir.myclinic.com/Appointment/",
      "instructions": {
        "http": {
          "method": "POST",
          "consumes": "application/phtal+json;profile=\"http://registry.phtal.com/fhir/profile/appointment\", application/phtal+xml;profile=\"http://registry.phtal.com/fhir/profile/appointment\"",
          "produces": "application/phtal+json;profile=\"http://registry.phtal.com/fhir/profile/operationOutcome\"",
          "isContentRequired": true,
          "security": {
            "type": "apiKey",
            "name": "auth-api-key",
            "in": "header"
          },
          "parameters": [{
            "name": "_pretty",
            "in": "query",
            "description": "Ask for a pretty printed response for human convenience",
            "schema": {
              "type": "boolean"
            }
          },
          {
            "$ref": "http://openapi.myclinic.com/fhir/parameter/operation#_elements"
          }]
        }
      }
    }
  }
}

About request/response formats phtal is designed to be both. Like regular "restful" APIs currently produce and consume plain app/json or app/xml they would now consume and produce app/phtal+json;profile="some-uri" or app/phtal+xml;profile="some-uri" which would be perfectly valid without any hypermedia for a request for example.

Yes, out-of-band meaning design time like OpenAPI or RAML, and in-band like HAL or CJ at runtime.

So the main parts are:
First, a profile format that's less abstract than ALPS but more meaningful than just a schema, in this case an extended OpenAPI Media Type Object.
Second, putting instructions into the representation via some new things like `consume` and `produces` and reusing others from OpenAPI Operation Object like `security` and `parameters`.

Basically bridging the gap between design time and run time by moving elements from the former into the latter. So for their clients, an OpenAPI author would only have to expose the Profile/MediaType Objects and create some new Ext Relation Type objects and nothing else. Everything else is concretely provided at runtime.

 - Jose

[mca]

Jose:

thanks for the quick response.

and replies are inline below...

On Thu, Feb 7, 2019 at 6:22 PM <jmon...@ms3-inc.com> wrote:

Hey Mike, glad to have your feedback.

No code or pseudo code at the moment. Interestingly enough I started by working on some proof of concept implementations before having the formats fleshed out, started running into a lot of questions marks so I decided to work on the definitions first. I think now I more or less have a clear image of what it looks like and how it should behave so I might get back to a POC soon.

POC would be excellent -- for the very reasons you mention. once it gets time to build a parser, etc details emerge. looking forward to what you can do here.

"extend generic hypermedia types"  So the wording there is now evidently off, initially phtal was sort of an extension to hal+json, that's why the "extends " but eventually turned into something else. So it's more of "takes some ideas from hypermedia types like hal or cj but adds some stuff" Throughout the I-D I provide examples of what would be pthal+json and phtal+xml, but here is a composed phtal+json example.

yes, i understand, prose stagnates, concepts drift. no problem.

I'd be intested in seeing a request body marked up in phtal. again, i ask this with an eye to how a service would receive, parse, and then operate on the incoming body is if key interest to me. we all know how form-urlencoded includes both meta and data it (simple name/value pairs), Cj does it in a similar way w/ the data objects. not sure how that works w/ phtal yet.

Yes, out-of-band meaning design time like OpenAPI or RAML, and in-band like HAL or CJ at runtime.

:thumbs-up:

 

So the main parts are:
First, a profile format that's less abstract than ALPS but more meaningful than just a schema, in this case an extended OpenAPI Media Type Object.
Second, putting instructions into the representation via some new things like `consume` and `produces` and reusing others from OpenAPI Operation Object like `security` and `parameters`.

Basically bridging the gap between design time and run time by moving elements from the former into the latter. So for their clients, an OpenAPI author would only have to expose the Profile/MediaType Objects and create some new Ext Relation Type objects and nothing else. Everything else is concretely provided at runtime.

i will admit i'm not up on OpenAPI yet to grok the use of custom type objects but we can deal with that along the way.

thanks again and looking forward to learning more. 

[jmon...@ms3-inc.com]

I work a lot with these design time formats, more RAML than OpenAPI but chose the second because it fit a little better with what I wanted to provide. This should provide a lot of reuse and minimal friction if adopting phtal.

If I understand your question about request markup correctly the answer is that is up to the client informed by the profile what is sent and to the server what's actually done.

Consider the following interaction:

GET clinic.com/
Accept: app/phtal+json

200: OK
Content-Type: app/phtal+json;profile="profile.com/clinic"
{
"clinicName": "best clinic",
"_links": {
 "rels.com/registerPatientService": {
  "href": ...
  "instructions": {"http": {
    "method": "post",
    "consumes": "app/json;profile="profiles.com/fhir-patient";q=0.5,
                app/json;profile="profiles.com/myclinic-patient-form";q=1.0",
    "produces": "app/json;profile="profiles.com/fhir-operationOutcome""
  }}
 }
}
}

profiles.com/myclinic-patient-form could resolve to phtal profile that'd include
description: This represents myclinic's patient registration form in json. "Some other information...."
schema: ...
examples: ...


basically a light weight standalone media type imo. This teaches the client the semantics behind a profiled representation and the schema for composing it. Which would enable it to...


POST ...
Content-Type: app/json;profile="profiles.com/myclinic-patient-form"
Accept: app/json;profile="profiles.com/fhir-operationOutcome"
{
  "patientName": "some name",
  "age": ...,
  "condition": "conditions.com/common-cold"
}

In this scenario the request would not include any hypermedia markup, any URIs needed by the server could be a regular json field in the request to be parsed by the server and then associated with a link. So the body of the request would seemingly be a regular hypermedia-less json or xml request, with the important difference that the media type metadata ie. phtal profile in content-type header, would identify the semantics and schema of the data for validating and parsing. In my experience that regular json or xml request is exactly how most "rest" apis work today, so there would be minimal friction there in adopting phtal.

coninuing the scenario...


201: Created
Content-Type: app/phtal+json;profile="profile.com/fhir-operationOutcome"
{
  "id": "allok",
  "text": {
    "status": "generated"
  },
  "issue": [
    {
      "severity": "information",
      "code": "informational",
      "details": {
        "text": "All OK"
      }
    }
  ],
  "_links": {
    "rels.com/patient": {
      "href": "clinic.com/patient/123",
      "instructions": {"http": {
        "produces": "app/phtal+json;profile="profile.com/fhir-patient""
      }}
    }
  }
}

GET clinic.com/patient/123
Accept: app/phtal+json;profile="profile.com/fhir-patient"

200: OK
Content-Type: app/phtal+json;profile="profile.com/fhir-patient"
{
"patientName": "some name",
"_links": {
 "rels.com/condition": {
  "href": "conditions.com/common-cold",
  "instructions": {"http": {
    "produces": "app/json;profile="profiles.com/condition""
  }}
 }
},
"_methods": { "http": {
  "put": {
    "consumes": "app/json;profile="profiles.com/fhir-patient";q=0.5,
                app/json;profile="profiles.com/myclinic-patient-form";q=1.0",
    "produces": "app/json;profile="profiles.com/fhir-operationOutcome""
  }
}
}
}


In this case the client has two request options for an HTTP PUT. The form we used in the first scenario to create the patient, which the server would probably process the same just specifically to the already existing patient, internally transform it into what ends up being hypermedia enabled format for a subsequent GET.

Or the hypermedia enabled format being presented currently "profiles.com/fhir-patient", I think here it's entirely up to the client and server what might happen, the client can choose to include the links it wants in the phtal format and the server is free to accept as is, transform that information however it sees fit, or reject it if something is not right. Both of those options I think are entirely consistent with HTTP PUT.

I would imagine the second scenario ought to be successful more often when done through code-on-demand, which I hope to add to phtal, because it'd know exactly what rels and values are supported internally.

I may have rambled a bit. Did that make sense?

[mca]

Jose:

thanks for the extended examples.

here's the root of my Q on "request formats":

content-type:app/phtal+json;profile="profiles.com/myclinic-patient-form"

does not seem to be a thing in any of your examples, right?  app.json w/ a profile is your example. i assume i could also use app/x-www-form-urlencoded could also be a thing, right?

secondarily, what (if anything) is at the end of "profiles.com/myclinic-patient-form"? did not see this in the spec, either.

just asking to learn about the common use you expect right now and where diff formats/patterns overlap, cross, etc.

thanks!

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund

[jmon...@ms3-inc.com]

Hey Mike,

That was my mistake for quickly writing the examples, *all* of those in my previous message should be app/phtal+json;profile="" instead of just app/json;profile="". IIRC json doesn't allow a profile param.

Through phtal a server could inform a client that it consumes application/x-www-form-urlencoded, but not in what format since it couldn't put a profile parameter along with it. Perhaps there's a way to move things around to make this possible though... OpenAPI does have schemas for application/x-www-form-urlencoded

What should be at the end of the profile URIs should be what's described and exemplified here https://phtal-org.github.io/spec/draft-ietf-phtal.html#rfc.section.2.1 though I suppose it's not clear that what I'm showing there is an example. But here is the same example:

x-phtal-description: A phtal enabled representation of a FHIR Patient, an agent may follow the links in the representation to other FHIR resources.
schema:
    $ref: "http://hl7.org/fhir/json-schema/Patient"
examples:
  newborn:
    $ref: "http://hl7.org/fhir/patient-example-newborn.json"
  deceased:
    $ref: "http://hl7.org/fhir/patient-example-c.json"

[mca]

Jose:

so, to make sure I'm understanding your approach:

as spec'd right now, when using app/phtal+JSON/XML as a response format

- your request bodies (e.g. for POST, PATCH, PUT) MUST be Content-Type: app/phtal+json/XML;priofile="..."

- your profile URLs MUST point to a document that looks like the one in https://phtal-org.github.io/spec/draft-ietf-phtal.html#rfc.section.2.1 

is that right?

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund

[jmon...@ms3-inc.com]

Yes, that's correct.

[jmon...@ms3-inc.com]

Well no, the server could ask the client to compose a request through a format different than phtal right, just phtal then would have no way to help the client compose that request. So for example a server could send:

{
  "name": [],

  "_links": {
    "http://registry.phtal.org/fhir/rel/bookAppointmentService": {
      "href": "http://fhir.myclinic.com/Appointment/",
      "instructions": {
        "http": {
          "method": "POST",

          "consumes": "application/phtal+json;profile=\"http://registry.phtal.org/fhir/profile/appointment\",

                              application/x-www-form-urlencoded",
          "produces": "application/phtal+json;profile=\"http://registry.phtal.org/fhir/profile/operationOutcome\"",
          "isContentRequired": true
        }
      }
    }
  }
}

So here the client would know how to compose application/phtal+json;profile=\"http://registry.phtal.org/fhir/profile/appointment\" from its profile, but not for the application/x-www-form-urlencoded. This is a currently specced, though there may be a way to make it work there too.

[mca]

cool.

eventually, you'll need to add that to the spec, then.

and i don't yet recognize that profile doc format. is that a media type that already exists? adding that as a reference in the docs will help implementors, too.

thanks for helping me sort this out.

cheers.

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund

[jmon...@ms3-inc.com]

Hey, happy to have your feedback.

Yes, it's an extension of an OpenAPI media type object defined here https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#mediaTypeObject

[mca]

What's the IS A registered media type string for openapi?

And is it valid to reply using the openapi IANA identifier with a body that has only the content in your profile example?

[jmon...@ms3-inc.com]

Unfortunately the openapi committee has not yet registered a media type with IANA, but from googling around I found this https://github.com/OAI/OpenAPI-Specification/issues/110#issuecomment-364498200 which suggests they plan to register

application/vnd.oai.openapi (YAML variant)
application/vnd.oai.openapi+json

with a "version" parameter.

About using openapi's media type for what I defined as a profile, I'm guessing it's a no at the moment. See OpenAPI opens any object to be extended with "x-" prefixed properties, so what I defined as a profile is still a perfectly valid OpenAPI media type object. But then there are a couple problems, openapi designates some components to be reusable through references so they could be standalone documents, but 1. it's not clear if those documents would use the openapi media type if/when registered and 2. media type object is not one of the currently designated as reference-able. For example, they decided to make Requests and Responses reusable instead of media type.

Would you suggest I register a type for the profile as well? I had considered just going at it with the requirement that the URI in the profile must be in that format even if just returned as app/json or app/yaml. Perhaps if phtal ends up being somewhat successful openapi might consider making the media type object reusable which might allow phtal to use that media type for profiles. Or perhaps making use of https://json-schema.org/latest/json-schema-core.html#rfc.section.12.2 something like app/schema-instance;schema="phtal.org/schema.json"

[mca]

yeah - i was hoping that OpenAPI had established a vnd or prs ide but that's not happened yet. no biggie.

MEDIA TYPES CONSIDERED HELPFUL

FWIW, i think it really helps the spec when there are clear media type identifier for responses. it allows client apps to sniff for formats and route the responses to the proper parser. the good news is that this "media type aware" client is not *required*. ppl can still hard-code assumptions about responses and both hard-coded and type-aware can interact successfully with the service. When the media type ID is missing this is not the case.

so, i encourage folks to spec out media type IDs whenever possible.

MINTING YOUR OWN

having said that, i think i agree that using OpenAPI as a type identifier for the profile response is not a good fit. for now at least, I'd just mint a new one for your profile format (e.g. app/vnd.phtal-profile). that does a couple things for you, i think. first, you get full control of what can be in that response (e.g. any future changes to OpenAPI won't fuss with your spec or implementations). second, you open yourself up to the possibility that you can spec support for _other_ profile formats in the future (DCAP, ALPS, etc.) without breaking any existing clients or servers using your spec. finally, you can make your own changes to what is in/out of that profile document w/o having to negotiate w/ anyone else as you add backward-compatible changes.

SCHEMA

and, yes, you *could* use schema= or even your own profile= (that'd be an inception moment, i guess) to solve this problem. that could work but might will you into dealing with additional work to serialization formats (JSON, XML YAML) variants and fiddly bits like app/json is not supposed to use additional arguments, blah blah blah.

RABBIT HOLE

lastly. i may be leading you down a rabbit hole since this profile format id is just a bootstrap element in order to get after the bigger picture of using phtal anyway, but i thought it worth addressing early in case it modifies some of your design and or spec decisions.

Cheers.

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund

[jmon...@ms3-inc.com]

You bring up a good point about controlling the format, I initially defined profiles to be something like:

description: A store's catalog item.
baseType: application/xml
schema:
  type: xsd
  externalValue: http://localhost/jam01/schemas/xsd/CatalogItem
examples:
  productItem:
    summary: A simple example that shows a product item, a soccer ball.
    externalValue: http://localhost/jam01/examples/xml/SoccerBall
  serviceItem:
    description: This item shows all the additional items that could be present on a catalog item for a service, these kind of items will only be available...
    externalValue: http://localhost/jam01/examples/xml/CarWash

which I think it's a better fit (openapi doesnt allow xsd for example). However I made the decision to just extend openapi's MT object to 1. be able to reuse some tooling and 2. encourage adoption by not having to introduce yet another format. It might still be the better option to define my own though.

PS. I'm always jumping from one rabbit hole to another haha, great way to learn.

[jmon...@ms3-inc.com]

Mike, did you ever consider how 'profile' and 'type' media type parameters could work with ALPS? Would something like `application/phtal;profile="alps.io/someAlps";type="alps.io/someAlps#someDesc"` make sense?

[mca]

well, i see that's possible, but not sure why you would do that.

what's the advantage?

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund

[Jose Montoya]

Say that we defined Alps profiles A and B. Each one developed independently and only describing state transitions within themselves. At runtime an API wants to offer an unsafe relationship between an A descriptors to a B desc. One way to do it would be to provide in the link something like

consumes: app/phtal;profile=B;type=B.x

produces: app/phtal; profile=B;type=B.y

This way clients that know both profiles can compose requests and clients that don't can see there's another profile they could possibly learn.

This is all because I've been revisiting the possibility of making phtals profiles more like alps, but still want to retain the ability to signal individual representations formats (equivalent to descriptors) in the header. 

What I'm trying to work through is what rules would make sense for the parameters. For example if a type param is used then the profile param should probably contain a single URI only, and if that's the case then the type param could just be a fragment. Or should it not? Talking out loud a bit.

[mca]

yeah - not sure i follow but will point out something that i've found in case we're talking around the sdame ideas.

adding multiple profiles to the same response can make it pretty tough for a client to parse and "understand" what to do with that response. adding profiles should be considered expontial, not cmulative. when i have two profiles (that describe data and action elements) and *some* of those overlap, it's tough to know what is to be done. just trying writing one (at least (I* had trouble writing a robust client in that case).

i've found it easier to provide a "profile negotiation" feature like media type negotiation. clients can make a request like this:

GET /customer-list

Accepts: app/vnd.collection+json, app/vnd.uber+json

Profile-Accepts: api.example.org/profiles/accounting-v1;api.example.org/profiles/accounting-v2;

and the server can then decide what format to use in the representation and which vocabulary to use when mapping the data into the format.

as for "unsafe relationship between A and B" there is nothing in ALPs (or DCAP, etc.) that prevents this kind of transition. just follow a transition in the representation from a resource that contains data/actions from profile "A" that takes you to a new resource that only supports data/actions from profile "B". i don't think you can be assured that that client will understand both profiles, but you can certainly navigate between them. just like when some resources only return HTML, not matter what Accept header you send.

does that make sense?

maybe i am missing the details of your example, tho.

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund

[jmon...@ms3-inc.com]

Hey Mike,

So phtal ended up turning into something different (sorry!). I've separated it into different drafts which I think definitely helps clarity. Results are XREL which is a format for extended relationships, and PHTAL which is now just a generic hypermedia type using RAML data types as profiles.

I've published versions -00 for both with the IETF, but I'll link the github HTML versions because they look nicer.

https://phtal-org.github.io/internet-draft-phtal/draft-montoya-phtal.html
https://phtal-org.github.io/internet-draft-xrel/draft-montoya-xrel.html

Appreciate any feedback.

- Jose

[mca]

thanks for the update. design, use, interate. a great pattern for finding what works and where boundaries appear.

continue to keep us all posted on progress.

cheers

Mike Amundsen

APIs, Microservices, and Digital Transformation

7310 Turfway Rd. Suite 550, Florence, KY, 41042

+1.859.372.6517

https://zoom.us/j/8593726715

http://linkedin.com/in/mamund

http://g.mamund.com/meetme
http://training.amundsen.com

http://twitter.com/mamund