Request for Comments: Web Services API - Second Draft

[Chris Davenport]

Over the past six weeks or so I have been working on the second draft of the web services API specification and I now have great pleasure in making it available for feedback from all interested parties.

It has taken quite a while to get to this point in large part because it turned out to be a very significant rewrite of the first draft (published just before Christmas).  This came about because of the excellent feedback from reviewers and also because I was able to read more extensively and think a lot more about how web services should be implemented and used in the future. The result is a more considered and comprehensive specification that I believe will enable a very wide range of services to be exposed while also enabling simple, elegant client code to be created allowing powerful applications to be built to access them.

Most of the original material is still there, greatly extended and suitably amended in the light of the feedback received, but it has also been rearranged into what I hope is a more logical format.

The most important changes that have been made are as follows:

• It has become more RESTful and in particular now adheres much more closely to the HATEOAS constraint ("hypermedia as the engine of application state").
• One document has now become seven! This is because the new specification centres around three new media types (plus a base type) that should probably be added to the vendor-specific tree of the IANA media types registry.  Each media type has its own specification document.  Implementation information on the proposed CMS resources and the proposed CLI API are also now in separate documents.
• The media type specifications can now be seen as extensions of the Hypertext Application Language (HAL) specification.  The first draft adopted some parts of HAL, but as I now understand it more deeply I can see no reason not to adopt it in full.
• One particularly interesting consequence is the addition of support for curies (compact URIs) which makes namespacing of resource quite straightforward.  This should make it easier for site integrators to avoid resource naming conflicts when installing web services-enabled extensions from multiple third-party developers.
• An interesting optimisation pattern has been added in the form of lazy-loading of resource properties. Provided that clients follow the procedure it should be possible for servers to dynamically optimise the data that they exchange with clients.
• The new draft is also now more representation format-agnostic in the sense that it is less JSON-centric, although most of the examples are in JSON and JSON is still the recommended format and the most likely to be implemented first.
• Versioning is now more clearly defined and it is also now possible to version individual resource schemas as well as the API; something that I think 3PDs will appreciate.
• Search is now accessed by following a link from the service resource, with separate links distinguishing basic and smart searc as well as autocompletion.
• checkin and checkout operations are now found by following links.
• Many more examples and diagrams have been added.

The documents are as follows:-

• API specification.  This should be considered the" master" document that provides an introduction to the material and binds it all together.  https://docs.google.com/document/d/1FVKGlV6BN6pu-YH2WR2pQHE3Ez7M6r7LD417GSw9ZSo/edit?usp=sharing
• application/vnd.joomla.base.v1.  This is the base media type from which the others are derived. Read this one before you read the others.  https://docs.google.com/document/d/11SqH-daKQV9SrFBMEpopjBk3vM1USIHnFWZB9rjJB94/edit?usp=sharing
  
• application/vnd.joomla.service.v1. Provides metadata about the API and a list of resources available from it. https://docs.google.com/document/d/1wg3AcgStA26UwDcbHVV1bub4sa_BhsKfzAmX21eG-FM/edit?usp=sharing
  
• application/vnd.joomla.item.v1. A representation of a single resource item. https://docs.google.com/document/d/16xwxSDDPW0U1CG9l7JcwOyGvyjm7wv5zOSd9JwgF2iQ/edit?usp=sharing
  
• application/vnd.joomla.list.v1. A representation of an ordered collection of resource items. https://docs.google.com/document/d/1PLym28MG5v1tWyvIyW-9483JNKh5AP21Fmsmg62plnA/edit?usp=sharing
  
• API implementation.  Looks at the schemas that might be used by the Joomla CMS.https://docs.google.com/document/d/1d5qQ16r1Bo1BlXXuyS_eFB4BQcfuSg05pn9hsMpAgqk/edit?usp=sharing
• CLI API.  A possible CLI which would allow access to services from the command line.  https://docs.google.com/document/d/1wI3cSm3y4aa8n8rojJKpiF6RUpSl63WFuLgJj2WqW8o/edit?usp=sharing

This is not the final version of this specification.  I have no doubt that changes will need to be made during the development process and in the light of experience.

What would be really great would be to hear from anyone who might be interested in starting to write some code for this stuff. The only real test of this standard is for people to actually use it to build real applications and feed their experiences back so that the rough edges can be smoothed and the ideas can be clarified and documented for the benefit of those who come after.

Have fun!

Chris.

--
Chris Davenport
Joomla Production Leadership Team

[David Hurley]

This looks amazing Chris! +1.

I look forward to reading through all of it. It's obvious this took a while to write. Nicely done!

-

Thanks,
David Hurley
Joomla! Community Development Manager

--
You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.
To post to this group, send an email to joomla-...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-cms?hl=en-GB.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Chad Windnagle]

This is incredible work. Well done. I'm very interested in seeing where this goes. 

--

You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.
To post to this group, send an email to joomla-...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-cms?hl=en-GB.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Regards,
Chad Windnagle
Fight SOPA

[Nick Savov]

+1 Looks amazing. Nice job, Chris! Kudos! :)

Kind regards,
Nick

> This is incredible work. Well done. I'm very interested in seeing where
> this goes.
>
> On Saturday, February 9, 2013, David Hurley wrote:
>
>> This looks amazing Chris! +1.
>>
>> I look forward to reading through all of it. It's obvious this took a
>> while to write. Nicely done!
>>
>>
>> -
>> Thanks,
>> David Hurley

>> *Joomla! Community Development Manager*

>>
>>
>> On Sun, Feb 10, 2013 at 5:36 AM, Chris Davenport <
>> chris.d...@joomla.org> wrote:
>>
>> Over the past six weeks or so I have been working on the second draft of
>> the web services API specification and I now have great pleasure in
>> making it available for feedback from all interested parties.
>>
>> It has taken quite a while to get to this point in large part because it
>> turned out to be a very significant rewrite of the first draft
>> (published
>> just before Christmas). This came about because of the excellent
>> feedback
>> from reviewers and also because I was able to read more extensively and
>> think a lot more about how web services should be implemented and used
>> in
>> the future. The result is a more considered and comprehensive
>> specification
>> that I believe will enable a very wide range of services to be exposed
>> while also enabling simple, elegant client code to be created allowing
>> powerful applications to be built to access them.
>>
>> Most of the original material is still there, greatly extended and
>> suitably amended in the light of the feedback received, but it has also
>> been rearranged into what I hope is a more logical format.
>>
>> The most important changes that have been made are as follows:
>>

>> - It has become more RESTful and in particular now adheres much more

>> closely to the HATEOAS constraint ("hypermedia as the engine of
>> application
>> state").

>> - One document has now become seven! This is because the new

>> specification centres around three new media types (plus a base type)
>> that
>> should probably be added to the vendor-specific tree of the IANA
>> media
>> types registry. Each media type has its own specification document.
>> Implementation information on the proposed CMS resources and the
>> proposed
>> CLI API are also now in separate documents.

>> - The media type specifications can now be seen as extensions of the

>> Hypertext Application Language (HAL) specification. The first draft
>> adopted some parts of HAL, but as I now understand it more deeply I
>> can see
>> no reason not to adopt it in full.

>> - One particularly interesting consequence is the addition of support

>> for curies (compact URIs) which makes namespacing of resource quite
>> straightforward. This should make it easier for site integrators to
>> avoid
>> resource naming conflicts when installing web services-enabled
>> extensions
>> from multiple third-party developers.

>> - An interesting optimisation pattern has been added in the form of

>> lazy-loading of resource properties. Provided that clients follow the
>> procedure it should be possible for servers to dynamically optimise
>> the
>> data that they exchange with clients.

>> - The new draft is also now more representation format-agnostic in

>> the
>> sense that it is less JSON-centric, although most of the examples are
>> in
>> JSON and JSON is still the recommended format and the most likely to
>> be
>> implemented first.

>> - Versioning is now more clearly defined and it is also now possible

>> to version individual resource schemas as well as the API; something
>> that I think 3PDs will appreciate.

>> - Search is now accessed by following a link from the service

>> resource, with separate links distinguishing basic and smart searc as
>> well
>> as autocompletion.

>> - checkin and checkout operations are now found by following links.
>> - Many more examples and diagrams have been added.

>>
>> The documents are as follows:-
>>
>> - API specification. This should be considered the" master" document
>> that provides an introduction to the material and binds it all
>> together.
>> https://docs.google.com/document/d/1FVKGlV6BN6pu-YH2WR2pQHE3Ez7M6r7LD417GSw9ZSo/edit?usp=sharing

>> - application/vnd.joomla.base.v1. This is the base media type from

>> which the others are derived. Read this one before you read the
>> others.
>> https://docs.google.com/document/d/11SqH-daKQV9SrFBMEpopjBk3vM1USIHnFWZB9rjJB94/edit?usp=sharing

>> - application/vnd.joomla.service.v1. Provides metadata about the
>> APIand a list of resources available from it.
>> https://docs.google.com/document/d/1wg3AcgStA26UwDcbHVV1bub4sa_BhsKfzAmX21eG-FM/edit?usp=sharing
>> - application/vnd.joomla.item.v1. A representation of a single
>> resource item.
>> <https://docs.google.com/document/d/16xwxSDDPW0U1CG9l7JcwOyGvyjm7wv5zOSd9JwgF2iQ/edit?usp=sharing>

>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups
>> "Joomla! CMS Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an

>> email to joomla-dev-cm...@googlegroups.com <javascript:_e({},
>> 'cvml', 'joomla-dev-cms%2Bunsu...@googlegroups.com');>.

>> To post to this group, send an email to

>> joomla-...@googlegroups.com<javascript:_e({}, 'cvml',
>> 'joomla-...@googlegroups.com');>
>> .

>> Visit this group at
>> http://groups.google.com/group/joomla-dev-cms?hl=en-GB
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
> --
> Regards,
> Chad Windnagle

> Fight SOPA <https://www.google.com/landing/takeaction/>

[Matias Aguirre]

Amazing work Chris!

I like to be part of the initial 'code workers', i have some experience with webservices so maybe could be useful to implement restful on Joomla!.

I have a question to make forward, its about URL Routing:

One problem that i found trying to implement RESTful on Joomla is the URL Routing on CMS, if we try to call GET /articles/1234 Joomla! routing will try to go to the CMS site and will not interpret that is a webservices call.
How this will impact on SEF components and Joomla SEF component? What we need to modify on CMS routing to get this working?

What is your idea to start the coding? There are some github project? I think that creating at first the CLI script could speed the logic to be implemented on CMS later.

Thanks for your effort and best regards.

[Chris Davenport]

Thanks for all your kind words guys. :-)

@Matias.  Great to hear that you'd like to work on making this happen.

I think the entry point would most likely be http://www.domain.com/api/index.php with an .htaccess rule so GET /articles/1234 in the documentation would translate to doing a GET on http://www.domain.com/api/articles/1234.  What do you think?

Ideally, I'd like it to be flexible so folks who would rather set it up to do http://api.domain.com/articles/1234 or http://www.domain.com/some/wher/else/articles/1234 instead can do so.  Is that possible?

To be honest, I'm still very unclear about how this will work architecturally in current or future Joomla's and I think that would be a great discussion to start (in another thread, I think).

I would suggest just forking the code in a branch and just going for it!  Which Joomla version were you thinking of targetting?

Best,
Chris.

--
You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.
To post to this group, send an email to joomla-...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-cms?hl=en-GB.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Matias Aguirre]

Good idea, if .htaccess rules can sort our problem is fantactic. There are another more complicated way. Exists a class into the joomla-cms libraries called JApplicationWebRouterRest that is a subclass of JApplicationWebRouter but i dont know if this classes are used by CMS currently. Would be awesome to know how to use this classes and adapt it using your API reference.

What im doing in jUpgradePro is just to create one plugin that read if the request are a RESTful, and ifs exists go to process the request and exit the CMS. This is the RESTful process class included into the plugin:

https://github.com/fastslack/jUpgradePro/blob/master/trunk/plugin/jupgrade/rest.php

Its based on the Louis Landry oauth work but modified for RESTful: https://github.com/LouisLandry/joomla-platform/tree/oauth/libraries/joomla/oauth

The possitive thing about using plugin is that you are allowed to enable/disable it and also is flexible about using api.example.org hostname too.

Regards

[Michael Babker]

Louis was working on an advanced version of the Platform Pull Tester using some of the new classes, including the web router.  The workflow is this:

- The web application is launched - https://github.com/LouisLandry/tester/blob/master/src/classes/application/web.php#L95

- In this instance, he stores the routes in a JSON string external to the class at https://github.com/LouisLandry/tester/blob/master/etc/routes.json

- The router then routes the request, calling the classes located at https://github.com/LouisLandry/tester/tree/master/src/classes/service based on the routing config linked previously.

[Matias Aguirre]

Hi guys!

I added a new project on github with an initial plugin to give support of webservices to Joomla! The url is https://github.com/fastslack/plg_restful

This is a very initial plugin. Im trying to get the same querys following the docs of Chris:

https://docs.google.com/document/d/1FVKGlV6BN6pu-YH2WR2pQHE3Ez7M6r7LD417GSw9ZSo/edit#
https://docs.google.com/document/d/1wI3cSm3y4aa8n8rojJKpiF6RUpSl63WFuLgJj2WqW8o/edit

If you like to test it you must install the plugin into Joomla 2.5/3.0 and add 2 new lines to your .htaccess file:

RewriteRule ^api/([^/]*)/([^/]*)$ index.php [L]

RewriteRule ^api/([^/]*)/([^/]*)/([^/]*)$ index.php [L]

For now, the only way to test it is using the CLI script included into the github project.

The TODO list:

* Authentication/Permission control

Now the plugin just let the user to get any articles without permissions control.

* Add more resources

Now the only resource is articles. What more resources we must to add?

* Better URI handler

Now only 2 ways we have to get an article:

http://example.org/api/v1/articles/5 -> to get a simple row (id 5)

http://example.org/api/v1/articles?page=2&perPage=20 -> to get a list using pagination

Anything more info you need just let me know.

Regards

On Sunday, 10 February 2013 17:02:38 UTC-3, Chris Davenport wrote: