the resource dsl, take 1

[Daniel Yoder]

Team,

I'm going to walk through some ideas here for implementing a DSL for defining resources. Tip of the hat to Eero for helping me really think through this.

The first realization is that, currently in Waves, a Resource is really a resource server. We might even consider it a meta-object, except that it may often provide access to multiple resources (if we define them, per the spec, as being associated with a specific URI). Essentially, it bundles a group of related resources together.

I still believe this should be a class (not a Module) to minimize concurrency issues. That is, each request corresponds to an instance of the necessary Resource Server and then calls whatever methods to actually get ahold of a response.

Let's get started:

class Blog

  include Waves::Resource::Server

  # ... magic happens here ...

end

The trick now is to define what the resources are that we are serving and what the caching policies are for them. We separate this from generating representations:

  resource :list, :expires => 3.days, [ ‘blogs’ ]  do

    get { model.find_all }

  end

  resource :element, :expires => 3.days, [ ‘blog’, :name ]  do

    get { model.find_by_name( captured.name ) }

  end

  schema :element, [ 'schema', 'blog', '2009-03' ] do

    attributes :title => String, :description => String

    link :entries, :list => Story

  end

     

We can still generate links easily enough, as before, but we don't have to worry about accept headers and so forth. We would still want to be able to support query string matching, though, but that could probably just be an optional 4th parameter.

The :element resource that we've defined also has a schema. This allows us to generate RDF and simplifies link creation. The link method could have taken the form of a link (to an RDF document) instead of a class name. Providing a class name, however, lets us know that this particular resource can be accessed locally (in process).

Next, we need to define the representations. Since these are tied to resources, it's natural to want to put them under the resource definition. As with the schema, I split them out because this makes reuse (say, via a mixin) much simpler.

 representations :list do

   as :html, :lang => :en do | blogs |

     view( :blog ).list( blogs )

   end

 end

 representations :element do

   as :html, :lang => :en do | blog |

     view( :blog ).show( blog )

   end

   end

Obviously, this is more verbose than just using the #on method. However, it makes it much easier to add a wide variety of representations while still keeping the code very DRY. In addition to explicitly declared representations, we can also add a fall-through method that checks each accept format in order of preference to see if there is a corresponding #to_format method defined on the resource. If none is found, then we fall through to a NoMethod exception. The schema resources additionally would have RDF representations pre-defined.

It is this last set of stanzas where we would actually define the resource functors using #on in the DSL implementation. We can store the exception meta-data in a hash as it is declared and then access this meta-data when invoking #on. This is straightforward to implement, which to some extent suggests the current design and existing DSL for handling requests is the right foundation. In fact, my hope is that we can actually implement 99% of this DSL by addition, perhaps with a few deprecations.

Please share your thoughts!

Regards,

Dan

[Eero Saynatkari]

I like Dan's take--here is my latest attempt as a continuum
of the same exploration. The biggest complication in this is
the relatively poorly understood nature of REST--I myself am
(hopefully) just now truly getting somewhere with it--and
the associated difficulty in introducing people to the "right"
way of doing things.

At this point, I am certainly not sure I have it right myself,
but I think the best possible test for this latest iteration
is to present it with only minimal comments to gauge if it
is intuitive enough (but ask for clarifications if needed.)

The example represents a page in a wiki. Please note some of
the syntax is handwavy.

</ado>

# TODO: Quite obviously there is no "static" routing mechanism
# defined here. Reverse lookups from linkers possible?

# TODO: These need to be scoped.
#
pseudo_mime "application/vnd.org.kittensoft.giraffe.page.editable",
:extension => ".editable",
:content => "text/html"

pseudo_mime "application/vnd.org.kittensoft.giraffe.page.new",
:extension => ".new",
:content => "text/html"

# As a general note, transitions are available in the output templates

# Global transitions.
#
Waves.site.transitions {
link "Back to homepage", Site
link "Page list", Site/Pages
link "History", Site/History

# Could also do something like POST for extra weirdness.
link "Search", Site/Grep/:query, "application/vnd.org.kittensoft.giraffe.grep"
}

# TODO: Do HTML layout templates etc. need to be defined
# here similar to global transitions, or should it
# be handled as automatic wrapping based on MIME?

# A wikipage
#
resource Page do

# GET
#
viewable {

page = Giraffe::Page.get_magic_pseudo_page

# Unrendered page contents
#
# TODO: This is a tricky one, cannot provide links unless
# there can be metadata with the raw content.
#
representation "text/plain", "text/x-markdown" do
page.raw_content
end

# TODO: Common transitions here, if any--is it possible or
# even feasible to truly close the blocks so that they
# are not made available to any representations above
# this?

# TODO: Page at specific commit here or as param?

# Regular rendered page
#
representation "text/html", "application/xhtml+xml" do
transitions {
view "Edit page", Site/Page/URL, "application/vnd.org.kittensoft.giraffe.page.editable"

view "View raw contents", Site/Page/URL, "text/plain"
view "Show page history", Site/History/URL
view "Other pages under #{path}", Site/Pages/path
}

render page.to_html # E.g. representations/page.view.html.erb ?
end

# Editable version of a brand new page
#
# TODO: Is this right or should this be in Pages (list) instead?
#
representation "application/vnd.org.kittensoft.giraffe.page.new" do
transitions {
view "Discard and return to page list", Site/Pages/path # GET
create "Create This Page", Site/Page/URL # PUT
}

# Editor has raw text
render page.raw_content
end

# Editable version of existing page
#
representation "application/vnd.org.kittensoft.giraffe.page.editable" do
transitions {
view "Cancel Editing", Site/Page/URL # GET default MIME
view "View raw contents", Site/Page/URL, "text/plain"
view "Show page history", Site/History/URL
view "Other pages under #{path}", Site/Pages/path
delete "Delete This Page", Site/Page/URL # DELETE
update "Commit Your Changes", Site/Page/URL # POST
}

render page.raw_content
end

}

# There are no other representations? Or maybe filter stuff out?

# Creating a Page
#
# TODO: This could fail due to conflict etc.
#
createable {
Giraffe::Page.make params.URL, params.contents
redirect_created
}

# Updating a Page
#
# TODO: This could fail due to conflict etc.
#
updateable {
Giraffe::Page.find(params.URL).contents = params.contents
redirect_ok
}

# Deleting a Page
#
deletable {
oldpath = path
Giraffe::Page.delete params.URL
redirect_ok Site/Pages/oldpath
}

end

--
Magic is insufficiently advanced technology.

[ab5tract]

Wow, this is some heavy Stuff! I really like the idea of splitting it
all up like this.

I'm a little confused by psuedo_mime, at least in this example -- is
this something that helps integrate with RDF? Otherwise I'm just not
sure why it is necessary to add extensions like 'blah.new' instead of
just using URI like 'blah/new'.

Also, transitions -- I like this idea but would like to hear more
about it. Also these seem to be the most hand-wavey of the code Eero
posted.

On Mar 30, 5:24 am, Eero Saynatkari <registrati...@kittensoft.org>
wrote:

[Eero Saynatkari]

Hi!

Excerpts from john.haltiwanger's message of Mon Apr 06 19:36:20 +0300 2009:

>
> Wow, this is some heavy Stuff! I really like the idea of splitting it
> all up like this.
>
> I'm a little confused by psuedo_mime, at least in this example -- is
> this something that helps integrate with RDF? Otherwise I'm just not
> sure why it is necessary to add extensions like 'blah.new' instead of
> just using URI like 'blah/new'.

The basic idea for the whole pseudo_mime is that in REST,
with reasonable clients, we could actually implement these
ad-hoc or otherwise unofficial types but it is not possible
with (most) browsers, without using XmlHttpRequest at least.

The pseudo_mime is a bit confusing nomenclature anyway, as
Dan pointed out earlier: the pseudo part I added merely to
signify that the "actual" response type will always be (e.g.)
HTML as the MIME type itself is not supported. The ad-hoc
type is used to distinguish the type of representation,
allowing a more REST-like definition of the interface/API.

The extension could, certainly, be emulated in a different
way. I chose the extension because it, to me, resembles an
actual MIME type specifier. A theoretical client (or an XHR
request) supporting setting Accept headers correctly could
send the request *without* the extension, and browsers can
include it as a workaround.

> Also, transitions -- I like this idea but would like to hear more
> about it. Also these seem to be the most hand-wavey of the code Eero
> posted.

Transitions are that into different states, and they would
be executed for each individual resource (i.e. an instance
of a class). Each transition defines the method ("view" is
really GET etc.), a description of the action and the method
by which the actual link is generated--that syntax is very
much in the hand-wavy category. But the basic idea, again,
is that the resource should not necessarily know the exact
details of any other resources, and queries the Site--or
some basic configuration--for the details when generating
the real URLs.

As an example, if we assume that a hypothetical Page is
mapped so that the URL is split into the path and the name
components in a manner similar to

on :get, [{:path => 0..-1}, {:name => true}] ...

Then a link specification like

Site/Pages/:path

Actually means that whatever the prefix for the page list
resource (Pages) ends up being is appended only the path
component of this resource's URL, e.g. something like

"/pages/path/to/"

Since Pages works on directories rather than individual
file names.

The resources are then, obviously, available when creating
the response representation, and can be formatted according
to whatever scheme is appropriate, <a /> tags for HTML and
so on.

Hopefully that answers some questions! I have intended to
extrapolate a bit more on the plan to incorporate Dan's
language, caching etc. selectors, but it would be good to
hear opinions on the overall direction from everyone. If
there are any specific things that would help in seeing
how this all in my opinion helps with REST, e.g. a logic
flow example, I would be happy to try to come up with an
expanded one (anyone else is free to do so too, of course,
if you see where this direction is taking us).

And, finally, please do not hesitate to tell me I am wrong
wherever you may disagree with me.

Eero (rue)

[ab5tract]

Well I think this looks interesting. Is it implemented to the point of
being test-able yet? I think it will take an example of some sort to
really explain some of what you are talking about.

> Site/Page/URL

Is that params.URL? If so, can we just say params.URL? Or maybe we can
use instance variables (@path, @URL) or some such? One problem that
Waves seems to hit a lot (IMO) is mysterious variables that come from
seemingly nowhere. Especially here where they are being used right
alongside Class names.

On Apr 6, 1:40 pm, Eero Saynatkari <registrati...@kittensoft.org>
wrote:

[Eero Saynatkari]

Excerpts from john.haltiwanger's message of Tue Apr 14 18:32:33 +0300 2009:

>
> Well I think this looks interesting. Is it implemented to the point of
> being test-able yet? I think it will take an example of some sort to
> really explain some of what you are talking about.

I think I have been able to massage the DSL to something
that can be implemented, so I shall begin speccing it in
waves/edge. We (or I) were not sure whether to move with
this change now or at a later point, but it seems better
to do it now.

> > Site/Page/URL
>
> Is that params.URL? If so, can we just say params.URL? Or maybe we can
> use instance variables (@path, @URL) or some such?

Yes, exactly!

> One problem that
> Waves seems to hit a lot (IMO) is mysterious variables that come from
> seemingly nowhere. Especially here where they are being used right
> alongside Class names.

You make a good point; I am loath of such magic myself, and
it was not the best direction to take the mapping. In fact,
I have settled on an initial draft of the DSL that will be
much more explicit, and build the more natural abstractions
on top of it or to replace it once the functionality is in
place.

One of the great things about Waves, as Dan alluded to in
an earlier post, is that by my reckoning the DSL does not
require much in the way of infrastructure work in Waves:
all or most of the components and functionality are already
in place and can just be abstracted with this new layer--
which can quite possibly be implemented as a Foundation!

Regards,

[Daniel Yoder]

> I think I have been able to massage the DSL to something
> that can be implemented, so I shall begin speccing it in
> waves/edge. We (or I) were not sure whether to move with
> this change now or at a later point, but it seems better
> to do it now.

Yes. I've been wanting to play with this, but have not had enough
time. I'd love to have something to present at GoRuCo on May 31,
though. So sooner is better, definitely. If nothing else, it gives us
something more concrete to start working with.

>>> Site/Page/URL
>>
>> Is that params.URL? If so, can we just say params.URL? Or maybe we
>> can
>> use instance variables (@path, @URL) or some such?
>
> Yes, exactly!

Hmm. What? The URL is not a "param" at least in the sense that the
params come from POST or query variables.

>> One problem that
>> Waves seems to hit a lot (IMO) is mysterious variables that come from
>> seemingly nowhere. Especially here where they are being used right
>> alongside Class names.

I'm lost. What mysterious variables?

> You make a good point; I am loath of such magic myself, and
> it was not the best direction to take the mapping. In fact,
> I have settled on an initial draft of the DSL that will be
> much more explicit, and build the more natural abstractions
> on top of it or to replace it once the functionality is in
> place.

As far as I am aware, we use instance methods to access object data
and class methods to access invariants. So I'm not sure what magic
we're referring to ... ? There is a fine line between "magic" and
"convenience" and sometimes I worry we forget about the developer
_using_ the framework, who may not worry so much about fully
understanding how it works beneath the covers. But, again, I have no
idea what we're even talking about here. =)

> One of the great things about Waves, as Dan alluded to in
> an earlier post, is that by my reckoning the DSL does not
> require much in the way of infrastructure work in Waves:
> all or most of the components and functionality are already
> in place and can just be abstracted with this new layer--
> which can quite possibly be implemented as a Foundation!

Yes, there need not be a one-size-fits all solution. Just Foundations
to suit various architectural constraints.

Regards,
Dan (zeraweb)

[ab5tract]

On Apr 15, 3:22 pm, Daniel Yoder <d...@zeraweb.com> wrote:
> > I think I have been able to massage the DSL to something
> > that can be implemented, so I shall begin speccing it in
> > waves/edge. We (or I) were not sure whether to move with
> > this change now or at a later point, but it seems better
> > to do it now.
>
> Yes. I've been wanting to play with this, but have not had enough  
> time. I'd love to have something to present at GoRuCo on May 31,  
> though. So sooner is better, definitely. If nothing else, it gives us  
> something more concrete to start working with.
>
> >>> Site/Page/URL
>
> >> Is that params.URL? If so, can we just say params.URL? Or maybe we  
> >> can
> >> use instance variables (@path, @URL) or some such?
>
> > Yes, exactly!
>
> Hmm. What? The URL is not a "param" at least in the sense that the  
> params come from POST or query variables.

The content in question is:

representation "application/

vnd.org.kittensoft.giraffe.page.new" do
transitions {
view "Discard and return to page list", Site/Pages/
path # GET
create "Create This Page", Site/Page/
URL # PUT
}

From his original post. If 'URL' and 'path' are not variables
containing information from 'params, then what are they? And what
makes them different from 'Site' or 'Page' ? Perhaps this is totally
obvious (well, I believe that Site and Page are resources that
presumably fill in their own URI) but to me the syntax does not make
it clear. This could be just a problem with Ruby's approach to
sigilization, but for the sake of clarity, if 'path' is just the value
of 'params.path' I think it would look more intuitive if it read 'Site/
Pages/params.path' instead.

>
> >>                                                   One problem that
> >> Waves seems to hit a lot (IMO) is mysterious variables that come from
> >> seemingly nowhere. Especially here where they are being used right
> >> alongside Class names.
>
> I'm lost. What mysterious variables?

Basically anything that is only shown to be there when you see it in
examples but are not fully explained either in the examples or the
documentation. I believe 'query' is/was an instance of this. And/or
where a lack of sigilization leaves you wondering if it is a variable
or a method. (This is a general complaint that I am no longer afraid
to make about Ruby).