Reconciling theory with practice on implementing service layers as resources

[Brian G]

I feel like I understand REST fairly well in a basic implementation of
CRUD. Where I am less confident is when we go beyond simple read/
update/delete and migrate method calls and business logic to resources
with states. Since we're not "supposed to" call something like

/my/uri/to/resource/foo/myFunctionCall

Then we need some way to trigger myFunctionCall() by way of a session-
less state change. I thought this article had an interesting point of
view:

http://blog.dhananjaynene.com/2009/06/rest-is-the-dbms-of-the-internet/

He goes to say:

"Traditional SOA based integration visualises different software
artifacts being able to interact with each other through procedures or
methods. REST effectively allows each software artifact to behave as a
set of tables, and these artifacts talk to each other using SELECT,
INSERT, UPDATE and DELETE. (or if you wish GET, PUT, POST, DELETE).
And where exactly is is the business logic ? Is it in the stored
procedures ? Not Quite. Its in the triggers."

In the triggers! Interesting way of relating REST to something we're
all familiar with. Let's talk in specifics about a specific workflow,
for an event registration (a domain problem near and dear to my
heart):

Client searches for events near location
Client picks an event and gets a description of the event from server
Client puts a registration
Server saves registration (status of new)
Server sends email receipt
Server sends email notification to event organizer
Server updates cached list of attendees
Client gets list of attendees for event (which now includes Client)
Client puts an updated registration (changing a lunch preference, for
example) (status = new)
Server sends email notification to event registrar
Client gets updated registration, payment status is = balance due
Client gets available credits -> returns list of payment credits
Client puts a payment that uses one $50 credit, supplies a discount
code and credit card credentials
Server processes payment. In our system, this means something very
specific and interdependent:
- If one or more discount codes are provided, apply them to the
balance first and reduce the total due by that amount (but don't go <
0)
- If one or more payment credits are supplied, apply them to the
balance and reduce the total. If one of the credits is larger than
the balance, roll the balance forward into a new credit
- Charge the credit card for any remaining balance
- Only if the credit card payment is successful should the discount
codes and credits be saved.
- Server returns answer of success or failure, let's assume success
Server updates registration status from new -> confirmed
Server sends email receipt to Client
Server sends email notification to registrar
Client gets updated registration, status is now confirmed

I think, if I am understanding these posts correctly, that the right
approach is for everything to have a status and you can have a
resource for anything. So in the case of the above, the convo might
look like:

GET /events?zipcode=94103 -> return list of events near zip code in
json/xml
GET /events/23 -> return details of event #23
POST /registration -> based on details of #23, send data required to
register, returns link to /registration/2903423 (registration
#2903423)
GET /events/23/attendees -> return list of attendees
PUT /registration/2903423 (update lunch)
GET /registration/2903423 (payment status is balance due)
GET /member/100/credits?event=23 (get all available credits that can
be used for #23, returns a $50 credit with id 5000)
POST /registration/2903423/payment (include credit id 5000, discount
code "CHEAP" and credit card details modeled in a json/xml packet) ->
returns success
GET /registration/2903423 -> representation is now in confirmed
status, all is done.

If we model things in the trigger model, it seems like my REST api
needs to look at an incoming resource and compare the current and
proposed state and generate actions based on that. Implementation-
wise, does this requires everything to have a status/state flag?
This doesn't sound a lot different than an event-based MVC framework
like Model-Glue, etc where the resources is the "event" and the
actions that are subsequently called on the Controller for
ValidationRegistration, SaveRegistration, SendEmailNotification, etc.
But what about when it's less of a resource and more of an action,
"search" for example? This guy would suggest you simply convert your
verb to a noun and have at it:

http://blog.dhananjaynene.com/2008/11/rest-fomenting-unrest-is-restfulness-a-semantics-game-why-does-rest-require-statelessness/

My question is - is this how people are doing it? If not, how are you
handling state changes that require additional actions? I'm
struggling with how to convert the minimal service-juggling logic
currently found in my Controllers into a stateless REST API (well, and
statelessness in general) The logic above is just one example but
concrete enough to hopefully have a practical discussion.

I would love to see notes from anyone who has created a REST api from
an application and how you modeled your resources and representations
compared to your Objects and methods.


Brian

[websolete]

On Mon, Jun 15, 2009 at 9:34 PM, Brian G <brian-...@vfive.com> wrote:

I feel like I understand REST fairly well in a basic implementation of
CRUD.  Where I am less confident is when we go beyond simple read/
update/delete and migrate method calls and business logic to resources
with states.  Since we're not "supposed to" call something like

/my/uri/to/resource/foo/myFunctionCall

Then we need some way to trigger myFunctionCall() by way of a session-
less state change.  I thought this article had an interesting point of
view:

http://blog.dhananjaynene.com/2009/06/rest-is-the-dbms-of-the-internet/

He goes to say:

"Traditional SOA based integration visualises different software
artifacts being able to interact with each other through procedures or
methods. REST effectively allows each software artifact to behave as a
set of tables, and these artifacts talk to each other using SELECT,
INSERT, UPDATE and DELETE. (or if you wish GET, PUT, POST, DELETE).
And where exactly is is the business logic ? Is it in the stored
procedures ? Not Quite. Its in the triggers."

I agree with this in principle, but while I don't like the overly simplistic 'REST is web-based CRUD', I'll grant the analogy is deeper than it appears. 

With a REST api, you are either requesting the current state of a resource (GET) or you are affecting some change to that resource's state (PUT,POST,DELETE) *at the server*.  Likewise, when manipulating data in a database, be it SELECT or UPDATE/INSERT/DELETE, you are sending requests to the database to change that data *in the database*; the resource (rows of data) is a 'static' resource which is found fully contained within the database and never leaves there.    Complex logic in both scenarios will be, in effect, 'triggers', as these are fired by changes in the state of the resources, not the act of making the request.
 

I don't think that everything must have a status, but there certainly are states in which a resource can be which preclude certain actions from being taken against it. 

For example, if a user creates a new user account but has not completed the step of confirming their email address in order to activate it, then the state of the user account resource would prevent update requests, delete requests, etc.  Whether this materializes as a single 'column' or attribute 'flag' or is derived from a more complex examination of multiple resource data points is really just an implementation detail.
 

GET /events?zipcode=94103 -> return list of events near zip code in
json/xml
GET /events/23 -> return details of event #23
POST /registration -> based on details of #23, send data required to
register, returns link to /registration/2903423  (registration
#2903423)
GET /events/23/attendees -> return list of attendees
PUT /registration/2903423 (update lunch)
GET /registration/2903423 (payment status is balance due)
GET /member/100/credits?event=23 (get all available credits that can
be used for #23, returns a $50 credit with id 5000)
POST /registration/2903423/payment (include credit id 5000, discount
code "CHEAP" and credit card details modeled in a json/xml packet) ->
returns success
GET /registration/2903423 -> representation is now in confirmed
status, all is done.

If we model things in the trigger model, it seems like my REST api
needs to look at an incoming resource and compare the current and
proposed state and generate actions based on that.  Implementation-
wise, does this requires everything to have a status/state flag?
This doesn't sound a lot different than an event-based MVC framework
like Model-Glue, etc where the resources is the "event" and the
actions that are subsequently called on the Controller for
ValidationRegistration, SaveRegistration, SendEmailNotification, etc.

Yeah, this is the RPC approach, and is obviously not RESTful.  The RESTful 'event' is what is triggered by the change to a resource's state, not the request to make the change. 
 

But what about when it's less of a resource and more of an action,
"search" for example?  This guy would suggest you simply convert your
verb to a noun and have at it:

http://blog.dhananjaynene.com/2008/11/rest-fomenting-unrest-is-restfulness-a-semantics-game-why-does-rest-require-statelessness/

I agree that simply redefining the verb to a noun goes a long way to solving this conundrum.  www.blah.com/api/doSearch?q=whatever is event-driven, and thats what you're trying to avoid (if you're going for REST purity - which it sounds like). 

Re-envisioning the search action as a static resource is the key; that is, www.blah.com/api/search or www.blah.com/api/search/ means 'i have a static resource 'bucket' called search, it just sits there until somebody drops a search term into it, at which time its state has changed to reflect 'search resource has unresolved search term for which results which must be retrieved'.  I think the implementation of a noun-based search 'service' could either be done as a POST or PUT, but POST seems more appropriate to me, so I would favor the /search/ resource 'collection' (POST) rather than the singular /search (PUT).
 

My question is - is this how people are doing it?  If not, how are you
handling state changes that require additional actions?   I'm
struggling with how to convert the minimal service-juggling logic
currently found in my Controllers into a stateless REST API (well, and
statelessness in general)  The logic above is just one example but
concrete enough to hopefully have a practical discussion.

As I said above, I think the statelessness 'problem' can be resolved by going back to the db CRUD analogy.  You don't (typically) maintain state across DML calls to a db; the calls themselves are atomic and don't require the calling application to say 'oh, by the way, this update is a continuation of update 54323 from two minutes ago'; it's most clear when approaching it from the stance that the data is always in the database, and that you must make REQUESTS to the db to change it.   Same goes for RESTful resources; you make REQUESTS to the api to make changes, it has no obligation to do so if required conditions aren't met, and the api consumer never 'has' the resource itself, just knowledge of where it can be found.

Also, as a corollary to database INSERT/UPDATE statements, in a REST api I see no reason to *require* a full resource representation to be sent by the consumer if its really not necessary (I'm speaking to the point of having to retransmit 'tons of information' when not relying on session state); if a PUT request only includes one data point out of twenty, then assuming other conditions are not violated, that one data point can be updated by itself, akin to updating a single column in a db without being required to update them all.

Kevin

 

[Brian G]

As some background, I'm considering a ground up rewrite of an existing
service as a REST API that could be consumed by many clients: my own
ColdFusion web site, an AIR client, a JS client, plugins for Joomla or
Drupal, etc. I can't expect these clients, some of which may not be
programmed by my team, to observe any kinds of rules so my REST API
must enforce all of my business logic but do so in a resource-oriented
way. I am struggling with that architecture.

On Jun 15, 12:55 pm, websolete <websol...@gmail.com> wrote:
> With a REST api, you are either requesting the current state of a resource
> (GET) or you are affecting some change to that resource's state

> (PUT,POST,DELETE) **at the server**.  Likewise, when manipulating data in a

> database, be it SELECT or UPDATE/INSERT/DELETE, you are sending requests to

> the database to change that data **in the database**; the resource (rows of

> data) is a 'static' resource which is found fully contained within the
> database and never leaves there.    Complex logic in both scenarios will be,
> in effect, 'triggers', as these are fired by changes in the state of the
> resources, not the act of making the request.

Working through my response, I may have come to understand this
better. Effectively my server-side code needs to look at every PUT
and POST and seek out any changes in the resource made via the
submitted representation. A simplified version would look at each
property, comparing what was provided to what already exists and
executing any necessary code in the event of a change. Where before
I would say cancelRegistration() which would trigger a handful of
activities, now I must look for when the status is now cancel and was
not before and execute cancelRegistration(). I guess in some respects
this makes the representation processing a Controller on top of a
service layer. That makes sense to me, but I was having a hard time
trying to think of the API *as* the service layer. I guess it is not,
rather the representations passed back and forth represent a data
exchange mechanism in the same way that HTML forms do.

> For example, if a user creates a new user account but has not completed the
> step of confirming their email address in order to activate it, then the
> state of the user account resource would prevent update requests, delete
> requests, etc.  Whether this materializes as a single 'column' or attribute
> 'flag' or is derived from a more complex examination of multiple resource
> data points is really just an implementation detail.

Agreed. It seems like since you're triggering off of a diff in the
representation though, that you need to move what would be a method
call to a property that can be monitored.

Is anyone using a mainstream framework like MG, M2 or Coldbox where
the views are just representations to power this? I have been using
PowerNap to expose little bits of my existing service layer as a REST
API and it works awesome. I don't think at present it would make a
good controller though without some additional work. Maybe Brian C
can chime in here, I assume he's done more with it than I have.

> > If we model things in the trigger model, it seems like my REST api
> > needs to look at an incoming resource and compare the current and
> > proposed state and generate actions based on that.  Implementation-
> > wise, does this requires everything to have a status/state flag?
> > This doesn't sound a lot different than an event-based MVC framework
> > like Model-Glue, etc where the resources is the "event" and the
> > actions that are subsequently called on the Controller for
> > ValidationRegistration, SaveRegistration, SendEmailNotification, etc.
>
> Yeah, this is the RPC approach, and is obviously not RESTful.  The RESTful

> 'event' is what is *triggered *by the change to a resource's state, not the
> *request *to make the change.

Why is the above RPC-like if I watch for a property in the
representation to be switched to "cancel" before triggering my cancel
logic? Or if I wait for someone to POST a registration before running
validation, save, send, etc?

> Also, as a corollary to database INSERT/UPDATE statements, in a REST api I
> see no reason to *require* a full resource representation to be sent by the
> consumer if its really not necessary (I'm speaking to the point of having to
> retransmit 'tons of information' when not relying on session state); if a

I like this concept. Rsync for representation transfer.


Brian

[websolete]

On Tue, Jun 16, 2009 at 11:01 PM, Brian G <brian-...@vfive.com> wrote:

As some background, I'm considering a ground up rewrite of an existing
service as a REST API that could be consumed by many clients:  my own
ColdFusion web site, an AIR client, a JS client, plugins for Joomla or
Drupal, etc.  I can't expect these clients, some of which may not be
programmed by my team, to observe any kinds of rules so my REST API
must enforce all of my business logic but do so in a resource-oriented
way.  I am struggling with that architecture.

I agree that any API should assume that clients will misbehave and cannot be trusted to enforce any rules.  Also, I don't see any major conceptual differences between how you treat resource-oriented business logic and object-oriented business logic under the hood.  See below.

On Jun 15, 12:55 pm, websolete <websol...@gmail.com> wrote:
> With a REST api, you are either requesting the current state of a resource
> (GET) or you are affecting some change to that resource's state
> (PUT,POST,DELETE) **at the server**.  Likewise, when manipulating data in a
> database, be it SELECT or UPDATE/INSERT/DELETE, you are sending requests to
> the database to change that data **in the database**; the resource (rows of
> data) is a 'static' resource which is found fully contained within the
> database and never leaves there.    Complex logic in both scenarios will be,
> in effect, 'triggers', as these are fired by changes in the state of the
> resources, not the act of making the request.

Working through my response, I may have come to understand this
better.  Effectively my server-side code needs to look at every PUT
and POST and seek out any changes in the resource made via the
submitted representation.  A simplified version would look at each
property, comparing what was provided to what already exists and
executing any necessary code in the event of a change.   Where before
I would say cancelRegistration() which would trigger a handful of
activities, now I must look for when the status is now cancel and was
not before and execute cancelRegistration().  I guess in some respects
this makes the representation processing a Controller on top of a
service layer.  That makes sense to me, but I was having a hard time
trying to think of the API *as* the service layer.  I guess it is not,
rather the representations passed back and forth represent a data
exchange mechanism in the same way that HTML forms do.

In my own framework, I take PUTs and POSTs submitted by clients and convert them internally to a bean representing the resource.  Since it's a fairly (I assume) traditional bean with properties and getters and setters, it would be a pretty simple affair to know what has changed in the event of a PUT.  In a nutshell:

• client PUTs an updated representation
• I convert that representation to a native CF struct (usually)
• I create a bean object for the resource and init it with the current values for the resource.
• I call a method in the bean, passing in the struct of updated values, and it applies the values to the bean as required

Given this approach, it would be trivial to add additional properties to the bean to denote changes in property values.  From there, it would just be passing the updated resource bean into controller methods for necessary processing (taking the bean as input and checking the 'changed values' properties in the bean and calling required service methods).

Under the hood, I convert resources to beans (not everything) and handle them in a traditional way.  The 'resource-ness' is only relevant to the client, and to me it doesn't necessarily affect how the internal processing works.

When I initially created my REST framework, I used my own in-house MVP framework as the basis and built the REST interface on top of it.  Once a request is past the REST api layer, there's really little difference in how business logic is executed or enforced compared to a 'standard' MVC app.
 

> For example, if a user creates a new user account but has not completed the
> step of confirming their email address in order to activate it, then the
> state of the user account resource would prevent update requests, delete
> requests, etc.  Whether this materializes as a single 'column' or attribute
> 'flag' or is derived from a more complex examination of multiple resource
> data points is really just an implementation detail.

Agreed.  It seems like since you're triggering off of a diff in the
representation though, that you need to move what would be a method
call to a property that can be monitored.

Yes, see above.  I'm not fully comfortable or knowledgeable enough in OO to do much more than use 'objects' as data containers.  If I understand correctly, there are those that like a lot of logic in their beans/objects, but I'm not one of them (yet?).  I try to keep logic to a minimum in objects and pass them into handler methods which do the complex method calls as required.  My approach may be wrong, and a lack of formal training may be showing;  I think it was my experience with a mod of BlogCFC I was forced to work on which really turned me off to the idea of having tons of methods within objects; code management was horrible, and I found it far from intuitive.  C'est la vie.
 

Is anyone using a mainstream framework like MG, M2 or Coldbox where
the views are just representations to power this?  I have been using
PowerNap to expose little bits of my existing service layer as a REST
API and it works awesome.  I don't think at present it would make a
good controller though without some additional work.  Maybe Brian C
can chime in here, I assume he's done more with it than I have.

I don't use any of the mainstream frameworks (and believe that you know this ;) so I'm just responding in the absence of any other dialog to keep the juices flowing...

Also, I don't think the REST framework even needs to be the controller.  I see it as a fairly thin layer which standardizes the communication between application and client, and don't think it qualifies as a controller beyond passing through requests to the underlying app and enforcing authentication and data formatting requirements.
 

> > If we model things in the trigger model, it seems like my REST api
> > needs to look at an incoming resource and compare the current and
> > proposed state and generate actions based on that.  Implementation-
> > wise, does this requires everything to have a status/state flag?
> > This doesn't sound a lot different than an event-based MVC framework
> > like Model-Glue, etc where the resources is the "event" and the
> > actions that are subsequently called on the Controller for
> > ValidationRegistration, SaveRegistration, SendEmailNotification, etc.
>
> Yeah, this is the RPC approach, and is obviously not RESTful.  The RESTful
> 'event' is what is *triggered *by the change to a resource's state, not the
> *request *to make the change.

Why is the above RPC-like if I watch for a property in the
representation to be switched to "cancel" before triggering my cancel
logic?  Or if I wait for someone to POST a registration before running
validation, save, send, etc?

I was referring to what I understood you to mean with the idea that a resource is an event.  If I take a resource, convert it to a bean internally, then its just a data-thing.  When that data-thing is passed to a handler method, and the handler determines that some processing is required, then THAT is the event to me; that is, a client submitting a change in a resource representation is not the event, the determination by a handler that something must be done about the change is.  I think we're saying the same thing and this is just semantics.  It may have been more clear if I used Model-Glue and understood the context of what you were saying better.

FWIW, I see resources as merely that part of your application which is exposed to the client via the api and little more, similar to the interface of an object, simplifying and abstracting the complexity.  I never read into it beyond that, and never felt that the innards of an application must follow some 'resource-centric' approach or must be dramatically different than standard application architecture.  To me, REST simply dictates a standardized communication/data exchange protocol and doesn't fundamentally affect how you can or should do anything internally.