Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Building Record interface to REST api
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  6 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
pelagic  
View profile  
 More options Oct 4 2012, 6:10 pm
From: pelagic <ventis...@gmail.com>
Date: Thu, 4 Oct 2012 15:10:54 -0700 (PDT)
Local: Thurs, Oct 4 2012 6:10 pm
Subject: Building Record interface to REST api

Hey Everyone,

I'm building a Record interface for use with RESTful api backends and would
like some advice. So far I've been using JSONRecord from the couchdb-record
project, which has turned out great for serializing json into Records, and
Databinder Dispatch for HTTP stuff.

Right now I'm trying to figure out the best way to define the different
endpoints (uri) associated with a record. For right now I would like to
handle the following cases:

/foo
/foo/:id
/foo/bar
/foo/:id/bar

Currently I've chosen to build the uri via three methods.

trait RestRecord[MyType <: RestRecord[MyType]] extends JSONRecord[MyType] {

  /**  Defines the RESTful endpoint for this resource: /foo   */
  val uri: List[String]

  /**  Defines the RESTful suffix for endpoint: /foo/:id/bar    */
  val uriSuffix: List[String] = Nil

  /**  Defines and uri identifier for this resource: /foo/:id or
/foo/:id/bar   */
  def id: Box[String] = Empty

}

Using HTTP verbs and the endpoints I can find, create, update and delete
the record. The uri is also partially defined my the HTTP verb:

GET         uri ::: maybe the id ::: uriSuffix
PUT         uri ::: maybe the id ::: uriSuffix
POST       uri ::: uriSuffix    // no id because it hasn't been created yet
DELETE  uri ::: maybe the id ::: uriSuffix

Now you can define Records like this

/** /foo/:id  */
class Foo extends RestRecord[Foo] {
  override val uri = "foo" :: Nil
  override def id = Full(id.is)

  object id extends StringField(this)

}

/** /foo/:id/bar  */
class Bar extends RestRecord[Bar] {
  override val uri = "foo" :: Nil
  override val uriSuffix = "bar" :: Nil
  override def id = Full(id.is)

  object id extends OptionalStringField(this)

}

Does this seems like a reasonable approach to define the uri? I would love
to hear any feedback.

Thanks for you help and time,
Greg


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tobias Pfeiffer  
View profile  
 More options Oct 4 2012, 7:02 pm
From: Tobias Pfeiffer <tob...@tesobe.com>
Date: Fri, 5 Oct 2012 01:02:34 +0200
Local: Thurs, Oct 4 2012 7:02 pm
Subject: Re: [Lift] Building Record interface to REST api
Hi,

Am Thu, 4 Oct 2012 15:10:54 -0700 (PDT) schrieb pelagic:

> I'm building a Record interface for use with RESTful api backends and
> would like some advice. [...] I would love to hear any feedback.

I recently read "REST in Practice" from Webber/Parastatidis/Robinson
<http://shop.oreilly.com/product/9780596805838.do> and one thing I
learned from that book is that it's not necessarily a good idea to
abstract away the "remote" character of a remote RESTful service.

Quoting them:
"Unfortunately, WSDL’s conceptual model and the naïve tooling based
around it attempt to hide distribution as though programmers need to be
protected from it. [Footnote: Nothing could be further from the truth.
We need to know about distribution boundaries so that we can write code
that deals gracefully with failures outside our control.] In hiding the
remote aspects of a distributed system, we hide necessary complexity
to the extent that we can’t build services that are tolerant of their
inherent latencies, failure characteristics, and ownership
boundaries." (p. 385)

"The classic paper by Waldo et al. emphasizes that distribution cannot
be safely hidden from developers. [Footnote:
http://research.sun.com/techrep/1994/abstract-29.html] If you try to
hide distribution in a distributed system, the abstractions will leak
at inconvenient times and cause significant problems. It’s best to
acknowledge the distribution in distributed systems—it is necessary
complexity— and to plan accordingly." (p. 409)

In the recent past, I've built two Lift applications that get (part of)
their data from RESTful backend services. In one, a 500 error or a
delayed response from the REST service will cause problems all over the
place, probably because I tried too hard to hide the REST data source
behind a normal model class. In the other one, I absolutely wanted the
application to continue running when the backend service has problems,
so I basically Box'ed everything and I'm now able to run that whole
application even when the backend service is not running. Probably there
are still better solutions, though ;-)

While I think it would be great to treat a RESTful CRUD service just
like any other Record, I'd keep in mind what should happen if, say, the
server needs 5 seconds to send you certain resource data.

Just my 2 cent
Tobias


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
pelagic  
View profile  
 More options Oct 5 2012, 12:02 pm
From: pelagic <ventis...@gmail.com>
Date: Fri, 5 Oct 2012 09:02:23 -0700 (PDT)
Local: Fri, Oct 5 2012 12:02 pm
Subject: Re: [Lift] Building Record interface to REST api

Hey Tobias,

Thanks for you input, I really appreciate you sharing your experience on
working with RESTful services.

Could you ellaborate more on what kind of problems you were having

>In one, a 500 error or a delayed response from the REST service will cause

problems all over the

>place, probably because I tried too hard to hide the REST data source
>behind a normal model class

I'm currently working on a app that is backed entirely by an internal
restful api and I'd really like to incorporate all the awesomeness of
Record. So for me it might be worth it go down this road, maybe not, i'm
trying to figure that out.

Greg


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tobias Pfeiffer  
View profile  
 More options Oct 7 2012, 5:52 pm
From: Tobias Pfeiffer <tob...@tesobe.com>
Date: Sun, 7 Oct 2012 23:52:32 +0200
Local: Sun, Oct 7 2012 5:52 pm
Subject: Re: [Lift] Building Record interface to REST api

Hi Greg,

Am Freitag, 5. Oktober 2012, 18:02 schrieb pelagic:

> Thanks for you input, I really appreciate you sharing your experience
> on working with RESTful services.

> Could you ellaborate more on what kind of problems you were having

Well, what you might be tempted to do if you want to map a model class
to a REST resource, is the following:

class Article extends LongKeyedMapper[Article] with IdPK {
  lazy val data: NodeSeq = (fetch XML data from RESTful API using id)

  def title = data \ "title" text
  def body = data \ "body" text
  def numberOfCcomments = asLong(data \ "comments" text) getOrElse 0L
  // ...

}

And then in your source code, you just treat that class as a normal
mapper class, as in:

  Article.find(By(Article.id, 7L)).get.title

and let's assume the ".get" is not a problem because you know there is
an entry with id 7, then this will work fine – if your backend is up and
running. If it takes a long time to respond, then the statement blocks,
and if it returns a response code that your code is not prepared to deal
with (for example, Http(myRequest as_str) in dispatch 0.8.x), then you
will get an exception. If the distributedness of your data store is
abstracted away and you treat Article as any other Mapper/Record class,
then you easily step into that trap.

In a second project where I was more aware of that issue, I defined my
REST-mapped model classes more like this:

class Product extends LongKeyedMapper[Product] with IdPK {
  lazy val data: Box[NodeSeq] = tryo {
    (try to fetch XML data using id)
  }

  def title = data.map(_ \ "name" text).getOrElse("missing title")
  def price = asDouble(data.map(_ \ "list_price" text).getOrElse("0.0")
).getOrElse(0.0)
  // ...

}

Here, I still have the problem of blocking, but I get around this by
LazyLoading snippets that show product data as much as possible. On the
other hand, if the service is down or returns an unexpected response
code, my page will still work fine, just display "missing title"
everywhere. Just need to make sure that nobody buys three items "missing
title" with a total price of 0.0 EUR ;-)

Now I'm also wondering what the best way to model this situation
actually is. Right after I sent my last email, I discovered that in
dispatch 0.9.x, network delays and failures are dealt with by wrapping
responses to HTTP requests into Promise objects. So maybe (!) one of the
following is the "right" way to do it?

class ProductOption extends LongKeyedMapper[ProductOption] with IdPK {
  lazy val data: Promise[Option[xml.Elem]] =
    Http(myRequest OK as.xml.Elem).option

  def title: Promise[Option[String]] = data.map(_.map(_ \ "name" text))
  def price: Promise[Option[Double]] = data.map(_.map(_ \ "price"
text).flatMap(asDouble))

}

OR:

class ProductEither extends LongKeyedMapper[ProductEither] with IdPK {
  lazy val data: Promise[Either[String, xml.Elem]] = {
    val res = Http(myRequest OK as.xml.Elem).either
    for (exc <- res.left) yield "ERROR: " + exc.getMessage
  }

  def title: Promise[Either[String, String]] = {
    def extractTitle(xml: scala.xml.Elem): Either[String, String] = {
      val nodes = xml \ "title" map (_.text)
      nodes.headOption.toRight("invalid data")
    }
    for {
      xml <- data.right
      t <- Promise(extractTitle(xml)).right
    } yield t
  }

  def price: Promise[Either[String, Double]] = {
    def extractPrice(xml: scala.xml.Elem): Either[String, Double] = {
      val nodes = xml \ "price" map (_.text) flatMap (asDouble)
      nodes.headOption.toRight("invalid data")
    }
    for {
      xml <- data.right
      t <- Promise(extractPrice(xml)).right
    } yield t
  }

  // ...

}

Both of these methods neither block nor do they throw exceptions if
something goes wrong in the backend. In particular the latter method
always keeps the error message, in case you want to log or display it.
However, you haven't dealt with updating or saving your data yet... ;-)

Tobias

  signature.asc
< 1K Download

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tobias Pfeiffer  
View profile  
 More options Oct 8 2012, 3:03 am
From: Tobias Pfeiffer <tob...@tesobe.com>
Date: Mon, 8 Oct 2012 09:02:44 +0200
Local: Mon, Oct 8 2012 3:02 am
Subject: Re: [Lift] Building Record interface to REST api

Hi again,

I just realized that actually using Lift's Box gives you the best of the
Option and Either world, as in:

class ProductBox extends LongKeyedMapper[ProductBox] with IdPK {
  lazy val data: Promise[Box[xml.Elem]] = {
    val res = Http(myRequest OK as.xml.Elem).either
    res.map(r => r match {
      case Left(t) => Failure(t.getMessage, Full(t), Empty)
      case Right(x) => Full(x)
    })
  }

  def title: Promise[Box[String]] = data.map(_.map(_ \ "name" text))
  def price: Promise[Box[Double]] = data.map(_.map(_ \ "price"
text).flatMap(asDouble))
  // ...

}

Kind regards
Tobias

  signature.asc
< 1K Download

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
pelagic  
View profile  
 More options Oct 11 2012, 6:12 pm
From: pelagic <ventis...@gmail.com>
Date: Thu, 11 Oct 2012 15:12:23 -0700 (PDT)
Local: Thurs, Oct 11 2012 6:12 pm
Subject: Re: [Lift] Building Record interface to REST api

Hey Tobias,

Thanks a ton for the response. Using a Promise seems like a slick way of
handling all of the connections to the backend. My current approach is
similar to the one where you wrap everything in a Box, expect I'm not using
the non-blocking io.

Thanks again,
Greg


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »