Automatic handling of Futures in a Lift request?

[Jeppe Nejsum Madsen]

Hi

As we start to compose our  app into more and more services, many being called asynchronously, I was wondering it it made sense to build something like this into Lift proper?

For instance we have something like this for downloading files from S3 (using Akka Futures):

".view [onclick]" #> SHtml.ajaxInvoke( () => {
        tryo (Await.result(fileUploadService.downloadURL(qs.fileId), Duration(20, TimeUnit.SECONDS))).map {url =>
          JsCmds.RedirectTo(url.toExternalForm())
        } openOr {
          S.error("Error generating download URL")
          JsCmds.Noop
        }


it would be nice to be able to just say something like:

".view [onclick]" #> SHtml.ajaxInvoke( () => {
        fileUploadService.downloadURL(qs.fileId), Duration(20, TimeUnit.SECONDS))).map {JsCmds.RedirectTo(_.toExternalForm())}
        })
 

and have Lift serve  the Future[JsCmd], when completed,  as response to the ajax request?

More generally, Lift could handle any Future[LiftResponse]. Error handling needs to be considered, but could configured in LiftRules (although I can't see how to get per-request error info)

Not sure how hard it would be to add :-)

WDYT?

/Jeppe

[David Whittaker]

Jeppe,

It's interesting.  If multiple SFuncs on a page returned Future[JsCmd], I suppose they could be combined pretty easily, and if some returned JsCmd itself, those could be wrapped in a pre-satisfied Future.  I've got to wonder though, if you'd doing asynchronous IO, why not just use comet and issue a partialUpdate when the data is available?

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
 
 

[Jeppe Nejsum Madsen]

On Fri, Nov 16, 2012 at 4:09 PM, David Whittaker <da...@iradix.com> wrote:

Jeppe,

It's interesting.  If multiple SFuncs on a page returned Future[JsCmd], I suppose they could be combined pretty easily, and if some returned JsCmd itself, those could be wrapped in a pre-satisfied Future.  I've got to wonder though, if you'd doing asynchronous IO, why not just use comet and issue a partialUpdate when the data is available?

Don't know :-) Not sure if it always fits or perhaps I just need to think harder.

In this case, the user clicks a download link (there could be many (100s)) this sends off an Ajax request which calls out (async) to S3 to generate a URL. When S3 returns the generated URL, it is passed back to the client. Just seems simpler with straight forward Ajax since, from a UI perspective, it is a blocking operation.

/Jeppe

[David Whittaker]

Hmmm..... so is the perceived benefit that rather than blocking while each FuncHolder is executed, Lift could execute them and then block on one Future that represented them all?  Or are you thinking that this would mainly lead to cleaner code for calls like your example?  I think a possible issue with this is that there are so many ways to represent an Async response.  You are using Akka, Lift has it's own Future implementation, I've been using a lot of the new dispatch lately, which has it's own Promise type.....

[Jeppe Nejsum Madsen]

On Fri, Nov 16, 2012 at 5:02 PM, David Whittaker <da...@iradix.com> wrote:

On Fri, Nov 16, 2012 at 10:20 AM, Jeppe Nejsum Madsen <je...@ingolfs.dk> wrote:

On Fri, Nov 16, 2012 at 4:09 PM, David Whittaker <da...@iradix.com> wrote:

Jeppe,

It's interesting.  If multiple SFuncs on a page returned Future[JsCmd], I suppose they could be combined pretty easily, and if some returned JsCmd itself, those could be wrapped in a pre-satisfied Future.  I've got to wonder though, if you'd doing asynchronous IO, why not just use comet and issue a partialUpdate when the data is available?

Don't know :-) Not sure if it always fits or perhaps I just need to think harder.

In this case, the user clicks a download link (there could be many (100s)) this sends off an Ajax request which calls out (async) to S3 to generate a URL. When S3 returns the generated URL, it is passed back to the client. Just seems simpler with straight forward Ajax since, from a UI perspective, it is a blocking operation.

Hmmm..... so is the perceived benefit that rather than blocking while each FuncHolder is executed, Lift could execute them and then block on one Future that represented them all?  Or are you thinking that this would mainly lead to cleaner code for calls like your example?  I think a possible issue with this is that there are so many ways to represent an Async response.  You are using Akka, Lift has it's own Future implementation, I've been using a lot of the new dispatch lately, which has it's own Promise type.....

Mostly for cleaner code. I haven't looked at the new LiftServlet code but it might also be possible to release the connection thread while waiting for the Future to complete, thus improving scalability. Right now a thread a handling an ajax request is tied up (I think?) while waiting for the async call to complete.

Yeah the multiple Future implementations might be a challenge. But may be somewhat solved in 2.10 or by using type classes.

/Jeppe

[Antonio Salazar Cardozo]

My biggest concern is definitely that we don't want to depend on akka just to support their futures in a syntax like this.

I like the idea of supporting futures, particularly if we can find a way to do it with continuations (right now you're right, the AJAX pipeline will tie up a thread waiting for it). As it currently stands, the path that is wrapped in continuations is the comet path. In a perfect world, we can add a way to send down new contentless comets to the client (maybe even comets with content, who knows) via AJAX. Then a response wrapped in a Future may trigger an AJAX response that sends down an anonymous comet that hooks into the future's success callback.

Thanks,

Antonio

[David Pollak]

The early implementations of Lift (well, Scala with Sails) were all Actor-based and didn't consume threads unless rendering was taking place. It mostly was a penalty.

I doubt you're going to have 500+ open Ajax requests on a single server (the point at which the threading issues become issues). So, I wouldn't want to mess around too much with trying to make some requests served Async and some synchronous. It's likely to be more damaging to performance and maintainability than it's worth.

There's an existing mechanism that I introduced as part of Wiring that allows you to collect the JsCmds to piggy-back onto a response. That mechanism could be used to deal with Futures such that you could have lots of Futures pending for a response and the response would not be served until the futures are returned.

I've been working on mechanisms for unifying Lift's rendering with Streams/Futures and Comet as part of Lift 3.0. I'm noodling right now on a mechanism that's not as heavy-weight as Lift's comet stuff, but still allows push as data becomes available. I'm also noodling with browser-based Actors that do asynchronous messaging between the client and the server. I'll work this requirement (sending the results of futures back to the browser) into my design thoughts.

 

/Jeppe

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
 
 

--

Telegram, Simply Beautiful CMS https://telegr.am

Lift, the simply functional web framework http://liftweb.net

Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im

[Jeppe Nejsum Madsen]

On Fri, Nov 16, 2012 at 7:06 PM, Antonio Salazar Cardozo <savedf...@gmail.com> wrote:

My biggest concern is definitely that we don't want to depend on akka just to support their futures in a syntax like this.

Agreed. But at least in 2.10 the Akka Future is in the stdlib :-)

/Jeppe

[Jeppe Nejsum Madsen]

I doubt you're going to have 500+ open Ajax requests on a single server (the point at which the threading issues become issues). So, I wouldn't want to mess around too much with trying to make some requests served Async and some synchronous. It's likely to be more damaging to performance and maintainability than it's worth.

There's an existing mechanism that I introduced as part of Wiring that allows you to collect the JsCmds to piggy-back onto a response. That mechanism could be used to deal with Futures such that you could have lots of Futures pending for a response and the response would not be served until the futures are returned.

Agreed. The ability to not tie up a thread is not a major issue (for us at least), just a nice side effect. But using Futures provide other benefits not directly tied to scalability. Ie. they compose nicely  and clearly demarcate when async stuff is happening etc

I've been working on mechanisms for unifying Lift's rendering with Streams/Futures and Comet as part of Lift 3.0. I'm noodling right now on a mechanism that's not as heavy-weight as Lift's comet stuff, but still allows push as data becomes available. I'm also noodling with browser-based Actors that do asynchronous messaging between the client and the server. I'll work this requirement (sending the results of futures back to the browser) into my design thoughts.

Sounds interesting. Is this in the lift_30 branch?

/Jeppe

[David Pollak]

I have serious concerns about Lift depending on Akka, Scala Actors and other stuff that has in the past had significant changes and performance problems. It's possible to wrap Lift's futures, etc. around other stuff with implicit conversions that are external to Lift.

 

/Jeppe

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
 
 

--

Telegram, Simply Beautiful CMS https://telegr.am

Lift, the simply functional web framework http://liftweb.net

[David Pollak]

Not yet. I'd swirling in my head and at some point in the next few weeks, it will vomit out of my fingers.

 

/Jeppe

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
 
 

[Naftoli Gugenheim]

On Sat, Nov 17, 2012 at 2:31 PM, David Pollak <feeder.of...@gmail.com> wrote:

out of my fingers.

To the list please?

[Antonio Salazar Cardozo]

I'm guessing to the codebase, so to github.

[Naftoli Gugenheim]

That was a request. I really believe a lot of things in Lift would turn out a lot better (not that they aren't excellent) if they would be planned collaboratively. Once something's in code, it's much harder to change it.

--

[Peter Petersson]

Here is a fairly resent podcast on Scala 2.10 Futures & Promises [1]
Lift is mentioned as a framework that can benefit from using it (in the slideds).


[1] http://skillsmatter.com/podcast/scala/futures-and-promises-a-new-take-on-concurrency-in-scala-2-10

2012/11/22 Naftoli Gugenheim <nafto...@gmail.com>

[Antonio Salazar Cardozo]

That's funny, I helped someone come up with an example of how Lift can already do everything in that presentation in the same amount or slightly less code. And IMO it's more expressive code to boot.

Antonio

[Naftoli Gugenheim]

Can you show the two code snippets in question? Also is the Lift code as composable?

[Antonio Salazar Cardozo]

Depends on which parts you're referring to. To be clear, I'm talking about the web part at the end that they did with Play. The futures themselves are a different matter, and I haven't really looked into how Lift's futures do or don't compare (though I seem to recall they don't support the for comprehension). Since Lift's domain is the web, I don't think the rest is really germaine (as in how could Lift benefit from it any more than anyone else?).

Antonio

[David Pollak]

On Wed, Nov 28, 2012 at 2:38 PM, Antonio Salazar Cardozo <savedf...@gmail.com> wrote:

Depends on which parts you're referring to. To be clear, I'm talking about the web part at the end that they did with Play. The futures themselves are a different matter, and I haven't really looked into how Lift's futures do or don't compare (though I seem to recall they don't support the for comprehension)

Lift Futures support for comprehensions.

--

Telegram, Simply Beautiful CMS https://telegr.am

Lift, the simply functional web framework http://liftweb.net