Should I use Play or Lift for doing web development in Scala?

[Shannon -jj Behrens]

I posted this question on Stack Overflow:
http://stackoverflow.com/questions/3670237/should-i-use-play-or-lift-for-doing-web-development-in-scala

I'm stuck on whether I should focus on Play or Lift for doing web
development in Scala.

Play looks very polished. The Scala-specific tutorial looks amazing.
Furthermore, since I've been coding in MVC frameworks for a long time,
it looks super familiar. However, it doesn't look like it can easily
accomplish all the brilliant things that Lift can do. For instance, I
can't find anywhere where it mentions Comet or Jetty Continuations.
Furthermore, I kind of like the "View First" methodology in Lift
because instead of using one controller, it lets me use a ton of
snippets to piece together a page.

Lift looks brilliant, but leaves me with a lot of questions
unanswered. Being highly stateful looks like it opens up a lot of
possibilities, but I wonder how it'll turn out in practice. The book
on Lift is a bit of a mess, and so is the wiki. The "Getting Started"
page is badly formatted and is no match for Play's tutorial.

Does Play support Jetty Continuations?

Is it painful to get started with Lift doing normal web application development?

How does Lift's statefulness work out in practice? How do you cope
with web servers going down in Lift? If I'm using Lift, and I push a
new version of my code on a daily basis, does that mean I have to
restart the application, and does that mean everyone's session gets
wiped out?

Does Lift's statefulness actually make it easier to code?

What happens if someone messes around with the back button in Lift?
What happens if a user is bouncing back and forth between several
tabs?

Thanks, guys!
-jj

--
In this life we cannot do great things. We can only do small things
with great love. -- Mother Teresa
http://jjinux.blogspot.com/

[Miles Egan]

I don't have an answer for you but, depending on what it's worth to
you, you should consider attending the Scala Lift Off in S.F. later
this month:

http://scalaliftoff.com/

You'll probably get a much better idea of what development is like. I
intend to go.

On Wed, Sep 8, 2010 at 4:50 PM, Shannon -jj Behrens <jji...@gmail.com> wrote:
> I'm stuck on whether I should focus on Play or Lift for doing web
> development in Scala.

--
miles

[Csaba Csoma]

Shannon,

I'm pretty happy with Lift. Once you get Scala the Lift part is not that hard.
The mailing list is awesome.

I would not worry about scalability.
Foursquare has over 3 million users on it with over 18,000 new signups/day: http://startupdata.rjmetrics.com/

Back button, multiple tabs etc. - all are working smoothly and it's very stable.
This was my first questions too. All the demos made it look like "it just works" and I was skeptical.

More on Lift: https://www.assembla.com/wiki/show/liftweb/Resources

Csaba

On Wed, Sep 8, 2010 at 4:50 PM, Shannon -jj Behrens <jji...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "Bay Area Scala Enthusiasts" group.
To post to this group, send email to scala...@googlegroups.com.
To unsubscribe from this group, send email to scala-base+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scala-base?hl=en.

[David Pollak]

Lift is better than Play if you care about the following:

• Security.  Lift is far more secure than Play.
• Scalability.  Lift apps have demonstrated pretty spectacular scaling (Foursquare is one example).
• Comet.  Lift's Comet support is the best of any web framework.
• Ajax.  Lift's Ajax support is the most concise and secure of any web framework.
• Multi-page input forms with back-button support.
• REST support.  Lift's REST support is concise and type safe.
• Scala support.  Lift takes full advantage of Scala including its type system, pattern matching, monads, etc.

On Wed, Sep 8, 2010 at 4:50 PM, Shannon -jj Behrens <jji...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "Bay Area Scala Enthusiasts" group.
To post to this group, send email to scala...@googlegroups.com.
To unsubscribe from this group, send email to scala-base+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scala-base?hl=en.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

[mkaz]

I think it comes down to personal preference to a degree. The Play
Framework was a very familiar structure for me, I come from a PHP
background, not Spring.

I found Play easy to setup and get going. I don't think it has as many
built-in features as Lift, but what it does have is documented well
and just works. I really like how tests are built-in, both Unit and
Selenium tests, makes it very easy to write and run. Plus I like the
template syntax it is quite nice and powerful.

My advice, try a demo app on each and see what fits to your liking.
Might be worth spending a day or two on each before "committing"

Hope it helps,
-Marcus


On Sep 8, 4:50 pm, Shannon -jj Behrens <jji...@gmail.com> wrote:
> I posted this question on Stack Overflow:http://stackoverflow.com/questions/3670237/should-i-use-play-or-lift-...

[David Taylor]

David,

While I'm not disagreeing with you, I feel like I need to add a little "[citation needed]" to some of those very broad statements, especially "Lift is far more secure than Play." In what ways? Due to what design decisions? Examples?

To make that statement, as opposed to something like just "Lift is very secure", I assume you have some sort of reason to believe Play is _less_ secure than Lift, and I'd be interested to know what it is.

Thanks,

David T

On Wed, Sep 8, 2010 at 11:13 PM, David Taylor <tinystat...@gmail.com> wrote:

While I'm not disagreeing with you, I feel like I need to add a little "[citation needed]" to some of those very broad statements, especially "Lift is far more secure than Play." In what ways? Due to what design decisions? Examples?

To make that statement, as opposed to something like just "Lift is very secure", I assume you have some sort of reason to believe Play is _less_ secure than Lift, and I'd be interested to know what it is.

Thanks,

David

[David Pollak]

On Wed, Sep 8, 2010 at 8:14 PM, David Taylor <maxim...@gmail.com> wrote:

David,

While I'm not disagreeing with you, I feel like I need to add a little "[citation needed]" to some of those very broad statements, especially "Lift is far more secure than Play." In what ways? Due to what design decisions? Examples?

Lift is more secure than Play because:

• Lift associates each user action with a GUID on the client and the function on the server.  This means that Lift apps are resistant to replay attack, cross site request forgeries, and vulnerabilities associated with leaking DB primary keys (aka insecure object references).
• Lift keeps the page around as XML during page composition.  This means that the developer has to do something affirmative to introduce a cross site scripting vulnerabilities.
• Lift's sitemap enforces access control to each page of the application, so Lift apps are generally resistant to URL access vulnerabilities.

If you take a look at the OWASP top 10 security vulnerabilities, Lift's defaults are resistant to most of the listed vulnerabilities (Lift doesn't address cryptographic storage [encrypting data at rest] or transportation layer issues [SSL]).

[David Pollak]

On Wed, Sep 8, 2010 at 9:01 PM, David Pollak <feeder.of...@gmail.com> wrote:

On Wed, Sep 8, 2010 at 8:14 PM, David Taylor <maxim...@gmail.com> wrote:

David,

While I'm not disagreeing with you, I feel like I need to add a little "[citation needed]" to some of those very broad statements, especially "Lift is far more secure than Play." In what ways? Due to what design decisions? Examples?

Lift is more secure than Play because:

• Lift associates each user action with a GUID on the client and the function on the server.  This means that Lift apps are resistant to replay attack, cross site request forgeries, and vulnerabilities associated with leaking DB primary keys (aka insecure object references).
• Lift keeps the page around as XML during page composition.  This means that the developer has to do something affirmative to introduce a cross site scripting vulnerabilities.
• Lift's sitemap enforces access control to each page of the application, so Lift apps are generally resistant to URL access vulnerabilities.

If you take a look at the OWASP top 10 security vulnerabilities, Lift's defaults are resistant to most of the listed vulnerabilities (Lift doesn't address cryptographic storage [encrypting data at rest] or transportation layer issues [SSL]).

Not to put too fine a point on it, but Yahoo! chief security guy couldn't find a single vulnerability in Foursquare's site and they were not coding for security.  See http://twitter.com/rasmus/status/5929904263

[Shannon -jj Behrens]

Hey David,

Thanks for the response. Lift looks like a fantastic framework. I
knew that already because I've a) seen you speak in person b) read all
the docs c) read your blog. However, I'm still struggling with a few
things.

Let me use an analogy. If someone sets you up on a blind date, and
all you keep hearing is "She's got a really great personality," it'll
make you really start wondering what the girl looks like. That's how
I feel about Lift. I can't seem to find anywhere a description of the
things Lift isn't so great at and how it copes with those things.

For instance, statefulness is a big deal. I've read on your blog that
you say, "statelessness is fail." I get that. However, I can't find
anywhere a description of the challenges imposed by statefulness and
session affinity. All I'm looking for is an honest assessment of the
pros *and* cons of Lift.

It's no small feat that the documentation and tutorial for Play look
so well done. I keep thinking that it'd be great if someone simply
ported the Play tutorial to Lift. Even though I've read most of the
Lift wiki, I don't feel like I'm capable of doing it. I.e. I fear
that I might not get it right.

I can't find an FAQ for Lift, so there are a lot of things I still
don't have a grasp on. I tried to mention some of these in my
original email:

* Based on what I've read, learning Lift is harder than learning RoR
or Play. How much harder? How long will it take me before I start
feeling comfortable in it? (Note, I'm no newbie to web frameworks.
I've used most of the Python frameworks available at one time or
another, and I've been using RoR for the last year.)

* How does Lift's statefulness work out in practice? How do you cope

with web servers going down in Lift? If I'm using Lift, and I push a
new version of my code on a daily basis, does that mean I have to
restart the application, and does that mean everyone's session gets
wiped out?

* How much state is kept in the database and how much state is kept on
the server? Do I have to worry about the state on the server being
stale compared to what's in the database? For instance, it seems
likely that I'll use state from the database when generating a form.
It seems possible that that state in the database might change by the
time the user submits the form. However, because of the way the
server is stateful, I'll have local variables on the server
referencing the old copy of the data when I'm handing the form
response. How do I cope with that?

* Does Lift's statefulness actually make it easier to code? How much easier?

* What happens if someone messes around with the back button in Lift?

What happens if a user is bouncing back and forth between several

tabs? What happens when a Mac user has both Firefox and Safari going
at the same time, playing around on the site? Earlier on this thread,
someone said that things do work with multiple tabs, so that makes me
feel better, but I'm still a little concerned. Let's imagine a user
is using my site. They're playing around with some data. They edit
the data. Then they hit the delete button to delete the data. Then
they hit the back button a few times and edit the data again. This is
a typical web application problem. I understand how to cope with this
with most web frameworks, but what happens with Lift? If the server
is stateful, what's going to happen when my form handler tries to edit
data that the user has actually already deleted?

* If someone is using Lynx or has JavaScript turned off (thus
disabling the heartbeat that Lift uses), and it takes them an hour to
fill in a credit card form (because their kids keep interrupting them
like mine interrupt me), when they submit the page, will it still
work? I'm worried that either a) sessions will get reaped too quickly
b) if sessions aren't reaped quickly enough, a simple web spider can
cause my web application to go into swap because of too many abandoned
sessions (I've seen this question partially addressed elsewhere).

* Lift's ORM doesn't look 100% polished. How does it compare to
ActiveRecord and JPA? What does it lack? How are people handling
migrations (someone gave me a link to
http://code.google.com/p/scala-migrations/ earlier)?

* I've read the Getting Started guide, and it looks cool. However,
what happens if you have so much traffic that a single machine can't
cut it? When I talked to you in person, you mentioned something about
some feature in the Java world that allows servers to cluster with
each other, but I'm really trying to wrap my mind around the
limitations and challenges. Nothing's perfect. I'm just trying to
understand the "backup plan" for when a single server can't cut it.
For instance, in the RDBMS world, I know the backup plan is to shard,
and I understand that it's a massive pain in the butt.

Having said all that, let me say that I think Lift is a great
framework, and it's probably the one I'll be using. I'm just trying
to find some of its "challenges", and how I can cope with them.

Thanks!
-jj

[David Pollak]

I can't really answer these open ended questions.

I find Lift far easier to code than Rails or any other framework I've used.  Other folks agree or they would not have chosen Lift.  But other people find other frameworks easier to code in.  It all depends on your application and your background and your taste for static vs. dynamic languages and a whole lot of other factors.

In terms of Lift and older browsers, Lift doesn't support them (or at least it doesn't support them well.)  At the end of the day, in order to use a Lift app, you need a modern browser (IE 6+, Firefox 1.5+ or WebKit-based [Chrome, Safari]).

In terms of persistence, Lift is persistence agnostic.  Use whatever you want... Lift doesn't care how the objects are materialized into the JVM.  Use JPA.  Use Mapper (I like Mapper and it does what I need for most of my projects.).  Use Squeryl.  Use Record/MongoDB.  Use case classes and MongoDB.  It makes no difference.

There are very few challenges related to statefulness and session affinity running Lift in production.  There's a module in Nginx that "just works."  There's support on HAProxy for session affinity.  Put another way, the challenges at both the code and operations layers for dealing with state in Memcached is far far nastier than putting Nginx and/or HAProxy in front of your Lift app.  But the scale that you have to worry about that is pretty high.  Is your app going to service 100+ requests per second sustained?  If not, you're running on a single instance and you've got not problems.

In terms of "where is state?"  That's up to your application and not proscribed in Lift.  You want to keep stuff in memory, then do it.  You want to put state in an RDBMS and pull it out on a request by request basis, then do it.  It's up to you.  The only place where Lift keeps state around in memory is when you associate a GUID on the client with an action on the server... the action is a Scala function and cannot be serialized.

With all this being said, if you have *specific* questions backed by a concrete description of your application, bring them to the Lift list.  Giving good answers to abstract questions just does yield good results.

[Miles Egan]

Has it been your experience that Lift apps can entirely do away with
the kinds of caching layers people usually implement in Memcached or
does it just delay the point at which they become necessary?

On Thu, Sep 9, 2010 at 10:22 AM, David Pollak
<feeder.of...@gmail.com> wrote:
> Put another way, the challenges at both the code and operations layers for
> dealing with state in Memcached is far far nastier than putting Nginx and/or
> HAProxy in front of your Lift app.

--
miles

[Shannon -jj Behrens]

I'm curious--a couple people on Stack Overflow mentioned the idea of
using Play and Akka together. Has anyone here tried that?

Thanks,
-jj

[David Pollak]

On Thu, Sep 9, 2010 at 10:24 AM, Miles Egan <mile...@gmail.com> wrote:

Has it been your experience that Lift apps can entirely do away with
the kinds of caching layers people usually implement in Memcached or
does it just delay the point at which they become necessary?

I've never seen memcached used with a Lift app.  They generally aren't necessary.  If you need to pre-compute a value (e.g., when Lift loads a template from the WAR file in production mode, it's cached in an LRU cache so it doesn't have to be re-parsed on each request), you can keep it around in the JVM and have much better rules about cache consistency, invalidation, cache recompution (avoiding cache stampedes when 10 threads/processes are recomputing the same popular cache item), etc.

Also, with Scala's immutable data structures, caching is much, much easier because you don't have to make defensive copies of a cached item each time you vend it.

Put another way, with 99%+ of the apps people are writing in Lift, they will live in a single JVM.  The stability of the JVM is on par with the stability of memcached, so it's not like a Rails process with needs to be killed every 5 minutes to avoid the MRI from sucking down all the swap space on your machine.

 

On Thu, Sep 9, 2010 at 10:22 AM, David Pollak

<feeder.of...@gmail.com> wrote:

> Put another way, the challenges at both the code and operations layers for
> dealing with state in Memcached is far far nastier than putting Nginx and/or
> HAProxy in front of your Lift app.

--
miles

--

You received this message because you are subscribed to the Google Groups "Bay Area Scala Enthusiasts" group.
To post to this group, send email to scala...@googlegroups.com.
To unsubscribe from this group, send email to scala-base+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scala-base?hl=en.

[Will Sargent]

On Thu, Sep 9, 2010 at 10:03 AM, Shannon -jj Behrens <jji...@gmail.com> wrote:

Hey David,

Thanks for the response.  Lift looks like a fantastic framework.  I
knew that already because I've a) seen you speak in person b) read all
the docs c) read your blog.  However, I'm still struggling with a few
things.

There's a Scala BASE session on the 13th that going to be about getting new people up to speed with Scala -- it's not specifically about Lift, but you might find it useful.

Will.

[Miles Egan]

That makes sense. A big part of the argument in favor of Memcached is
that cache can be shared by multiple front-end servers. Would it be
fair to say that the worker/cache ratio you see in something like a
rails app is equalled or surpassed by the ratio you see in a single
JVM? In other words, does a shared JVM give you similar overall
caching benefits?

I can definitely see the greater simplicity of dumping a separate
caching layer in the infrastructure.

On Thu, Sep 9, 2010 at 10:33 AM, David Pollak
<feeder.of...@gmail.com> wrote:
> Put another way, with 99%+ of the apps people are writing in Lift, they will
> live in a single JVM.  The stability of the JVM is on par with the stability
> of memcached, so it's not like a Rails process with needs to be killed every
> 5 minutes to avoid the MRI from sucking down all the swap space on your
> machine.

--
miles

[Shannon -jj Behrens]

> In terms of Lift and older browsers, Lift doesn't support them (or at least it doesn't support them well.) At the end of the day, in
> order to use a Lift app, you need a modern browser (IE 6+, Firefox 1.5+ or WebKit-based [Chrome, Safari]).

Uh, that's kind of a big deal ;) That's exactly the sort of thing I
need to know before I try to use a new framework on a new project. I
don't remember reading that in the Lift documentation. That'd be the
perfect sort of thing to have in a FAQ.

> Put another way, with 99%+ of the apps people are writing in Lift, they will
> live in a single JVM.

That's also the sort of thing I was looking for. The fact that most
Lift users haven't experienced the challenges of using Lift with
multiple web servers is worth knowing. Performance != scalability. I
was looking for rules of thumb such as "If you have multiple servers
running Lift, make sure you keep persistent data in the database and
beware of stale cached data on the web servers. Otherwise, if a user
has two browsers up, and the browsers are each affiliated with
different Lift servers, one of the browsers might see stale data."

Look, Lift is fantastic, but a project is defined as much by what it
doesn't do as what it does do. I'm just trying to get my head around
what it doesn't do.

Thanks,

[Will Sargent]

On Thu, Sep 9, 2010 at 10:54 AM, Shannon -jj Behrens <jji...@gmail.com> wrote:

> In terms of Lift and older browsers, Lift doesn't support them (or at least it doesn't support them well.)  At the end of the day, in
> order to use a Lift app, you need a modern browser (IE 6+, Firefox 1.5+ or WebKit-based [Chrome, Safari]).

What's the limiting factor on older browsers?  AJAX / XHTML support?

Will.

[Mateo Barraza]

Will, can someone record the getting upto speed with Scala session on
the 13th. It would be great for people miles aways that can not
attend.

M>

[David Pollak]

On Thu, Sep 9, 2010 at 10:54 AM, Miles Egan <mile...@gmail.com> wrote:

That makes sense. A big part of the argument in favor of Memcached is
that cache can be shared by multiple front-end servers. Would it be
fair to say that the worker/cache ratio you see in something like a
rails app is equalled or surpassed by the ratio you see in a single
JVM? In other words, does a shared JVM give you similar overall
caching benefits?

So, this gets into your overall architecture.  If you've got a 2 tiered app with nothing shared at the app tier, then you need to put cached stuff somewhere.  The RDBMS is typically a bad place for that (both because of performance and the structured nature of the RDBMS), so memcached is a non-stupid place to store intermediate results.

So, if you've got a medium traffic site running on Rails, then you're going to have 4-10 front end web app servers.  This is because Rails needs the entire address space per request being services (yeah, yeah, there's some multi-threaded Rails foo, but it doesn't work well).  A Rails instance will consume about 250MB once it's warmed up (all the classes have been loaded).  This means in your typical web server, you can have have about 6 Rails instances.  Rails requests typically take 500ms to service, which means if you've got traffic of 20 requests per second peaking at 100 requests per second, you need 50 Rails instances warmed up to keep your site responsive.

The rule of thumb I've seen with memcached and Rails is that you need about 2 Rails boxes per memcached box, so in this configuration, you're looking at 15 or so boxes running Rails and memcached.

The same level of traffic can be served with a single Lift instance running with 1.5GB of heap and all caching done in the JVM (yes, you've got less memory dedicated to cache, but you can typically cache less with a Lift app because the cost of calculating content is typically 100 times faster with the JVM than with Ruby.)

But once you're at the point where you are pre-computing/caching significant data (rather than simply caching the home page to a CNET-like site), you should be moving to a multi-tiered architecture.  The cached items (other than cached markup which will consume a relatively small amount of space) will be cached by your back-end services.  This is what Akka and Stambecco do particularly well.  So, rather than working about keeping everything in the web tier, you distribute the computation and caching out into a services that speak asynchronous messages.

 

I can definitely see the greater simplicity of dumping a separate
caching layer in the infrastructure.

On Thu, Sep 9, 2010 at 10:33 AM, David Pollak

<feeder.of...@gmail.com> wrote:

> Put another way, with 99%+ of the apps people are writing in Lift, they will
> live in a single JVM.  The stability of the JVM is on par with the stability
> of memcached, so it's not like a Rails process with needs to be killed every
> 5 minutes to avoid the MRI from sucking down all the swap space on your
> machine.

--

miles

--
You received this message because you are subscribed to the Google Groups "Bay Area Scala Enthusiasts" group.
To post to this group, send email to scala...@googlegroups.com.
To unsubscribe from this group, send email to scala-base+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scala-base?hl=en.

[David Pollak]

Lift does a lot to detect the browser and render the output accordingly.  Lift does not have built in support for every browser, although support can be added (there is a site that did support for older Blackberry and Nokia browsers because there was a business advantage in doing so).  The JavaScript requirement (page-level GC) can be turned off depending on the user agent.

But at the end of the day, doing all that work winds up reducing your site to a lowest common denominator green-screen app.  So, I guess technically, Lift can render any form of response to anything that can speak HTTP, but practically, it's something that you have to work hard for.  So, if you've got the requirement of supporting Lynx and you don't care about Lift's power zone (security, scalability, interactivity, etc.), then start with something that's not going to let you go up, but not cost you anything to server Lynx.

 

Will.

--
You received this message because you are subscribed to the Google Groups "Bay Area Scala Enthusiasts" group.
To post to this group, send email to scala...@googlegroups.com.
To unsubscribe from this group, send email to scala-base+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scala-base?hl=en.

[Shannon -jj Behrens]

Hey David,

I appreciate your taking the time to answer all our questions.

I have a question about continuous deployment. A lot of companies I
know (but none that I've worked for) like to deploy changes to the
live site anytime anything is checked into source control. Some
companies like to just rely on their customers to find all their bugs
(which I consider negligent) whereas some companies have very strong
test suites and push changes to the live site after the code passes
all the tests. Hence, some companies manage to push new versions of
the code 10s of times a day.

Is continuous deployment possible with Lift? I suspect you have to
restart the application every time someone pushes new code to the
server, and I can't imagine how the strong statefulness of Lift can
survive application restarts since actual function references are
stored in the session.

As for scalability, I think you've convinced us all that Lift has
superb *performance*--that it is fast and uses modest amounts of RAM
and CPU. However, I spend a lot of my time thinking about horizontal
*scalability*. I.e. if I have 10 servers running Lift, and one of the
servers has a harddrive crash, does that mean every one who had items
in their shopping cart and were affiliated with that particular server
will lose everything in their cart? I'm not saying that that's the
end of the world--I'm just trying to understand the tradeoffs.

Thanks!
-jj

> Lift does a lot to detect the browser and render the output accordingly.
>  Lift does not have built in support for every browser, although support can
> be added (there is a site that did support for older Blackberry and Nokia
> browsers because there was a business advantage in doing so).  The
> JavaScript requirement (page-level GC) can be turned off depending on the
> user agent.
> But at the end of the day, doing all that work winds up reducing your site
> to a lowest common denominator green-screen app.  So, I guess technically,
> Lift can render any form of response to anything that can speak HTTP, but
> practically, it's something that you have to work hard for.  So, if you've
> got the requirement of supporting Lynx and you don't care about Lift's power
> zone (security, scalability, interactivity, etc.), then start with something
> that's not going to let you go up, but not cost you anything to server Lynx.

--

[Miles Egan]

Thanks for the detailed explanation. I'm looking forward to hearing
more about Lift later this month.

On Thu, Sep 9, 2010 at 1:48 PM, David Pollak
<feeder.of...@gmail.com> wrote:
> So, this gets into your overall architecture.

--
miles

[Sean Rhea]

Agreed. As a long-time Rails user and enthusiastic Scala newbie I'm
finding this whole discussion really informative. Thanks to both David
and JJ for continuing it!

Sean

> --
> You received this message because you are subscribed to the Google Groups "Bay Area Scala Enthusiasts" group.
> To post to this group, send email to scala...@googlegroups.com.
> To unsubscribe from this group, send email to scala-base+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scala-base?hl=en.
>
>

--
"I refuse to accept the idea that the ‘isness’ of man’s present nature
makes him morally incapable of reaching up for the eternal ‘oughtness’
that forever confronts him." --MLK

[s t ong]

On Sep 9, 7:22 pm, Shannon -jj Behrens <jji...@gmail.com> wrote:
> Hey David,
>
> I appreciate your taking the time to answer all our questions.
>
> I have a question about continuous deployment.  A lot of companies I
> know (but none that I've worked for) like to deploy changes to the
> live site anytime anything is checked into source control.  Some
> companies like to just rely on their customers to find all their bugs
> (which I consider negligent) whereas some companies have very strong
> test suites and push changes to the live site after the code passes
> all the tests.  Hence, some companies manage to push new versions of
> the code 10s of times a day.

I'm neither a fan of Lift (the tooling is very weak) or David (I
didn't like the flip attitude in his book), but last thing that David
posted to you was:

With all this being said, if you have *specific* questions backed by a
concrete description of your application, bring them to the Lift list.

You've given no information about your application. You keep asking
open ended questions and then saying stupid things like "I'm just
trying to get my head around what it [Lift] doesn't do." If you spent
just 3 seconds telling the group about what you're trying to build or
helping David understand your requirements or even following his
instructions (take this discussion to the Lift list), maybe you'd get
better answers. If you know what you want to accomplish and can
articulate it, then you might get some real answers that you can use
to make a decision about which web framework to use.

I'm also sure that whatever app you're building isn't going to get 5
requests per minute, let alone 100 or more requests per second.
Wasting David's time asking about a server farm of 10 servers doesn't
show very much respect for him. If you're going to deploy to
production 10+ times a day, you're quite simply an idiot and you've
got bigger problems that the statefulness of your app.

[Miles Egan]

Nice attitude.

--
miles

[Ian Kallen]

Continuous deployment is becoming a common practice. While I agree
that this discussion should have moved to the lift list long ago, the
flames are uncalled for. A lot of Shannon's questions were valid, even
if they really seems more appropriate for the Lift group. I'm actually
surprised nobody else has pointed out the peculiar capacity planning
metrics and performance broad brushes that @dpp posted. Nonetheless, I
appreciate the work that @dpp has done with Lift and the time he takes
to work on it with the community.

On Thu, Sep 9, 2010 at 8:42 PM, s t ong <ston...@gmail.com> wrote:
> I'm also sure that whatever app you're building isn't going to get 5
> requests per minute, let alone 100 or more requests per second.
> Wasting David's time asking about a server farm of 10 servers doesn't
> show very much respect for him.  If you're going to deploy to
> production 10+ times a day, you're quite simply an idiot and you've
> got bigger problems that the statefulness of your app.

--
Ian Kallen
blog: http://www.arachna.com/roller/spidaman
tweetz: http://twitter.com/spidaman
vox: 415.505.5208

[Bill Venners]

Hi All,

On Thu, Sep 9, 2010 at 8:42 PM, s t ong <ston...@gmail.com> wrote:
>

I've ever heard of anyone deploying live changes to production ten
times a day, but if someone does do that I would give them the benefit
of the doubt and figure they must have some good reason (i.e., reserve
judgment on the "idiocy" of it until I heard the details). That said,
I was hoping to hear an answer about this question. David, I was
surprised to hear you answer clustering questions with 99% of Lift
apps just need one server because of Lift's performance. I've heard
you give that answer before. The concern I've had for our situation is
deploying software and dealing gracefully with crashes, which is
exactly what JJ brought up here.

No matter what your traffic, when you need to deploy changes, the
question arises can you keep the website running while those changes
are being deployed. Will people in the process of buying something be
kicked off and need to log in again and start over, for example? The
answer I like to hear is yes, you can run two or three instances, and
when one reboots, the other one can take over for the first for the 30
seconds it takes to restart the app server. Once the first one is up,
you can restart the second, etc. This is good because if a server
crashes unexpectedly, it is nice to know that people won't get kicked
out of the shop where they were about to make a purchase. This also
means that whatever state exists needs a way to migrate from one
server to another.

The other thing I like to hear is that I can deploy new code without
restarting the server. It is important to have state migration for
crashes, and the occasional reboot that is likely necessary, but the
JVM has class loaders and it seems like a good use of them is to
support deploying updates.

If Lift's state is in the session, then there are tools to migrate
session data that could probably be applied. Also some app servers may
be able to assist with do live updates, I'm not sure. But I'm curious
if there's a recommended or built-in way to do these things.

Thanks.

Bill

> --
> You received this message because you are subscribed to the Google Groups "Bay Area Scala Enthusiasts" group.
> To post to this group, send email to scala...@googlegroups.com.
> To unsubscribe from this group, send email to scala-base+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scala-base?hl=en.
>
>

--
Bill Venners
Artima, Inc.
http://www.artima.com

[Raoul Duke]

> On Thu, Sep 9, 2010 at 8:42 PM, s t ong <ston...@gmail.com> wrote:
>> show very much respect for him.  If you're going to deploy to
>> production 10+ times a day, you're quite simply an idiot and you've
>> got bigger problems that the statefulness of your app.

http://www.google.com/search?q=imvu+release+production+interview

[David Pollak]

Bill,

I did ask JJ to take his discussion to the Lift list with specifics.  While I disagree with the tone of S T Ong's post, I generally agree with its content (except for the negative comments about me and Lift ;-) )  If he's after answers to concrete questions based on his actual application requirements, I'm sure there's somebody on the Lift list that can help him... and hopefully he won't make the mistake of calling the authors of the Lift Book ADAH in the Lift forum, ignore questions, or make sweeping and unfounded generalities about Lift.

To the substance....

Lift has a mechanism for extended sessions and a mechanism for detecting that a session has gone away and causing the current page to be reloaded.  Those two mechanisms working in concert means that current page state is generally restored on server restart or when you sleep your laptop and unsleep it hours later.  Try it out with Foursquare.

Lift also has a session keep-alive mechanism so that even if you're not doing anything on a page, your session stays alive as long as your browser is open.  This means that you don't get the annoying issue of posting a form to a session that's timed out.

 

No matter what your traffic, when you need to deploy changes, the
question arises can you keep the website running while those changes
are being deployed. Will people in the process of buying something be
kicked off and need to log in again and start over, for example?

Most of the site deployments that I do in production are during well defined maintenance windows in which the entire service is shut down.  I realize that there's a class of services for which that's not acceptable, but the vast majority of sites (the bottom 98% or so) are going to be cool with the maintenance window.

If that's not acceptable, there will be a commercial Lift Cluster Manager that deals with starving sessions on machines that are being "taken down" and forwarding new traffic to the new machines (as well as sending "friendly" traffic to beta servers, etc.)  But, taken in concert with extended sessions, most users will not perceive the switch-over.

So, when I say "most users," I've taken an informal poll of high traffic Lift sites and among the sites that I've talked to, there have been zero, none, nada complaints about lost sessions.  Perhaps it's that most web users are conditioned to the occasional 500 error or session timeout or other examples of things not working, but real world Lift apps that cycle in and out of production are not perceived by users as being complaint-worthy.

 

The
answer I like to hear is yes, you can run two or three instances, and
when one reboots, the other one can take over for the first for the 30
seconds it takes to restart the app server. Once the first one is up,
you can restart the second, etc. This is good because if a server
crashes unexpectedly, it is nice to know that people won't get kicked
out of the shop where they were about to make a purchase. This also
means that whatever state exists needs a way to migrate from one
server to another.

"State" is a big word.  For the most part, one can reconstruct a session pretty easily, unless the state of the session spans multiple screens (e.g., Lift's Wizard).  Yes, I've explored the idea of marking certain forms as "keep alive" such that they will be sent back to the server as they are filled out, kept in persistent backing store and reconstituted in the event that the session shows up on a new machine.  The amount of actual demand for that kind of feature has been zero... the amount of theoretical demand has been high (just like in this thread.)  Put another way, the Lift stateful boogey man generally tends to be premature optimization and has not been demonstrated as an issue in production.

 

The other thing I like to hear is that I can deploy new code without
restarting the server. It is important to have state migration for
crashes, and the occasional reboot that is likely necessary, but the
JVM has class loaders and it seems like a good use of them is to
support deploying updates.

Do you also have hot spares for all your persistent stores?  Do you make sure that your memcached instances keep duplicates so if one goes down, you won't lose state?  These are not idle questions.  Folks worry so much about the web tier and lose site of the other places where failure can take place.

My real-world experience is that JVMs crash very, very infrequently.  Planning for a JVM crash vs. planning for a router crash, a database server crash, etc. has never been fruitful.

Unplanned outages (vs. deploys, which I've covered above) are less than optimal no matter what component (or machine running the component) goes down.  But by the time people are going to care, you've likely moved off a 2 tier architecture and moved to some distributed services based architecture... and in that scenario (see a prior post to this thread), the cost in terms of user experience of reconstructing web-tier state is pretty low.

In terms of continuous deployment, yeah is the current "in vogue" thing just like the LAMP stack being infinitely scalable was a few years ago.  I've found that in practice, continuous deployment only really works for small (3-4 person) teams.  Doing continuous deployment while insuring (1) serialized state can be shared across versions; (2) all message formats are backwards compatible across all your services; (3) there are no db schema changes or there are views that mask the differences; and (4) your new libraries (e.g., the consistent hashing algorithm in your memcached client) don't break any assumptions has in my observation had resulted in more "ooopsss... roll it back, roll is back!!" pain than it's solved with "getting the latest and greatest into production."  As a practical matter, having releases timed to agile iterations with hot deploys only in the event of a material defect or security problem has had the highest success rate.

 

If Lift's state is in the session, then there are tools to migrate
session data that could probably be applied.

Once again, State is a very big word.  Lift has non-serializable state which is the association between GUIDs in the browser and functions on the server.  This state cannot (short of using Terracotta) be migrated outside the current JVM.  The balance of the state in your application is your business and you can migrate/store it however you want.  As discussed above (and in about 50 threads on the Lift list), reconstructing the GUID -> Function state is generally done automatically on session loss (when using extended sessions).

So, in summary, Lift's GUID -> Function mechanism helps Lift apps be far more secure than most (any?) other web frameworks.  This non-migratable state also leads to the ability to create more complex and interactive applications.  The cost of this is session affinity.  As a practical matter, the deployment and crash scenarios that folks raise are premature optimizations because in the real world, nobody notices (or cares to complain) when Lift sessions stop and are restarted elsewhere.  And there's plenty of precedent for Lift's stateful model in WebObjects which powers most of Apple's web properties... what's the uptime of Apple's store or iTunes?

If any of you have real-world applications you're planning to build with Lift and have specific requirements (rather than generic architecture astronauting), please bring them to the Lift list.  There are plenty of folks from the likes of Xerox, Novell, Foursquare, Innovation Games, etc. that can give you real-world assessments of the risks of a particular issue with Lift.  I am also available to do consulting to help you meet real-world SLAs that you need to meet, there are plenty of things that can be done beyond what Lift does out of the box.

Thanks,

David

--

[Bill Venners]

Hi David,

Thanks for the detailed reply. You answered my questions.

Bill