Postmortem thoughts on a small production Lift site
cody koeninger
www.goldenfrog.com/) using Lift. Thought I'd offer up a postmortem on
what worked well and what didn't, in case it's useful to anyone
considering Lift. We're a small team with essentially no prior Scala
experience, so take this with a grain of salt & feel free to offer
suggestions.
The good:
We went with Lift mainly because we wanted: 1. clean separation of
code from html; 2. easy localization. With a few minor caveats (see
below), we got that.
An html designer was able to work on templates & commit directly to
our repo without touching any scala code. When he broke things, it
was usually immediately obvious because the xml stopped parsing
correctly and the page would no longer render (even though it usually
took a programmer to interpret the stacktrace & fix the xml).
Regarding localization, the use of separate templates (instead of e.g.
java properties file references for every single sentence) was a big
help. Because everything in templates is well-formed xml, I was able
to write less than 100 lines of code to automatically extract
translatable text and generate a translated version of all the
templates. This is done during build, so essentially duplicate html
doesn't get into our repo, until some point when translated templates
need to actually be structurally different.
The bad:
Dreamweaver doesn't deal with head merge, it will automatically move
the head tags outside of the lift:surround. Not really a lift issue.
PCDataXmlParser converts entity references to actual utf-8
characters. This means templates with entity references won't round-
trip correctly through view source, nor through localization
properties files / gettext files.
Jetty seems to do questionable things with filehandles, opening
multiple copies of the same template file and keeping them open too
long. Setting ulimit "fixes" the issue of "Too many open files"
exceptions & failed requests, but it still seems like at most
templates should need 1 access to check mtime. Not really a lift
issue, and not a deal breaker for us to switch to a different servlet
container, but I am curious to see what the eventual solution for
comet performance is.
Lift doesn't seem to have any central place to handle URL issues
related to servlet context paths and domains / subdomains. We're
currently running lift as a root servlet to avoid forcing context path
to be visible in urls; this is fine for us but I can see it being
useful for lift to know that the frontend proxy is taking care of
mapping foo.com/ to the context /foo-lift-app/, instead of always
adding the context path to urls. Similarly, we're running multiple
subdomains from a single lift instance, because it's all basically
common code and templates. The way we ended up making this work was
to have 1 folder per subdomain, and subclass Link so that it knows how
to createLink correctly based on headers the frontend is sending; e.g.
the link for http://foo.bar.com/baz is put into the sitemap as /
foo.bar/baz and will be rendered as /baz if the request is already in
the foo subdomain, or fully qualified http://foo.bar.com/baz
otherwise. In one sense, it was nice that we were even able to do
this without modifying Lift. However, we already ran into the issue
of default form redirects wanting to go to /foo.bar/baz, not just /
baz, and there doesnt seem to be a clean place to handle that.
The public face of 1.0 vs snapshot could be handled a little better,
in terms of website / liftbook / documentation. If the intention is
for all users to track snapshot for now, that should be made explicit
and the docs updated.
The ugly:
There are aspects of the API that are frankly quite difficult for
developers to take seriously -
"object User extends User with MetaMegaProtoUser[User]" is a common
whipping boy around the office, and leads to jokes about giant robots.
Similarly, there are a number of places that seem to force syntactic
overhead instead of catering to concise default usage, eg the first
time I saw the liftbook example of
case RewriteRequest(
ParsePath(List("account",acctName),_,_,_),_,_) =>
RewriteResponse("viewAcct" :: Nil, Map("name" -> acctName))
I thought how often are you actually going to fill in those
underscores? One of the things we did as part of our link subclass
was an implicit conversion to allow writing links in the sitemap as
"foo.bar/baz/quux/*" rather than ("foo.bar" :: "baz" :: "quux" :: Nil)
-> true. I don't know if that's bad style or not, but it certainly
makes the sitemap more scannable as a list of urls. I know
conciseness isn't this framework's top priority, but it seems like
some judicious use of implicits would help.
Thanks to the devs for writing lift, and like I said, please offer any
suggestions or counterpoints.
David Pollak
Thanks for such a thoughtful set of feedback.
Is there a way we could do something to make Lift stuff more DW friendly?
PCDataXmlParser converts entity references to actual utf-8
characters. This means templates with entity references won't round-
trip correctly through view source, nor through localization
properties files / gettext files.
Hmmm... is this worth a flag someplace to disable this behavior?
Jetty seems to do questionable things with filehandles, opening
multiple copies of the same template file and keeping them open too
long. Setting ulimit "fixes" the issue of "Too many open files"
exceptions & failed requests, but it still seems like at most
templates should need 1 access to check mtime. Not really a lift
issue, and not a deal breaker for us to switch to a different servlet
container, but I am curious to see what the eventual solution for
comet performance is.
If you can put together a reproducible case on this, we'll see what we can do to close stuff that Jetty hands us more explicitly.
Lift doesn't seem to have any central place to handle URL issues
related to servlet context paths and domains / subdomains. We're
currently running lift as a root servlet to avoid forcing context path
to be visible in urls; this is fine for us but I can see it being
useful for lift to know that the frontend proxy is taking care of
mapping foo.com/ to the context /foo-lift-app/, instead of always
adding the context path to urls. Similarly, we're running multiple
subdomains from a single lift instance, because it's all basically
common code and templates. The way we ended up making this work was
to have 1 folder per subdomain, and subclass Link so that it knows how
to createLink correctly based on headers the frontend is sending; e.g.
the link for http://foo.bar.com/baz is put into the sitemap as /
foo.bar/baz and will be rendered as /baz if the request is already in
the foo subdomain, or fully qualified http://foo.bar.com/baz
otherwise. In one sense, it was nice that we were even able to do
this without modifying Lift. However, we already ran into the issue
of default form redirects wanting to go to /foo.bar/baz, not just /
baz, and there doesnt seem to be a clean place to handle that.
Yeah. I agree this is a problem. I opened a ticket on it forever ago: http://github.com/dpp/liftweb/issues#issue/68
If you have any thoughts on how to deal with this, input would help formulate a solution.
The public face of 1.0 vs snapshot could be handled a little better,
in terms of website / liftbook / documentation. If the intention is
for all users to track snapshot for now, that should be made explicit
and the docs updated.
Yep.
The ugly:
There are aspects of the API that are frankly quite difficult for
developers to take seriously -
"object User extends User with MetaMegaProtoUser[User]" is a common
whipping boy around the office, and leads to jokes about giant robots.
I guess I shouldn't name things when I'm in a goofy mood.
Similarly, there are a number of places that seem to force syntactic
overhead instead of catering to concise default usage, eg the first
time I saw the liftbook example of
case RewriteRequest(
ParsePath(List("account",acctName),_,_,_),_,_) =>
RewriteResponse("viewAcct" :: Nil, Map("name" -> acctName))
I thought how often are you actually going to fill in those
underscores?
This particular construct was the subject of a lot of debate, both on-list and with some folks on a given project. I think the best way to deal with it is to have a bunch of helper objects w/unapply to do better matching.
One of the things we did as part of our link subclass
was an implicit conversion to allow writing links in the sitemap as
"foo.bar/baz/quux/*" rather than ("foo.bar" :: "baz" :: "quux" :: Nil)
-> true. I don't know if that's bad style or not, but it certainly
makes the sitemap more scannable as a list of urls. I know
conciseness isn't this framework's top priority, but it seems like
some judicious use of implicits would help.
Yep. If you care to contribute your various implicits and object.unapply stuff into Lift, we'd love to have them.
Thanks to the devs for writing lift, and like I said, please offer any
suggestions or counterpoints.
These issues were specific, well reasoned and actionable (well, except for the MetaMegaUsertron issue... ;-) ) This is the kind of feedback that helps make Lift better.
Thanks,
David
PS -- Concise code is a goal of Lift.
--
You received this message because you are subscribed to the Google Groups "Lift" group.
To post to this group, send email to lif...@googlegroups.com.
To unsubscribe from this group, send email to liftweb+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics
Nathan
> > Jetty seems to do questionable things with filehandles, opening
> > multiple copies of the same template file and keeping them open too
> > long. Setting ulimit "fixes" the issue of "Too many open files"
> > exceptions & failed requests, but it still seems like at most
> > templates should need 1 access to check mtime. Not really a lift
> > issue, and not a deal breaker for us to switch to a different servlet
> > container, but I am curious to see what the eventual solution for
> > comet performance is.
>
> If you can put together a reproducible case on this, we'll see what we can
> do to close stuff that Jetty hands us more explicitly.
I'm a total Lift and Scala newbie, but file-handle problems and last-
modified checks remind me of this bug in Facelets:
https://facelets.dev.java.net/issues/show_bug.cgi?id=278
It looks like ResourceServer.scala may be hitting the same problem
between the findResourceInClasspath and calcLastModified methods, but
again - total newbie - so take that with a boulder sized grain of
salt.
--
Nathan
David Pollak
I'm a total Lift and Scala newbie, but file-handle problems and last-
> > Jetty seems to do questionable things with filehandles, opening
> > multiple copies of the same template file and keeping them open too
> > long. Setting ulimit "fixes" the issue of "Too many open files"
> > exceptions & failed requests, but it still seems like at most
> > templates should need 1 access to check mtime. Not really a lift
> > issue, and not a deal breaker for us to switch to a different servlet
> > container, but I am curious to see what the eventual solution for
> > comet performance is.
>
> If you can put together a reproducible case on this, we'll see what we can
> do to close stuff that Jetty hands us more explicitly.
modified checks remind me of this bug in Facelets:
https://facelets.dev.java.net/issues/show_bug.cgi?id=278
Thanks. Looks like you found a defect and I've opened a defect.
It looks like ResourceServer.scala may be hitting the same problem
between the findResourceInClasspath and calcLastModified methods, but
again - total newbie - so take that with a boulder sized grain of
salt.
--
Nathan
--
You received this message because you are subscribed to the Google Groups "Lift" group.
To post to this group, send email to lif...@googlegroups.com.
To unsubscribe from this group, send email to liftweb+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Nathan Parry
> > > > multiple copies of the same template file and keeping them open too
> > > > long. Setting ulimit "fixes" the issue of "Too many open files"
> > > > exceptions & failed requests, but it still seems like at most
> > > > templates should need 1 access to check mtime. Not really a lift
> > > > issue, and not a deal breaker for us to switch to a different servlet
> > > > container, but I am curious to see what the eventual solution for
> > > > comet performance is.
>
> > > If you can put together a reproducible case on this, we'll see what we
> > can
> > > do to close stuff that Jetty hands us more explicitly.
>
> > I'm a total Lift and Scala newbie, but file-handle problems and last-
> > modified checks remind me of this bug in Facelets:
>
> >https://facelets.dev.java.net/issues/show_bug.cgi?id=278
>
> Thanks. Looks like you found a defect and I've opened a defect.
In case it is helpful, I think you can reproduce the behavior by doing
something like this:
1. mvn jetty:run
2. Figure out the pid for jetty
3. Figure out how many file descriptors are open after startup using:
lsof | grep <pid> | wc -l
4. Run a command like the one copied below to request the same file
over and over.
5. Run the lsof command line every few seconds to watch the FD count.
Sample command to fetch a file in a loop:
while (`true`); do curl -H "If-Modified-Since: Tue, 29 Dec 2009
20:02:16 UTC" -H "Cache-Control: max-age=0" http://localhost:8080/classpath/blueprint/screen.css;
done
Running this locally the FD count seems to fluctuate between the
baseline count and around 2x the baseline. I would guess that might
be based on how frequently the garbage collector is running and
cleaning up any connections that were not explicitly closed.
--
Nathan
cody koeninger
On Dec 30 2009, 3:16 pm, David Pollak <feeder.of.the.be...@gmail.com>
wrote:
> > Dreamweaver doesn't deal with head merge, it will automatically move
> > the head tags outside of the lift:surround. Not really a lift issue.
>
> Is there a way we could do something to make Lift stuff more DW friendly?
Not that I can see - short of making head tags such a special case
that you could put them at the top of any template, outside of
lift:surround, which doesn't seem reasonable. I'll try to dig into
dreamweaver options more to see if there's a way to disable it. To be
clear, a designer can manually fix it in dreamweaver before
committing, it's just that every time it loads a file from the svn
repo, it silently moves things around.
> > PCDataXmlParser converts entity references to actual utf-8
> > characters. This means templates with entity references won't round-
> > trip correctly through view source, nor through localization
> > properties files / gettext files.
>
> Hmmm... is this worth a flag someplace to disable this behavior?
In our case, yeah, a flag to disable this behavior would save me a lot
of work. For localization it's invaluable to have the strings in the
source correspond exactly to the strings in the properties / gettext
files; so far my "get it done" alternative has been a lot of nastiness
with sed and/or manual editing.
> > Jetty seems to do questionable things with filehandles, opening
> > multiple copies of the same template file and keeping them open too
> > long. Setting ulimit "fixes" the issue of "Too many open files"
> > exceptions & failed requests, but it still seems like at most
> > templates should need 1 access to check mtime. Not really a lift
> > issue, and not a deal breaker for us to switch to a different servlet
> > container, but I am curious to see what the eventual solution for
> > comet performance is.
>
> If you can put together a reproducible case on this, we'll see what we can
> do to close stuff that Jetty hands us more explicitly.
The steps that nathan described are pretty representative of how we
were diagnosing the issue. The same thing didn't seem to happen under
tomcat, so I figured it was jetty specific and left it at that. It
looks like you are already working on getting that issue fixed, which
is awesome.
> > Lift doesn't seem to have any central place to handle URL issues
> > related to servlet context paths and domains / subdomains. We're
> > currently running lift as a root servlet to avoid forcing context path
> > to be visible in urls; this is fine for us but I can see it being
> > useful for lift to know that the frontend proxy is taking care of
> > mapping foo.com/ to the context /foo-lift-app/, instead of always
> > adding the context path to urls. Similarly, we're running multiple
> > subdomains from a single lift instance, because it's all basically
> > common code and templates. The way we ended up making this work was
> > to have 1 folder per subdomain, and subclass Link so that it knows how
> > to createLink correctly based on headers the frontend is sending; e.g.
> > the link forhttp://foo.bar.com/bazis put into the sitemap as /
> > foo.bar/baz and will be rendered as /baz if the request is already in
> > the foo subdomain, or fully qualifiedhttp://foo.bar.com/baz
> > otherwise. In one sense, it was nice that we were even able to do
> > this without modifying Lift. However, we already ran into the issue
> > of default form redirects wanting to go to /foo.bar/baz, not just /
> > baz, and there doesnt seem to be a clean place to handle that.
>
> Yeah. I agree this is a problem. I opened a ticket on it forever ago:http://github.com/dpp/liftweb/issues#issue/68
>
> If you have any thoughts on how to deal with this, input would help
> formulate a solution.
Yeah, I noticed that ticket after sending the message, sorry. I
haven't had a chance to really read the related code yet, so my only
input for now would be that when looking at the solution for context
path issues, don't restrict it to depend on just the context path.
Speaking naively, I'd like to have a consistent place to override (or
define in liftrules, etc) a function from filesystem path to external
url, such that it could depend on arbitrary request headers (in our
case, the top level domain and subdomain), not just the context path.
> > There are aspects of the API that are frankly quite difficult for
> > developers to take seriously -
> > "object User extends User with MetaMegaProtoUser[User]" is a common
> > whipping boy around the office, and leads to jokes about giant robots.
>
> I guess I shouldn't name things when I'm in a goofy mood.
Well, to be fair, we did decide it was an intentionally silly name,
and there are plenty of people around here who like Transformers ;) I
know there's been some back and forth about naming on this list; not
going to weigh in on that, just providing a data point that naming
issues did actually create a bit of unnecessary initial friction,
given that I was already trying to convince people to try out a new
language.
> > One of the things we did as part of our link subclass
> > was an implicit conversion to allow writing links in the sitemap as
> > "foo.bar/baz/quux/*" rather than ("foo.bar" :: "baz" :: "quux" :: Nil)
> > -> true. I don't know if that's bad style or not, but it certainly
> > makes the sitemap more scannable as a list of urls. I know
> > conciseness isn't this framework's top priority, but it seems like
> > some judicious use of implicits would help.
>
> Yep. If you care to contribute your various implicits and object.unapply
> stuff into Lift, we'd love to have them.
I certainly don't mind contributing - in this particular case we're
talking a couple lines of code here and there, but I'll get
clarification from my management regarding permission. Let me know
the best way to proceed, whether that's posting short examples to the
list and letting you guys do your own expression of the idea if you
like it, vs signing a contributor agreement, whatever makes sense
(speaking as someone who suffered through law school + the bar, I
actually do appreciate attention to detail on those issues).
> These issues were specific, well reasoned and actionable (well, except for
> the MetaMegaUsertron issue... ;-) ) This is the kind of feedback that helps
> make Lift better.
>
> Thanks,
>
> David
>
> PS -- Concise code is a goal of Lift.
Thanks for the response & for taking the ribbing about metamegatron in
the spirit it was intended. As for conciseness, we've got a lot of
Perl experience so we may have a warped view of conciseness, but I'll
be happy to share any hacks we end up using to keep things
streamlined.
David Pollak
Nathan,
Now don't take this the wrong way, but I could kiss you. ;-)
This is a great defect report... it's led to: http://github.com/dpp/liftweb/issues/#issue/275
As well as a fix that I'm posting to review board right now.
Thanks,
David
--
Nathan
--
You received this message because you are subscribed to the Google Groups "Lift" group.
To post to this group, send email to lif...@googlegroups.com.
To unsubscribe from this group, send email to liftweb+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.