Interactive Development with ring 0.2 and compojure 0.4

[Rick Moynihan]

Hi all,

I've been taking a look at compojure 0.4, and have been having some
trouble getting started.

It appears that the new method of web server is to use ring's
adapters, as the 0.4 branch seems to have removed the jetty-server
function.

No bother, as ring supplies ring.adapter.jetty/run-jetty function,
unfortunately though it's behaviour doesn't seem to encourage lisp
style interactive development.

I have the following code taken from rings hello_world example:

(ns ring.example.hello-world
(:use ring.adapter.jetty)
(:import java.util.Date java.text.SimpleDateFormat))

(defn app
[req]
{:status 200
:headers {"Content-Type" "text/html"}
:body (str "<h3>Hello World from Ring</h3>"
"<p>The current time is "
(.format (SimpleDateFormat. "HH:mm:ss") (Date.))
".</p>")})

(run-jetty app {:port 8080})

Running this code works, however once its running modifying the
definition of the app function (e.g. changing it to say "Hello Rick")
and re-evaluating the form in Emacs/SLIME doesn't cause the changes to
propogate into Jetty. Also re-evaluating the whole file fails as
jetty's socket is left open.

I believe the ability to redefine functions and have them
automagically be deployed was a feature of Compojure 0.3.2. What is
the preferred way to do this now?

Thanks for your help,

R.

p.s. I hope you don't mind the cross-post, but I'm not sure where this
responsibilty lies...

[Scott Jaderholm]

What do you think about changing the line

(run-jetty example {:port 8080})

to

(run-jetty (var example) {:port 8080})

at http://weavejester.github.com/compojure/docs/getting-started.html ?

See http://groups.google.com/group/ring-clojure/browse_thread/thread/5606760b86cfd49c/f2be9cf207cf0f11?lnk=gst&q=interactive
(the reply to this thread that didn't make it to compojure ml) for
details about why this is important.

Is this something that should only be done in development or is it
fine for production?

Not knowing about this caused me a lot of frustration when trying to
converting a 0.3.2 app to 0.4.0 and my changes weren't taking effect.
I suspect it others will expect (and want) the old interactive
behavior as well.

Also, what's the proper way to stop the server with 0.4.0? The code
for run-jetty makes it look like it returns the server s, but when i
call run-jetty so long as the server is running that function doesn't
return so I don't get an #^Server s to call .stop (a method I assume
exists) on.

Thanks,
Scott

> --
> You received this message because you are subscribed to the Google Groups "Compojure" group.
> To post to this group, send email to comp...@googlegroups.com.
> To unsubscribe from this group, send email to compojure+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/compojure?hl=en.
>
>

--
You received this message because you are subscribed to the Google Groups "Compojure" group.
To post to this group, send email to comp...@googlegroups.com.
To unsubscribe from this group, send email to compojure+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/compojure?hl=en.

[James Reeves]

On May 22, 3:07 am, Scott Jaderholm <jaderh...@gmail.com> wrote:
> Not knowing about this caused me a lot of frustration when trying to
> converting a 0.3.2 app to 0.4.0 and my changes weren't taking effect.
> I suspect it others will expect (and want) the old interactive
> behavior as well.

I think the "Hello World" example shouldn't be overcomplicated, but it
seems like we probably need some documentation covering interactive
development.

> Also, what's the proper way to stop the server with 0.4.0? The code
> for run-jetty makes it look like it returns the server s, but when i
> call run-jetty so long as the server is running that function doesn't
> return so I don't get an #^Server s to call .stop (a method I assume
> exists) on.

Ring adapters are a "lowest common denominator" interface. Since you
can't guarantee all servers will have an explicit "stop" function,
adapters don't have an explicit stop either. The only certain way to
stop them is to terminate the environment.

That said, maybe someone needs to create a Clojure Jetty wrapper. Then
you could use the Ring "servlet" function to convert your handler into
a servlet and run it under Jetty directly.

- James

[Brian Carper]

On May 26, 5:07 am, James Reeves <weavejes...@googlemail.com> wrote:
> That said, maybe someone needs to create a Clojure Jetty wrapper. Then
> you could use the Ring "servlet" function to convert your handler into
> a servlet and run it under Jetty directly.
>
> - James

The way things worked in Compojure 0.3 seemed OK to me. But if you
made ring.adapter.jetty/create-server and proxy-handler public, it'd
still be pretty easy to do something like this:

(ns my.site (:require (ring.adapter [jetty :as jetty])))

(def server (jetty/create-server {:port 8080}))

(defn start [routes]
(doto server
(.setHandler (jetty/proxy-handler routes))
(.start)))

(defn stop []
(.stop server))

Is there anything wrong with that approach?

[Brian Carper]

On May 26, 9:11 pm, Brian Carper <briancar...@gmail.com> wrote:
>
> Is there anything wrong with that approach?

Never mind, my code doesn't work. For now I'm resorting to throwing
run-jetty into a background thread and forgetting about it. Requiring
a JVM reboot to restart Jetty is a bit of show-stopper though
(literally).

[James Reeves]

On 27 May 2010 01:43, Brian Carper <brian...@gmail.com> wrote:
> Never mind, my code doesn't work.  For now I'm resorting to throwing
> run-jetty into a background thread and forgetting about it.  Requiring
> a JVM reboot to restart Jetty is a bit of show-stopper though
> (literally).

Out of interest, why is it a show-stopper?

- James

[Remco van 't Veer]

I agree with Brain. It is pretty annoying needing to restart the swank
process when you want to restart the server process. Or is there
another way to fiddle with, for instance, middleware without restarting?

Remco

[James Reeves]

You shouldn't need to restart the server process when developing. Just
pass your handler in as a var:

(run-jetty (var your-app) {:port 8080})

Altering the definition of your-app will automatically show up in the
browser. I don't have the time to confirm it this morning, but I
believe just recompiling a file with C-c C-k should be enough for you
to see changes in the browser.

- James

[Chas Emerick]

Indeed...and there's lots of ways to get to that point, too.

We don't use run-jetty (the jetty-maven-plugin meshes better with our
environment), but we get the same behaviour by passing the var of the
"tip" of our routes to defservice in the ns that we AOT-compile as our
(only) servlet. Once jetty is running, we can then connect an
enclojure REPL to it, and load whatever code we want, and see the
changes instantly.

- Chas

[Remco van 't Veer]

Confirmed, it does. Sorry about the noise.

[Brian Carper]

On May 27, 3:49 am, James Reeves <jree...@weavejester.com> wrote:

> On 27 May 2010 01:43, Brian Carper <briancar...@gmail.com> wrote:
>
> > Never mind, my code doesn't work.  For now I'm resorting to throwing
> > run-jetty into a background thread and forgetting about it.  Requiring
> > a JVM reboot to restart Jetty is a bit of show-stopper though
> > (literally).
>
> Out of interest, why is it a show-stopper?
>
> - James

I run multiple sites out of one JVM instance. Each site is its own
server running on a different port. Up to now I could restart one
site without affecting all the others. Now I don't see how to do that
other than bringing the whole JVM down and up again.

For development, doing (run-jetty (var your-app) {:port 8080}) does
work, as long as I don't want to switch the server to run with a new
var. During development I define new routes and wrap routes in
container vars to group them together, etc. I don't run into that
problem very often, admittedly. It could be a matter of defining one
"top-level" var and sticking with it.

--Brian

[Chas Emerick]

Everything in computer science is solvable by adding another level of
indirection. :-)

This is veering off topic some, but are people actually using run-
jetty to run "production" sites (where "production" is scare-quoted
appropriately)? I remember seeing posts about people running blogs
and such via run-jetty via emacs screen, but I've always presumed that
that was tinkering.

There's a lot of stuff that's orthogonal to one's actual application
that is handled very, very well by any of the very capable free-
standing servlet containers out there (of which jetty is itself one)
-- with which ring and compojure interop with quite nicely (with live
code reloading, if you so desire).

Cheers,

- Chas

[James Reeves]

On 27 May 2010 12:59, Brian Carper <brian...@gmail.com> wrote:
> I run multiple sites out of one JVM instance.  Each site is its own
> server running on a different port.  Up to now I could restart one
> site without affecting all the others.  Now I don't see how to do that
> other than bringing the whole JVM down and up again.

In which case, you probably want a Clojure wrapper for Jetty. The Ring
adapters take a "lowest common denominator" approach, but you seem
like you need more control over your server.

So rather than trying to extend the functionality of specific Ring
adapters, perhaps you should design a more functional (but less
generic) solution using the Ring Jetty adapter code as a basis.

I've been meaning to write something like that for a while now, but
I've been snowed under with work and other projects. It's not very
high up on my priority list, I'm afraid.

- James

[James Reeves]

On 27 May 2010 13:18, Chas Emerick <ceme...@snowtide.com> wrote:
> There's a lot of stuff that's orthogonal to one's actual application that is

> handled very, very well by any of the very capable free-standing servlet

> containers out there (of which jetty is itself one) -- with which ring and
> compojure interop with quite nicely (with live code reloading, if you so
> desire).

This is another good approach, and one with less wheel-reinventing. :)

- James

[Brian Carper]

On May 27, 10:18 am, Chas Emerick <cemer...@snowtide.com> wrote:
> This is veering off topic some, but are people actually using run-
> jetty to run "production" sites (where "production" is scare-quoted  
> appropriately)?  I remember seeing posts about people running blogs  
> and such via run-jetty via emacs screen, but I've always presumed that  
> that was tinkering.
>

It's possibly one of my posts you're remembering. I'm using it only
for hobby sites, so yes, tinkering. I'm coming from a Linux
background where there's a long tradition of kludging together shell
scripts.

> There's a lot of stuff that's orthogonal to one's actual application  
> that is handled very, very well by any of the very capable free-
> standing servlet containers out there (of which jetty is itself one)  
> -- with which ring and compojure interop with quite nicely (with live  
> code reloading, if you so desire).

Can you point me in the right direction to begin learning about the
proper way to do it? Even the name of a library would help. I know
nothing about the Java way of deploying websites.

I don't think this is off-topic. I know for a fact that a lot of
people would be interested to know how to deploy a Compojure site in a
solid and reliable way.

--Brian

[Nick Dimiduk]

On Thu, May 27, 2010 at 12:37 PM, Brian Carper <brian...@gmail.com> wrote:

I'm coming from a Linux background where there's a long tradition of kludging together shell scripts.

Here in Java-land we believe in monolithic deployment strategies! None of your "interchangeable parts" here. [/snark]

 

Can you point me in the right direction to begin learning about the

proper way to do it?  Even the name of a library would help.  I know
nothing about the Java way of deploying websites.

Traditional java webapps are deployed in a java application server container. Most places where I've been a part of java webapps, the preferred method of deployment is pushing a war file ("Web ARchive") from your build process out to the application server. If you like extra complexity, package up all your war's into an ear ("Enterprise ARchive", I believe). If your stars are particularly well aligned, the archive will be automatically detected and deployed  for immediate availability. For getting started, first do some reading on "production ready" tomcat or jetty. That should give you the context you need for understanding how "most people do it".

My guess, however, is that here in Clojure-land we'd prefer a more dynamic method of production deployments. I think the desirable compromise will be a method based upon a stable appserver installation, a build process for generating a clojure/compojure/ring war husk running swank, and a dependable way to connect, load code, and unload/reload dependency jars. Extra points if I can update dependencies automagically using leiningen.

Give these smug clojure weenies enough of the good old lispy ways to continue with the smug while bringing in enough of the modern to give us a chance :p

[/0.02$]

Cheers,

-Nick

[Chas Emerick]

On May 27, 2010, at 5:13 PM, Nick Dimiduk wrote:

On Thu, May 27, 2010 at 12:37 PM, Brian Carper <brian...@gmail.com> wrote:

I'm coming from a Linux background where there's a long tradition of kludging together shell scripts.

Here in Java-land we believe in monolithic deployment strategies! None of your "interchangeable parts" here. [/snark]

That's sort of snark and sarcasm all in one, right? ;-)  The funny thing is that JVM web apps have traditionally been extraordinarily modular, probably unnecessarily so -- this is part of what led to the over-componentization that was/is EJBs and such, which you refer to later.

Can you point me in the right direction to begin learning about the

proper way to do it?  Even the name of a library would help.  I know
nothing about the Java way of deploying websites.

Traditional java webapps are deployed in a java application server container. Most places where I've been a part of java webapps, the preferred method of deployment is pushing a war file ("Web ARchive") from your build process out to the application server. If you like extra complexity, package up all your war's into an ear ("Enterprise ARchive", I believe). If your stars are particularly well aligned, the archive will be automatically detected and deployed  for immediate availability. For getting started, first do some reading on "production ready" tomcat or jetty. That should give you the context you need for understanding how "most people do it".

A war file is a zip file that contains your web resources (i.e. html/css/js/image files), your application code (either classfiles or clojure source will do), and all of your dependencies, all in a self-contained bundle that can get dropped into any of a couple dozen containers that support the servlet spec.  To me, that sounds pretty darn simple, though it's true that some tools and server vendors have often stacked unnecessary proprietary complexity on top of that.  Use tomcat (or jetty, though it's far, far happier in as an embedded server), and you'll be really, really surprised with how easy it is.  Seriously, if entry-level *Java* developers can roll out apps using it, clever Clojure hackers shouldn't be so stumped by this stuff.

You're right about EAR files, although the evil there isn't so much in the packaging as what's in it -- the only reason to use EARs is if you're going to be using EJBs, message beans, JMX management beans, and the like.  That stuff is almost certainly not worth touching now unless you really, really have to (i.e. perhaps it's mandated some policy somewhere, or you need to go with the flow in an existing app, etc).

My guess, however, is that here in Clojure-land we'd prefer a more dynamic method of production deployments. I think the desirable compromise will be a method based upon a stable appserver installation, a build process for generating a clojure/compojure/ring war husk running swank, and a dependable way to connect, load code, and unload/reload dependency jars. Extra points if I can update dependencies automagically using leiningen.

I'd note that what you describe is very nearly exactly what we use:

- Tomcat (jetty for local dev / testing purposes)

- a standard war file that contains a single AOT-compiled clojure namespace as a servlet entry point (defined using defservice as I mentioned earlier in the thread)

- an enclojure repl server running on every deployed node so that we can connect with a REPL at any time, to any app server (usually tunneled through SSH) -- IIRC, there's a corresponding swank-clojure repl server library (by technomancy, I think)

You're going to run into trouble with the notion of unloading / reloading dependency jars, though.  That's just not how it works in JVM-land.  There's some nasty hacks (i.e. JavaRebel) for getting around this, but it's far, far, far easier to just rebundle and deploy a new war file -- doing this will allow the old app context to fall away, and your newly-deployed version will take its place.  This takes seconds using any sane app server, and is entirely, remarkably reliable.

So, there's your dream deployment configuration, ready to go! :-D

FYI, here's what our deployment process looks like -- which uses a clojure project called pallet (which further uses a java library called jclouds to automate the provisioning of instances on EC2 [or rackspace or gogrid or any other cloud you'd care to use]) that I was lucky enough to come across a few months ago: http://muckandbrass.com/web/x/CoBZ

It all depends on what you choose to care about, of course, but IMVHO, I'd much rather work on my application, than spend time and energy on what are, at their core, commodity tools.  The Java world has been working on web app deployment and operations for a long, *long* time.  They've certainly gotten some things wrong, but they've absolutely gotten the bulk of it very right, and it seems like sheer lunacy to me to even think about not standing on those giants' shoulders.  Let the cruft fall away, and use the gems that you have available to you by dint of Clojure being part of the JVM ecosystem.  If you haven't yet, I'd recommend you take a listen to Rich on the recent interview he did on SE-Radio (http://www.se-radio.net/podcast/2010-03/episode-158-rich-hickey-clojure), and pay particular attention to the part about not being on an island with your lisp.....

Give these smug clojure weenies enough of the good old lispy ways to continue with the smug while bringing in enough of the modern to give us a chance :p

This is definitely way, way off-topic, but...I winced hard at this, even with the smiley.  The 'smug lisp weenie' concept, however much it's invoked in jest, is a poisonous one that has so far been (mostly) absent from the Clojure community.  Conviction is good, nay, extraordinary when you can back up your conviction with tangible benefits and results, which will largely speak for themselves.  But smugness can only exist if there is an 'other' that is placed in a lesser position -- and I don't think that stance serves anyone well, especially if one cares about the broader future of Clojure.

Definitely not meaning to pick on you at all, Nick, you just gave me an opening to rant a bit. :-)

All the best,

- Chas

[Chas Emerick]

On May 27, 2010, at 3:37 PM, Brian Carper wrote:

>> There's a lot of stuff that's orthogonal to one's actual application
>> that is handled very, very well by any of the very capable free-
>> standing servlet containers out there (of which jetty is itself one)
>> -- with which ring and compojure interop with quite nicely (with
>> live
>> code reloading, if you so desire).
>
> Can you point me in the right direction to begin learning about the
> proper way to do it? Even the name of a library would help. I know
> nothing about the Java way of deploying websites.
>
> I don't think this is off-topic. I know for a fact that a lot of
> people would be interested to know how to deploy a Compojure site in a
> solid and reliable way.

Ah, sorry, I saw Nick's post before yours -- look there for what we
do. The other point worth mentioning is ..... (/me dons fire-proof
suit!) ..... we use maven and hudson for our build process, and for
kicking off our pallet-driven provisioning and deployment operations.
Lots of people are attached to lein, but IMO, there's just too many
useful tools and plugins for maven that make driving this whole thing
really, really easy to ignore it. Various info on maven, its jetty
plugin, and other bits can be had on my blog: http://muckandbrass.com/chas

Cheers,

- Chas