Clojure as a CGI language?
robert.e...@googlemail.com
writing CGI web applications would be. I know that it can be used for
web programming using java web servers, however I want to be able to
develop application which can be simply uploaded to a server and `just
work', like PHP. for which CGI is the only option I know of.
rob
Clojure servelets (why not do it that way by the way?) CGI allows you
to operate any program via the web, but it will be alot slower if you
have load everything every time a request is received by the server.
My guess is that it will be prohibitively clunky. I saw a good
article on using emacs lisp via CGI, which if that is what you want to
do, makes sense as the way to do it since there is no other more
integrated way (in contrast with Java which is used for server-side
web application development all the time). You could use that article
to learn about CGI, but there's really not much to it.
There may be some way of keeping Clojure running and thereby avoiding
that overhead. This would be similar to if there were a "mod_clojure"
for Apache, except mod_clojure (or mod_java-- which really does exist)
should allow you greater access and control at the various stages of
the request and serving of content. I have not tried using servelets,
so I'm not certain what the advantages/ disadvantages are. Mod_java
is less developed and has had less people working on it and using it
than Tomcat/ servelets, but I don't know much about either really, and
have no personal experiences with either of these approaches.
On Jul 20, 5:01 pm, "robert.e.hick...@googlemail.com"
Niels Mayer
App development using a database-backed web development environment like Xwiki gets rid of a lot of the issues that might give one pause programming traditional "java web servers," most of which is boilerplate cruft that should just be hidden away and encapsulated by a higher level interface.... which is exactly what Xwiki gives in terms of app-development.
I don't think it would be *that* hard (compared to the heavy lifting already done) to support Clojure as a scripting language in java-based Xwiki. Xwiki already has strong support for scripting in Groovy and Velocity. (Although this video is old and talks about Xwiki 1.0 (2.0 is nearing release) it's still a good intro to app-development in Xwiki: http://www.youtube.com/watch?v=xs3LuzwqemM )
http://code.xwiki.org/xwiki/bin/view/Snippets/DisplayFormatedCodeAttachmentsSnippet
provides a suggestion for extending Xwiki with Clojure as a scripting language. Xwiki already has extensive java "plugins" that create a library for web app-development that you "script" in either Velocity or Groovy. If Clojure has a JSR-223 implementation, you could easily leverage all the same Xwiki API's you normally access from Velocity or Groovy:
VincentMassol | 2009/05/05 20:03
Niels, no there's no plan but we can now support any language using macros so feel free to provide a clojure macro if you're interested. Shouldn't be too hard to do, especially if there's a JSR-223 implementation (in which case simply dropping the jar in WEB-INF/lib should be enough - you'd then use it using the script macro).I hope someone is inspired to try this out. I think it could be a surprisingly easy way to do web-programming in Lisp, by leveraging a flexible platform like Xwiki.
http://nielsmayer.com
Daniel
>
> It sounds like you want to avoid the approach of Apache Tomcat and
> Clojure servelets (why not do it that way by the way?) CGI allows you
> to operate any program via the web, but it will be alot slower if you
> have load everything every time a request is received by the server.
> My guess is that it will be prohibitively clunky. I saw a good
> article on using emacs lisp via CGI, which if that is what you want to
> do, makes sense as the way to do it since there is no other more
> integrated way (in contrast with Java which is used for server-side
> web application development all the time). You could use that article
> to learn about CGI, but there's really not much to it.
>
> There may be some way of keeping Clojure running and thereby avoiding
> that overhead. This would be similar to if there were a "mod_clojure"
> for Apache, except mod_clojure (or mod_java-- which really does exist)
> should allow you greater access and control at the various stages of
> the request and serving of content. I have not tried using servelets,
> so I'm not certain what the advantages/ disadvantages are. Mod_java
> is less developed and has had less people working on it and using it
> than Tomcat/ servelets, but I don't know much about either really, and
> have no personal experiences with either of these approaches.
>
I mostly agree with the above, just to extend it a bit: PHP is an
interpreter which means a new process is fired up on every request.
Java is abysmally slow if you try to use it in the same way since you
have to instantiate the whole VM everytime (besides the fact that you
have to serialize all state incl. caches to disk to survive a
request). What you need is a resident JVM process to answer your
requests. If you're using Apache in front to handle the requests you
can use more than one strategy, there should be enough tutorials
around to help you with that (start at
http://tomcat.apache.org/connectors-doc/webserver_howto/apache.html to
get an idea).
This is quite a bit different from PHP, but also Rails etc. usually
have a resident process that you're talking to (FastCGI etc.). It
makes some other things in the end a lot easier. If you try to just
have something small up and running without a lot of hassle, try to
make it on Google App Engine. There you have a (restricted) managed
JVM and you just have to supply a servlet (standard API for writing
HTTP handler in Java land). Otherwise, there are a few companies that
have JVM hosting in Virtual machines on offer (e.g. Kattare:
http://www.kattare.com/).
Rant: I also find it annoying and unnecessarily complex to have JVM
based webapplications running online. Not many managed JVM
environments are available in comparison to PHP, Python, Perl, Ruby,
et al. The reason is that until now it's still not really possible to
reliably partition applications (code, resources consumption, kill of
only one applicaton) within a single JVM, that means, JVM don't really
mix with shared hosting plans (the solutions are so called MVMs with
true separation of resources inside a JVM). Until these MVMs arrive,
you really need are Hypervisor based solutions, Xen etc. to constrain
ressource usage. I hope that someone is going to tackle that problem
soon, there's a pile of cash waiting to be made if you can pull it off
for the masses.
Hope that helps.
Cheers,
Daniel
Jeff Marder
you want to get fancy later you could always run it behind Apache or
lighttpd. I haven't done this myself with Clojure, but I've used an
embedded Jetty for test cases in a plain Java application. It's very
simple- probably much more so than CGI- and it will perform much
better than CGI.
I saw a blog post a while back where a guy made a quick and dirty
Clojure web app with Jetty:
http://robert.zubek.net/blog/2008/04/26/clojure-web-server/
robert hickman
---- security issues
I have read trough the page that you linked to and the issues listed
ain't any different from the issues associated with developing PHP
applications.
---- I mostly agree with the above, just to extend it a bit: PHP is an
---- interpreter which means a new process is fired up on every request.
---- Java is abysmally slow if you try to use it in the same way since you
---- have to instantiate the whole VM everytime (besides the fact that you
---- have to serialize all state incl. caches to disk to survive a
---- request).
Why is the jvm so slow at starting up? Both PHP and Python are byte code
compiled and run in a VM, but the runtime environments for both of these
languages start up almost instantly.
---- Rant: I also find it annoying and unnecessarily complex to have JVM
---- based webapplications running online. Not many managed JVM
---- environments are available in comparison to PHP, Python, Perl, Ruby,
---- et al. The reason is that until now it's still not really possible to
---- reliably partition applications (code, resources consumption, kill of
---- only one applicaton) within a single JVM, that means, JVM don't really
---- mix with shared hosting plans (the solutions are so called MVMs with
---- true separation of resources inside a JVM). Until these MVMs arrive,
---- you really need are Hypervisor based solutions, Xen etc. to constrain
---- ressource usage. I hope that someone is going to tackle that problem
---- soon, there's a pile of cash waiting to be made if you can pull it off
---- for the masses.
This is exactly why I do not want the complexity of proxying various servers
behind apache, I am running my own development server, which is set up to
mimic the average shared hosting environment, PHP and CGI applications run
as separate users to prevent the compromise of one application form effecting
anything else.
Running other servers doesn't fit with this environment very well, and
embedding more modules into apache is something that I am trying to avoid,
as they often provide no way of running code as different users.
---- You could use FastCGI to accomplish this, though you would have to
---- write the interface.
---- http://www.fastcgi.com/
---- FastCGI would remove the long startup times for the JVM, etc.
I will look into fast CGI.
---- One thing I would say, though, is that PHP apps don't "Just Work", you
---- have to have mod_php (in Apache) or
---- some other support in the web server.
I know, however from the viewpoint of the end user, php does just work, just
upload a .php file, and run it. Which is why PHP has bean so successful
compared to other technologies, even though the language is a complete mess.
---- If you want very easy to deploy web apps, I would suggest Compojure:
---- http://github.com/weavejester/compojure/tree/master
---- You can always use a proxy to front a compojure app, which is how a
---- lot of web apps do it.
Compojure requires running a second server and proxying it, which as
described above, I am trying to avoid. Is there any way of developing
a application with Compojure completely without shell access to the server?
Lisp seams like a very good language for developing web applications, mostly
due to its macro system and doorman specific languages. However there seam
to be no practical ways of developing with it.
Daniel Renfer
hickman<robert.e...@googlemail.com> wrote:
> ---- If you want very easy to deploy web apps, I would suggest Compojure:
> ---- http://github.com/weavejester/compojure/tree/master
> ---- You can always use a proxy to front a compojure app, which is how a
> ---- lot of web apps do it.
>
> Compojure requires running a second server and proxying it, which as
> described above, I am trying to avoid. Is there any way of developing
> a application with Compojure completely without shell access to the server?
>
It is possible to compile your Compojure application to a WAR file
that can be deployed into any standard servlet container. You would
still need a host that supports Tomcat (or similar) but you wouldn't
need shell access.
Chouser
> ---- write the interface.
> ---- http://www.fastcgi.com/
> ---- FastCGI would remove the long startup times for the JVM, etc.
>
> I will look into fast CGI.
Your pain is my pain. Writing your web app in PHP, Ruby,
Python etc. means that you have your pick of a very large
number of cheap hosting services. Depending on the service,
you may be able to simply copy over your script source and
have it run almost automatically, or you may need to create
a CGI and/or ask that the interpreter for whatever language
be installed, but still this is very commonly available.
Especially for a low-volume, low-revenue sites these
inexpensive options are very attractive.
Java and therefore Clojure does not play nicely in this
niche as far as I can tell. Even if a service allows ssh
access such that you can install a JVM, Clojure, etc. it's
almost certain you will not be allowed to keep
a long-running process going (which rules out simple Jetty
solutions). A simple CGI, while usually possible in such an
environment, incurs such a heavy CPU and time cost on JVM
startup that it's not a practical solution.
FastCGI is indeed a theoretical option. While not as
widely supported as plain CGI by cheap hosting services,
you'd still have many providers to choose from. There is
even a Java interface. I attempted to use this several
months ago, but failed for reasons I don't remember --
I hope you have more success.
The reason that FastCGI is works when other kinds of
separate-server-processes don't is because the web server
(Apache or lighttpd) can start up and shut down these
processes on demand based on policies controlled by the
server admin.
I even got working a solution where a very small CGI
(written in C) forwards requests to a separate server
process (which could be running Jetty or whatever), but if
no such server is available the little CGI would start it
up. Unfortunately my hosting service still got sore about
the long-running process -- perhaps if the server shut
itself down after a few idle minutes that would keep the
admins happy, but I didn't pursue it.
So as far as I can tell, your best options for hosting
a Clojure web app are still either to get FastCGI working or
pay for a more expensive service: Java servlet container
provider, private virtual machine, co-locate, or on up from
there.
--Chouser
Mike Hinchey
See http://elhumidor.blogspot.com/2009/04/clojure-on-google-appengine.html
-Mike
robert hickman
> As Daniel mentioned, Google App Engine can host java. It's very easy, just
> upload a war with your clj AOT-compiled.
The google app engine looks like an interesting platform, However I
cannot create an account as I don't have a phone.
Niels Mayer
---- Trying to use CGI sounds like a bad idea. It's always full of
---- security issues
I have read trough the page that you linked to and the issues listed
ain't any different from the issues associated with developing PHP
applications.
That's true. You could run your Java web server as "root" and enjoy the same lack of security you can get with all the PHP, Perl, and other "1st gen" solutions that seem to be the target of every script-kiddies repertoire (just watch your web logs, see what's being targetted...). And since java can now bind port 80, you can get rid of apache too. :-) Ah, the simple life!
Ultimately, security (and the fact that someone thought about security ahead of time) is one of the big differences between hosting with java versus other "easier" alternatives. But it's also why you hear of those sites getting "owned" more regularly than others. (The java-based sites just go belly-up garbage collecting strings on each slashdotting :-) )
What exactly is the big win in reinventing the wheel in developing some special fastCGI or
mod_php equivalent for running your java when you can use something standard and supported, e.g. a a different special-purpose apache httpd module written to serve your Java '/app/' running securely as a low priv user via AJP?
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
ProxyPass /app/ ajp://127.0.0.1:8009/app/
ProxyRequests Off
#<Proxy *>
# Order deny,allow
# Deny from all
# Allow from .example.com
# </Proxy>
## NPM: see http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances#HModProxyAJPConfiguration
ProxyPreserveHost On
As apache/httpd must bind a privileged port (80), it must run as a privileged user. Because admins implicitly "trust" very little software that needs to run privileged, the actual choices of "trusted" implementations and configurations are limited. Given it's extensive history and broad support, apache httpd and most of it's modules are "trusted" by admins. One of the "trusted" modules typically available with apache (e.g. it's part of Fedora's 'httpd' rpms) allows tomcat to run more securely as it's own "low privilege" user on a nonprivileged, private port. This follows the principle of least priviledge and reduction of overlapping concerns, low privilege users such as 'tomcat' running "open" and potentially dangerous code, should only be able to write to a very specific part of the filesystem and not access system-level and network resources....
All you need to do is enable AJP connector in tomcat:
<Connector address="127.0.0.1" port="8009" URIEncoding="UTF-8" protocol="AJP/1.3"></Connector>
Why is the jvm so slow at starting up? Both PHP and Python are byte code
compiled and run in a VM, but the runtime environments for both of these
languages start up almost instantly.
Perhaps they use undump to get good startup performance? Java has introduced this facility as well. The inherent unportability of doing this (i've undumped plenty an emacsen and xlisp/winterp's in my time) is apparent in the spotty platform coverage:
http://blog.headius.com/2009/01/my-favorite-hotspot-jvm-flags.html
-Xshare:dump can help improve startup performance on some installations. When run as root (or whatever user you have the JVM installed as) it will dump a shared-memory file to disk containing all of the core class data. This file is much faster to load then re-verifying and re-loading all the individual classes, and once in memory it's shared by all JVMs on the system. Note that -Xshare:off, -Xshare:on, -Xshare:auto set whether "Class Data Sharing" is enabled, and it's not available on the -server VM or on 64-bit systems. Mac users: you're already using Apple's version of this feature, upon which Hotspot's version is based.
If hosting services actually provided a specially undumped jvm, also set to "share", then multiple JVM's could be running in parallel, all sharing their huge libraries of code. Seems like this is the better solution for hosting Java -- pick a particular trusted baseline of libs that all apps/hosts/users will be using, and "preload" that into your java impl's "shared segment." Rather than reinventing some mechanism of regulating access or curtailing runaway peformance, you simply have a bunch of JVM's all running at a reasonable size, all sharing class files, and using the OS's built-in mechanisms to catch and signal runaway processes, audit usage, etc. Also by setting thread limits on individual JVM's, a single process spawning a lot of threads (perhaps runaway) doesn't swamp other JVM's from getting their fair share.
-- Niels
http://nielsmayer.com
Niels Mayer
As Daniel mentioned, Google App Engine can host java. It's very easy, just upload a war with your clj AOT-compiled.
See http://elhumidor.blogspot.com/2009/04/clojure-on-google-appengine.html
http://xwiki1.appspot.com/bin/view/Main/ ( http://markmail.org/thread/krlqzdzgia2j72r7 )
Support for java-based scripting is provided via xwiki's default scripting language, Velocity and JSR223. Xwiki has good support for JSR223-based scripting languages in Java: http://jerome.myxwiki.org/xwiki/bin/view/Blog/GroovyWebConsoleInXWiki
Previously I asked:
If Clojure has a JSR-223 implementation, you could easily leverage all the same Xwiki API's you normally access from Velocity or Groovy:
Niels, no there's no plan but we can now support any language using macros so feel free to provide a clojure macro if you're interested. Shouldn't be too hard to do, especially if there's a JSR-223 implementation (in which case simply dropping the jar in WEB-INF/lib should be enough - you'd then use it using the script macro).VincentMassol | 2009/05/05 20:03
http://github.com/pmf/clojure-jsr223/tree/master
http://groovy.codehaus.org/JSR-223+access+to+other+JVM+languages
http://sayspy.blogspot.com/2009/03/interacting-between-jvm-lang-here-and.html
Niels
http://nielsmayer.com
cody koeninger
On Jul 22, 10:15 am, Chouser <chou...@gmail.com> wrote:
> Java and therefore Clojure does not play nicely in this
> niche as far as I can tell. Even if a service allows ssh
> access such that you can install a JVM, Clojure, etc. it's
> almost certain you will not be allowed to keep
> a long-running process going (which rules out simple Jetty
> solutions).
> a Clojure web app are still either to get FastCGI working or
> pay for a more expensive service: Java servlet container
> provider, private virtual machine, co-locate, or on up from
> there.
>
cody 12139 12138 0 Jul07 pts/1 01:31:03 java -cp .:lib/*:/usr/
local/src/compojure/src
I've had a jetty instance running on slicehost for over 2 weeks now,
which was just the last time I happened to start it.
I dunno if $20 / month counts as expensive (it's less than I was
paying for hosting before), but I'm not sure you can even find
reliable shared hosting for the more usual suspects like Rails for
much less than $10.
Jetty / compojure has been pretty painless to get running, I'd rather
use routes than mess with mod_rewrite rules for instance. My instinct
is that a simple apache / lighttpd / whatever on port 80 to serve
static content and proxy back to jetty for anything else would be
plenty sufficient for most production needs (along with a real hosting
plan, but that's true of anything).