easy way to get Closure doing serious web apps?

[Niels Mayer]

(assuming you agree that xwiki-backed sites like http://www.idiva.com , http://www.curriki.org or http://www.bestventes.com constitute
"serious web apps"...)

On the Xwiki devs list, I inquired: http://lists.xwiki.org/pipermail/devs/2009-July/013763.html about using Clojure as a scripting language in Xwiki,   via JSR-223. I got two very helpful replies:

(1) http://lists.xwiki.org/pipermail/devs/2009-July/013764.html

Asiri Rathnayake Sun Jul 26 10:43:25 CEST 2009
[...]
> Clojure <http://groups.google.com/group/clojure> has a JSR-223 implementation
> according to:
> http://github.com/pmf/clojure-jsr223/tree/master
> http://groovy.codehaus.org/JSR-223+access+to+other+JVM+languages
> http://sayspy.blogspot.com/2009/03/interacting-between-jvm-lang-here-and.html
>
> Are there any examples, documentation, or suggestions of writing a "script macro" to call a new jar in WEB-INF/lib ?

1. Put the Clojure jar file <http://repo1.maven.org/maven2/org/clojure/clojure/1.0.0/clojure-1.0.0.jar> inside WEB-INF/lib directory

2. Build the Clojure JSR223 engine from http://github.com/pmf/clojure-jsr223/tree/master and place it inside
WEB-INF/lib

3. Restart your xwiki and use the script macro as below:

{{script language="Clojure"}}
// Put your clojure script here.
{{/script}}

(Note that since context is a reserved binding in JSR-223 specifications
used for Scripting Macros, the XWiki Api Context is now accessible through
xcontext.)

I haven't myself tried out clojure inside xwiki but I built a ruby
macro<http://code.xwiki.org/xwiki/bin/view/Macros/RubyMacro>(for
2.0M2) just by following the steps I described to you :)

Question: does  anybody have any hints, warnings, potential "gotchas" regarding this part: "Build the Clojure JSR223 engine from http://github.com/pmf/clojure-jsr223/tree/master and place it inside WEB-INF/lib" ??

(2)  http://lists.xwiki.org/pipermail/devs/2009-July/013767.html

From: Vincent Massol
Date: Sun, Jul 26, 2009 at 10:21 AM
[...]
Well all you need is drop the JSR223 jar in WEB-INF/lib and then you can use the new language using the script macro:
http://code.xwiki.org/xwiki/bin/view/Macros/ScriptMacro

Then if you want you can write a wrapping macro such as we've done for groovy and jruby so that it can be called with {{clojure}} instead of {{script language="clojure"}}.

> PS: Here's examples of easy "scripted java" programming you can do in
> Clojure (note the helpful parallelism constructs):
> http://travis-whitton.blogspot.com/2009/07/network-sweeping-with-clojure.html
> http://travis-whitton.blogspot.com/2009/06/clojures-agents-scientists-monkeys-and_18.html
>
> It could be very useful to employ massive parallelism via such Clojure
> scripts, which could achieve a xwiki-based web portal performance akin to
> Yahoo's, Google's, etc. For example, the following describes how Yahoo works
> -- and would be quite easy to implement this kind of processing "for free"
> in Clojure with very little code:
>
> http://research.yahoo.com/files/pnuts.pdf

I'm definitely +1 to make it part of the XWiki platform (same as for jruby) if you develop it. Similar to jruby I don't think we should
package it in the default WAR though but as an optional download to install (will be even easier with the app manager).

Niels
http://nielsmayer.com

[Niels Mayer]

> easy way to get Closure doing serious web apps?

Ooops. That's supposed to read "Clojure" in the title. Sorry about that.

By the way, I previously wrote: 

> It could be very useful to employ massive parallelism via such Clojure
> scripts, which could achieve a xwiki-based web portal performance akin to
> Yahoo's, Google's, etc. For example, the following describes how Yahoo works
> -- and would be quite easy to implement this kind of processing "for free"

> in Clojure with very little code:  http://research.yahoo.com/files/pnuts.pdf ...

Wondering whether Google App Engine would be able to host such an application, I check

http://elhumidor.blogspot.com/2009/04/clojure-on-google-appengine.html which says:

THE BIG CAVEAT

Two unusual aspects of the Google AppEngine environment create pretty major constraints on your ability to write idiomatic Clojure.

First, an AppEngine application runs in a security context that doesn't permit spawning threads, so you won't be able to use Agents, the clojure.parallel library, or Futures.

Second, one of the most exciting features of AppEngine is that your application will be deployed on Google's huge infrastructure, dynamically changing its footprint depending on demand. That means you'll potentially be running on many JVMs at once. Unfortunately this is a strange fit for Clojure's concurrency features, which are most useful when you have precise control over what lives on what JVM (and simplest when everything runs on one JVM). Since shared references (Vars, Refs, and Atoms) are shared only within a single JVM, they are not suitable for many of their typical uses when running on AppEngine. You should still use Clojure's atomic references (and their associated means of modification) for any state that it makes sense to keep global per-JVM, since there may be multiple threads serving requests in one JVM. But remember JVMs will come and go during the lifetime of your application, so anything truly global should go in the Datastore or Memcache.

The first point means closure isn't that interesting on GAE. Any new information to dispel the fact that GAE's Java is "single threaded" ? That seems pointless. Why even use Java, use something like server-side JavaScript, like http://www.freebase.com/docs/acre ? :-)

Regarding the second point: why would one be using shared references in code that one wanted purely functional for the purpose of making it decompose into parallel code without needing to worry about any of the issues raised? How could one be doing functional programming if side-effects are happening to the shared references? And how could one expect that code to decompose into parallel code w/o explicitly handling locking on shared references that are being changed?

As followup to GAE limitations, see http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine?pli=1 which points to 

https://sites.google.com/a/wildstartech.com/adventures-in-java/Java-Platform-Enterprise-Edition/JavaServer-Faces/sun-javaserver-faces-reference-implementation/configuring-jsf-20-to-run-on-the-google-appengine 

 <!-- Disable use of threading for single-threaded environments such as the Google AppEngine. -->

...

If the com.sun.faces.enableThreading context parameter is not set to false (or omitted completely from the web.xml configuration file) an application deployed on the Google AppEngine will display a number of different errors all caused by the JSF reference implementation's attempts to construct threads.

....

Sadly, current stable releases of Xwiki probably  won't be well supported on GAE, either, due to lack of proper database support. http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine?pli=1 states:

You cannot currently use Hibernate directly. The differences between the App Engine datastore and SQL were too great to get the standard Hibernate up and running under App Engine. App Engine does support JDO and JPA, so you may be able to convert your Hibernate code to use one of these ORM interfaces.

 

I believe, future releases of Xwiki will better support JDO/JPA through the XWQL effort. Current work, e.g.

http://xwiki1.appspot.com/ or http://xwiki1.appspot.com/bin/view/Main/ seems to be based on a non-hibernate-based datastore.

-- Niels

http://nielsmayer.com