roll call of production use?

[Raoul Duke]

hi,

i'd be interested to hear who has successfully used clojure in
production. i know of some, as some folks have been vocal; any other
interesting-but-so-far-silent uses people'd be willing to fess up
about?

many thanks.

[David Brown]

I've thrown together a small clojure program to present an internal
dashboard of active projects and states and stuff. Having the JVM was
invaluable, since I could easily make postgreSQL queries as well as
use JGit to walk the history of a git repository.

I've mentioned this before, but <http://github.com/d3zd3z/webweight>
is a small program I wrote to learn compojure.

One significant part is that I've written an org.davidb.contrib.html
and xml that wrap around the same data structure used by clojure.xml.
It allows for easier construction of html/xml in code:

(html/table :border "1"
(html/tr
(html/td :valign "top" "Cell") ..))

I also wrote an html/xml emitter that uses ones from the standard Java
libraries, since the one in clojure.xml doesn't actually generate
valid xml.

The other interesting part of webweight is that the build.xml and the
ivy and files under etc are sufficient to build most Clojure projects.
It's similar to the idea behind Leiningen, but it will also build Java
and native code into the project as well.

The production webpage is internal, and has a lot more data on it, but
I've captured and scrubbed a snapshot
<http://www.davidb.org/dashboard/sample.html>

I was able to throw the whole thing together in just a couple of days
using Clojure. I haven't really felt this productive writing code
since using Common Lisp.

David

[Wojciech Kaczmarek]

I use Clojure for web crawling / scraping of certain services with
storing extracted data in a custom database. I could marry existing
Java libs with an excellent xml/html walking via zippers; and could
test it in REPL which is a pleasure to use. After finishing a
prototype it became a valid production version which Just Works in a
near-zero time. Ah, and I eradicated many bugs early by having set
*warn-of-reflection* - esp. for the code heavily calling Java.

Nothing revolutionary, it was just use of typical web-related
techniques, which are unfortunately still a major pain in the ass on
many programming platforms (and they don't have macros!! ;)

More uses incoming. It will be bulk data load of QDBM and TokyoCabinet
storages, for example.

[Phlex]

> i'd be interested to hear who has successfully used clojure in
> production.

Hello,

1- We have this license server, used to control the use of a
professional software (this one written using delphi).
This was tested with thousands of simutaneous connections, and has been
working with no hiccup for 6 months. It uses nio + ssl + a mysql database.

2- An application that connects to telephony switches and gives real
time call informations to web users. With the whole administration part
(managing customer, agents and their different price lists, showing
statistics). It has many users permanently logged in as they need to
monitor those calls (they are call shops). So they can see the calls
counting for their (typically 8) booths, and can do various operation on
the calls (like interrupting a prepaid call for instance). That's a lot
of polling. We're still in the ramp-up period so we only have a fraction
of the existing customers using our application yet. This project uses
compojure + jetty + ssl + a postgres database.

There isn't much to report, everything worked as expected. The jvm
integration was a big help, providing access to some java third party
libraries. Also using java, i can develop on windows, and just drop the
jar on a solaris or linux server, it just works.
The REPL helps a lot too.

I'd like to see support for database connection pooling, and those pesky
sql underscored column names to end up with dashes on the clojure side.
Also compojure has somewhat of an attitude. Actually i miss cl-sql and
hunchentooth, but hey you can't get everything, and what we've got is
already fantastic.

The language itself is perfect for the job, though i'm looking forward
to the protocols and deftype extensions. The type metadata trick always
seems wrong, and including it in a map feels like a hack. Refs, atoms
and agents helped a lot, as well as the immutable data structures.

There it is, an application programmer's view on clojure !

Sacha De Vos

[John Harrop]

On Mon, Nov 23, 2009 at 8:35 PM, Phlex <Ph...@telenet.be> wrote:

> i'd be interested to hear who has successfully used clojure in
> production.

Hello,

1- We have this license server, used to control the use of a
professional software (this one written using delphi).

What are the ethics of using an open source product like Clojure to implement DRM restrictions for some other product? Seems there might be something a bit iffy there -- if not legally, perhaps morally.

[Richard Newman]

1- We have this license server, used to control the use of a
professional software (this one written using delphi).

What are the ethics of using an open source product like Clojure to implement DRM restrictions for some other product? Seems there might be something a bit iffy there -- if not legally, perhaps morally.

I don't agree. So long as they abide by the Clojure license, everything is A-OK… and the Clojure license doesn't impose restrictions on its use for this purpose.

It's only morally "iffy" if you consider preventing contract violation (which is what a license server does) to be iffy… in which case there'd be no problem with breaking the Clojure license, no?

This use of Clojure is internally consistent, and so I suspect that you're simply slightly offended by the idea of someone making money by using open-source software. In that case, I'd suggest you look first at Red Hat (market cap: $5.09B), and the Linux community's attitude towards them (generally positive).

Probably you'd be more cheerful if Phlex's company contributed to the Clojure community. (For all I know, they do.)

Anyway, apologies for possibly starting a "closed-source is evil" debate. Let's hope it fizzles.

[Phlex]

1- We have this license server, used to control the use of a
professional software (this one written using delphi).

What are the ethics of using an open source product like Clojure to implement DRM restrictions for some other product? Seems there might be something a bit iffy there -- if not legally, perhaps morally.

-- 

Ah nice to see you've finally adopted a lisp !

This customer's request seemed reasonable, and the project looked like a perfect test bed for clojure.
The scope was small, and the requirements well defined. It involved multithreading, networking, databases and long up-times.

The ethics are not mine to define. I'm just programmer, and will leave this discussion to you philosophers.
Once a consensus will be reached you may drop me a mail.
In the meantime, I'll just go hack at my next evil project.

Sacha De Vos

[mbrodersen]

I use an internal DSL (Domain Specific Language) in Clojure to
generate C++ and C# code.

Cheers
Morten

[John Harrop]

On Mon, Nov 23, 2009 at 9:47 PM, Richard Newman <holy...@gmail.com> wrote:

1- We have this license server, used to control the use of a
professional software (this one written using delphi).

What are the ethics of using an open source product like Clojure to implement DRM restrictions for some other product? Seems there might be something a bit iffy there -- if not legally, perhaps morally.

I don't agree. So long as they abide by the Clojure license, everything is A-OK… and the Clojure license doesn't impose restrictions on its use for this purpose.

Hence my "if not legally".

 

This use of Clojure is internally consistent, and so I suspect that you're simply slightly offended by the idea of someone making money by using open-source software. In that case, I'd suggest you look first at Red Hat (market cap: $5.09B), and the Linux community's attitude towards them (generally positive).

Oh, I have no problem with making money by using open source software, when it's done in the manner that companies like Red Hat do it. It's the use to lock down some piece of proprietary software even more than it already is that seems, at the very least, ironic.

[Robert Campbell]

I've deployed two small mashup apps which combine OpenCalais and our
content repository to annotate documents with metadata (named
entities, relationships, etc) and expose the results over the web.
Good experiences all around, including with the clojure-http-client
and saxon wrapper libs + Compojure.

I did miss having a nice Clojure RDF lib, so maybe I'll make a Jena
wrapper in the future. Parsing the XML-RDF as pure XML w/xpath was
enough for my purposes.

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

[Jay Fields]

DRW (http://drw.com) uses Clojure for several production applications.

Cheers, Jay

[Meikel Brandmeyer]

Hi,

On Nov 24, 6:06 am, John Harrop <jharrop...@gmail.com> wrote:

> Oh, I have no problem with making money by using open source software, when
> it's done in the manner that companies like Red Hat do it. It's the use to
> lock down some piece of proprietary software even more than it already is
> that seems, at the very least, ironic.

A license server does not necessarily mean restriction. We have a
small in-house app which is also handled via a license server. It just
records, who uses the app and charges 25€ per year and user on the
cost center of the using department. No restriction whatsoever.

To say something productive for the discussion:

I use Clojure to drive our regular quality reporting. It takes
information from different source, mainly different SAP systems but
also supplier data provided via Excel. Everything is imported into
database and after a lot of JOINs, SELECTs and UNIONs a pdf containing
the charts with a home-grown analysis is generated. The charts and pdf
are still work in progress, but everything else works nicely so far.

Sincerely
Meikel

[John Harrop]

On Tue, Nov 24, 2009 at 1:04 AM, Meikel Brandmeyer <m...@kotka.de> wrote:

Hi,

On Nov 24, 6:06 am, John Harrop <jharrop...@gmail.com> wrote:

> Oh, I have no problem with making money by using open source software, when
> it's done in the manner that companies like Red Hat do it. It's the use to
> lock down some piece of proprietary software even more than it already is
> that seems, at the very least, ironic.

A license server does not necessarily mean restriction. We have a
small in-house app which is also handled via a license server. It just
records, who uses the app and charges 25€ per year and user on the
cost center of the using department. No restriction whatsoever.

Not enabling people to use it for free = restriction, given that marginal increased use by them does not correspond to marginal increased costs to you. (If it means marginal increased costs for support, why not sell the support and let them use it for free unsupported if they choose to? Charge per incident, or for a maintenance plan, or whatever. Red Hat makes millions doing that.)

 

To say something productive for the discussion:

I use Clojure to drive our regular quality reporting. It takes
information from different source, mainly different SAP systems but
also supplier data provided via Excel. Everything is imported into
database and after a lot of JOINs, SELECTs and UNIONs a pdf containing
the charts with a home-grown analysis is generated. The charts and pdf
are still work in progress, but everything else works nicely so far.

There's a Clojure or a Java library for generating pdf? 

[Rich Hickey]

On Nov 23, 9:47 pm, Richard Newman <holyg...@gmail.com> wrote:

> Anyway, apologies for possibly starting a "closed-source is evil"  
> debate. Let's hope it fizzles.

Yes, please, let's end this here. Any further non-Clojure content on
this thread might be moderated.

Thanks,

Rich

[RandyHudson]

On Nov 24, 11:30 am, John Harrop <jharrop...@gmail.com> wrote:

> There's a Clojure or a Java library for generating pdf?

Apache FOP is an XSL-FO processor than can generate PDF documents.
I've heard good things about iText, a Java library for generating or
modifying PDF docs.

[Meikel Brandmeyer]

Hi,

Am 24.11.2009 um 17:30 schrieb John Harrop:

> There's a Clojure or a Java library for generating pdf?

I use enlive to generate a HTML version with CSS of the report. This
is turned by virtue of flying saucer and iText into PDF. At least that
is the plan. Whether it will work satisfactory is a different story.
At least the initial results look promising. :)

Sincerely
Meikel

[Anniepoo]

At my workplace (University of Houston, dept. of Health and Human
Performance) Clojure is our primary language for interacting with our
virtual world presence in Second Life.
We have an automated lesson path building system currently in
production, and several other projects in various states.

automated lesson path building system - user (currently it's too
scratchy to turn the profs loose on this, so I always use it) draws a
learning path out in a 2D swing based clojure app. It compiles a set
of 'prims' that fill the needed shapes (a sort of variation on the
knapsack problem, where you also get a small 'rubber' block that can
fill in small spaces, but it's an optimal packing problem), creates
the textures, controls a C# robot (no java libs for that task) to
upload the textures, and sends commands to a robot in-world to do the
actual build.

Nutrition Goggles system - What I should be doing today, if I weren't
noodling around in Clojure 8cD
(OK, in theory the injectable agent system I'm doing today should be
usable for all these projects)
This is a HUD (tool that attaches to the users screen as additional
UI), when they look at a food item in the virtual environment it shows
the nutrition data. They also can enter nutrition data the same way.
Back end in clojure.

Summoner monitor system - I'm supposed to write this, I'd dearly love
to get there, as it's a tool that alerts me
that I'm needed in world to help a student. (currently I depend on my
email alerter, which is suboptimal)

Configurable teleporting sign system - a web app that lets users set
up teleporters around the campus sim, while retaining some control
over the appearance of the central entry area. Might not end up being
in clojure, I might do this totally in world.

[Adrian Cuthbertson]

For a data analysis/reporting application, I use colt for some matrix
processing, joda time for period calculations, jfreechart to generate
charts, an adaptation clj-html to create some dynamic html, also now
some StringTemplate templates to bring in and manipulate static
content and finally all this (as html) gets fed into XhtmlRenderer to
emit some beautiful css stylized PDF reports.

Sounds like a mish-mash and it is, but clojure has allowed me to
create all the right abstractions, manage the processing flows and
build a complex application leveraging many powerful java libraries
that would have just been too much effort (for me) to do in plain old
java.

The other spin-off of this is that using the repl, one is able to
really explore the api's of these big libraries dynamically and get to
know them much more intimately than when doing static compilation/run
cycles as in the past. So, a major benefit for me of learning clojure
has also been a substantial increase in my depth of knowledge of the
workings of some of the excellent java libraries out there.

- Adrian.

[eyeris]

I used clojure to write an application that interpolates SQL results
(from clojureql) into Excel spreadsheets (using Java interop to
Apache's POI lib). I have a two users putting this to use for their
project progress reporting.

[Daniel Simms]

--
Daniel Simms <dsi...@dsimms.com>


On Nov 24, 2009, at 22:23, Adrian Cuthbertson <adrian.cu...@gmail.com

> wrote:

> The other spin-off of this is that using the repl, one is able to
> really explore the api's of these big libraries dynamically and get to
> know them much more intimately than when doing static compilation/run
> cycles as in the past.

Oh, +1 to that. Similarly, having the repl in production contexts is
super useful. (No, the repl-ness isn't unique to clojure but it's
really useful.)
>

[Stefan Kamphausen]

Hi,

we replaced a command line interface for an internal, mission-critical
application with a custom Clojure-REPL. It is used for administrative
tasks as well as testing.

Cheers,
Stefan

[John Harrop]

We're maintaining a large database of tagged images and had a need to perform "fuzzy search" of the database. The existing search tool takes exact queries only. So it was necessary to hack up a little tool to sit between the query source and the engine and transform the query into a "fuzzy query". You can think of it like this: the input query is something like x AND y and the output is something like (x AND (y OR y2 OR y3)) OR (y AND (x OR x2 OR x3)) where y2 and y3 are in some sense "near" y and x2 and x3 "near" x. The query transformation is simple enough, but it was really handy being able to test it at a REPL, and even hot-swap modifications and do live testing and tweaking of e.g. the neighborhood graphs used to "fuzzify" query terms.

(Right now it only allows "slop" in one query term; sort of a Hamming-distance-1 matcher. Extending it further would invite a combinatorical explosion as well as lead to noisier, less useful results. Distance 1 seems to be the "sweet spot" for our app.)

The code takes the input query, takes apart the terms, and generates seqs of alternatives, with some (butlast (interleave foo (repeat "OR"))) type stuff here and there, and cobbles them together again using seq functions, str, and java.lang.String methods. (Some query terms need to be parsed, e.g. are a hyphenated entity the first part of which should be fuzzy-matchable while the second part should stay constant, etc.)

Rather boring? Little things like this incrementally improve services. Two guys hacking like this in their garage went on to start Google. :) We don't have such lofty aspirations, but it's still something to keep in mind.

[Joel Westerberg]

I have written a production web app with compojure. It was a very nice
experience. First project I have pulled of totally TDD and the
functional aspect of clojure really makes tests so much easier and
more truthful, although there sometimes seems to sneak in some
exceptions I didn't expect from the java side.

It was a small ad campaign site with some SMS functionality, with web
pages formatted for mobile phones. Really nice to be able to just wrap
our java libs for sending SMS and device recognition. The compojure
HTML generator is fantastic, extremely powerful and easy to use. I am
amazed how compact the end code is.

I deployed a war file on tomcat 5.5. The app is stable and very fast.
One thing I would have liked to have was to start swank-server from
the war to provide an easy way to do hot updates and inspection, but I
ran into some problem with *1 etc not being defined so I gave up on
that. I prefer to deploy a war because I did not want to write my own
init.d script and bug test that.

Overall - a very nice experience.

[prhlava]

Hello,

I have done a smallish project (mainly to stop my friend doing some
monkey
work) - it basically opened an ms excel spreadsheet (using library
developed
by apache project folk), validated input, categorised groups of rows
by needed criteria and
wrote the results to a new spread-sheet (formated), with interesting
results on separate page
(these were done by hand before).

In the proces, I have re-written the application once (as I learnt
more over time +
switched to apache library for excel files) and was very pleased with
the result (the code) -
both, performance and "beauty".

Kind regards,

Vladimir

[Jamie]

On Nov 23, 5:00 pm, Raoul Duke <rao...@gmail.com> wrote:

Our real-world use reported here:

http://groups.google.com/group/clojure/browse_thread/thread/ffcd4bc722852b4/d07a1ca449e83a8b

Summary: Moderate-scale data processing. Millions of records.
Timeseries and regressions. Works great. Based on this pilot, we're
expanding our use of Clojure.

It's surprising to me just how solid this innovative platform is.
Outstanding work.

Thanks again to Rich and the other contributors.

--Jamie

[Luc Préfontaine]

[Luc Préfontaine]

> On Nov 23, 5:00 pm, Raoul Duke <rao...@gmail.com> wrote:
>
> > i'd be interested to hear who has successfully used clojure in
> > production. i know of some, as some folks have been vocal; any other
> > interesting-but-so-far-silent uses people'd be willing to fess up
> > about?

http://www.infoq.com/news/2009/01/clojure_production

The bus has been up for months now and down times have been related
to either some hardware changes (system is fully redundant so most
of the time no downtime occurred)
and network stability issues over which we have little control (managed
by the customer's IT group).

We are adding more functions tapping on the information flowing on the
bus, of course these are now written in Clojure...

Census tracking, decentralized service request management, ...

This year the bus will also extend to interconnect other hospital
services. The bus is also a way to convince users to move to a paperless
environment and focus on data entry completeness to achieve that goal.

We have also in the works a Clojure/Terracotta/Java messaging layer to
improve parallelism. This should be rolled out in summer time.

Transitioning to Clojure V1.0 was not a problem and we do not expect
that moving 1.1 will prove difficult.

The bus is still running on a cluster of small foot print computers but
we moved to Atom 330 motherboards to boost processing power.

This thing will celebrate it's first year in production very soon
without significant pain and this a proof to me that Clojure is mature.


Luc

[Daniel Werner]

On Dec 1, 5:20 pm, Luc Préfontaine <lprefonta...@softaddicts.ca>
wrote:
> http://www.infoq.com/news/2009/01/clojure_production

Slightly off-topic:

What prompted you to choose ActiveMQ over other popular message bus
systems like RabbitMQ? Was it the ease of operability with Clojure/
Java, or are there other strong factors?

[Luc Préfontaine]

We picked one that met our minimal requirements... our prototype was in Java however and that probably biased
the choice a bit.

But for us it's an intermediate step. We need a more flexible solution.
We will keep ActiveMQ in the picture to link different clusters (or an alternative) but for intra cluster processing
we want something more in line with the contraints we need to meet especially regarding event serialization.

We think that custom fit message serialization is not easily managed using a message bus generic layer.
We need a more specific solution and some management hooks to deal with this in day to day operations.

Luc