Clojure alternatives to Esper

[Rogier Peters]

Hi,

For a java project I have been looking at Esper (esper.codehaus.org),
a component for complex event processing:

"Complex event processing (CEP) delivers high-speed processing of many
events across all the layers of an organization, identifying the most
meaningful events within the event cloud, analyzing their impact, and
taking subsequent action in real time (source:Wikipedia).

Esper offers a Domain Specific Language (DSL) for processing events.
The Event Processing Language (EPL) is a declarative language for
dealing with high frequency time-based event data.

Some typical examples of applications are:

Business process management and automation (process monitoring, BAM,
reporting exceptions, operational intelligence)
Finance (algorithmic trading, fraud detection, risk management)
Network and application monitoring (intrusion detection, SLA monitoring)
Sensor network applications (RFID reading, scheduling and control of
fabrication lines, air traffic)"

tl;dr:

My question is: is there an alternative for this in the clojure
ecosystem (since clojure seems like a good fit for this), and if not,
what kind of clojure components/libraries would be a good starting
point to implement someting similar

--
Rogier Peters
rogier@twitter, flickr, delicious

[Toby DiPasquale]

On Monday, April 23, 2012 9:51:39 AM UTC-4, Rogier wrote:

My question is: is there an alternative for this in the clojure
ecosystem (since clojure seems like a good fit for this), and if not,
what kind of clojure components/libraries would be a good starting
point to implement someting similar

I know of no complete alternatives, but there are some projects doing similar things for similar reasons. There is one that I saw recently that looks interesting:

https://github.com/aphyr/riemann

Stream processing for monitoring. Not quite sure why he didn't use ESPer for this, but there it is. There is also Lamina:

https://github.com/ztellman/lamina

which is more of a concurrency library but you could build a stream processing engine with it pretty simply.

--

Toby

[Mark Rathwell]

I would start with storm:

https://github.com/nathanmarz/storm

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

[Rogier Peters]

Hi Mark,

Thanks. Some work has been done by Thomas Dudziak on integrating storm
with esper [1][2].

What esper offers is:
a. queries (select average(price) from events where type=buy)
b. windows (the above for a window of 30 minutes)
c. patterns (if event a is not followed within 5 minutes by event b)
d. the dsl (everything can be done in sql-like query)

So it can probably be built in storm, but it's not as easy to use. And
since I don't need storm's distributed nature it could probably even
be done in raw clojure.

[1] http://tomdzk.wordpress.com/2011/09/28/storm-esper/
[2] https://github.com/tomdz/storm-esper

[Paul deGrandis]

I've had some success using Esper directly in Clojure for an internal
project at work.

Are you just looking to avoid interop? Are you just curious about
what other Clojure-specific options exist?
Or is there a specific tradeoff, design constraint, or quality
attribute you're working with?

Paul

On Apr 23, 1:49 pm, Rogier Peters <rogier.pet...@gmail.com> wrote:
> Hi Mark,
>
> Thanks. Some work has been done by Thomas Dudziak on integrating storm
> with esper [1][2].
>
> What esper offers is:
> a. queries (select average(price) from events where type=buy)
> b. windows (the above for a window of 30 minutes)
> c. patterns (if event a is not followed within 5 minutes by event b)
> d. the dsl (everything can be done in sql-like query)
>
> So it can probably be built in storm, but it's not as easy to use. And
> since I don't need storm's distributed nature it could probably even
> be done in raw clojure.
>
> [1]http://tomdzk.wordpress.com/2011/09/28/storm-esper/
> [2]https://github.com/tomdz/storm-esper
>
>
>
>
>
>
>
>
>
> On Mon, Apr 23, 2012 at 6:44 PM, Mark Rathwell <mark.rathw...@gmail.com> wrote:
> > I would start with storm:
>
> >https://github.com/nathanmarz/storm
>

[Rogier Peters]

Good questions. Mostly that it seems a technology domain (a dsl for
event streams) that would fit clojure well. If it hasn't been done,
interop would be no problem.

[ronen]

Iv been integrating Drools fusion with Clojure using a DSL to describe rules:

https://github.com/narkisr/gelfino/wiki/drools

I think it showcases nicely how interop can be done

Ronen

>> >> clojure+unsubscribe@googlegroups.com

>> >> For more options, visit this group at
>> >>http://groups.google.com/group/clojure?hl=en
>>
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Clojure" group.
>> > To post to this group, send email to clo...@googlegroups.com
>> > Note that posts from new members are moderated - please be patient with your first post.
>> > To unsubscribe from this group, send email to

>> > clojure+unsubscribe@googlegroups.com

>> > For more options, visit this group at
>> >http://groups.google.com/group/clojure?hl=en
>>
>> --
>> Rogier Peters
>> rogier@twitter, flickr, delicious
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to

> clojure+unsubscribe@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

[Yann Schwartz]

Since rxjs is being considered with ClojureScript (https://github.com/Reactive-Extensions/rxjs-jquery/blob/master/examples/KonamiCode.html), I would love to see something akin to Reactive Extensions ( http://msdn.microsoft.com/en-us/data/gg577609 and http://www.codefornuts.com/2010/05/reactive-extensions-rx-for-net-few.html for some examples) make its way into Storm...

Rx is not distributed, but its monadic goodness would be a great fit for clojure (and storm). 

[Rogier Peters]

Aphyr looks interesting, hadn't seen it before.

Mentioning lamina and reactive extensions reminded me of the clojure
asynchronous events page,
http://dev.clojure.org/display/design/Asynchronous+Events which I
probably should read once more ;)

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

[Paul Ingles]

Hi,

I agree- I think Clojure would be very well suited to solving these
problems. I'd used Esper a little in the past so started doing some
digging around what would be involved in replicating it. In the end, I
started working on a little lib to make working with Esper a bit
easier (http://github.com/pingles/clj-esper).

Since then I've been thinking of having another go, and took a look
through Riemann (http://aphyr.github.com/riemann/) and Pulse (https://
github.com/heroku/pulse) recently for inspiration. I also found a few
papers that covered efficient (time + space) algorithms for some of
the statistical measures that were important for me (medians +
quantiles) but haven't had the time to implement them- Esper already
solved our problem well enough :)

Paul