Periodic tasks

[Stefan Arentz]

What is a good and simple way to run periodic tasks in Clojure? I need
to run a simple function every couple of minutes. And make sure that
if it throws an exception that it won't kill the periodic task.

I come from a Spring world where XML, Timers, Jobs and Quartz rule the
world, so am hoping for something small and elegant for Closure :-)

S.

[John Harrop]

See 

http://groups.google.com/group/clojure/browse_thread/thread/5b197faf820ab77c/ 

or another recent thread about "actors". You'll need to wrap your function body in (try ... (catch Throwable _ xxx)) to halt exceptions. (The "xxx" can be any logging action you want to take, or nothing.)

[Albert Cardona]

How about:

(import '(java.util.concurrent Executors TimeUnit))

(let [s (Executors/newSingleThreadScheduledExecutor)]
(.scheduleAtFixedRate s
#(try
(println "I did it again")
(catch Exception e
(.printStackTrace e)))
(long 0)
(long 1)
TimeUnit/SECONDS))

Admittedly very java-ish.

Albert
--
http://albert.rierol.net

[AndrewC.]

On 30 Oct, 16:18, Albert Cardona <sapri...@gmail.com> wrote:
> How about:
>
> (import '(java.util.concurrent Executors TimeUnit))

...<snip>...

> Admittedly very java-ish.

Personally, I think Java-ish is the way to go here. John's actor lib
is pretty nifty, but it is relying on implementation details of the
threading of agent send handlers to try and ensure timely activity.
This seems like a case where we need certain functionality, Java has a
good library (the Executor framework, or Quartz), therefore we should
just use it.

[alxtoth]

Hi

Why not use the OS task scheduler? On un*x there is good old cron or
at. On windoze there is similar task scheduler.

If you think a little that's what it takes: simple reliable and if it
fails with whatever error will start again in 2 minutes. Just don't
forget to write to log files , and rotate those files sometimes.

I am sure there are more fancy solutions than cron, and some that
don't require megabytes of XML as the usual Java stuff

-Alex

[cody koeninger]

On Oct 31, 5:22 am, alxtoth <alexandru.t...@gmail.com> wrote:
> Why not use the OS task scheduler? On un*x there is good old cron or
> at. On windoze there is similar task scheduler.
>

Overhead from starting and stopping the JVM every couple of minutes
would probably be unacceptable. My understanding is that solutions
like Nailgun don't work correctly with Clojure either.

[Luke VanderHart]

Why not just run an agent that does something, then calls sleep for N
seconds, then calls the next thing?

Granted, it will eat up a thread in your agent thread pool, but if
you've only got one of these in the app it shouldn't be a problem.

Or you could go the Java route, and start a daemon thread manually.

-Luke

[Richard Newman]

>
> Overhead from starting and stopping the JVM every couple of minutes
> would probably be unacceptable. My understanding is that solutions
> like Nailgun don't work correctly with Clojure either.

VimClojure relies on Nailgun, with a bunch of people on this list
using it with Clojure every day.

[cody koeninger]

On Oct 31, 11:42 am, Richard Newman <holyg...@gmail.com> wrote:
> VimClojure relies on Nailgun, with a bunch of people on this list  
> using it with Clojure every day.

My recollection from list and IRC was that (aside from random nailgun
issues + the project not being updated in 4 years) there was an issue
with dynamic classes being created and never expired. Is that
inaccurate? I'm also not sure how one would work around the security
issues of running a single nailgun for a variety of tasks that needed
different user permissions

I'd love to hear success stories from people using nailgun to actually
run frequent scripted tasks out of cron, as opposed to for
development. It would make clojure more palatable for my work
environment.

[John Harrop]

On Sat, Oct 31, 2009 at 12:36 PM, Luke VanderHart <luke.va...@gmail.com> wrote:

Why not just run an agent that does something, then calls sleep for N
seconds, then calls the next thing?

Granted, it will eat up a thread in your agent thread pool, but if
you've only got one of these in the app it shouldn't be a problem.

Or you could go the Java route, and start a daemon thread manually.

Or you could use send-off instead of send; send-off doesn't consume from a finite thread pool and is thus suitable for long-running (or blocking, e.g. I/O) tasks.

Incidentally, if you ever want to simply reset the state of an agent to a particular value, analogously to ref-set and reset!, this can work:

(defn set-agent [a v] (send a (fn [_] v)))

[Emeka]

John,

 

Is like I am missing something? This thread mentioned making periodic tasks, there was a java example to it. However, what you posted I am yet to figure out how to make a periodic tasks possible.

 

Regards,

Emeka

[John Harrop]

On Sun, Nov 1, 2009 at 3:20 PM, Emeka <emeka...@gmail.com> wrote:

John,

 

Is like I am missing something? This thread mentioned making periodic tasks, there was a java example to it. However, what you posted I am yet to figure out how to make a periodic tasks possible.

Agents can perform periodic tasks, if under some conditions they send to themselves.

Also, in my earlier post I linked to another post of mine. The pq-agent-jobs and ppq-agent-jobs there allow giving agents jobs that are done in an order determined by a priority queue instead of in the order the jobs were enqueued. In another thread I posted "actor" code that explicitly set up jobs to be done periodically: http://groups.google.com/group/clojure/msg/b20c9390046b9746

[Jonathan Smith]

Maybe I'm confused, but can't you just do a regular java thread for
this?

(defn periodicly [fun time]
"starts a thread that calls function every time ms"
(let [thread (new Thread (fn [] (loop [] (fun) (Thread/sleep time)
(recur))))]
(.start thread)
thread))

(periodicly #(println "foo foo foo") 3000)

I'm not really sure about the error handling thing, but I guess you
could wrap the function in a try-catch where it reinitialized itself.
(although I guess that could get dangerous if you start throwing a lot
of errors).

[John Harrop]

On Sun, Nov 1, 2009 at 7:39 PM, Jonathan Smith <jonathan...@gmail.com> wrote:

Maybe I'm confused, but can't you just do a regular java thread for
this?

(defn periodicly [fun time]
 "starts a thread that calls function every time ms"
 (let [thread (new Thread (fn [] (loop [] (fun) (Thread/sleep time)
(recur))))]
   (.start thread)
   thread))

(periodicly #(println "foo foo foo") 3000)

I'm not really sure about the error handling thing, but I guess you
could wrap the function in a try-catch where it reinitialized itself.
(although I guess that could get dangerous if you start throwing a lot
of errors).

The actor code makes it easy to start, stop, and modify the periodic task and not just to start it. :) 

[MarkSwanson]

> I'd love to hear success stories from people using nailgun to actually
> run frequent scripted tasks out of cron, as opposed to for
> development. It would make clojure more palatable for my work
> environment.

Nailgun has been a boon to me. I don't believe Nailgun has a problem
with dynamic classes - only that you have to remember to stop/start
Nailgun to ensure its ClassLoader gets the latest classes (I would be
genuinely interested in any details that show otherwise). F.E. I can:
1. (def q (vec (range 1000000)))
2. use jvisualvm to see q takes up ~20MB RAM; 20 bytes / Integer.
3. (def q "abc")
4. use jvisualvm and it will show the 20MB has been GC'd just fine.
(I'm not sure what sort of dynamic class / unload/ GC? issue you were
referring to).

Since it takes Spring a long time to instantiate all of the configured
objects using Nailgun really does save me time. I have a lot of nifty
command line tools that help me resolve support issues and waiting for
Spring (not the JVM really) to instantiate and wire everything up
really slowed me down. Nailgun solves the speed problem and I haven't
had any problems with it.

[sross]

You could also try cron4j which, while less customisable than quartz,
is far simpler to use.
It also accepts Runnable's as as tasks which allows for much easier
clojure integration.

user=> (def sched (it.sauronsoftware.cron4j.Scheduler.))
#'user/sched

user=> (.schedule sched "* * * * *" #(println "Hello, World!"))

"dca735307ae35bb7649a8a6800000124bec35ff926b54d06"
user=> (.start sched)