Scheduling tasks in dropwizard services

[Andrew]

All,

What is the best way to schedule a repeating task in a dropwizard service?

I need to update some data in my service (a) every hour, and (b) at a specific time every day.

I could use ScheduledExecutor and other java.util.concurrent classes, but I'd really like to use Quartz, but I'm not sure where to start with integrating the two.

Any tips?

Thanks

Andrew

[Brian O'Neill]

I'm not sure this is the best answer, but we've found a lot of benefit in integrating some of Spring's facilities with Dropwizard. (for JMS, JPA, Workflow, Scheduling, etc.)

We're not in love with our solution and we'd be glad to hear if others have a better integration strategy, but for now its working.  

You can see the details here:

http://brianoneill.blogspot.com/2012/05/dropwizard-and-spring-sort-of.html

By integrating Spring, you should be able to use Spring's scheduling facilities:

http://static.springsource.org/spring/docs/3.0.5.RELEASE/reference/scheduling.html

If you go this route, let me know how it turns out.

-brian

--
Brian ONeill
Lead Architect, Health Market Science (http://healthmarketscience.com)
mobile:215.588.6024
blog: http://weblogs.java.net/blog/boneill42/
blog: http://brianoneill.blogspot.com/

[Ryan Kennedy]

I've never done this, but you could wrap Quartz as a managed object:

http://dropwizard.codahale.com/manual/core/#managed-objects

Then you could ship your schedule as an XML resource. Dropwizard will fire up your Managed instance (starting Quartz in the process) on startup and shut it down on shutdown.

-- 

Ryan Kennedy

[Tatu Saloranta]

On Mon, May 28, 2012 at 10:14 PM, Ryan Kennedy <rcke...@yahoo.com> wrote:
> I've never done this, but you could wrap Quartz as a managed object:
>
> http://dropwizard.codahale.com/manual/core/#managed-objects
>
> Then you could ship your schedule as an XML resource. Dropwizard will fire
> up your Managed instance (starting Quartz in the process) on startup and
> shut it down on shutdown.

And for what it's worth, after using a tiny subset of Quartz earlier,
I found cron4j [http://www.sauronsoftware.it/projects/cron4j/] a
really nice alternative for the stuff I actually need, and little
more. Quartz definitely has way more stuff, which may be good thing
for many users.

-+ Tatu +-

[Arul Dhesiaseelan]

I put together a simple example using Flux (https://github.com/aruld/dropwizard-flux-sample). Feel free to fork and adapt it to Quartz. It is that simple with DW.

-Arul

[Tim Molter]

I've integrated Xeiam's Sundial scheduler library into DropWizard for running scheduled tasks which you can see here: https://github.com/timmolter/XDropWizard

Sundial is a fork of Quartz that I created a while ago. I stripped out tons of the bloat that is in Quartz. It depends only on slf4j and the jar size is only 250Kb. The artifacts are on MavenCentral. If you prefer Quartz, you can integrate Quartz in roughly the same way. You'll just have more dependencies, a bigger footprint, and a more complicated configuration.

With Sundial integrated into DropWizard, you can simply create a class that extends com.xeiam.sundial.Job and either run it on a schedule and/or fire it off by calling "curl -X POST http://localhost:9091/tasks/myjob" for example, the same mechanism DropWizard promotes for running tasks.

The example above (https://github.com/timmolter/XDropWizard) runs a Job called SampleJob3 every 45 seconds which just logs some stuff. The cron expression is found in jobs.xml.

[Jared Stehler]

Here's a simple wrapper for integrating the real quartz into dropwizard: https://github.com/jaredstehler/dropwizard-quartz

[Jared Stehler]

But, I do like the integration with the DW tasks resource for running jobs ad hoc!

[Barclay Dunn]

I'm having trouble getting this wrapper to execute my job. I added the following to my service's run method:

        MPDSDIModule mpdsDIModule = new MPDSDIModule(configuration, hibernate);

        DwQuartzModule dwQuartzModule = new DwQuartzModule();

        dwQuartzModule.configure();

        Scheduler scheduler = dwQuartzModule.provideScheduler();

        scheduler.start();

        Injector injector = Guice.createInjector(mpdsDIModule, dwQuartzModule);

When I start my service, I do show output that the scheduler has been instantiated and, later, has been started:

     INFO  [2013-10-29 15:30:34,428] org.quartz.core.QuartzScheduler: Scheduler meta-data: Quartz Scheduler (v2.2.1) 'MPDSScheduler' with instanceId 'NON_CLUSTERED'

       Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.

       NOT STARTED.

       Currently in standby mode.

       Number of jobs executed: 0

       Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 5 threads.

       Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

     INFO  [2013-10-29 15:30:34,428] org.quartz.impl.StdSchedulerFactory: Quartz scheduler 'MPDSScheduler' initialized from default resource file in Quartz package: 'quartz.properties'

     INFO  [2013-10-29 15:30:34,428] org.quartz.impl.StdSchedulerFactory: Quartz scheduler version: 2.2.1

     INFO  [2013-10-29 15:30:34,428] org.quartz.core.QuartzScheduler: Scheduler MPDSScheduler_$_NON_CLUSTERED started.

I have a basic job that is annotated to run every 5 mins, just to see if it runs:

     @Scheduled(interval = 5, unit = TimeUnit.MINUTES)

     public class MerchantNexusUpdateJob implements Job {

         public void execute(JobExecutionContext context) throws JobExecutionException {

             logger.info("starting MerchantNexusUpdateJob.execute method");

             ...

             logger.info("ending MerchantNexusUpdateJob.execute method");

         }

     }

I have debug statements in the execute method of the job, but I never see them and surmise that the job is never executed.

I'd appreciate any help setting this up. It's a nice little wrapper, super simple and I'd prefer to use it if possible.

Thanks,

Barclay

On Wednesday, April 3, 2013 4:49:18 PM UTC-4, Jared Stehler wrote:

[Barclay Dunn]

Ugh, the problem stems from our having some shared code we have written for several services to use. So my class definition right off the bat is the problem. I cannot extend AutoConfigService... I will have to find another way.

[Kyle Boon]

I prefer to create a dropwizard task for updating the data and then using an outside entity (a separate quartz server, cron, chronus) to execute that task. We're trying to make sure there is a single place where all scheduled tasks across the organization can be monitored and tracked. 

[Lance N.]

On the topic, Jenkins is a nice enterprise scheduler for (very) coarse-grained tasks. It tracks history, makes charts, has a nice editor, chains tasks, sends mail, has hysteresis for sending mail, has a ton of integration plugins like send-to-IRC, has APIs, etc.

Lance

[Tim Molter]

I've recently released a project called dropwizard-sundial, which nicely integrates Sundial into a Dropwizard app, adding many new convenience features. With [dropwizard-sundial](https://github.com/timmolter/dropwizard-sundial), you can easily integrate the lightweight multi-threaded Java job scheduling library, [Sundial](https://github.com/timmolter/Sundial), and add Jobs either using a `SimpleTrigger` or a `CronTrigger`. First, you'd start out with a class defining your job logic with a `SimpleTrigger` annotation: 

    @SimpleTrigger(repeatInterval = 60, timeUnit = TimeUnit.SECONDS)    
    public class SampleJob extends com.xeiam.sundial.Job {
       
      @Override
      public void doRun() throws JobInterruptException {
        // Do something interesting...
      }
    }

A `CronTrigger` annotation is also available (@CronTrigger(cron = "0/5 * * * * ?")). In your Dropwizard app's `yaml` file, you'll need to define what package `dropwizard-sundial` should search for annotated job classes. Following is an example config with the `annotated-jobs-package-name` config param along with several other optional params for fine-tuning the scheduler:

    sundial:
      thread-pool-size: 10
      shutdown-on-unload: true
      wait-on-shutdown: false
      start-delay-seconds: 0
      start-scheduler-on-load: true
      global-lock-on-load: false
      annotated-jobs-package-name: com.foo.bar.jobs

Additionally, you can control the scheduler asynchronously via Curl while the app is running to do things like locking and unlocking the scheduler, starting, stopping, adding, removing jobs and triggers. Here are a few examples:

    curl -X POST http://localhost:9090/admin/tasks/locksundialscheduler
    curl -X POST http://localhost:9090/admin/tasks/unlocksundialscheduler
    curl -X POST "http://localhost:9090/admin/tasks/startjob?JOB_NAME=MyJob"

[Jeyashree Narayanan]

Hi all,

I am trying to use dropwizard-sundial and am having trouble with a resource. Not sure if its classpath issue or if I am failing to register resources properly.

This is my application class' run method:

public void run(DataLoaderApplicationConfiguration configuration, Environment environment) throws Exception {

    logger.info("Started DataLoader Application");

    final String template = configuration.getTemplate();

    environment.healthChecks().register("TemplateHealth", new TemplateHealthCheck(template));

      // JOBS

    environment.jersey().packages("com.yyy.dataloader.jobs");

}

I get the following error at runtime.

INFO  [2015-04-07 15:00:19,737] com.xeiam.sundial.plugins.AnnotationJobTriggerPlugin: Loading annotated jobs from com.yyy.dataloader.jobs.

[WARNING] 

java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:601)

at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)

at java.lang.Thread.run(Thread.java:722)

Caused by: java.lang.RuntimeException: Unexpected problem: No resource for com/yyy/dataloader/jobs

at org.quartz.classloading.CascadingClassLoadHelper.getJobClasses(CascadingClassLoadHelper.java:217)

at com.xeiam.sundial.plugins.AnnotationJobTriggerPlugin.start(AnnotationJobTriggerPlugin.java:72)

at org.quartz.QuartzScheduler.startPlugins(QuartzScheduler.java:1102)

at org.quartz.QuartzScheduler.start(QuartzScheduler.java:211)

at com.xeiam.sundial.SundialJobScheduler.startScheduler(SundialJobScheduler.java:102)

[Tim Molter]

@ Jeyashree 

I was able to recreate that exception if I specified a package in the yml file (annotated-jobs-package-name: com.foo.bar.jobs)
, which wasn't on my class path. This leads me to believe you have a classpath issue and that package missing.

~Tim

[Jeyashree Narayanan]

Thanks Tim. Appreciate the quick response. If I remove the jobs.xml file, and do not specify the annotated-jobs-package-name, and instead use the ad-hoc method of adding a job, I was able to do it fine, which led me to the same conclusion as you. Finally figured out the the resources directory was incorrectly set in my pom.xml file. Once that was fixed, was able to pick up the jobs correctly. Thanks.

Now that I have your attention, would appreciate your help on another issue - this time in Yank. I am connecting to MySQL DB and I get the error "Unable to get driver". In my DB.properties file, I am trying to specify the driver but I dont see any documentation on the driver variable. I have the following:

#MySQL DB

jdbcUrl=jdbc:mysql://localhost:3306/mydb

username=root

password=root

maximumPoolSize=10

I tried a few variables for driver but reflection fails.

Thanks

jeyashree

[Tim Molter]

That should be all you need. I suspect you don't have a mysql jdbc jar on your classpath.

[Jeyashree Narayanan]

Sorry I left off this thread to pursue the jobs.xml option. That works fine. However, we prefer to use the job annotations, which means in the application we add

SundialJobScheduler.startScheduler(threadCount, package with jobs);

The CascadingClassLoadHelper is not locating the jobclasses though and throws an exception. The package is in the JAR file I can see and jobs.xml option also recognizes it. However, when using the annotated approach, the classloader is unable to load the classes. Any suggestions will be appreciated.

thanks

jeyashree

[Tim Molter]

What does your annotated-jobs-package-name:
look like in your *.yml file?

~Tim

[Abhishek Kotwal]

Hi Tim,

I followed your article on followed link https://github.com/timmolter/dropwizard-sundial 

I must say article was very much thorough and clear. Hats off to you man !!!

I am struck with a hibernate session in my Job class, I need to update DB through my JOB which I have created by following your article, How will I get a hibernate session or Dao reference in my Job class so I can use that to update DB ? Looking forward to hear from you soon.

Thanks,

Abhishek K

[Tim Molter]

See: https://github.com/timmolter/dropwizard-sundial#inject-global-objects-or-config-parameters-into-a-job

Just put your reference in the servlet context.