Cron replacement using Consul

[Pete Emerson]

Hmmm, thought I sent this in yesterday, apologies if it shows up multiple times.

Has anyone written a cron replacement using Consul? Something along the lines of AirBnB's Chronos, but without it built on top of Mesos and Zookeeper?

I'd like to have our scheduling be better than cron, and using Consul as the backend seems like a natural fit.

Pete

[Brian Lalor]

I haven’t done it, but I’ve considered it.  Seems like a cool idea. :-)

--
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Brian Lalor

bla...@bravo5.org

[Evan Phoenix]

Over this holiday I've been sketching up a distrubed scheduler similar the Mesos that uses Consul and Vega. Can you tell me more about your use case for cron-style jobs? What features are you looking for? Could easily fit he bill for you I think. 

[Pete Emerson]

What's Vega?

I was thinking of creating two pieces, a scheduler and a worker.

The scheduler decides if there's something to run or not, the worker takes jobs off of the queue and runs them.

For a job's configuration, off the top of my head:

    name

    command

    schedule ( cronstyle  */5 * * * * ) and / or human readable ("every 6 hours"), up for suggestions and improvements

    first_run_datetime (sets when it will run initially)

    last_run_datetime (stop running this job after a certain date/time, blank to run to infinity)

    max_time_to_fail (mark the job as failed if we haven't heard back from the worker)

    max_timeout (give up if the job goes for this long without succeeding)

    max_retries (if the job fails, how often do we retry?)

    retry_backoff (0, linear, exponential?)

    last_start_time

    last_finish_time

    last_status_code

    last_output

A lot of that list I'd consider kitchen sink and not MVP! I'd want to keep logs of every job, too.

To go completely in a different direction, how about a cron wrapper that uses Consul to obtain atomic locks? Then you have the same cronjob on a bunch of different machines, and maybe use some randomness or other simple algorithm to distribute load a bit. It wouldn't be as flexible as what I've proposed above, but it would give you redundancy and load distribution.

0 0 * * * /usr/local/bin/cronwrapper /usr/local/bin/my_daily_report

Pete

[Evan Phoenix]

Vega is a distributed mailbox system (http://github.com/vektra/vega) we wrote for doing fully distrubed message passing. 

Your needs seem reasonable, though I'd move first_run out to be held in dynamic state rather than configuration.

Your thoughts are right along with what I'm thinking, though it adds a separate piece than acts as authority on cluster state (see Google's Omega scheduler for inspiration) that a scheduler uses to cooperate with other schedulers. You don't need that part, just giving you a more complete picture of what I'm thinking.

Using atomic locks via consul is certainly a quick way to get what you want. It leaves most your features out, but it could be done in about 30 mins. 

[Carl Johnson]

consul-do is designed for this. https://github.com/zeroXten/consul-do

Though it does not have any scheduling itself like chronos.