Job queue for Node.js

[Ω Alisson]

Does anyone knows a good job queue that has atomicity and scheduled jobs?

[Sameer Joshi]

I have used Agenda which is backed with MongoDB. That has served my purpose.

There is also an AgendaUI package where you can view all the status of the jobs which have been schsduled

Reference:

1) https://www.npmjs.org/package/agenda

2) https://www.npmjs.org/package/agenda-ui

[Ω Alisson]

It is very nice, but it doesn't seem to delegate the job processing to other servers, or am I wrong?

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/c3782711-53e7-45bb-a0c9-a2a91c49dda8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Kelley]

On Monday, November 10, 2014 5:13:09 PM UTC-7, Alisson Cavalcante Agiani wrote:

Does anyone knows a good job queue that has atomicity and scheduled jobs?

Here's one: https://github.com/andrewrk/node-redis-dist-job-queue

You'd have to use `setInterval` for scheduling.

[Ω Alisson]

Andrew, is your implementation atomic?

--

Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/fa3fff07-f321-407e-a23f-62621a9b3e1d%40googlegroups.com.

[Andrew Kelley]

On Monday, November 10, 2014 9:46:27 PM UTC-7, Alisson Cavalcante Agiani wrote:

Andrew, is your implementation atomic?

Yes. Lua scripts are used to do atomic operations on redis. 

[Behrad Zari]

Have you tried https://github.com/LearnBoost/kue ?

[Aria Stewart]

> On Nov 10, 2014, at 6:14 PM, Ω Alisson <thelin...@gmail.com> wrote:
>
> Does anyone knows a good job queue that has atomicity and scheduled jobs?

Atomicity of what?

It may be that you want to break out these functions: if you make your tasks idempotent, you become resilient to all kinds of failures of your job queue -- and in fact can use one that's very fast and limited like Gearman.

You can use cron for scheduling -- or just a simple daemon that looks up the next time to run something and stuffs it in the queue then.

What guarantees do you need from your queue vs the jobs you schedule on it? Why?

Aria

[Buschini Edouard]

Hi,

I use RabbitMQ with a max length of 1 to ensure that only one of my nodejs node catches the job and executes it.

But I use setInterval to launch them.

The advantage is that you can remotely send a message through the queue to force a node executing your callback task and with any langage supporting the amqp (this can even be done with the rabbit management UI).

--
Edouard Buschini
@moonbocal

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/BD86CEFE-8927-41F9-971A-868673E20BFD%40nbtsc.org.

[Paul Jensen]

At my former employer we created a Job Queue (to replace Kue) for our needs: 

https://github.com/webcast-io/jobukyu

It's essentially a REST API on top of MongoDB, with support for web hooks.

MongoDB does has document-level atomicity, see here: http://docs.mongodb.org/manual/tutorial/model-data-for-atomic-operations/

It doesn't have scheduled jobs. If you wanted that feature, you may be better off looking at other systems. Otherwise, if you're upto it, you could fork it and add it in.

Regards,

Paul Jensen

[Behrad]

2014-11-12 17:42 GMT+03:30 Paul Jensen <paulb...@gmail.com>:

At my former employer we created a Job Queue (to replace Kue) for our needs: 

​

​

Kue has fixed some serious atomicity problems ​it had a year ago, and is planned to move into a fully atomic (server side LUA based redis) state management from 1.0

May I ask what was your actual problems with it?

​

https://github.com/webcast-io/jobukyu

It's essentially a REST API on top of MongoDB, with support for web hooks.

MongoDB does has document-level atomicity, see here: http://docs.mongodb.org/manual/tutorial/model-data-for-atomic-operations/

It doesn't have scheduled jobs. If you wanted that feature, you may be better off looking at other systems. Otherwise, if you're upto it, you could fork it and add it in.

Regards,

Paul Jensen

On Tuesday, 11 November 2014 00:13:09 UTC, Alisson Cavalcante Agiani wrote:

Does anyone knows a good job queue that has atomicity and scheduled jobs?

--

Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---

You received this message because you are subscribed to a topic in the Google Groups "nodejs" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nodejs/Rgwn_lQ3m-k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nodejs+un...@googlegroups.com.

To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/d6477342-56f0-4bb9-b3ce-c9e3a27ae8f6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

--Behrad