Clarification on job priority
ty...@monkeypox.org
clients fail to, with regards to job priority.
I've implemented a crude locking mechanism around memcached in my workers since
we have a business requirement to execute some jobs non-concurrently (to
prevent swamping some sites).
If my queue looks like this (at least in my head):
.__________________________.__________________________.__________________________.
| {"id":1,"payload":"foo"} | {"id":1,"payload":"foo"} | {"id":2,"payload":"foo"} |
`--------------------------`--------------------------`--------------------------`
A B C
If I have two workers, the flow would be as follows:
[worker1] * reserve A
[worker1] * check lock for id: 1
[worker1] * lock 1
[worker1] * delete A
[worker1] * <work>
[worker1] * unlock 1
[worker2] * reserve B
[worker2] * check lock for id: 1
[worker2] * release B with priority 0
Assuming both worker1 and worker2 finish their flows around the same time,
What gets scheduled next in this scenario? Does priority of the job override
order in the tube? How does priority actually play into the scheduling of jobs
for reservation?
Cheers,
-R. Tyler Ballance
--------------------------------------
GitHub: http://github.com/rtyler
Twitter: http://twitter.com/agentdero
Blog: http://unethicalblogger.com
Keith Rarick
> I'm wondering if somebody could clarify where the documentation of various
> clients fail to, with regards to job priority.
how priorities work in general terms.
When a client requests a job, the server picks one as if it were using
this procedure:
1. Sort all the jobs (in relevant tubes), first by priority, then by
order of submission.
2. Pick the first job.
So we could submit job A with priority 6, then job B with priority 5.
A subsequent reserve command would get job B first.
In your example, it depends on the priorities and submission order of
the other jobs.
kr