[rabbitmq-discuss] Schedule Messages

[Darren Lee]

Hi everyone,

I’m looking for advice on the best way to schedule messages to be received after a certain time. I have a system where files get published out to a CDN and then the system needs to verify the files were propagated out correctly, this can sometimes take hours. I would like my subscriber to see a message, check for completion and if not put the message back on the queue to be dealt with in X minutes time.

Thanks for any help,
Darren

[Jon Brisbin]

I do something very similar with retries when operations triggered by a message fail. On failure, I write the message, exchange, routing key, and header information to a Riak server. A scheduled job comes along later and removes these messages from Riak and re-publishes them. The process may or may not be repeated N times, depending on whether later runs were successful or not.

I don't know if Rabbit has this functionality built-in, but having a "delayed delivery" would be awesome.

The only other way to do it is to actually publish and deliver a message, but have the consumer delay acting on it. I felt like a more reliable solution is to persist a message to a NoSQL store where I can delete, retrieve, or replay the message as many times as I want to. It adds another component to the mix, but I use Riak for things like this anyway, so why not? :)

Jon Brisbin

Portal Webmaster

NPC International, Inc.

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq...@lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[Doug Barth]

We needed similar functionality. The way we implemented it was to
create a service queue that takes a message and headers to indicate
the target queue and when to deliver it. Messages are either delivered
immediately because their delivery window has past, or they are queued
to a beanstalk server with a delay. The beanstalk server will send the
message back to our process after the delay time is up, which then
delivers the queue to its target queue.

[lusis]

On Sep 27, 9:54 am, Darren Lee <darr...@10cms.com> wrote:
<snip>

You could also stand up an instance of Apache Camel and have it manage
the process for you too. The DSL and spring xml methods are both very
easy to understand and configure.

[Christian Legnitto]

Can't the client just not ack the message if the files aren't there, thus leaving the message in the queue for the next retry?

Christian

[Darren Lee]

It's more a case of not actually attempting to do the work until after a certain amount of time has passed to conserve resources.

[Matthias Radestock]

Doug Barth wrote:
> We needed similar functionality.

There are two features which we are planning to implement in RabbitMQ
(please don't ask when ;) that in their combination can provide
delayed/scheduled message functionality: 1) per-queue message TTLs, and
2) dead-letter exchanges. The setup would look something like this:

The app creates a queue with the desired delay as the per-queue message
ttl. That queue is configured to route purged messages to an exchange
(using the dead-letter exchange feature). A second queue is bound to
that exchange to receive all the messages. The app consumes from the
second queue.

Regards,

Matthias.

[Matthias Radestock]

Matthias Radestock wrote:
> Doug Barth wrote:
>> We needed similar functionality.
>
> There are two features which we are planning to implement in RabbitMQ
> (please don't ask when ;) that in their combination can provide
> delayed/scheduled message functionality: 1) per-queue message TTLs, and
> 2) dead-letter exchanges. The setup would look something like this:

> [...]

Another solution, which you can implement right now, is to get the
publishers to stick an expiry header into the message and to publish the
message to a dedicated delay queue. A helper app consumes messages from
that queue one at a time and checks the expiry. If the expiry is in the
past it re-publishes the message to the appropriate exchange for
routing. Otherwise it waits until the deadline.

The expiries must be monotonically increasing per delay queue, though
you can setup several delay queues for different expiry intervals.

[David Wragg]

Matthias Radestock <matt...@rabbitmq.com> writes:
> Another solution, which you can implement right now, is to get the
> publishers to stick an expiry header into the message and to publish
> the message to a dedicated delay queue. A helper app consumes messages
> from that queue one at a time and checks the expiry. If the expiry is
> in the past it re-publishes the message to the appropriate exchange
> for routing. Otherwise it waits until the deadline.

This (any many other) AMQP intermiediaries would be assisted by:

- A way to consume messages without receiving the message content, just
the properties and headers.

- A way to republish consumed messages without having to re-send the
message content to the server.

An with these features, implementing a general and efficient message
scheduler based on timer wheels would be fairly simple.

David

--
David Wragg
Staff Engineer, RabbitMQ
SpringSource, a division of VMware

[David Wragg]

David Wragg <da...@rabbitmq.com> writes:
> This (any many other) AMQP intermiediaries would be assisted by:

By "intermediaries" (when I can spell it), I mean AMQP clients that
shuffle messages around, rather than publishing or consuming for their
own ends.

[Michael Bridgen]

>> Another solution, which you can implement right now, is to get the
>> publishers to stick an expiry header into the message and to publish
>> the message to a dedicated delay queue. A helper app consumes messages
>> from that queue one at a time and checks the expiry. If the expiry is
>> in the past it re-publishes the message to the appropriate exchange
>> for routing. Otherwise it waits until the deadline.
>
> This (any many other) AMQP intermiediaries would be assisted by:
>
> - A way to consume messages without receiving the message content, just
> the properties and headers.
>
> - A way to republish consumed messages without having to re-send the
> message content to the server.
>
> An with these features, implementing a general and efficient message
> scheduler based on timer wheels would be fairly simple.

Interesting thought; like a basic.head (just the headers please) and
basic.redirect (please dequeue this and publish it again). The latter
could work with (unacked) messages acquired through basic.consume or
basic.get, too.

In the case of plugins, using a direct connection is effectively the
same thing, by the way.

mkb

[Matthew Sackman]

On Fri, Oct 01, 2010 at 01:45:46PM +0100, Michael Bridgen wrote:
> In the case of plugins, using a direct connection is effectively the
> same thing, by the way.

No it's not. The msg may be fully on disk and not in RAM, at which point
you have to issue a disk read regardless of whether the client is
embedded or not.

Furthermore, we do not store the message headers separately from the msg
body. Thus the only thing you're saving by getting one, not the other,
is network transfer. And given the average message body size, ethernet
frame size and the fact that we turn nagel off, I'm not convinced in the
average case you'd actually save anything at all, but I could be missing
something.

Matthew

[Matthew Sackman]

On Fri, Oct 01, 2010 at 02:00:50PM +0100, Matthew Sackman wrote:
> On Fri, Oct 01, 2010 at 01:45:46PM +0100, Michael Bridgen wrote:
> > In the case of plugins, using a direct connection is effectively the
> > same thing, by the way.
>
> No it's not.

Well, it's not if the goal is to avoid unnecessary disk access. It is if
the goal is to avoid unnecessary TCP stack usage.

[David Wragg]

Matthew Sackman <mat...@rabbitmq.com> writes:
> On Fri, Oct 01, 2010 at 01:45:46PM +0100, Michael Bridgen wrote:
>> In the case of plugins, using a direct connection is effectively the
>> same thing, by the way.
>
> No it's not. The msg may be fully on disk and not in RAM, at which point
> you have to issue a disk read regardless of whether the client is
> embedded or not.
>
> Furthermore, we do not store the message headers separately from the msg
> body. Thus the only thing you're saving by getting one, not the other,
> is network transfer. And given the average message body size, ethernet
> frame size and the fact that we turn nagel off, I'm not convinced in the
> average case you'd actually save anything at all, but I could be missing
> something.

We'd get the network traffic saving immediately. And if worthwhile,
later on we could store message bodies separately, above a certain size
threshold. At the moment there is no motivation to consider doing that,
because AMQP always transmits the properties and body together.

And I doubt there is a meaningful notion of the average message body
size. Different AMQP applications have different characteristics.

--
David Wragg
Staff Engineer, RabbitMQ
SpringSource, a division of VMware

[Jon Brisbin]

Sorry to hijack the discussion here, but I'm ++1 on a head-style query of messages in a queue. I can think of several places right off where I could use this.

And I would be careful lumping all AMQP applications into a single bucket when it comes to message size. I would not write an application that interrogated the properties/headers of my batch file processor because it's sending zip files as message bodies but I would if I could get just the props/headers without the body. 

I also have applications that send a key value as the message body; maybe a few bytes in size. It just depends.

jb

Thanks!

J. Brisbin

http://jbrisbin.com/