Redis suitablity for queueing application
jberkus
I'm preparing to compare Redis against RabbitMQ and ActiveMQ for
queueing in a medium-sized multi-server application. It would be very
helpful to me if folks would answer some of the questions below about
suitability and load. This application will be public and have some
exposure, and I can see to it that Redis is mentioned if we use it (I
also want to do an article on Redis, but that's another post).
Application Spec:
* 4 queues. 3 of the four need to be accessible from the same
interface; the other does not.
* Highest volume queue will receive up to 5000 messages/minute
* Messages are only a UUID.
* The queues will have up to 20 writers and 12 readers.
* If a component in the system goes down, one or more queues can get
"backed up". This would require storing up to 1 million items in the
queue before they start to get cleared out.
* We would prefer to have the queue recover from unexpected shutdown
* Losing items from the queue is not permitted
Questions:
a) Does the above sound appropriate for Redis?
b) How does Redis handle this level of contention (up to 20 clients
wanting to hit the same list at once) when using push/pop off the same
list?
c) What are my options for persistence and/or queue forwarding? How
slow is Redis if I sync every write to disk?
d) Are there any mechanisms for "locking" a queue item while it's
being processed, as you can in ActiveMQ?
e) Any other thoughts?
--Josh
Josiah Carlson
> Redis folks,
>
> I'm preparing to compare Redis against RabbitMQ and ActiveMQ for
> queueing in a medium-sized multi-server application. It would be very
> helpful to me if folks would answer some of the questions below about
> suitability and load. This application will be public and have some
> exposure, and I can see to it that Redis is mentioned if we use it (I
> also want to do an article on Redis, but that's another post).
>
> Application Spec:
> * 4 queues. 3 of the four need to be accessible from the same
> interface; the other does not.
> * Highest volume queue will receive up to 5000 messages/minute
> * Messages are only a UUID.
> * The queues will have up to 20 writers and 12 readers.
> * If a component in the system goes down, one or more queues can get
> "backed up". This would require storing up to 1 million items in the
> queue before they start to get cleared out.
> * We would prefer to have the queue recover from unexpected shutdown
> * Losing items from the queue is not permitted
>
> Questions:
> a) Does the above sound appropriate for Redis?
It's all reasonable to Redis. You'd want to use AOF + compaction for
the last item.
> b) How does Redis handle this level of contention (up to 20 clients
> wanting to hit the same list at once) when using push/pop off the same
> list?
Not an issue. Seriously. I've personally benchmarked about a
thousand readers and writers on a single queue. Redis had no
problems. I thought about offering it a drink to toast to it's
success, then I realized that it's software and doesn't drink. So I
drank alone. (not really, but it's late, and this is what passes for
humor from me at this hour)
> c) What are my options for persistence and/or queue forwarding? How
> slow is Redis if I sync every write to disk?
Persistence can either be snapshotted, or you can use an
append-only-file (AOF) with compaction. If you sync every write to
disk, then you can only move as many events as your disk is fast. I
assure you, your disk is not very fast, maybe 100-300 IOs/second peak
on a standard spinning disk (if you are lucky, 50 with metadata
updates), which *might* be okay for inserting 5000 events/minute, but
won't when you also have to remove them. Using RabbitMQ or ActiveMQ
would run into the same limitations. Redis would work fine with the
once/second syncing, which should be able to handle at least a few
thousand (if not a couple hundred thousand) UUID writes/second.
Incidentally, unless RabbitMQ has solved their saving problems that
Reddit ran into this past spring, I wouldn't use it.
We've had good experience with ActiveMQ, though we've had to build
custom interfaces in Python (because the existing ones for Python were
broken in one way or another). Also, though ActiveMQ suggests KahaDB
for it's scalability and recovery, last time I checked, no recovery
tools are actually available (we've had ActiveMQ crash on us, which
lost us items in our Queues, and which caused ActiveMQ to fail to
load; uck for requiring a sysadmin to get working again).
> d) Are there any mechanisms for "locking" a queue item while it's
> being processed, as you can in ActiveMQ?
Not natively, but you can easily write locking mechanisms using Redis.
There are numerous recipes in the mailing list and scattered online.
> e) Any other thoughts?
If you are using Redis for other things, re-using it for queueing is
just fine. If you are okay with 1 second persistence limitations,
and/or are using a SSD for persistence (which may have a relatively
short couple-year lifetime with your Redis load), then all of your
bases are covered. Note that a message queue with Redis is going to
be very bare-bones, unless you spend time building it (scheduled
messages, delayed messages, etc.), which you can get pre-built with
the other queue systems.
One thing to note is that neither RabbitMQ nor ActiveMQ offer any sort
of magic pixie dust to solve the disk IO issue, and actually, the
recovery tools with Redis are much more mature than the recovery tools
that I've actually had the pleasure of using for either RabbitMQ or
ActiveMQ (both of which are effectively nonexistent). That is, the
recovery tool for Redis will tell you the offset at which to truncate
your AOF file to recover all but your last set of writes. The best
case scenario I've lived through with RabbitMQ or ActiveMQ is to move
their databases somewhere else, start the process back up, and hope
that the data I've just pulled will at some point be recoverable.
Regards,
- Josiah
teleo
'pending' Redis list to 'inProgress' Redis list via RPOPLPUSH command.
Once the item is executed, it should be removed from 'inProgress'
queue.
You will need a daemon to periodically detect incomplete queue items
which timed out, and move them back to 'pending' list or to 'unknown'.
The timeout information can be stored in a ZSET.
See http://code.google.com/p/redis/wiki/RpoplpushCommand for further
details.
On Nov 9, 10:37 am, Josiah Carlson <josiah.carl...@gmail.com> wrote:
Didier
I don't think Redis can compete on the same level than full-fledged
solutions like RabbitMQ, ActiveMQ, HornetQ and even less with
proprietary products like TIBCO, Websphere-MQ, or Oracle AQ.
IMO, it has never been the intented purpose.
With Redis you get extreme simplicity, good performance and a useful
publish-and-subscribe API. You will lack many high-level features
provided by most MOMs though.
Regards,
Didier.
Matt Ranney
To address 'locking', you can atomically move queue items from
'pending' Redis list to 'inProgress' Redis list via RPOPLPUSH command.
Once the item is executed, it should be removed from 'inProgress'
queue.
Salvatore Sanfilippo
Inviato da iPhone
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
Josh Berkus
Thanks for the incredibly complete answer!
> I
> assure you, your disk is not very fast, maybe 100-300 IOs/second peak
> on a standard spinning disk (if you are lucky, 50 with metadata
> updates), which *might* be okay for inserting 5000 events/minute, but
> won't when you also have to remove them.
I primarily work in SQL databases, so I'm very familiar with disk speeds.
(Salvatore, at some point you might consider adding a small write buffer
for the AOF, like 1MB. You can tremendously improve the throughput of
disk writing at the cost of only 1ms or so of response time.)
> Using RabbitMQ or ActiveMQ
> would run into the same limitations.
Ok, that's good to know.
> Incidentally, unless RabbitMQ has solved their saving problems that
> Reddit ran into this past spring, I wouldn't use it.
They claim to have. Haven't tested, of course.
> We've had good experience with ActiveMQ, though we've had to build
> custom interfaces in Python (because the existing ones for Python were
> broken in one way or another).
Yeah, and we're doing Python, so that's a significant concern. Thrift--
> Not natively, but you can easily write locking mechanisms using Redis.
> There are numerous recipes in the mailing list and scattered online.
Yeah. I'd already thought of having a 2nd queue, which someone on this
list actually recommended.
> If you are using Redis for other things, re-using it for queueing is
> just fine.
Actually, we *have* to add a tool to the stack regardless. Redis is less
invasive than a Java-based queue. And potentially if we're using it
anyway, we could swap memcached out.
> Note that a message queue with Redis is going to
> be very bare-bones, unless you spend time building it (scheduled
> messages, delayed messages, etc.), which you can get pre-built with
> the other queue systems.
Yeah, we don't need any of those things, which is why I'm looking at
Redis. The only thing which would potentially be useful which would
require a lot of build-out in Redis might be queue forwarding.
> One thing to note is that neither RabbitMQ nor ActiveMQ offer any sort
> of magic pixie dust to solve the disk IO issue, and actually, the
> recovery tools with Redis are much more mature than the recovery tools
> that I've actually had the pleasure of using for either RabbitMQ or
> ActiveMQ (both of which are effectively nonexistent).
Very good to know. And here I thought I just couldn't find them in the
docs.
Thanks everyone else for your feedback!
--
-- Josh Berkus
PostgreSQL Experts Inc.
http://www.pgexperts.com