Redis Triggers / Watch?

[tom]

Following use-case: I have plenty of (zset) keys that form a 2-D array
of the world. Each key holds trucks in a approx. 1km x 1km area. I
would like to implement a "watch" feature, where someone could
subscribe to watch an area, say 20X20 km in size, equalling 200 such
keys.

To display the trucks there i could either loop through all of the 200
keys in an intervall and display their contents e.g. on a map or -
ideally - I could tell redis to hand me just those few keys, that have
changed since last check.

Is this something that can be done with "Watch"? I just saw it as an
ingredient in CAS transacations for now.

[Josiah Carlson]

Use publish/subscribe.

- Josiah

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

[tom]

As far as I understand it pub/sub is an efficient means to distribute
info. But how do I get the info about changes in many keys WITHOUT
polling many keys frequently?
Or can pub/sub do this?

[Pieter Noordhuis]

What Josiah probably means is that you can easily curry every update
to your sorted set with a PUBLISH. This way, if some subscriber is
interested in being notified of changes to a 10x10 square, he
subscribes to 100 channels (being possibly named the same way as the
keys, as pub/sub channel names and the keyspace are unrelated). Now,
when you change, let's say square:123:456, with whatever command
(ZADD, ZINCRBY, ...), you instead do the following:

MULTI
ZINCRBY square:123:456 1 foo
PUBLISH square:123:456 "foo is updated"
EXEC

You can think about the payload to publish to allow incremental
updates of the client-side data, but you get the point. Every
subscriber to the channel "square:123:456" will receive this message
and can act accordingly.

Cheers,
Pieter

[Александр Лозовюк]

Pub/Sub in our test is very CPU-load :( if the message flowrate is
more then 200 - 300 msg/sec (with 3 channel, and one psub client)

2010/8/24 Pieter Noordhuis <pcnoo...@gmail.com>:

--
C уважением, Александр Лозовюк
Alpha-Beta-Release Blog
http://abrdev.com

[Pedro Melo]

Hi,

2010/8/24 Александр Лозовюк <aleks....@gmail.com>:

> Pub/Sub in our test is very CPU-load :(  if the message flowrate is
> more then 200 - 300 msg/sec (with 3 channel, and one psub client)

Didn't test this, but what about having a Redis slave just for SUBSCRIBE's?

Now that I think about it, is PubSub master-slave-aware?

Bye,
--
Pedro Melo
http://www.simplicidade.org/
xmpp:me...@simplicidade.org
mailto:me...@simplicidade.org

[Pieter Noordhuis]

Yes, PUBLISH is replicated.

@Aleks: I don't share the experience of pub/sub being this slow. On my
laptop I can easily get a throughput of 10k messages/sec with a couple
of subscribers.

Cheers,
Pieter

[Александр Лозовюк]

Hmm, we have two small EC2 instance, one publisher and second slave
instance. Normally, we have 35 msg/sec to publish, and peak (at 00:10,
01:10 eg.) have 100 or little more messages, and redis at master node
use up to 100% CPU. Tuned for performance has no results. At now i see
to beanstalk or stomp MQ servers... but try to another testing.. may
be dependence of client (we using nodejs client and C publisher
program)

2010/8/24 Pieter Noordhuis <pcnoo...@gmail.com>:

--

[Marc Byrd]

Pieter wrote:  "...pub/sub channel names and the keyspace are unrelated."

In fact almost everything about pubsub is unrelated to the rest of redis, right?

Has there ever been consideration of spinning off pubsub into its own project?  The two code bases might each be easier to maintain, smaller, faster, etc. 

Else, what's the compelling use case for having them live in the same code base?  

Cheers,

m

[Pieter Noordhuis]

The channel names are indeed unrelated, but having pub/sub in Redis
allows you to do the exact thing I described in my previous mail,
which is notifying others of your actions in one call.

For instance, I have an app where a producer publishes updates to
consumers, but where the consumer can update state and trigger certain
actions on the producer side of things. When the two would live in
different projects, I would lose the strict ordering of updates to the
key space and the corresponding triggers (not to mention the need to
run another daemon).

I hope this example clarifies a bit.

Cheers,
Pieter