Hello, starting from 2.6 you can use a Redis Lua Script.
There are no solutions for 2.4, nor planned since 2.6 solved the issue.
Salvatore
On Thu, Aug 30, 2012 at 5:24 PM, Parham Negahdar <pnegah...@gmail.com> wrote:
> Is there a way to publish to multiple channels at once to save on bandwidth?
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/ecntVZn-y9IJ.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
-- Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org
Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter
I'm not sure if you're talking about sending the same message to many
channels at once or many messages.
1. you can do it with a pipeline or transaction. but that doesn't save
bandwidth, just network roundtrips.
2. if you have one message to many, you can write a Lua function (in redis
2.6) that takes a message and replicates it to many channels.
3. you can have the clients listen to a single channel at once, or to have
the clients use PSUBSCRIBE to subscribe to a channel _pattern_, not a
specific channel, then any publish to any channel in the pattern will get
to them.
On Thu, Aug 30, 2012 at 6:24 PM, Parham Negahdar <pnegah...@gmail.com>wrote:
> Is there a way to publish to multiple channels at once to save on
> bandwidth?
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/ecntVZn-y9IJ.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
Related question: what happens when you publish to redis at say 1Gb/s and the subscribers are connected with a 100Mb/s connection? What happens to the data that cant be sent over fast enough?
On Thursday, August 30, 2012 11:35:49 AM UTC-4, dvirsky wrote:
> I'm not sure if you're talking about sending the same message to many > channels at once or many messages.
> 1. you can do it with a pipeline or transaction. but that doesn't save > bandwidth, just network roundtrips.
> 2. if you have one message to many, you can write a Lua function (in redis > 2.6) that takes a message and replicates it to many channels.
> 3. you can have the clients listen to a single channel at once, or to have > the clients use PSUBSCRIBE to subscribe to a channel _pattern_, not a > specific channel, then any publish to any channel in the pattern will get > to them.
> On Thu, Aug 30, 2012 at 6:24 PM, Parham Negahdar <pneg...@gmail.com<javascript:> > > wrote:
>> Is there a way to publish to multiple channels at once to save on >> bandwidth?
>> -- >> You received this message because you are subscribed to the Google Groups >> "Redis DB" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/redis-db/-/ecntVZn-y9IJ. >> To post to this group, send email to redi...@googlegroups.com<javascript:> >> . >> To unsubscribe from this group, send email to >> redis-db+u...@googlegroups.com <javascript:>. >> For more options, visit this group at >> http://groups.google.com/group/redis-db?hl=en.
On Thu, Aug 30, 2012 at 5:35 PM, Dvir Volk <dvir...@gmail.com> wrote:
> I'm not sure if you're talking about sending the same message to many
> channels at once or many messages.
My guess is that the OP wants to send the exact same message to
multiple channels. A 2.6 script seems the best way to do this. Or
maybe a change in the overall design to avoid duplication in some way?
Salvatore
-- Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org
Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter
since PUBLISH replicates the message to all listeners already and does it
fast, the right design would be to attack this via the channel pattern, or
use PSUBSCRIBE.
On Thu, Aug 30, 2012 at 6:45 PM, Salvatore Sanfilippo <anti...@gmail.com>wrote:
> On Thu, Aug 30, 2012 at 5:35 PM, Dvir Volk <dvir...@gmail.com> wrote:
> > I'm not sure if you're talking about sending the same message to many
> > channels at once or many messages.
> My guess is that the OP wants to send the exact same message to
> multiple channels. A 2.6 script seems the best way to do this. Or
> maybe a change in the overall design to avoid duplication in some way?
> Beauty is more important in computing than anywhere else in technology
> because software is so complicated. Beauty is the ultimate defence
> against complexity.
> — David Gelernter
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
The reason I want to send to multiple channels is because I want publish to a channel based on how much data the subscribers connection can digest. Say I publish data to a single channel at 1Gb/s to redis and some subscribers have a full 1Gb/s connection and some have a 100Mb/s how would redis handle this? I'm fine with the interim data getting lost, but I am not fine with the data stream backing up.
On Thursday, August 30, 2012 11:46:14 AM UTC-4, Salvatore Sanfilippo wrote:
> On Thu, Aug 30, 2012 at 5:35 PM, Dvir Volk <dvi...@gmail.com <javascript:>> > wrote:
> > I'm not sure if you're talking about sending the same message to many > > channels at once or many messages.
> My guess is that the OP wants to send the exact same message to > multiple channels. A 2.6 script seems the best way to do this. Or > maybe a change in the overall design to avoid duplication in some way?
> Beauty is more important in computing than anywhere else in technology > because software is so complicated. Beauty is the ultimate defence > against complexity. > — David Gelernter
On Thu, Aug 30, 2012 at 5:46 PM, Parham Negahdar <pnegah...@gmail.com> wrote:
> Related question: what happens when you publish to redis at say 1Gb/s and
> the subscribers are connected with a 100Mb/s connection? What happens to the
> data that cant be sent over fast enough?
In 2.4 land, what happens is an Armageddon (crash for out of memory eventually).
In 2.6 land you can configure limits, and the slow-reader connection
is closed when those limits are reached (see redis.conf for more
info).
Cheers,
Salvatore
-- Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org
Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter
So does each client have its own buffer/queue? Is there a way to neglect the backed up data instead of disconnecting the client? Like in 2.4 can we just set a memory limit to redis with a MAXMEMORY policy which deletes the publishing queue once the memory limit is reached or is the memory limit just for keys and not subscribers as well?
On Thursday, August 30, 2012 12:01:27 PM UTC-4, Salvatore Sanfilippo wrote:
> On Thu, Aug 30, 2012 at 5:46 PM, Parham Negahdar <pneg...@gmail.com<javascript:>> > wrote:
> > Related question: what happens when you publish to redis at say 1Gb/s > and > > the subscribers are connected with a 100Mb/s connection? What happens to > the > > data that cant be sent over fast enough?
> In 2.4 land, what happens is an Armageddon (crash for out of memory > eventually). > In 2.6 land you can configure limits, and the slow-reader connection > is closed when those limits are reached (see redis.conf for more > info).
> Beauty is more important in computing than anywhere else in technology > because software is so complicated. Beauty is the ultimate defence > against complexity. > — David Gelernter
IMHO if you're thinking of solving this type of problem with redis, you're
looking in the wrong place. To the best of my knowledge, redis was not
designed to be a streaming server. While it can do that, you are better off
using servers that were designed for this.
sent from my Sinclair ZX48
On Aug 30, 2012 8:03 PM, "PN" <pnegah...@gmail.com> wrote:
> For example if I could truncate the queue/buffer of the slow clients it'd
> be golden for multiple clients of variable speeds.
> On Thursday, August 30, 2012 11:24:03 AM UTC-4, PN wrote:
>> Is there a way to publish to multiple channels at once to save on
>> bandwidth?
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/AFzNfa8DUKAJ.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
since this is a redis group it's off topic, but I'd have a look at zeromq.
they have this stuff configurable and you can limit the buffer size per
connection and the desired behavior.
On Thu, Aug 30, 2012 at 9:15 PM, PN <pnegah...@gmail.com> wrote:
> Any recommendations?
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/2ET_zA1FEi8J.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
If you want to talk huge scale, I'm not sure what Twitter does but I'm
guessing it's custom Scala code, or at least it was at one time.
Personally I'm a huge fan of custom code in languages built on a
really tight VM, like the JVM, V8, the Erlang run-time or LuaJIT for
this sort of thing. You end up rewriting a bunch of stuff when your
users outgrow your prototype, so you might as well pick a runtime VM
you can grow with at the beginning.
In the context of Redis, 2.6 (and Luvit) have forced me to look at Lua
as a viable language for constructing large scalable systems. Well
played, all. ;-)
On Thu, Aug 30, 2012 at 11:19 AM, Dvir Volk <dvir...@gmail.com> wrote:
> since this is a redis group it's off topic, but I'd have a look at zeromq.
> they have this stuff configurable and you can limit the buffer size per
> connection and the desired behavior.
> On Thu, Aug 30, 2012 at 9:15 PM, PN <pnegah...@gmail.com> wrote:
>> Any recommendations?
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Redis DB" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/redis-db/-/2ET_zA1FEi8J.
>> To post to this group, send email to redis-db@googlegroups.com.
>> To unsubscribe from this group, send email to
>> redis-db+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/redis-db?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
