Switching to 2.0 protocol

7 views
Skip to first unread message

Salvatore Sanfilippo

unread,
Apr 13, 2010, 9:02:31 AM4/13/10
to redis-cl...@googlegroups.com
Hello hackers!

this is just a "let's check the feelings" thread ;) Basically I think
it would be great for all the client libs to switch to the new "multi
bulk request" protocol before the 2.0 RC-1 release, so we'll get:

1) binary safe keys
2) a much more hard to desync protocol
3) clients that will work out of the box without knowing the "type" of
the commands

From the latest benchmarks I performed it seems like the performance
hit is very low compared to the old protocol, and I think I can
improve it a bit. I'll try to run some profiling and benchmark and
report back, I'm waiting for a box where I'll be able to perform
benchmarks without the fluctuations I see in the VM environments.

Redis-rb now has support for the new protocol at object creation time:

r = Redis.new(:binary_keys => true)

this will switch to the new protocol. What I think is that at some
point we can make this the default, and instead use

r = Redis.new(:use_old_protocol => true) when operating against Redis
1.0 instances...

I think this is a decent way of introducing the new protocol without
breaking old production environments, but I wonder if you have
different ideas.
At some point I would like to totally drop support for the old
protocol in Redis... maybe for 2.2.

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay

Andy McCurdy

unread,
Apr 13, 2010, 12:12:15 PM4/13/10
to redis-cl...@googlegroups.com
The Python library has already been converted to the new protocol.  We chose not to keep support for the older protocol as it's not part of the public API.  All the Redis commands will continue to work identically as before, so users shouldn't notice.



--
To unsubscribe, reply using "remove me" as the subject.

Michel Martens

unread,
Apr 13, 2010, 12:31:58 PM4/13/10
to redis-cl...@googlegroups.com

As we speak, Damian and I are working on a refactoring of redis-rb
which will use the new protocol by default.

clayton collie

unread,
Apr 13, 2010, 1:52:00 PM4/13/10
to redis-cl...@googlegroups.com
Is the new protocol outlined in the wiki ? I'm developing a new C# client and i'd like to support it exclusively since i have no legacy concerns.


Dobrica Pavlinušić

unread,
Apr 13, 2010, 1:58:07 PM4/13/10
to redis-cl...@googlegroups.com
Perl bindings have experimental branch with multi-bulk support:

http://github.com/dpavlin/perl-Redis/tree/multi-bulk

but I have problem with setting nul values:

*3
$3
SET
$3
foo
$0

will return '' (empty string), while

*3
$3
SET
$3
foo
$-1
-ERR invalid bulk write count

doesn't work. I'm doing something wrong, but I can't figure out how to
set nul value correctly in multi-bulk protocol.

While we are at it, Salvatore could you please update link to perl
bindings to point to github repo at
http://github.com/dpavlin/perl-Redis

Thanks.

--
...2share!2flame... http://blog.rot13.org

ccollie

unread,
Apr 13, 2010, 3:10:13 PM4/13/10
to Redis Clients Development
Nevermind. Found it in the protocol spec docs.

Daniele Alessandri

unread,
Apr 13, 2010, 3:57:14 PM4/13/10
to redis-cl...@googlegroups.com
On Tue, Apr 13, 2010 at 15:02, Salvatore Sanfilippo <ant...@gmail.com> wrote:

> this is just a "let's check the feelings" thread ;) Basically I think
> it would be great for all the client libs to switch to the new "multi
> bulk request" protocol before the 2.0 RC-1 release, so we'll get:

Switching to the new protocol is totally fine with me and I shouldn't
have any problem implementing this change for Predis in a way that it
automatically uses the new protocol when a server profile for a
version of Redis >= 2.x is loaded. Predis 0.6.0 will likely be out in
a month or so (delayed for good reasons as it will ship with a ton of
internal changes and new features), which means this is indeed the
right time for me to make such a change.

I don't know if I will be able to update redis-lua in time for the RC1
(support for the commands set of Redis 2.0 is totally missing, sigh)
but I will make sure to do so by the time 2.0 hits the stable release.

> I think this is a decent way of introducing the new protocol without
> breaking old production environments, but I wonder if you have
> different ideas.

Yeah it seems a good way to progressively phase out the old protocol.

--
Daniele Alessandri
http://www.clorophilla.net/
http://twitter.com/JoL1hAHN

Brian Hammond

unread,
Apr 13, 2010, 5:36:22 PM4/13/10
to Redis Clients Development
The node.js client uses the multi-bulk request format exclusively as
of v0.2.0.

I like the simplicity of it.

In my case there's no equivalent to method_missing. However, I made
it easy for someone to add a command:

http://github.com/fictorial/redis-node-client/blob/master/lib/redis-client.js#L567-666

Thanks,
Brian

> Salvatore 'antirez' Sanfilippohttp://invece.org

Joubin Houshyar

unread,
Apr 13, 2010, 9:54:53 PM4/13/10
to redis-cl...@googlegroups.com
Its not supported yet in Java/Go but will switch entirely over to the new protocol.  

/R

Daniele Alessandri

unread,
Apr 19, 2010, 11:41:35 AM4/19/10
to redis-cl...@googlegroups.com
On Tue, Apr 13, 2010 at 21:57, Daniele Alessandri <suppa...@gmail.com> wrote:

> Switching to the new protocol is totally fine with me and I shouldn't
> have any problem implementing this change for Predis in a way that it

Quoting myself for a quick recap on my part:

Predis: I've just dropped default support for Redis 1.0 (but it's
optionally available through a simple require) and now the new
protocol is used for Redis >= 1.2.0

redis-lua: it will probably undergo a major rewrite soon and I think
it won't be backwards compatible with the Redis 1.x, which means that
support for the old protocol will be completely scrapped.
--
Subscription settings: http://groups.google.com/group/redis-clients-dev/subscribe?hl=it

Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:06:42 AM4/20/10
to redis-cl...@googlegroups.com
2010/4/13 Dobrica Pavlinušić <dpa...@gmail.com>:
> Perl bindings have experimental branch with multi-bulk support:
>
> http://github.com/dpavlin/perl-Redis/tree/multi-bulk
>
> but I have problem with setting nul values:

Hello Dobrica!

in Redis it's not possible to set null values, they are only returned
to signal the absence of an element.

Sorry for the delay in the reply.

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay


Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:08:42 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 13, 2010 at 6:12 PM, Andy McCurdy <sed...@gmail.com> wrote:
> The Python library has already been converted to the new protocol.  We chose
> not to keep support for the older protocol as it's not part of the public
> API.  All the Redis commands will continue to work identically as before, so
> users shouldn't notice.

That's a very good news, thanks.

Yesterday I performed some test and there are evidences that the new
protocol is no slower than the old one, so there is really no point in
using the old one that is pretty broken at this point (because we have
a lot of commands requiring multiple binary safe arguments, and
because it does not recover well from errors).

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay


--
Subscription settings: http://groups.google.com/group/redis-clients-dev/subscribe?hl=it

Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:09:16 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 13, 2010 at 6:31 PM, Michel Martens <sov...@gmail.com> wrote:

> As we speak, Damian and I are working on a refactoring of redis-rb
> which will use the new protocol by default.

Thanks Michel, and now I know it's already 99% done :)

Cheers,
Salvatore

>
> --
> To unsubscribe, reply using "remove me" as the subject.
>



Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:09:43 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 13, 2010 at 7:52 PM, clayton collie <gbo...@gmail.com> wrote:
> Is the new protocol outlined in the wiki ? I'm developing a new C# client
> and i'd like to support it exclusively since i have no legacy concerns.

Hello,

Sure, it's in the protocol specification page "multi bulk request protocol".

Cheers,
Salvatore

Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:10:44 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 13, 2010 at 11:36 PM, Brian Hammond <br...@fictorial.com> wrote:
> The node.js client uses the multi-bulk request format exclusively as
> of v0.2.0.
>
> I like the simplicity of it.
>
> In my case there's no equivalent to method_missing.  However, I made
> it easy for someone to add a command:
>
> http://github.com/fictorial/redis-node-client/blob/master/lib/redis-client.js#L567-666

Thanks Brian,

and you also support Pub/Sub that's nice, even if we have still the
problem of the original pattern that would be very useful in order to
bind functions to subscribe operations...

Cheers,
Salvatore
> --
> To unsubscribe, reply using "remove me" as the subject.
>



--
Salvatore 'antirez' Sanfilippo

Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:12:09 AM4/20/10
to redis-cl...@googlegroups.com
On Wed, Apr 14, 2010 at 3:54 AM, Joubin Houshyar <sun...@gmail.com> wrote:
> Its not supported yet in Java/Go but will switch entirely over to the new
> protocol.

Thanks Joubin!

Cheers,
Salvatore


--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay


--
Subscription settings: http://groups.google.com/group/redis-clients-dev/subscribe?hl=it

Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:13:12 AM4/20/10
to redis-cl...@googlegroups.com
On Mon, Apr 19, 2010 at 5:41 PM, Daniele Alessandri
<suppa...@gmail.com> wrote:
> On Tue, Apr 13, 2010 at 21:57, Daniele Alessandri <suppa...@gmail.com> wrote:
>
>> Switching to the new protocol is totally fine with me and I shouldn't
>> have any problem implementing this change for Predis in a way that it
>
> Quoting myself for a quick recap on my part:
>
> Predis: I've just dropped default support for Redis 1.0 (but it's
> optionally available through a simple require) and now the new
> protocol is used for Redis >= 1.2.0

Great, thanks.
I think there are very little reasons for using Redis 1.0, as Redis
1.2 can be used as a better drop in replacement.

> redis-lua: it will probably undergo a major rewrite soon and I think
> it won't be backwards compatible with the Redis 1.x, which means that
> support for the old protocol will be completely scrapped.

Ok, but Redis 1.2 supports the new protocol without problems.

Cheers,
Salvatore

litao cheng

unread,
Apr 20, 2010, 4:21:56 AM4/20/10
to redis-cl...@googlegroups.com
let's do it!
In erl-redis client(http://github.com/litaocheng/erl-redis), we had implement the mbulk commands.
I will change all the commands to mbulk format.
I love the safe binary key string.

Best Regards
litao cheng

Salvatore Sanfilippo

unread,
Apr 20, 2010, 4:29:01 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 20, 2010 at 10:21 AM, litao cheng <litao...@gmail.com> wrote:
> let's do it!
> In erl-redis client(http://github.com/litaocheng/erl-redis), we had
> implement the mbulk commands.
> I will change all the commands to mbulk format.
> I love the safe binary key string.

Thanks Litao,

indeed binary keys are good, also no difference between commands is a
good feature, and the ability to recover better from an error (for
instance sending a command to a Redis instance that does not support
it as it was introduced later for instance).

Cheers,
Salvatore

Daniele Alessandri

unread,
Apr 20, 2010, 5:34:25 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 20, 2010 at 10:13, Salvatore Sanfilippo <ant...@gmail.com> wrote:

>> redis-lua: it will probably undergo a major rewrite soon and I think
>> it won't be backwards compatible with the Redis 1.x, which means that
>> support for the old protocol will be completely scrapped.
>
> Ok, but Redis 1.2 supports the new protocol without problems.

It's not really a matter of protocol, it's just that I won't bother
with testing it against Redis 1.2 at all. Anyway, there's nothing set
in stone yet :-)

Dobrica Pavlinusic

unread,
Apr 20, 2010, 7:45:26 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 20, 2010 at 10:06:42AM +0200, Salvatore Sanfilippo wrote:
> 2010/4/13 Dobrica Pavlinušić <dpa...@gmail.com>:
> > Perl bindings have experimental branch with multi-bulk support:
> >
> > http://github.com/dpavlin/perl-Redis/tree/multi-bulk
> >
> > but I have problem with setting nul values:
>
> in Redis it's not possible to set null values, they are only returned
> to signal the absence of an element.

I'm interested what other binding authors are doing when requested to
set null value: creating empty element or not creating element at all.

It would be useful to have consistent behavior between various language
bindings for this case so we have interoperability.

--
Dobrica Pavlinusic 2share!2flame dpa...@rot13.org
Unix addict. Internet consultant. http://www.rot13.org/~dpavlin

Salvatore Sanfilippo

unread,
Apr 20, 2010, 7:49:06 AM4/20/10
to redis-cl...@googlegroups.com
On Tue, Apr 20, 2010 at 1:45 PM, Dobrica Pavlinusic <dpa...@gmail.com> wrote:

> I'm interested what other binding authors are doing when requested to
> set null value: creating empty element or not creating element at all.
>
> It would be useful to have consistent behavior between various language
> bindings for this case so we have interoperability.

Hello Dobrica,

I think that if the type passed is not trivially turnable into a
string it's better to raise an error.

So for instance an integer will be converted into a string, but a null
value converted to string is almost surely not what the user wants.

Cheers,
Salvatore

> --
> Dobrica Pavlinusic               2share!2flame            dpa...@rot13.org
> Unix addict. Internet consultant.             http://www.rot13.org/~dpavlin
>
>
> --
> Subscription settings: http://groups.google.com/group/redis-clients-dev/subscribe?hl=it
>



Reply all
Reply to author
Forward
0 new messages