Unix domain sockets support

554 views
Skip to first unread message

tav

unread,
Aug 3, 2010, 11:32:42 PM8/3/10
to redi...@googlegroups.com
Hey guys,

I've implemented support for Unix domain sockets in Redis:

* http://github.com/tav/redis/compare/upstream...master

It yields anywhere between 20-90% performance increases on the Linux
and OS X 10.5 systems that I've tried.

The code includes updates for the redis core as well as
redis-benchmark and redis-cli.

The redis.conf file now has a new ``connection`` setting, which can be
either "tcp" or "unix". If tcp is chosen, then the usual port/bind
settings are used. Otherwise, it looks for a ``unixsocket`` setting
which defaults to /tmp/redis.sock.

The redis-benchmark and redis-cli take a new optional -u parameter for
the path to the domain socket.

It'd be great to have this merged upstream. Do let me know if there's
anything that I need to do.

--
Thanks, tav

plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

Tim Lossen

unread,
Aug 4, 2010, 1:51:15 AM8/4/10
to redi...@googlegroups.com
On 2010-08-04, at 5:32 AM, tav wrote:
> I've implemented support for Unix domain sockets in Redis:
>
> * http://github.com/tav/redis/compare/upstream...master
>
> It yields anywhere between 20-90% performance increases on the Linux
> and OS X 10.5 systems that I've tried.

tav, this is super awesome! thanks a lot for tackling this, hope your
code
gets merged soon.

tim

--
http://tim.lossen.de

Mathias Meyer

unread,
Aug 4, 2010, 4:04:12 AM8/4/10
to redi...@googlegroups.com
On Wed, Aug 4, 2010 at 5:32 AM, tav <t...@espians.com> wrote:
> Hey guys,
>
> I've implemented support for Unix domain sockets in Redis:
>
> * http://github.com/tav/redis/compare/upstream...master
>
> It yields anywhere between 20-90% performance increases on the Linux
> and OS X 10.5 systems that I've tried.
>
Great work, but seems like Pieter and you worked on the same things,
because he has Unix socket support on his fork too:
http://github.com/pietern/redis/compare/unixsocket

Would be neat to have this included either way though.

Cheers, Mathias
--
http://scalarium.com | http://paperplanes.de
http://twitter.com/roidrage

Александр Лозовюк

unread,
Aug 4, 2010, 4:43:33 AM8/4/10
to redi...@googlegroups.com
Great, but some question:

If Redis using only as local storage, unix socket are excellent, but
what if i need use network replication? In typically usage pattern,
one or more client work with Redis local, all data replicated over the
net to second node, and few client work at this node too. For better
usage, we need two interface at one time - unix soket for local client
and network for inter-node communication and remote client. This
behavior can be implemented?

2010/8/4 Mathias Meyer <pomon...@googlemail.com>:

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

--
C уважением, Александр Лозовюк
Alpha-Beta-Release Blog
http://abrdev.com

Pieter Noordhuis

unread,
Aug 4, 2010, 5:36:22 AM8/4/10
to redi...@googlegroups.com
Listening on both a socket file and a network port at the same time is
already implemented, so this won't be a problem.

Cheers,
Pieter

2010/8/4 Александр Лозовюк <aleks....@gmail.com>:

tav

unread,
Aug 4, 2010, 6:54:31 AM8/4/10
to redi...@googlegroups.com
> Great work, but seems like Pieter and you worked on the same things,
> because he has Unix socket support on his fork too:
> http://github.com/pietern/redis/compare/unixsocket

Yeah, that was a bummer =(

I've since spoken to him and have incorporated supporting both types
together as he'd done. Just set:

connection tcp+unix

The code is structured so that it'd be possible to add other
connection types in the future. In particular, I'd like support for
TLS connections to communicate securely across datacenters without
proxying.

The updated code can be seen at the same place:

* http://github.com/tav/redis/compare/upstream...master

--
Cheers, tav

Nate Wiger

unread,
Aug 4, 2010, 2:47:36 PM8/4/10
to Redis DB
This is a fantastic addition, much thanks both Pieter and tav!

On Aug 4, 3:54 am, tav <t...@espians.com> wrote:
> > Great work, but seems like Pieter and you worked on the same things,
> > because he has Unix socket support on his fork too:
> >http://github.com/pietern/redis/compare/unixsocket
>
> Yeah, that was a bummer =(
>
> I've since spoken to him and have incorporated supporting both types
> together as he'd done. Just set:
>
>     connection   tcp+unix
>
> The code is structured so that it'd be possible to add other
> connection types in the future. In particular, I'd like support for
> TLS connections to communicate securely across datacenters without
> proxying.
>
> The updated code can be seen at the same place:
>
> *http://github.com/tav/redis/compare/upstream...master
>
> --
> Cheers, tav
>
> plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369http://tav.espians.com|http://twitter.com/tav| skype:tavespian

Jak Sprats

unread,
Aug 4, 2010, 6:34:24 PM8/4/10
to Redis DB
Tav and Pieter, nice work. I have wanted this feature for a LONG time,
so thanks.

Is this going to be merged into 2.2, or is the plan that is remains on
a fork?

trung

unread,
Aug 4, 2010, 9:36:45 PM8/4/10
to Redis DB
I was going to ask for the same thing :)

When will it be merged to the master branch? Thanks.

Pieter Noordhuis

unread,
Aug 5, 2010, 3:17:41 AM8/5/10
to redi...@googlegroups.com
As the code is fairly trivial and non-intrusive, it is very likely
that it will be merged into master soon.

Cheers,
Pieter

On Thu, Aug 5, 2010 at 3:36 AM, trung <tr...@phamcom.com> wrote:
> I was going to ask for the same thing :)
>
> When will it be merged to the master branch? Thanks.
>

Salvatore Sanfilippo

unread,
Aug 5, 2010, 5:45:42 AM8/5/10
to redi...@googlegroups.com
On Thu, Aug 5, 2010 at 9:17 AM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:
> As the code is fairly trivial and non-intrusive, it is very likely
> that it will be merged into master soon

Hey, I want to merge this indeed.

Only stopper after I read the whole code tomorrow morning, is that in
the variable names you used "socket" to refer to unix domain sockets.
Actually everything is a socket ;) And for sure the common usage of
the word socket is more biased to actual TCP or UDP sockets, so I
would change this var naming. All the rest is ok for me.

Another small note is that probably it is possible to avoid closing
the sockets after the fork() in the childs. This was my fault, you
just followed my code indeed. But I think that as it's not possible to
return to the event loop in the saving code there is no point in
closing the socket.

Cheers,
Salvatore

> Cheers,
> Pieter
>
> On Thu, Aug 5, 2010 at 3:36 AM, trung <tr...@phamcom.com> wrote:
>> I was going to ask for the same thing :)
>>
>> When will it be merged to the master branch? Thanks.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Redis DB" group.
>> To post to this group, send email to redi...@googlegroups.com.
>> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay

Berry Tza

unread,
Oct 2, 2010, 5:53:12 AM10/2/10
to Redis DB
1+ on compliments for you work tav.
We are testing it under high load and it is FASTTTTT.
One thing that relates to my ignorance in unix sockets compared to
tcp.
Is it safe for multiple client process/threads open different
connections but all use /tmp/redis.sock (which is the one used by the
listening server). Is it up to the developer to sync read/write from
the unix socket or is it the same as tcp (totally separated channels).
What is the best practice for multiple clients over unix domain in
order to avoid race conditions?

On Aug 4, 12:54 pm, tav <t...@espians.com> wrote:
> > Great work, but seems like Pieter and you worked on the same things,
> > because he hasUnixsocket support on his fork too:
> >http://github.com/pietern/redis/compare/unixsocket
>
> Yeah, that was a bummer =(
>
> I've since spoken to him and have incorporated supporting both types
> together as he'd done. Just set:
>
>     connection   tcp+unix
>
> The code is structured so that it'd be possible to add other
> connection types in the future. In particular, I'd like support for
> TLS connections to communicate securely across datacenters without
> proxying.
>
> The updated code can be seen at the same place:
>
> *http://github.com/tav/redis/compare/upstream...master
>
> --
> Cheers, tav
>
> plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369http://tav.espians.com|http://twitter.com/tav| skype:tavespian
Reply all
Reply to author
Forward
0 new messages