clients that are multi-threaded, benefits?

[S Ahmed]

Seeing as redis is single threaded, what benefit does a redis client have in being multi-threaded? 

For example, jedis has a pool of connections into redis.

Is there really a benefit?  If yes, in what scenerio would this be?

If you make calls to redis, and redis is working a previous call, I guess the call will queue up at the o/s level and then send the call when redis completes.  

The only thing I can thing of, which is really is a guess, is when redis responds to a request, having a multi-threaded client can then immedietely send a new request.  But in a single threaded client, you loose the time the response is coming over the wire?  Again just a guess....

[Josiah Carlson]

On Sun, Dec 9, 2012 at 2:53 PM, S Ahmed <sahme...@gmail.com> wrote:
> Seeing as redis is single threaded, what benefit does a redis client have in
> being multi-threaded?
>
> For example, jedis has a pool of connections into redis.
>
> Is there really a benefit? If yes, in what scenerio would this be?

Assume Redis is running on another machine. If each thread wants to
execute some sequence of WATCH/MULTI/EXEC operations, by having
multiple connections, you can hide the latency between your clients
and your Redis server. Also, with multiple clients, you can ensure
that Redis itself can be kept busy.

> If you make calls to redis, and redis is working a previous call, I guess
> the call will queue up at the o/s level and then send the call when redis
> completes.

Yes.

> The only thing I can thing of, which is really is a guess, is when redis
> responds to a request, having a multi-threaded client can then immedietely
> send a new request. But in a single threaded client, you loose the time the
> response is coming over the wire? Again just a guess....

That's one, yes. But it's not just when Redis responds to the request
(or is busy), it's the entire round trip.

Say that your Redis instance is in AWS, along with your clients that
are connecting to Redis. You can figure that when the network between
your instances isn't heavily loaded, you can get a .5-1ms ping time.
At .5-1 ms, that is 1k-2k operations/second. Unless you are doing
fully pipelined async requests with callbacks (like the node.js client
does, which makes it impossible to use WATCH/MULTI/EXEC), that is your
limit per process if you only have one shared client for that process.
But if you use your threaded client pool, now you are limited to 1k-2k
round-trips *per thread*. Well hey, now you've just managed to hide
network latency with network bandwidth and concurrency.

Incidentally, this is *exactly* the same reason why Intel and other
manufacturers (like Sun with the UltraSPARC T1) run multiple threads
per core with "hyperthreading" (or similar technologies). Latency to
main memory is slow (compared to registers, L1, L2, and/or L3 cache),
so when you are waiting on that data (or waiting on the result of a
branch, as was the case with the T1 design), switch to a thread that
you have data for and continue executing it. When that data comes
back, you can continue executing. Again: hiding latency with bandwidth
and concurrency.

Regards,
- Josiah

[Jay A. Kreibich]

On Sun, Dec 09, 2012 at 05:53:40PM -0500, S Ahmed scratched on the wall:

> Seeing as redis is single threaded, what benefit does a redis client have
> in being multi-threaded?

It works with multi-threaded client environments, like web app servers
and other middle layers in a multi-tier architecture. Such systems
tend to be heavily threaded, so any support library must also be
thread safe.

Additionally, threading can make it easier to handle multiple connections
in language environments that tend to be thread-centric. Thanks to
publish/subscribe messaging and blocking list options, having
multiple connections open for moderate lengths of time is not that
unusual or unexpected.

-j

--
Jay A. Kreibich < J A Y @ K R E I B I.C H >

"Intelligence is like underwear: it is important that you have it,
but showing it to the wrong people has the tendency to make them
feel uncomfortable." -- Angela Johnson

[S Ahmed]

But say you are running a web application on jetty/tomcat, which in threaded in itself.

Why would your redis client need to be threaded also, each web request is on its own thread anyhow.

One thing I can think of is, if you get 1000 requests per second, then you have 1000 connections into redis.

But if your redis client has a pool, it will actually limit the threads in this case to e.g. 20 threads.  So it is actually a way to limit the # of threads.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

[Josiah Carlson]

On Mon, Dec 10, 2012 at 8:12 AM, S Ahmed <sahme...@gmail.com> wrote:
> But say you are running a web application on jetty/tomcat, which in threaded
> in itself.
>
> Why would your redis client need to be threaded also, each web request is on
> its own thread anyhow.

Maybe threads are not what you think they are. The purpose of a thread
is that it shares the same memory space as all other threads in the
same process, so it can have shared global variables for common
resources (like using a thread-safe hash map for a local cache), or
thread-local resources that are not shared.

Presumably Jedis connections are a shared global resource.

> One thing I can think of is, if you get 1000 requests per second, then you
> have 1000 connections into redis.

No. You only have as many connections as are necessary to sustain the
requests, based on the number of sequential commands, the latency of
those commands, and the number of serving threads.

If you only have 10 threads serving those 1k requests/second, you will
only need 10 connections, because each thread will (in the worst case)
need a connection. If you have 1k threads serving those 1k requests in
a second, then you *may* need 1k connections, but only if your typical
request requires 1 full second to complete (or if all 1k requests come
all at the same time, requiring every thread to have a connection
simultaneously). If your request (including all Redis commands) can be
served in 10 ms from Redis, you may only need 10-20 connections to
Redis.

> But if your redis client has a pool, it will actually limit the threads in
> this case to e.g. 20 threads. So it is actually a way to limit the # of
> threads.

No. Connections are released back into the pool when they are done being used.

As a case in point, I've got a web server running 2 processes and 5
threads each (I'm in Python, so heavy threading isn't quite so good).
At times, it will serve 200+ requests/second (I've got a lot of
headroom on that machine), but the most connections I've seen to Redis
from those (10 total) web serving threads over long periods of time is
4 (2 connection for 5 threads in each process). I'm not limiting the
number of connections, but the number of connections still hangs
around 4, because the actual number of concurrent requests to Redis is
about 4 over time. On the other hand, when I'm performing some
resource-intensive process (like a manual re-indexing/re-caching of a
bunch of data, pushing Redis to about 75% processor), latency per
command increases. Due to increased command latency, more requests
overlap, so more connections are necessary. Usually it gets to 6-7
connections, but I have seen it max out to 10 on occasion.

Regards,
- Josiah

[Jay A. Kreibich]

On Mon, Dec 10, 2012 at 11:12:31AM -0500, S Ahmed scratched on the wall:

> But say you are running a web application on jetty/tomcat, which in
> threaded in itself.

Exactly? If the app is threaded, the client library needs to be
thread-aware and thread-safe.

> Why would your redis client need to be threaded also, each web request is
> on its own thread anyhow.

Most clients manage global data within the client library... the
connection pool, for example. That means they need to be thread-safe,
use proper locking, etc. The client library is unlikely to use its
own threads (i.e. it doesn't actually create or destroy threads), but
the library needs to be aware that the application will likely be using
threads, and needs to behave correctly.

> One thing I can think of is, if you get 1000 requests per second, then you
> have 1000 connections into redis.

No idea what this means.

> But if your redis client has a pool, it will actually limit the threads in
> this case to e.g. 20 threads. So it is actually a way to limit the # of
> threads.

That's not how connection pools work.