Multithreaded Jedis

[JammyZ]

Hi,
  what's the recommended way to use Jedis in a multithreaded environment for best performance? I have multiple threads that connect to Redis, if I use a single instance it works fine, but I presume I can do better. If I use an instance per thread I end up getting an error connecting to Redis (too many files open).

Any suggestions?

Thanks,
Iker.

[Jonathan Leibiusky]

Hi Iker,
To use Jedis in a multithreaded environment you should use JedisPool. This is a pool of Jedis instances. So you can reuse them and achieve great performance.

You just need to:

JedisPool pool = new JedisPool("localhost");
pool.init();

you can store the pool somewhere statically, it is threadsafe.

and then you use it just:

Jedis jedis = pool.getResource();
... do stuff here ...
pool.returnResource(jedis);


It is important to return the jedis instance to the pool once you've finished using it. And when you close your app it is good to call:

pool.destroy();

And btw, just added this to the Wiki under Advanced Usage.  :)

http://github.com/xetorthio/jedis/wiki

Jonathan

[JammyZ]

Hi,
  thanks for the quick response. I'm looking into integrating all this (multithreaded Jedis) within Spring. Will work on that tomorrow but I would love to hear if somebody else has done it before.

Iker.

[tedpe...@gmail.com]

I have a Spring @Aspect I wrote to manage Jedis instances from a JedisPool. It should address what you're looking for. I use it in my Spring MVC powered blog, http://ted.pennin.gs . I'll send it and post it on the wiki later tonight when I'm on my home computer. 

Ted

Sent from my iPhone

[JammyZ]

I wasn't planning on using an aspect, but it might be a good idea. Looking forward to look at your code.

[Ted Pennings]

Here's the code I used

@Aspect code: http://paste.ly/16um

Cache @Repository it decorates: http://paste.ly/16un

Spring Xml configuration for pool: http://paste.ly/16uo

... with the caveat that I've had it working for a few weeks, but I haven't had a chance to come back to it to refactor much.  I'd like to do that posting it anywhere else. (And before writing the inevitable blog entry.)

I hope it helps!

Ted

[Ted Pennings]

There's also an annotation I use, @FreshJedisRequired, to demarcate which methods on the @Repository should be wrapped with the execution pointcut. The code is *very* boilerplate, but here it is in case you were wondering: http://paste.ly/16uv

[JammyZ]

Hi Ted,
   thanks for sharing your code, I used a few things from it and would be interested in reading a blog entry about it.
  You might want to add a call to destroy in your xml config for the Pool class, it hung my tomcat without it.

Cheers,
Iker.

[Ted Pennings]

Thanks for the comment. I'll add that into my implementation.

I agree with you that an @Aspect is a hack to get this functionality. It would be much better to use a proxy that could manage JedisPool instances and execute all method invocations against a fresh Jedis instance. I've written a rough, reference implementation of this that appears to work for my blog (http://github.com/xetorthio/jedis/pull/36). What do you guys think?

As a side note, this reference implementation is slightly crippled because Jedis doesn't heavily code to interfaces. del(String... keys) is not supported, for instance. For the large release, it might be a worthwhile to use interfaces and factories more heavily. But I'm not sure if that bloated Java approach kind of defeats the purpose of using Redis in the first place...

I'm interested to hear everyone's thoughts.

[Jonathan Leibiusky]

Hi!
I think that JedisProxy is a nice thing to have, Probably I will push it to master branch during this week.

I didn't heavily code Jedis to interfaces because I wanted to do it very simple and easy to follow, and go from there. Actually one of the main reasons I started Jedis is because of JRedis code... wanted to add stuff there, found it was so obscure and hard to follow, created my own one easier and simpler.

As long as we keep it damn simple, we can add whatever makes our lives easier :)

Jonathan

[Sam]

Hi Jonathan,

There are some interesting sentence on wiki page:

enable replication

Redis is primarily built for master/slave distribution. This means that write requests have to be explicitly addressed to the master (a redis server), which replicates changes to slaves (which are also redis servers). Read requests then can be (but must not necessarily) addressed to the slaves, which alleviates the master.

I'm just wondering how we can ensure that jedisPool.getResource() will return a master if we want to write some key-val pairs to redis in the example code-snippet on that wiki page:

Jedis jedis = pool.getResource();
try {
  /// ... do stuff here ... for example
  jedis.set("foo", "bar");
  String foobar = jedis.get("foo");
  jedis.zadd("sose", 0, "car"); jedis.zadd("sose", 0, "bike"); 
  Set<String> sose = jedis.zrange("sose", 0, -1);
} catch (JedisConnectionException e) {
    // returnBrokenResource when the state of the object is unrecoverable
    pool.returnBrokenResource(jedis);
} finally {
  /// ... it's important to return the Jedis instance to the pool once you've finished using it
  pool.returnResource(jedis);
}

[Jonathan Leibiusky]

So this is not something that has to do with Jedis at all.

JedisPool is just a pool of reusable Jedis instances. All of those instances are connected to the same redis server. Which means that if you want to write on the master, and read from the slave, you should have 2 Jedis instances, one connected to your master and the other one to the slave.

If you need a reusable pool (for threading reasons, or performance) you would end up with 2 JedisPools. One connected to the master and the other one to the slave.

It is on the application logic to decide which Jedis instance (or pool) to use, depending on the operation.

--
You received this message because you are subscribed to the Google Groups "Jedis" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jedis_redis...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Sam]

Hi Jonathan

Thanks for your enlightenment!

I have one more question here regarding JedisPool :)

do I need to call jedis.slaveOf(masterHost, masterPort) each time one Jedis is returned from a slave master pool by calling jedisPool.getResource() on slave master? or just call it once after a new master is chosen?

Thanks in advance!

[Jonathan Leibiusky]

You actually don't need to do it through Jedis.

That command is for doing dynamic changes in the topology of your cluster.

If machine A is master and machine B is slave, and this is static (which means, they will always be like that), you just need to add in B's redis.conf a line like this:

slaveof <A's IP> <A's port>

You would use slaveOf from Jedis only if you want change this behavior from your app dynamically.

[Sam]

Currently we have one master and to slaves. We do need to dynamically promote a slave to master when the original master is not available. In this situation, should I call jedis.slaveOf(host, port) on each jedis returned from jedisPool.getResource() for a slave or just once?

currentMaster = promoteSlaveToMaster();

// get jedis from a slave pool for reading
Jedis jedis = jedisSlavePool.getResource();
//should I call slaveOf for all jedis instance returned from getResource() method over??
jedis.slaveOf(currentMaster.getHost(), currentMaster.getPort())

[Jonathan Leibiusky]

Yes. You should call slaveOf just once. This tells redis to start replicating from the maste you specify.
But I would consider using Sentinel.

[Sam]

Hi again,

I have one question regarding replication between master and slaves. For example, one extreme situation where one thread/application has written a new value to redis master, and exactly at this point, another thread/application tries to read the same key from a redis slave. How can it be ensured that the reading operation will get a fresh value from redis since it'll take ms for redis master to issue SYNC command, and the replication also takes time on slave.

The redis online documentation says something as following, however what if the writing operation just comes in the interval between WRITE->ISSUING SYNC (Master) and ACCEPT SYNC->DO SYNC (Slave), It might have caused inconsistency between master and slaves in a short short period in theory, doesn't it?

"Replication is non blocking on the slave side: while the slave is performing the first synchronization it can reply to queries using the old version of the data set, assuming you configured Redis to do so in redis.conf. Otherwise you can configure Redis slaves to send clients an error if the link with the master is down. However there is a moment where the old dataset must be deleted and the new one must be loaded by the slave where it will block incoming connections."

Thanks a lot in advance!

[Jonathan Leibiusky]

As far as I know you can't.
If you want consistency, you should read from master.

[David Weir]

When I implemented static JedisPool across multiple threads, my application falls over immediately with a cast error (cant cast "[B" to Long). I'm mainly using brpop(), lpush(), set() and get(). I'm using a compiled version of 3.0.0 with commons-pool2-2.0.

What's the issue here? Is the pool not thread safe?

This email is intended solely for the recipients named above and may contain information that is confidential, privileged or legally protected.  If you receive this communication in error, please immediately notify the sender by return e-mail and delete all copies of the original email and attachments.

[David Weir]

Actual error:

java.lang.ClassCastException: java.lang.Long cannot be cast to [B

at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:202) ~[jedis-3.0.0-ASYNC.jar:?]

at redis.clients.jedis.Connection.getBulkReply(Connection.java:192) ~[jedis-3.0.0-ASYNC.jar:?]

at redis.clients.jedis.JedisAsync.rpop(JedisAsync.java:1021) ~[jedis-3.0.0-ASYNC.jar:?]

I’m sharing the static JedisPool across my threads. I’ve got synchronized statements around Pool.getResource(), JedisAsync.rpop().

Should it be on Protocol.sendCommand()?

Thanks,

David.

-- 

David Weir

You received this message because you are subscribed to a topic in the Google Groups "Jedis" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jedis_redis/igyK85LwCss/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jedis_redis...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Weir]

or Protocol.process()…

-- 

David Weir

[Nils Kilden-Pedersen]

On Thu, Jan 22, 2015 at 12:16 PM, David Weir <d...@millanetworks.com> wrote:

When I implemented static JedisPool across multiple threads, my application falls over immediately with a cast error (cant cast "[B" to Long). I'm mainly using brpop(), lpush(), set() and get(). I'm using a compiled version of 3.0.0 with commons-pool2-2.0.

What's the issue here? Is the pool not thread safe?

Pool should be, but connection is not. Are you either sharing the connection among threads or interleaving pipeline calls with non-pipeline calls?

 

For more options, visit https://groups.google.com/d/optout.

[David Weir]

1 connection pool configured for unlimited connections.

Every call to redis is via:

jedis = pool.getResource();

jedis.<some function> ;

pool.returnResource(jedis);

the pool.getResource() is ‘synchronized’, the calls to jedis.pop, jedis.brpop & jedis.get() are all synchronized… 

So, what gives?

-- 

David Weir

You received this message because you are subscribed to a topic in the Google Groups "Jedis" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jedis_redis/igyK85LwCss/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jedis_redis...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.