Connecting to Redis using a pool

[Daniel López]

Hey there,

I'm using Lettuce to connect to a Redis DB and I'm not sure if I'm doing it correctly. I'm using a pool to get the connections (Cacheable is a custom type that I serialise etc.)

RedisURI redisUri = getRedisURI();//

RedisClient redisClient = RedisClient.create(redisUri);

...

RedisConnectionPool<RedisCommands<String, Cacheable>> redisConnectionPool = redisClient.pool(new SerializedObjectCodec(), 5, 20);

...

and then

try (RedisCommands<String, Cacheable> redisConnection = redisConnectionPool.allocateConnection()) {

  redisConnection.set(key, value);

}

In some cases, I'd like to make sure the operation is synchronous but I can't find how to specify so when using a pool. My issue is that in some tests, if we try to get the value I just set, sometimes we get nothing back, and then after some interval, we then find the value. That's what made me think the operation is being sent asnychronously.

Is there some way to specify the "synchronicity" of the operation when using a pool? I've browsed the documentation and found several ways of performing sync/async operations, but not when using a pool of connections. Maybe it's not the right way of using the client when you want to do so?

Thanks in advance,

D.

[Mark Paluch]

Hi Daniel, 

great to see you use the Codec API. 

lettuce is internally fully asynchronous, and the API is thread-safe. You can use one connection for all your operations by multiple threads as long as you don't use transactions or blocking operations.

Multiple calls on one connection are pipelined, meaning the preceding command does not need to be finished at the time you invoke the next command. This is true for the sync and the async API. 

While you cannot invoke two commands at the same time using the sync API (from one thread), you can do so by using one connection (with the sync API) by two threads. This means that synchronicity is only visible to the calling thread but not to the full application. 

Redis works single-threaded but there are cases where you set a value, then get it (on two different threads), and the set is not fully finished.

If you need _full_ synchronization, you need to synchronize access to the connection on your own. That all above applies when using one connection with one/multiple threads.

The connection pool creates multiple connections and, in that case, every connection is used by just one thread (as far as I can see). In that case, multiple threads can perform truly asynchronous operations that send multiple commands at a time by using different connections. Redis then performs the synchronization. 

The synchronous aspect applies only on thread-level, never globally. Other clients, that do connection pooling usually run into the same issue, since you have multiple decoupled connections.

Please come back to me, if you have questions or issues. The whole sync/async thing can be quite confusing.

HTH, Mark

[Daniel López]

Hi Mark,

No problem, I understand how it works and why. The only issue we are having is that doing tests (integrarion or load) the tests run "too" fast and after storing a value, we sometimes find it's not there. We have resorted to adding a sleep to the tests, but that looks ugly and we have to make it way too big or it is still uncertain when load testing. That's why we were wondering if there was any way we could get the operation to be "synchronous" by waiting on the result, some flag...

S!

D.

[Tuco]

I know its probably very late, but redisConnection.set(key, value) returns a Future, so just make sure to do a "redisConnection.set(key, value).get();" instead, that will make sure that the set is synchronous...

[Daniel López]

Thanks, we'll make sure we use that.

Cheers!

D.

--
You received this message because you are subscribed to the Google Groups "lettuce-redis-client-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lettuce-redis-clien...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lettuce-redis-client-users/c7cbd28b-ed46-4ba5-9cbc-17f5c2ab6a91%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.