Client & Connection - close/shutdown

[mandar....@gmail.com]

Hi,

*I am using 4.3.0.Final version of the library*

Do I need to shutdown a client? What are the repercussions of keeping a client open forever?

Earlier, I'd client as static member of the class and I was creating a client in constructor. The application run absolutely fine on HP Stackato Cloud Foundry. I was not calling shutdown on the client in this case.

public class RedisConnector {

private static RedisClient client;

private RedisConnector() {

RedisURI redisUri = RedisURI.Builder.redis(ConfigPropertiesReader.getRedisEndpoint(), ConfigPropertiesReader.getRedisPort())

.withSsl(true).withPassword(ConfigPropertiesReader.getRedisCredentials()).build();

 

client = RedisClient.create(redisUri);

}

However, as soon as I made it a member of method, and run it 7-8 times consecutively, I see tomcat shuts down and restarts the application.

Here is my newer code that runs into issues (after 7-8 consecutive runs). Hope you can let me know?

public void insertInBatchAsync(Map<String, Object> dbRecords) throws CustomException {

RedisURI redisUri = RedisURI.Builder.redis(ConfigPropertiesReader.getRedisEndpoint(), ConfigPropertiesReader.getRedisPort())

.withSsl(true).withPassword(ConfigPropertiesReader.getRedisCredentials()).build();

 

RedisClient client = RedisClient.create(redisUri);

StatefulRedisConnection<String, Object> connection = client.connect(new KryoCodec());

try {

RedisAsyncCommands<String, Object> asyncCommands = connection.async();

// Disable auto-flushing

connection.setAutoFlushCommands(false);

// Perform a series of independent calls

List<RedisFuture<?>> futures = new ArrayList<RedisFuture<?>>(); 

for (Map.Entry<String, Object> entry : dbRecords.entrySet()) {

if (null != entry) {

futures.add(asyncCommands.set(entry.getKey(), entry.getValue()));

}

}

// Write all commands to the transport layer

connection.flushCommands();

// Wait until all futures complete

boolean success = LettuceFutures.awaitAll(10, TimeUnit.SECONDS,  

                  futures.toArray(new RedisFuture[futures.size()]));

if (!success) {

RedisFuture [] rf = (RedisFuture[]) futures.toArray(new RedisFuture[futures.size()]);

StringBuilder fails = new StringBuilder();

List<String> keys = new ArrayList<String>(dbRecords.keySet());

for (int i=0; i<rf.length; i++) {

AsyncCommand asyncCommand = (AsyncCommand)rf[i];

if (null != asyncCommand) {

StatusOutput so = (StatusOutput) asyncCommand.getOutput();

if (null != so) {

if (!"OK".equals(so.get())) {

String key = keys.get(i);

fails.append(key);

fails.append(", ");

}

}

}

}

// Check if there is any failures and throw an exception in case there are any.

if (fails.length() > 0) {

logger.error("++++++++ Failed to insert some or all objects into Redis cache ++++++++");

throw new CustomException(500, "Objects with following keys not inserted = " + fails.toString());

}

}

logger.info("======== Inserting Object objects in batch - Success = " + success + " ========");

} finally {

connection.close();

client.shutdown();

}

}

[Mark Paluch]

RedisClient should be shut down at the very end when your application is closed. It should be initialized once at first use/application startup and reused until application shutdown. Here are some reasons about the design:

1. RedisClient spins up expensive resources (multiple thread pools, additional threads for timers, …) that eat some of your system's resources. While it does not hurt spinning up one instance, creating multiple client instances will eat more system resources.
2. Initializing and shutting down RedisClient multiple times will create and destroy threads quite often which is expensive again (in terms of initialization time). 
3. RedisClient shutdown will not only free all computational resources but also closes client connections that might linger around.

You can keep the client instance in a static field and deallocate it on application shutdown (with a Servlet ContextListener) or better, if you run some sort of Dependency Injection container (like Spring or CDI), have it managed by the context.

Cheers, 

Mark

[mandar....@gmail.com]

Great.

My application is intended to run once every 24 hours (like an ETL job) by 1 user only (not multiple users). Can I keep alive forever that single instance of client and use the same instance every 24 hours (to create & close connections)?

Is there any downside to that - like memory leak? Will the client instance hold huge amount of resources if I were to not shut it down and keep it alive forever?

Regards,

Mandar

[Mark Paluch]

I think the answer is tied to the JVM lifecycle. If you spin up every 24 hours a JVM that uses lettuce and does some data loading, then it basically does not matter.

If you have a JVM that is up and running for a long time, then you should go with the single client instance approach and just open and close connections.

Cheers, 

Mark

[mandar....@gmail.com]

Thank you Mark!

Regards,

Mandar