What is the best way to get all keys of a redis DB

[E. Ulrich Kriegel]

Hi there,

we are implementing a cache in redis. Sometimes, all the keys matching a certain pattern are needed.

The API provides 2 ways to get them:

• rxScan
• rxKeys

The challenge with  rescan is, that one has to loop,  whereas the result of rekeys might become very large. Our solution is to use 

rxScan(..).toBlocking().value()

 in a loop which is embedded in an executeBlocking-clause, but that's not very elegant. 

Is there a better solution we do not see or is is there a need for a new API function in redis?

Looking at the SQLConnection-API, the problem of large result sets was solved by a streaming API.

IMHO it would be very helpful, to have an API like

Stream<String>rxKeysStream(pattern)

for redis.

Best regards

--Ulrich

[Paulo Lopes]

Yes we should probably refactor to stream redis too. Currently it works like the old JDBC used to work, read the whole bulk to memory convert to JSON and return to the handler.

[Arnaud Estève]

If a refactor of Redis-Client has to be done, can we also avoid the JSON deserialization ? (not avoid, but provide raw APIs that don't use JsonArray / JsonObject).

It's causing major issues when the number of, say, keys is huge. If you're working in Java and stored values as Strings (maybe containing pure JSON or something else), creating JsonArray is can lead to a huge performance penalty. I measured the difference between jedis (blocking) and redis-client in a case where I a huge number of huge hashsets : jedis was performing fine, and vertx-redis-client was itself blocking the event-loop.

[Paulo Lopes]

Good point.