Lettuce ScanArgs with match and limit

1,603 views
Skip to first unread message

matth...@gmail.com

unread,
Jun 16, 2017, 1:00:20 PM6/16/17
to lettuce-redis-client-users
In order to achieve a non-blocking transaction that handles the same method as a KEYS * query, I have the following code:

private int scanArgsLimit = 100000;

public List<String> listKeys(String keyPattern) {
    KeyScanCursor<String> scanCursor = null;

    List<String> keys = new ArrayList<String>();

    RedisFuture<KeyScanCursor<String>> future = null;

    do {
        if (scanCursor == null) {
            future = asyncRedisConnection.scan(ScanArgs.Builder.matches(keyPattern).limit(scanArgsLimit));
        } else {
            future = asyncRedisConnection.scan(scanCursor, ScanArgs.Builder.matches(keyPattern).limit(scanArgsLimit));
        }

        try {
          scanCursor = future.get();
        } catch (Exception e) {
          return null;
        }
        keys.addAll(scanCursor.getKeys());

    } while (!scanCursor.isFinished());

    if (keys != null) {
      keys.sort(String::compareTo);
    }
    return keys;
}

However, I've found that on larger key sets this does not appear to return all available keys. Is scanCursor.isFinished() called when 0 matching keys are returned from a cursor? If so, how do I scan the entire key space? (these are relatively small spaces, < 200k keys)

Thank you!

Mark Paluch

unread,
Jun 17, 2017, 11:50:37 AM6/17/17
to lettuce-redis-client-users, matth...@gmail.com
What kind of Redis setup (Standalone, Master/Slave, Cluster) are you using? The code above does the right thing but why are you synchronizing the futures instead of using the synchronous API?

Mind the guarantees [0] that SCAN gives you if keys are changed while you're iterating the keyspace. Check out the [1] ScanIterator API which will be available with Lettuce 4.4. It will simplify your code a lot.

Reply all
Reply to author
Forward
0 new messages