Jedis ScanParams Count - missing keys

[Samuel Facchinello]

Hello. 

Using the code below to search for all keys, when I set the parameter COUNT = 100 it returns all keys correctly, if I put any value above 100, some key does not return.

//START CODE

static JedisSentinelPool pool =null;

static {

JedisPoolConfig conf = new JedisPoolConfig();

conf.setMaxTotal(2048);

conf.setMaxIdle(1024);

conf.setMinIdle(1);

conf.setMaxWaitMillis(1000);

Set<String> sentinels = getSentinels();

JedisSentinelPool pool = new JedisSentinelPool("mymaster", sentinels, conf, "pass");

}

public static Set<String> keys(String pattern) {

Jedis redis = null;

try {

redis = pool.getResource();

Set<String> keys = new HashSet<String>();

ScanParams params = new ScanParams();

params.count(100); // <<<====================

params.match(pattern);

ScanResult<String> scanResult = redis.scan("0", params);

while (!"0".equals(scanResult.getCursor())) {

keys.addAll(scanResult.getResult());

scanResult = redis.scan(scanResult.getCursor(), params);

}

return keys;

} finally {

if (redis != null) {

redis.close();

}

}

}

//END CODE  

-------------------

INFO:

• Java 7;
• Jedis 3.2.0 (also tested with 2.10.2);
• Redis 4.0.6 (there is no way to update) (with 20000 keys)

---------------

Example:

public static void main(String[] args) throws Exception { 

String key1 = "RF:P:450:*"; 

Set<String> entryInCache = keys(key1);

System.out.println("amount returned="+entryInCache.size()); 

}

//with count(100) print in log:

amount returned=8

//with count(500) print in log:  

amount returned=7

--------------

could someone explain to me why this happens?

[Allan Wax]

Setting the count just sets a max return set size.  There is no guarantee of any particular number of keys returned.  You just have to loop until it tells you the scan is done.

Setting the max size means you're trying to be nice to the server.  The problem with KEYS (evil, evil, evil) is that it will lock up the server on your request until all the keys have been transmitted to you.  SCAN provides a cap, and between calls, allows other processes to get work done.

[Samuel Facchinello]

I know this theory, but what I want to understand is why it does not return all records in certain scans.

Is there any explanation for why this happens?

[Allan Wax]

I don't know the underlying code so please ignore or completely disregard the following.

Have you tried using KEYS with your pattern on the command line to see what gets returned.

Second, and this is a pure guess, since you are using a Sentinel pool and not a Cluster, is it possible that the call you made only returned the keys from one of the pool members.  Again, just a thought.

[Sagar Sitap]

This is wired. I assume there is no expiry or evict mechanism in place. Is is consistently giving 7 for count 500 and 8 for count 100 ?

--
You received this message because you are subscribed to the Google Groups "Jedis" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jedis_redis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jedis_redis/ebc7588c-f8ee-4c08-ab29-0a3a5467bcabo%40googlegroups.com.

--

Thanks,

Sagar Sitap

[Samuel Facchinello]

I finally found the problem.

I changed my WHILE to DO WHILE as below, now all records are returned correctly, regardless of the COUNT parameter.

//START CODE  

public static Set<String> keys(String pattern) {

Jedis redis = null;

try {

redis = pool.getResource();

Set<String> keys = new HashSet<String>();

ScanParams params = new ScanParams();

params.count(100); // <<<====================

params.match(pattern);

String cursor = ScanParams.SCAN_POINTER_START;

do {

ScanResult<String> scanResult = redis.scan(cursor, params);

keys.addAll(scanResult.getResult());

cursor = scanResult.getCursor();

} while (!cursor.equals(ScanParams.SCAN_POINTER_START));

return keys;

} finally {

if (redis != null) {

redis.close();

}

}

}

//END CODE  

sorry for the delay in the response, but only today I went through the project again.

Thank you all.

[Samuel Facchinello]

RESOLVED!