AdminClient fetchKeys performance

[ctasada]

Hi guys,

What's the expected performance of the AdminClient.fetchKeys? I'm using it but the behaviour is a bit heretic. I get something like 10K keys every 1.5 seconds, which is good, but then suddenly, the streaming stops for 30 to 60 seconds.

My suspicious is that the stops are caused by BDB reading, since some keys may not be cached. Is there any way to validate this theory? 

Also, checking the code I see that internally is doing storeEngine.get(key). Wouldn't it be more optimal to use getAll with blocks of keys?

Regards,

Carlos.

[Zhongjie Wu]

10k sounds reasonable. I got 1k/s for fetchEntries when I was migrating test data to MySQL.

getAll uses get anyway I think.

[Lei Gao]

When you do fetchKeys, do you give a list of partitions or you are fetching all partitions? If you are just fetching a subset of the partitions, the 30-60 seconds interval where you don't see any keys is the time we spend to scan and filter keys that are not part of the partitions being fetched.

Lei

On Tuesday, October 2, 2012 1:49:54 PM UTC-7, ctasada wrote: