Getting a subset or all keys?

[Christian van der Leeden]

Hi,

I haven't found documentation about it, but is there a way to get all keys
or a subset of keys (e.g. by wildcard query)? A longer runtime is acceptable,
since it is more for statistc or for cleanup purposes. Or should one
keep a global value as index?

We're currently running redis next to membase for the indexing part.

Christian

[Perry Krug]

Christian, at the moment you will need to use our TAP interface: http://wiki.membase.org/display/membase/TAP+Protocol

It will stream out all the keys, and continue to stream out incoming mutations.  

We are currently working on a robust interface called Node Code that will allow you to implement queries and indices on top of this TAP interface.

Perry

--

Perry Krug System Engineering direct: 831-824-4123 email: pe...@membase.com

 

[Chad Kouse]

I modified the tap.py that comes with membase to spit out all keys, then I dump that output to a file and process it with my favorite scripting language.  I'm able to parse/cleanup about 600,000 items in a minute.. works for me, but can't wait for the Node Code stuff...

--chad

[Chad Kouse]

Here is the modified tap python program (99.99% of it written by Dustin Stallings)

I just changed the opts variable to look like:

opts = {memcacheConstants.TAP_FLAG_DUMP: 0x0}

I am also attaching my memcacheConstants.py file as I believe I had to modify the variable TAP_FLAG_TYPES to look like:

TAP_FLAG_TYPES = {TAP_FLAG_BACKFILL: ">Q", TAP_FLAG_DUMP: ">Q"}

and I'm not sure if I added the TAP_FLAG_DUMP variable (I don't think so)

Finally I'm attaching a php script to be used to parse the output (since I'm more proficient in php).

The flow is like this:

1. python tap-dumpall.py > dump.txt

2. php parse_tap_output.php dump.txt someKey__or_key_prefix__to_find

Let me know if you have any questions... 

--chad