Input on Redis and Memcache protocol support

[HarryR]

Hey all,

  I put together a parser using Ragel* to handle Redis protocol commands (in both Unified and Inline format), the Inline support is generic and could conceivably act as a Memcache frontend too. This was spurred by pchapuis's comment - I think maintaining the DFA parser by hand isn't a feasible option when there are excellent tools out there to do it (like Ragel).

  - https://gist.github.com/1551348

 My goals for the new parser mean I have to re-evaluate if the DB interface I started in my fork/branch would work well.

 Currently the API works with fixed key sizes to accept one or more Key parameters and a variable sized Value parameter to one of the calls (GET/PUT/DEL/WALK), one downside is that unless the database layer returns the key and value in the same format in a single buffer then a new buffer of key+value size must be constructed for the reply - this makes zero-copy mostly impossible.

 The second approach I looked at is a multi-message based protocol which fits well with ZeroMQ.

 Example Redis input:

 *3\r\n$3\r\nSET\r\n$5\r\nmykey\r\n$9\r\nmyva\r\nlue\r\n 

 Example ZMQ output:

 msg("SET"[3], "mykey"[5], "myva\r\nlue"[9])

 This format/protocol is used by leveldb-server* - a very lightweight interface and client to LevelDB.

 * https://github.com/srinikom/leveldb-server

 The database backend would only have to support basic operations (GET/PUT/DEL/WALK/CAS), and the frontend would translate between Redis or Memcache commands and streams of basic operations - unless it was something supported natively by the backend (like list/field operations).

 Any input is appreciated, but so far this looks like the right approach to take.