[Feature Idea] Command batches

[Thomas Fritz]

Hi

I thought a bit about sending only a single command at a time to redis
getting the return value to just form the next query to redis.
Wouldn't this be better if redis could handle that for us. Which would
save bandwidth and would result in greater performance, i think.
So i thought it maybe would be a good idea to have the ability to send
a batch of commands ( I am not talking of transactions -MULTI, EXEC,
etc) with the ability to use the return value(s) of one command as the
input for the next command in the batch.
So a "client" could stick a sequence of commands together to a batch
of commands, sent the whole batch in one RoundTrip to the redis
server, the redis server process the commands sequentially (Actually
the same way as described and used for transactions).
The kicker about such a batch is that you could use the return
value(s) as parameters in the next command. A simple example for
instance to get batch the retrieval of the uid for a given username
and return all profile properties.

GET usernames:UserXYZ_66:uid // will return the unique id, lets say 1234
HGETALL uid:$1 profile // returns username "UserXYZ" password
"<HASHVALUE>" email "f...@bar.com"

I think performance, where more sequential commands are used than in
my example, could greatly benefit. Especially where the redis server
is split across multiple servers over a "slow" tcp connection.
This could be even further optimized by allowing to save such a batch
on the redis server with a name so one is just calling that name - in
fact like a simple stored procedure.

What do you think?

Kind regards

---
Thomas FRITZ
web http://fritzthomas.com
twitter http://twitter.com/thomasf

[Pedro Melo]

Hi,

On Mon, Oct 10, 2011 at 11:42 AM, Thomas Fritz <frit...@gmail.com> wrote:
> I thought a bit about sending only a single command at a time to redis
> getting the return value to just form the next query to redis.
> Wouldn't this be better if redis could handle that for us. Which would
> save bandwidth and would result in greater performance, i think.
> So i thought it maybe would be a good idea to have the ability to send
> a batch of commands ( I am not talking of transactions -MULTI, EXEC,
> etc) with the ability to use the return value(s) of one command as the
> input for the next command in the batch.
> So a "client" could stick a sequence of commands together to a batch
> of commands, sent the whole batch in one RoundTrip to the redis
> server, the redis server process the commands sequentially (Actually
> the same way as described and used for transactions).
> The kicker about such a batch is that you could use the return
> value(s) as parameters in the next command. A simple example for
> instance to get batch the retrieval of the uid for a given username
> and return all profile properties.

...
> What do you think?

My old pitch to Salvatore about this was a STORE prefix available to
most commands. You would

STORE dest_key REGULAR_REDIS_COMMAND

and the output of the REGULAR_REDIS_COMMAND would be stored in
dest_key. Useful for commands that return a large result, be they
GET's or set operations.

At the time (1 or 2 years ago at Codebits) he explained to me that the
problem is that the code generates the responses directly to the
output buffer, and they would need a major rewrite to generate the
response to a buffer that could be reused for stuff like this.

So basically I don't think anybody hates the idea, but there are
practical problems that prevent it right now.

Bye,
--
Pedro Melo
@pedromelo
http://www.simplicidade.org/
http://about.me/melo
xmpp:me...@simplicidade.org
mailto:me...@simplicidade.org

[Colin Vipurs]

It sounds like the lua scripting is exactly what you want here

On Mon, Oct 10, 2011 at 11:42 AM, Thomas Fritz <frit...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

--
Maybe she awoke to see the roommate's boyfriend swinging from the
chandelier wearing a boar's head.

Something which you, I, and everyone else would call "Tuesday", of course.

[Thomas Fritz]

Hi
Thanks for your replys.

@Colin: Maybe... I found the blog post after i wrote this mail. I
think the LUA Scripting is a very good idea to extend redis. But the
batch commands are easier to use and is in my opinion easier for
simple tasks where i just want to use the output as a "variable" or
placeholder in the next command. I think ( without any proof ) that
this solution could satisfy many, maybe the most complex queries by
allowing to save or store this batch of commands with a name on the
server side.

@Pedro: Maybe that changed in the meanwhile. But your suggestions
would be a good solution too.

I added an issue on github just for completness.

Kind regards

---
Thomas FRITZ
web http://fritzthomas.com
twitter http://twitter.com/thomasf

2011/10/10 Colin Vipurs <zodi...@gmail.com>:

[Colin Vipurs]

If you look back through the list and Salvatore's blog you'll see
discussions of why storing the commands on the server isn't a desired
proposition.

[Thomas Fritz]

See Issue on Github:
https://github.com/antirez/redis/issues/129#issuecomment-2347402

"antirez: Hi! We believe that tis is better fixed with Scripting
support that is coming in the next version of Redis (2.6 or 3.0, not
2.4).

Thanks for your proposal, but really, scripting can do for you that
and even more!"