Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
optional STORE argument for read operations?
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  5 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Brian P O'Rourke  
View profile  
 More options Nov 1 2012, 1:42 pm
From: "Brian P O'Rourke" <brianorou...@gmail.com>
Date: Thu, 1 Nov 2012 10:41:58 -0700
Local: Thurs, Nov 1 2012 1:41 pm
Subject: optional STORE argument for read operations?
I don't want to hijack Peter's thread about his specific problem[1],
so let me re-start my suggestion here... this is a bit of a thought
experiment.

It would be nice if the STORE suffix were generalized to an argument
that works wherever there's a read operation going on. I think this
change would simplify Redis's language significantly, though the
implementation would have to map return values of all commands
returning integer/bulk/multi-bulk to Redis types in a way that seemed
natural.

Examples of operations which would have new uses

Atomic copy on a string:
    GET STORE

Grab a snapshot of the keys available in a hash as a new set:
    HKEYS STORE

Partial copies of keys:
    GETRANGE STORE
    HMGET STORE
    ZRANGE STORE
    LRANGE STORE

And all of these special commands could be dropped in favor of their
composite alternative:

    SDIFFSTORE becomes "SDIFF STORE"
    SUNIONSTORE
    SINTERSTORE
    ZUNIONSTORE
    ZINTERSTORE

Most of this can be done with Lua now, but it seems like a way to at
once simplify and add functionality - thoughts?

Cheers,
Brian P O'Rourke

[1] https://groups.google.com/forum/#!msg/redis-db/TXhODw23YW0/sruwnzIm5xQJ


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Salvatore Sanfilippo  
View profile  
 More options Nov 1 2012, 2:23 pm
From: Salvatore Sanfilippo <anti...@gmail.com>
Date: Thu, 1 Nov 2012 19:22:56 +0100
Local: Thurs, Nov 1 2012 2:22 pm
Subject: Re: optional STORE argument for read operations?
Hi, you can use Lua Scripting in Redis 2.6 to solve all the scenarios
you describe.

I think that while many protocol extensions like this seem natural,
the risk is to add STORE, then a generic NX option, and so forth, and
you get with half-a-language that does not benefit in the long run
IMHO. Instead just going for a Lua script is the simplest thing :-)

It is much better to provide fast Lua primitives in the future for
things we'll need. For instance we can have C-coded functions in Lua
that do things like redis.mass_add_to_set(lua_array) or alike, that
can avoid writing a loop inside Lua.

Cheers!
Salvatore

On Thu, Nov 1, 2012 at 6:41 PM, Brian P O'Rourke <brianorou...@gmail.com> wrote:

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
       — David Gelernter


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Brian P O'Rourke  
View profile  
 More options Nov 1 2012, 2:32 pm
From: "Brian P O'Rourke" <brianorou...@gmail.com>
Date: Thu, 1 Nov 2012 11:32:45 -0700
Local: Thurs, Nov 1 2012 2:32 pm
Subject: Re: optional STORE argument for read operations?
On Thu, Nov 1, 2012 at 11:22 AM, Salvatore Sanfilippo <anti...@gmail.com> wrote:

<snip>

> It is much better to provide fast Lua primitives in the future for
> things we'll need. For instance we can have C-coded functions in Lua
> that do things like redis.mass_add_to_set(lua_array) or alike, that
> can avoid writing a loop inside Lua.

This makes a lot of sense. Thanks, Salvatore.

Cheers,
Brian


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Josiah Carlson  
View profile  
 More options Nov 1 2012, 2:56 pm
From: Josiah Carlson <josiah.carl...@gmail.com>
Date: Thu, 1 Nov 2012 11:56:51 -0700
Local: Thurs, Nov 1 2012 2:56 pm
Subject: Re: optional STORE argument for read operations?

On Thu, Nov 1, 2012 at 11:22 AM, Salvatore Sanfilippo <anti...@gmail.com> wrote:

> Hi, you can use Lua Scripting in Redis 2.6 to solve all the scenarios
> you describe.

> I think that while many protocol extensions like this seem natural,
> the risk is to add STORE, then a generic NX option, and so forth, and
> you get with half-a-language that does not benefit in the long run
> IMHO. Instead just going for a Lua script is the simplest thing :-)

> It is much better to provide fast Lua primitives in the future for
> things we'll need. For instance we can have C-coded functions in Lua
> that do things like redis.mass_add_to_set(lua_array) or alike, that
> can avoid writing a loop inside Lua.

So a special C-implemented API for interacting with the underlying
system, as opposed to using the redis.call() interface?

That seems to me to be rife with "I want internal API X", which is the
same problem, only now there are 2 APIs (the external and internal).

What about just offering the ability to call 3rd party C extensions
stored in .so modules via Lua? Combine that with a relatively fixed
C-API for major components in Redis for major releases, and you
basically have the Postgres 3rd party situation (which has worked
great for them).

 - Josiah


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Salvatore Sanfilippo  
View profile  
 More options Nov 1 2012, 3:16 pm
From: Salvatore Sanfilippo <anti...@gmail.com>
Date: Thu, 1 Nov 2012 20:16:22 +0100
Local: Thurs, Nov 1 2012 3:16 pm
Subject: Re: optional STORE argument for read operations?

On Thu, Nov 1, 2012 at 7:56 PM, Josiah Carlson <josiah.carl...@gmail.com> wrote:
> So a special C-implemented API for interacting with the underlying
> system, as opposed to using the redis.call() interface?

There is no real need to break the interface, but it is possible to
extend it considering we are calling it from Lua in some way.
For example, instead of:

redis.sadd("foo",1,2,3,4,5,...) maybe we could support
redis.sadd("foo",my_array)

Or as proposed before, Lua could have access to a few well-specified
APIs that don't have any sense called as commands, that can operate
keys in very fast ways, as mass-add to list, set, zset, ...

I don't think this will be a big issue, but this is one of those ideas
that in order to enter Redis need 6 months or more and a clear
understanding that we: 1) need more speed compared to Lua loops. 2) We
can't optimize it further. 3) A minimal set of API calls can do a
great difference in some case.

For now I'll simply not add anything :-) We'll need a lot of
experience to understand what's a sensible thing to implement or not
IMHO.

> That seems to me to be rife with "I want internal API X", which is the
> same problem, only now there are 2 APIs (the external and internal).

This is bad only if the second API makes sense as a standard command
itself, this would be odd.

But: mass_add_to_set(key,array) is in the command API simply SADD.

However I think it's much better if mass_add_to_set() can just be an
optimization the user is not aware of at all if possible.

Like if the engine detects "SADD","key", ... many args, it will take a
code path that will add stuff directly to the set instead of really
going along the normal code path to dispatch the command using the
"fake client".

> What about just offering the ability to call 3rd party C extensions
> stored in .so modules via Lua? Combine that with a relatively fixed
> C-API for major components in Redis for major releases, and you
> basically have the Postgres 3rd party situation (which has worked
> great for them).

I'm really against that for a number of reasons already exposed. But
this is a short summary:

1) Every Redis instance should be the same with a set of minimal API.
2) I don't think that external C-coded extensions will allow Redis to
stay as stable as it is today.
3) I think Redis is Redis because of it's API. If everybody can extend
the API Redis will no longer be Redis, in some way.

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
       — David Gelernter


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »