Weight of SELECT command

[Dean]

Hi All,

I know that SELECT is essentially deprecated, though it's really
convenient to set up multiple test environments. We plan to use it
for testing in the interim until it is actually removed (this is not
a plea to keep it as a feature).

Redis.io doesn't list time complexity for SELECT. Based on some
topical observations, it looks like SELECT tends to be heavy. Is
there a rough rule of thumb for the relative time complexity of
SELECT? I'm trying to explain some performance issues that we're
seeing, and I suspect it's due to a naiive client that issues SELECT
in front of each command rather than once after establishing the
connection.

Cheers.

[Salvatore Sanfilippo]

On Thu, Jun 30, 2011 at 6:03 PM, Dean <deanw...@gmail.com> wrote:
> Redis.io doesn't list time complexity for SELECT.  Based on some
> topical observations, it looks like SELECT tends to be heavy.  Is

Hi! SELECT is among the fastest Redis commands! Maybe your delay is
due to FLUSHDB?

Btw there is currently no firm plan to remove SELECT, since there is
the possibility that adding a command to swap databases on the fly
could provide some impressive advantage for some use case. In other
words Redis cluster will NOT support SELECT and multiple DBs but it is
not clear if it will be supported in the single-instance case or not.

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele

[Dave Peticolas]

2011/6/30 Salvatore Sanfilippo <ant...@gmail.com>

On Thu, Jun 30, 2011 at 6:03 PM, Dean <deanw...@gmail.com> wrote:
> Redis.io doesn't list time complexity for SELECT.  Based on some
> topical observations, it looks like SELECT tends to be heavy.  Is

Hi! SELECT is among the fastest Redis commands! Maybe your delay is
due to FLUSHDB?

Btw there is currently no firm plan to remove SELECT, since there is
the possibility that adding a command to swap databases on the fly
could provide some impressive advantage for some use case. In other
words Redis cluster will NOT support SELECT and multiple DBs but it is
not clear if it will be supported in the single-instance case or not.

SELECT really is quite convenient for testing setups. I would be sorry
to see it go in the single-instance case.

dave
 

[Marc Gravell]

For info, we find the multi-DB scenario very useful and efficient, as a single redis process *easily* covers our usage, and it allows very simple admin (compared to the same number of exes on different ports, especially by the time you have a slave).

If/when it goes, we'll manage - but we're fans of it :)

Marc
(stackoverflow)

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>

[Salvatore Sanfilippo]

On Thu, Jun 30, 2011 at 6:43 PM, Marc Gravell <marc.g...@gmail.com> wrote:
> If/when it goes, we'll manage - but we're fans of it :)

Thanks for all the positive feedbacks about SELECT... the reason I
wrote that I'm reconsidering the dropping of SELECT is that in general
I learned how to don't care and love the bomb: in practice I'm no
longer trying to make sure that the cluster and single-node versions
of Redis are going to be similarly featured. This shift is clear if
you read the Redis Manifesto ;)

---------- from the manifesto -----------
6 - Two levels of API. The Redis API has two levels: 1) a subset of
the API fits naturally into a distributed version of Redis and 2) a
more complex API that supports multi-key operations. Both are useful
if used judiciously but there's no way to make the more complex
multi-keys API distributed in an opaque way without violating our
other principles. We don't want to provide the illusion of something
that will work magically when actually it can't in all cases. Instead
we'll provide commands to quickly migrate keys from one instance to
another to perform multi-key operations and expose the tradeoffs to
the user.
----------------------------------------------------

So basically if we are no longer going to unify it is a shame to drop
it. But if we take it, I would like to have two additional features.

1) Ability to programmatically swap two DBs:

SWAPDB 0 1

and in no time the DBs will be swapped (just by pointer).

2) A lazy version of FLUSHDB, like "FLUSHDB 10 LAZY", that is able to
don't stop but will remove objects in the background incrementally.

This may make the ability to have multiple DBs an important advantage
to model some kind of problems.

[Hampus Wessman]

On 2011-06-30 18:44, Hampus Wessman wrote:

> The SELECT command basically just updates a pointer, so it should take
> constant time and be extremely fast. On the other hand, if you issue a
> select without using pipelining it will still cost you a round trip to
> the server. That could make a big difference if you run select before
> every other command. Assuming that all commands are fast, this could
> make everything take around twice the time (i.e. two round trip times to
> the server per command instead of one). I think that makes a difference
> even over the local loopback interface, but I haven't measured.
>
> In fact, the time a really fast command takes to reply is not only
> influenced by network latency. The command may have to wait for other
> (potentially slower) commands to finish before it gets a chance to run,
> so even though the select command is really fast it can take a while
> until you get a reply from Redis. Remember that Redis is single threaded.
>
> Hope that helps. Pipelining could be a good solution, if you can't get
> rid of the selects.
>
> Cheers,
> Hampus

Client libraries could optimize this by filtering out redundant selects.
In that case it wouldn't cost anything to issue extra selects... Not
sure if any libraries do so right now, but it would be a nice
improvement otherwise. Anyone who knows how client libraries handle this?

Hampus

[Marc Gravell]

BookSleeve (the .NET client I wrote specifically to minimise all unnecessary overheads) will indeed optimise around SELECTs (like I said earlier, we use this lots) - so it is both avoided when unnecessary, and pipelined when it *is* necessary.

Obviously each client needs to handle this separately.

Marc

[Marc Gravell]

Interesting. SWAPDB might have some interesting effects on other connected clients, though - Presumably any clients in the affected DBs would need an implicit/automatic SELECT to ensure consistency, and there is the issue of keeping the clients informed, for example SELECT {olddb} would then *not* be a no-op, but some clients might be tracking the DB to avoid unnecessary SELECT (hence not issuing it). Of course, it is easier if there are no clients in the affected DBs

Just some thoughts,

Marc

[Hampus Wessman]

On 2011-06-30 18:03, Dean wrote:

The SELECT command basically just updates a pointer, so it should take

[Dean]

Thanks, all, for the replies.

Salvatore, thanks for the refresher from the Manifesto. I like your
approach!

It sounds like we have another performance culprit somewhere. I'll
start a new thread on that topic.

Cheers,

Dean