Redis.pm multi bulk commands
Dirk Vleugels
attached a patch which uses only the multi-bulk protocol for
communication with redis (and therefor supports all commands).
My bulk tests showed no measurable slowdown.
Cheers,
Dirk
ps: nice project!
Salvatore Sanfilippo
> Hi,
>
> attached a patch which uses only the multi-bulk protocol for
> communication with redis (and therefor supports all commands).
>
> My bulk tests showed no measurable slowdown.
Hello Dirk,
thanks, probably in the future the way to go is to support only the
new protocol and drop the old one at all.
This has a number of good effects, including for instance the removal
of many lines of code from Redis.c parsing code, the support for
binary keys, and client libs that are mostly ready for any kind of new
command without any effort.
Cheers,
Salvatore
--
Salvatore 'antirez' Sanfilippo
http://invece.org
"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay
Salvatore Sanfilippo
> thanks, probably in the future the way to go is to support only the
> new protocol and drop the old one at all.
Actually there is a minor difference:
PING: 32518.34 requests per second
PING (multi bulk): 30751.66 requests per second
(this test is against 1,000,000 requests), but the current
implementation ok multi bulk is not as fast as it could be, as my goal
was to make the code not too much complex so I did a trick to use the
old code to parse a bulk reply in order to parse every argument of the
multi bulk reply.
Probably once I fix this problem both protocols should perform the same.
dirk
> On Sat, Feb 6, 2010 at 1:14 PM, Salvatore Sanfilippo <anti...@gmail.com> wrote:
> > thanks, probably in the future the way to go is to support only the
> > new protocol and drop the old one at all.
>
> Actually there is a minor difference:
>
> PING: 32518.34 requests per second
> PING (multi bulk): 30751.66 requests per second
It's more work after all.
Using Redis.pm to insert roughly 1m keys + approx. 4m set and ordered
set (very useful btw.) values didn't show any difference in runtime:
approx. 20min runtime for a day of logprocessing for both versions.
Btw.: one useful feature could be to support serverside key prefixes
for a session, e.g.:
KEYPREFIX my.reports.2010-02-01.
the next
SET key value
or
INCR key
would create or increase a key named 'my.reports.2010-02-01.key'.
When doing bulk key creations this would save some bandwith maybe.
But then maybe im just doing it wrong, i use something like 'KEYS *.
2010-02-1.*.distinct_matches' a lot to fetch the values im interested
in....
Cheers,
Dirk
Gleicon Moraes
Thats why I abuse SETs when I use key prefixes. The extra space doesnt bother me as I gain flexibility for searches.
Also, sometimes I use lists to keep hotter items so I can slice them when needed.
Cheers
-- mobile cowbell
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
Salvatore Sanfilippo
> I found that using keys * for medium sized (i.e. < 20K objs) leads to bigger times and sometimes broken pipes (in python drivers) messages.
>
> Thats why I abuse SETs when I use key prefixes. The extra space doesnt bother me as I gain flexibility for searches.
>
> Also, sometimes I use lists to keep hotter items so I can slice them when needed.
Gleicon, from what I read, the Redis Power is with you :)
Sorted sets, Sets and Lists, are very useful even to build indexes
indeed. It's just a matter of finding the best data types and
organization to obtain the best results. KEYS is an O(N) operation
that should only be used for maintenance operations, like scripts to
modify the DB schema, debugging, and so forth.
dirk
> Thats why I abuse SETs when I use key prefixes. The extra space doesnt bother me as I gain flexibility for searches.
>
> Also, sometimes I use lists to keep hotter items so I can slice them when needed.
Could you explain what you mean exactly? Do you duplicate all keys
into a SET and query this set then? Using SORT ... GET?
I don't see why the query for the candidate keys should be faster than
KEYS (the key namespace is a set after all). I do understand that you
get the "values" of your search result without doing a further client-
server roundtrip (sending the MGET for the keys), which will speed up
the whole query, but the search performance should be comparable
right?
Cheers,
Dirk
Gleicon Moraes
I didn't delve too much into keys * internals, but I feel that SADDing keys to a set avoid running the keyname comparison which the argument for KEYS require.
Doing it just solved the broken pipes and long query times I had.
Also, I began to break down these sets according to my data model, and for a full key scan I sunion them.
As there is no norm (or normal form) to these kind of dataset, its really hard to tell the right way...
Best regards,
Gm
-- mobile cowbell
-----Original Message-----
From: dirk <dirk.v...@gmail.com>
Date: Sat, 6 Feb 2010 14:31:26
To: Redis DB<redi...@googlegroups.com>
Subject: Re: Redis.pm multi bulk commands