Date: Tue, 30 Oct 2012 20:54:46 -0700 (PDT)
From: Royi Hagigi <rhag...@gmail.com>
To: redis-db@googlegroups.com
Message-Id: <8ffc4e84-18c2-4d68-ba7e-93b588940c1e@googlegroups.com>
In-Reply-To: <042d7d24-0bdc-4bda-92b0-6fba993349a7@googlegroups.com>
References: <042d7d24-0bdc-4bda-92b0-6fba993349a7@googlegroups.com>
Subject: Re: Why would running the same command (SINTERSTORE) successively
 on Redis produce faster and faster results?
MIME-Version: 1.0
Content-Type: multipart/mixed; 
	boundary="----=_Part_347_2319389.1351655686932"

------=_Part_347_2319389.1351655686932
Content-Type: multipart/alternative; 
	boundary="----=_Part_348_33444639.1351655686940"

------=_Part_348_33444639.1351655686940
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Just an update, reran the same test on a physical box. Running the same 
test on the same "hardware" as the virtual hardware (8gb dataset on 16gb 
memory) gave us significantly improved performance. For reference sake, it 
never showed a timing, even from the first time I ran it, and continued to 
give us results that showed much more consistent timings for similar 
commands.


On Wednesday, October 10, 2012 4:38:18 PM UTC-4, Royi Hagigi wrote:
>
> Apologies for the cross-post from StackOverflow, but I figured I'd get a 
> faster response here than there. I don't believe that redis caches the 
> results of commands, correct? If so, then why would I see the following on 
> my redis server for the same query. For reference, this is redis running in 
> a VM. I checked the smaps file as described in 
> http://redis.io/topics/latency and see no swapping on the OS level (all 
> 0kb in Swap for the process), but is it possible that running redis in a VM 
> pages memory to disk and back? Or... are these results expected due to some 
> kind of redis optimization for commonly run commands? 
>
>
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (4.46s)
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (3.77s)
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (0.92s)
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (0.64s)
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (0.67s)
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (0.73s)
> redis 127.0.0.1:6379[4]> scard ClientId:1637
> (integer) 796529
> redis 127.0.0.1:6379[4]> scard PublisherId:1
> (integer) 311092
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (1.88s)
> redis 127.0.0.1:6379[4]> sinterstore testdb ClientId:1637 PublisherId:1
> (integer) 240001
> (0.69s)
>
>
------=_Part_348_33444639.1351655686940
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Just an update, reran the same test on a physical box. Running the same tes=
t on the same "hardware" as the virtual hardware (8gb dataset on 16gb memor=
y) gave us significantly improved performance. For reference sake, it never=
 showed a timing, even from the first time I ran it, and continued to give =
us results that showed much more consistent timings for similar commands.<d=
iv><br></div><div><br>On Wednesday, October 10, 2012 4:38:18 PM UTC-4, Royi=
 Hagigi wrote:<blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-l=
eft: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><span style=3D"c=
olor:rgb(0,0,0);font-family:Arial,'Liberation Sans','DejaVu Sans',sans-seri=
f;font-size:14px;line-height:18px">Apologies for the cross-post from StackO=
verflow, but I figured I'd get a faster response here than there. I don't b=
elieve that redis caches the results of commands, correct? If so, then why =
would I see the following on my redis server for the same query. For refere=
nce, this is redis running in a VM. I checked the smaps file as described i=
n</span><span style=3D"color:rgb(0,0,0);font-family:Arial,'Liberation Sans'=
,'DejaVu Sans',sans-serif;font-size:14px;line-height:18px">&nbsp;</span><a =
href=3D"http://redis.io/topics/latency" rel=3D"nofollow" style=3D"color:rgb=
(74,107,130);font-family:Arial,'Liberation Sans','DejaVu Sans',sans-serif;f=
ont-size:14px;line-height:18px;background-color:transparent" target=3D"_bla=
nk">http://redis.io/topics/<wbr>latency</a><span style=3D"color:rgb(0,0,0);=
font-family:Arial,'Liberation Sans','DejaVu Sans',sans-serif;font-size:14px=
;line-height:18px">&nbsp;</span><span style=3D"color:rgb(0,0,0);font-family=
:Arial,'Liberation Sans','DejaVu Sans',sans-serif;font-size:14px;line-heigh=
t:18px">and see no swapping on the OS level (all 0kb in Swap for the proces=
s), but is it possible that running redis in a VM pages memory to disk and =
back? Or... are these results expected due to some kind of redis optimizati=
on for commonly run commands?&nbsp;</span><div><font color=3D"#000000" face=
=3D"Arial, Liberation Sans, DejaVu Sans, sans-serif"><span style=3D"font-si=
ze:14px;line-height:18px"><br></span></font><div><table style=3D"border-col=
lapse:collapse;color:rgb(0,0,0);font-family:Arial,'Liberation Sans','DejaVu=
 Sans',sans-serif;line-height:12px"><tbody style=3D"background-color:transp=
arent"><tr style=3D"background-color:transparent"><td style=3D"vertical-ali=
gn:top;background-color:transparent;width:60px"><br></td><td style=3D"backg=
round-color:transparent"><div style=3D"background-color:transparent"><div s=
tyle=3D"margin-right:5px;margin-bottom:5px;font-size:14px;background-color:=
transparent;width:660px;line-height:18px"><pre style=3D"margin-bottom:10px;=
padding:5px;background-color:rgb(238,238,238);font-family:Consolas,Menlo,Mo=
naco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera =
Sans Mono','Courier New',monospace,serif;overflow:auto;width:auto;max-heigh=
t:600px"><code style=3D"font-family:Consolas,Menlo,Monaco,'Lucida Console',=
'Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier Ne=
w',monospace,serif">redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId=
:1637 PublisherId:1
(integer) 240001
(4.46s)
redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId:1637 PublisherId:1
(integer) 240001
(3.77s)
redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId:1637 PublisherId:1
(integer) 240001
(0.92s)
redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId:1637 PublisherId:1
(integer) 240001
(0.64s)
redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId:1637 PublisherId:1
(integer) 240001
(0.67s)
redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId:1637 PublisherId:1
(integer) 240001
(0.73s)
redis 127.0.0.1:6379[4]&gt; scard ClientId:1637
(integer) 796529
redis 127.0.0.1:6379[4]&gt; scard PublisherId:1
(integer) 311092
redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId:1637 PublisherId:1
(integer) 240001
(1.88s)
redis 127.0.0.1:6379[4]&gt; sinterstore testdb ClientId:1637 PublisherId:1
(integer) 240001
(0.69s)</code></pre></div></div></td></tr></tbody></table></div></div></blo=
ckquote></div>
------=_Part_348_33444639.1351655686940--

------=_Part_347_2319389.1351655686932--