Regressions in 2.6 when out of memory & writing during multi-exec? - Redis DB

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Regressions in 2.6 when out of memory & writing during multi-exec?
Received-SPF: neutral (google.com: 209.85.219.42 is neither permitted nor denied by best guess record for domain of mar...@new-bamboo.co.uk) client-ip=209.85.219.42;
MIME-Version: 1.0
From: Martyn Loughran <mar...@pusher.com>
Date: Thu, 15 Nov 2012 17:56:01 +0000
Message-ID: <CAP0aOtoTMd8qY0+XENa9mbfWNtr1R2BkcNuDYOZ=OC-y34u...@mail.gmail.com>
Subject: Regressions in 2.6 when out of memory & writing during multi-exec?
To: redis-db@googlegroups.com
Content-Type: multipart/alternative; boundary=f46d0444808366b86e04ce8c5e35

--f46d0444808366b86e04ce8c5e35
Content-Type: text/plain; charset=ISO-8859-1

While upgrading from 2.4 to 2.6 I noticed some changes which I haven't seen
documented, and look like regressions to me.

Scenario: out of memory & writing during multi-exec

# Change 1 - exec succeeds even if one of commands was an error

On redis 2.4 (tested 2.4.17)

    redis 127.0.0.1:6379> config get maxmemory
    1) "maxmemory"
    2) "0"
    redis 127.0.0.1:6379> multi
    OK
    redis 127.0.0.1:6379> set foo yyy
    QUEUED

In another client set memory limit `config set maxmemory 1`

    redis 127.0.0.1:6379> set bar uuu
    (error) ERR command not allowed when used memory > 'maxmemory'
    redis 127.0.0.1:6379> exec
    (error) ERR command not allowed when used memory > 'maxmemory'

    Transaction still open, need to call discard

On redis 2.6 (tested 2.6.4)

    redis 127.0.0.1:6379> config get maxmemory
    1) "maxmemory"
    2) "0"
    redis 127.0.0.1:6379> multi
    OK
    redis 127.0.0.1:6379> set foo yyy
    QUEUED

In another client set memory limit `config set maxmemory 1`

    redis 127.0.0.1:6379> set bar uuu
    (error) OOM command not allowed when used memory > 'maxmemory'.
    redis 127.0.0.1:6379> exec
    1) OK
    redis 127.0.0.1:6379> get foo
    "yyy"
    redis 127.0.0.1:6379> get bar
    (nil)

This seems like a pretty serious issue. foo and bar were set during a
transaction, but one one of the keys was written!

The only way to code around this is to check the response from each of the
commands during a multi, and to only exec if there were no errors. This is
problematic for all sorts of reasons. Previously it was easy to see whether
the transaction succeeded by reading the exec reply.

The OOM error is also inconsistent with this line in the transaction docs
"When a Redis connection is in the context of a MULTI request, all commands
will reply with the string QUEUED unless they are syntactically incorrect".

# Change 2 - memory limit enforced during queuing rather than exec

This is related to the first change, but I just wanted to be explicit

On redis 2.4 (tested 2.4.17) the memory limit is enforced when exec is
called

    redis 127.0.0.1:6379> multi
    OK
    redis 127.0.0.1:6379> set foo asdf
    QUEUED

In another client set memory limit `config set maxmemory 1`, then exec the
transaction

    redis 127.0.0.1:6379> exec
    (error) ERR command not allowed when used memory > 'maxmemory'

As expected the set fails.

For redis 2.6 (tested 2.6.4), the set does not fail

    redis 127.0.0.1:6379> multi
    OK
    redis 127.0.0.1:6379> set foo asdf
    QUEUED

In another client set memory limit `config set maxmemory 1`, then exec the
transaction

    redis 127.0.0.1:6379> exec
    1) OK

This feels like a bug to me.

Thanks very much for all your work on redis guys :)

Martyn

--f46d0444808366b86e04ce8c5e35
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>While upgrading from 2.4 to 2.6 I noticed some changes which I haven&#=
39;t seen documented, and look like regressions to me.</div><div><br></div>=
<div>Scenario: out of memory &amp; writing during multi-exec</div><div><br>

</div><div># Change 1 - exec succeeds even if one of commands was an error<=
/div><div><br></div><div>On redis 2.4 (tested 2.4.17)</div><div><br></div><=
div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt; =
config get maxmemory</div>

<div>=A0 =A0 1) &quot;maxmemory&quot;</div><div>=A0 =A0 2) &quot;0&quot;</d=
iv><div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&=
gt; multi</div><div>=A0 =A0 OK</div><div>=A0 =A0 redis <a href=3D"http://12=
7.0.0.1:6379">127.0.0.1:6379</a>&gt; set foo yyy</div>

<div>=A0 =A0 QUEUED</div><div><br></div><div>In another client set memory l=
imit `config set maxmemory 1`</div><div><br></div><div>=A0 =A0 redis <a hre=
f=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt; set bar uuu</div><div>=
=A0 =A0 (error) ERR command not allowed when used memory &gt; &#39;maxmemor=
y&#39;</div>

<div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt;=
 exec</div><div>=A0 =A0 (error) ERR command not allowed when used memory &g=
t; &#39;maxmemory&#39;</div><div>=A0 =A0=A0</div><div>=A0 =A0 Transaction s=
till open, need to call discard</div>

<div><br></div><div>On redis 2.6 (tested 2.6.4)</div><div><br></div><div>=
=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt; conf=
ig get maxmemory</div><div>=A0 =A0 1) &quot;maxmemory&quot;</div><div>=A0 =
=A0 2) &quot;0&quot;</div>

<div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt;=
 multi</div><div>=A0 =A0 OK</div><div>=A0 =A0 redis <a href=3D"http://127.0=
.0.1:6379">127.0.0.1:6379</a>&gt; set foo yyy</div><div>=A0 =A0 QUEUED</div=
><div><br></div>

<div>In another client set memory limit `config set maxmemory 1`</div><div>=
<br></div><div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:63=
79</a>&gt; set bar uuu</div><div>=A0 =A0 (error) OOM command not allowed wh=
en used memory &gt; &#39;maxmemory&#39;.</div>

<div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt;=
 exec</div><div>=A0 =A0 1) OK</div><div>=A0 =A0 redis <a href=3D"http://127=
.0.0.1:6379">127.0.0.1:6379</a>&gt; get foo</div><div>=A0 =A0 &quot;yyy&quo=
t;</div><div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379=
</a>&gt; get bar</div>

<div>=A0 =A0 (nil)</div><div><br></div><div>This seems like a pretty seriou=
s issue. foo and bar were set during a transaction, but one one of the keys=
 was written!</div><div><br></div><div>The only way to code around this is =
to check the response from each of the commands during a multi, and to only=
 exec if there were no errors. This is problematic for all sorts of reasons=
. Previously it was easy to see whether the transaction succeeded by readin=
g the exec reply.</div>

<div><br></div><div>The OOM error is also inconsistent with this line in th=
e transaction docs &quot;When a Redis connection is in the context of a MUL=
TI request, all commands will reply with the string QUEUED unless they are =
syntactically incorrect&quot;.</div>

<div><br></div><div># Change 2 - memory limit enforced during queuing rathe=
r than exec</div><div><br></div><div>This is related to the first change, b=
ut I just wanted to be explicit</div><div><br></div><div>On redis 2.4 (test=
ed 2.4.17) the memory limit is enforced when exec is called</div>

<div><br></div><div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0=
.1:6379</a>&gt; multi</div><div>=A0 =A0 OK</div><div>=A0 =A0 redis <a href=
=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt; set foo asdf</div><div>=
=A0 =A0 QUEUED</div>

<div><br></div><div>In another client set memory limit `config set maxmemor=
y 1`, then exec the transaction</div><div><br></div><div>=A0 =A0 redis <a h=
ref=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt; exec</div><div>=A0 =A0=
 (error) ERR command not allowed when used memory &gt; &#39;maxmemory&#39;<=
/div>

<div><br></div><div>As expected the set fails.</div><div><br></div><div>For=
 redis 2.6 (tested 2.6.4), the set does not fail</div><div><br></div><div>=
=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0.1:6379</a>&gt; mult=
i</div>

<div>=A0 =A0 OK</div><div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">1=
27.0.0.1:6379</a>&gt; set foo asdf</div><div>=A0 =A0 QUEUED</div><div><br><=
/div><div>In another client set memory limit `config set maxmemory 1`, then=
 exec the transaction</div>

<div><br></div><div>=A0 =A0 redis <a href=3D"http://127.0.0.1:6379">127.0.0=
.1:6379</a>&gt; exec</div><div>=A0 =A0 1) OK</div><div><br></div><div>This =
feels like a bug to me.</div><div><br></div><div>Thanks very much for all y=
our work on redis guys :)</div>

<div><br></div><div>Martyn</div><div><br></div>

--f46d0444808366b86e04ce8c5e35--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
