Redis will not delete a key from a list when it should

[Tampa312]

Hi,

The below will not delete the key in a a list using msgpack in python 2.7

import redis
import msgpack
redis_master_conn = redis.StrictRedis()

data = {'data': {u'account_hash': u'a8682994fe92180813627b37bea83d86', u'account_user': u'00', u'alert_email': u'fad...@gmail.com', u'account_password': u'jCjCEIwmIC0idins98vlj/o=', u'service_name': u'Aesop', u'service_id': 1, u'full_day_only': 0}}

redis_master_conn.set('remove:%s' % data['data']['account_hash'],1)
packed = msgpack.packb(data,use_bin_type=False)
redis_master_conn.lpush('queue1',packed)
data = redis_master_conn.brpoplpush('queue1','queue2',timeout=0)
data = msgpack.unpackb(data)
if redis_master_conn.exists('remove:%s' % data['data']['account_hash']):
    packed = msgpack.packb(data,use_bin_type=False)
    result = redis_master_conn.lrem('queue2',1,packed)
    if result==1:
        redis_master_conn.delete('remove:%s' % data['data']['account_hash'])
    else:
        print 'sad'
    print 'result:',result
else:
    print 'no delete'
   
   
sad
result: 0


Now...if I remove a key then it works:

e.g I removed this key ->  u'account_user': u'00'
   
data = {'data': {u'account_hash': u'a8682994fe92180813627b37bea83d86', u'alert_email': u'fad...@gmail.com', u'account_password': u'jCjCEIwmIC0idins98vlj/o=', u'service_name': u'Aesop', u'service_id': 1, u'full_day_only': 0}}

result: 1

Then one would think there is smething wrong with that key....so I placed back in and removed this key u'alert_email': u'fad...@gmail.com'

data = {'data': {u'account_hash': u'a8682994fe92180813627b37bea83d86', u'account_user': u'00', u'account_password': u'jCjCEIwmIC0idins98vlj/o=', u'service_name': u'Aesop', u'service_id': 1, u'full_day_only': 0}}

result: 1

So why the issue?  Is there a min lenth that redis can user to compare keys to delete?  makes no sense to me why this is happing

[Jan-Erik Rediger]

Hi,

use 'redis-cli MONITOR' in a second terminal to see the actual commands
that are received on Redis' side. Make sure the data is correct.

There is no minimum/maximum length Redis can handle on LREM (besides the
limit on the internal sds data type, which should be 2 GB, or was it 4 GB?)

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.

[Itamar Haber]

After commenting out the 2nd `packed = msgpack.packb(data,use_bin_type=False)` (line 15), this works as expected. That means that the 1st `packed` (pushed) isn't the same as the 2nd `packed` (searched for) so LREM fails to find it in the list.

@Jan-Erik isn't it 512MB? :)

[Salvatore Sanfilippo]

On Mon, Feb 23, 2015 at 11:27 AM, Itamar Haber <ita...@redislabs.com> wrote:

@Jan-Erik isn't it 512MB? :)

Max string 512MB

Max sds string internals 2GB in older Redis code

Max sds stirng internals 4GB in more recent Redis code 

--

Salvatore 'antirez' Sanfilippo
open source developer - Pivotal http://pivotal.io

"If a system is to have conceptual integrity, someone must control the concepts."
       — Fred Brooks, "The Mythical Man-Month", 1975.

[Jan-Erik Rediger]

Ah, well indeed. the sds itself should be able to handle more, but the
protocol limits it to 512 MB.

[Tampa312]

I see what happened using monitor,

unpacking and packing chaged the order of compression.  Works now with some modification.

Thanks

On Monday, February 23, 2015 at 5:27:53 PM UTC+7, Itamar Haber wrote:

After commenting out the 2nd `packed = msgpack.packb(data,use_bin_type=False)` (line 15), this works as expected. That means that the 1st `packed` (pushed) isn't the same as the 2nd `packed` (searched for) so LREM fails to find it in the list.

@Jan-Erik isn't it 512MB? :)

On Mon, Feb 23, 2015 at 11:22 AM, Jan-Erik Rediger <jan...@fnordig.de> wrote:

Hi,

use 'redis-cli MONITOR' in a second terminal to see the actual commands
that are received on Redis' side. Make sure the data is correct.

There is no minimum/maximum length Redis can handle on LREM (besides the
limit on the internal sds data type, which should be 2 GB, or was it 4 GB?)

On Sun, Feb 22, 2015 at 11:58:05PM -0800, Tampa312 wrote:
> Hi,
>
> The below will not delete the key in a a list using msgpack in python 2.7
>
> import redis
> import msgpack
> redis_master_conn = redis.StrictRedis()
>
> data = {'data': {u'account_hash': u'a8682994fe92180813627b37bea83d86',

> u'account_user': u'00', u'alert_email': u'fa...@gmail.com',

> u'account_password': u'jCjCEIwmIC0idins98vlj/o=', u'service_name':
> u'Aesop', u'service_id': 1, u'full_day_only': 0}}
>
> redis_master_conn.set('remove:%s' % data['data']['account_hash'],1)
> packed = msgpack.packb(data,use_bin_type=False)
> redis_master_conn.lpush('queue1',packed)
> data = redis_master_conn.brpoplpush('queue1','queue2',timeout=0)
> data = msgpack.unpackb(data)
> if redis_master_conn.exists('remove:%s' % data['data']['account_hash']):
>     packed = msgpack.packb(data,use_bin_type=False)
>     result = redis_master_conn.lrem('queue2',1,packed)
>     if result==1:
>         redis_master_conn.delete('remove:%s' % data['data']['account_hash'])
>     else:
>         print 'sad'
>     print 'result:',result
> else:
>     print 'no delete'
>
>
> sad
> result: 0
>
>
> Now...if I remove a key then it works:
>
> e.g I removed this key ->  u'account_user': u'00'
>
> data = {'data': {u'account_hash': u'a8682994fe92180813627b37bea83d86',

> u'alert_email': u'fa...@gmail.com', u'account_password':

> u'jCjCEIwmIC0idins98vlj/o=', u'service_name': u'Aesop', u'service_id': 1,
> u'full_day_only': 0}}
>
> result: 1
>
> Then one would think there is smething wrong with that key....so I placed

> back in and removed this key u'alert_email': u'fa...@gmail.com'

[Jan-Erik Rediger]

As always: you can't rely on the order of hashes/dictionaries :)

> >> > u'account_user': u'00', u'alert_email': u'fa...@gmail.com <javascript:>

> >> ',
> >> > u'account_password': u'jCjCEIwmIC0idins98vlj/o=', u'service_name':
> >> > u'Aesop', u'service_id': 1, u'full_day_only': 0}}
> >> >
> >> > redis_master_conn.set('remove:%s' % data['data']['account_hash'],1)
> >> > packed = msgpack.packb(data,use_bin_type=False)
> >> > redis_master_conn.lpush('queue1',packed)
> >> > data = redis_master_conn.brpoplpush('queue1','queue2',timeout=0)
> >> > data = msgpack.unpackb(data)
> >> > if redis_master_conn.exists('remove:%s' % data['data']['account_hash']):
> >> > packed = msgpack.packb(data,use_bin_type=False)
> >> > result = redis_master_conn.lrem('queue2',1,packed)
> >> > if result==1:
> >> > redis_master_conn.delete('remove:%s' %
> >> data['data']['account_hash'])
> >> > else:
> >> > print 'sad'
> >> > print 'result:',result
> >> > else:
> >> > print 'no delete'
> >> >
> >> >
> >> > sad
> >> > result: 0
> >> >
> >> >
> >> > Now...if I remove a key then it works:
> >> >
> >> > e.g I removed this key -> u'account_user': u'00'
> >> >
> >> > data = {'data': {u'account_hash': u'a8682994fe92180813627b37bea83d86',

> >> > u'alert_email': u'fa...@gmail.com <javascript:>', u'account_password':

> >> > u'jCjCEIwmIC0idins98vlj/o=', u'service_name': u'Aesop', u'service_id':
> >> 1,
> >> > u'full_day_only': 0}}
> >> >
> >> > result: 1
> >> >
> >> > Then one would think there is smething wrong with that key....so I
> >> placed
> >> > back in and removed this key u'alert_email': u'fa...@gmail.com

> >> <javascript:>'

> >> >
> >> > data = {'data': {u'account_hash': u'a8682994fe92180813627b37bea83d86',
> >> > u'account_user': u'00', u'account_password':
> >> u'jCjCEIwmIC0idins98vlj/o=',
> >> > u'service_name': u'Aesop', u'service_id': 1, u'full_day_only': 0}}
> >> >
> >> > result: 1
> >> >
> >> > So why the issue? Is there a min lenth that redis can user to compare
> >> keys
> >> > to delete? makes no sense to me why this is happing
> >> >
> >> >
> >> >
> >> > --
> >> > You received this message because you are subscribed to the Google
> >> Groups "Redis DB" group.
> >> > To unsubscribe from this group and stop receiving emails from it, send

> >> an email to redis-db+u...@googlegroups.com <javascript:>.

> >> > To post to this group, send email to redi...@googlegroups.com

> >> <javascript:>.

> >> > Visit this group at http://groups.google.com/group/redis-db.
> >> > For more options, visit https://groups.google.com/d/optout.
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Redis DB" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an

> >> email to redis-db+u...@googlegroups.com <javascript:>.

> >> To post to this group, send email to redi...@googlegroups.com

> >> <javascript:>.