Protocol error sometimes with django-redis

[Владислав Полухин]

Sometimes I get:

Protocol error, got "\xb2" as reply type byte

Protocol error, got "a" as reply type byte

Protocol error, got "\x82" as reply type byte

Protocol error, got "\x00" as reply type byte

Protocol error, got "\xd1" as reply type byte

Used: 

Kernel 2.6.32-5-amd64 #1 SMP  x86_64 GNU/Linux

Redis server version 2.4.10 (00000000:0)

Python packages:

Django==1.3.1

django-redis==1.4.5

hiredis==0.1.0

redis==2.4.11

Redis settings:

daemonize yes

port 6379

bind 127.0.0.1

unixsocket /tmp/redis.sock

unixsocketperm 0777

timeout 10

loglevel notice

logfile /var/log/redis/redis-server.log

databases 16

save 60 10

rdbcompression yes

dbfilename dump.rdb

dir /var/lib/redis

maxclients 0

appendonly yes

appendfsync no

Django settings:

CACHES = {'default': {'BACKEND': 'redis_cache.cache.RedisCache',

                      'LOCATION': 'localhost:6379',

                      'TIMEOUT': 24 * 24 * 60 * 30, # month

                      'OPTIONS': {'DB': 0,

                                  'PARSER_CLASS': 'redis.connection.HiredisParser',

                                  'MAX_ENTRIES': 0}}}

Django runs as fastcgi daemon on highload server (about 110 000 views in day).

[Pieter Noordhuis]

This is usually a sign of the connection being shared between threads
or processes. If you fork your application server at start, do you
reconnect every child process? Similarly, if you use threads, does
every thread have its own Redis connection, or do you use locks to
protect the client from being used by concurrent threads?

Cheers,
Pieter

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/L7lAGjpD7oUJ.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.

[Andy McCurdy]

2.4.11 should be completely thread safe. If you're sharing connections across multiple processes via fork(), that's a different story.

[Josiah Carlson]

I don't think anyone is sharing the connections on purpose. This is
one of the reasons why I implemented my own connection pooling rather
than using the pooling built into redis-py.

Regards,
- Josiah

[Andy McCurdy]

Hey Josiah,

Can you point me to your implementation? What are the benefits over the connection pooling built in? Is it something we could potentially include in redis-py?

-andy

[Josiah Carlson]

Let me fork and send you a pull request. It's faster to code it than
it is to explain how to fix it.

Regards,
- Josiah

[Josiah Carlson]

This should be close to a minimal fix:
https://github.com/andymccurdy/redis-py/pull/236

As long as a fork() doesn't happen while a command is being run in the
forking thread, this should be sufficient.

Regards,
- Josiah

[Владислав Полухин]

I tried that patch on our servers. Now works fine, waiting tomorrow.

четверг, 12 апреля 2012 г., 7:29:46 UTC+8 пользователь Josiah Carlson написал:

>> redis-db+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/redis-db?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Redis DB" group.
>> To post to this group, send email to redi...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> redis-db+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/redis-db?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Redis DB" group.
>> To post to this group, send email to redi...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> redis-db+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/redis-db?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Redis DB" group.
>> To post to this group, send email to redi...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> redis-db+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/redis-db?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Redis DB" group.
>> To post to this group, send email to redi...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> redis-db+unsubscribe@googlegroups.com.