Strange slave behavior
Alex Arnell
We have a single master and a single slave setup. The master
currently holds about 5,223,423 in 1.65 GB of memory and has been
running solid since we turned it on almost 3 weeks ago.
The slave is another story. We have to shut it down every 3 days,
delete it's disk backup and restart it. After the 3rd day of
operation it consumes roughly twice the memory, 3.43 GB and even holds
more keys than the master, 6,167,490 according to DBSIZE.
Our application performs the occasional KEYS command on the slave to
prevent locking in the master but that is it.
Are the DELETE commands synced across to the slave?
Salvatore Sanfilippo
>
> We're seeing some strange Redis slave behavior in our current setup.
Hello Alex,
this problems looks like a missing server->dirty++; call in some
delete or delete-alike operation (like list pop).
I'll check it right now all the commands, but please if you can send
me a list of commands you are using against Redis.
Thank you!
Salvatore
--
Salvatore 'antirez' Sanfilippo
http://invece.org
"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay
Salvatore Sanfilippo
> this problems looks like a missing server->dirty++; call in some
> delete or delete-alike operation (like list pop).
> I'll check it right now all the commands, but please if you can send
> me a list of commands you are using against Redis.
Ok... EXPIRE was not propagated to slaves for a missing server->dirty increment.
Now all is ok if you ack you are using expire. Otherwise there is
another bug to fix :)
Cheers,
Salvatore
Alex Arnell
Thanks for the quick reply. Yes we are using EXPIRE for some of the
keys in the database.
Here is a complete list of the commands our application is using.
SET
GET
GETSET
MGET
SETNX
INCR
INCRBY
DEL
KEYS
EXPIRE
RPUSH
LPUSH
LRANGE
LPOP
RPOP
SADD
SREM
SMEMBERS
Cheers,
Alex Arnell
On Jul 3, 10:58 am, Salvatore Sanfilippo <anti...@gmail.com> wrote:
> > this problems looks like a missing server->dirty++; call in some
> > delete or delete-alike operation (like list pop).
> > I'll check it right now all the commands, but please if you can send
> > me a list of commands you are using against Redis.
>
> Ok... EXPIRE was not propagated to slaves for a missing server->dirty increment.
> Now all is ok if you ack you are using expire. Otherwise there is
> another bug to fix :)
>
> Cheers,
> Salvatore
>
>
>
> > Thank you!
> > Salvatore
>
> > --
> > Salvatore 'antirez' Sanfilippo
> >http://invece.org
>
> > "Once you have something that grows faster than education grows,
> > you’re always going to get a pop culture.", Alan Kay
>
> --
Salvatore Sanfilippo
>
> Hi Salvatore,
>
> Thanks for the quick reply. Yes we are using EXPIRE for some of the
> keys in the database.
Ok, so it's almost sure we fixed your problem.
In the latest email forgot to mention that the fix is already on the
Git repo. Please if you can upgrade the slave and try if the problem
is gone. Thank you very much!
Cheers,
Salvatore
p.s. The version of Redis in Git is almost the same as the latest
tar.gz so you are not going to run less stable code, just a bit more
fixed code :)
Alex Arnell
back to you on whether or not the problem still exists or not.
Thanks again,
Alex ARnell
On Jul 3, 12:23 pm, Salvatore Sanfilippo <anti...@gmail.com> wrote:
>
> > Hi Salvatore,
>
> > Thanks for the quick reply. Yes we are using EXPIRE for some of the
> > keys in the database.
>
> Ok, so it's almost sure we fixed your problem.
> In the latest email forgot to mention that the fix is already on the
> Git repo. Please if you can upgrade the slave and try if the problem
> is gone. Thank you very much!
>
> Cheers,
> Salvatore
>
> p.s. The version of Redis in Git is almost the same as the latest
> tar.gz so you are not going to run less stable code, just a bit more
> fixed code :)
>
> --