Problem with memcache - very bad performance
Szymon
I'm using latest Django trunk and have problems with memcache. For few
days my site is struggling with performance. I've blamed mod_wsgi,
I've blamed Apache, I've blamed Postgres on dedicated box, I've blamed
everything but today I've done profiling... and the winner is...
72.1% 4.862 build/bdist.freebsd-7.1-RELEASE-amd64/egg/memcache.py
5 secs. (!) 72% of execution time. Something that should boost
performance, kills my WWW box! There were about 30-40 calls.
It's not normal? What can cause problem? There are no any infos in
logs. Nothing. :S
Best regards,
Szymon Kosok
Szymon
> 5 secs.
CPU secs. of course.
Now, on lighter traffic (same view).
12.1% 0.069 build/bdist.freebsd-7.1-RELEASE-amd64/egg/memcache.py
Strange. Probably on higher load I will get faster response times just
querying DB. :S Any suggestions? Maybe it's normal?
Javier Guerra
are you sure you get memcached store on RAM? maybe it's being swapped out
--
Javier
Szymon
> are you sure you get memcached store on RAM? maybe it's being swapped out
Yes. Swap hasn't been touched yet since last reboot.
Mem: 400M Active, 2915M Inact, 354M Wired, 66M Cache, 214M Buf, 193M
Free
Swap: 4000M Total, 4000M Free
But that's strange:
# netstat | grep localhost.11211 | wc -l
652
It's ok? Almost all in "TIME_WAIT" state. Now, when there is really
small traffic.
Szymon
I've turned -vv logging and sometimes it throws:
>18 ERROR
<18 ccopy_reg
>18 ERROR
<18 _reconstructor
>18 ERROR
<18 p1
>18 ERROR
<18 (cdjango.db.models.query
>18 ERROR
<18 QuerySet
>18 ERROR
<18 p2
>18 ERROR
<18 c__builtin__
>18 ERROR
<18 object
[snip - it's long]
Some problem with caching QuerySets?
Szymon
I've changed backend to pylibmc (using some custom backend found in
Google, that needed some changes to work) , installed Memcached on
local WWW box (it was on DB box before), connecting via socket...
and...
2.5% 0.009 /home/www/dev/mc.py
But we will see in few hours, when traffic will be heavy (now is light/
moderate).
Best regards,
Szymon