How Redis is behaving in production with lloogg.com
Salvatore Sanfilippo
Friday, 19 June I switched the lloogg.com service from MySQL to Redis.
I plan to post a detailed article about the switch, but I've already
some interesting data on how it is behaving in a production
environment with a decent load.
I've two servers, one is a database, one is a web server. The database
runs just Redis, and the web server an apache optimized for high load.
The web server also runs a Redis slave connected to the Redis master
running on the DB in order to make sure the data is safe. In order to
better stress Redis I enabled object sharing only on the slave, so I
can compare the memory usage in the two processes. Saving is enabled
in both the Redis servers.
Hardware: both the boxes are powered by an Intel(R) Xeon(R) CPU X3320
@ 2.50GHz, 32bit, with 4 GB of RAM each. In other words, this are very
very cheap servers.
Redis master:
redis_version:0.900
uptime_in_seconds:770018
uptime_in_days:8
connected_clients:1
connected_slaves:1
used_memory:165710581
changes_since_last_save:10801
bgsave_in_progress:0
last_save_time:1246112512
total_connections_received:7661948
total_commands_processed:49106754
role:master
db0: keys=1315382,expires=0
This server received around 50 millions of queries in an 8 days period
(around 72 queries for second). Given that most lloogg users are
Italians actually there are hours of the day when this queries are
around 150 per second or more. In order to serve all this queries
Redis used very little cpu time:
root 30106 0.2 6.4 235292 234116 pts/1 S+ Jun18 34:23
./redis-server redis.conf
A total of 35 milutes of CPU time in 8 days. Doing some trivial math
you'll find that in theory in order to saturate the CPU it requires a
load of around 23k queries for second. This is perfectly in line to
what we can expect from benchmarks, given that currently LLOOGG is
opening and closing the TCP connection again and again for every
client request. (see the total_connections_received numbers). To take
the same socket connected should make it much more cheaper. Let's look
at the slave:
Redis slave:
redis_version:0.900
uptime_in_seconds:430181
uptime_in_days:4
connected_clients:2
connected_slaves:0
used_memory:131705620
changes_since_last_save:19571
bgsave_in_progress:0
last_save_time:1246112403
total_connections_received:3
total_commands_processed:15671577
role:slave
master_host:db
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
db0: keys=1315382,expires=0
The slave is using less memory because of object sharing, even if
lloogg is not a good match for object sharing given that the main data
is about Redis lists with serialized data inside.
root 32056 0.1 4.4 164256 163120 pts/2 S+ Jun22 7:55
./redis-server redis.conf
Obviously the slave used less CPU, just 8 minutes, because of the
single connection and because all the read-only queries are avoided.
After the switch from MySQL to Redis the DB server load is very low:
http://db.lloogg.com/load.dir/load-1month.png
Compare this with the load data before the switch. So the conclusion
is that Redis is working very well for the application domain it was
conceived for. We never experienced any stability problem, even in
other production sites where more features are used (like expires). I
think that after applying the latest patches received it's worth to
add more documentation (especially a quick start guide about
compiling, configuring, ...) and release Redis 1.0 stable.
Cheers,
Salvatore
--
Salvatore 'antirez' Sanfilippo
http://invece.org
"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay
Brian Hammond
janek
>
> http://db.lloogg.com/load.dir/load-1month.png
>
> Compare this with the load data before the switch. So the conclusion
> is that Redis is working very well for the application domain it was
> conceived for.
best regards,
janek