performance tuning to the last bits.

[sting-]

Hello there, we are currently beta testing the mod_cached server
(v1.8.2 on lighty 1.4.23)
This server is currently serving nearly 440 million image hits/per day
(real traffic,thumbnails mostly) and I see there is too much CPU as
it's currently the bottleneck of the system. I was wondering if I can
improve the CPU usage with any other technique?
I haven't optimized the TCP memory or send/received buffer as
improving will only help for bigger files.
Below a snapshot of top and mod_cache config below that,
PS: I just noticed that 1.8.3. how much would that improve the
performance ?
Thanks in advance,

# dual quad (16 core) 2,5Ghz with 8MB of L3 cache, 24GB RAM on 2.6.18
- x86 platform.

top - 13:58:42 up 41 days, 16:52, 1 user, load average: 15.04,
11.73, 10.37
Tasks: 311 total, 9 running, 302 sleeping, 0 stopped, 0 zombie
Cpu(s): 23.1%us, 28.0%sy, 0.0%ni, 33.8%id, 0.1%wa, 1.6%hi,
13.5%si, 0.0%st
Mem: 24677072k total, 24594692k used, 82380k free, 226780k
buffers
Swap: 26738680k total, 204k used, 26738476k free, 21957884k
cached

Here is the mod_cache config

cache.enable = "enable"
#cache.max-memory-size = 8000
cache.lru-remove-count = 6500
cache.support-queries = "enable"
cache.bases = ("/stor/lcache")
cache.refresh-pattern = (
"\.(?i)(flv)$" => "2880 fetchall-for-range-request flv-
streaming",
"\.(?i)(js|css|xml)$" => "240",
"\.(?i)(htm|html|shtml)$" => "30",
"\.(?i)(jpg|bmp|jpeg|gif|png)$" => "2880",
"\.(?i)(rar|swf|zip|wmv|avi|mp3|mp4|m4a|m4v|ape|rm|mpeg|mpg|
wma|asf|rmvb|flv)$" => "2880 fetchall-for-range-request",
"." => "30 update-on-refresh"
)
proxy.server = ( "/" =>
(
( "host" => "127.0.0.1", "port" => 81 )
)
)

# Performance options below;

server.max-fds = 232768
server.max-connections = 15000
server.max-worker = 32
server.max-keep-alive-requests = 10000
server.max-keep-alive-idle = 60
server.max-read-idle = 30
server.max-write-idle = 120
server.event-handler = "linux-sysepoll"
server.network-backend = "linux-sendfile"
server.stat-cache-engine = "simple"

[shellcode]

For servers hosting small images, it's better to use memory cache of mod_cache instead of local disk cache.

I have one 32G memory, Xeon E5310 server, which can handle 4k thumbnails req/s easily, system load never goes up to 1.

here is config for you:

--------------

cache.bases = ("/opt/cache")

cache.support-queries = "enable"

cache.dynamic-mode = "enable"

cache.max-memory-size = 17000

cache.refresh-pattern = (

       "\.(?i)(js|css|xml)$" => "240",

       "\.(?i)(htm|html|shtml)$" => "30",

       "\.(?i)(jpg|bmp|jpeg|gif|png)$" => "0 use-memory",

       "." => "0 no-cache"

)

proxy.server  = ( "/" =>

        (

                 ( "host" => "127.0.0.1", "port" => 81 )

        )

)

server.event-handler = "linux-sysepoll"

server.max-keep-alive-requests = 32

server.max-keep-alive-idle = 5

server.max-request-size = 64

server.max-fds = 60000

server.max-connections = 30000

-------------------

please note: 

1) don't use server.max-worker

2) try modcache 1.8.3 + lighttpd 1.4.26 tarball

[sting-]

well, without the use of server.max-worker you are not utilizing the
CPU, as your load really can't go up to 1.
Because, a thread can only have %100 of CPU whereas you're actually
utilizing only 1/16 of real CPU power. (16 core CPU)
It totals %3-4 CPU being used.
Could you elaborate the logic behind not using the max-worker ?
Thanks,

> *1) don't use server.max-worker*
> *2) try modcache 1.8.3 + lighttpd 1.4.26 tarball*

[shellcode]

accessing disk files is cpu intensive, so server.max-worker helps when modcache stores cache in local disk.

memory accessing is really fast and isn't cpu intensive, so server.max-worker won't help too much.

memory cache of modcache doesn't share between different workers, so it's better to run one lighttpd process

[sting-]

OK.
I tried the latest version of mod_cache (modcache 1.8.3 + lighttpd
1.4.26) However, restarting the server crashes the lighty, instantly.
I'm only compiling it with --with-fam option nothing else. Only one
request is needed to crash the lighty, nothing in the logs.
I tracked it down a bit, removing the proxy.server lines fixes the
problem (contradicting whole function though)
Going back to 1.4.23 source and make install ; restart lighty and it
works again.
Has this release been tested ?

[shellcode]

I fix the segfault bug of mod_proxy, new tarball is http://lighttpd-improved.googlecode.com/files/lighttpd-1.4.26.modcache.v.1.8.3-2.tar.gz

sorry for the inconvenience.