Too many backend server connections

[Bo Yu]

Hi All,

I'm using lighttpd 1.4.23 with mod_cache 1.8.0 for static pic cache
service. But I encountered a problem: when I started lighty online, it
can just serve normally for 2-3 minutes, then the pics can't be
accessed. I found there were too many connections established to the
backend server( more than 30,000 ), therefore, the backend server
can't handle these too many connections/requests. Is there any
solution that can help release the connections to the backend server
very quickly? Don't talk to me about the hitrate, cause it's the first
time to start lighty+modcache as the cache server that it need to
create the cache content for the first time.

followed is my configuration:

server.modules =
("mod_redirect","mod_access","mod_status","mod_cache","mod_mem_cache","mod_proxy","mod_compress","mod_expire","mod_accesslog")
server.max-keep-alive-requests = 4
server.max-keep-alive-idle = 4
server.max-read-idle = 30
server.max-write-idle = 180
server.event-handler = "linux-sysepoll"
server.network-backend = "linux-sendfile"
server.max-fds = 65535
server.stat-cache-engine = "fam"
cache.support-queries = "enable"
cache.dynamic-mode = "enable"
cache.bases = ("/srv/lcache")
cache.max-memory-size = 1536
cache.refresh-pattern = (
"\.(?i)(flv)$" => "0 fetchall-for-range-request flv-streaming",
"\.(?i)(js|css|xml)$" => "10080″,
"\.(?i)(htm|html|shtml)$" => "10080″,
"\.(?i)(jpg|bmp|jpeg|gif|png)$" => "10080 use-memory″,
"\.(?i)(rar|zip|wmv|avi|mp3|ape|rm|mpeg|mpg|wma|asf|rmvb|flv)$" => "0
fetchall-for-range-request",
"." => "10080 update-on-refresh"
)
cache.purge-host = "^(192\.168\.)"
cache.ignore-hostname = "enable"
proxy.server = ( "/" =>
( "localhost" =>
(
"host" => "192.168.1.18",
"port" => 3080
)
)
)
proxy.worked-with-mod-cache = "enable"

Any help is much appreciated!

[shellcode]

1) try to change proxy.server to

proxy.server  = ( "/" =>
        (

                ( "host" => "192.168.1.18", "port" => 3080 )
        )
)

2) disable keepalive of backend http server

3) what's the capacity of your cache server? how many memory?

[Kevin Yu]

Hi shellcode,

Thanks for your advice. I will find another chance to try. any result
I will give feedback.
My server's total memory is 4G.

Kevin Yu
Mail: flyi...@gmail.com

[Kevin Yu]

BTW: isn't the connection number increasing if I disable keepalive of
backend http server?

Kevin Yu
Mail: flyi...@gmail.com



On Thu, Aug 6, 2009 at 11:10 AM, shellcode<shel...@gmail.com> wrote:

[shellcode]

it is better to provide the http response header using 'curl -v http://host/uri'

there maybe some http header to prevent modcache from cache response

[Kevin Yu]

here are the response headers:

< HTTP/1.1 200 OK
< Content-Type: image/jpeg
< Last-Modified: Thu, 09 Jul 2009 04:58:51 GMT
< X-Powered-By: ASP.NET
< Expires: Thu, 13 Aug 2009 08:16:10 GMT
< Cache-Control: max-age=604800
< X-Cache: HIT
< Content-Length: 2298
< Date: Thu, 06 Aug 2009 08:16:10 GMT
< Server: lighttpd

Kevin Yu
Mail: flyi...@gmail.com