HELP: cache repopulation under heavy load

[Pete]

I'm running a site that was recently slammed - 180,000 unique visitors
yesterday.

Lighty + mod_cache saved my pants! So thank you to everyone involved.

My setup is to have lighty in "front" of an apache based php site

I've noticed some behavior that stresses the system when a cached file
is expired. It seems that when a cached file is expired, if
additional requests come in for that file between the time the first
request hit the backend and the time the cache is repopulated, they
all go through to the backend. This causes "rushes" against the
backend - behaving as if the caching server isn't there at all. There
also appears to be some conflict opportunity on cache writing? It
seems like these "rushes" that I'm getting dont' resolve quickly and
can go on for a minute or so.

I'm also seeing "fatal error"s on these files presumably due to the
file being deleted before the sendfile actually gets around to finding
it.

Below are snippets from my lighty config and error.log (with cache
debug enabled).

Is there anything I can do to fix this?

Cheers,

pete

------- lighttpd.conf ------

cache.support-queries = "enable" #ignore '?' in url
cache.dynamic-mode = "enable"
cache.debug = "enable"
cache.bases = ("/root/taxes/lighty/cache") #write cached files in /
data/cache directory
cache.refresh-pattern = (
"/wp-admin" => "nocache",
"/jscomments.php" => "5 override-expire ignore-cache-control-
header",
"." => "60 override-expire ignore-cache-control-header"
)


------- error.log -------

mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.1860: (trace) /root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php expired^M
mod_cache.c.1860: (trace) /root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php expired^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.1860: (trace) /root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php expired^M
mod_cache.c.1860: (trace) /root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php expired^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2178: (trace) cache saved to /root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php successfully^M
mod_cache.c.2096: (trace) successfully delete cache file /root/taxes/
lighty/cache/www.electiontaxes.com/jscomments.php before update^M
network_linux_sendfile.c.79: (error) opening '/root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php' failed: No such file or
directory^M
connections.c.1412: (trace) (network-subsys sent us a fatal-error)^M
network_linux_sendfile.c.79: (error) opening '/root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php' failed: No such file or
directory^M
connections.c.1412: (trace) (network-subsys sent us a fatal-error)^M
network_linux_sendfile.c.79: (error) opening '/root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php' failed: No such file or
directory^M
connections.c.1412: (trace) (network-subsys sent us a fatal-error)^M
network_linux_sendfile.c.79: (error) opening '/root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php' failed: No such file or
directory^M
connections.c.1412: (trace) (network-subsys sent us a fatal-error)^M
network_linux_sendfile.c.79: (error) opening '/root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php' failed: No such file or
directory^M
connections.c.1412: (trace) (network-subsys sent us a fatal-error)^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
mod_cache.c.2178: (trace) cache saved to /root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php successfully^M
mod_cache.c.2096: (trace) successfully delete cache file /root/taxes/
lighty/cache/www.electiontaxes.com/jscomments.php before update^M
mod_cache.c.2047: (trace) somebody is handling /root/taxes/lighty/
cache/www.electiontaxes.com/jscomments.php now^M
network_linux_sendfile.c.79: (error) opening '/root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php' failed: No such file or
directory^M
connections.c.1412: (trace) (network-subsys sent us a fatal-error)^M
mod_cache.c.2178: (trace) cache saved to /root/taxes/lighty/cache/
www.electiontaxes.com/jscomments.php successfully^M

[Hongyu]

   Modcache keeps list of urls which are handled to avoid serveral connections writing to one cache file at the same time. That's the reason of "somebody is handling /root/taxes/lighty/cache/www.electiontaxes.com/jscomments.php now^M" in lighttpd error logfile