Apache suddenly stops serving
andy thomas
stopped serving up any pages. On investigation, it was still running but
instead of the usual 30 or so running httpd processes, there were about
90 of them. 'apachectl restart' does not fix the problem, but stopping
Apache altogether and then starting up afresh restores normal operation.
Has anyone come across this problem before? If it's relevant to the
problem, the Apache is version 1.3.19 compiled with PHP4/MySQL support and
it runs in conjunction with Allaire ColdFusion 5.0 server, loading the
latter's interface as a module. The platform is Linux (Slackware 7.1) on
Intel-based hardware (733 MHz pentium 3, 512 MB RAM and 2 GB swap). It
does not appear to be load-related - the first failure occurred when the
server was serving about 6 hits/sec and the second when it was servicing
about 30 hits/sec.
We use remote monitoring to alert us of any problems on the server but
this one is fooling the monitoring system as the server reports (via
ps -ax) httpd processes are still running! So we don't know it's failed
until the phones start ringing. (I've now written a script that uses lynx
to access a particular page on the server every 2 minutes and if
unsuccessful, it stops and restarts the server).
Any suggestions would be gratefully received.
Andy
Webmaster (Nemesis Services)
MaxSpareServers and same for StartServers
get your server checking tool to actually do a check on the www server ie
HTTP REQUEST have a look at: www.ipsentry.net
"andy thomas" <an...@ic.ac.uk> wrote in message
news:Pine.LNX.4.33.011122...@anahata.ma.ic.ac.uk...
Joshua Slive
> you could make the MinSpareServers x2 as much as it is same with
> MaxSpareServers and same for StartServers
Whatever.
> "andy thomas" <an...@ic.ac.uk> wrote in message
> news:Pine.LNX.4.33.011122...@anahata.ma.ic.ac.uk...
>> We have had two incidents in the last 5 weeks where Apache suddenly
>> stopped serving up any pages. On investigation, it was still running but
>> instead of the usual 30 or so running httpd processes, there were about
>> 90 of them. 'apachectl restart' does not fix the problem, but stopping
>> Apache altogether and then starting up afresh restores normal operation.
>>
Try setting up mod_status and see what the apache processes are doing
when they freeze. Check to see if you can "kill" the individual child
processes.
If you find that the processes are stuck in "keepalive", then check
the apache bug database for some ideas.
--
Joshua Slive
ne...@slive.ca
http://slive.ca/
Webmaster (Nemesis Services)
your nappies :))
"Joshua Slive" <ne...@slive.ca> wrote in message
news:9tjck5$aaq$4...@nntp.itservices.ubc.ca...
andy thomas
On Thu, 22 Nov 2001, Webmaster (Nemesis Services) wrote:
> you could make the MinSpareServers x2 as much as it is same with
> MaxSpareServers and same for StartServers
I'm using the current default settings at the moment. But I didn't think
this was the problem as according to the Apache documentation, if the number
of incoming requests exceeds MaxSpareServers, Apache simply ignores these
and doesn't start up any more child servers. It doesn't say anything about
it stopping serving pages either!
> get your server checking tool to actually do a check on the www server ie
> HTTP REQUEST have a look at: www.ipsentry.net
I'll go have a look at this. Is it destructive? (If so I'll wait until the
middle of the night at the weekend).
thanks for the suggestions,
cheers,
Andy
andy thomas
On 22 Nov 2001, Joshua Slive wrote:
> > "andy thomas" <an...@ic.ac.uk> wrote in message
> > news:Pine.LNX.4.33.011122...@anahata.ma.ic.ac.uk...
> >> We have had two incidents in the last 5 weeks where Apache suddenly
> >> stopped serving up any pages. On investigation, it was still running but
> >> instead of the usual 30 or so running httpd processes, there were about
> >> 90 of them. 'apachectl restart' does not fix the problem, but stopping
> >> Apache altogether and then starting up afresh restores normal operation.
> >>
>
> Try setting up mod_status and see what the apache processes are doing
> when they freeze. Check to see if you can "kill" the individual child
> processes.
Accessing the status page won't work because the server is effectively
dead and won't serve any pages, even though it appears to be running.
> If you find that the processes are stuck in "keepalive", then check
> the apache bug database for some ideas.
Andy
Joshua Slive
> Accessing the status page won't work because the server is effectively
> dead and won't serve any pages, even though it appears to be running.
You need to access the status page before the server locks up.
Dr Hackenbush
> stopped serving up any pages. On investigation, it was still running
but
> instead of the usual 30 or so running httpd processes, there were
about
> 90 of them. 'apachectl restart' does not fix the problem, but
stopping
> Apache altogether and then starting up afresh restores normal
operation.
When you restart your server any resource leaks go away.
What is the value of MaxRequestsPerChild ?
The default is 0, which means &infinity; and is in effect
if you don't specify anything. The documentation notes
this is fine as long as there are no leaks. Now your
situation suggests that something is wrong and it will
do very little harm to set this parameter to something less
than infinity.
If you do this and the problem goes away, you might
report it or look for a problem in your custom code.
If you do this and the problem does not go away, you
will have an indication that the problem is not a leak
[It could still be a leak, but one that becomes serious
before MaxRequestsPerChild is reached.]
Webmaster (Nemesis Services)
andy thomas
On Fri, 23 Nov 2001, Dr Hackenbush wrote:
>
> "andy thomas" <an...@ic.ac.uk> wrote in message
> news:Pine.LNX.4.33.011122...@anahata.ma.ic.ac.uk...
> > We have had two incidents in the last 5 weeks where Apache suddenly
> > stopped serving up any pages. On investigation, it was still running
> but
> > instead of the usual 30 or so running httpd processes, there were
> about
> > 90 of them. 'apachectl restart' does not fix the problem, but
> stopping
> > Apache altogether and then starting up afresh restores normal
> operation.
>
> When you restart your server any resource leaks go away.
>
> What is the value of MaxRequestsPerChild ?
Zero, the default.
> The default is 0, which means &infinity; and is in effect
> if you don't specify anything. The documentation notes
> this is fine as long as there are no leaks. Now your
> situation suggests that something is wrong and it will
> do very little harm to set this parameter to something less
> than infinity.
I'll try setting this to, say, 10000.
> If you do this and the problem goes away, you might
> report it or look for a problem in your custom code.
Thanks for your suggestion, I'll try it now. This problem has only
happened twice so far so it's not easy to reproduce. There is no real
custom code - it's a standard 1.3.19 build with PHP4 compiled in and
the ColdFusion module, mod_coldfusion.so, was supplied with ColdFusion 5.0
Professional from Macromedia. (We've had more trouble with ColdFusion
- including total crashes - than anything else on our servers but that's
another story and one which now appears to have been resolved).
> If you do this and the problem does not go away, you
> will have an indication that the problem is not a leak
> [It could still be a leak, but one that becomes serious
> before MaxRequestsPerChild is reached.]
Many thanks,
Andy
Rich Bowen
>
> On Thu, 22 Nov 2001, Webmaster (Nemesis Services) wrote:
>
> > you could make the MinSpareServers x2 as much as it is same with
> > MaxSpareServers and same for StartServers
>
> I'm using the current default settings at the moment. But I didn't think
> this was the problem as according to the Apache documentation, if the number
> of incoming requests exceeds MaxSpareServers, Apache simply ignores these
> and doesn't start up any more child servers. It doesn't say anything about
> it stopping serving pages either!
No, that's not quite what MaxSpareServers does.
Apache will launch new children in such a way that there are always AT
LEAST MinSpareServers idle child processes, and AT MOST MaxSpareServers
processes. MaxSpareServers is the max number of SPARE servers, not the
max total.
There are a number of things that are possibly happening that can cause
this sort of a condition. For example, if you are out of memory, and
unable to start new processes, Apache might be unable to serve incoming
requests. I would suggest that you tail the logfiles:
tail -f error_log access_log
and watch what is going on at the moment that things hang. Of course,
this means that you have to actually be watching when things start going
sour, but I would expect that once it hangs, you'll stop seeing new log
entries. You're probably looking for someone accessing some dynamic
content - a CGI program, or PHP page, or cfm page, for example - which
is causing the problem. If you notice the same resource being accessed
consistently before hangs, then you have your culprit.
--
Rich Bowen <rbo...@rcbowen.com>
http://www.apacheadmin.com/