Recipe for verifying a log is growing?

[Dustin]

Does anyone have anything handy that will allow me to set a restart
condition on a log not growing?

I've got ruby processes that just hang hard.

[deaconbradley]

Hello,

I have been doing this recently using a restart condition polling a
ruby-script. This was a quick-fix to get me through the weekend so
please excuse the style :) ::

First have this custom PollCondition loaded somewhere in your configs:

******* BEGIN *********
module God
module Conditions

class LogWatch < PollCondition
attr_accessor :log_file, :above

def initialize
super
self.log_file = nil
self.above = nil
end

def valid?
valid = true
valid &= complain("Attribute 'log_file' must be specified",
self) if self.log_file.nil?
valid &= complain("Attribute 'above' must be specified", self)
if self.above.nil?
valid
end

def test
begin
seconds = `/path/to/exec/script/below #{self.log_file}`
seconds.to_i > self.above
rescue
true
end
end
end # LogWatch
************ END *************

Then I used this script which right now is completely external to the
God stuff (but I'd like to integrate it if i ever had time)

******** script BEGIN **********
#!/usr/local/bin/ruby
#
# Print the number of seconds since the last log statement in the
given log
#
# $Id:$
#

require 'rubygems'
require 'activesupport'
require 'date'

log_dir = "/var/rails/log"

if ARGV.length != 1
puts "You must provide ONE log file name to check!"
exit 1
end

log_file = ARGV.first

if ! File.readable?("#{log_dir}/#{log_file}")
puts "Cannot read #{log_file}"
exit 1
end

begin
t = File.mtime("#{log_dir}/#{log_file}")
last_log_time = DateTime.parse(t.to_s)
rescue
last_log_time = DateTime.now
end

# convert to DateTime
diff = (Time.now - last_log_time.to_time).to_i
puts diff
******** script END **********

Then in my God watch I add the following poll condition:

******* watch BEGIN *********
w.restart_if do |restart|
restart.condition(:log_watch) do |c|
c.log_file = "really_important.log"
c.above = 300 # 5 min (in seconds)
c.notify = 'warnings'
end
end
******* watch END *********

-- Deacon Bradley <dea...@otherinbox.com>

[Richard Heycock]

Excerpts from dsallings's message of Tue Nov 04 09:54:34 +1100 2008:

>
> Does anyone have anything handy that will allow me to set a restart
> condition on a log not growing?
>
> I've got ruby processes that just hang hard.

Just a thought, but couldn't you attach an inotify watch to the log file
and every time an event is generated check to see if the size has
changed. If there is no notification within a certain time you could
restart or do something to generate a log message just to see if it
responds.

Not sure if this would make life more difficult or easy, as I said just
a thought!

rgh
--
+61 (0) 410 646 369
[e]: r...@neoss.com.au
[im]: r...@jabber.org

You're worried criminals will continue to penetrate into cyberspace, and
I'm worried complexity, poor design and mismanagement will be there to meet
them - Marcus Ranum