Understanding the Rule and Filter classes

[Mohanaraj]

Hi ,

In trying to solve my earlier challenge I have been trying to
understand how loghetti works. What confuses me is the Rule class and
how its applied in the Filter class - specifically the strainer
method.

Firstly the Rule class. Looking at the names of the variables

attr - Which I presume is referring to the names of the attributes of
the line object
cmp - Which I understood to mean the 'comparison' operator
val - which I understand to be the value which we are looking for that
is picked up from the command line

What I find to be confusing is that the 'cmp' attribute is not used at
in the strainer method ?

Am I missing something here ? Is there some design decision I do not
fully appreciate ?

I have done some minor changes so that I could leverage the self.cmp
setting of the Rule class as follows:

Firstly I implemented a __call__ method for the Rule class:

def __call__(self,line):
if self.cmp == "==":
return (line.__dict__[self.attr] == self.val)
elif self.cmp == "in":
return (line.__dict__[self.attr] in self.val)
elif self.cmp == ">":
return (line.__dict__[self.attr] > self.val)
elif self.cmp == ">=":
return (line.__dict__[self.attr] >= self.val)
elif self.cmp == "<":
return (line.__dict__[self.attr] < self.val)
elif self.cmp == "<=":
return (line.__dict__[self.attr] <= self.val)
elif self.cmp == "!=":
return (line.__dict__[self.attr] != self.val)
else :
raise NotImplementedError("Unknown comparator %s"%self.cmp)

Then I changed the Filter class strain method as follows:

for rule in self.rules:
try:
- show_line &= (rule.val == eval(rule.attr, line.__dict__)) #
returns false if the right side is false.
+ show_line &= (True == rule(line)) # returns false if the
right side is false.
except KeyError:

The instantiations of Rule will also as used in the optionHandler_X
series of methods would also need to be changed from:

self.x = Rule("minute", "=", X)
to:
self.x = Rule("minute", "==",x)


This allows for the cmp attribute of the Rule class to be used to
determine the nature of the Rule. This might also open possibilities
for further options to be passed in at runtime by the user. What do
you think ? Please do not mind my ignorance if I completely missed
something here.

I attach my complete diff below:

Mohan

--

Index: loghetti.py
===================================================================
--- loghetti.py (revision 10)
+++ loghetti.py (working copy)
@@ -23,6 +23,24 @@

def __str__(self):
return ','.join([self.attr, self.cmp, self.val])
+
+ def __call__(self,line):
+ if self.cmp == "==":
+ return (line.__dict__[self.attr] == self.val)
+ elif self.cmp == "in":
+ return (line.__dict__[self.attr] in self.val)
+ elif self.cmp == ">":
+ return (line.__dict__[self.attr] > self.val)
+ elif self.cmp == ">=":
+ return (line.__dict__[self.attr] >= self.val)
+ elif self.cmp == "<":
+ return (line.__dict__[self.attr] < self.val)
+ elif self.cmp == "<=":
+ return (line.__dict__[self.attr] <= self.val)
+ elif self.cmp == "!=":
+ return (line.__dict__[self.attr] != self.val)
+ else :
+ raise NotImplementedError("Unknown comparator %s"%self.cmp)

class Filter(object):
"""
@@ -48,6 +66,7 @@
line.date = time.strptime(line.time, '%d/%b/%Y:%H:%M:%S')
line.year, line.month, line.day, line.hour, line.minute,
line.second = line.date[0:6]
if self.process_url:
+
line.urlbase = urlparse.urlparse(line.url)[2]
line.base = self.baserex.search(line.urlbase)
if not line.base:
@@ -62,7 +81,7 @@

for rule in self.rules:
try:
- show_line &= (rule.val == eval(rule.attr, line.__dict__)) #
returns false if the right side is false.
+ show_line &= (True == rule(line)) # returns false if the
right side is false.
except KeyError:
show_line = False
if not show_line:
@@ -96,7 +115,7 @@
"""
Return all lines in file containing the user-supplied HTTP
response code (500, 404, 200, etc)
"""
- self.coderule = Rule("http_response_code", "=", respcode)
+ self.coderule = Rule("http_response_code", "==", respcode)
self.ruleset.append(self.coderule)
return

@@ -113,7 +132,7 @@
"""
Return lines in the log that match the given IP address.
"""
- self.iprule = Rule("ip", "=", ip)
+ self.iprule = Rule("ip", "==", ip)
self.ruleset.append(self.iprule)
return

@@ -123,7 +142,7 @@
to see lines from January 31, the way to do that is to pass
"131". February 3rd? Pass "23". Lame, I know.
I'm working on it :)
"""
- self.monthrule = Rule("month", "=", int(month))
+ self.monthrule = Rule("month", "==", int(month))
self.ruleset.append(self.monthrule)
self.process_date = True
return
@@ -133,7 +152,7 @@
Pass in a non-zero-padded day (1-31). Sorry, doesn't yet accept a
range, though passing
multiple --day arguments should work.
"""
- self.dayrule = Rule("day", "=", int(day))
+ self.dayrule = Rule("day", "==", int(day))
self.ruleset.append(self.dayrule)
self.process_date = True
return
@@ -143,7 +162,7 @@
Pass in a non-zero-padded month (1-12). Sorry, doesn't yet accept
a range, though passing
multiple --year arguments should work.
"""
- self.yearrule = Rule("year", "=", int(year))
+ self.yearrule = Rule("year", "==", int(year))
self.ruleset.append(self.yearrule)
self.process_date = True
return
@@ -153,7 +172,7 @@
Pass in a non-zero-padded hour (0-23). Sorry, doesn't yet accept
a range, though passing
multiple --hour arguments should work.
"""
- self.hourrule = Rule("hour", "=", int(hour))
+ self.hourrule = Rule("hour", "==", int(hour))
self.ruleset.append(self.hourrule)
self.process_date = True
return
@@ -163,7 +182,7 @@
Pass in a non-zero-padded minute (0-59). Sorry, doesn't yet
accept a range, though passing
multiple --minute arguments should work.
"""
- self.minuterule = Rule("minute", "=", int(minute))
+ self.minuterule = Rule("minute", "==", int(minute))
self.ruleset.append(self.minuterule)
self.process_date = True
return
@@ -175,7 +194,7 @@
'/file.php?foo=bar&bar=baz&abc=def&stuff=idunno', you just ask
for 'file.php' instead of that big long
url.
"""
- self.urlrule = Rule("base", "=", urlbase)
+ self.urlrule = Rule("base", "==", urlbase)
self.ruleset.append(self.urlrule)
self.process_url = True
return
@@ -184,7 +203,7 @@
"""
Pass in an http method (probably GET or POST, but any should
work) to filter on.
"""
- self.methodrule = Rule("http_method" , "=", method)
+ self.methodrule = Rule("http_method" , "==", method)
self.ruleset.append(self.methodrule)
return

@@ -195,7 +214,7 @@
"""
key, val = keyval.split(':')
attr = "urldata['%s'][0]" % key
- self.urldatarule = Rule(attr, "=", val)
+ self.urldatarule = Rule(attr, "==", val)
self.ruleset.append(self.urldatarule)
self.process_qstring = True
return

[Brian Jones]

Hi,

I haven't looked thoroughly at this code, but it appears to be exactly
the concept I was looking to implement in loghetti, but I kept putting
it off. Some time this week I hope to get it in place, and perhaps
your code will make lighter work of it :-D

As for your earlier issue, I've reproduced it here, and will also try
to work through that by the end of this week. Thanks very much for
bringing the issue to light! Again, I've not had time yet today to
look into why that would happen, but will do my best to get to it
asap.

Oh! Also - can you please let me know which version of loghetti you're
running? I've just put a new tarball up in the "Downloads" area, so
the latest version is now 0.9 (I seem to remember you saying you were
running straight out of 'trunk' - just checking). Also, assuming you
*are* using trunk, do you know if this query worked at one time, but
has since stopped working?

Thanks again,
brian.

--
Brian K. Jones
Python Magazine http://www.pythonmagazine.com
My Blog http://www.protocolostomy.com

[Mohanaraj Gopala Krishnan]

On Mon, Mar 31, 2008 at 8:15 PM, Brian Jones <bkj...@gmail.com> wrote:
>
> Hi,
>
> I haven't looked thoroughly at this code, but it appears to be exactly
> the concept I was looking to implement in loghetti, but I kept putting
> it off. Some time this week I hope to get it in place, and perhaps
> your code will make lighter work of it :-D

I hope it has been useful.

>
> As for your earlier issue, I've reproduced it here, and will also try
> to work through that by the end of this week. Thanks very much for
> bringing the issue to light! Again, I've not had time yet today to
> look into why that would happen, but will do my best to get to it
> asap.

Thanks !

> Oh! Also - can you please let me know which version of loghetti you're
> running? I'

I am running straight out of the svn trunk.

>Also, assuming you
> *are* using trunk, do you know if this query worked at one time, but
> has since stopped working?

Nope. AFAIK ,this issue has always been there.

I look forward to your additions later Brian.

Cheers.

Mohan