Distributed pymon work begins
Duncan M. McGreggor
I've updated the site with a news item about it. You can read it here:
http://pymon.sourceforge.net/
The work is in branch and will remain there until the componentization
code is completed, at which point these changes will all be merged back
into trunk.
I expect there will also be additional development on the web front end
as a result of presenting the peering data. This will likely include
more MochiKit. The page refresh will go away to be replaced by regular
XMLHttpRequests to JSON state data.
More later...
d
Duncan M. McGreggor
On Apr 30, 2006, at 1:08 AM, Duncan M. McGreggor wrote:
> Hey everyone, I started work on the distributed pymon code tonight.
> I've updated the site with a news item about it. You can read it here:
> http://pymon.sourceforge.net/
Some more thinking "out loud":
When performing unit tests tests in pymon, individual components will
need to be created. Here are some of the basics:
* a monitor
* a client
* a notification rule
* a notification mechanism (email, log, asserts or exceptions, etc.)
* a service to monitor
* configuration for the unit test service
There are a couple obvious examples for adaptation here, specifically
with "notification system" and "configuration":
Notifications (for testing purposes) should adapt to raise and asserts
so that alerts are seamlessly integrated into unit tests. Likewise, for
application extendibility, they should adapt to various modes of
communication like email (currently in place), jabber, irc, etc.
Writing adaptors for notifications should be as simple as calling
implements(INotifiable) in your adaptor and writing any additional code
necessary to perform the adaptation.
Likewise for configuration. During unit tests, we will want to setup
minimal examples for configuring services, etc., and configuring a
service with something like a simple dictionary should entail something
along the lines of calling implements(IConfigurable) in your adaptor
and writing any additional code necessary to perform the adaptation.
The underlying machinery that makes writing adaptors easy in pymon is
my job right now :-) There should be minimal work for developers to
write new adaptors. This will improve plugin development processes and
release schedules as well as encourage community contributions. The
only learning curve that should be required for this should entail an
understanding of interfaces, their use in adaptation, and an
understanding of adaptation itself.
This means I really have to sit down and consider what *exactly* and
*minimally* defines a monitor, a service, a client, etc. I want each to
have ONLY the methods and attributes they absolutely must have. I want
to put as much of the monitor development burden as possible myself,
writing it into pymon core, and then exposing only what is necessary
and useful for developers.
I am even considering another attempt at using plugins, where there
would be a single file or directory for each type of monitor (in
/usr/local/pymon/plugins) and a single configuration file to edit
(/usr/local/pymon/etc/pymon.conf). This would mean that each plugin
would needs to contain something along the following, probably as a
python package in the plugin directory:
* configuration schema (these is where you define your config
directives in XML)
* configuration file (this is the Apache-style user-editable config
file)
* interface(s) (including IYourMonitor which will have to extend
pymon's IMonitor)
* a monitor class implementing you monitor interface
* processing of service status
* any custom notification policies
* any needed adaptors
I would put together an ultra-minimal example of this that would
definitively show the bare minimum necessary to write a plugin. I would
also provide an full-override example that made every possible
customization. Of course, the default monitors that come with pymon
would be included, and these would be examples for middle ground.
I've actually had problems with integrating multiple ZConfig schema.xml
files and their respective .conf files. I will explore this and see if
there is an easy resolution. A worst-case scenario would mean plugin
developers have to edit the base pymon schema.xml pymon.conf files
themselves.
In any case, even while working on other project, pymon continues to
get lots of background processing from me. As I said last night, I'm
actually writing code, so keep those fingers crossed.
Later,
d
Duncan M. McGreggor
On Apr 30, 2006, at 2:30 PM, Duncan M. McGreggor wrote:
> When performing unit tests tests in pymon, individual components will
> need to be created. Here are some of the basics:
> * a monitor
> * a client
> * a notification rule
> * a notification mechanism (email, log, asserts or exceptions, etc.)
> * a service to monitor
> * configuration for the unit test service
I did some more work on this, and as part of that work I did a HUGE
amount of code clean up. Maybe not huge by volume, but certainly a huge
relief of removing the dead albatross from around my neck.
Most importantly: pymon.config.cfg is no longer imported in library
code. It is imported at the application-level only, right were it
belongs: in the pymon.tac file. That being said, there are still a few
places were it is imported, but these are places like the web ui (for
getting the location of static files), pymon.application (for getting
the location of where backups need to be written), and utils (for
getting the configured log level). These are all justifiably
application-level needs, and so I don't feel the least bit bad about
importing them there.
I can't even tell you how much better this makes me feel.
The other big clean up was pymon.monitors.MonitorMixin.__init__(). It
was a mess, and now it's lost about 15 lbs and is working out regularly
at the gym. Next up is __call__() in that same class. __call__() needs
more than a face lift, though: I'm going to be amputating... there's no
way around it. In order to do that, though, I'll need to update
pymon.application.MonitorState. That requires some thought.
Essentially, MonitorState is like a table schema (or more accurately, a
row in a table), so I'll need to think about how that might be
used/adapted in the future, and then shoot for a general solution.
With MonitorMixin cleaned up, I'll be ready to get rid of it ;-)
Seriously, I am leaning heavily towards making the monitors adapt a
Monitor class (which is what MonitorMixin would become). I haven't
worked all that out yet, and the fact that there are so many different
client factories that would be adapting it has been tricky in the
prototyping stage. We'll see.
[one hour later]
Okay, I moved the horrible crap out of __call__() now, too and
consolidated it to be non-redundant and full of goodness.
I'm also starting to get good ideas for unit tests, so that should
begin sometime soon.
But now, it's time to unit test by bed.
d