progress!

[duncan.m...@gmail.com]

Significant progress has been made made in the last few weeks of pymon
development.

Here are the things on which I am currently focusing:
* Scalability of configuration (supporting 100s to 100s of config
files)
* Scalability of monitoring (checking 1000s to 100,000s of hosts)
* Plugin framework
* Workflow
* Monitor cleanup
* Migration away from pickled data to database support
* Administrator/User API
* pymon "Monitoring OS" (the pymon shell)

In order, here's the status of each of those:

Scaling config: Done!
Scaling monitoring: not started
Plugin framework: Done!
Workflow:
* this was started ages ago and code is in-place, but needs more
testing
* a plan for migration from rules to workflow is needed
Monitor cleanup:
* this work is underway and has been waiting on the plugin framework
to be completed
* part of this effort involves the componentization of pymon; we've
done this for State objects so far, but more is planned
Databases: not started
API: not started (will include at least XML-RPC and possibly a RESTful
API as well)
pymon MOS: started.

What's next? I want to finish monitoring cleanup and workflow, at
which point the database work and testing for scalability can move
ahead unhindered. With those in place, it will make sense to start
exposing an API for administrators and users. The pymon MOS will be
further developed once the API is in place.

To keep track of the latest, be sure to check out the following at
cia.vc:
web: http://cia.vc/stats/project/pymon
RSS: http://cia.vc/stats/project/pymon/.rss

[duncan.m...@gmail.com]

On Dec 5, 12:09 am, duncan.mcgreg...@gmail.com wrote:
> Monitor cleanup:
> * this work is underway and has been waiting on the plugin framework
> to be completed
> * part of this effort involves the componentization of pymon; we've
> done this for State objects so far, but more is planned
>

> What's next? I want to finish monitoring cleanup

Background

* a pymon plugin is composed of a client, a monitor, and a
configuration
* a pymon monitor is responsible for actually connecting to a network
service and getting the data back
* a pymon client is responsible for parsing the returned data,
processing it, and updating state

Foreground

pymon Monitor Issues:
* the current monitor code in (for example) the ping plugin is hairy
with legacy and needs to be cleaned/reamed out
* even more so with the BaseMonitor class: way, way too many class
attributes (we need to save memory, not CPU, so make local variables)

pymon Client Issues:
* the plugin client code currently does too much rules stuff; all it
should have to do is initiate a rules check
* in fact, we probably want to do something like
IWorkflow.processTransition(self), where processTransition would
1) check thresholds
2) dispatch the appropriate workflow transition methods based on
the threshold check result, and
3) update the state data (current state and state history)
* ClientMixin needs to be rewritten along these lines, too; a lot of
it can probably go in workflow code

d