Logging in web2py
Yarin
I've posted an updated slice with an in-depth treatment of logging in
web2py.
http://web2pyslices.com/main/slices/take_slice/91
Based on our earlier discussion
http://groups.google.com/group/web2py/browse_thread/thread/d3b534113a904cd7/4d3f9f10d402210d?lnk=gst&q=logging#4d3f9f10d402210d
The slice addresses the issues of application-level logging, once-only
configuration, and simpler syntax. It offers an approach that fully
leverages Python's native logging capabilities, allows for full
flexibility in configuration, and doesn't interfere with existing
logging implementations.
Some notes:
-I decided against creating a separate contrib module for the code as
it would impede flexibility and obscure the simplicity of solution.
If included in the framework, I think it should be as model code.
-I abandoned the cache approach to creating singleton loggers, and
just inspect handlers instead. (Logger caching was causing problems
on GAE)
Let me know what you think. Thanks Massimo, Iceberg...
Jonathan Lundell
>
> I've posted an updated slice with an in-depth treatment of logging in
> web2py.
> http://web2pyslices.com/main/slices/take_slice/91
I've got a patch pending that adds a (standard format) logging configuration file, and converts all of web2py's logging calls to use it.
It has a logger named "web2py.app", so you can getLogger("web2py.app.myapp") and log calls will propagate to that logger (or you can create a new "web2py.app.myapp" logger definition in the config file).
Initialization happens when web2py loads, so you don't have to test for configuration.
Yarin
On Aug 16, 5:03 pm, Jonathan Lundell <jlund...@pobox.com> wrote:
> On Aug 16, 2010, at 1:31 PM, Yarin wrote:
>
>
>
> > I've posted an updated slice with an in-depth treatment of logging in
> > web2py.
> >http://web2pyslices.com/main/slices/take_slice/91
>
> I've got a patch pending that adds a (standard format) logging configuration file, and converts all of web2py's logging calls to use it.
>
> It has a logger named "web2py.app", so you can getLogger("web2py.app.myapp") and log calls will propagate to that logger (or you can create a new "web2py.app.myapp" logger definition in the config file).
>
> Initialization happens when web2py loads, so you don't have to test for configuration.
>
>
>
>
>
> > Based on our earlier discussion
Jonathan Lundell
> @Jonathan- I'd like to see what you've got- keep me updated
The patch is here, if anyone would like to review it.
http://web.me.com/jlundell/filechute/logging.zip
Most of the changes are to use a specific logger instead of making a generic logging call. The only interesting code is in main.py, along with logging.example.conf.
Nothing much to it: just loading a configuration file, and using specific loggers.
Iceberg
And I wish GAEHandler and get_configured_logger() could go to web2py trunk, and the scaffold's models/model.py contains just one extra line:
logger = get_configured_logger(folder = 'static', maxBytes=10240000, backupCount=5)
so it will be even easier to use.
Best regards,
Iceberg, 2010-Aug-18, 02:20(AM), Wed
----------------------- Original Message -----------------------
From: Yarin <ykes...@gmail.com>
To: web2py-users <web...@googlegroups.com>
Cc: ice...@21cn.com
Date: Mon, 16 Aug 2010 13:31:16 -0700 (PDT)
Subject: Logging in web2py
-------------------
Yarin
but beyond that I'm not clear on what your patch is meant to solve-
What's your reason for introducing named loggers and sub-loggers at
the site level? Couldn't the root logger just handle all this?
My intention was to use named loggers to allow for logging control at
the application level, but for site level logging I see no reason to
stray from root.
@Iceberg- Yeah, I played around with abstracting
get_configured_logger() into a contrib module, but it got messy
because
a) The request object we use to get application name and log file path
isn't available outside the mvc classes, and would have to be passed
in.
b) More importantly, it killed the flexibility of being able to do
custom configuration, use custom or multiple handlers, etc.
The GAEHandler and others I include in the link are useful, but
they're not web2py specific so Massimo might not want to clutter the
framework with them.
On Aug 17, 2:32 pm, Iceberg <iceb...@21cn.com> wrote:
> Nice work, Yarin! You use a more decent way, logging.getLogger(request.application), to achive app-wide logger. It is better than my cache.ram() trick. Oh, I was so blind, man. :-)
>
> And I wish GAEHandler and get_configured_logger() could go to web2py trunk, and the scaffold's models/model.py contains just one extra line:
>
> logger = get_configured_logger(folder = 'static', maxBytes=10240000, backupCount=5)
>
> so it will be even easier to use.
>
> Best regards,
> Iceberg, 2010-Aug-18, 02:20(AM), Wed
>
>
>
> ----------------------- Original Message -----------------------
> To: web2py-users <web...@googlegroups.com>
>
> Cc: iceb...@21cn.com
> Subject: Logging in web2py
> -------------------
>
> > I've posted an updated slice with an in-depth treatment of logging in
> > web2py.
> >http://web2pyslices.com/main/slices/take_slice/91
>
> > Based on our earlier discussion
Jonathan Lundell
> @Jonathan- Like your use of config files for default configurations,
> but beyond that I'm not clear on what your patch is meant to solve-
> What's your reason for introducing named loggers and sub-loggers at
> the site level? Couldn't the root logger just handle all this?
The main purpose of the configuration file is to allow a site to configure the logging destination. The current logging all goes to stdout (which in the case of Fedora or RHEL, at least, is /dev/null for system-started daemons). The configuration file lets us log to files, rotating files, or syslog, local or remote.
The reason for named loggers is twofold. One is trivial: to identify the source of log messages. The other is not so trivial, but mainly for development: to let us have different log levels for different loggers. If I'm debugging URL rewrite, I can lower the level of the web2py.rewrite logger to (say) debug, without enabling all the other debug/info loggers in the system.
Note that having named loggers is a pretty trivial addition, once we have a config file; it's not central, but it's a distinct convenience.
Note also that we already have named loggers (eg Rocket and its kin), but no way to configure them.
Yarin
a stack trace
> The other ... to let us have different log levels for different loggers.
But it looks like you're introducing several named loggers only to log
several messages at startup. For example, your "web2py.sql" logger's
only function is to log a single driver exception. As far as I can
tell, only the "web2py.rewrite" logger does any continuous work. And
I assume the Rocket logger does a lot of work. But beyond that I
can't see the justification in forcing the framework to create and
configure a number of different loggers that will only be used once,
if at all?
Jonathan Lundell
>> The reason for named loggers is twofold. One is trivial: to identify the source of log messages.
> Logging already tracks module, function, lineno... we can even insert
> a stack trace
>> The other ... to let us have different log levels for different loggers.
> But it looks like you're introducing several named loggers only to log
> several messages at startup. For example, your "web2py.sql" logger's
> only function is to log a single driver exception. As far as I can
> tell, only the "web2py.rewrite" logger does any continuous work. And
> I assume the Rocket logger does a lot of work. But beyond that I
> can't see the justification in forcing the framework to create and
> configure a number of different loggers that will only be used once,
> if at all?
Mostly for purposes of example. The web2py.foo loggers needn't appear in the config file; they can all use the web2py logger (I keep writing 'blogger').
The Rocket module is (I think) independent of web2py; if not, I'd have changed its logger (or asked Tim to do so) to something like 'web2py.rocket.*' (maybe we could pass a logger base name in to it).
Identifying the source of the log messages through the logger name is useful for more than human consumption. log levels is one use; another is choice of handler.
I'd have no objection to consolidating the startup-only loggers into 'web2py.startup', though I think that a noisy logger (even if noisy only at startup) ought to have its own name.
