Re: [modwsgi] Re: mod_wsgi: Possible enhancement: making log level adjustable per message

[Graham Dumpleton]

I have not been keen on providing a way of logging Apache error messages at specific levels, because as soon as some one uses it in conjunction with the Python logging module, it will not yield the results I expect many would think they should see.

Specifically, if there was an API and a logging module handler for emitting records to the Apache error log at the levels attributed to them by the logging module, but the Apache LogLevel is set lower than what is being passed through because of logging module configuration, then you wouldn't see them in the Apache error log. This mismatch would be confusing and where you don't have access to Apache configuration wouldn't be something you could change.

There are also some inherit limitations in the Apache error log API. One is that messages if over about 8Kb will be truncated. Thus you can result in loss of information that you may want.

So it all sounds like it may be a reasonable idea, but not convinced it is the best idea. Still tend to think that web application logging should be kept separate from Apache error log.

If you are after ways of logging events and exceptions, you may be better off using the free version of Sentry, the paid hosted version of Sentry, or if also wanting application performance monitoring, a system such as New Relic.

Graham

On 20 November 2012 23:22, elwynd <elw...@googlemail.com> wrote:

[Moved to Google Group from private email]

Thanks for the response Graham.

Yes, the standalone version makes use of Python logging.  It seemed that it was adding an
additional and potentially unnecessary layer to use Python logging on top of the existing
Apache multilevel logging given that Apache logging has the leveling needed.

Is there anything inherently difficult or undesirable to making the Apache logging level
control accessible that I haven't spotted?  Threading comes to mind if the level setting
is not in the actual log write method - I am not sure whether the wsgi.errors logger is
per-thread or shared in a multiple thread per-process situation.

Regards,
Elwyn

 

Are you using:

http://docs.python.org/2/library/logging.html

It provides log levels and the output of what does get through can still be
sent to stderr and captured in the Apache error log.

The logging module should be used in preference to using wsgi.errors.

The only cosmetic issue is that in the Apache error log everything shows as
at error level that goes to stdout/stderr. This is something that can't be
changed.

To see the proper log level you can still set the logging module logger
message format to include the logging module log level. It will end up
being part of the logged message. Thus the entries in the Apache error log
might end up looking like:

[Fri Nov 16 05:02:40 2012] [error] 2012-11-16 05:02:40,095 (5010/Dummy-6)
newrelic.core.agent INFO - New Relic Python Agent Shutdown
[Fri Nov 16 05:02:40 2012] [error] 2012-11-16 05:02:40,119
(5010/NR-Harvest-Thread) newrelic.core.agent DEBUG - Commencing harvest of
all application data and forcing a shutdown at the same time.

In this case the log format was still including its own date/time stamp,but
you could drop that to make it more compact.

BTW, can you use the mod_wsgi mailing list in future.

http://code.google.com/p/modwsgi/wiki/WhereToGetHelp?tm=6#Asking_Your_Questions

Thanks.

Graham





On 20 November 2012 08:06, Elwyn Davies <davi...@scss.tcd.ie> wrote:

> Hi.
>
> I am just starting out with mod_wsgi.
>
> I have an existing application which is currently running using a basic
> BaseHTTPServer/BaseHTTPHandler with a Python logger in a standalone
> program.  I am now moving the code to run as a mod_wsgi module.
>
> The code contains extensive logging both for debugging and monitoring
> using multiple logging levels to tune the amount of output according to my
> confidence in the code.  I would like to be able to set the logging level
> for each logged message.  As I understand mod_wsgi the 'wsgi.errors'
> logging object has a fixed logging level, so that whilst I can control
> whether any messages from the mod_wsgi application are logged by Apache
> using the log level setting in Apache, this is rather a blunt instrument.
>
> Having had a quick look at the code, it looks as if it would be very easy
> to add a getter/setter pair to allow adjustment of the 'level' value in the
> log structure.
>
> Would this be an effective and sane enhancement?  I have some code that I
> am currently trying out - not exactly a difficult mod!
>
> Regards,
> Elwyn Davies
> Folly Consulting/Trinity College Dublin
>
>

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To view this discussion on the web visit https://groups.google.com/d/msg/modwsgi/-/yKcQ7J7HJ6kJ.
To post to this group, send email to mod...@googlegroups.com.
To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.

[elwynd]

Hi.

One point for the record...

Using the Python Logger module works OK except that you have to be careful to shut the StreamHandler down very thoroughly before returning from your app function.  It appeared from the Logger code that calling flush and close on the StreamHandler would stop it using the wsgi.error stream after it was shut down but this didn't seem to work as expected.  The Logger module installs an 'atexit' function that flushes the stream and hence provokes an exception in mod_wsgi.  I worked around this by explicitly setting the StreamHandler stream to None after flushing the stream before exiting.

Regards,
Elwyn