mod_wsgi and pylons, Logging

[PyDevler]

I am currently running pylons using mod_wsgi, as a daemon process
running as a separate user from apache. I am having problems logging
to anything under mod_wsgi. Any special config needed to log under
mod_wsgi. The logging works fine as a separate process, however not
using mod_wsgi. Any Ideas?

Thanks, your feedback is greatly apreciated,

Hatem

[Graham Dumpleton]

On Nov 15, 12:20 pm, PyDevler <hnass...@gmail.com> wrote:
> I am currently running pylons using mod_wsgi, as a daemon process
> running as a separate user from apache. I am having problems logging
> to anything under mod_wsgi. Any special config needed to log under
> mod_wsgi. The logging works fine as a separate process, however not
> using mod_wsgi. Any Ideas?

Logging using what? Where would the log output normally go to when you
are running the application in standalone process?

If the logging system you are using is trying to output to sys.stdout,
then you will need to configure mod_wsgi to allow that, as writing to
sys.stdout by WSGI applications is not portable, so to discourage
people doing that in WSGI applications mod_wsgi doesn't allow you to
do it by default. It should raise a Python exception if you try, but
your logging system may well be catching them and ignoring them. Your
logging system may also be getting confused because of sys.stdout not
actually being the original Python sys.stdout object but a wrapper
around Apache error log.

Presuming that sys.stdout was being used, once enabled the output
would then end up in the Apache error log. Whether it will end up in
the main Apache error log or a virtual host specific error log will
depend on how you have configured Apache. As output is sent to Apache
error log as 'error' level, you need to ensure Apache LogLevel
directive is set with a sensible value so as to ensure that 'error'
level output is being output. By default it should, but if it has been
changed to be more restrictive it wouldn't.

If not using sys.stdout but a file of your own as output for logging,
also be aware that you can't use relative path names for files as when
using Apache there isn't really a guarantee about what the working
directory of your application will be.

That all said, can you please provide a minimal code example which
shows how you are logging stuff, preferably something that could be
run to try and duplicate the problem.

For further reading on restrictions on sys.stdout under mod_wsgi and
other material about logging with mod_wsgi, plus issues related to
user permissions and working directories of applications, ensure you
have read:

http://code.google.com/p/modwsgi/wiki/ApplicationIssues
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques

BTW, the mod_wsgi mailing list/group is at:

http://groups.google.com/group/modwsgi

Graham

[PyDevler]

> If the logging system you are using is trying to output to sys.stdout

...
No, I am not logging to stderr or stdout

> If not using sys.stdout but a file of your own as output for logging,
> also be aware that you can't use relative path names for files as when
> using Apache there isn't really a guarantee about what the working
> directory of your application will be.

I am also using absolute paths.

I was initially logging using the Syslog Handler, however when I
switched over to mod_wsgi this did not work. How I performed this, was
using the standard pylons config, which uses the python config for
logging. I created a syslog handler with its respective formatter and
made it the default output for the root logger.When this didn't work
in
mod_wsgi I thought it mght be specific to syslog to I attempted t use
a
FileHandler. Which ended up having the same effect, where it works in
standalone mode and not under mod_wsgi. After trying these two
versions
and failed I wanted to see if I can acheive any logging using the
documented wsgi.errors handlers, however not able to see any logging
through my calls to log anywhere. All I see is the startup, any
exceptions and shutdown in the apache error.log. I have not turned out
stdout in mod_wsgi and is not willing to do so.

I did some invistigating and realized, it is not loading my config
specified in the .ini file. It works fine in standalone mode, but it
seems to ignore the logging part under mod_wsgi. Any ideas how the
logging can be enabled in pylons under mod_wsgi.

Thanks,

Hatem

[PyDevler]

On Nov 15, 10:51 pm, PyDevler <hnass...@gmail.com> wrote:
> How I performed this, was
> using the standard pylons config, which uses the python config for

I have attached my config, maybe there is something missing here.

Excerpt from production.ini
---------------------------------------

# Logging configuration
[loggers]
keys = root, helloworld

[handlers]
keys = console, syslog, logfile

[formatters]
keys = generic, syslog, logfile

[logger_root]
level = INFO
handlers = console

[logger_helloworld]
level = DEBUG
handlers =
qualname = course

[handler_syslog]
class=handlers.SysLogHandler
args=('/dev/log', handlers.SysLogHandler.LOG_USER)
level = NOTSET
formatter=syslog

[handler_logfile]
class=handlers.RotatingFileHandler
main_dir=/var/local/xyz
args=('%(main_dir)s/log/math0010.log', 'a', 1048576, 0)
level=INFO
formatter=logfile

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = logfile

[formatter_generic]
format = %(asctime)s,%(msecs)03d %(levelname)-5.5s [%(name)s] %
(message)s
datefmt = %H:%M:%S

[formatter_syslog]
format = [%(clientip)s]:%(module)s[%(process)d]:%(message)s

[formatter_logfile]
format = %(asctime)s [%(clientip)s]:%(module)s[%(process)d]:%
(message)s
datefmt = %H:%M:%S

--

Hatem

[Graham Dumpleton]

What does the WSGI script file for mod_wsgi that you are using
contain?

Graham

[PyDevler]

> What does the WSGI script file for mod_wsgi that you are using
> contain?

It contains the one line that was documented.

[code]
import os, site; site.addsitedir(eggs_dir)
from paste.deploy import loadapp

application = loadapp('config:Path.../production.ini' )
[/code]

[PyDevler]

When I previously mentioned invistigating. I managed to get it to log
by manually adding a handler from within my controller-xyz.py. However
it is refusing to load my config. It is for some reason refusing to
use the Formatter line, that I adjust from within controller-xyz.py.
However, it changes the log-level which I also set in the same command
I pass the formatter string. I am unable to explain what pylons is
doing under mod-wsgi.

[Graham Dumpleton]

Sorry for taking so long to get back to this, got diverted on more
important things.

In the documentation for Pylons logging it says:

"""paster, when loading an application via the paster serve, shell or
setup-app commands, calls the logging.fileConfig function on that
specified ini file if it contains a 'loggers' entry.
logging.fileConfig reads the logging configuration from a ConfigParser
file."""

This would suggest using 'paster' it does special stuff which wouldn't
be getting done if using mod_python, mod_wsgi or any other hosting
solution besides 'paster'.

The documentation is a bit deceiving here as took that to mean
'fileConfig' with 'logging' module, but on Python 2.3 at least, no
such function exists. Turns out what you need in WSGI script file is:

import os, sys
__here__ = os.path.dirname(__file__)
__parent__ = os.path.dirname(__here__)

sys.path.append(__parent__)

from paste.script.util.logging_config import fileConfig
fileConfig('%s/development.ini' % __parent__)

from paste.deploy import loadapp

application = loadapp('config:%s/development.ini' % __parent__)

Ie., fileConfig comes from 'paste.script_util.logging_config'.

If that function is called with Pylons ini file then logging if then
output.

Graham

[PyDevler]

> In the documentation for Pylons logging it says:
>
> """paster, when loading an application via the paster serve, shell or
> setup-app commands, calls the logging.fileConfig function on that
> specified ini file if it contains a 'loggers' entry.
> logging.fileConfig reads the logging configuration from a ConfigParser
> file."""

I did read this in passing, However did not think it doesn't do this
through loadapp
...

> from paste.script.util.logging_config import fileConfig
> fileConfig('%s/development.ini' % __parent__)

...
Works perfect, Thanks Graham.

[prog...@gmail.com]

Hey Graham,

Actually, I thought I was having the same issue since I was getting no
logging at all from Pylons when using mod_wsgi. However, after trying
this and it not working, it looks like it has to do with using
mod_wsgi in daemon mode. (No, not on FreeBSD this time). I can seem to
log from the wsgi script, but once in Pylons it just doesn't spit
anything out unless running in embedded mode.

I'm simply using:

WSGIApplicationGroup %{GLOBAL}
WSGIDaemonProcess localdev-site user=me group=me
WSGIProcessGroup localdev-site
WSGIScriptAlias / /path/to/script.wsgi

And I get nothing.

WSGIApplicationGroup %{GLOBAL}
#WSGIDaemonProcess localdev-site user=me group=me
#WSGIProcessGroup localdev-site
WSGIScriptAlias / /path/to/script.wsgi

Will get me exception traces (the main reason I'm doing this) and
general logging, without any special setup, just using the default
pylons logging config and without the hack mentioned in this thread.

On a dual proc, dual core amd64 setup btw

-jeff

On Nov 17, 8:44 pm, Graham Dumpleton <Graham.Dumple...@gmail.com>
wrote:

[Graham Dumpleton]

Which Apache error log file are you looking in? Do you have
VirtualHost specific CusomLog defined?

When run in mod_wsgi daemon mode, the sys.stderr output will be
redirected to a VirtualHost specific error log file if
WSGIDaemonProcess was defined in the context of the VirtualHost.

When in mod_wsgi embedded mode, the sys.stderr output will always go
to the main Apache error log file even if a VirtualHost specific error
log file has been defined.

Graham

[Jeff Lindsay]

That doesn't seem to be the case. We're using this inside our VirtualHost:

ErrorLog /path/to/error_log
CustomLog /path/to/access_log combined

We're looking at the error_log file for this vhost and in embedded
mode we *do* see Pylons errors when raised but in daemon mode we do
not, which seems the opposite of what you say... except that we don't
get the errors in the main apache log either when in daemon mode.

This is Gentoo btw.

-jeff

--
Jeff Lindsay
http://devjavu.com -- Free Trac and Subversion
http://blogrium.com -- A blog about things

[Graham Dumpleton]

I am only referring to anything output directly via sys.stderr.

Any messages output via wsgi.errors passed in the WSGI environment,
which is how most WSGI application would tend to log, would go to the
log file associated with the context the request is handled in. If you
have a CustomLog in a VirtualHost container, then that is where those
messages would go. Any error messages generated by mod_wsgi, including
error tracebacks, which correspond to a specific request will
similarly be output to the error log file associated with the
VirtualHost if CustomLog is used.

The case which differs is when using sys.stderr directly, or where
using the logging module since it defaults to using sys.stderr also.
Because sys.stderr is global to the interpreter it isn't associated
with a specific request and so cannot normally be associated with the
custom log of the virtual host. This is the case because technically
it is possible for requests against different virtual hosts to be
directed to the same interpreter instance.

Daemon mode, where WSGIDaemonProcess is used inside of a VirtualHost,
is a special case because in that scenario, that the directive appears
inside of the VirtualHost means that only requests bound for that
virtual host could be sent to that daemon process. This means that
mod_wsgi can associat sys.stderr for that daemon process with the
error log for that VirtualHost rather than it going to the global
Apache error log.

Graham

On Dec 20, 2:30 pm, "Jeff Lindsay" <progr...@gmail.com> wrote:
> That doesn't seem to be the case. We're using this inside our VirtualHost:
>
> ErrorLog /path/to/error_log
> CustomLog /path/to/access_log combined
>
> We're looking at the error_log file for this vhost and in embedded
> mode we *do* see Pylons errors when raised but in daemon mode we do
> not, which seems the opposite of what you say... except that we don't
> get the errors in the main apache log either when in daemon mode.
>
> This is Gentoo btw.
>
> -jeff
>

> Jeff Lindsayhttp://devjavu.com -- Free Trac and Subversionhttp://blogrium.com-- A blog about things

[Jeff Lindsay]

Ok, that makes sense now but only concerns me more because logging in
daemon mode is not working, using wsgi.errors or sys.stderr. However,
I'm a bit confused because CustomLog is used for access logs, which
does work btw if I wasn't clear. It's only error logs that don't work.

-jeff

--

[Graham Dumpleton]

Sorry, got myself confused, where I said CustomLog I meant ErrorLog.

Can you summarise what works and doesn't for basic use case inside a
hello world application. Ie., what gets output to Apache error log
(and which one) for the following:

import sys

def application(environ, start_response):
status = '200 OK'
output = 'Hello World!'

print >> sys.stderr, "sys.stderr"
print >> environ["wsgi.errors"], "wsgi.errors"

response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)

return [output]

This will tell me if we are talking about an underlying mod_wsgi
issue, or something specific to how Pylons does logging.

Note that LogLevel directive in Apache must be at least 'error' for
either of these to be displayed. Generally the default is 'warn' which
should capture them.

If both of those works in both embedded and daemon mode, but a Pylons
specific application is not generating anything to error log when you
expect it, can you give a small example which would demonstrate the
problem.

Thanks.

Graham

On Dec 20, 5:41 pm, "Jeff Lindsay" <progr...@gmail.com> wrote:
> Ok, that makes sense now but only concerns me more because logging in
> daemon mode is not working, using wsgi.errors or sys.stderr. However,
> I'm a bit confused because CustomLog is used for access logs, which
> does work btw if I wasn't clear. It's only error logs that don't work.
>
> -jeff
>

> > > Jeff Lindsayhttp://devjavu.com-- Free Trac and Subversionhttp://blogrium.com--A blog about things