mod_wsgi and .pyc files

[Matt Craighead]

Hey all,

 

Porting from mod_python to mod_wsgi.  I put the following in my httpd.conf: 

WSGIScriptAlias /cascade "C:\Program Files\Conifer Systems\Cascade\csc_manager_wsgi.pyc"

 

<Directory "C:\Program Files\Conifer Systems\Cascade">

Order allow,deny

Allow from all

</Directory>

I got the following error in my Apache log:

 

[Mon Dec 01 12:27:08 2008] [error] [client 127.0.0.1]  SyntaxError: Non-ASCII character '\\xb3' in file C:/Program Files/Conifer Systems/Cascade/csc_manager_wsgi.pyc on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details (C:/Program Files/Conifer Systems/Cascade/csc_manager_wsgi.pyc, line 1)

 

 

Works fine if I give it the .py file, but I prefer to ship my software as .pyc files rather than .py files.  Is there a way to tell mod_wsgi to load a precompiled .pyc script?

 

Thanks,

 

--

Matt Craighead

Founder/CEO, Conifer Systems LLC

http://www.conifersystems.com

512-772-1834

 

[Clodoaldo Pinto Neto]

2008/12/1 Matt Craighead <matt.cr...@conifersystems.com>:

> Hey all,
>
> Porting from mod_python to mod_wsgi. I put the following in my httpd.conf:
>
> WSGIScriptAlias /cascade "C:\Program Files\Conifer
> Systems\Cascade\csc_manager_wsgi.pyc"
>
> <Directory "C:\Program Files\Conifer Systems\Cascade">
> Order allow,deny
> Allow from all
> </Directory>
>
> I got the following error in my Apache log:
>
> [Mon Dec 01 12:27:08 2008] [error] [client 127.0.0.1] SyntaxError:
> Non-ASCII character '\\xb3' in file C:/Program Files/Conifer
> Systems/Cascade/csc_manager_wsgi.pyc on line 1, but no encoding declared;
> see http://www.python.org/peps/pep-0263.html for details (C:/Program
> Files/Conifer Systems/Cascade/csc_manager_wsgi.pyc, line 1)

You need to tell the interpreter what is the source file encoding. If
it is utf-8 then put a line like this after the line with the
interpreter path:

# -*- coding: utf-8 -*-

Since you are in Windows the encoding will be something like cp1252

Regards, Clodoaldo

[Matt Craighead]

Clodoaldo,

 

This is a precompiled binary .pyc file -- I don't believe character encodings are meaningful in this context.

512-772-1834

[Clodoaldo Pinto Neto]

2008/12/1 Matt Craighead <matt.cr...@conifersystems.com>:

> Clodoaldo,
>
> This is a precompiled binary .pyc file -- I don't believe character
> encodings are meaningful in this context.

Yes, if the .py works then i guess the problem is that he is expecting
a text file.

Clodoaldo

[Brett Hoerner]

On Mon, Dec 1, 2008 at 2:17 PM, Matt Craighead
<matt.cr...@conifersystems.com> wrote:
> Works fine if I give it the .py file, but I prefer to ship my software as
> .pyc files rather than .py files. Is there a way to tell mod_wsgi to load a
> precompiled .pyc script?

The docs say,

"In all ways that the WSGIScriptAlias can be used, the target script
is not required to have any specific extension type and in particular
it is not necessary to use a '.py' extension just because it contains
Python code. Because the target script is not treated exactly like a
traditional Python module, if an extension is used, it is recommended
that '.wsgi' be used rather than '.py'."

Especially note the last sentence. The wsgi script isn't a normal
Python file, really. Is there any reason you want it to be a pyc?
(You're aware that pyc files can be easily decompiled - right?)

Brett

[Matt Craighead]

Brett,

 

Yes, I'm aware that pyc files can be decompiled -- but this does at least strip comments from the code, slightly speed up loading, and a few other minor things.

 

I guess the simplest workaround would be to create a 1-line file "csc_manager.wsgi" as follows?

 

from csc_manager_wsgi import application

512-772-1834

[Brett Hoerner]

On Mon, Dec 1, 2008 at 3:31 PM, Matt Craighead
<matt.cr...@conifersystems.com> wrote:
> Yes, I'm aware that pyc files can be decompiled -- but this does at least
> strip comments from the code, slightly speed up loading, and a few other
> minor things.

Do you really have many comments or much code in your wsgi script
itself? I guess I can understand the desire for pyc files elsewhere,
but does this one file being "plain source" really hold you back? I
suppose all of my wsgi scripts have been pretty basic.

> I guess the simplest workaround would be to create a 1-line file
> "csc_manager.wsgi" as follows?
>
> from csc_manager_wsgi import application

That'd probably work, certainly worth a try.

Brett

[Robert Coup]

On Tue, Dec 2, 2008 at 10:31 AM, Matt Craighead <matt.cr...@conifersystems.com> wrote:

Yes, I'm aware that pyc files can be decompiled -- but this does at least strip comments from the code, slightly speed up loading, and a few other minor things.

If you/anyone didn't already know, mod_wsgi can compile .py -> .pyc/.pyo files at runtime, via the WSGIPythonOptimize directive:

http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIPythonOptimize

Rob :)

[Graham Dumpleton]

2008/12/2 Matt Craighead <matt.cr...@conifersystems.com>:

> Brett,
>
> Yes, I'm aware that pyc files can be decompiled -- but this does at least
> strip comments from the code, slightly speed up loading, and a few other
> minor things.

Any speedup in load time isn't going to help much since the code is
only loaded once at process start and then persists for life of
process. This isn't CGI, loading is not done on every request.

Thus, if load time is the only reason you are using .pyc files, I
wouldn't be bothering. You will get more from simply ensuring that
application code is preloaded at process start rather than mod_wsgi
default of lazy load on first request.

Doing preloading helps to ensure that you don't see slight pauses on
process reloads. If using daemon mode, you would though probably also
want to run 2 or more processes for this to be completely effective.

To ensure preloading occurs, use WSGIImportScript directive to point
at the WSGI script file.

> I guess the simplest workaround would be to create a 1-line file
> "csc_manager.wsgi" as follows?
>
> from csc_manager_wsgi import application

Yes, that will work. Just ensure that your real .py/.pyc files are not
in the same directory but are instead somewhere outside of directories
exposed via Apache, with Python module search path set to get them
from this alternate location. You don't want them in Apache document
directories because if you don't configure Apache right, you might
create situation where they can be downloaded by external people.

Also suggest reading:

http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode

It talks more about how WSGI script file is treated in special ways.

Overall I would counsel against premature optimisations of the sort
you are trying to do, especially if you don't have evidence or
supporting documentation that suggests it is worthwhile.

Alone using .pyc files, also warn against using WSGIPythonOptimize and
.pyo files. The justification isn't really there for using it.

Graham