How do I get the request header from mod_wsgi?

[Carl Nobile]

I am thinking about using mod_wsgi as a possible service
implementation is an SOA environment. I also want to use a RESTful
interface as opposed to SOAP. However, if my understanding of RFC2616
is correct, many of the request methods may need to send along
specific request headers such as Authorization: credentials when using
the OPTION request method.

The issue I am having is how to get the request headers from mod_wsgi?
They are obviously not in the "wsgi.input" stream, so where are they?

Carl

[Brian Smith]

> The issue I am having is how to get the request headers from mod_wsgi?
> They are obviously not in the "wsgi.input" stream, so where are they?

All WSGI implementations store the request headers in the environ
dictionary passed to the applicatoin. See

http://www.python.org/dev/peps/pep-0333/#environ-variables:

Variables corresponding to the client-supplied HTTP request headers
(i.e., variables whose names begin with "HTTP_"). The presence or
absence of these variables should correspond with the presence or
absence of the appropriate HTTP header in the request.

For example, value of the Accept-Language header is found using
environ['HTTP_ACCEPT_LANGUAGE'].

- Brian

[Carl Nobile]

Duh, yup and I knew this, matter-of-fact I read it a few days ago, but
I guess it didn't sink in.
Sorry for the dumb question.

Carl

[Graham Dumpleton]

Do note though that mod_wsgi doesn't pass through HTTP_AUTHORIZATION
variable by default because of the security implications. You will
need to tell mod_wsgi to pass it through if you require it. See
section 'User Authentication' of:

http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines

and description of WSGIPassAuthorization directive in:

http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives

Graham

[Brian Smith]

Graham Dumpleton wrote:
> WSGIPassAuthorization:
>
> http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives

Can this be made to work with .htaccess?

- Brian

[Carl Nobile]

I suppose that is a good question if you are using it at an ISP, but in my case as a web service, which is usually on a private network behind the ESB (Enterprise Service Bus), doing it in the apache config is fine with me.

I personally don't like to use .htaccess files since the server needs to read it on every request, great for testing but too slow for a production site.

Carl

--
-------------------------------------------------------------------------------
Carl J. Nobile (Software Engineer)
carl....@gmail.com
-------------------------------------------------------------------------------

[Graham Dumpleton]

No.

If one allowed that, then one would for example in a corporate setting
where HTTP auth was used across a whole web site, allow any individual
user who may have ability to run WSGI stuff to then collect the logins
and passwords of his work mates. For reasons like that it is
explicitly only allowed to be set from main Apache configuration where
the system administrator has control and can make the decision if
someone is trusted and should be able to see the passwords.

Being extra secure like this seems to be the more prudent option.

Graham

[Graham Dumpleton]

On 15/01/2008, Carl Nobile <carl....@gmail.com> wrote:
> I suppose that is a good question if you are using it at an ISP, but in my
> case as a web service, which is usually on a private network behind the ESB
> (Enterprise Service Bus), doing it in the apache config is fine with me.
>
> I personally don't like to use .htaccess files since the server needs to
> read it on every request, great for testing but too slow for a production
> site.

Whether .htaccess is slow really depends on what you are doing and how
you are using it. In the scheme of things, a roughly five percent
reduction through prudent use of .htaccess files may be quite
acceptable when you balance it against the ability to dynamically
change the configuration of the application without having to restart
the whole of Apache.

I vaguely remember there being a comment against this recent Python
and PHP conversation brought up by Ian Bicking and others which
suggested that one of the major Internet web sites still used
.htaccess files for this exact reason.

Also, that five percent figure I give is based on looking at what one
can achieve as maximum throughput for a site when the application is
doing the least possible. In this case I used the mod_wsgi performance
figures as a guide.

http://code.google.com/p/modwsgi/wiki/PerformanceEstimates

When you consider though that the WSGI application and any database
access is going to cause your request through to drop quite a fair bit
of the theoretical maximum, that five percent actually drops down to
be a much much lower percentage when you look at the much longer
request time of a complex application.

So, my feeling is that .htaccess files are probably not as bad as many
people make out, provided that is that they aren't abused and you have
deep directory hierarchies with lots of .htaccess file spread all
other the place.

Graham

[Carl Nobile]

You are totally correct in everything you say when one stops to think about it. With a web service that is designed, debugged, tested, then put into production and probably seldom changed I still think the best place for the apache configuration is in the apache config files.

This leads me to another thought I've been having and why I am leaning toward using your mod_wsgi for web services. Python has not in general been considered the greatest language for SOA. I'm also a Java programmer, but my love is really Python; however, Java/SOAP seems to be what people gravitate to when they think about SOA. I guess .NET/C# is in there too, but generally not Python especially when one combines it with REST. Believe me, some people think I'm nuts going down this path. I feel that mod_wsgi has potentially great value in the web services arena.

I know from the comments from you and others ISP support for Python seems to be the most prevalent goal. I hope to be able to provide feedback on how mod_wsgi performs when used in SOA backends. We are just starting down this path, so it may be months before we get working prototypes if ever--a lot of people here still need convincing yet.

This should maybe be another thread, but I would like to see if anybody else has given SOA/REST/Web Services/mod_wsgi any thought. I think what will make Python a serious contender in this area will be WSGI and having mod_wsgi will surely help.

Graham, I very much appreciate your work. A few months ago we had an email correspondance and you gave of your time freely to me then as you seem to do with everybody, not something I often see.

Thanks,
Carl

[Brian Smith]

Can't it be conditional based on "AllowOverride AuthConfig" or a new
AllowOverride option?

This isn't a valid (effective) security mechanism when mod_rewrite is
active. See http://www.rittau.org/blog/20061119-00,
http://codex.gallery2.org/Gallery2:Modules:httpauth#Authorization_Header
, and other sources:

RewriteEngine on
RewriteBase /
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule ^(.*)$ $1 [e=HTTP_AUTHORIZATION:%1]

Misusing mod_rewrite like this is more likely to result in a security
breach than a straightforward mechanism.

There are a lot of (Python) web applications that are implementing new
authentication protocols like OpenID, or emulating the Amazon.com Web
Services or Google Auth authentication mechanisms. If you want these
applications to be deployable behind mod_wsgi in a shared web hosting
environment, then they need to be able to process the Authorization
header.

- Brian

[Graham Dumpleton]

On 15/01/2008, Brian Smith <br...@briansmith.org> wrote:
>
> Graham Dumpleton wrote:
> > On 15/01/2008, Brian Smith <br...@briansmith.org> wrote:
> > > Graham Dumpleton wrote:
> > > > WSGIPassAuthorization:
> > > >
> > > > http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives
> > >
> > > Can this be made to work with .htaccess?
> >
> > No.
> >
> > If one allowed that, then one would for example in a
> > corporate setting where HTTP auth was used across a whole web
> > site, allow any individual user who may have ability to run
> > WSGI stuff to then collect the logins and passwords of his
> > work mates. For reasons like that it is explicitly only
> > allowed to be set from main Apache configuration where the
> > system administrator has control and can make the decision if
> > someone is trusted and should be able to see the passwords.
> >
> > Being extra secure like this seems to be the more prudent option.
>
> Can't it be conditional based on "AllowOverride AuthConfig" or a new
> AllowOverride option?

One can't extend AllowOverride beyond the fixed values it takes now. I
did think about using AuthConfig to allow it in .htaccess but at the
time decided to just err on side of caution and defer making that
decision.

The problem I had at the time was that I really wanted a way to
optionally allow WSGIProcessGroup and WSGIApplicationGroup, as well as
WSGIPassAuthorization to be defined in .htaccess file. The
AllowOverride couldn't be extended and wasn't too keen on adding a
WSGIAllowOverride directive equivalent just for mod_wsgi.

To get around WSGIProcessGroup and WSGIApplicationGroup the solution
was to allow in main configuration:

SetEnv APPLICATION_GROUP %{GLOBAL}
WSGIApplicationGroup %{ENV:APPLICATION_GROUP}

That way if user has FileInfo override they could say:

SetEnv APPLICATION_GROUP xyz

as a way of overriding it. To allow that did require it to be setup in
main configuration first.

So I had a solution for those two directives at least. I didn't then
allow WSGIPassAuthorization to be controlled by AuthConfig as saw them
as allowing how auth information was used, but didn't in any other
cases, eg, CGI, allow it to be passed through to a script of some
form. But then, the CGI cases is more problematic because the
information actually appears in environment variables which are more
easily accessed.

> This isn't a valid (effective) security mechanism when mod_rewrite is
> active. See http://www.rittau.org/blog/20061119-00,
> http://codex.gallery2.org/Gallery2:Modules:httpauth#Authorization_Header
> , and other sources:
>
> RewriteEngine on
> RewriteBase /
> RewriteCond %{HTTP:Authorization} ^(.*)
> RewriteRule ^(.*)$ $1 [e=HTTP_AUTHORIZATION:%1]
>
> Misusing mod_rewrite like this is more likely to result in a security
> breach than a straightforward mechanism.

Yeah, well I didn't think of that one. :-(

One can't even stop this through mod_wsgi always ensuring that
HTTP_AUTHORISATION is deleted from environment when
WSGIPassAuthorization not set. This is because user could just pass it
through a variable of another name.

Since that hack is available when FileInfo is allowed, can't see any
downside then of allowing WSGIPassAuthorization when AuthConfig is
allowed, as less likely that AuthConfig would be allowed anyway.

> There are a lot of (Python) web applications that are implementing new
> authentication protocols like OpenID, or emulating the Amazon.com Web
> Services or Google Auth authentication mechanisms. If you want these
> applications to be deployable behind mod_wsgi in a shared web hosting
> environment, then they need to be able to process the Authorization
> header.

The way I saw it was that the documentation for shared web hosting
would explain setting WSGIPassAuthorization to enable passing of the
information if site configurations were always such that only one user
own the site. In other words, still wanted it to be a conscious
decision of the person setting up the box after they understood the
consequences.

Graham

[Madhukumar Seshadri]

What if you want to get list of headers where you don't have the keys. Something like for every header in self.something.headers. You cannot do this today with mod_wsgi and I must acknowledge I have not read Pep 333.

Regards,
Madhu
www.letustalkweb.org

[Graham Dumpleton]

See:

http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Displaying_Request_Environment

As environ is a dictionary you can just iterate over it and dump out or compare the key/values as needed.

As HTTP header names are encoded as per CGI specification, they are the ones with HTTP_ prefix. The exception to this is the CONTENT_LENGTH and CONTENT_TYPE keys.

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.