SSL + web2py problem

240 views
Skip to first unread message

voltron

unread,
Jun 18, 2008, 3:46:41 AM6/18/08
to web2py Web Framework
I have my certificate installed, the SSL module in Apache works. Inmy
controller I have this:


if request.env.wsgi_url_scheme == "http":
path = 'https://%s/' % (URL(r=request, f="register"))
redirect(path)


The path generated is correct, the URL in the browser is correct but
Apache notifies me of this error:


Service Temporarily Unavailable

The server is temporarily unable to service your request due to
maintenance downtime or capacity problems. Please try again later.

This is what my Apache config looks like

<VirtualHost *:443>
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain_ssl.key

<Location "/accounts/register">
Order deny,allow
Allow from all
ProxyPass https://localhost:8000/accounts/register
ProxyPassReverse https:/localhost:8000/accounts/register
</Location>


Is there anything else I have to do? Thanks

voltron

unread,
Jun 18, 2008, 4:54:14 AM6/18/08
to web2py Web Framework
Ok, I got it to work. But why are all my static files not being found
anymore in the templates?

Thanks

voltron

unread,
Jun 18, 2008, 7:38:42 AM6/18/08
to web2py Web Framework
Anybody? This is really slowing me down.


Thanks

Mark

unread,
Jun 18, 2008, 9:23:29 AM6/18/08
to web2py Web Framework
Voltron,

I don't know that I can help, but your post leaves me scratching my
head in a number of places.

First, your code to create the https redirect generates "https:///
modemo/default/register/" which isn't a valid URL. You'll need to add
the host name which should be "localhost:8000" if your Apache redirect
is as you've said. The net result would be a URL that looks like:
"https://localhost:8000/modemo/default/register/"

This is where things start to get interesting to me. First, https is
usually served on port 443, not port 8000. I haven't gotten web2py.py
to work with the -c and -k settings (yet), so I can't say personally
that it works with https pointed at port 8000, but it should. I've
recently been wrestling with this issue, and it looks like the 1.35
version of web2py doesn't support OpenSSL. (See [1].)

If web2py doesn't handle https directly, you can still get it to work,
but you'll need to change the Apache2 redirect to strip off the https:

<Location "/accounts/register">
Order deny,allow
Allow from all
ProxyPass http://localhost:8000/accounts/register
ProxyPassReverse http://localhost:8000/accounts/register
</Location>

(Note the http://localhost:8000 instead of https://localhost:8000.)
This is how Massimo recommends in his posts and the FAQ.

The good part is this will definitely redirect to your Web2Py
instance. (I'm using it now.) The bad news is your test for
redirecting an incoming non-secure connection (if
request.env.wsgi_url_scheme == "http") will *always* return true. I
believe that will give you infinite recursion.

This may be obvious to you, but I definitely missed it for a while,
and maybe I can save others some wasted time: You need to recognize
that with the Apache2 front end, the *only* thing that ever connects
to Web2Py is Apache. It intercepts every incoming connection (on
ports 80 and 443) and then *Apache* connects to Web2Py on port 8000 on
behalf of the incoming http/https request.

One of the upshots of this is that using an https connection between
Apache and Web2Py (as your original post implies) is complete over
kill, and the only reason to do it is because there isn't any other
reliable way to detect whether the originating incoming connection was
on http ot https. Using https between Apache and Web2Py, you're
spending a lot of cycles encrypting data that is literally passing
from one place in RAM to another on the same physical CPU. You've
probably got multiple GHz and multiple cores, so you've probably got
cycles to waste, but it's important to understand the design trade
offs you're making so they are intentional and not accidental. ;o)

Massimo, this issue about detecting whether the originating request
was http or https is something you probably want to think about. It's
something we are wrestling with, and it looks like others are as well.

-MM

[1] http://groups.google.com/group/web2py/t/27275e16c1462951

voltron

unread,
Jun 18, 2008, 9:38:35 AM6/18/08
to web2py Web Framework
Thanks Mark! An answer! Heavens be praised! I have spent the whole day
on this :-(

I had taken a look at AlterEgo and adjusted my Apache settings as you
also suggested, I just did not post them, sorry for making you scratch
your head. This is what it looks like

<VirtualHost *:443>
ProxyRequests Off
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain_ssl.key
ProxyVia On
#ProxyPreserveHost On

<Location "/parents/accounts/register">
Order deny,allow
Allow from all
ProxyPass http://127.0.0.1:8000/parents/accounts/register
ProxyPassReverse http://127.0.0.1:8000/parents/accounts/register/
</Location>
</VirtualHost>

If I do this:
URL = 'https://www.domain.com/%s' % (URL(a = request.application, c
='accounts', f='register' , vars = {'subscription_type': 'trial'}))
it works, but all the static stuff is not rendered( JS, CSS, Images
e.t.c). You made a suggestion once about passing an attribute to
URL(), '_secure = True' if I can recall well. So, to cut things
short. what should I do to get a few URLs to use SSL Mark before my
boss decides to get his shotgun after me?. Oh, an no, we do not have
CPU cycles to waste


Thanks





Massimo Di Pierro

unread,
Jun 18, 2008, 10:16:13 AM6/18/08
to web...@googlegroups.com
I think

  <Location "/accounts/register">
    Order deny,allow
    Allow from all
        ProxyPassReverse https:/localhost:8000/accounts/register
  </Location>

should be

  <Location "/accounts/register">
    Order deny,allow
    Allow from all
        ProxyPassReverse https:/localhost:8000/
  </Location>

Mark

unread,
Jun 18, 2008, 10:30:32 AM6/18/08
to web2py Web Framework
Voltron,

I'm not part of Web2Py, just a user. So, I'm struggling right along
with you, but that doesn't stop me from offering my 2 cents. ;o)

On the _secure=True attribute you want to add to your URL() object, I
don't remember suggesting this. Maybe you can post a link? I would
be dubious since that only affects the local URL() object in memory.
Once it's turned in to a string (e.g. as part of your 'https://%s/' %
(URL(r=request, f="register")) operation), it's just a string. No
more attribute. No more magic.

The very problem is there is simply no place easy to tuck the
information that a particular request was originated over a secure
connection. That's precisely where I came to as I have been banging
on this problem, and it's why I suggested Massimo might want to start
a thread about his thoughts.

Currently, there is only one solution that works (that I'm aware of).
It's quick and dirty, but it's right there in the Apache settings
Massimo posted. Basically, anything that comes in on the "admin" app
is guaranteed to be from a secure connection. Anything else is not.
Period. (Until and unless we get -c and -k working on Web2Py.)

Leveraging the /admin/c/f path is probably illadvised since that would
be a maintenance nightmare as Web2Py is modified and updated. But,
the concept could be duplicated for your own "secure path". Here's a
sample:

<VirtualHost *:80>
<Location /admin>
SSLRequireSSL
</Location>
<Location /foo>
SSLRequireSSL
</Location>
<Location />
Order deny,allow
Allow from all
ProxyPass http://localhost:8000/
ProxyPassReverse http://localhost:8000/
</Location>
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog /var/log/apache2/access.log common
</VirtualHost>

<VirtualHost *:443>
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/server.crt
SSLCertificateKeyFile /etc/apache2/ssl/server.key
<Location />
Order deny,allow
Allow from all
ProxyPass http://localhost:8000/
ProxyPassReverse http://localhost:8000/
</Location>
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog /var/log/apache2/access.log common
</VirtualHost>

Now, anything connecting to your "foo" application is guaranteed to be
secure (just like admin). You can vector from there to wherever you
need to go,

I don't like this approach since it kind of crams everything into a
single application, but that probably works fine for a large class of
problems. I also don't like it because it means you need to
coordinate changes in two radically different places which makes it
brittle (e.g. the Apache2 config files in /etc/apache2/sites-available
folder has to agree with the application structure within Web2Py).
And finally, I don't like it because there is a bunch of code in
Web2Py that implies you can check for wsgi_url_scheme, and this code
is an absolute distraction. Using the documented Apache proxy
settings, Web2Py will *never* see an https URL. Ever.

But, if you need to get yourself out off the critical path, creating
your own SSLRequireSSL URL path will work.

-MM

Mark

unread,
Jun 18, 2008, 10:31:37 AM6/18/08
to web2py Web Framework
Massimo,

> >>   <Location "/accounts/register">
> >>     Order deny,allow
> >>     Allow from all
> >>     ProxyPass  https://localhost:8000/accounts/register
> >>         ProxyPassReverse https:/localhost:8000/
> >>   </Location>

Won't work until the -c and -k command line switches work on Web2Py.

-MM

voltron

unread,
Jun 18, 2008, 10:46:31 AM6/18/08
to web2py Web Framework
I'm sorry Mark, It was Michael that suggested this:

http://groups.google.com/group/web2py/browse_thread/thread/297267bb97331160/5603d91b239cdc31?lnk=gst&q=ssl#5603d91b239cdc31

For give me for being dense today, but why are my templates trashed? I
can imagine that the links to the templates don't work anymore because
the protocol is then 'https' instead of 'http' I would not want to be
putting conditionals in my templates to check if the protocol

Thanks for your help I really appreciate it

Mark

unread,
Jun 18, 2008, 11:49:36 AM6/18/08
to web2py Web Framework
All,

This may be obvious to others, but it only just ocurred to me...
(Doh!)

In another thread, I'm (still) wrestling with getting SSL working
under Windows with the -c and -k command-line switches. But... I've
also got a Ubuntu Linux box as well.

You will need to "sudo apt-get install python-pyopenssl" in order to
install the OpenSSL modules for Python. Here's a mini doctest that
should pass if you've got OpenSSL installed:

>>> import OpenSSL
>>> "Whew!"
'Whew!'

If you don't, you'll get a trace dump. Here's what I get under
Windows:

>>> import OpenSSL
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python25\Lib\site-packages\OpenSSL\__init__.py", line
11, in <module>
import rand, crypto, SSL, tsafe
ImportError: DLL load failed: The specified module could not be
found.

Now my Ubuntu Web2Py can connect with an https request. Woo hoo!

Unfortunately, this opens up a whole host of issues. Massimo, it
looks like you don't think this warrants a whole seperate thread, but
I've already mentioned that setting up a DES3 cypher between the
Apache daemon and the Web2Py daemon is completely useless overkill.

Also, now you need to buy (or generate[1]) and support 2 sets of
server certificates. One for the outward facing Apache server, and
another pair for the Web2Py server. You can't share the cert since
the Web2Py links are all on localhost:8000 or 127.0.0.1:8000, so the
certificate will need to match.

Also, it looks like you need to run 2 parallel instances of Web2Py
(e.g. on on port 8000 for http requests, and another on port 8001 for
https requests). I haven't tested this extensively, so there may be a
way to get a single Web2Py instance to serve both http *and* https,
but it doesn't look like it. If you have two parallel instances, it
seems like there would be a host of concurrent access issues to the
resources being served by the 2 Web2Py's. I understand that Web2Py
was designed to run multiple simultaneous instances. But, it seems
like it would be *waaay* easy for someone using Web2Py to implement a
syncronization policy that would break.

-MM

[1] http://mdp.cti.depaul.edu/AlterEgo/default/show/140

Massimo Di Pierro

unread,
Jun 18, 2008, 11:56:53 AM6/18/08
to web...@googlegroups.com
> Massimo, it
> looks like you don't think this warrants a whole seperate thread,

I am just not sure what you mean. You asking for an AlterEgo page?

> but
> I've already mentioned that setting up a DES3 cypher between the
> Apache daemon and the Web2Py daemon is completely useless overkill.
>
> Also, now you need to buy (or generate[1]) and support 2 sets of
> server certificates. One for the outward facing Apache server, and
> another pair for the Web2Py server. You can't share the cert since
> the Web2Py links are all on localhost:8000 or 127.0.0.1:8000, so the
> certificate will need to match.

I'd recommend using apache + mod_wsgi in production or apache
+mod_proxy and only one certificate.

>
> Also, it looks like you need to run 2 parallel instances of Web2Py
> (e.g. on on port 8000 for http requests, and another on port 8001 for
> https requests).

The suggestion above is to avoid this.

> I haven't tested this extensively, so there may be a
> way to get a single Web2Py instance to serve both http *and* https,
> but it doesn't look like it. If you have two parallel instances, it
> seems like there would be a host of concurrent access issues to the
> resources being served by the 2 Web2Py's. I understand that Web2Py
> was designed to run multiple simultaneous instances. But, it seems
> like it would be *waaay* easy for someone using Web2Py to implement a
> syncronization policy that would break.

Please elaborate.

Mark

unread,
Jun 18, 2008, 12:17:20 PM6/18/08
to web2py Web Framework
Voltron,

Now I'm sorry for being anal... I read the post from Michael, but it
only mentions some testable internal value (self.is_secure) inside the
Django framework, not an attribute to add to the URL() object in
Web2Py. You don't need to go dig up more, but I wanted to make sure I
wasn't missing what you were trying to say.

On your trashed templates, I think you are asking about the "Service
Temporarily Unavailable" message that started this thread. If so,
that's an indication that your Apache redirect (ProxyPass) is not
connecting to anything. This could be for a number of reasons, but if
your Web2Py instance is running, it's probably because of a
disagreement about http/https-ness. (I'm almost certain this is
what's going on for you.)

The snippet of your Apache2 config you included above shows that you
map https://<whatever-your-domain-is>/parents/accounts/register/<anything-else-even-nothing>
gets routed to ---> http://localhost:8000/parents/accounts/register/<anything-else-even-nothing>.

NOTE: Massimo posted later that you should have it redirect to
https://localhost:8000/... I believe that will get you back to the
"Service Temporarily Unavailable" state, but you should try.

The issue with your static content is it gets requested from
https://<yourdomain>/parents/static/<whatever>. This doesn't match
any of your ProxyPass patterns, so it won't get remapped. The Apache
server will simply say it doesn't know where to find that resource.

The fix is to trim your Apache config down some:

<VirtualHost *:443>
ProxyRequests Off
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain_ssl.key
ProxyVia On
#ProxyPreserveHost On

<Location "/parents">
Order deny,allow
Allow from all
ProxyPass http://127.0.0.1:8000/parents/
ProxyPassReverse http://127.0.0.1:8000/parents/
</Location>
</VirtualHost>

That will allow your static content to come through as well. This is
exactly equivalent to my "special foo folder" suggestion above. The
downside is that all external accesses to /<your-domain>/parents/
<anything> will need to be through an https:// URL.

-MM


On Jun 18, 7:46 am, voltron <nhy...@googlemail.com> wrote:
> I'm sorry Mark, It was Michael that suggested this:
>
> http://groups.google.com/group/web2py/browse_thread/thread/297267bb97...

Mark

unread,
Jun 18, 2008, 12:28:02 PM6/18/08
to web2py Web Framework
Massimo,

> > looks like you don't think this warrants a whole seperate thread,
>
> I am just not sure what you mean. You asking for an AlterEgo page?

I mean here in http://groups.google.com/group/web2py. It seems like
there needs toi be a thread about the architectural decisions on a
production Web2Py setup supporting an application that has some open
(unsecured/plain http) functionality, and some secured/https
functionality. It may be obvious to you, but it's not to me (and I'm
guessing Voltron).


> I'd recommend using apache + mod_wsgi in production or apache
> +mod_proxy and only one certificate.

Can you elaborate? Is this different than the setup you've documented
everywhere else? Specifically, use Apache to ProxyPass from port 80
and 443 to 127.0.0.1:8000 against a single instance of Web2Py running
from "cd .../web2py-install-dir; python web2py.py <-a '<recycle>'
etc.>"?


> > Also, it looks like you need to run 2 parallel instances of Web2Py
> > (e.g. on on port 8000 for http requests, and another on port 8001 for
> > https requests).
>
> The suggestion above is to avoid this.

Excellent! I'm just unclear on how it accomplishes this.


> > I haven't tested this extensively, so there may be a
> > way to get a single Web2Py instance to serve both http *and* https,
> > but it doesn't look like it.  If you have two parallel instances, it
> > seems like there would be a host of concurrent access issues to the
> > resources being served by the 2 Web2Py's.  I understand that Web2Py
> > was designed to run multiple simultaneous instances.  But, it seems
> > like it would be *waaay* easy for someone using Web2Py to implement a
> > syncronization policy that would break.
>
> Please elaborate.

I'm not sure what you are asking about, but the problem goes away if
(as you imply) a single instance of Web2Py can service both http and
https requests.

-MM

voltron

unread,
Jun 18, 2008, 12:40:41 PM6/18/08
to web2py Web Framework
Hi Mark!

What I mean by trashed by templates is that I see the forms, SSL
works, but the pages look strange because the CSS and images are not
displayed



On Jun 18, 6:17 pm, Mark <mark.mo...@notlimited.com> wrote:
> Voltron,
>
> Now I'm sorry for being anal...  I read the post from Michael, but it
> only mentions some testable internal value (self.is_secure) inside the
> Django framework, not an attribute to add to the URL() object in
> Web2Py.  You don't need to go dig up more, but I wanted to make sure I
> wasn't missing what you were trying to say.
>
> On your trashed templates, I think you are asking about the "Service
> Temporarily Unavailable" message that started this thread.  If so,
> that's an indication that your Apache redirect (ProxyPass) is not
> connecting to anything.  This could be for a number of reasons, but if
> your Web2Py instance is running, it's probably because of a
> disagreement about http/https-ness.  (I'm almost certain this is
> what's going on for you.)
>
> The snippet of your Apache2 config you included above shows that you
> map https://<whatever-your-domain-is>/parents/accounts/register/<anything-else-even-nothing>
> gets routed to --->http://localhost:8000/parents/accounts/register/<anything-else-even-nothing>.
>
> NOTE: Massimo posted later that you should have it redirect tohttps://localhost:8000/...  I believe that will get you back to the
> "Service Temporarily Unavailable" state, but you should try.
>
> The issue with your static content is it gets requested from
> https://<yourdomain>/parents/static/<whatever>.  This doesn't match
> any of your ProxyPass patterns, so it won't get remapped.  The Apache
> server will simply say it doesn't know where to find that resource.
>
> The fix is to trim your Apache config down some:
>
> <VirtualHost *:443>
>   ProxyRequests Off
>   SSLEngine On
>   SSLCertificateFile /etc/apache2/ssl/domain.crt
>   SSLCertificateKeyFile /etc/apache2/ssl/domain_ssl.key
>   ProxyVia On
>   #ProxyPreserveHost On
>
>   <Location "/parents">
>     Order deny,allow
>     Allow from all
>     ProxyPass  http://127.0.0.1:8000/parents/
>    ProxyPassReversehttp://127.0.0.1:8000/parents/

voltron

unread,
Jun 18, 2008, 12:51:15 PM6/18/08
to web2py Web Framework
Thank you very much for your help Mark

Hmm, I am in big trouble then, I really cant put the whole 'parents'
controller under SSL. I wonder if Massimo has another idea. I have to
get this done this week. Oh boy, this is bad, very bad...

Mark

unread,
Jun 18, 2008, 1:12:17 PM6/18/08
to web2py Web Framework
Voltron,

There is some (semi-)interesting info you can root around for in
request.env.http_x_forwarded_{for|host|server}, but I don't think that
will get you out of the woods.

I would suggest splitting your top-level application into parents and
parents-secure (or whatever you would like to call it). Then modify
your Apache config:

<VirtualHost *:443>
ProxyRequests Off
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain_ssl.key
ProxyVia On
#ProxyPreserveHost On


<Location "/parents">
Order deny,allow
Allow from all
ProxyPass http://127.0.0.1:8000/parents-sec/
ProxyPassReverse http://127.0.0.1:8000/parents-sec/
</Location>
</VirtualHost>

You should also add this somewhere at the top of your <VirtualHost *:
80> section:

<Location /parents-sec>
SSLRequireSSL
</Location>

Now, inside your parents-sec controller(s) you can rest assured that
you are talking over an SSL connection. If you want, you can set some
handy global variable to indicate things are secure and then call code
in your parents appication folder.

I'm not sure what the idiom is for sharing code between applications
under Web2Py, but you should be able to do it very simply.

-MM

voltron

unread,
Jun 18, 2008, 1:37:30 PM6/18/08
to web2py Web Framework
Thanks Mark,

I ll go that route for now.
Reply all
Reply to author
Forward
0 new messages