how to use python 3.6.1 in an anaconda3 env w/ Apache + mod_wsgi + flask on aws ubuntu ec2 instance

[Brian Lehman]

Good day. When I try to pip install mod_wsgi inside of an anaconda env called test1.0 on an aws ec2 instance w/ ubuntu, I get the folowing error. Any suggestions?

(test1.0) ubuntu$ python --version 

Python 3.6.1 :: Continuum Analytics, Inc.

(test1.0) ubuntu$ 

(test1.0) ubuntu$ 

(test1.0) ubuntu$ pip install mod_wsgi

Collecting mod_wsgi

  Using cached mod_wsgi-4.5.15.tar.gz

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-build-80walwu3/mod-wsgi/setup.py", line 164, in <module>

        'missing Apache httpd server packages.' % APXS)

    RuntimeError: The 'apxs' command appears not to be installed or is not executable. Please check the list of prerequisites in the documentation for this package and install any missing Apache httpd server packages.

    

    ----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-80walwu3/mod-wsgi/

[Graham Dumpleton]

As per instructions in:

    https://pypi.python.org/pypi/mod_wsgi

You need to ensure you have the 'dev' packages for Apache installed.

If you are running Debian or Ubuntu Linux with Apache 2.4 system packages, regardless of which Apache MPM is being used, you would need both:

    apache2

    apache2-dev

Note that if using Anaconda Python, after having installed mod_wsgi with pip, ensure you run:

    mod_wsgi-express module-config

to show the configuration you need to include in the system Apache if you intend manually configuring it for mod_wsgi, instead of using 'mod_wsgi-express start-server'.

This is necessary as Anaconda Python can require extra configuration directives to force load its shared library. That command should output the appropriate lines, although just check that the generated line for LoadFile directive actually points at a valid path. Seen one report where for Anaconda Python the path it generated was wrong and it had to be changed.

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Brian Lehman]

Thanks Graham. So everything is installed. Being brand new to this world, I'm looking for a tutorial to (and please forgive me as I'm likely using the wrong terminology) see how to tell Apache to use mod_wsgi to run my flask app. I'm guessing that I need to change some config files in Apache to use my conda environment and then write some .wsgi file? However, I'm seeing quite a few different threads on this subject. Any advice is greatly appreciated.   

## get apache2

sudo apt-get install apache2

sudo apt-get install apache2-dev

## get mod_wsgi

pip install mod_wsgi

#force load Anaconda's shared library

mod_wsgi-express module-config

## start/restart apache

sudo service apache2 restart 

[Graham Dumpleton]

In the first instance you should ensure things work with a WSGI hello world, no Flask framework. This ensures setup with Anaconda Python is okay, which sometimes doesn't always work because of stupid things Anaconda Python does which breaks the ability to use it in embedded systems.

As to deploying Flask, the Flask documentation has details:

* http://flask.pocoo.org/docs/0.12/deploying/mod_wsgi/

Also ensure you read:

* http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

so you are setting up any Python virtual environment correctly as guides for frameworks often don't suggest the preferred method, or simply don't address it.

Graham

[Brian]

Thank you Graham. 

Any recs on testing and/or debugging a 500 internal server error when I try to hit the site? I'm assuming that it's in the configuration so I'm going to add full details of the process and config files below. 

All commands ran:

## build conda environment

conda create --name test1.0 python==3.6.1 pandas jupyter requests flask

## activate conda environment 

source activate test1.0

 

## get apache2

sudo apt-get install apache2

sudo apt-get install apache2-dev

## get mod_wsgi

#sudo apt-get install libapache2-mod-wsgi

pip install mod_wsgi

## force load Anaconda's shared library

mod_wsgi-express module-config

## enable mod_wsgi

sudo a2enmod wsgi 

## get git 

sudo apt-get install git

## create config for apache to know about wsgi and flask app

touch /etc/apache2/sites-available/FlaskApp.conf

## disable default config

sudo a2dissite 000-default

## provide new config

sudo a2ensite FlaskApp.conf

## start/restart apache

sudo service apache2 restart

Contents of FlaskApp.conf:

<VirtualHost *:80>

                ServerName <IPv4 PUBLIC IP>

                ServerAdmin ubuntu@<IPv4 PUBLIC IP>.us-west-2.compute.amazonaws.com

                WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

                <Directory /var/www/FlaskApp/FlaskApp/>

                        Order allow,deny

                        Allow from all

                </Directory>

                Alias /static /var/www/FlaskApp/FlaskApp/static

                <Directory /var/www/FlaskApp/FlaskApp/static/>

                        Order allow,deny

                        Allow from all

                </Directory>

                ErrorLog ${APACHE_LOG_DIR}/error.log

                LogLevel warn

                CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

Location of flask app and .wsgi file.

(test1.0) ubuntu:/var/www$ tree

.

├── FlaskApp

   ├── FlaskApp

   │   ├── __init__.py

   │   ├── static

   │   └── templates

   └── flaskapp.wsgi

Contents of flaskapp.wsgi:

#!/usr/bin/python

import sys

import logging

logging.basicConfig(stream=sys.stderr)

sys.path.insert(0,"/var/www/FlaskApp/")

from FlaskApp import app as application

application.secret_key = '<secret>'

Contents of __init__.py:

from flask import Flask

app = Flask(__name__)

@app.route("/")

def hello():

    return "Hello World (flask via conda env)!"

if __name__ == "__main__":

    app.run()

When visiting the site, I get a 500 internal server error. I likely messed up something with the config. Any help is greatly appreciated.

Best,

-Brian

[Brian]

Note: the previous question is also on SO.

Best,

-Brian

[Vanessa Marques]

Brian, check out /var/log/apache2/error.log to see the errors and let us know what is in there. 

I'm having the exact same challenge as you (trying to configure flask + apache + anaconda + mod_wsgi on an ubuntu EC2 instance). 

[Brian]

The conda environment must not be set correctly because it can't find flask:

[Mon May 08 16:39:11.002873 2017] [:error] [pid 31696:tid 139896253531904] [client 208.184.3.194:36413] Traceback (most recent call last):

 

[Mon May 08 16:39:11.002923 2017] [:error] [pid 31696:tid 139896253531904] [client 208.184.3.194:36413]   File "/var/www/FlaskApp/flaskapp.wsgi", line 7, in <module>

[Mon May 08 16:39:11.002989 2017] [:error] [pid 31696:tid 139896253531904] [client 208.184.3.194:36413]     from FlaskApp import app as application

[Mon May 08 16:39:11.003012 2017] [:error] [pid 31696:tid 139896253531904] [client 208.184.3.194:36413]   File "/var/www/FlaskApp/FlaskApp/__init__.py", line 1, in <module>

[Mon May 08 16:39:11.003044 2017] [:error] [pid 31696:tid 139896253531904] [client 208.184.3.194:36413]     from flask import Flask

[Mon May 08 16:39:11.003081 2017] [:error] [pid 31696:tid 139896253531904] [client 208.184.3.194:36413] ImportError: No module named flask

[Graham Dumpleton]

Did you read:

    http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

as I linked to.

There is no evidence that you have added anything for a Python virtual environment if you are using one.

Graham

[Brian]

Thanks Graham. 

I did edit the FlaskApp.config file:

<VirtualHost *:80>
                ServerName <public ip>.us-west-2.compute.amazonaws.com
                ServerAdmin ubuntu@ec2-<public ip>.us-west-2.compute.amazonaws.com
                WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi
                <Directory /var/www/FlaskApp/FlaskApp/>

                        Order allow,deny
                        Allow from all
                </Directory>
                Alias /static /var/www/FlaskApp/FlaskApp/static
                <Directory /var/www/FlaskApp/FlaskApp/static/>
                        Order allow,deny
                        Allow from all
                </Directory>

                WSGIDaemonProcess FlaskApp python-path=/home/ubuntu/anaconda3/envs/test1.0/lib/python3.6/site-packages

                ErrorLog ${APACHE_LOG_DIR}/error.log
                LogLevel warn
                CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

And then applied following new step in the process:

## get apache2
sudo apt-get install apache2
sudo apt-get install apache2-dev

## get mod_wsgi
#sudo apt-get install libapache2-mod-wsgi
pip install mod_wsgi

## force load Anaconda's shared library
mod_wsgi-express module-config

## enable mod_wsgi
sudo a2enmod wsgi

## Install and register the mod_wsgi module with Apache  <--------------------------
sudo `which mod_wsgi-express` install-module

## get git
sudo apt-get install git

## create apache config

touch /etc/apache2/sites-available/FlaskApp.conf

## disable default config
sudo a2dissite 000-default

## provide new config
sudo a2ensite FlaskApp.conf

## start/restart apache
sudo service apache2 restart

The same error occurs in the log:

 Traceback (most recent call last):

   File "/var/www/FlaskApp/flaskapp.wsgi", line 7, in <module>

     from FlaskApp import app as

application
   File "/var/www/FlaskApp/FlaskApp/__init__.py", line 1, in <module>
     from flask import Flask

 ImportError: No module named flask

Thoughts?

Best,

-Brian

[Graham Dumpleton]

Just read again. You are using system package for mod_wsgi, not that installed for Anaconda. You should not be using both installs. Will send back instructions later when have time.

Graham

[Graham Dumpleton]

If you want to use Anaconda Python with mod_wsgi, the steps you need to follow are as follows.

1. Uninstall any system wide mod_wsgi package as you do not need it and it can interfere.

     sudo apt-get uninstall libapache2-mod-wsgi

2. Ensure 'dev' packages for Python and Apache installed.

3. You are better off using virtual environment created using virtualenv rather than using a conda env. I am not sure what set up steps need to be if using conda env right now. With virtual environment created using virtualenv, activate it.

4. Install mod_wsgi using pip.

5. Run:

    mod_wsgi-express module-config

and take the output of this and put it in a 'wsgi.load' file in the 'mods-available' directive of your Apache installation.

6. Enable the Apache module. Either run:

    sudo a2enmod wsgi

or symlink wsgi.load in to the 'mods-enabled' directory and restart Apache.

Verify that Apache starts up okay.

7. Manually configure Apache for mod_wsgi with a WSGI hello world program, restart Apache and verify that it works.

8. Add configuration for mod_wsgi to Apache sites file for your Flask application. Use the recommended steps for setting up a Python virtual environment as documented in:

    http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

Don't use 'python-path' with 'site-packages'. Use 'python-home' with the root of the Python virtual environment instead.

See if it works.

Would like to see you use virtualenv before we sort out what needs to be done for conda env. I can't remember right now how compatible conda env directories are with normal Python virtual environments.

Graham

[Brian]

Graham,

Thank you so much. I plan to use Anaconda for this project and so maybe using another tool is best? I don't want to give up on this direction if you think it will be fruitful, but maybe nginx + gunicorn + flask in a conda installed env is better? I'm even considering just deploying to Heroku. 

Thanks again and if you have any thoughts, I'm certainly game to hear what you think.

Best,

-Brian

[Graham Dumpleton]

If want to use conda env, just try it. Differences are:

3. Activate conda env.

8. Do same, but just use directory that 'sys.prefix' refers to for conda env.

It is quite likely it will work, I just don't remember right now and can't quickly test.

Graham

[Rajeev Jain]

HI, Did yu have any luck? I'm having the same exact issue.

best,

-R

[Graham Dumpleton]

You are better off restating your problem from scratch, with errors you see, any configuration etc. One cannot assume it is the same problem. So do that and can then help.

Graham

[Rajeev Jain]

Your point is well taken. Its 100% the exact same situation. Same objective, appears to be a similar software setup and same exact error. No need to repeat what has already been stated.

error I'm trying to resolve:

Wed Jun 13 00:15:55.875613 2018] [mpm_prefork:notice] [pid 5367] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g mod_wsgi/4.3.0 Python/3.5.2 configured -- resuming normal operations

[Wed Jun 13 00:15:55.875649 2018] [core:notice] [pid 5367] AH00094: Command line: '/usr/sbin/apache2'

[Wed Jun 13 00:16:02.051821 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] mod_wsgi (pid=5371): Target WSGI script '/var/www/FlaskApp/flaskapp.wsgi' cannot be loaded as Python module.

[Wed Jun 13 00:16:02.051892 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] mod_wsgi (pid=5371): Exception occurred processing WSGI script '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 00:16:02.061170 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] Traceback (most recent call last):

[Wed Jun 13 00:16:02.061202 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]   File "/var/www/FlaskApp/flaskapp.wsgi", line 7, in <module>

[Wed Jun 13 00:16:02.061207 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]     from FlaskApp import app as application

[Wed Jun 13 00:16:02.061214 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]   File "/var/www/FlaskApp/FlaskApp/__init__.py", line 2, in <module>

[Wed Jun 13 00:16:02.061217 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]     from flask import Flask

[Wed Jun 13 00:16:02.061234 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] ImportError: No module named 'flask'

In your step 4. Install mod_wsgi using pip. this is giving same error as the start of this thread.

user@ubuntu:/etc/apache2/mods-available$ pip install mod_wsgi

Collecting mod_wsgi

  Using cached https://files.pythonhosted.org/packages/9e/37/dd336068ece37c43957aa337f25c59a9a6afa98086e5507908a2d21ab807/mod_wsgi-4.6.4.tar.gz

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-install-k87n4cm_/mod-wsgi/setup.py", line 168, in <module>

        'missing Apache httpd server packages.' % APXS)

    RuntimeError: The 'apxs' command appears not to be installed or is not executable. Please check the list of prerequisites in the documentation for this package and install any missing Apache httpd server packages.

your help is appreciated.

[Graham Dumpleton]

On 13 Jun 2018, at 6:38 pm, Rajeev Jain <jain...@gmail.com> wrote:

Your point is well taken. Its 100% the exact same situation. Same objective, appears to be a similar software setup and same exact error. No need to repeat what has already been stated.

So you know for future. In a multi part email chain it can be difficult to go back and extract what configuration may have been used that you might be referring to. So when you ask questions you should always repeat what is occurring in your case. It is even better not to reply on a previous email chain and the history of the previous issue, which may not be the same, can just confuse things. So I am just going to ask for the configuration again.

error I'm trying to resolve:

Wed Jun 13 00:15:55.875613 2018] [mpm_prefork:notice] [pid 5367] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g mod_wsgi/4.3.0 Python/3.5.2 configured -- resuming normal operations

[Wed Jun 13 00:15:55.875649 2018] [core:notice] [pid 5367] AH00094: Command line: '/usr/sbin/apache2'

[Wed Jun 13 00:16:02.051821 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] mod_wsgi (pid=5371): Target WSGI script '/var/www/FlaskApp/flaskapp.wsgi' cannot be loaded as Python module.

[Wed Jun 13 00:16:02.051892 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] mod_wsgi (pid=5371): Exception occurred processing WSGI script '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 00:16:02.061170 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] Traceback (most recent call last):

[Wed Jun 13 00:16:02.061202 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]   File "/var/www/FlaskApp/flaskapp.wsgi", line 7, in <module>

[Wed Jun 13 00:16:02.061207 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]     from FlaskApp import app as application

[Wed Jun 13 00:16:02.061214 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]   File "/var/www/FlaskApp/FlaskApp/__init__.py", line 2, in <module>

[Wed Jun 13 00:16:02.061217 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318]     from flask import Flask

[Wed Jun 13 00:16:02.061234 2018] [wsgi:error] [pid 5371] [client 127.0.0.1:50318] ImportError: No module named 'flask'

Where is the Flask package installed?

From the command line, if you run the 'python' you want to use, what do you get when from the interpreter you do:

    import flask

    print(flask.__file__)

Also what do you get for:

    import sys

    print(sys.prefix)

The prior configuration example in the email chain was not setting up the Python virtual environment correctly so can't be used as a guide.

I need to see what is in the wsgi.load file if still using system package for mod_wsgi.

I then need to see what you may have set WSGIPythonHome, WSGIPythonPath, and general configuration for mod_wsgi in the VirtualHost, including WSGIDaemonProcess. Basically, include all what you are using in your response. I really need to see what you are using, and not what someone else used, even if you think it is the same.

Also show anything you have added in the WSGI script file to try and activate a virtual environment, or changes you are making to sys.path.

In your step 4. Install mod_wsgi using pip. this is giving same error as the start of this thread.

user@ubuntu:/etc/apache2/mods-available$ pip install mod_wsgi

Collecting mod_wsgi

  Using cached https://files.pythonhosted.org/packages/9e/37/dd336068ece37c43957aa337f25c59a9a6afa98086e5507908a2d21ab807/mod_wsgi-4.6.4.tar.gz

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-install-k87n4cm_/mod-wsgi/setup.py", line 168, in <module>

        'missing Apache httpd server packages.' % APXS)

    RuntimeError: The 'apxs' command appears not to be installed or is not executable. Please check the list of prerequisites in the documentation for this package and install any missing Apache httpd server packages.

Which means you do not have the httpd-dev package installed on the system.

[Graham Dumpleton]

For apxs problem, the system package missing is actually apache2-dev on Ubuntu.

[Rajeev Jain]

So you know for future. In a multi part email chain it can be difficult to go back and extract what configuration may have been used that you might be referring to. So when you ask questions you should always repeat what is occurring in your case. It is even better not to reply on a previous email chain and the history of the previous issue, which may not be the same, can just confuse things. So I am just going to ask for the configuration again.

Understood. No problem. 

Where is the Flask package installed?

On my Ubuntu 16.04 system, anaconda python and all python modules are installed in my home directory. There is no virtual environment. When I log-in the PATH environment variable is set so any invocation of python uses the desired python distribution. All my python code is able to find all other installed modules. There is history and legacy here. 

My immediate objective is to have the Apache server boot and use the python distribution in my home directory for all Flask/WSGI related processing.

From the command line, if you run the 'python' you want to use, what do you get when from the interpreter you do:

    import flask

    print(flask.__file__)

/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/__init__.py

 

Also what do you get for:

    import sys

    print(sys.prefix)

/home/rajeev/anaconda3

 

The prior configuration example in the email chain was not setting up the Python virtual environment correctly so can't be used as a guide.

I need to see what is in the wsgi.load file if still using system package for mod_wsgi.

/etc/apache2/mods-enabled$ cat wsgi.load 

LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so

notes:

1) was able to successfully pip install mod_wsgi

pip install mod_wsgi

Requirement already satisfied: mod_wsgi in ./anaconda3/lib/python3.6/site-packages (4.6.4)

2) updated softlink to use new object

/usr/lib/apache2/modules$ ll mod_wsg*

-rwxr-xr-x 1 root root 974744 Jun 13 14:45 mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so*

lrwxrwxrwx 1 root root     45 Jun 13 14:47 mod_wsgi.so -> mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so*

-rw-r--r-- 1 root root 207952 Jan 25  2016 mod_wsgi.so-3.5

3) server log shows the new mod_wsgi is being loaded.

tail -l /var/log/apache2/error.log

[Wed Jun 13 15:08:13.942743 2018] [mpm_prefork:notice] [pid 23851] AH00169: caught SIGTERM, shutting down

[Wed Jun 13 15:12:14.024133 2018] [ssl:warn] [pid 24110] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Wed Jun 13 15:12:14.114890 2018] [ssl:warn] [pid 24111] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Wed Jun 13 15:12:14.121149 2018] [mpm_prefork:notice] [pid 24111] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g mod_wsgi/4.6.4 Python/3.6 configured -- resuming normal operations

[Wed Jun 13 15:12:14.121185 2018] [core:notice] [pid 24111] AH00094: Command line: '/usr/sbin/apache2'

 

I then need to see what you may have set WSGIPythonHome, WSGIPythonPath, and general configuration for mod_wsgi in the VirtualHost, including WSGIDaemonProcess. Basically, include all what you are using in your response. I really need to see what you are using, and not what someone else used, even if you think it is the same.

/etc/apache2/mods-enabled$ cat wsgi.conf 

<IfModule mod_wsgi.c>

    WSGIPythonHome /home/rajeev/anaconda3

    WSGIPythonPath /home/rajeev/anaconda3/lib/python3.6/site-packages

</IfModule>

 /etc/apache2/sites-available$ cat FlaskApp.conf 

<VirtualHost *:83>

    ServerName flaskapp.com

    WSGIDaemonProcess flaskapp.com python-path=/home/rajeev/anaconda3

    WSGIProcessGroup %{GLOBAL}

    WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

    <Directory /var/www/FlaskApp/FlaskApp/>

        Require all granted

    </Directory>

</VirtualHost>

Also show anything you have added in the WSGI script file to try and activate a virtual environment, or changes you are making to sys.path.

/var/www/FlaskApp$ cat flaskapp.wsgi 

#!/home/rajeev/anaconda3/bin/python

import sys

import logging

logging.basicConfig(stream=sys.stderr)

sys.path.insert(0,"/var/www/FlaskApp/")

from FlaskApp import app as application

application.secret_key = 'Add your secret key'

Testing:

curl -sH 'Host: flaskapp.com' localhost:83|grep title

<title>500 Internal Server Error</title>

tail -l /var/log/apache2/error.log

[Wed Jun 13 15:46:22.636400 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]     from flask import Flask

[Wed Jun 13 15:46:22.636406 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/__init__.py", line 21, in <module>

[Wed Jun 13 15:46:22.636409 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]     from .app import Flask, Request, Response

[Wed Jun 13 15:46:22.636413 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/app.py", line 25, in <module>

[Wed Jun 13 15:46:22.636416 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]     from . import cli, json

[Wed Jun 13 15:46:22.636421 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/cli.py", line 18, in <module>

[Wed Jun 13 15:46:22.636424 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]     import ssl

[Wed Jun 13 15:46:22.636429 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]   File "/home/rajeev/anaconda3/lib/python3.6/ssl.py", line 101, in <module>

[Wed Jun 13 15:46:22.636432 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068]     import _ssl             # if we can't import it, let the error propagate

[Wed Jun 13 15:46:22.636453 2018] [wsgi:error] [pid 2199] [client 127.0.0.1:38068] ImportError: /home/rajeev/anaconda3/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so: undefined symbol: SSLv2_method

/var/www/FlaskApp/FlaskApp$ python __init__.py 

 * Serving Flask app "__init__" (lazy loading)

 * Environment: production

   WARNING: Do not use the development server in a production environment.

   Use a production WSGI server instead.

 * Debug mode: off

 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

127.0.0.1 - - [13/Jun/2018 15:48:53] "GET / HTTP/1.1" 200 -

127.0.0.1 - - [13/Jun/2018 15:48:53] "GET /favicon.ico HTTP/1.1" 404 -

Using browser on server (http://localhost:5000)is able to load test page

This is where I am at...

Your help is greatly appreciated. Perhaps I should consider setting a new virtual environment...

Thanks,

--Rajeev

[Graham Dumpleton]

Comments inline.

On 14 Jun 2018, at 8:52 am, Rajeev Jain <jain...@gmail.com> wrote:

So you know for future. In a multi part email chain it can be difficult to go back and extract what configuration may have been used that you might be referring to. So when you ask questions you should always repeat what is occurring in your case. It is even better not to reply on a previous email chain and the history of the previous issue, which may not be the same, can just confuse things. So I am just going to ask for the configuration again.

Understood. No problem. 

Where is the Flask package installed?

On my Ubuntu 16.04 system, anaconda python and all python modules are installed in my home directory. There is no virtual environment. When I log-in the PATH environment variable is set so any invocation of python uses the desired python distribution. All my python code is able to find all other installed modules. There is history and legacy here. 

My immediate objective is to have the Apache server boot and use the python distribution in my home directory for all Flask/WSGI related processing.

From the command line, if you run the 'python' you want to use, what do you get when from the interpreter you do:

    import flask

    print(flask.__file__)

/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/__init__.py

 

Also what do you get for:

    import sys

    print(sys.prefix)

/home/rajeev/anaconda3

 

The prior configuration example in the email chain was not setting up the Python virtual environment correctly so can't be used as a guide.

I need to see what is in the wsgi.load file if still using system package for mod_wsgi.

/etc/apache2/mods-enabled$ cat wsgi.load 

LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so

It is probably better to have run:

    mod_wsgi-express module-config

That should give you a LoadModule line for loading the mod_wsgi.so file from the place it is installed, as well as a LoadFile which loads the correct Python shared library for Anaconda, so the system Python library isn't picked up by mistake. Add the output of that into wsgi.load in place of what you have.

notes:

1) was able to successfully pip install mod_wsgi

pip install mod_wsgi

Requirement already satisfied: mod_wsgi in ./anaconda3/lib/python3.6/site-packages (4.6.4)

2) updated softlink to use new object

/usr/lib/apache2/modules$ ll mod_wsg*

-rwxr-xr-x 1 root root 974744 Jun 13 14:45 mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so*

lrwxrwxrwx 1 root root     45 Jun 13 14:47 mod_wsgi.so -> mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so*

-rw-r--r-- 1 root root 207952 Jan 25  2016 mod_wsgi.so-3.5

3) server log shows the new mod_wsgi is being loaded.

tail -l /var/log/apache2/error.log

[Wed Jun 13 15:08:13.942743 2018] [mpm_prefork:notice] [pid 23851] AH00169: caught SIGTERM, shutting down

[Wed Jun 13 15:12:14.024133 2018] [ssl:warn] [pid 24110] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Wed Jun 13 15:12:14.114890 2018] [ssl:warn] [pid 24111] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Wed Jun 13 15:12:14.121149 2018] [mpm_prefork:notice] [pid 24111] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g mod_wsgi/4.6.4 Python/3.6 configured -- resuming normal operations

[Wed Jun 13 15:12:14.121185 2018] [core:notice] [pid 24111] AH00094: Command line: '/usr/sbin/apache2'

 

I then need to see what you may have set WSGIPythonHome, WSGIPythonPath, and general configuration for mod_wsgi in the VirtualHost, including WSGIDaemonProcess. Basically, include all what you are using in your response. I really need to see what you are using, and not what someone else used, even if you think it is the same.

/etc/apache2/mods-enabled$ cat wsgi.conf 

<IfModule mod_wsgi.c>

    WSGIPythonHome /home/rajeev/anaconda3

    WSGIPythonPath /home/rajeev/anaconda3/lib/python3.6/site-packages

</IfModule>

Instead of WSGIPythonHome and WSGIPythonPath here, have:

    WSGIRestrictEmbedded On

That will turn off embedded mode and if configuration is wrong and application is not delegated to daemon mode processes correctly, you will get a 500 error to tell you you have an issue.

 /etc/apache2/sites-available$ cat FlaskApp.conf 

<VirtualHost *:83>

    ServerName flaskapp.com

    WSGIDaemonProcess flaskapp.com python-path=/home/rajeev/anaconda3

This is not correct. Use:

    WSGIDaemonProcess flaskapp.com python-home=/home/rajeev/anaconda3

    WSGIProcessGroup %{GLOBAL}

The problem with the daemon process configuration wasn't noticeable because you are using the wrong value for WSGIProcessGroup. You were telling it to use embedded mode instead.

    WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

I would suggest removing the WSGIProcessGroup above and change this to:

    WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi process-group=flaskapp.com application-group=%{GLOBAL}

This is ensuring daemon process group is used, and also the main interpreter context.

    <Directory /var/www/FlaskApp/FlaskApp/>

This path is incorrect. Should be:

    <Directory /var/www/FlaskApp>

Better still use:

    <Directory /var/www/FlaskApp>

    <Files flaskapp.wsgi>

    Require all granted

   </Files>

    </Directory>

since you have your project source code under the same directory. 

That you don't get forbidden at the moment suggests for some reason the overall Apache configuration has been weakened in some way such that anything on the whole file system can be served up if mapped, which isn't really a good idea.

When using Anaconda Python, you cannot used mod_ssl in Apache. This is because Anaconda Python ships its own SSL libraries which aren't compatible, and by having mod_ssl loaded, the system SSL libraries will get loaded first, resulting in any use of SSL in Python to fail.

If not using SSL in Apache, disable mod_ssl. If you are using it, the only way to do it is to use mod_wsgi-express to create a distinct instance and have the main Apache proxy to it.

/var/www/FlaskApp/FlaskApp$ python __init__.py 

 * Serving Flask app "__init__" (lazy loading)

 * Environment: production

   WARNING: Do not use the development server in a production environment.

   Use a production WSGI server instead.

 * Debug mode: off

 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

127.0.0.1 - - [13/Jun/2018 15:48:53] "GET / HTTP/1.1" 200 -

127.0.0.1 - - [13/Jun/2018 15:48:53] "GET /favicon.ico HTTP/1.1" 404 -

Using browser on server (http://localhost:5000)is able to load test page

This is where I am at...

After all that is addressed, we will see if any issues with permissions. That is, whether the Apache user can actually read everything in your home directory.

Graham

[Rajeev Jain]

It is probably better to have run:

    mod_wsgi-express module-config

That should give you a LoadModule line for loading the mod_wsgi.so file from the place it is installed, as well as a LoadFile which loads the correct Python shared library for Anaconda, so the system Python library isn't picked up by mistake. Add the output of that into wsgi.load in place of what you have.

Done.

Some logs to verify done correctly:

mod_wsgi-express module-config

LoadModule wsgi_module "/home/rajeev/anaconda3/lib/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so"

WSGIPythonHome "/home/rajeev/anaconda3"

cat wsgi.load 

LoadModule wsgi_module "/home/rajeev/anaconda3/lib/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so"

 

Instead of WSGIPythonHome and WSGIPythonPath here, have:

    WSGIRestrictEmbedded On

That will turn off embedded mode and if configuration is wrong and application is not delegated to daemon mode processes correctly, you will get a 500 error to tell you you have an issue.

Done.

    WSGIDaemonProcess flaskapp.com python-home=/home/rajeev/anaconda3

    WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi process-group=flaskapp.com application-group=%{GLOBAL}

    <Directory /var/www/FlaskApp>

    <Files flaskapp.wsgi>

    Require all granted

    </Files>

    </Directory>

    Alias /static /var/www/FlaskApp/FlaskApp/static 

    <Directory /var/www/FlaskApp/FlaskApp/static/>

Order allow,deny

Allow from all

    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    LogLevel warn

    CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

When using Anaconda Python, you cannot used mod_ssl in Apache. This is because Anaconda Python ships its own SSL libraries which aren't compatible, and by having mod_ssl loaded, the system SSL libraries will get loaded first, resulting in any use of SSL in Python to fail.

If not using SSL in Apache, disable mod_ss

 

l. If you are using it, the only way to do it is to use mod_wsgi-express to create a distinct instance and have the main Apache proxy to it.

I disabled the SSL

/etc/apache2/sites-available$ sudo a2dismod ssl

Module ssl disabled.

To activate the new configuration, you need to run:

  service apache2 restart

After all that is addressed, we will see if any issues with permissions. That is, whether the Apache user can actually read everything in your home directory.

A few observations:

1) I made the above changes. After rebooting the Ubuntu and logging in it reported "System program problem detected"

2) On the Ubuntu:

curl -sH 'Host: flaskapp.com' localhost:83|grep title

<title>500 Internal Server Error</title>

tail -l /var/log/apache2/error.log

[Wed Jun 13 17:50:04.857839 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]     from flask import Flask

[Wed Jun 13 17:50:04.857844 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/__init__.py", line 21, in <module>

[Wed Jun 13 17:50:04.857847 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]     from .app import Flask, Request, Response

[Wed Jun 13 17:50:04.857852 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/app.py", line 25, in <module>

[Wed Jun 13 17:50:04.857854 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]     from . import cli, json

[Wed Jun 13 17:50:04.857859 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/cli.py", line 18, in <module>

[Wed Jun 13 17:50:04.857862 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]     import ssl

[Wed Jun 13 17:50:04.857866 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]   File "/home/rajeev/anaconda3/lib/python3.6/ssl.py", line 101, in <module>

[Wed Jun 13 17:50:04.857869 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208]     import _ssl             # if we can't import it, let the error propagate

[Wed Jun 13 17:50:04.857884 2018] [wsgi:error] [pid 2177] [remote 127.0.0.1:60208] ImportError: /home/rajeev/anaconda3/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so: undefined symbol: SSLv2_method

I verified ssl was disabled:

sudo a2dismod ssl

[sudo] password for rajeev: 

Module ssl already disabled

I'm concerned what change is generating that "System program problem detected" and why its still complaining about SSL sysmbol even though we disabled the mod_ssl.

Suggestions, next steps? We should be close...just a configuration issue, yes?

 --Rajeev

[Graham Dumpleton]

There may be another Apache module which is dragging in the SSL libraries.

Is PHP module enabled?

What modules are linked in the 'mods-enabled' directory?

Graham

[Rajeev Jain]

Here is a more complete error log:

[Wed Jun 13 17:55:33.173679 2018] [mpm_prefork:notice] [pid 2174] AH00169: caught SIGTERM, shutting down

[Wed Jun 13 17:55:34.421101 2018] [mpm_prefork:notice] [pid 2501] AH00163: Apache/2.4.18 (Ubuntu) mod_wsgi/4.6.4 Python/3.6 configured -- resuming normal operations

[Wed Jun 13 17:55:34.421158 2018] [core:notice] [pid 2501] AH00094: Command line: '/usr/sbin/apache2'

[Wed Jun 13 17:55:34.576866 2018] [wsgi:error] [pid 2504] mod_wsgi (pid=2504): Failed to exec Python script file '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 17:55:34.576923 2018] [wsgi:error] [pid 2504] mod_wsgi (pid=2504): Exception occurred processing WSGI script '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 17:55:34.578129 2018] [wsgi:error] [pid 2504] Traceback (most recent call last):

[Wed Jun 13 17:55:34.578169 2018] [wsgi:error] [pid 2504]   File "/var/www/FlaskApp/flaskapp.wsgi", line 7, in <module>

[Wed Jun 13 17:55:34.578182 2018] [wsgi:error] [pid 2504]     from FlaskApp import app as application

[Wed Jun 13 17:55:34.578189 2018] [wsgi:error] [pid 2504]   File "/var/www/FlaskApp/FlaskApp/__init__.py", line 2, in <module>

[Wed Jun 13 17:55:34.578192 2018] [wsgi:error] [pid 2504]     from flask import Flask

[Wed Jun 13 17:55:34.578197 2018] [wsgi:error] [pid 2504]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/__init__.py", line 21, in <module>

[Wed Jun 13 17:55:34.578200 2018] [wsgi:error] [pid 2504]     from .app import Flask, Request, Response

[Wed Jun 13 17:55:34.578205 2018] [wsgi:error] [pid 2504]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/app.py", line 25, in <module>

[Wed Jun 13 17:55:34.578207 2018] [wsgi:error] [pid 2504]     from . import cli, json

[Wed Jun 13 17:55:34.578212 2018] [wsgi:error] [pid 2504]   File "/home/rajeev/anaconda3/lib/python3.6/site-packages/flask/cli.py", line 18, in <module>

[Wed Jun 13 17:55:34.578218 2018] [wsgi:error] [pid 2504]     import ssl

[Wed Jun 13 17:55:34.578223 2018] [wsgi:error] [pid 2504]   File "/home/rajeev/anaconda3/lib/python3.6/ssl.py", line 101, in <module>

[Wed Jun 13 17:55:34.578225 2018] [wsgi:error] [pid 2504]     import _ssl             # if we can't import it, let the error propagate

[Wed Jun 13 17:55:34.578243 2018] [wsgi:error] [pid 2504] ImportError: /home/rajeev/anaconda3/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so: undefined symbol: SSLv2_method

As a side note, I have 2 other php virtualhosts running on the same ubuntu box and pages are being served correctly.

Server is complaining about => /var/www/FlaskApp/flaskapp.wsgi

How can that file be unit tested?

[Rajeev Jain]

FYI,

/etc/apache2/mods-available$ ls

access_compat.load  authn_file.load       cache_disk.conf     deflate.conf     heartmonitor.load         macro.load        php7.0.load          proxy.load           session.load           suexec.load

actions.conf        authn_socache.load    cache_disk.load     deflate.load     ident.load                mime.conf         proxy_ajp.load       proxy_scgi.load      setenvif.conf          unique_id.load

actions.load        authnz_fcgi.load      cache.load          dialup.load      include.load              mime.load         proxy_balancer.conf  proxy_wstunnel.load  setenvif.load          userdir.conf

alias.conf          authnz_ldap.load      cache_socache.load  dir.conf         info.conf                 mime_magic.conf   proxy_balancer.load  ratelimit.load       slotmem_plain.load     userdir.load

alias.load          authz_core.load       cgid.conf           dir.load         info.load                 mime_magic.load   proxy.conf           reflector.load       slotmem_shm.load       usertrack.load

allowmethods.load   authz_dbd.load        cgid.load           dump_io.load     lbmethod_bybusyness.load  mpm_event.conf    proxy_connect.load   remoteip.load        socache_dbm.load       vhost_alias.load

asis.load           authz_dbm.load        cgi.load            echo.load        lbmethod_byrequests.load  mpm_event.load    proxy_express.load   reqtimeout.conf      socache_memcache.load  wsgi.conf

auth_basic.load     authz_groupfile.load  charset_lite.load   env.load         lbmethod_bytraffic.load   mpm_prefork.conf  proxy_fcgi.load      reqtimeout.load      socache_shmcb.load     wsgi.load

auth_digest.load    authz_host.load       data.load           expires.load     lbmethod_heartbeat.load   mpm_prefork.load  proxy_fdpass.load    request.load         speling.load           xml2enc.load

auth_form.load      authz_owner.load      dav_fs.conf         ext_filter.load  ldap.conf                 mpm_worker.conf   proxy_ftp.conf       rewrite.load         ssl.conf

authn_anon.load     authz_user.load       dav_fs.load         file_cache.load  ldap.load                 mpm_worker.load   proxy_ftp.load       sed.load             ssl.load

authn_core.load     autoindex.conf        dav.load            filter.load      log_debug.load            negotiation.conf  proxy_html.conf      session_cookie.load  status.conf

authn_dbd.load      autoindex.load        dav_lock.load       headers.load     log_forensic.load         negotiation.load  proxy_html.load      session_crypto.load  status.load

authn_dbm.load      buffer.load           dbd.load            heartbeat.load   lua.load                  php7.0.conf       proxy_http.load      session_dbd.load     substitute.load

[Graham Dumpleton]

Those PHP sites are likely going to be the issue then as the PHP modules are likely pulling in the SSL libraries. It could also be the LDAP related modules, but more likely the PHP modules.

What Anaconda Python does with supplying its own SSL libraries, as well as other replacements for system libraries is a right pain and is going to break any system where Python is being embedded in another process, such as is the case with mod_wsgi and Apache. So isn't just the SSL libraries. Because PHP can pull in image libraries, that can conflict with those in Anaconda Python if using image manipulation packages like Pillow (PIL).

If you have the existing PHP sites I don't really see a solution besides running mod_wsgi-express to create a separate Apache/mod_wsgi instance and then proxy that behind the main Apache.

Some links which talk about mod_wsgi-express and running it like this are below. Note that when using mod_wsgi-express it will automatically configure the separate Apache instance for you, so it isn't as bad as it may seem.

Introduction in:

* http://blog.dscpl.com.au/2015/04/introducing-modwsgi-express.html

Details on having it behind a proxy. Ignore that it talks about Docker, is still relevant.

* http://blog.dscpl.com.au/2015/06/proxying-to-python-web-application.html

* http://blog.dscpl.com.au/2015/07/redirection-problems-when-proxying-to.html

Pre-create config so easier to integrate with system startup scripts.

* https://github.com/GrahamDumpleton/mod_wsgi#running-mod_wsgi-express-as-root

Ignore that mentions root. You would need to run it on port other than 80 and so doesn't need to run as root.

So sorry, don't have a simple solution if you want to run with Anaconda Python because of what Anaconda Python does with supplying its own libraries which conflict with system libraries.

Graham

[Rajeev Jain]

I want to thank you for the time you spent with me. It is clear, using Anaconda with Apache is not a good idea. My objective now is to setup a virtual environment, and then configure a Apache virtual host to use it. There is a multitude of half complete solutions out there to accomplish this. Would you be kind enough to point me to from your perspective the best guide for this objective? I want to make sure the essential parameters are correctly set.

Any examples for the following files would very helpful:

- wsgi.conf

- wsgi.load

- myhost.conf

I guess at this point wsgi.conf should contain nothing, wsgi.load would load the appropriate mod_wsgi. Most of the magic will be in the virtualhost.conf file, yes?  

I believe this to be a good path for getting this to work.

thank-you again.

--Rajeev

[Graham Dumpleton]

Using a virtual environment will not help if you intend to still use Anaconda Python. You will still have the issue with the SSL libraries.

Are you thinking that a virtual environment will somehow solve the problem?

On 14 Jun 2018, at 11:34 am, Rajeev Jain <jain...@gmail.com> wrote:

I want to thank you for the time you spent with me. It is clear, using Anaconda with Apache is not a good idea. My objective now is to setup a virtual environment, and then configure a Apache virtual host to use it. There is a multitude of half complete solutions out there to accomplish this. Would you be kind enough to point me to from your perspective the best guide for this objective? I want to make sure the essential parameters are correctly set.

Any examples for the following files would very helpful:

- wsgi.conf

Would stay the same.

- wsgi.load

Would still be output of mod_wsgi-express module-config.

The difference would be that you would pip install mod_wsgi into the virtual environment first and run mod_wsgi-express module-config from there so loading module from correct place.

- myhost.conf

Only difference here would be what python-home is set to.

Should be value of sys.prefix for the virtual environment.

Details in:

* http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

Note I missed one thing in the config.

You also likely needed python-path option to WSGIDaemonProcess as well. Probably:

    python-path=/var/www/FlaskApp

This is in addition to python-home.

The python-path says where your project code is.

If you use mod_wsgi-express as standalone server behind front end Apache proxy, then it will pretty well worry about this stuff for you. Just run it in the right directory and additional tell it where your static files should be mounted.

[Rajeev Jain]

To be clear, I'll create a virtual environment and use standard python3.6 distribution (i.e. non anaconda). I'll follow the suggestions you have laid out here...

[Rajeev Jain]

Latest update:

No virtual environment. Python3.6 is now installed into the system folder:

python location:

/etc/apache2/sites-available$ python --version

Python 3.6.5

python

Python 3.6.5 (default, May  3 2018, 10:08:28) 

[GCC 5.4.0 20160609] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import sys

>>> print(sys.prefix)

/usr

Flask location:

>>> import flask

>>> print(flask.__file__)

/usr/local/lib/python3.6/dist-packages/flask/__init__.py

mod_wsgi-express module-config

LoadModule wsgi_module "/usr/local/lib/python3.6/dist-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so"

WSGIPythonHome "/usr"

cat wsgi.load 

LoadModule wsgi_module /usr/local/lib/python3.6/dist-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so 

/etc/apache2/sites-available$ cat FlaskApp.conf 

<VirtualHost *:83>

    ServerName flaskapp.com

    WSGIDaemonProcess flaskapp.com python-home=/usr python-path=/var/www/FlaskApp

    WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi process-group=flaskapp.com application-group=%{GLOBAL}

    <Directory /var/www/FlaskApp/>

    <Files flaskapp.wsgi>

    Require all granted

    </Files>

    </Directory>

</VirtualHost>

/etc/apache2/mods-available$ cat wsgi.conf

<IfModule mod_wsgi.c>

    WSGIRestrictEmbedded On

</IfModule>

Testing:

curl -sH 'Host: flaskapp.com' localhost:83|grep title

<title>500 Internal Server Error</title>

tail -l /var/log/apache2/error.log

[Wed Jun 13 22:18:08.678744 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]     from werkzeug.exceptions import abort

[Wed Jun 13 22:18:08.678749 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/__init__.py", line 151, in <module>

[Wed Jun 13 22:18:08.678752 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]     __import__('werkzeug.exceptions')

[Wed Jun 13 22:18:08.678756 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/exceptions.py", line 67, in <module>

[Wed Jun 13 22:18:08.678759 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]     from werkzeug._internal import _get_environ

[Wed Jun 13 22:18:08.678764 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/_internal.py", line 15, in <module>

[Wed Jun 13 22:18:08.678766 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]     from datetime import datetime, date

[Wed Jun 13 22:18:08.678771 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]   File "/usr/lib/python3.6/datetime.py", line 8, in <module>

[Wed Jun 13 22:18:08.678773 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590]     import math as _math

[Wed Jun 13 22:18:08.678786 2018] [wsgi:error] [pid 3095] [remote 127.0.0.1:57590] ModuleNotFoundError: No module named 'math'

Running FlaskApp on its own:

/var/www/FlaskApp/FlaskApp$ python __init__.py 

 * Serving Flask app "__init__" (lazy loading)

 * Environment: production

   WARNING: Do not use the development server in a production environment.

   Use a production WSGI server instead.

 * Debug mode: off

 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

127.0.0.1 - - [13/Jun/2018 22:25:23] "GET / HTTP/1.1" 200 -

127.0.0.1 - - [13/Jun/2018 22:25:23] "GET /favicon.ico HTTP/1.1" 404 -

curl -sH 'Host: flaskapp.com' localhost:5000

Hello, this is running from flasK!

As you can see the FlaskApp is running fine.

Python and all modules are now installed in system locations and they are running fine.

Apache2 is running fine. I have 2 other virtualhosts running serving php/html pages

Still WSGI is still not setup correctly.

I'm shocked at the lack of decent documentation to get WCGI enabled on a Apache2 server. What am I missing??

Don't you have some basic test code or files one can run to verify the WSGI is setup correctly?

Please advise.

--Rajeev

 

[Graham Dumpleton]

How did you install Python 3.6?

Unless that is a very up to date Ubuntu version, which I would doubt AWS would be using if this is one of their images, you would need to install it from source code. Installing Python from source code yourself can be a bit tricky on Linux as the default configure options aren't what you need. Also not a good idea to install from Python source code into system directories as it is then hard to extract it out if you need to remove it.

There is a whole web site on it at:

http://modwsgi.readthedocs.io/en/develop/

It just doesn't cover the specifics of particular ways that Linux distros need things done because they depart from standard conventions for Apache and do things their own way. Lot of work to document how every different Linux distro does it.

Don't you have some basic test code or files one can run to verify the WSGI is setup correctly?

Usually if you can get the WSGI hello world program working you are good.

* http://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html

There are various checks you can run in:

* http://modwsgi.readthedocs.io/en/develop/user-guides/checking-your-installation.html

This more for when debugging issues when things don't work.

Graham

Please advise.

--Rajeev

 

[Rajeev Jain]

On Jun 13, 2018, at 10:42 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

How did you install Python 3.6?

followed instructions here:

https://askubuntu.com/questions/889535/how-to-install-pip-for-python-3-6-on-ubuntu-16-10

Let's suppose that you have a system running Ubuntu 16.04, 16.10, or 17.04, and you want Python 3.6 to be the default Python.

If you're using Ubuntu 16.04 LTS, you'll need to use a PPA:

sudo add-apt-repository ppa:jonathonf/python-3.6  # (only for 16.04 LTS)

Then, run the following (this works out-of-the-box on 16.10 and 17.04):

sudo apt update
sudo apt install python3.6
sudo apt install python3.6-dev
sudo apt install python3.6-venv
wget https://bootstrap.pypa.io/get-pip.py
sudo python3.6 get-pip.py
sudo ln -s /usr/bin/python3.6 /usr/local/bin/python3
sudo ln -s /usr/local/bin/pip /usr/local/bin/pip3

# Do this only if you want python3 to be the default Python
# instead of python2 (may be dangerous, esp. before 2020):
# sudo ln -s /usr/bin/python3.6 /usr/local/bin/python

When you have completed all of the above, each of the following shell commands should indicate Python 3.6.1 (or a more recent version of Python 3.6):

python --version   # (this will reflect your choice, see above)
python3 --version
$(head -1 `which pip` | tail -c +3) --version
$(head -1 `which pip3` | tail -c +3) --version

This is running on a local Ubuntu 16.04 computer. 

python

Python 3.6.5 (default, May  3 2018, 10:08:28) 

[GCC 5.4.0 20160609] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import math

[Graham Dumpleton]

Odd.

If you run python3.6 and enter in the interpreter:

    import datetime

it works?

Graham

[Rajeev Jain]

yes.

/etc/apache2/sites-available$ python

Python 3.6.5 (default, May  3 2018, 10:08:28) 

[GCC 5.4.0 20160609] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import datetime

>>> 

—Rajeev

[Graham Dumpleton]

From root, can you su to the user for Apache and try the same thing.

Graham

[Rajeev Jain]

root is not enabled. but i guess i can enable it for testing purposes.

[Graham Dumpleton]

Instead then. Print out sys.path from the interpreter and then also from your WSGI script file.

[Rajeev Jain]

on Jun 13, 2018, at 10:58 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

Instead then. Print out sys.path from the interpreter and then also from your WSGI script file.

python 

Python 3.6.5 (default, May  3 2018, 10:08:28) 

[GCC 5.4.0 20160609] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import sys

>>> print(sys.path)

['', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages']

>>>

cat flaskapp.wsgi 

#!/usr/local/python

import sys

import logging

logging.basicConfig(stream=sys.stderr)

sys.path.insert(0,"/var/www/FlaskApp/")

from FlaskApp import app as application

application.secret_key = 'Add your secret key'

[Graham Dumpleton]

On 14 Jun 2018, at 4:06 pm, Rajeev Jain <jain...@gmail.com> wrote:

on Jun 13, 2018, at 10:58 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

Instead then. Print out sys.path from the interpreter and then also from your WSGI script file.

python 

Python 3.6.5 (default, May  3 2018, 10:08:28) 

[GCC 5.4.0 20160609] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import sys

>>> print(sys.path)

['', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages']

>>>

cat flaskapp.wsgi 

#!/usr/local/python

import sys

import logging

logging.basicConfig(stream=sys.stderr)

sys.path.insert(0,"/var/www/FlaskApp/")

Can you add here:

    print('VERSION', sys.version)

    print('PREFIX', sys.prefix)

    print('PATH', sys.path)

and then extract what was logged for it from the Apache error log.

That way can compare what is being set under mod_wsgi compared to the command line.

[Rajeev Jain]

here is updated flask.wsgi

here is error.log

[Wed Jun 13 23:23:03.445837 2018] [wsgi:error] [pid 4758] ModuleNotFoundError: No module named 'math'

[Wed Jun 13 23:23:15.854525 2018] [wsgi:error] [pid 4758] VERSION 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:17:00) 

[Wed Jun 13 23:23:15.854556 2018] [wsgi:error] [pid 4758] [GCC 7.2.0]

[Wed Jun 13 23:23:15.854563 2018] [wsgi:error] [pid 4758] 

[Wed Jun 13 23:23:15.854581 2018] [wsgi:error] [pid 4758] PREFIX /usr

[Wed Jun 13 23:23:15.854599 2018] [wsgi:error] [pid 4758] PATH ['/var/www/FlaskApp/', '/var/www/FlaskApp/', '/var/www/FlaskApp', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages']

[Wed Jun 13 23:23:15.856916 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696] mod_wsgi (pid=4758): Failed to exec Python script file '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 23:23:15.856949 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696] mod_wsgi (pid=4758): Exception occurred processing WSGI script '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 23:23:15.857057 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696] Traceback (most recent call last):

[Wed Jun 13 23:23:15.857091 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]   File "/var/www/FlaskApp/flaskapp.wsgi", line 11, in <module>

[Wed Jun 13 23:23:15.857095 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]     from FlaskApp import app as application

[Wed Jun 13 23:23:15.857100 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]   File "/var/www/FlaskApp/FlaskApp/__init__.py", line 2, in <module>

[Wed Jun 13 23:23:15.857103 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]     from flask import Flask

[Wed Jun 13 23:23:15.857108 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]   File "/usr/local/lib/python3.6/dist-packages/flask/__init__.py", line 17, in <module>

[Wed Jun 13 23:23:15.857110 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]     from werkzeug.exceptions import abort

[Wed Jun 13 23:23:15.857115 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/__init__.py", line 151, in <module>

[Wed Jun 13 23:23:15.857117 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]     __import__('werkzeug.exceptions')

[Wed Jun 13 23:23:15.857122 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/exceptions.py", line 67, in <module>

[Wed Jun 13 23:23:15.857124 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]     from werkzeug._internal import _get_environ

[Wed Jun 13 23:23:15.857129 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/_internal.py", line 15, in <module>

[Wed Jun 13 23:23:15.857131 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]     from datetime import datetime, date

[Wed Jun 13 23:23:15.857135 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]   File "/usr/lib/python3.6/datetime.py", line 8, in <module>

[Wed Jun 13 23:23:15.857138 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696]     import math as _math

[Wed Jun 13 23:23:15.857150 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57696] ModuleNotFoundError: No module named 'math'

[Wed Jun 13 23:23:30.077689 2018] [wsgi:error] [pid 4758] VERSION 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:17:00) 

[Wed Jun 13 23:23:30.077714 2018] [wsgi:error] [pid 4758] [GCC 7.2.0]

[Wed Jun 13 23:23:30.077720 2018] [wsgi:error] [pid 4758] 

[Wed Jun 13 23:23:30.077729 2018] [wsgi:error] [pid 4758] PREFIX /usr

[Wed Jun 13 23:23:30.077745 2018] [wsgi:error] [pid 4758] PATH ['/var/www/FlaskApp/', '/var/www/FlaskApp/', '/var/www/FlaskApp/', '/var/www/FlaskApp', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages']

[Wed Jun 13 23:23:30.080656 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698] mod_wsgi (pid=4758): Failed to exec Python script file '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 23:23:30.080701 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698] mod_wsgi (pid=4758): Exception occurred processing WSGI script '/var/www/FlaskApp/flaskapp.wsgi'.

[Wed Jun 13 23:23:30.080846 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698] Traceback (most recent call last):

[Wed Jun 13 23:23:30.080893 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]   File "/var/www/FlaskApp/flaskapp.wsgi", line 11, in <module>

[Wed Jun 13 23:23:30.080907 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]     from FlaskApp import app as application

[Wed Jun 13 23:23:30.080916 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]   File "/var/www/FlaskApp/FlaskApp/__init__.py", line 2, in <module>

[Wed Jun 13 23:23:30.080921 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]     from flask import Flask

[Wed Jun 13 23:23:30.080927 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]   File "/usr/local/lib/python3.6/dist-packages/flask/__init__.py", line 17, in <module>

[Wed Jun 13 23:23:30.080931 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]     from werkzeug.exceptions import abort

[Wed Jun 13 23:23:30.080937 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/__init__.py", line 151, in <module>

[Wed Jun 13 23:23:30.080941 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]     __import__('werkzeug.exceptions')

[Wed Jun 13 23:23:30.080947 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/exceptions.py", line 67, in <module>

[Wed Jun 13 23:23:30.080951 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]     from werkzeug._internal import _get_environ

[Wed Jun 13 23:23:30.080957 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]   File "/usr/local/lib/python3.6/dist-packages/werkzeug/_internal.py", line 15, in <module>

[Wed Jun 13 23:23:30.080961 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]     from datetime import datetime, date

[Wed Jun 13 23:23:30.080967 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]   File "/usr/lib/python3.6/datetime.py", line 8, in <module>

[Wed Jun 13 23:23:30.080971 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698]     import math as _math

[Wed Jun 13 23:23:30.080988 2018] [wsgi:error] [pid 4758] [remote 127.0.0.1:57698] ModuleNotFoundError: No module named 'math'

How is Anaconda being picked up?

Can we check PATH setting for Apache boot ?

[Graham Dumpleton]

Do:

    pip uninstall mod_wsgi

Repeat until says not installed.

Then reinstall it.

Run 'ldd' on the mod_wsgi .so file that is built to see what Python shared library it is finding.

The installed version is possibly compiled against Anaconda Python somehow.

Also do a complete Apache stop and start, rather than reload. There is also a chance the Anaconda Python library hasn't unloaded from Apache properly.

Graham

[Rajeev Jain]

did that and tried to restart the server. getting an error:

sudo service apache2 restart

Job for apache2.service failed because the control process exited with error code. See "systemctl status apache2.service" and "journalctl -xe" for details.

rajeev@cubi-ubuntu:/var/www/FlaskApp/FlaskApp$ systemctl status apache2.service

● apache2.service - LSB: Apache2 web server

   Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)

  Drop-In: /lib/systemd/system/apache2.service.d

           └─apache2-systemd.conf

   Active: failed (Result: exit-code) since Thu 2018-06-14 00:03:05 PDT; 8s ago

     Docs: man:systemd-sysv-generator(8)

  Process: 5405 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)

  Process: 5531 ExecStart=/etc/init.d/apache2 start (code=exited, status=1/FAILURE)

Jun 14 00:03:05 cubi-ubuntu apache2[5531]:  * The apache2 configtest failed.

Jun 14 00:03:05 cubi-ubuntu apache2[5531]: Output of config test was:

Jun 14 00:03:05 cubi-ubuntu apache2[5531]: AH00526: Syntax error on line 5 of /etc/apache2/sites-enabled/FlaskApp.conf:

Jun 14 00:03:05 cubi-ubuntu apache2[5531]: Invalid command 'WSGIDaemonProcess', perhaps misspelled or defined by a module not included in the server configuration

Jun 14 00:03:05 cubi-ubuntu apache2[5531]: Action 'configtest' failed.

Jun 14 00:03:05 cubi-ubuntu apache2[5531]: The Apache error log may have more information.

Jun 14 00:03:05 cubi-ubuntu systemd[1]: apache2.service: Control process exited, code=exited status=1

Jun 14 00:03:05 cubi-ubuntu systemd[1]: Failed to start LSB: Apache2 web server.

Jun 14 00:03:05 cubi-ubuntu systemd[1]: apache2.service: Unit entered failed state.

Jun 14 00:03:05 cubi-ubuntu systemd[1]: apache2.service: Failed with result 'exit-code’.

/etc/apache2/sites-available$ cat FlaskApp.conf 

<VirtualHost *:83>

    ServerName flaskapp.com

    WSGIDaemonProcess flaskapp.com python-home=/usr

    WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi process-group=flaskapp.com application-group=%{GLOBAL}

    <Directory /var/www/FlaskApp/>

    <Files flaskapp.wsgi>

    Require all granted

    </Files>

    </Directory>

</VirtualHost>

Do you see anything wrong?

—Rajeev

[Graham Dumpleton]

Check your wsgi.load contents is correct. Ie., current output of running:

    mod_wsgi-express module-config

for version installed in system Python 3.6.

Check that mods-enabled is symlink to it and not copy with wrong value.

The error indicates module wasn't loaded.

[Rajeev Jain]

Enabling WSGI on Apache2/Ubuntu16.04 is a very popular configuration. It should not be this difficult.

What python version do you want me to use?

Where do you want it installed?

Please tell me precisely what steps to follow to get this running?

Please send me the following 2 files correctly and fully populated:

- wsgi.conf

- virtualhost.conf

I'll revisit this in the morning.

thanks,

—Rajeev

[Graham Dumpleton]

On 14 Jun 2018, at 6:06 pm, Rajeev Jain <jain...@gmail.com> wrote:

Enabling WSGI on Apache2/Ubuntu16.04 is a very popular configuration. It should not be this difficult.

It was complicated by starting out with trying to use Anaconda Python. That has confused things along the way.

What python version do you want me to use?

Since there is a system package for Python 3.6, use it.

Where do you want it installed?

You already have it installed.

Please tell me precisely what steps to follow to get this running?

I already explained the steps.

Since though Python 3.6 does appear to be available as a system package, since you are having trouble with using pip install method, see if there is also a system Python 3.6 package for mod_wsgi.

Disable the mod_wsgi module using:

    a2dismod wsgi

Make a copy of the wsgi.load file just in case need it later. Then remove the wsgi.load file.

For good measure make sure you uninstall mod_wsgi for Python 2 in case it is still installed.

    apt-get uninstall libapache2-mod-wsgi

Ensure you remove any stuff you put in /usr/lib/apache2/modules previously for mod_wsgi:

-rwxr-xr-x 1 root root 974744 Jun 13 14:45 mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so*
lrwxrwxrwx 1 root root     45 Jun 13 14:47 mod_wsgi.so -> mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so*
-rw-r--r-- 1 root root 207952 Jan 25  2016 mod_wsgi.so-3.5

Then run:

    apt-get install libapache2-mod-wsgi-py3

This should install mod_wsgi for Python 3 using system package.

Run ldd on the mod_wsgi module in /usr/lib/apache2/modules/ to make sure what Python version it is linking to.

Enable mod_wsgi.

    a2enmod wsgi

This has taken you back to stock standard system packages.

Check that wsgi.conf still has:

    WSGIRestrictEmbedded on

Everything in virtualhost.conf should be same as what I told you to use previously. Remember my follow comment about setting python-path as well as python-home.

Graham

[Rajeev Jain]

Sorry for my frustration. You are correct, installation was complicated by several factors. Your notes are very helpfulI and i really appreciate.  Im going to cross check my work today and will provide an update later today.

===

Rajeev Jain

Jain...@yahoo.com

[Rajeev Jain]

I have one question regarding the mod_wsgi doc:

http://modwsgi.readthedocs.io/en/develop/index.html

Its mentioned in the project status “In general, the documentation is in a bit of a mess right now and somewhat outdated”.

Can the info in the getting started section be relied on or not. The reason i ask is i don't want to waste time following outdated or in accurate info:

http://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html

If accurate, i’ll spend time to follow that section. I like how a simple app is provided to unit test the setup. Can that be followed?

-Rajeev

===

Rajeev Jain

Jain...@yahoo.com

[Graham Dumpleton]

The only thing which doesn't apply in your case is that the examples use:

<Directory /usr/local/www/wsgi-scripts>

Order allow,deny
Allow from all
</Directory>

This is old Apache 2.2 way of doing it.

Use what you have now for Apache 2.4 of:

<Directory /usr/local/www/wsgi-scripts>
Require all granted
</Directory>

The comment about being out of date is more that there are various new features things are explained except in configuration settings docs. So no examples for them beyond that. The core information is still valid except for that one Apache 2.2/2.4 difference.

So where you see me using options to directives in email you don't see in setting up guide, check the directives documentation under:

http://modwsgi.readthedocs.io/en/develop/configuration.html

Graham

[Rajeev Jain]

Ok good. 

===

Rajeev Jain 

From mobile

[Rajeev Jain]

I’ve decided to scrap this effort using Anaconda python. You’ll see a new post

thx,

—Rajeev

[Rajeev Jain]

background: my system is Ubuntu16.04 LTS running on a home network behind a firewall. Apache2 server is installed and successfully hosting 3 virtual hosts using ports 80, 81, 82. These 3 sites are working 100% fine. My objective is to setup a 4th virtual host using mod_wsgi for serving a python3.6 based Flask Application.

After many trials and tribulations the 4th virtual host is now partially working...the site is successfully served when using the curl command but the site is not accessible when using a browser.

Here is the tutorial I used:

https://www.digitalocean.com/community/tutorials/how-to-deploy-a-flask-application-on-an-ubuntu-vps

Here are my test results:

Using curl:

curl http://192.168.1.14:83

Hello, this is running from flasK!

Using a browser:

I just rebooted my server and now the 4th virtual host is not even working with curl. Are there any out who have this desired configuration working?

I relatively confident the python3.6 and the mod_wsgi modules are correctly installed and some configuration settings are not correct.

Any guidance or trouble-shooting steps would be greatly appreciated. I’m happy to provide any kind of details required to get this working.

TIA,

—Rajeev

[Graham Dumpleton]

Would need to see the VirtualHost definition including what is in it.

Also, was that VirtualHost in its own file in the sites-available directory and was it enabled so that it is linked into the sites-enabled directory and would be found.

Graham

On 24 Jun 2018, at 7:55 am, 'Rajeev Jain' via modwsgi <mod...@googlegroups.com> wrote:

background: my system is Ubuntu16.04 LTS running on a home network behind a firewall. Apache2 server is installed and successfully hosting 3 virtual hosts using ports 80, 81, 82. These 3 sites are working 100% fine. My objective is to setup a 4th virtual host using mod_wsgi for serving a python3.6 based Flask Application.

After many trials and tribulations the 4th virtual host is now partially working...the site is successfully served when using the curl command but the site is not accessible when using a browser.

Here is the tutorial I used:

https://www.digitalocean.com/community/tutorials/how-to-deploy-a-flask-application-on-an-ubuntu-vps

Here are my test results:

Using curl:

curl http://192.168.1.14:83

Hello, this is running from flasK!

Using a browser:

<PastedGraphic-1.png>

I just rebooted my server and now the 4th virtual host is not even working with curl. Are there any out who have this desired configuration working?

I relatively confident the python3.6 and the mod_wsgi modules are correctly installed and some configuration settings are not correct.

Any guidance or trouble-shooting steps would be greatly appreciated. I’m happy to provide any kind of details required to get this working.

TIA,

—Rajeev

[Rajeev Jain]

Hi Graham, thank-you for your reply.

1) wsgi.load

cat /etc/apache2/mods-available/wsgi.load

LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so 

rajeev@cubi-ubuntu:~$ ldd /usr/lib/apache2/modules/mod_wsgi.so

linux-vdso.so.1 =>  (0x00007fffdf7af000)

libpython3.6m.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0 (0x00007f494f3a6000)

libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f494f189000)

libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f494edbf000)

libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f494eb96000)

libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f494e97c000)

libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f494e778000)

libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f494e575000)

libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f494e26c000)

/lib64/ld-linux-x86-64.so.2 (0x00007f494fca7000)

2) wsgi.conf

cat /etc/apache2/mods-available/wsgi.conf

<IfModule mod_wsgi.c>

    

    #WSGIRestrictEmbedded on

    WSGIPythonHome "/usr"

    #This config file is provided to give an overview of the directives,

    #which are only allowed in the 'server config' context.

    #For a detailed description of all avaiable directives please read

    #http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives

    #WSGISocketPrefix: Configure directory to use for daemon sockets.

    #

    #Apache's DEFAULT_REL_RUNTIMEDIR should be the proper place for WSGI's

    #Socket. In case you want to mess with the permissions of the directory,

    #you need to define WSGISocketPrefix to an alternative directory.

    #See http://code.google.com/p/modwsgi/wiki/ConfigurationIssues for more

    #information

        #WSGISocketPrefix /var/run/apache2/wsgi

    

    #WSGIPythonOptimize: Enables basic Python optimisation features.

    #

    #Sets the level of Python compiler optimisations. The default is '0'

    #which means no optimisations are applied.

    #Setting the optimisation level to '1' or above will have the effect

    #of enabling basic Python optimisations and changes the filename

    #extension for compiled (bytecode) files from .pyc to .pyo.

    #When the optimisation level is set to '2', doc strings will not be

    #generated and retained. This will result in a smaller memory footprint,

    #but may cause some Python packages which interrogate doc strings in some

    #way to fail. 

        #WSGIPythonOptimize 0

    #WSGIPythonPath: Additional directories to search for Python modules,

    #                overriding the PYTHONPATH environment variable.

    #

    #Used to specify additional directories to search for Python modules.

    #If multiple directories are specified they should be separated by a ':'.

        #WSGIPythonPath directory|directory-1:directory-2:...

    #WSGIPythonEggs: Directory to use for Python eggs cache.

    #

    #Used to specify the directory to be used as the Python eggs cache

    #directory for all sub interpreters created within embedded mode.

    #This directive achieves the same affect as having set the

    #PYTHON_EGG_CACHE environment variable.

    #Note that the directory specified must exist and be writable by the user

    #that the Apache child processes run as. The directive only applies to

    #mod_wsgi embedded mode. To set the Python eggs cache directory for

    #mod_wsgi daemon processes, use the 'python-eggs' option to the

    #WSGIDaemonProcess directive instead. 

        #WSGIPythonEggs directory

    #WSGIRestrictEmbedded: Enable restrictions on use of embedded mode.

    #

    #The WSGIRestrictEmbedded directive determines whether mod_wsgi embedded

    #mode is enabled or not. If set to 'On' and the restriction on embedded

    #mode is therefore enabled, any attempt to make a request against a

    #WSGI application which hasn't been properly configured so as to be

    #delegated to a daemon mode process will fail with a HTTP internal server

    #error response. 

        #WSGIRestrictEmbedded On|Off

    #WSGIRestrictStdin: Enable restrictions on use of STDIN.

    #WSGIRestrictStdout: Enable restrictions on use of STDOUT.

    #WSGIRestrictSignal: Enable restrictions on use of signal().

    #

    #Well behaved WSGI applications neither should try to read/write from/to

    #STDIN/STDOUT, nor should they try to register signal handlers. If your

    #application needs an exception from this rule, you can disable the

    #restrictions here.

        #WSGIRestrictStdin On

        #WSGIRestrictStdout On

        #WSGIRestrictSignal On

    #WSGIAcceptMutex: Specify type of accept mutex used by daemon processes.

    #

    #The WSGIAcceptMutex directive sets the method that mod_wsgi will use to

    #serialize multiple daemon processes in a process group accepting requests

    #on a socket connection from the Apache child processes. If this directive

    #is not defined then the same type of mutex mechanism as used by Apache for

    #the main Apache child processes when accepting connections from a client

    #will be used. If set the method types are the same as for the Apache

    #AcceptMutex directive.

        #WSGIAcceptMutex default

    #WSGIImportScript: Specify a script file to be loaded on process start. 

    #

    #The WSGIImportScript directive can be used to specify a script file to be

    #loaded when a process starts. Options must be provided to indicate the

    #name of the process group and the application group into which the script

    #will be loaded.

        #WSGIImportScript process-group=name application-group=name

    #WSGILazyInitialization: Enable/disable lazy initialisation of Python. 

    #

    #The WSGILazyInitialization directives sets whether or not the Python

    #interpreter is preinitialised within the Apache parent process or whether

    #lazy initialisation is performed, and the Python interpreter only

    #initialised in the Apache server processes or mod_wsgi daemon processes

    #after they have forked from the Apache parent process. 

        #WSGILazyInitialization On|Off

</IfModule>

3) FlaskApp.conf

cat /etc/apache2/sites-available/FlaskApp.conf

<VirtualHost *:83>

ServerName flaskapp.com

ServerAdmin ad...@flaskapp.com

WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

<Directory /var/www/FlaskApp/FlaskApp/>

Order allow,deny

Allow from all

</Directory>

ErrorLog ${APACHE_LOG_DIR}/error.log

LogLevel warn

CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

I change change anything you advise and re-run/test….

—Rajeev

[Graham Dumpleton]

Is FlaskApp.conf symlinked into sites-enabled directory?

If it isn't then it will not even be used.

    WSGIPythonHome "/usr"

Are you not using a Python virtual environment?

Also good practice to use mod_wsgi daemon mode.

3) FlaskApp.conf

cat /etc/apache2/sites-available/FlaskApp.conf

<VirtualHost *:83>

ServerName flaskapp.com

ServerAdmin ad...@flaskapp.com

WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

<Directory /var/www/FlaskApp/FlaskApp/>

This directory path is wrong. Should be:

    <Directory /var/www/FlaskApp/>

 

Order allow,deny

Allow from all

For Apache 2.4 you should be using:

    Require all granted

</Directory>

ErrorLog ${APACHE_LOG_DIR}/error.log

No point setting this here if not setting it different to main Apache error log.

LogLevel warn

Set this to:

    LogLevel info

That way mod_wsgi will log details about when it loads scripts and you will know if request is actually making it to mod_wsgi.

So fix above things, ensuring your site is enabled and do curl again.

What appears in the error log from when the request is made? Does it show mod_wsgi loading your script?

Graham

[Rajeev Jain]

I fixed the items in you email. See my comments inline and below for details:

I rebooted.

 and unfortunately curl command is not working:

curl http://192.168.1.14:83

curl: (7) Failed to connect to 192.168.1.14 port 83: Operation timed out

Here is error.log

tail -f /var/log/apache2/error.log

[Sat Jun 23 14:48:49.945490 2018] [mpm_prefork:notice] [pid 2625] AH00169: caught SIGTERM, shutting down

[Sat Jun 23 14:49:22.977595 2018] [ssl:warn] [pid 2170] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Sat Jun 23 14:49:23.145018 2018] [ssl:warn] [pid 2171] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Sat Jun 23 14:49:23.151437 2018] [mpm_prefork:notice] [pid 2171] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g mod_wsgi/4.6.4 Python/3.6 configured -- resuming normal operations

[Sat Jun 23 14:49:23.151475 2018] [core:notice] [pid 2171] AH00094: Command line: '/usr/sbin/apache2'

[Sat Jun 23 15:40:33.636711 2018] [mpm_prefork:notice] [pid 2171] AH00169: caught SIGTERM, shutting down

[Sat Jun 23 15:41:06.959491 2018] [ssl:warn] [pid 2149] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Sat Jun 23 15:41:07.131568 2018] [ssl:warn] [pid 2151] AH01909: 198.105.244.228:443:0 server certificate does NOT include an ID which matches the server name

[Sat Jun 23 15:41:07.135495 2018] [mpm_prefork:notice] [pid 2151] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g mod_wsgi/4.6.4 Python/3.6 configured -- resuming normal operations

[Sat Jun 23 15:41:07.135532 2018] [core:notice] [pid 2151] AH00094: Command line: '/usr/sbin/apache2'

FYI, updated/most recent:

cat /etc/apache2/sites-available/FlaskApp.conf

<VirtualHost *:83>

ServerName flaskapp.com

ServerAdmin ad...@flaskapp.com

WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

<Directory /var/www/FlaskApp/>

Require all granted

</Directory>

LogLevel info

CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

can we use mod_wsgi-express to help us get this setup?

—Rajeev

On Jun 23, 2018, at 3:24 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

Is FlaskApp.conf symlinked into sites-enabled directory?

If it isn't then it will not even be used.

[rajeev]

yes it is

/etc/apache2/sites-enabled$ ll Flask*

lrwxrwxrwx 1 root root 32 Jun 12 19:50 FlaskApp.conf -> ../sites-available/FlaskApp.conf

and it is enabled:

/etc/apache2/sites-available$ sudo a2ensite FlaskApp

[sudo] password for rajeev: 

Site FlaskApp already enabled

    WSGIPythonHome "/usr"

Are you not using a Python virtual environment?

[rajeev]

my thinking was to get this working at the system level and then migrate to a virtual environment. if better to start with a virtualenv then let’s do so. i have one already setup:

virtualenv-location: /home/rajeev/.virtualenvs/smdb-prod

either system or virtualenv is fine for me. i’ll go with your recommendation. please do advise.

Also good practice to use mod_wsgi daemon mode.

[rajeev]

please advise how to use mod_wsgi daemon mode.

3) FlaskApp.conf

cat /etc/apache2/sites-available/FlaskApp.conf

<VirtualHost *:83>

ServerName flaskapp.com

ServerAdmin ad...@flaskapp.com

WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

<Directory /var/www/FlaskApp/FlaskApp/>

This directory path is wrong. Should be:

    <Directory /var/www/FlaskApp/>

 

Order allow,deny

Allow from all

For Apache 2.4 you should be using:

    Require all granted

</Directory>

ErrorLog ${APACHE_LOG_DIR}/error.log

No point setting this here if not setting it different to main Apache error log.

LogLevel warn

Set this to:

    LogLevel info

That way mod_wsgi will log details about when it loads scripts and you will know if request is actually making it to mod_wsgi.

So fix above things, ensuring your site is enabled and do curl again.

What appears in the error log from when the request is made? Does it show mod_wsgi loading your script?

Graham

[Graham Dumpleton]

Have you added a:

    Listen 83

directive to Apache so it knows to listed on that port?

Graham

[Rajeev Jain]

good question. yes that has been added:

ccat /etc/apache2/ports.conf

# If you just change the port or add more ports here, you will likely also

# have to change the VirtualHost statement in

# /etc/apache2/sites-enabled/000-default.conf

Listen 80

Listen 81

Listen 82

Listen 83

<IfModule ssl_module>

Listen 443

</IfModule>

<IfModule mod_gnutls.c>

Listen 443

</IfModule>

any chance you can call me or i can call you? if yes please email my personal email: cjainraje@yahoo.

[Rajeev Jain]

sorry. typo: my email: jain...@yahoo.com

[Graham Dumpleton]

Run the curl on the same host and use:

    curl http://127.0.0.1:83

Try doing a complete 'stop' and 'start' of Apache, not just a 'reload'.

BTW, you don't need the CustomLog directive either since would be same as main Apache one.

Graham

[Rajeev Jain]

FYI Out for the evening but will run this later tonight and provide results.

===

Rajeev Jain 

From mobile

[Rajeev Jain]

On Jun 23, 2018, at 4:42 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

Run the curl on the same host and use:

    curl http://127.0.0.1:83

From the server both the curl command and browser are working correctly:

$curl http://192.168.1.14:83

Hello, this is running from flasK!

$curl http://127.0.0.1:83

Hello, this is running from flasK!

Browser not shown but working correctly.

Those same URLs do not work at all (curl or browser) from a different computer on the network.

Note the other 3 virtualservers are working using curl or browser from any computer on the network:

http://192.168.1.14:80

http://192.168.1.14:81

http://192.168.1.14:82

Why 192.168.1.14:83 is not accessible from other computer?

Try doing a complete 'stop' and 'start' of Apache, not just a 'reload’.

To restart apache I’m using the following:

$ sudo service apache2 restart

Is this the right command (i.e. will is reload changes in the configuration and restart the server)? 

If not, which command to use?

BTW, you don't need the CustomLog directive either since would be same as main Apache one.

ok i removed that line. here is most recent:

cat /etc/apache2/sites-available/FlaskApp.conf

<VirtualHost *:83>

ServerName flaskapp.com

ServerAdmin ad...@flaskapp.com

WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi

<Directory /var/www/FlaskApp/>

Require all granted

</Directory>

LogLevel info

</VirtualHost>

Whats’s next…it seems we are close.

—Rajeev

[Graham Dumpleton]

Possibly because for the other ports you opened up the firewall on the host so they can be accessed from outside, but you didn't do that for port 83.

Graham

[Rajeev Jain]

On Jun 23, 2018, at 9:36 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

Possibly because for the other ports you opened up the firewall on the host so they can be accessed from outside, but you didn't do that for port 83.

Bingo! 

sudo ufw allow 83

It is working from all the other computers!!

Thank-you so much. You are awesome. 

[Rajeev Jain]

This is the final part of the puzzle for me. If I wanted to run things from a virtualenv, what would be the required changes? 

Location of desired virtualenv:

/home/user/.virtualenvs/myenv

—Rajeev

[Graham Dumpleton]

See:

http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

Graham

[Rajeev Jain]

On Jun 23, 2018, at 10:02 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

See:

http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

Graham

Perfect.

[Rajeev Jain]

On Jun 23, 2018, at 10:02 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

See:

http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

Graham

After reading the virtualenv section I reverted back to the wsgi and virtualhost conf files from our other thread.

Here are snapshots:

cat /etc/apache2/mods-available/wsgi.conf

<IfModule mod_wsgi.c>

   WSGIRestrictEmbedded On

</IfModule>

cat /etc/apache2/sites-available/FlaskApp.conf

<VirtualHost *:83>

ServerName flaskapp.com

ServerAdmin ad...@flaskapp.com

WSGIDaemonProcess flaskapp.com python-home=/usr/local/venvs/myenv python-path=/var/www/FlaskApp

WSGIScriptAlias / /var/www/FlaskApp/flaskapp.wsgi process-group=flaskapp.com application-group=%{GLOBAL}

<Directory /var/www/FlaskApp/>

    <Files flaskapp.wsgi>

    Require all granted

    </Files>

</Directory>

LogLevel info

</VirtualHost>

Anything above look concerning? Any further best practices or optimizations you advise?

It appears to be working well. Again thank you for your very good support and patience especially dealing with my user errors (arg didn’t open firewall ports and others).

—Rajeev

[Graham Dumpleton]

Looks good.

Maybe watch:

    https://www.youtube.com/watch?v=CPz0s1CQsTE

and check out some of those options for WSGIDaemonProcess mentioned.

    http://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIDaemonProcess.html

Graham

[Rajeev Jain]

Very good. I appreciate the pointers.

—Rajeev