Problem with new install on FreeBSD

[David Gessel]

I'm pretty confident this is a path issue, but I haven't had much luck in resolving it. 

I am getting "500 server error" and looking at my logs I see:

[Fri Mar 24 11:10:23.282925 2017] [:error] [pid 21554:tid 34741511168] [client xxx.127.65.15:22736] python_handler: Can't get/create interpreter.

ImportError: No module named mod_python.apache
[Fri Mar 24 11:10:23.282850 2017] [:error] [pid 21554:tid 34741511168] make_obcallback: Python path being used "['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload']".
[Fri Mar 24 11:10:23.282888 2017] [:error] [pid 21554:tid 34741511168] get_interpreter: no interpreter callback found.

My vhost config looks like:

<VirtualHost 10.3.69.139:80>
    # registrar nic.st
    DocumentRoot /usr/local/www/trac
    ServerName trac.my.domain
    ErrorLog /var/log/trac-error_log
    CustomLog /var/log/trac-access_log combined
    <Location />
        SetHandler mod_python
        PythonInterpreter main_interpreter
        PythonOption PYTHON_EGG_CACHE /usr/local/www/trac/tmp
        PythonPath "sys.path + ['/usr/local/www/trac']"
        PythonPath /usr/local/bin/mod_python
        PythonHandler trac.web.modpython_frontend
        PythonOption TracEnv /usr/local/www/trac
        PythonOption TracUriRoot /
    </Location>
</VirtualHost>

I am confident it's a trivial misconfiguration issue, but most answers I find by googling are along the lines of "why aren't you using WSGIScript

There's an obvious crude attempt to add the path for mod_python explicitly in my vhost declaration, which didn't help.


[Logan Anderson]

A shot in the dark because I haven't really messed with FreeBSD in years..

Is it possible you have python3 on the system and the version of mod_python is compiled for python3 while you are using python2.7 for Trac? Or vice versa, is it compiled for 2.7 and you only have python3 installed?

Just because you are calling mod_python doesn't mean you installed the module.. have you installed it?

Im guessing apache on FreeBSD has support for mod_wsgi, that may be a reasonable alternative to mod_python. This may be why the links you are seeing reference using that instead.

---

From: "David Gessel" <ges...@blackrosetech.com>
To: "Trac Users" <trac-...@googlegroups.com>
Sent: Friday, March 24, 2017 2:30:02 PM
Subject: [Trac] Problem with new install on FreeBSD

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.
To post to this group, send email to trac-...@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.

[RjOllos]

On Friday, March 24, 2017 at 12:03:18 PM UTC-7, Logan Anderson wrote:

A shot in the dark because I haven't really messed with FreeBSD in years..

Is it possible you have python3 on the system and the version of mod_python is compiled for python3 while you are using python2.7 for Trac? Or vice versa, is it compiled for 2.7 and you only have python3 installed?

Just because you are calling mod_python doesn't mean you installed the module.. have you installed it?

That sounds like a good thing to check. It appears it should be possible to run "import mod_python" in an interpreter in order to check.
 

Im guessing apache on FreeBSD has support for mod_wsgi, that may be a reasonable alternative to mod_python. This may be why the links you are seeing reference using that instead.

See also:
https://trac.edgewall.org/wiki/TracModPython#Troubleshooting

- Ryan
 

[David Gessel]

I think's so:

# pkg version -v | grep py
ap24-mod_python35-3.5.0_2          =   up-to-date with index
py27-Babel-2.3.4                   =   up-to-date with index
py27-Genshi-0.7_1                  =   up-to-date with index
py27-acme-0.12.0,1                 =   up-to-date with index
py27-cffi-1.7.0                    =   up-to-date with index
py27-configargparse-0.11.0         =   up-to-date with index
py27-configobj-5.0.6_1             =   up-to-date with index
py27-cryptography-1.7.2            =   up-to-date with index
py27-dnspython-1.15.0              =   up-to-date with index
py27-docutils-0.13.1               =   up-to-date with index
py27-enum34-1.1.6                  =   up-to-date with index
py27-funcsigs-1.0.2                =   up-to-date with index
py27-idna-2.0                      =   up-to-date with index
py27-ipaddress-1.0.18              =   up-to-date with index
py27-mock-1.3.0_1                  =   up-to-date with index
py27-ndg_httpsclient-0.4.2         =   up-to-date with index
py27-openssl-16.2.0                =   up-to-date with index
py27-parsedatetime-2.1             =   up-to-date with index
py27-pbr-1.8.1                     =   up-to-date with index
py27-pip-9.0.1                     =   up-to-date with index
py27-psutil-5.2.0_1                =   up-to-date with index
py27-pyasn1-0.2.2                  =   up-to-date with index
py27-pycparser-2.10                =   up-to-date with index
py27-pygments-2.1.3_1              =   up-to-date with index
py27-pyrfc3339-1.0                 =   up-to-date with index
py27-python2-pythondialog-3.4.0    =   up-to-date with index
py27-pytz-2016.10,1                =   up-to-date with index
py27-requests-2.11.1               =   up-to-date with index
py27-setuptools-32.1.0_1           =   up-to-date with index
py27-six-1.10.0                    =   up-to-date with index
py27-sqlite3-2.7.13_7              =   up-to-date with index
py27-subversion-1.9.5              =   up-to-date with index
py27-werkzeug-0.12.1               =   up-to-date with index
py27-zope.component-4.2.2          =   up-to-date with index
py27-zope.event-4.1.0              =   up-to-date with index
py27-zope.interface-4.1.3          =   up-to-date with index
python2-2_3                        =   up-to-date with index
python27-2.7.13_1                  =   up-to-date with index

mod_python doesn't have any configurable options, but the make file indicates that it wants Python 2.7 (as installed)

 # less Makefile
# Created by: Hye-Shik Chang
# $FreeBSD: head/www/mod_python35/Makefile 422338 2016-09-17 18:13:34Z ohauer $

PORTNAME=       mod_python
PORTVERSION=    3.5.0
PORTREVISION=   2
CATEGORIES=     www python
MASTER_SITES=   http://dist.modpython.org/dist/ \
                LOCAL/ohauer
PKGNAMEPREFIX=  ${APACHE_PKGNAMEPREFIX}
PKGNAMESUFFIX=  35
DIST_SUBDIR=    apache2

MAINTAINER=     apache@FreeBSD.org
COMMENT=        Apache module that embeds the Python interpreter within the server

LICENSE=        APACHE20

CONFLICTS_INSTALL=      ap2[24]-mod_python3[^5]*

USE_APACHE=     22+
USES=           cpe python:2.7 tar:tgz
CPE_VENDOR=     apache
SUB_FILES=      pkg-message 270_mod_python.conf.sample

GNU_CONFIGURE=          yes
CONFIGURE_ARGS+=        --with-apxs="${APXS}"           \
                        --with-python="${PYTHON_CMD}"   \
                        --without-flex

LDFLAGS+=       -L${LOCALBASE}/lib
MAKE_ARGS+=     APXS=${APXS}
MAKE_ENV=       EXPR_COMPAT=yes
PLIST_SUB+=     PORTVERSION=${PORTVERSION} \
                PYTHON_VER=${PYTHON_VER}

post-install:
        @${MKDIR} ${STAGEDIR}${PREFIX}/${APACHEETCDIR}/modules.d
        ${INSTALL_DATA} ${WRKDIR}/270_mod_python.conf.sample ${STAGEDIR}${PREFIX}/${APACHEETCDIR}/modules.d

# You need to install apache & mod_python before you run this
regression-test: build
        ${REINPLACE_CMD} -e 's|%%APACHEMODDIR%%|${APACHEMODDIR}|' ${WRKSRC}/test/test.py
        -@${MAKE} test -C ${WRKSRC}

[David Gessel]

Also, just to verify that it is apache 24 that's installed:

# pkg version -v | grep apache
apache24-2.4.25_1                  =   up-to-date with index

[RjOllos]

From:

http://modpython.org/live/current/doc-html/directives.html#pythonpath

"When multiple PythonPath directives are specified, the effect is not cumulative, last directive will override all previous ones. "

Try combining your PythonPath directives:
PythonPath "sys.path + ['/usr/local/www/trac', '/usr/local/bin/mod_python']"

- Ryan

[David Gessel]

Thanks!  I tried and
# apachectl graceful

but no change, though the error log might provide a clue with respect to what paths are still getting called (no change):

Enter code here.Sat Mar 25 09:50:25.025855 2017] [:error] [pid 69623:tid 34741511168] make_obcallback: could not import mod_python.apache.\n

ImportError: No module named mod_python.apache

[Sat Mar 25 09:50:25.026263 2017] [:error] [pid 69623:tid 34741511168] make_obcallback: Python path being used "['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload']".
[Sat Mar 25 09:50:25.026292 2017] [:error] [pid 69623:tid 34741511168] get_interpreter: no interpreter callback found.
..

[RjOllos]

On Saturday, March 25, 2017 at 9:52:30 AM UTC-7, David Gessel wrote:

Thanks!  I tried and
# apachectl graceful

but no change, though the error log might provide a clue with respect to what paths are still getting called (no change):

Enter code here.Sat Mar 25 09:50:25.025855 2017] [:error] [pid 69623:tid 34741511168] make_obcallback: could not import mod_python.apache.\n
ImportError: No module named mod_python.apache
[Sat Mar 25 09:50:25.026263 2017] [:error] [pid 69623:tid 34741511168] make_obcallback: Python path being used "['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload']".
[Sat Mar 25 09:50:25.026292 2017] [:error] [pid 69623:tid 34741511168] get_interpreter: no interpreter callback found.
..

Are you able to?:
1. Open python from the terminal by typing "python"
2. Import mod_python from the Python interpreter session by typing "import mod_python"
3. What is the output of sys.path from your Python terminal session?

- Ryan

[David Gessel]

First step fail! 

Thanks for the diagnostic - good starting point.

[RjOllos]

On Saturday, March 25, 2017 at 3:08:35 PM UTC-7, David Gessel wrote:

First step fail! 

Thanks for the diagnostic - good starting point.

Okay, make sense because the  Python path printed in your earlier error message looked incomplete. I'd focus on trying to figure out why the PYTHONPATH is incorrect.

- Ryan

[David Gessel]

But the freeBSD command is "python2" which works.
import mod_python works

# python2
Python 2.7.13 (default, Mar 22 2017, 10:45:35)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/site-packages']
>>>

[David Gessel]

This link seems to be in the right direction...   https://forums.freebsd.org/threads/46449/

[David Gessel]

executing # portmaster lang/python  and now # python does the right thing and....  closer, but still getting the 500 error. 

Same errors logged

ImportError: No module named mod_python.apache

[Sat Mar 25 15:20:19.399938 2017] [:error] [pid 83737:tid 34741511168] make_obcallback: Python path being used "['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload']".
[Sat Mar 25 15:20:19.399964 2017] [:error] [pid 83737:tid 34741511168] get_interpreter: no interpreter callback found

However, perhaps mod_python needs to be rebuilt with the proper environment added now that the lang/python package is installed.  I will do a rebuild of all packages, which takes a few hours, and try again.

[David Gessel]

Alas, I rebuilt the system and rebooted the jail to no avail.

An additional detail is that when I test trac with the internal server, it works fine.  Not surprising, but reinforces this is almost certainly a mod_python issue.  :/

[RjOllos]

On Sunday, March 26, 2017 at 10:44:56 AM UTC-7, David Gessel wrote:

Alas, I rebuilt the system and rebooted the jail to no avail.

An additional detail is that when I test trac with the internal server, it works fine.  Not surprising, but reinforces this is almost certainly a mod_python issue.  :/

Given that it's probably a issue with mod_python on FreeBSD, you'd probably be better off reaching out to a FreeBSD mailing list for help. I haven't seen a lot of FreeBSD users on here. Maybe you could setup a simple mod_python site that has nothing to do with Trac in order to narrow the scope of the issue. That might give you more traction in getting help from the FreeBSD folks. Something like:
http://modpython.org/live/current/doc-html/tutorial.html

- Ryan

[RjOllos]

On Saturday, March 25, 2017 at 9:52:30 AM UTC-7, David Gessel wrote:

Thanks!  I tried and
# apachectl graceful

but no change, though the error log might provide a clue with respect to what paths are still getting called (no change):

Enter code here.Sat Mar 25 09:50:25.025855 2017] [:error] [pid 69623:tid 34741511168] make_obcallback: could not import mod_python.apache.\n
ImportError: No module named mod_python.apache
[Sat Mar 25 09:50:25.026263 2017] [:error] [pid 69623:tid 34741511168] make_obcallback: Python path being used "['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload']".
[Sat Mar 25 09:50:25.026292 2017] [:error] [pid 69623:tid 34741511168] get_interpreter: no interpreter callback found.
..

It seems that your PythonPath directive is having no effect. Do you get the same "Python path being used" error if you comment out the PythonPath directive?

With regard to testing, (1) shows how to setup a simple test case using modpython.test_handler.

- Ryan

(1) https://trac.edgewall.org/wiki/TracModPython#Simpleconfiguration

[David Gessel]

I apologize for the late response, but tested and yes, same error.  Commented out, apachectl graceful and

[Sat Apr 01 08:19:28.674481 2017] [core:notice] [pid 75658:tid 34401707008] AH00094: Command line: '/usr/local/sbin/httpd'
[Sat Apr 01 08:20:28.444978 2017] [:error] [pid 45481:tid 34741511168] make_obcallback: could not import mod_python.apache.\n

ImportError: No module named mod_python.apache

[Sat Apr 01 08:20:28.445426 2017] [:error] [pid 45481:tid 34741511168] make_obcallback: Python path being used "['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload']".
[Sat Apr 01 08:20:28.445464 2017] [:error] [pid 45481:tid 34741511168] get_interpreter: no interpreter callback found.

[David Gessel]

Also, given your helpful hint (thanks) I tracked down this : https://code.activestate.com/lists/python-tutor/23472/

however, adding the paths to site.py didn't help.  There are two further hints, but I'm not sure where to apply them - adding the paths to the Trac code or creating *.pth files (as suggested in site.py) but it isn't clear where to install them or if they'll only enumerate subdirectories of an existing path directory (sort of implied).

[David Gessel]

And I found this useful hint. 
http://modpython.org/pipermail/mod_python/2004-November/016815.html
https://groups.google.com/forum/#!topic/comp.lang.python/zqm-uvhfzHY

And indeed:

# python
Python 2.7.13 (default, Mar 25 2017, 16:20:16)

[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.

>>> import mod_python.apache
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/mod_python/apache.py", line 30, in <module>
    import _apache
ImportError: No module named _apache

And

[Sat Apr 01 08:59:34.694122 2017] [:notice] [pid 75658:tid 34401707008] mod_python: Creating 8 session mutexes based on 7 max processes and 25 max threads.
[Sat Apr 01 08:59:34.694155 2017] [:notice] [pid 75658:tid 34401707008] mod_python: using mutex_directory /tmp
[Sat Apr 01 08:59:34.865517 2017] [mpm_worker:notice] [pid 75658:tid 34401707008] AH00292: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k mod_python/3.5.0 Python/2.7.13 PHP/5.6.30 configured -- resuming normal operations
[Sat Apr 01 08:59:34.865568 2017] [core:notice] [pid 75658:tid 34401707008] AH00094: Command line: '/usr/local/sbin/httpd'
[Sat Apr 01 08:59:40.077629 2017] [:error] [pid 46654:tid 34745705472] make_obcallback: could not import mod_python.apache.\n

ImportError: No module named mod_python.apache

[Sat Apr 01 08:59:40.078052 2017] [:error] [pid 46654:tid 34745705472] make_obcallback: Python path being used "['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd10', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload']".
[Sat Apr 01 08:59:40.078087 2017] [:error] [pid 46654:tid 34745705472] get_interpreter: no interpreter callback found.

[RjOllos]

On Saturday, April 1, 2017 at 8:57:04 AM UTC-7, David Gessel wrote:

Also, given your helpful hint (thanks) I tracked down this : https://code.activestate.com/lists/python-tutor/23472/

however, adding the paths to site.py didn't help.  There are two further hints, but I'm not sure where to apply them - adding the paths to the Trac code or creating *.pth files (as suggested in site.py) but it isn't clear where to install them or if they'll only enumerate subdirectories of an existing path directory (sort of implied).

You could try editing site-packages.pth, as described in:

https://superuser.com/a/247649

Or maybe create or edit a file with a name like mod_python.pth in /usr/local/lib/python2.7/site-packages. Which .pth files exist in that directory?

To make the svn bindings available on Linux and OSX, I have to create an svn.pth and two symbolic links:

$pwd

/Users/rjollos/Documents/Workspace/trac-dev/pve/lib/python2.7/site-packages

$ls -l svn libsvn

libsvn -> /usr/local/Cellar/subversion/1.9.5_1/lib/python2.7/site-packages/libsvn

svn -> /usr/local/Cellar/subversion/1.9.5_1/lib/python2.7/site-packages/svn

$cat svn.pth

/usr/local/Cellar/subversion/1.9.5_1/lib/svn-python/libsvn

So I guess something similar might be required for FreeBSD with mod_python.

When you build mod_python, does everything get installed into /usr/local/lib/python2.7/site-packages/mod_python ?

- Ryan

[David Gessel]

It looks like it is all there:

/usr/local/lib/python2.7/site-packages/mod_python
/usr/local/lib/python2.7/site-packages/mod_python/Cookie.py
/usr/local/lib/python2.7/site-packages/mod_python/Cookie.pyc
/usr/local/lib/python2.7/site-packages/mod_python/Cookie.pyo
/usr/local/lib/python2.7/site-packages/mod_python/Session.py
/usr/local/lib/python2.7/site-packages/mod_python/Session.pyc
/usr/local/lib/python2.7/site-packages/mod_python/Session.pyo
/usr/local/lib/python2.7/site-packages/mod_python/__init__.py
/usr/local/lib/python2.7/site-packages/mod_python/__init__.pyc
/usr/local/lib/python2.7/site-packages/mod_python/__init__.pyo
/usr/local/lib/python2.7/site-packages/mod_python/_psp.so
/usr/local/lib/python2.7/site-packages/mod_python/apache.py
/usr/local/lib/python2.7/site-packages/mod_python/apache.pyc
/usr/local/lib/python2.7/site-packages/mod_python/apache.pyo
/usr/local/lib/python2.7/site-packages/mod_python/cache.py
/usr/local/lib/python2.7/site-packages/mod_python/cache.pyc
/usr/local/lib/python2.7/site-packages/mod_python/cache.pyo
/usr/local/lib/python2.7/site-packages/mod_python/cgihandler.py
/usr/local/lib/python2.7/site-packages/mod_python/cgihandler.pyc
/usr/local/lib/python2.7/site-packages/mod_python/cgihandler.pyo
/usr/local/lib/python2.7/site-packages/mod_python/httpdconf.py
/usr/local/lib/python2.7/site-packages/mod_python/httpdconf.pyc
/usr/local/lib/python2.7/site-packages/mod_python/httpdconf.pyo
/usr/local/lib/python2.7/site-packages/mod_python/psp.py
/usr/local/lib/python2.7/site-packages/mod_python/psp.pyc
/usr/local/lib/python2.7/site-packages/mod_python/psp.pyo
/usr/local/lib/python2.7/site-packages/mod_python/publisher.py
/usr/local/lib/python2.7/site-packages/mod_python/publisher.pyc
/usr/local/lib/python2.7/site-packages/mod_python/publisher.pyo
/usr/local/lib/python2.7/site-packages/mod_python/python22.py
/usr/local/lib/python2.7/site-packages/mod_python/python22.pyc
/usr/local/lib/python2.7/site-packages/mod_python/python22.pyo
/usr/local/lib/python2.7/site-packages/mod_python/testhandler.py
/usr/local/lib/python2.7/site-packages/mod_python/testhandler.pyc
/usr/local/lib/python2.7/site-packages/mod_python/testhandler.pyo
/usr/local/lib/python2.7/site-packages/mod_python/util.py
/usr/local/lib/python2.7/site-packages/mod_python/util.pyc
/usr/local/lib/python2.7/site-packages/mod_python/util.pyo
/usr/local/lib/python2.7/site-packages/mod_python/version.py
/usr/local/lib/python2.7/site-packages/mod_python/version.pyc
/usr/local/lib/python2.7/site-packages/mod_python/version.pyo
/usr/local/lib/python2.7/site-packages/mod_python/wsgi.py
/usr/local/lib/python2.7/site-packages/mod_python/wsgi.pyc
/usr/local/lib/python2.7/site-packages/mod_python/wsgi.pyo
/usr/local/lib/python2.7/site-packages/mod_python-3.5.0-py2.7.egg-info

[RjOllos]

Not sure if it will be helpful for your situation, but you could try a "test handler", like:

<Location />
    SetHandler mod_python
    PythonInterpreter main_interpreter

    PythonHandler mod_python.testhandler
    Require all granted
</Location> 

Discussed in https://trac.edgewall.org/wiki/TracModPython#Simpleconfiguration

- Ryan