cannot use pyodbc from wsgi/apache

[fred....@sunrise.com]

I know this is a bit off topic; but I'm really stuck. I've got a
pyodbc connection to Sybase that works from the command line but not
from within a wsgi app and Apache on linux; my /etc/.odbc.ini looks
like this (also /etc/odbc.ini since I've got conflicting info) and
they have permissions of 777

[root]# more /etc/.odbc.ini
[ODBC Data Sources]
Homeworks=SQLAnywhere 12.0

[Homeworks]
UID=xxxxxxxxxxxxxxxxxx
PWD=xxxxxxxxxxxxxxxxxx
DatabaseName=Production
ServerName=Homeworks
CommLinks=tcpip(host=192.168.xxx.xxx;DoBroadcast=NONE)
Driver=/opt/sqlanywhere12/lib32/libdbodbc12.so

Uid, pwd and ip deleted for security.

Either Apache cannot access this and I'm not an Apache expert or if
someone knows how to convert this into the pyodbc connection string
that does not use the DSN, that might be a solution.

From the command line
pyodbc.connect('DSN=HomeworksProduction') works but that confuses me
because I don't see that name in the .ini unless it is "assembled"
from the individual lines.

I've googled all day on this and just cannot find a solution.

[Graham Dumpleton]

Do yo have an actual error message you can share which gives any
indication of why it is failing?

Graham

> --
> You received this message because you are subscribed to the Google Groups "modwsgi" group.
> To post to this group, send email to mod...@googlegroups.com.
> To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
>
>

[fred....@sunrise.com]

Sorry about the oversight; here's the dump of apache's error_log;
browser just says "Internal Server Error"

[Tue Jul 19 20:43:17 2011] [notice] Apache/2.2.15 (Unix) DAV/2
mod_wsgi/3.2 Python/2.6.5 configured -- resuming normal operations
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] mod_wsgi
(pid=31858): Exception occurred processing WSGI script '/var/www/wsgi-
scripts/mdsquery.py'.
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] Traceback
(most recent call last):
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] File "/
var/www/wsgi-scripts/mdsquery.py", line 136, in application
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] output
+= unittest('HomeworksProduction', resid)
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] File "/
var/www/wsgi-scripts/mdsquery.py", line 122, in unittest
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] HWDB =
MDSQuery(dsn)
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] File "/
var/www/wsgi-scripts/mdsquery.py", line 78, in __init__
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17]
self.Connection = pyodbc.connect(connection_string)
[Tue Jul 19 20:43:32 2011] [error] [client 192.168.252.17] Error:
('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not
found, and no default driver specified (0) (SQLDriverConnectW)')
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] mod_wsgi
(pid=31858): Exception occurred processing WSGI script '/var/www/wsgi-
scripts/mdsquery.py'.
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] Traceback
(most recent call last):
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] File "/
var/www/wsgi-scripts/mdsquery.py", line 136, in application
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] output
+= unittest('HomeworksProduction', resid)
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] File "/
var/www/wsgi-scripts/mdsquery.py", line 122, in unittest
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] HWDB =
MDSQuery(dsn)
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] File "/
var/www/wsgi-scripts/mdsquery.py", line 78, in __init__
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17]
self.Connection = pyodbc.connect(connection_string)
[Wed Jul 20 08:18:46 2011] [error] [client 192.168.252.17] Error:
('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not
found, and no default driver specified (0) (SQLDriverConnectW)')

[Graham Dumpleton]

When doing stuff from the command line, have you perhaps set any
special environment variables in your user environment. These
environment variables wouldn't be present when run under
Apache/mod_wsgi.

Graham

[fred....@sunrise.com]

I think you found the issue. Now is the proper procedure to just set
the missing environment variables into the environ that is passed into
the
def application(environ, start_response): function?

os.environ when run from the wsgi app shows:
HOME =/var/www
LANG =C
PATH =/sbin:/usr/sbin:/bin:/usr/bin
PWD =/
SHLVL =2
TERM =vt100
_ =/usr/sbin/httpd

----------while when run from shell command as root, I
get-------------
CVS_RSH =ssh
G_BROKEN_FILENAMES =1
HISTCONTROL =ignoredups
HISTSIZE =1000
HOME =/root
HOSTNAME =xxxxxxxx.sunbelt.org
LANG =en_US.UTF-8
LD_LIBRARY_PATH =/opt/sqlanywhere12/lib32:
LESSOPEN =|/usr/bin/lesspipe.sh %s
LOGNAME =root
LS_COLORS =rs=0:di=01;34:....:
MAIL =/var/spool/mail/root
OLDPWD =/home/sybase
PATH =/opt/sqlanywhere12/bin32:/usr/lib/
qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/
bin:/root/bin
PWD =/var/www/wsgi-scripts
QTDIR =/usr/lib/qt-3.3
QTINC =/usr/lib/qt-3.3/include
QTLIB =/usr/lib/qt-3.3/lib
SHELL =/bin/bash
SHLVL =1
SQLANY12 =/opt/sqlanywhere12
SQLANYSAMP12 =/opt/sqlanywhere12/samples
SSH_ASKPASS =/usr/libexec/openssh/gnome-ssh-askpass
SSH_CLIENT =192.168.xxx.xxx 64809 22
SSH_CONNECTION =192.168.xxx.xxx64809 192.168.xxx.xxx 22
SSH_TTY =/dev/pts/1
TERM =vt100
USER =root
_ =/usr/bin/python

[fred....@sunrise.com]

I tried this to see if it would solve the problem with no success; no
change in error message.

('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not
found, and no default driver specified (0) (SQLDriverConnectW)')

def fix_environment():
os.environ['LD_LIBRARY_PATH'] ='/opt/sqlanywhere12/lib32'
os.environ['PATH'] =os.environ['PATH']+':/opt/sqlanywhere12/
bin32:/usr/local/sbin:/usr/local/bin:/usr/bin:/root/bin'
os.environ['SQLANY12'] ='/opt/sqlanywhere12'
os.environ['SQLANYSAMP12'] ='/opt/sqlanywhere12/samples'

[Graham Dumpleton]

On 22 July 2011 06:33, fred....@sunrise.com <fred....@sunrise.com> wrote:
> I tried this to see if it would solve the problem with no success; no
> change in error message.
> ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not
> found, and no default driver specified (0) (SQLDriverConnectW)')
>
> def fix_environment():
>    os.environ['LD_LIBRARY_PATH']  ='/opt/sqlanywhere12/lib32'

You can't set this on in WSGI script file as is too late. This
variable needs to be set at time process starts up.

If you have a standard Apache installation, then look for the
'envvars' file in same directory as Apache 'httpd' executable and add
to the 'envvars' file:

LD_LIBRARY_PATH='/opt/sqlanywhere12/lib32':$LD_LIBRARY_PATH
export LD_LIBRARY_PATH

Alternatively, if you compiled the Python client module for pyodbc
yourself, then we can set an environment variable at the time you
compile it so the library directory is embedded in the client module
and therefore don't need to set it at run time.

>    os.environ['PATH']  =os.environ['PATH']+':/opt/sqlanywhere12/
> bin32:/usr/local/sbin:/usr/local/bin:/usr/bin:/root/bin'

Not usually a good idea for a web application to be overriding PATH.
Code shouldn't be relying on picking up executables to be run from
PATH but should use absolute paths instead. In all probability this
isn't needed anyway, as code probably not trying to execute command
line applications.

>    os.environ['SQLANY12']  ='/opt/sqlanywhere12'
>    os.environ['SQLANYSAMP12']  ='/opt/sqlanywhere12/samples'

These are okay set like that.

Graham

[fred....@sunrise.com]

Sorry for being so dense, I know just enough about Apache to be dangerous and our SysAdmin just quit.
I'm running RHEL6 with mod_wsgi and Apache 2.2 whith Python 2.6 --  all default components.
there is no envvars or envars file anywhere on the system.

adding the following to /etc/profile (which others said would apply to Apache) and rebooting did not make these visible to Apache
SQLANY12="/opt/sqlanywhere12"
export SQLANY12

PATH="$SQLANY12/bin32:${PATH:-}"
export PATH
LD_LIBRARY_PATH="$SQLANY12/lib32:${LD_LIBRARY_PATH:-}"
export LD_LIBRARY_PATH


export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL
------------------------------------------------------------
my /etc/httpd/conf.d/wsgi.conf has
LoadModule wsgi_module modules/mod_wsgi.so

WSGIScriptAlias /myapp /var/www/wsgi-scripts/myapp.wsgi
WSGIScriptAlias /wsgi /var/www/wsgi-scripts
WSGIPythonPath /var/www/djangodeploy
WSGIScriptReloading On
#WSGIReloadMechanism Process
WSGIDaemonProcess djangoapps processes=2 threads=15 python-path=/var/www/djangodeploy
WSGIProcessGroup djangoapps
LogLevel info
WSGISocketPrefix /var/run/wsgi

<Directory /var/www/wsgi-scripts>
Order allow,deny
Allow from all
</Directory>

#<Directory /var/www/djangodeploy>
#Order allow,deny
#Allow from all
#</Directory>


-----------------------------------------------------------------------------
on an older system running mod_python I have
<Location "/django/">
    SetHandler python-program
    PythonHandler django.core.handlers.modpython
    SetEnv DJANGO_SETTINGS_MODULE mds30.settings
    PythonOption django.root /var/www//django/mds30
    PythonPath "['/var/www/django'] + sys.path"
    PythonDebug On
    PythonAutoReload On
</Location>

Now I really don't comprehend the difference between the Location and the Directory directives, but can i put the equivalent of the  "SetEnv ..." statements in the wsgi.conf  <Directory settings?


logging os.environ returns {'LANG': 'C', 'CONSOLETYPE': 'vt', 'TERM': 'linux', 'SHLVL': '3', 'runlevel': '5', 'UPSTART_EVENTS': 'runlevel', 'PREVLEVEL': 'N', 'LANGSH_SOURCED': '1', 'UPSTART_INSTANCE': '', 'PWD': '/', 'RUNLEVEL': '5', 'UPSTART_JOB': 'rc', 'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'HOME': '/var/www', '_': '/usr/sbin/httpd', 'previous': 'N'}
 even after a brute force attempt in my wsgi script as follows:
os.environ['SQLANY12'] = '/opt/sqlanywhere12'
os.environ['PATH'] = '/opt/sqlanywhere12/bin32:'+os.environ['PATH']
os.environ['LD_LIBRARY_PATH'] = '/opt/sqlanywhere12/lib32:'+os.environ['LD_LIBRARY_PATH']


I't apparent that I'm failing to set the environment variables when running as Apache. 

[Graham Dumpleton]

On 30 July 2011 02:16, fred....@sunrise.com <fred....@sunrise.com> wrote:
> Sorry for being so dense, I know just enough about Apache to be dangerous
> and our SysAdmin just quit.
> I'm running RHEL6 with mod_wsgi and Apache 2.2 whith Python 2.6 --  all
> default components.
> there is no envvars or envars file anywhere on the system.
>
> adding the following to /etc/profile (which others said would apply to
> Apache) and rebooting did not make these visible to Apache

Don't believe /etc/profile is used. If envvars doesn't exist, which
file you change depends on the platform.

Could be one of the follow:

/etc/sysconfig/httpd
/etc/init.d/httpd
/etc/init.d/apache

> SQLANY12="/opt/sqlanywhere12"
> export SQLANY12
>
> PATH="$SQLANY12/bin32:${PATH:-}"
> export PATH
> LD_LIBRARY_PATH="$SQLANY12/lib32:${LD_LIBRARY_PATH:-}"
> export LD_LIBRARY_PATH
>
>
> export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL
> ------------------------------------------------------------
> my /etc/httpd/conf.d/wsgi.conf has
> LoadModule wsgi_module modules/mod_wsgi.so
>
> WSGIScriptAlias /myapp /var/www/wsgi-scripts/myapp.wsgi
> WSGIScriptAlias /wsgi /var/www/wsgi-scripts
> WSGIPythonPath /var/www/djangodeploy

Not needed since you are using daemon mode and using python-path
option to WSGIDaemonProcess.

> WSGIScriptReloading On

Not needed as is the default.

> #WSGIReloadMechanism Process

Not needed and newer mod_wsgi versions don't even recognise that directive.

SetEnv can be put in mod_wsgi but it only results in variables going
into per request environ and not os.environ.

It was a hack in mod_python Django adapter that SetEnv variables got
pushed to os.environ. It is actually really bad practice to update
os.environ on every request for a persistent process like Django was
doing in their mod_python adapter.

Graham

> logging os.environ returns {'LANG': 'C', 'CONSOLETYPE': 'vt', 'TERM':
> 'linux', 'SHLVL': '3', 'runlevel': '5', 'UPSTART_EVENTS': 'runlevel',
> 'PREVLEVEL': 'N', 'LANGSH_SOURCED': '1', 'UPSTART_INSTANCE': '', 'PWD': '/',
> 'RUNLEVEL': '5', 'UPSTART_JOB': 'rc', 'PATH':
> '/sbin:/usr/sbin:/bin:/usr/bin', 'HOME': '/var/www', '_': '/usr/sbin/httpd',
> 'previous': 'N'}
>  even after a brute force attempt in my wsgi script as follows:
> os.environ['SQLANY12'] = '/opt/sqlanywhere12'
> os.environ['PATH'] = '/opt/sqlanywhere12/bin32:'+os.environ['PATH']
> os.environ['LD_LIBRARY_PATH'] =
> '/opt/sqlanywhere12/lib32:'+os.environ['LD_LIBRARY_PATH']
>
>
> I't apparent that I'm failing to set the environment variables when running
> as Apache.
>

> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/modwsgi/-/qLeOsHe3ES4J.

[fred....@sunrise.com]

Thank You, Thank You, Thank You, Thank You, Thank You, Thank You,
It's working like a charm.

Graham wrote:

Don't believe /etc/profile is used. If envvars doesn't exist, which file you change depends on the platform

Could be one of the follow:

   /etc/sysconfig/httpd  it was this one for RHEL/Centos.

   /etc/init.d/httpd

   /etc/init.d/apache

In all my trial and error I've got multiple .odbc.ini files scattered about.  I'll start deleting those one by one to verify which one ( /var/www I think) is the right one. 

Again, thank you for your help and patience with ignorance of SysAdmin issues.

Fred.

[Varun Boyapati]

Hi All,

My situation was exactly same to this. But I was using Apache 2.4 Version.

I tried to add the environment variables in bashrc but still getting the same issue. Can you please help me on this. Hard for me to get it down from 2 days. Not able to fix that.

Thanks in Advance.

[Graham Dumpleton]

Ideally you would create a fresh discussion thread which properly explains your own problem, rather than referencing an existing discussion, that could well end up not being the same even if you think it is.

As explained in that discussion, what file you need to change depends on what platform you are using, but you don’t say what platform you are using, whether you are using the operating system Apache or an Apache which you installed yourself.

Setting environment variables in startup scripts for Apache is also not necessarily recommended anyway, as it affects the whole of Apache and so you can’t be specific and have it only affect one specific application.

The better way is to set environment variables in the WSGI script file.

If you are using Django where it already has a wsgi.py file and that is part of your code repo, then you would want to create a new WSGI script file outside of your code repo and which is just on the host, which sets the environment variables and then imports the original module with the WSGI application in it.

This is of course is Django specific, so you also need to indicate what framework you are using, how you currently have mod_wsgi configured so any suggested solution can be tailored to your circumstances.

Graham

-- 
You received this message because you are subscribed to the Google Groups "modwsgi" group.

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.

To post to this group, send email to mod...@googlegroups.com.

Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Varun Boyapati]

I was new using google groups. Sorry for referring existing discussion.

Mine was Flask Application running under Apache 2.4 ( which was installed manually ) and mod_wsgi in Linux RHEL 6.

I had installed ODBC driver and changed the perimissions of .ini files to chmod 777 and owner group of files to root. So Pyodbc connection works fine on Python Shell and causing problem while running under Apache.

Error Message:

Error: ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')

Earlier I was using bashrc file to export the environment variables  and those working fine for Oracle drivers and others. But Even after adding to bashrc the connection raises an error for pyodbc. Don't know the reason.

I had tried setting it up in the Wsgi Script using os.putenv(). But still hadn't helped.

Any suggestion regarding that?

Thanks.