I try to access a Microsoft SQL database from Linux (Debian testing):
from sqlalchemy.ext.sqlsoup import SqlSoup
conn_string = 'mssql+pymssql://user:pass@freetds_name'
db = SqlSoup(conn_string)
v = db.some_table.first()
print v
freetds_name is the section name from /etc/freetds/freetds.conf
[freetds_name]
host = ...
port = 1433
tds version = 7.1
asa database = DB
The above script fails in pymssql on line 83, because line 81 sets vers to None:
def _get_server_version_info(self, connection):
vers = connection.scalar("select @@version")
m = re.match(
r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
if m:
return tuple(int(x) for x in m.group(1, 2, 3, 4))
else:
return None
But the following works in tsql:
1> select @@version
2> go
Microsoft SQL Server 2008 R2 (RTM) - 10.50.1617.0 (X64)
Apr 22 2011 19:23:43
Copyright (c) Microsoft Corporation
Workgroup Edition (64-bit) on Windows NT 6.1 <X64>
(Build 7601: Service Pack 1) (Hypervisor)
> in pymssql on line 83, because line 81 sets vers to None:
> def _get_server_version_info(self, connection):
> vers = connection.scalar("select @@version")
> m = re.match(
> r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
> if m:
> return tuple(int(x) for x in m.group(1, 2, 3, 4))
> else:
> return None
> But the following works in tsql:
> 1> select @@version
> 2> go
> Microsoft SQL Server 2008 R2 (RTM) - 10.50.1617.0 (X64)
> Apr 22 2011 19:23:43
> Copyright (c) Microsoft Corporation
> Workgroup Edition (64-bit) on Windows NT 6.1 <X64>
> (Build 7601: Service Pack 1) (Hypervisor)
> (1 row affected)
> Any idea what is wrong?
> Thank you,
> Ladislav Lenart
> -- > You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
I don't have access to my development environment during the weekend, so I cannot provide you with a stacktrace, but I try to better describe the issue:
def _get_server_version_info(self, connection)
vers = connection.scalar("select @@version")
m = re.match(r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
...
The above code snippet is from the file pymssql (around line 80). The variable vers is set to None and because of that the following regex fails with error "Expected string or buffer". The None is returned by the call to scalar(). The code snippet (from memory, I don't remember its exact location and form):
iter(resultproxy).next()
is called to get a next (first) result from the result proxy and it simply returns None as if there were no rows.
> On Oct 5, 2012, at 9:40 AM, Ladislav Lenart wrote:
> > Hello.
> > I try to access a Microsoft SQL database from Linux (Debian testing):
> > from sqlalchemy.ext.sqlsoup import SqlSoup
> > conn_string = 'mssql+pymssql://user:pass@freetds_name'
> > db = SqlSoup(conn_string)
> > v = db.some_table.first()
> > print v
> > freetds_name is the section name from /etc/freetds/freetds.conf
> > [freetds_name]
> > host = ...
> > port = 1433
> > tds version = 7.1
> > asa database = DB
> > The above script fails
> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
> > in pymssql on line 83, because line 81 sets vers to None:
> > def _get_server_version_info(self, connection):
> > vers = connection.scalar("select @@version")
> > m = re.match(
> > r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
> > if m:
> > return tuple(int(x) for x in m.group(1, 2, 3, 4))
> > else:
> > return None
> > But the following works in tsql:
> > 1> select @@version
> > 2> go
> > Microsoft SQL Server 2008 R2 (RTM) - 10.50.1617.0 (X64)
> > Apr 22 2011 19:23:43
> > Copyright (c) Microsoft Corporation
> > Workgroup Edition (64-bit) on Windows NT 6.1 <X64>
> > (Build 7601: Service Pack 1) (Hypervisor)
> > (1 row affected)
> > Any idea what is wrong?
> > Thank you,
> > Ladislav Lenart
> > -- > > You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> > To post to this group, send email to sqlalchemy@googlegroups.com.
> > To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
> > For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
> -- > You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
what I can do for the moment is this patch, if you want to try it:
diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
--- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
+++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
@@ -80,7 +80,7 @@
def _get_server_version_info(self, connection):
vers = connection.scalar("select @@version")
m = re.match(
- r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
+ r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
if m:
return tuple(int(x) for x in m.group(1, 2, 3, 4))
else:
otherwise, I'd consider using pyodbc for which the dialect and DBAPI are production quality. I use pyodbc with FreeTDS on unix platforms in production.
On Oct 5, 2012, at 1:40 PM, lenart...@volny.cz wrote:
>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
> I don't have access to my development environment during the weekend, so I cannot provide you with a stacktrace, but I try to better describe the issue:
> def _get_server_version_info(self, connection)
> vers = connection.scalar("select @@version")
> m = re.match(r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
> ...
> The above code snippet is from the file pymssql (around line 80). The variable vers is set to None and because of that the following regex fails with error "Expected string or buffer". The None is returned by the call to scalar(). The code snippet (from memory, I don't remember its exact location and form):
> iter(resultproxy).next()
> is called to get a next (first) result from the result proxy and it simply returns None as if there were no rows.
> Ladislav Lenart
> Od: "Michael Bayer" <mike...@zzzcomputing.com>
>> On Oct 5, 2012, at 9:40 AM, Ladislav Lenart wrote:
>>> Hello.
>>> I try to access a Microsoft SQL database from Linux (Debian testing):
>>> from sqlalchemy.ext.sqlsoup import SqlSoup
>>> conn_string = 'mssql+pymssql://user:pass@freetds_name'
>>> db = SqlSoup(conn_string)
>>> v = db.some_table.first()
>>> print v
>>> freetds_name is the section name from /etc/freetds/freetds.conf
>>> [freetds_name]
>>> host = ...
>>> port = 1433
>>> tds version = 7.1
>>> asa database = DB
>>> The above script fails
>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>>> in pymssql on line 83, because line 81 sets vers to None:
>>> def _get_server_version_info(self, connection):
>>> vers = connection.scalar("select @@version")
>>> m = re.match(
>>> r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> if m:
>>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>>> else:
>>> return None
>>> But the following works in tsql:
>>> 1> select @@version
>>> 2> go
>>> Microsoft SQL Server 2008 R2 (RTM) - 10.50.1617.0 (X64)
>>> Apr 22 2011 19:23:43
>>> Copyright (c) Microsoft Corporation
>>> Workgroup Edition (64-bit) on Windows NT 6.1 <X64>
>>> (Build 7601: Service Pack 1) (Hypervisor)
>>> (1 row affected)
>>> Any idea what is wrong?
>>> Thank you,
>>> Ladislav Lenart
>>> -- >>> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
>>> To post to this group, send email to sqlalchemy@googlegroups.com.
>>> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
>>> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>> -- >> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
>> To post to this group, send email to sqlalchemy@googlegroups.com.
>> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
> -- > You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are
production quality. I use pyodbc with FreeTDS on unix platforms in production.
Ok, I can use pyodbc if it is the preferred choice. However I cannot make it
work either. I suspect that I supply bad connection string but am a little lost
in the docs...
The code:
from sqlalchemy.ext.sqlsoup import SqlSoup
if __name__ == '__main__':
conn_string = 'mssql+pyodbc://username:pass\@w...@10.230.128.140:1433/ZFP_CRM'
db = SqlSoup(conn_string)
x = db.zfp_mlm_spol.first()
fails (see the traceback below). Note the password contains the character '@'.
The preceeding '\' is my attempt to escape it.
What connection string should I use to connect to MSSQL via pyodbc using freetds?
Thank you,
Ladislav Lenart
THE TRACEBACK:
Traceback (most recent call last):
File
"/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
line 1392, in <module>
debugger.run(setup['file'], None, None)
File
"/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
line 1085, in run
pydev_imports.execfile(file, globals, locals) #execute the script
File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 11, in <module>
x = db.zfp_mlm_spol.first()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
in __getattr__
return self.entity(attr)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
in entity
return self.map_to(attr, tablename=attr, schema=schema)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
in map_to
schema=schema or self.schema)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
table._init(name, metadata, *args, **kw)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
self._autoload(metadata, autoload_with, include_columns)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
_autoload
self, include_columns, exclude_columns
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2424,
in run_callable
conn = self.contextual_connect()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2490,
in contextual_connect
self.pool.connect(),
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 224, in connect
return _ConnectionFairy(self).checkout()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 387, in __init__
rec = self._connection_record = pool._do_get()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 741, in _do_get
con = self._create_connection()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 188, in
_create_connection
return _ConnectionRecord(self)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 270, in __init__
self.connection = self.__connect()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 330, in __connect
connection = self.__pool._creator()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
80, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
281, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.DBAPIError: (Error) ('IM002', '[IM002] [unixODBC][Driver
Manager]Data source name not found, and no default driver specified (0)
(SQLDriverConnectW)') None None
> what I can do for the moment is this patch, if you want to try it:
> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
> @@ -80,7 +80,7 @@
> def _get_server_version_info(self, connection):
> vers = connection.scalar("select @@version")
> m = re.match(
> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
> if m:
> return tuple(int(x) for x in m.group(1, 2, 3, 4))
> else:
> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are production quality. I use pyodbc with FreeTDS on unix platforms in production.
> On Oct 5, 2012, at 1:40 PM, lenart...@volny.cz wrote:
>> Hello.
>>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>> I don't have access to my development environment during the weekend, so I cannot provide you with a stacktrace, but I try to better describe the issue:
>> def _get_server_version_info(self, connection)
>> vers = connection.scalar("select @@version")
>> m = re.match(r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>> ...
>> The above code snippet is from the file pymssql (around line 80). The variable vers is set to None and because of that the following regex fails with error "Expected string or buffer". The None is returned by the call to scalar(). The code snippet (from memory, I don't remember its exact location and form):
>> iter(resultproxy).next()
>> is called to get a next (first) result from the result proxy and it simply returns None as if there were no rows.
>> Ladislav Lenart
>> Od: "Michael Bayer" <mike...@zzzcomputing.com>
>>> On Oct 5, 2012, at 9:40 AM, Ladislav Lenart wrote:
>>>> Hello.
>>>> I try to access a Microsoft SQL database from Linux (Debian testing):
>>>> from sqlalchemy.ext.sqlsoup import SqlSoup
>>>> conn_string = 'mssql+pymssql://user:pass@freetds_name'
>>>> db = SqlSoup(conn_string)
>>>> v = db.some_table.first()
>>>> print v
>>>> freetds_name is the section name from /etc/freetds/freetds.conf
>>>> [freetds_name]
>>>> host = ...
>>>> port = 1433
>>>> tds version = 7.1
>>>> asa database = DB
>>>> The above script fails
>>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>>>> in pymssql on line 83, because line 81 sets vers to None:
>>>> def _get_server_version_info(self, connection):
>>>> vers = connection.scalar("select @@version")
>>>> m = re.match(
>>>> r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>>> if m:
>>>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>>>> else:
>>>> return None
>>>> But the following works in tsql:
>>>> 1> select @@version
>>>> 2> go
>>>> Microsoft SQL Server 2008 R2 (RTM) - 10.50.1617.0 (X64)
>>>> Apr 22 2011 19:23:43
>>>> Copyright (c) Microsoft Corporation
>>>> Workgroup Edition (64-bit) on Windows NT 6.1 <X64>
>>>> (Build 7601: Service Pack 1) (Hypervisor)
>>>> (1 row affected)
>>>> Any idea what is wrong?
>>>> Thank you,
>>>> Ladislav Lenart
>>>> -- >>>> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
>>>> To post to this group, send email to sqlalchemy@googlegroups.com.
>>>> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
>>>> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>>> -- >>> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
>>> To post to this group, send email to sqlalchemy@googlegroups.com.
>>> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
>>> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>> -- >> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
>> To post to this group, send email to sqlalchemy@googlegroups.com.
>> To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
It turned out that I was missing some ODBC-related packages and also needed to
configure freetds to work with unixodbc. I managed to finally do it, though it
was by no means easy for me (trial and error of several tutorials).
import pyodbc
from sqlalchemy.engine import create_engine
from sqlalchemy.ext.sqlsoup import SqlSoup
def connect():
return
pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0;')
engine = create_engine('mssql+pyodbc://', creator=connect)
db = SqlSoup(engine)
row = db.mlm_spol.first()
print row
fails with:
Traceback (most recent call last):
File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 16, in <module>
x = db.mlm_spol.first()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
in __getattr__
return self.entity(attr)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
in entity
return self.map_to(attr, tablename=attr, schema=schema)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
in map_to
schema=schema or self.schema)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
table._init(name, metadata, *args, **kw)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
self._autoload(metadata, autoload_with, include_columns)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
_autoload
self, include_columns, exclude_columns
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2426,
in run_callable
return conn.run_callable(callable_, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1969,
in run_callable
return callable_(self, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
260, in reflecttable
return insp.reflecttable(table, include_columns, exclude_columns)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/reflection.py", line
412, in reflecttable
raise exc.NoSuchTableError(table.name)
sqlalchemy.exc.NoSuchTableError: mlm_spol
>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are
> production quality. I use pyodbc with FreeTDS on unix platforms in production.
> Ok, I can use pyodbc if it is the preferred choice. However I cannot make it
> work either. I suspect that I supply bad connection string but am a little lost
> in the docs...
> The code:
> from sqlalchemy.ext.sqlsoup import SqlSoup
> if __name__ == '__main__':
> conn_string = 'mssql+pyodbc://username:pass\@w...@10.230.128.140:1433/ZFP_CRM'
> db = SqlSoup(conn_string)
> x = db.zfp_mlm_spol.first()
> fails (see the traceback below). Note the password contains the character '@'.
> The preceeding '\' is my attempt to escape it.
> What connection string should I use to connect to MSSQL via pyodbc using freetds?
> Thank you,
> Ladislav Lenart
> THE TRACEBACK:
> Traceback (most recent call last):
> File
> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
> line 1392, in <module>
> debugger.run(setup['file'], None, None)
> File
> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
> line 1085, in run
> pydev_imports.execfile(file, globals, locals) #execute the script
> File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 11, in <module>
> x = db.zfp_mlm_spol.first()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
> in __getattr__
> return self.entity(attr)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
> in entity
> return self.map_to(attr, tablename=attr, schema=schema)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
> in map_to
> schema=schema or self.schema)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
> table._init(name, metadata, *args, **kw)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
> self._autoload(metadata, autoload_with, include_columns)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
> _autoload
> self, include_columns, exclude_columns
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2424,
> in run_callable
> conn = self.contextual_connect()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2490,
> in contextual_connect
> self.pool.connect(),
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 224, in connect
> return _ConnectionFairy(self).checkout()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 387, in __init__
> rec = self._connection_record = pool._do_get()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 741, in _do_get
> con = self._create_connection()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 188, in
> _create_connection
> return _ConnectionRecord(self)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 270, in __init__
> self.connection = self.__connect()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 330, in __connect
> connection = self.__pool._creator()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
> 80, in connect
> return dialect.connect(*cargs, **cparams)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
> 281, in connect
> return self.dbapi.connect(*cargs, **cparams)
> sqlalchemy.exc.DBAPIError: (Error) ('IM002', '[IM002] [unixODBC][Driver
> Manager]Data source name not found, and no default driver specified (0)
> (SQLDriverConnectW)') None None
> On 6.10.2012 00:47, Michael Bayer wrote:
>> what I can do for the moment is this patch, if you want to try it:
>> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
>> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
>> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
>> @@ -80,7 +80,7 @@
>> def _get_server_version_info(self, connection):
>> vers = connection.scalar("select @@version")
>> m = re.match(
>> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>> if m:
>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>> else:
>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are production quality. I use pyodbc with FreeTDS on unix platforms in production.
>> On Oct 5, 2012, at 1:40 PM, lenart...@volny.cz wrote:
>>> Hello.
>>>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>>> I don't have access to my development environment during the weekend, so I cannot provide you with a stacktrace, but I try to better describe the issue:
>>> def _get_server_version_info(self, connection)
>>> vers = connection.scalar("select @@version")
>>> m = re.match(r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> ...
>>> The above code snippet is from the file pymssql (around line 80). The variable vers is set to None and because of that the following regex fails with error "Expected string or buffer". The None is returned by the call to scalar(). The code snippet (from memory, I don't remember its exact location and form):
>>> iter(resultproxy).next()
>>> is called to get a next (first) result from the result proxy and it simply returns None as if there were no rows.
>>> Ladislav Lenart
>>> Od: "Michael Bayer" <mike...@zzzcomputing.com>
>>>> On Oct 5, 2012, at 9:40 AM, Ladislav Lenart wrote:
>>>>> Hello.
>>>>> I try to access a Microsoft SQL database from Linux (Debian testing):
>>>>> from sqlalchemy.ext.sqlsoup import SqlSoup
>>>>> conn_string = 'mssql+pymssql://user:pass@freetds_name'
>>>>> db = SqlSoup(conn_string)
>>>>> v = db.some_table.first()
>>>>> print v
>>>>> freetds_name is the section name from /etc/freetds/freetds.conf
>>>>> [freetds_name]
>>>>> host = ...
>>>>> port = 1433
>>>>> tds version = 7.1
>>>>> asa database = DB
>>>>> The above script fails
>>>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>>>>> in pymssql on line 83, because line 81 sets vers to None:
>>>>> def _get_server_version_info(self, connection):
>>>>> vers = connection.scalar("select @@version")
>>>>> m = re.match(
>>>>> r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>>>> if m:
>>>>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>>>>> else:
>>>>> return None
import pyodbc
from sqlalchemy.engine import create_engine
from sqlalchemy.ext.sqlsoup import SqlSoup
def connect():
return
pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0;')
engine = create_engine('mssql+pyodbc://', creator=connect)
db = SqlSoup(engine)
x = db.execute("select * from mlm_spol").fetchone()
print x
Any ideas what I have to do to make SqlSoup's ORM work? I am out of ideas.
> It turned out that I was missing some ODBC-related packages and also needed to
> configure freetds to work with unixodbc. I managed to finally do it, though it
> was by no means easy for me (trial and error of several tutorials).
> Traceback (most recent call last):
> File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 16, in <module>
> x = db.mlm_spol.first()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
> in __getattr__
> return self.entity(attr)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
> in entity
> return self.map_to(attr, tablename=attr, schema=schema)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
> in map_to
> schema=schema or self.schema)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
> table._init(name, metadata, *args, **kw)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
> self._autoload(metadata, autoload_with, include_columns)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
> _autoload
> self, include_columns, exclude_columns
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2426,
> in run_callable
> return conn.run_callable(callable_, *args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1969,
> in run_callable
> return callable_(self, *args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
> 260, in reflecttable
> return insp.reflecttable(table, include_columns, exclude_columns)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/reflection.py", line
> 412, in reflecttable
> raise exc.NoSuchTableError(table.name)
> sqlalchemy.exc.NoSuchTableError: mlm_spol
> Any ideas?
> Thanks,
> Ladislav Lenart
> On 8.10.2012 11:11, Ladislav Lenart wrote:
>> Hello.
>>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are
>> production quality. I use pyodbc with FreeTDS on unix platforms in production.
>> Ok, I can use pyodbc if it is the preferred choice. However I cannot make it
>> work either. I suspect that I supply bad connection string but am a little lost
>> in the docs...
>> The code:
>> from sqlalchemy.ext.sqlsoup import SqlSoup
>> if __name__ == '__main__':
>> conn_string = 'mssql+pyodbc://username:pass\@w...@10.230.128.140:1433/ZFP_CRM'
>> db = SqlSoup(conn_string)
>> x = db.zfp_mlm_spol.first()
>> fails (see the traceback below). Note the password contains the character '@'.
>> The preceeding '\' is my attempt to escape it.
>> What connection string should I use to connect to MSSQL via pyodbc using freetds?
>> Thank you,
>> Ladislav Lenart
>> THE TRACEBACK:
>> Traceback (most recent call last):
>> File
>> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
>> line 1392, in <module>
>> debugger.run(setup['file'], None, None)
>> File
>> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
>> line 1085, in run
>> pydev_imports.execfile(file, globals, locals) #execute the script
>> File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 11, in <module>
>> x = db.zfp_mlm_spol.first()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
>> in __getattr__
>> return self.entity(attr)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
>> in entity
>> return self.map_to(attr, tablename=attr, schema=schema)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
>> in map_to
>> schema=schema or self.schema)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
>> table._init(name, metadata, *args, **kw)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
>> self._autoload(metadata, autoload_with, include_columns)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
>> _autoload
>> self, include_columns, exclude_columns
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2424,
>> in run_callable
>> conn = self.contextual_connect()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2490,
>> in contextual_connect
>> self.pool.connect(),
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 224, in connect
>> return _ConnectionFairy(self).checkout()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 387, in __init__
>> rec = self._connection_record = pool._do_get()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 741, in _do_get
>> con = self._create_connection()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 188, in
>> _create_connection
>> return _ConnectionRecord(self)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 270, in __init__
>> self.connection = self.__connect()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 330, in __connect
>> connection = self.__pool._creator()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
>> 80, in connect
>> return dialect.connect(*cargs, **cparams)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
>> 281, in connect
>> return self.dbapi.connect(*cargs, **cparams)
>> sqlalchemy.exc.DBAPIError: (Error) ('IM002', '[IM002] [unixODBC][Driver
>> Manager]Data source name not found, and no default driver specified (0)
>> (SQLDriverConnectW)') None None
>> On 6.10.2012 00:47, Michael Bayer wrote:
>>> what I can do for the moment is this patch, if you want to try it:
>>> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
>>> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
>>> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
>>> @@ -80,7 +80,7 @@
>>> def _get_server_version_info(self, connection):
>>> vers = connection.scalar("select @@version")
>>> m = re.match(
>>> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> if m:
>>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>>> else:
>>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are production quality. I use pyodbc with FreeTDS on unix platforms in production.
>>> On Oct 5, 2012, at 1:40 PM, lenart...@volny.cz wrote:
>>>> Hello.
>>>>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>>>> I don't have access to my development environment during the weekend, so I cannot provide you with a stacktrace, but I try to better describe the issue:
>>>> def _get_server_version_info(self, connection)
>>>> vers = connection.scalar("select @@version")
>>>> m = re.match(r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>>> ...
>>>> The above code snippet is from the file pymssql (around line 80). The variable vers is set to None and because of that the following regex fails with error "Expected string or buffer". The None is returned by the call to scalar(). The code snippet (from memory, I don't remember its exact location and form):
>>>> iter(resultproxy).next()
>>>> is called to get a next (first) result from the result proxy and it simply returns None as if there were no rows.
>>>> Ladislav Lenart
>>>> Od: "Michael Bayer" <mike...@zzzcomputing.com>
>>>>> On Oct 5, 2012, at 9:40 AM, Ladislav Lenart wrote:
>>>>>> Hello.
>>>>>> I try to access a Microsoft SQL database from Linux (Debian testing):
>>>>>> from sqlalchemy.ext.sqlsoup import SqlSoup
>>>>>> conn_string = 'mssql+pymssql://user:pass@freetds_name'
>>>>>> db = SqlSoup(conn_string)
>>>>>> v = db.some_table.first()
>>>>>> print v
"no such table" usually means you're not connected to the database that you think you are.
I'd strongly suggest configuring an ODBC datasource within FreeTDS, and using standard connection techniques. Hostname, port, tds version go into freetds.conf, and database names go into odbc.ini.
In freetds conf for example I have:
[ms_2005]
host = 172.16.248.128
port = 1213
tds version = 8.0
client charset = UTF8
text size = 50000000
and on my mac in /Library/ODBC/odbc.ini I have:
[ODBC Data Sources]
ms_2005 = test
[ms_2005]
Driver = /usr/local/lib/libtdsodbc.so
Description = test
Trace = No
Servername = ms_2005
> It turned out that I was missing some ODBC-related packages and also needed to
> configure freetds to work with unixodbc. I managed to finally do it, though it
> was by no means easy for me (trial and error of several tutorials).
> Traceback (most recent call last):
> File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 16, in <module>
> x = db.mlm_spol.first()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
> in __getattr__
> return self.entity(attr)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
> in entity
> return self.map_to(attr, tablename=attr, schema=schema)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
> in map_to
> schema=schema or self.schema)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
> table._init(name, metadata, *args, **kw)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
> self._autoload(metadata, autoload_with, include_columns)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
> _autoload
> self, include_columns, exclude_columns
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2426,
> in run_callable
> return conn.run_callable(callable_, *args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1969,
> in run_callable
> return callable_(self, *args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
> 260, in reflecttable
> return insp.reflecttable(table, include_columns, exclude_columns)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/reflection.py", line
> 412, in reflecttable
> raise exc.NoSuchTableError(table.name)
> sqlalchemy.exc.NoSuchTableError: mlm_spol
> Any ideas?
> Thanks,
> Ladislav Lenart
> On 8.10.2012 11:11, Ladislav Lenart wrote:
>> Hello.
>>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are
>> production quality. I use pyodbc with FreeTDS on unix platforms in production.
>> Ok, I can use pyodbc if it is the preferred choice. However I cannot make it
>> work either. I suspect that I supply bad connection string but am a little lost
>> in the docs...
>> The code:
>> from sqlalchemy.ext.sqlsoup import SqlSoup
>> if __name__ == '__main__':
>> conn_string = 'mssql+pyodbc://username:pass\@w...@10.230.128.140:1433/ZFP_CRM'
>> db = SqlSoup(conn_string)
>> x = db.zfp_mlm_spol.first()
>> fails (see the traceback below). Note the password contains the character '@'.
>> The preceeding '\' is my attempt to escape it.
>> What connection string should I use to connect to MSSQL via pyodbc using freetds?
>> Thank you,
>> Ladislav Lenart
>> THE TRACEBACK:
>> Traceback (most recent call last):
>> File
>> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
>> line 1392, in <module>
>> debugger.run(setup['file'], None, None)
>> File
>> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
>> line 1085, in run
>> pydev_imports.execfile(file, globals, locals) #execute the script
>> File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 11, in <module>
>> x = db.zfp_mlm_spol.first()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
>> in __getattr__
>> return self.entity(attr)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
>> in entity
>> return self.map_to(attr, tablename=attr, schema=schema)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
>> in map_to
>> schema=schema or self.schema)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
>> table._init(name, metadata, *args, **kw)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
>> self._autoload(metadata, autoload_with, include_columns)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
>> _autoload
>> self, include_columns, exclude_columns
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2424,
>> in run_callable
>> conn = self.contextual_connect()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2490,
>> in contextual_connect
>> self.pool.connect(),
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 224, in connect
>> return _ConnectionFairy(self).checkout()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 387, in __init__
>> rec = self._connection_record = pool._do_get()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 741, in _do_get
>> con = self._create_connection()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 188, in
>> _create_connection
>> return _ConnectionRecord(self)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 270, in __init__
>> self.connection = self.__connect()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 330, in __connect
>> connection = self.__pool._creator()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
>> 80, in connect
>> return dialect.connect(*cargs, **cparams)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
>> 281, in connect
>> return self.dbapi.connect(*cargs, **cparams)
>> sqlalchemy.exc.DBAPIError: (Error) ('IM002', '[IM002] [unixODBC][Driver
>> Manager]Data source name not found, and no default driver specified (0)
>> (SQLDriverConnectW)') None None
>> On 6.10.2012 00:47, Michael Bayer wrote:
>>> what I can do for the moment is this patch, if you want to try it:
>>> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
>>> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
>>> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
>>> @@ -80,7 +80,7 @@
>>> def _get_server_version_info(self, connection):
>>> vers = connection.scalar("select @@version")
>>> m = re.match(
>>> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> if m:
>>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>>> else:
>>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are production quality. I use pyodbc with FreeTDS on unix platforms in production.
>>> On Oct 5, 2012, at 1:40 PM, lenart...@volny.cz wrote:
>>>> Hello.
>>>>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>>>> I don't have access to my development environment during the weekend, so I cannot provide you with a stacktrace, but I try to better describe the issue:
>>>> def _get_server_version_info(self, connection)
>>>> vers = connection.scalar("select @@version")
>>>> m = re.match(r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>>> ...
>>>> The above code snippet is from the file pymssql (around line 80). The variable vers is set to None and because of that the following regex fails with error "Expected string or buffer". The None is returned by the call to scalar(). The code snippet (from memory, I don't remember its exact location and form):
>>>> iter(resultproxy).next()
>>>> is called to get a next (first) result from the result proxy and it simply returns None as if there were no rows.
>>>> Ladislav Lenart
>>>> Od: "Michael Bayer" <mike...@zzzcomputing.com>
>>>>> On Oct 5, 2012, at 9:40 AM, Ladislav Lenart wrote:
>>>>>> Hello.
>>>>>> I try to access a Microsoft SQL database from
Ok, I will give it yet another try, but please note that the following works:
import pyodbc
from sqlalchemy.engine import create_engine
from sqlalchemy.ext.sqlsoup import SqlSoup
def connect():
return
pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0;')
engine = create_engine('mssql+pyodbc://', creator=connect)
db = SqlSoup(engine)
x = db.execute("select * from mlm_spol").fetchone()
print x
i.e. raw SqlSoup.execute() sees the table and returns a valid result.
I regard this as a proof that the connection was established successfully and
the problem lies elsewhere. Am I wrong?
> "no such table" usually means you're not connected to the database that you think you are.
> I'd strongly suggest configuring an ODBC datasource within FreeTDS, and using standard connection techniques. Hostname, port, tds version go into freetds.conf, and database names go into odbc.ini.
> In freetds conf for example I have:
> [ms_2005]
> host = 172.16.248.128
> port = 1213
> tds version = 8.0
> client charset = UTF8
> text size = 50000000
> and on my mac in /Library/ODBC/odbc.ini I have:
> [ODBC Data Sources]
> ms_2005 = test
> [ms_2005]
> Driver = /usr/local/lib/libtdsodbc.so
> Description = test
> Trace = No
> Servername = ms_2005
> On Oct 8, 2012, at 9:02 AM, Ladislav Lenart wrote:
>> Hello again.
>> It turned out that I was missing some ODBC-related packages and also needed to
>> configure freetds to work with unixodbc. I managed to finally do it, though it
>> was by no means easy for me (trial and error of several tutorials).
>> Traceback (most recent call last):
>> File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 16, in <module>
>> x = db.mlm_spol.first()
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
>> in __getattr__
>> return self.entity(attr)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
>> in entity
>> return self.map_to(attr, tablename=attr, schema=schema)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
>> in map_to
>> schema=schema or self.schema)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
>> table._init(name, metadata, *args, **kw)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
>> self._autoload(metadata, autoload_with, include_columns)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
>> _autoload
>> self, include_columns, exclude_columns
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2426,
>> in run_callable
>> return conn.run_callable(callable_, *args, **kwargs)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1969,
>> in run_callable
>> return callable_(self, *args, **kwargs)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
>> 260, in reflecttable
>> return insp.reflecttable(table, include_columns, exclude_columns)
>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/reflection.py", line
>> 412, in reflecttable
>> raise exc.NoSuchTableError(table.name)
>> sqlalchemy.exc.NoSuchTableError: mlm_spol
>> Any ideas?
>> Thanks,
>> Ladislav Lenart
>> On 8.10.2012 11:11, Ladislav Lenart wrote:
>>> Hello.
>>>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are
>>> production quality. I use pyodbc with FreeTDS on unix platforms in production.
>>> Ok, I can use pyodbc if it is the preferred choice. However I cannot make it
>>> work either. I suspect that I supply bad connection string but am a little lost
>>> in the docs...
>>> The code:
>>> from sqlalchemy.ext.sqlsoup import SqlSoup
>>> if __name__ == '__main__':
>>> conn_string = 'mssql+pyodbc://username:pass\@w...@10.230.128.140:1433/ZFP_CRM'
>>> db = SqlSoup(conn_string)
>>> x = db.zfp_mlm_spol.first()
>>> fails (see the traceback below). Note the password contains the character '@'.
>>> The preceeding '\' is my attempt to escape it.
>>> What connection string should I use to connect to MSSQL via pyodbc using freetds?
>>> Thank you,
>>> Ladislav Lenart
>>> THE TRACEBACK:
>>> Traceback (most recent call last):
>>> File
>>> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
>>> line 1392, in <module>
>>> debugger.run(setup['file'], None, None)
>>> File
>>> "/home/lada/.eclipse/org.eclipse.platform_3.8_155965261/plugins/org.python. pydev_2.6.0.2012062818/pysrc/pydevd.py",
>>> line 1085, in run
>>> pydev_imports.execfile(file, globals, locals) #execute the script
>>> File "/home/lada/mine/devel/python/ZFP/zfp_connect.py", line 11, in <module>
>>> x = db.zfp_mlm_spol.first()
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 807,
>>> in __getattr__
>>> return self.entity(attr)
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 804,
>>> in entity
>>> return self.map_to(attr, tablename=attr, schema=schema)
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/ext/sqlsoup.py", line 684,
>>> in map_to
>>> schema=schema or self.schema)
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 318, in __new__
>>> table._init(name, metadata, *args, **kw)
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 381, in _init
>>> self._autoload(metadata, autoload_with, include_columns)
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 397, in
>>> _autoload
>>> self, include_columns, exclude_columns
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2424,
>>> in run_callable
>>> conn = self.contextual_connect()
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2490,
>>> in contextual_connect
>>> self.pool.connect(),
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 224, in connect
>>> return _ConnectionFairy(self).checkout()
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 387, in __init__
>>> rec = self._connection_record = pool._do_get()
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 741, in _do_get
>>> con = self._create_connection()
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 188, in
>>> _create_connection
>>> return _ConnectionRecord(self)
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 270, in __init__
>>> self.connection = self.__connect()
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 330, in __connect
>>> connection = self.__pool._creator()
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
>>> 80, in connect
>>> return dialect.connect(*cargs, **cparams)
>>> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line
>>> 281, in connect
>>> return self.dbapi.connect(*cargs, **cparams)
>>> sqlalchemy.exc.DBAPIError: (Error) ('IM002', '[IM002] [unixODBC][Driver
>>> Manager]Data source name not found, and no default driver specified (0)
>>> (SQLDriverConnectW)') None None
>>> On 6.10.2012 00:47, Michael Bayer wrote:
>>>> what I can do for the moment is this patch, if you want to try it:
>>>> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
>>>> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
>>>> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
>>>> @@ -80,7 +80,7 @@
>>>> def _get_server_version_info(self, connection):
>>>> vers = connection.scalar("select @@version")
>>>> m = re.match(
>>>> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>>> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>>> if m:
>>>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>>>> else:
>>>> otherwise, I'd consider using pyodbc for which the dialect and DBAPI are production quality. I use pyodbc with FreeTDS on unix platforms in production.
>>>> On Oct 5, 2012, at 1:40 PM, lenart...@volny.cz wrote:
>>>>> Hello.
>>>>>> I dont have easy access to pymssql here so can you fully define what "fails" means ? stack trace ?
>>>>> I don't have access to my development environment during the weekend, so I cannot provide you with a stacktrace, but I try
> Ok, I will give it yet another try, but please note that the following works:
> import pyodbc
> from sqlalchemy.engine import create_engine
> from sqlalchemy.ext.sqlsoup import SqlSoup
> def connect():
> return
> pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0;')
> engine = create_engine('mssql+pyodbc://', creator=connect)
> db = SqlSoup(engine)
> x = db.execute("select * from mlm_spol").fetchone()
> print x
> i.e. raw SqlSoup.execute() sees the table and returns a valid result.
> I regard this as a proof that the connection was established successfully and
> the problem lies elsewhere. Am I wrong?
perhaps. Maybe the connection doesn't have correct access to the information schema tables, as SQLSoup relies upon table reflection. you'd need to run with echo='debug' on your engine to see exactly what queries are being emitted and the rows being returned, and determine why an information schema row for "mlm_spol" isn't being returned.
> perhaps. Maybe the connection doesn't have correct access to the information
schema tables, as SQLSoup relies upon table reflection. you'd need to run with
echo='debug' on your engine to see exactly what queries are being emitted and
the rows being returned, and determine why an information schema row for
"mlm_spol" isn't being returned.
I did it and it seems that there is a str/unicode problem when dealing with the
information shema (see below, it's rather long).
I tried it with charset=UTF8 to pyodbc.connect string. I also tried various
combinations of create_engine options supports_unicode_binds and convert_unicode
but none of them worked.
Ladislav Lenart
CODE:
import pyodbc
from sqlalchemy.engine import create_engine
from sqlalchemy.ext.sqlsoup import SqlSoup
def connect():
return
pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0;')
engine = create_engine('mssql+pyodbc://', creator=connect, echo='debug')
db = SqlSoup(engine)
x = db.execute("select * from mlm_spol").fetchone()
print x
ITS DEBUG OUTPUT:
pydev debugger: starting
2012-10-08 17:19:53,239 INFO sqlalchemy.engine.base.Engine SELECT user_name() as
user_name;
2012-10-08 17:19:53,240 INFO sqlalchemy.engine.base.Engine ()
2012-10-08 17:19:53,246 DEBUG sqlalchemy.engine.base.Engine Col ('user_name',)
2012-10-08 17:19:53,247 DEBUG sqlalchemy.engine.base.Engine Row
(u'\U00660065\U00610072\U00740063\U006c0061', )
2012-10-08 17:19:53,253 INFO sqlalchemy.engine.base.Engine
SELECT default_schema_name FROM
sys.database_principals
WHERE name = ?
AND type = 'S'
import pyodbc
from sqlalchemy.engine import create_engine
from sqlalchemy.ext.sqlsoup import SqlSoup
def connect():
return
pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0;')
engine = create_engine('mssql+pyodbc://', creator=connect, echo='debug')
db = SqlSoup(engine)
x = db.mlm_spol.fetchone()
print x
ITS DEBUG OUTPUT:
pydev debugger: starting
2012-10-08 17:33:07,175 INFO sqlalchemy.engine.base.Engine SELECT user_name() as
user_name;
2012-10-08 17:33:07,176 INFO sqlalchemy.engine.base.Engine ()
2012-10-08 17:33:07,181 DEBUG sqlalchemy.engine.base.Engine Col ('user_name',)
2012-10-08 17:33:07,182 DEBUG sqlalchemy.engine.base.Engine Row
(u'\U00660065\U00610072\U00740063\U006c0061', )
2012-10-08 17:33:07,188 INFO sqlalchemy.engine.base.Engine
SELECT default_schema_name FROM
sys.database_principals
WHERE name = ?
AND type = 'S'
>> perhaps. Maybe the connection doesn't have correct access to the information
> schema tables, as SQLSoup relies upon table reflection. you'd need to run with
> echo='debug' on your engine to see exactly what queries are being emitted and
> the rows being returned, and determine why an information schema row for
> "mlm_spol" isn't being returned.
> I did it and it seems that there is a str/unicode problem when dealing with the
> information shema (see below, it's rather long).
> I tried it with charset=UTF8 to pyodbc.connect string. I also tried various
> combinations of create_engine options supports_unicode_binds and convert_unicode
> but none of them worked.
> Ladislav Lenart
> CODE:
> import pyodbc
> from sqlalchemy.engine import create_engine
> from sqlalchemy.ext.sqlsoup import SqlSoup
> def connect():
> return
> pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0;')
> engine = create_engine('mssql+pyodbc://', creator=connect, echo='debug')
> db = SqlSoup(engine)
> x = db.execute("select * from mlm_spol").fetchone()
> print x
> ITS DEBUG OUTPUT:
> pydev debugger: starting
> 2012-10-08 17:19:53,239 INFO sqlalchemy.engine.base.Engine SELECT user_name() as
> user_name;
> 2012-10-08 17:19:53,240 INFO sqlalchemy.engine.base.Engine ()
> 2012-10-08 17:19:53,246 DEBUG sqlalchemy.engine.base.Engine Col ('user_name',)
> 2012-10-08 17:19:53,247 DEBUG sqlalchemy.engine.base.Engine Row
> (u'\U00660065\U00610072\U00740063\U006c0061', )
> 2012-10-08 17:19:53,253 INFO sqlalchemy.engine.base.Engine
> SELECT default_schema_name FROM
> sys.database_principals
> WHERE name = ?
> AND type = 'S'
from sqlalchemy.engine import create_engine
from sqlalchemy.ext.sqlsoup import SqlSoup
if __name__ == '__main__':
engine = create_engine("mssql://efractal:efR@cZFP13@zfp", echo='debug')
db = SqlSoup(engine)
x = db.mlm_spol.fetchone()
print x
still fails with NoSuchTableError: mlm_spol
Its debug output is the same as in my previous email which used different
connect style.
And the code that uses db.execute('select * from mlm_spol').fetchone() still
works...
if you didnt have this problem with pymssql then please apply the patch I sent previously.
However, I'm going to bet the problem remains as it seems something is not right with how your database and/or client is configured. I'm not familiar with the encoding pattern seen in your information_schema queries. you may have to reproduce the issue using plain SELECT statements against information_schema and then seek outside help.
diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
--- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
+++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
@@ -80,7 +80,7 @@
def _get_server_version_info(self, connection):
vers = connection.scalar("select @@version")
m = re.match(
- r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
+ r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
if m:
return tuple(int(x) for x in m.group(1, 2, 3, 4))
else:
On Oct 8, 2012, at 12:10 PM, Ladislav Lenart wrote:
No, I was not able to connect via pymssql. Furthemore I would like to use pyodbc
if it is the preferred way. I just did not know that when I started with pymssql.
It really is a misconfigured character encoding issue in pyodbc / freetds on my
part. I am just clueless as to what should I set to what to make it all work.
The code:
cnxn =
pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0')
cursor = cnxn.cursor()
cursor.execute("select prijmeni from osoba where id_osoba = 462493")
row = cursor.fetchone()
print row
prints:
('Ne?asov', )
The correct value is 'Nečasová'. Ideally I would like to see the following:
(u'Nečasová', )
On a side note, the returned value is one character shorter. I would expected:
('Ne?asov?', )
However, when I connect via tsql, it works correcly:
tsql -S zfp -U efractal
Password:
locale is "cs_CZ.UTF-8"
locale charset is "UTF-8"
using default charset "utf8"
1> select prijmeni from osoba where id_osoba = 462493
2> go
prijmeni
Nečasová
(1 row affected)
1>
I guess I am on my own now. Anyway, thank you for your kind assistance.
> if you didnt have this problem with pymssql then please apply the patch I sent previously.
> However, I'm going to bet the problem remains as it seems something is not right with how your database and/or client is configured. I'm not familiar with the encoding pattern seen in your information_schema queries. you may have to reproduce the issue using plain SELECT statements against information_schema and then seek outside help.
> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
> @@ -80,7 +80,7 @@
> def _get_server_version_info(self, connection):
> vers = connection.scalar("select @@version")
> m = re.match(
> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
> if m:
> return tuple(int(x) for x in m.group(1, 2, 3, 4))
> else:
> On Oct 8, 2012, at 12:10 PM, Ladislav Lenart wrote:
>> Hello.
>> I adjusted the ODBC/FreeTDS condifugration according to your suggestions but
>> still get the "NoSuchTableError: mlm_spol".
>> freetds.conf:
>> [zfp]
>> host = 10.230.128.140
>> port = 1433
>> tds version = 8.0
>> asa database = ZFP_CRM
>> client charset = utf8
>> text size = 50000000
>> odbc.ini:
>> [ODBC Data Sources]
>> zfp = test
>> [zfp]
>> Driver = /usr/lib/libtdsodbc.so
>> Description = test
>> Trace = No
>> Servername = zfp
> No, I was not able to connect via pymssql. Furthemore I would like to use pyodbc
> if it is the preferred way. I just did not know that when I started with pymssql.
> It really is a misconfigured character encoding issue in pyodbc / freetds on my
> part. I am just clueless as to what should I set to what to make it all work.
> The code:
> cnxn =
> pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0')
> cursor = cnxn.cursor()
> cursor.execute("select prijmeni from osoba where id_osoba = 462493")
> row = cursor.fetchone()
> print row
> prints:
> ('Ne?asov', )
> The correct value is 'Nečasová'. Ideally I would like to see the following:
> (u'Nečasová', )
> On a side note, the returned value is one character shorter. I would expected:
> ('Ne?asov?', )
> However, when I connect via tsql, it works correcly:
> tsql -S zfp -U efractal
> Password:
> locale is "cs_CZ.UTF-8"
> locale charset is "UTF-8"
> using default charset "utf8"
> 1> select prijmeni from osoba where id_osoba = 462493
> 2> go
> prijmeni
> Nečasová
> (1 row affected)
> 1>
> I guess I am on my own now. Anyway, thank you for your kind assistance.
> Ladislav Lenart
> On 8.10.2012 19:06, Michael Bayer wrote:
>> if you didnt have this problem with pymssql then please apply the patch I sent previously.
>> However, I'm going to bet the problem remains as it seems something is not right with how your database and/or client is configured. I'm not familiar with the encoding pattern seen in your information_schema queries. you may have to reproduce the issue using plain SELECT statements against information_schema and then seek outside help.
>> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
>> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
>> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
>> @@ -80,7 +80,7 @@
>> def _get_server_version_info(self, connection):
>> vers = connection.scalar("select @@version")
>> m = re.match(
>> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>> if m:
>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>> else:
>> On Oct 8, 2012, at 12:10 PM, Ladislav Lenart wrote:
>>> Hello.
>>> I adjusted the ODBC/FreeTDS condifugration according to your suggestions but
>>> still get the "NoSuchTableError: mlm_spol".
>>> freetds.conf:
>>> [zfp]
>>> host = 10.230.128.140
>>> port = 1433
>>> tds version = 8.0
>>> asa database = ZFP_CRM
>>> client charset = utf8
>>> text size = 50000000
>>> odbc.ini:
>>> [ODBC Data Sources]
>>> zfp = test
>>> [zfp]
>>> Driver = /usr/lib/libtdsodbc.so
>>> Description = test
>>> Trace = No
>>> Servername = zfp
> No, I was not able to connect via pymssql. Furthemore I would like to use pyodbc
> if it is the preferred way. I just did not know that when I started with pymssql.
> It really is a misconfigured character encoding issue in pyodbc / freetds on my
> part. I am just clueless as to what should I set to what to make it all work.
> The code:
> cnxn =
> pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0')
> cursor = cnxn.cursor()
> cursor.execute("select prijmeni from osoba where id_osoba = 462493")
> row = cursor.fetchone()
> print row
> prints:
> ('Ne?asov', )
you definitely, definitely need "client charset" to be part of your FreeTDS config, either in that URL string up there (not sure if that's supported), or preferably in your freetds.conf. pyodbc is pretty sensitive to this.
> However, when I connect via tsql, it works correcly:
> tsql -S zfp -U efractal
> Password:
> locale is "cs_CZ.UTF-8"
> locale charset is "UTF-8"
> using default charset "utf8"
> 1> select prijmeni from osoba where id_osoba = 462493
> 2> go
> prijmeni
> Nečasová
> (1 row affected)
> 1>
> I guess I am on my own now. Anyway, thank you for your kind assistance.
> Ladislav Lenart
> On 8.10.2012 19:06, Michael Bayer wrote:
>> if you didnt have this problem with pymssql then please apply the patch I sent previously.
>> However, I'm going to bet the problem remains as it seems something is not right with how your database and/or client is configured. I'm not familiar with the encoding pattern seen in your information_schema queries. you may have to reproduce the issue using plain SELECT statements against information_schema and then seek outside help.
>> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
>> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
>> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
>> @@ -80,7 +80,7 @@
>> def _get_server_version_info(self, connection):
>> vers = connection.scalar("select @@version")
>> m = re.match(
>> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>> if m:
>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>> else:
>> On Oct 8, 2012, at 12:10 PM, Ladislav Lenart wrote:
>>> Hello.
>>> I adjusted the ODBC/FreeTDS condifugration according to your suggestions but
>>> still get the "NoSuchTableError: mlm_spol".
>>> freetds.conf:
>>> [zfp]
>>> host = 10.230.128.140
>>> port = 1433
>>> tds version = 8.0
>>> asa database = ZFP_CRM
>>> client charset = utf8
>>> text size = 50000000
>>> odbc.ini:
>>> [ODBC Data Sources]
>>> zfp = test
>>> [zfp]
>>> Driver = /usr/lib/libtdsodbc.so
>>> Description = test
>>> Trace = No
>>> Servername = zfp
I made some progress. I have client charset in my freetds config file. I am also
certain that the config and the charset is used by pyodbc / freetds combo.
Without it I get 'Ne?...'. With it I get back a str encoded in utf-8. I also
enabled freetds logging where I clearly see two conversions being prepared:
iconv.c:351:preparing iconv for "UTF-8" <-> "UCS-2LE" conversion
iconv.c:391:preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion
If I comment the client charset line in the config, I see:
iconv.c:391:preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion
iconv.c:391:preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion
I beleive the second one has to do with description_encoding which is 'latin-1'
by default (in MSDialect_pyodbc.__init__). However I tried to set it to utf-8,
utf-16le and whatnot without any effect (in debugger).
I also found out that my problem is not in information_schema per se but with
all nvarchar values / columns. For example this
though I have no idea how and where it comes into existence. Furthermore the
correct value is 'osoba' (a table name). I guess the conversion stripped out the
odd character.
Current state of nvarchar processing in my dev env:
tsql OK
isql OK
pyodbc KO
> On Oct 9, 2012, at 6:03 AM, Ladislav Lenart wrote:
>> Hello.
>> No, I was not able to connect via pymssql. Furthemore I would like to use pyodbc
>> if it is the preferred way. I just did not know that when I started with pymssql.
>> It really is a misconfigured character encoding issue in pyodbc / freetds on my
>> part. I am just clueless as to what should I set to what to make it all work.
>> The code:
>> cnxn =
>> pyodbc.connect('DRIVER={FreeTDS};SERVER=10.230.128.140;PORT=1433;DATABASE=Z FP_CRM;UID=efractal;PWD=efR@cZFP13;TDS_VERSION=8.0')
>> cursor = cnxn.cursor()
>> cursor.execute("select prijmeni from osoba where id_osoba = 462493")
>> row = cursor.fetchone()
>> print row
>> prints:
>> ('Ne?asov', )
> you definitely, definitely need "client charset" to be part of your FreeTDS config, either in that URL string up there (not sure if that's supported), or preferably in your freetds.conf. pyodbc is pretty sensitive to this.
>> However, when I connect via tsql, it works correcly:
>> tsql -S zfp -U efractal
>> Password:
>> locale is "cs_CZ.UTF-8"
>> locale charset is "UTF-8"
>> using default charset "utf8"
>> 1> select prijmeni from osoba where id_osoba = 462493
>> 2> go
>> prijmeni
>> Nečasová
>> (1 row affected)
>> 1>
>> I guess I am on my own now. Anyway, thank you for your kind assistance.
>> Ladislav Lenart
>> On 8.10.2012 19:06, Michael Bayer wrote:
>>> if you didnt have this problem with pymssql then please apply the patch I sent previously.
>>> However, I'm going to bet the problem remains as it seems something is not right with how your database and/or client is configured. I'm not familiar with the encoding pattern seen in your information_schema queries. you may have to reproduce the issue using plain SELECT statements against information_schema and then seek outside help.
>>> diff -r 17cab4ad55d5 lib/sqlalchemy/dialects/mssql/pymssql.py
>>> --- a/lib/sqlalchemy/dialects/mssql/pymssql.py Thu Oct 04 18:26:55 2012 -0400
>>> +++ b/lib/sqlalchemy/dialects/mssql/pymssql.py Fri Oct 05 18:46:01 2012 -0400
>>> @@ -80,7 +80,7 @@
>>> def _get_server_version_info(self, connection):
>>> vers = connection.scalar("select @@version")
>>> m = re.match(
>>> - r"Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> + r"\s*Microsoft SQL Server.*? - (\d+).(\d+).(\d+).(\d+)", vers)
>>> if m:
>>> return tuple(int(x) for x in m.group(1, 2, 3, 4))
>>> else:
>>> On Oct 8, 2012, at 12:10 PM, Ladislav Lenart wrote:
>>>> Hello.
>>>> I adjusted the ODBC/FreeTDS condifugration according to your suggestions but
>>>> still get the "NoSuchTableError: mlm_spol".
>>>> freetds.conf:
>>>> [zfp]
>>>> host = 10.230.128.140
>>>> port = 1433
>>>> tds version = 8.0
>>>> asa database = ZFP_CRM
>>>> client charset = utf8
>>>> text size = 50000000
>>>> odbc.ini:
>>>> [ODBC Data Sources]
>>>> zfp = test
>>>> [zfp]
>>>> Driver = /usr/lib/libtdsodbc.so
>>>> Description = test
>>>> Trace = No
>>>> Servername = zfp
I finally managed to solve the problem (with the great help of my colleagues)!
The culprit: python-pyodbc package in Debian/testing is nearly THREE years old
now (version 2.1.7). I removed it and easy-installed the version 3.0.6 which
does not have the encoding bug anymore. Now everything works like a charm,
inluding SqlSoup. I even receive unicode objects directly for both VARCHAR and
NVARCHAR columns!
For future reference: If you want to connect to MSSQL server from a Linux
(Debian) machine, you need to...
[1] Install unixodbc:
aptitude install unixodbc
I also have unixodbc-dev installed, but I am not sure if it is necessary.
[2] Install freetds and the driver:
aptitude install freetds-bin tdsodbc
[3] Edit /etc/odbcinst.ini (note: DO NOT indent lines in the odbc config files!):
[zfp]
Driver = FreeTDS # The section name from /etd/odbcinst.ini
Description = test
Servername = zfp # The section name from /etc/freetds/freetds.conf (see below)
# Do NOT use option 'Server'.
TDS_Version = 8.0 # Important
[5] Edit /etc/freetds/freetds.conf:
[zfp]
host = 10.230.128.140
port = 1433
asa database = ZFP_CRM # Database name
tds version = 8.0 # Important
client charset = UTF-8 # Important
text size = 50000000
[6] Install python2.7-dev which is needed for the next step, because pyodbc
compiles some stuff and thus needs header files:
aptitude install python2.7-dev
[7] Install pyodbc "manually" (i.e. without aptitude):
easyinstall pyodbc
[8] Verify that freetds works (zfp is the section name in
/etc/freetds/freetds.conf):
tsql -S zfp -U username
Try to select some strings to see that they display correctly.
[9] Verify that unixodbc works (zfp here is the section name in /etc/odbc.ini):
isql zfp username password
Again, try to select some strings to see that they display correctly.
[10] Verify that pyodbc works:
import pyodbc
cnxn = pyodbc.connect("DSN=zfp;UID=username;PWD=password", unicode_results=True)
cursor = cnxn.cursor()
# Select some VARCHAR column.
(x,) = cursor.execute("select...").fetchone()
# Should print a meaningful unicode representation of the result string.
print repr(x)
# Repeat the same for NVARCHAR column. The result should be the same, i.e.
# proper unicode representation).
(x,) = cursor.execute("select...").fetchone()
print repr(x)
[11] Connect via SQLAlchemy:
from sqlalchemy.engine import create_engine
from sqlalchemy.ext.sqlsoup import SqlSoup
# zfp here is the section name in /etc/odbc.ini
engine = create_engine(
"mssql+pyodbc://username:password@zfp",
convert_unicode=True,
echo='debug'
)
db = SqlSoup(engine)
# Same as in step 10
db.execute(...)
# OR
x = db.some_table.filter(...).one()
The ODBC Driver 1.0 for Linux from Microsoft (http://www.microsoft.com/en-us/download/details.aspx?id=28160) works very well for me. It allows me to use the same driver (SQL Server Native Client 11.0) on both Linux-based development machine and Windows-based production server.
With this added to odbcinst.ini:
[SQL Server Native Client 11.0] Description=Microsoft SQL Server ODBC Driver V1.0 for Linux Driver=/opt/microsoft/sqlncli/lib64/libsqlncli-11.0.so.1790.0 Threading=1 UsageCount=1
> The ODBC Driver 1.0 for Linux from Microsoft (http://www.microsoft.com/en-us/download/details.aspx?id=28160) works very well for me. It allows me to use the same driver (SQL Server Native Client 11.0) on both Linux-based development machine and Windows-based production server.
> With this added to odbcinst.ini:
> [SQL Server Native Client 11.0]
> Description=Microsoft SQL Server ODBC Driver V1.0 for Linux
> Driver=/opt/microsoft/sqlncli/lib64/libsqlncli-11.0.so.1790.0
> Threading=1
> UsageCount=1
OK this is really fascinating, as my sysadmin scared me away from MS's product, but if you're saying its working, then that is really amazing. Of course the proof would be that I can run unicode through it in any way I see fit and it wouldn't choke. Will try to look into this, as never using FreeTDS again would be a good thing.