MS SQL backend as a proper external backend (was: RFC: Django 1.0 roadmap and timeline)
Ramiro Morales
I've been working in my free time for the last few days on updating the
pyodbc-based MS SQL Server backend so it a) can be an external Django
backend and b) to post qs-rf merge.
First I tried to participate by testing django-pyodbc and opening a
[1]ticket about
missing files in the repo in the the project issue tracker but later
found the code
there does not add much to what Filip Wasilewski had already implemented on
#5246 and in fact has more serious problems.
Since then I've opened ticket [2]#7420 with a patch that would reduce the list
of things needed to patch in Django to just *one item: Taking in account the
fact that in pyodbc seems to be the only DB-API2 adapter where
connection.autocommit is an attribute and not a method (didn't find a way to
cleanly monkeypatch this).
Hopefully, this will help clear the way to get this backend closer to
completion, exposure and much needed testing and feedback by users without
forcing them to touch Django core code and without the need to pester Django
devs to give official support and infrastructure.
What are you opinions about this? Do you think a collective effort can
be made?. I'm learning my way in the internals of Django and have a very
basic knowledge of SQL and SQL Server but I could make use of such
a backend plus Django trunk for some work and personal projects.
Regards,
* Perhaps I would also need to open a ticket for this 4-line test/utils.py
[3]modification.
--
Ramiro Morales
1. http://code.google.com/p/django-pyodbc/issues/detail?id=1
2. http://code.djangoproject.com/ticket/7420
3. http://code.djangoproject.com/attachment/ticket/5246/mssql_pyodbc.2.patch
Leo Soto M.
>
> Jack Moffitt, mamcx (and everyone interested),
>
> I've been working in my free time for the last few days on updating the
> pyodbc-based MS SQL Server backend so it a) can be an external Django
> backend and b) to post qs-rf merge.
[...]
> Since then I've opened ticket [2]#7420 with a patch
I see that part of the patch deals with the fact that the underlying
adapter prefer to receive actual date and time instances for date and
time parameters, instead of strings. I have the same problem when
talking to JDBC drivers.
To solve it I proposed[1] another strategy: delegate type conversion
to the backend. Otherwise, I think we will end with too many backend
flags. You can see my first attempt on
https://hg.leosoto.com/django.doj/rev/8b0c71447f98. I've made more
adjustments later[2] and plan to submit a patch once I fix all the
remaining backend related test failures with postgresql_zxjdbc (Which,
according to my schedule should be this week)
This won't help with the interprets_empty_strings_as_nulls flag you
are proposing, tough.
> that would reduce the list
> of things needed to patch in Django to just *one item: Taking in account the
> fact that in pyodbc seems to be the only DB-API2 adapter where
> connection.autocommit is an attribute and not a method (didn't find a way to
> cleanly monkeypatch this).
You are definitely not alone. Same happens with zxJDBC. Same
workaround as yours:
https://hg.leosoto.com/django.doj/rev/235b40d3da2e
[1] http://groups.google.com/group/django-developers/browse_thread/thread/fb1afa93451008a2/0fed055c00cb64e7
[2] https://hg.leosoto.com/django.doj/log?rev=leo.soto+get_db_prep
--
Leo Soto M.
http://blog.leosoto.com
Ian Kelly
> Since then I've opened ticket [2]#7420 with a patch that would reduce the list
> of things needed to patch in Django to just *one item: Taking in account the
> fact that in pyodbc seems to be the only DB-API2 adapter where
> connection.autocommit is an attribute and not a method (didn't find a way to
> cleanly monkeypatch this).
It's also an attribute in cx_Oracle. I'll have to take a look at it
some time and figure out why it's not broken.
Adam V.
I maintain the external django-mssql backend, and I would HUGELY
prefer to get "real Python types" instead of to-string types for
things like dates/times/decimal/etc.
(Converting everything to strings "feels" like a side-effect of MySQL
accepting strings for pretty much everything and doing implicit
conversions?)
Ramiro Morales
On Thu, Jun 12, 2008 at 10:49 PM, Ian Kelly <ian.g...@gmail.com> wrote:
>> connection.autocommit is an attribute and not a method (didn't find a way to
>> cleanly monkeypatch this).
>
> It's also an attribute in cx_Oracle. I'll have to take a look at it
> some time and figure out why it's not broken.
>
/me greps over the cx_Oracle 4.3.3 and 4.4 source trees.
It's me or cx_Oracle doesn't have an autocommit symbol at all?. In
fact, it hasn't
a set_isolation_level one either.
Confused,
--
Ramiro Morales
Ivan Illarionov
> to the backend. Otherwise, I think we will end with too many backend
> flags.
I maintain the external Firebird backend and I would also prefer this
solution.
Ivan Illarionov
Ian Kelly
> /me greps over the cx_Oracle 4.3.3 and 4.4 source trees.
>
> It's me or cx_Oracle doesn't have an autocommit symbol at all?. In
> fact, it hasn't
> a set_isolation_level one either.
It does as of version 4.3.2. I know that it does because I wrote the
patch for it. The attribute is defined in Connection.c and referenced
in the Cursor_InternalExecute function in Cursor.c.
In any case, I looked at this and found that the only time Django
calls the autocommit method is when creating or destroying a test
database. The Oracle backend defines its own custom procedures for
creating and destroying test databases, so those lines never get
executed when using Oracle, and that's why it doesn't cause a problem.
-Ian
mamcx
This could be added to google code? Is more easy run on trunk thta
doing the patching dance...
I test this shortly. I hope we could do this and prove that is doable
to all the *nix db guys ;)
Leo Soto M.
> On Thu, Jun 12, 2008 at 8:35 PM, Ramiro Morales <cra...@gmail.com> wrote:
> [...]
>
> > Since then I've opened ticket [2]#7420 with a patch
>
> I see that part of the patch deals with the fact that the underlying
> adapter prefer to receive actual date and time instances for date and
> time parameters, instead of strings. I have the same problem when
> talking to JDBC drivers.
>
> To solve it I proposed[1] another strategy: delegate type conversion
> to the backend.
datetime and decimal python -> db conversion to the database backend.
It also ensures that for basic field types (IntegerField,
BooleanField, ...) the backend will receive the "expected" basic
python type (int, bool, ...).
http://code.djangoproject.com/ticket/7560
This should help other external-backend mantainers.
--
Leo Soto M.