[Django] #21603: Inconsistent column names in cursor.description under SQLite break RawQuerySet
Django
RawQuerySet
----------------------------------------------+--------------------
Reporter: alex@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 1
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+--------------------
See pull request at https://github.com/django/django/pull/2069
SQLite will sometimes return column names in cursor.description in the
form `table_alias."column_name"` or simply `"column_name"` instead of just
`column_name`. This can happen when returning columns from subqueries,
views or unions.
This means that `RawQuerySet` can't match up the columns when trying to
create model instances, and raises `InvalidQuery: Raw query must include
the primary key`.
This change introduces a `column_name_converter()` to the backend
introspection code to mirror `table_name_converter()` and implements the
fix on the SQLite backend.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
`test_subquery_in_raw_sql` doesn't fail on my system. Is this behavior
specific to some version of SQLite?
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:1>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by alexh):
This bug report suggests it might be platform/version specific:
http://bugs.python.org/issue19167
I'm on OSX 10.9, Python 2.7.5, and if I run sqlite3 from a command line it
reports version 3.7.13.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:2>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by claudep):
I'm on Linux, that may explain why I cannot reproduce the issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:3>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by vajrasky):
Ladies and gentlemen, this has nothing to do with OS. Mine, sqlite 3.7 on
Fedora Core 18 is afflicted.
This is a bug from Sqlite. Sqlite 3.7 is afflicted. Solution: upgrade to
sqlite 3.8.
http://sqlite.1065341.n5.nabble.com/sqlite3-column-name-contains-quotes-
for-views-td65226.html
http://www.sqlite.org/src/info/5526e0aa3c
I guess it is not our place to fix this bug. So, I am inclined to make
this ticket as invalid. But I let OP or core developers to do that.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:4>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by claudep):
Thanks for the research! And yes, I have the 3.7.17 version of the debian
mozilla team which is posterior to the fixing patch. However plain Debian
Wheezy (currently stable) package seems to be affected. I'm still
undecided about the value of working around this bug in Django.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:5>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by alexh):
Yes, thanks for unearthing that!
I personally would like to see it fixed in Django, because a feature I'm
hoping to get merged (#21604) relies on getting this data from
`cursor.description`. As well as Debian, 3.7 is the system installed
version on the current versions of Ubuntu and OS X - I think it's
widespread enough to warrant including the workaround.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:6>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: master
Component: Database layer | Resolution:
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 1
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by aaugustin):
* needs_better_patch: 0 => 1
* type: Bug => Cleanup/optimization
* stage: Unreviewed => Accepted
Comment:
Given that the patch isn't too invasive, yes, we could include it... It
needs a comment stating which versions of SQLite are affected, so we can
eventually remove the workaround when we consider them sufficiently
outdated.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:7>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: master
Component: Database layer | Resolution:
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 1
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by alexh):
OK, I've added the version affected and a link to the bugfix commit in the
comments. Also added a TODO to remove the change at some point in the
future. Let me know if it needs anything else.
Alex
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:8>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: master
Component: Database layer | Resolution:
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 1
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by timo):
I left comments for improvement on the PR. Please uncheck "Patch needs
improvement" when you update it, thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:9>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: master
Component: Database layer | Resolution:
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
Comment:
Have updated the PR to incorporate your feedback, should be good to go.
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:10>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: | Status: closed
Cleanup/optimization | Version: master
Component: Database layer | Resolution: fixed
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"938da36cb12c07c93a44bb1b77984427e809d213"]:
{{{
#!CommitTicketReference repository=""
revision="938da36cb12c07c93a44bb1b77984427e809d213"
Fixed #21603 -- Fixed complex RawQuerySets queries on some versions of
SQLite.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:11>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: master
Component: Database layer | Resolution:
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* cc: shai, timgraham (added)
* status: closed => new
* has_patch: 1 => 0
* resolution: fixed =>
Comment:
The query added in this test doesn't work on Oracle (`DatabaseError:
ORA-00920: invalid relational operator`). Should we try to make it work
there or just skip it?
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:12>
Django
RawQuerySet
Reporter: alex@… | Owner: nobody
Type: | Status: closed
Cleanup/optimization | Version: master
Component: Database layer | Resolution: fixed
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 0
Has patch: 0 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"2a9f44dfbe293a1217097acc34993f48c0139e99"]:
{{{
#!CommitTicketReference repository=""
revision="2a9f44dfbe293a1217097acc34993f48c0139e99"
Fixed #21603 (again) -- Made a raw query in a test Oracle-compliant
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21603#comment:13>