[Django] #25265: DB Backend cannot specify query class.
Django
----------------------------------------------+--------------------
Reporter: techdragon | Owner: nobody
Type: New feature | Status: new
Component: Database layer (models, ORM) | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+--------------------
It would be beneficial if the Queryset class
(django.db.models.query.QuerySet) could use some mechanism to find out
which query class the database backend wants to use.
Without some mechanism to specify the query class, non sql based database
backends are made drastically more complex.
Adding something like "get_query" on the database backends that returns
'sql.Query' in the existing backends would enable new backends to provide
their own query class.
--
Ticket URL: <https://code.djangoproject.com/ticket/25265>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: techdragon | Owner: nobody
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by akaariai):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
The problem is that QuerySets aren't tied to any given backend. What
should happen if you have two backends, one for mongodb and other for
postgres, and the user does User.objects.filter(id=10).using('postgres')?
Maybe this could be an error?
I'm not sure what we could do to help nonsql backends. Some sort of
User.objects.all() returns nonsql QuerySet seems like a possible addition,
but I'm not sure what that mechanism should be.
--
Ticket URL: <https://code.djangoproject.com/ticket/25265#comment:1>
Django
Reporter: techdragon | Owner: nobody
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by jarshwah):
* stage: Unreviewed => Accepted
Comment:
I'm not sure what the solution to this problem is, but after discussing it
at pycon with techdragon I agree that it'd be a nice thing to fix.
Here's a couple of things off the top of my head:
1) Meta option per model: Meta: query_backend =
'django.db.models.sql.query'
2) A DATABASES option
3) A new queryset method
`Model.objects.with('django.db.models.sql.query')`
But I also think that we need to define the API between queryset and
query, so that actually writing different query backends is realistic. Not
a requirement for a solution here though.
--
Ticket URL: <https://code.djangoproject.com/ticket/25265#comment:2>
Django
Reporter: techdragon | Owner: nobody
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by techdragon):
RE: akaariai
--
The situation between 'comparable' database backends should actually be
more like what will happen if you did the same query
{{{User.objects.filter(id=10).using('postgres')}}} with a MySQL and
PostgreSQL backend. I can't say I've done that myself, but if it raises an
error, it should raise an error between a sql and a nonsql backend.
'Non comparable' backends should probably raise an error, something like
{{{NotImplemented}}} perhaps.
--
RE: jarshwah
--
1 - This offers little improvement over a custom model that uses the Meta
API, and overrides the save method.
2 - This is more like what I'm thinking. While its definitely possible to
enhance the flexibility by providing a SETTINGS powered override, I'm not
sure that defining the 'default' query for any particular backend in the
SETTINGS dictionary is the right way to go.
3 - Just like suggestion 1, this doesn't improve the 'compatibility'
situation significantly, beyond what can be done with existing mechanisms
such as custom models using the Meta API and overriding their save
methods.
--
Ideally with a database backend that supports all the correct methods, we
should be able to run Django entirely on top of it, so that means either
extensive modifications to contrib.user and other contrib modules, to
support some kind of 'optional non sql path', or we need to 'fix' the
'issues' that currently exist with this and related section of code that
result in the public API to the ORM mandating that any Django database
backend behave like an SQL database.
I'm fairly sure the optimal path is shifting some of this back into the
database backend 'layer'. Possibly creating a new "base_sql" class that
the existing SQL database backends inherit from instead of
{{{db.backends.base}}} and things like defining 'what is my query class'
can be set here and shared by all the SQL backends. Otherwise we just
implement the new behaviour in each backend.
--
Ticket URL: <https://code.djangoproject.com/ticket/25265#comment:3>
Django
Reporter: techdragon | Owner: nobody
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by akaariai):
Its also worth noting that there are three ways to implement nonsql
backends:
1. Implement Compiler
2. Implement Query (and possibly Compiler)
3. Implement QuerySet (and possibly Query and Compiler)
Currently implementing a different Compiler for nonsql backend is the
suggested approach. In principle it should work. In practice, I don't
know. I guess the biggest problem is that when doing this, the author is
targeting an often changing internal API.
I'm not sure implementing Query is that much better. The API for Query
isn't public. How about implementing QuerySet itself? Now that iterator is
separated from QuerySet, it isn't that much more work when compared to
just implementing Query and/or Compiler.
For .using() incompatibility - I think we should throw an error. And the
default backend should be used to decide what is the default queryset
type. So, if you have default db = mongodb, other = postgres, then
User.objects.filter(foo=bar).using('other') should throw an error, but
User.objects.using('other').filter(foo=bar) shouldn't.
It is also worth noting that maybe the right answer is to use different
*models* altogether - Django's default user, permission and groups system
for example isn't a good design for non-relational databases. If you
control the models, then you also control the default queryset type, and
we don't have a problem to begin with.
I guess I am +½ to manager .get_queryset() checking the query/queryset
class from the default backend, with the idea that objects.using('other')
will get you the other backend's query/queryset class.
Finally, I think whatever we come up with here, we should present the
solution on django-developers and ask for opinions.
--
Ticket URL: <https://code.djangoproject.com/ticket/25265#comment:4>
Django
Reporter: Samuel Bishop | Owner: nobody
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* stage: Accepted => Someday/Maybe
Comment:
I don't think there is a way to move it forward without PoC and a
discussion on DevelopersMailingList.
--
Ticket URL: <https://code.djangoproject.com/ticket/25265#comment:5>
Django
Reporter: Samuel Bishop | Owner: nobody
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by GitHub <noreply@…>):
In [changeset:"14c8504a37afad96ab93cf82f47b13bcc4d00621" 14c8504a]:
{{{
#!CommitTicketReference repository=""
revision="14c8504a37afad96ab93cf82f47b13bcc4d00621"
Refs #25265 -- Allowed Query subclasses to build filters.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25265#comment:6>
Django
Reporter: Samuel Bishop | Owner: nobody
Type: New feature | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"547656c85027eda85a24edcab907022ce313f772" 547656c8]:
{{{
#!CommitTicketReference repository=""
revision="547656c85027eda85a24edcab907022ce313f772"
Refs #25265 -- Allowed customizing Query's datastructure classes.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25265#comment:7>