[Django] #26338: Getting Queryset item by index with PostgreSQL behaves randomly
Django
----------------------------------------------+------------------------
Reporter: pawelad | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.9
Severity: Normal | Keywords: postgresql
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+------------------------
I originally 'reported' it on [http://stackoverflow.com/q/35831698/3023841
StackOverflow], but this is probably where it should be.
This code on SQLite works as intended and gets two random items from
queryset:
{{{
random.seed()
index1, index2 = random.sample(range(0, qs.count()), 2)
all_objects = qs.all()
object1 = all_objects[index1]
object2 = all_objects[index2]
}}}
but when run on PostgreSQL, getting items from Queryset by index seems to
behave randomly.
Here are {{{index1, index2}}} and {{{object1, object2}}} pairs when that
code is run 5 times:
{{{
9 12
ID: 2754 ID: 2365
15 11
ID: 1626147 ID: 200811
12 1
ID: 2365 ID: 203112
1 12
ID: 203112 ID: 2365
1 3
ID: 203112 ID: 203112
}}}
The last one is especially important, because what it means is
{{{qs.all()[1] == qs.all()[3]}}} which just can't be (all ID's are
unique). And this occurred quite regularly.
Converting queryset to list fixed the issue ({{{all_objects =
list(qs.all())}}}), so I am guessing it has something to do with Queryset
laziness in PostgreSQL implementation. I tried to look it up in the source
code but with no luck.
--
Ticket URL: <https://code.djangoproject.com/ticket/26338>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: pawelad | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: postgresql | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
Does the queryset have an `order_by()`? If a query doesn’t have an
ordering specified, results are returned from the database in an
unspecified order.
--
Ticket URL: <https://code.djangoproject.com/ticket/26338#comment:1>
Django
Reporter: pawelad | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by pawelad):
Yes, both the model and queryset has {{{order_by}}} set.
But even if it's in unspecified order, could the order change between
accessing it's items? Because from my understanding, event if it's random,
{{{qs.all()[1] == qs.all()[3]}}} should never be true.
--
Ticket URL: <https://code.djangoproject.com/ticket/26338#comment:2>
Django
Reporter: pawelad | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
I believe `all_objects[index1]` and `all_objects[index2]` are running two
separate queries. When you first cast to a list, you only have one query.
Please verify by checking
[https://docs.djangoproject.com/en/stable/faq/models/#how-can-i-see-the-
raw-sql-queries-django-is-running connections.queries].
--
Ticket URL: <https://code.djangoproject.com/ticket/26338#comment:3>
Django
Reporter: pawelad | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 1.9
(models, ORM) |
Keywords: postgresql | Triage Stage:
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: new => closed
* resolution: => needsinfo
Comment:
Please reopen with a test for Django's test suite that demonstrates the
problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/26338#comment:4>
Django
Reporter: pawelad | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution: needsinfo
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by pawelad):
You were right with the {{{order_by}}}, but why setting it on model level
isn't taking care of that?
Shouldn't that be enough?
{{{
class Meta:
ordering = ['-field1, '-field2', 'field3']
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26338#comment:5>
Django
Reporter: pawelad | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution: needsinfo
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
Again, I'd ask if you could please provide more details such as a test for
Django's test suite (ideally) or a sample project to reproduce the issue.
Although I suspect a bug in your code rather than in Django, there aren't
enough details here currently to investigate the issue. Ideally, "is it a
bug?" questions would first be directed to our support channels
(TicketClosingReasons/UseSupportChannels) and reported here only if
someone else confirms the problem as a probable bug. Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/26338#comment:6>
Django
Reporter: pawelad | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 1.9
(models, ORM) |
Severity: Normal | Resolution: needsinfo
| Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by pawelad):
OK, sorry for the inconvenience.
--
Ticket URL: <https://code.djangoproject.com/ticket/26338#comment:7>