[Django] #28035: Help Django Programmer to link a Query in their database logs with the Python line of codes that generated the request.

7 views
Skip to first unread message

Django

unread,
Apr 6, 2017, 5:57:59 AM4/6/17
to django-...@googlegroups.com
#28035: Help Django Programmer to link a Query in their database logs with the
Python line of codes that generated the request.
-----------------------------------------+------------------------
Reporter: Rémy Hubscher | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
The QuerySet creation and the QuerySet execution doesn't happens at the
same time in Django.

However when you want to link a query in your database logs with your
Python code, you want to see where the QuerySet have been created as well
as were it was executed.

Each time you call a new method on your queryset it will create a new
queryset with the new filter or order_by or annotate calls.

The plan would be to keep track of all these calls that generated a new
QuerySet and to add it as a database comment when executing the request.

Then you would be able to link easily where your request have been
generated from in your python code.

SELECT * FROM users WHERE id = 1; -- users/views.py L.21 `user =
Users.objects.get(pk=request.user.id)`

It is especially useful when you create conditionals QuerySet:

SELECT * FROM users WHERE name LIKE 'enac%' ORDER BY username; --
users/views.py L.18 `qs = Users.objects.all() \n users/views.py L.25 `qs =
qs.filter(name__startswith="enac")` \n users/views.py L.30 `qs =
qs.order_by('username')` \n templates/user_search.html L.55 `{% for user
in users %}`

If we want to do that as a library outside of Django, it means we would
have to monkey patch the QuerySet object.
Do we want to make it a Django Feature?

We could add the comment only when `DEBUG = True` or `DATABASE_QUERY_LOGS
= True`

--
Ticket URL: <https://code.djangoproject.com/ticket/28035>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Apr 6, 2017, 5:58:35 AM4/6/17
to django-...@googlegroups.com
#28035: Help Django Programmer to link a Query in their database logs with the
Python lines of code that generated the request.
-------------------------------+--------------------------------------

Reporter: Rémy Hubscher | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------

--
Ticket URL: <https://code.djangoproject.com/ticket/28035#comment:1>

Django

unread,
Apr 6, 2017, 2:03:55 PM4/6/17
to django-...@googlegroups.com
#28035: Help Django Programmer to link a Query in their database logs with the
Python lines of code that generated the request.
-------------------------------------+-------------------------------------

Reporter: Rémy Hubscher | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

* type: Uncategorized => New feature
* component: Uncategorized => Database layer (models, ORM)


Old description:

> The QuerySet creation and the QuerySet execution doesn't happens at the
> same time in Django.
>
> However when you want to link a query in your database logs with your
> Python code, you want to see where the QuerySet have been created as well
> as were it was executed.
>
> Each time you call a new method on your queryset it will create a new
> queryset with the new filter or order_by or annotate calls.
>
> The plan would be to keep track of all these calls that generated a new
> QuerySet and to add it as a database comment when executing the request.
>
> Then you would be able to link easily where your request have been
> generated from in your python code.
>
> SELECT * FROM users WHERE id = 1; -- users/views.py L.21 `user =
> Users.objects.get(pk=request.user.id)`
>
> It is especially useful when you create conditionals QuerySet:
>
> SELECT * FROM users WHERE name LIKE 'enac%' ORDER BY username; --
> users/views.py L.18 `qs = Users.objects.all() \n users/views.py L.25 `qs
> = qs.filter(name__startswith="enac")` \n users/views.py L.30 `qs =
> qs.order_by('username')` \n templates/user_search.html L.55 `{% for user
> in users %}`
>
> If we want to do that as a library outside of Django, it means we would
> have to monkey patch the QuerySet object.
> Do we want to make it a Django Feature?
>
> We could add the comment only when `DEBUG = True` or `DATABASE_QUERY_LOGS
> = True`

New description:

--

Comment:

I'm not sure if this is feasible to implement. How would it work?

--
Ticket URL: <https://code.djangoproject.com/ticket/28035#comment:2>

Django

unread,
Apr 9, 2017, 2:06:11 PM4/9/17
to django-...@googlegroups.com
#28035: Help Django Programmer to link a Query in their database logs with the
Python lines of code that generated the request.
-------------------------------------+-------------------------------------
Reporter: Rémy Hubscher | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Rémy Hubscher):

The idea I had in mind was to add an new property to the queryset if the
DEBUG setting is set to True that contains a traceback to the current
QuerySet creation.
The new queryset is created with the traceback list of the previous
queryset in which we append the new traceback from the current creation.

It is then possible to look at the queryset property when we actually
build the request from it to have a full traceback to each call that was
used to create the queryset.

--
Ticket URL: <https://code.djangoproject.com/ticket/28035#comment:3>

Django

unread,
Apr 12, 2017, 12:39:29 PM4/12/17
to django-...@googlegroups.com
#28035: Allowing logging the queryset location for a given query
-------------------------------------+-------------------------------------

Reporter: Rémy Hubscher | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

* stage: Unreviewed => Accepted


Comment:

Tentatively accepting. I guess we'll have to see the implementation to see
if it looks okay and works well enough.

--
Ticket URL: <https://code.djangoproject.com/ticket/28035#comment:4>

Reply all
Reply to author
Forward
0 new messages