[Django] #34962: Support for overriding result of model field values
Django
-------------------------------------+-------------------------------------
Reporter: | Owner: nobody
piraka9011 |
Type: New | Status: new
feature |
Component: Database | Version: 4.2
layer (models, ORM) |
Severity: Normal | Keywords: QuerySet.extra
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
We have many models with timestamp fields (ex. `created_at`, `updated_at`,
etc.) that store `DateTimeField`s.
We use Django as a REST API for web and mobile clients written using
JavaScript where time is stored as milliseconds since the UNIX epoch (i.e.
`Date.now()`).
To minimize the amount of processing we do client side, we convert
`DateTimeField`s to milliseconds using a Django Rest Framework (DRF)
serializer that looks like this:
{{{
class TimestampDateTimeField(serializers.DateTimeField):
"""Convert Python datetime to/from Unix timestamp in milliseconds.
For use with JavaScript's Date objects which use ms since epoch, not
seconds.
"""
def to_representation(self, value):
return round(value.timestamp() * 1000)
def to_internal_value(self, value):
try:
result = datetime.fromtimestamp(value / 1000,
timezone.get_default_timezone())
return super(TimestampDateTimeField,
self).to_internal_value(result)
except TypeError:
raise serializers.ValidationError("Datetime must be a number,
not a string")
}}}
Ideally, we could instead do this data manipulation at the database layer
using the `Extract` function **but** with the same behavior as the
serializer where we can override the original name of the field.
Example:
{{{
# This will currently throw an error
# ValueError: The annotation 'updated_at' conflicts with a field on the
model.
m = MyModel.objects.filter(user=user).annotate(
updated_at=Extract("updated_at", "epoch") * 1000
)
}}}
The way we can do this instead is using the `extra()` method from the
`QuerySet` API to generate a query that would look like:
{{{
SELECT (EXTRACT(EPOCH FROM app_mymodel.updated_at) * 1000) AS updated_at
FROM app_mymodel
WHERE app_mymodel.user_id = 42
LIMIT 1;
}}}
I did my best to look online and in the docs for potential
solutions/workarounds, but all point back to using the `extra()`. Is there
a way to do this w/o using `extra()`?
--
Ticket URL: <https://code.djangoproject.com/ticket/34962>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: New feature | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: QuerySet.extra | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => wontfix
Comment:
Thanks for the request but this was an intentional change to prevent data
loss.
You can either setup a new annotation get DRF to refer to that (it's quite
easy by specifying the `source` kwarg for that field in the model
serializer's `extra_kwargs`) or you'll need to manually manipulate the
query underneath the queryset eg: https://github.com/shangxiao/stupid-
django-tricks/tree/master/annotation_overwrite (Standard disclaimer
applies here as that's not an official Django recommendation).
Good luck!
--
Ticket URL: <https://code.djangoproject.com/ticket/34962#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: piraka9011 | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: QuerySet.extra | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Natalia Bidart):
Hello!
(EDIT: I was writing this message when David submitted theirs, so while
David's answer is more accurate, I'll post mine to share the links and doc
pointers).
This report seems like a mix of a new feature request and a support
request. The best place to get answers to your final question ("Is there a
way to do this w/o using `extra()`?") is using any of the user support
channels from [https://docs.djangoproject.com/en/dev/faq/help/#how-
do-i-do-x-why-doesn-t-y-work-where-can-i-go-to-get-help this link]. For
feature requests, the current procedure is to start a new conversation on
the [https://forum.djangoproject.com/c/internals/5 Django Forum] to
present the idea and get community consensus, as per
[https://docs.djangoproject.com/en/stable/internals/contributing/bugs-and-
features/#requesting-features the documented guidelines for requesting
features].
Since the goal of this issue tracker is to track issues about Django
itself, I'll be closing this ticket as invalid following the
[https://docs.djangoproject.com/en/4.2/internals/contributing/triaging-
tickets/#closing-tickets Ticket Triaging policy].
Having said that, what I personally would do is to have different field
names for the `DateTimeField` and its "processed" milliseconds since
epoch, and I would have the latter be a `GeneratedField` (or an
annotation) of the former.
Thank you!
--
Ticket URL: <https://code.djangoproject.com/ticket/34962#comment:2>