default values on database level

828 views
Skip to first unread message

Podrigal, Aron

unread,
Jul 29, 2015, 1:05:10 AM7/29/15
to Django developers (Contributions to Django itself)

I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some  constants like fields.CURRENT_TIMESTAMP, etc.  The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.

Any thoughts?

Loïc Bistuer

unread,
Jul 29, 2015, 1:55:48 AM7/29/15
to django-d...@googlegroups.com
Hi Aron,

+1 on db defaults as well, I've actually started experimenting on this last week. It's a bit tricky because migrations do use db defaults in some operations (and drop them once they are done) so we have to ensure that the new feature doesn't interfere with them.

However I don't think we should create them by default, instead I propose the introduction of a db_default kwarg, which should be either a constant, or an expression (e.g. db_default=contrib.postgres.functions.TransactionNow).

Cheers,
Loïc

> On Jul 29, 2015, at 12:04, Podrigal, Aron <ar...@guaranteedplus.com> wrote:
>
> I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some constants like fields.CURRENT_TIMESTAMP, etc. The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.
>
> Any thoughts?
>
>
> --
> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANJp-yi5%2Bu%3DW8PdUXtcot%2BO8%3Df35dVLbStwVArJhU7gDnaNXoA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

Carl Meyer

unread,
Jul 29, 2015, 11:32:16 AM7/29/15
to django-d...@googlegroups.com
On 07/28/2015 11:55 PM, Loïc Bistuer wrote:
> +1 on db defaults as well, I've actually started experimenting on
> this last week. It's a bit tricky because migrations do use db
> defaults in some operations (and drop them once they are done) so we
> have to ensure that the new feature doesn't interfere with them.
>
> However I don't think we should create them by default, instead I
> propose the introduction of a db_default kwarg, which should be
> either a constant, or an expression (e.g.
> db_default=contrib.postgres.functions.TransactionNow).

+1 to this approach. We should not try to conflate Python-level defaults
and db-level defaults into a single kwarg (that way lies madness), nor
implicitly introduce db-level defaults for existing models where Django
never used them historically.

A separate `db_default` kwarg is clear, explicit, and matches precedent
from SqlAlchemy.

Carl

signature.asc

Podrigal, Aron

unread,
Jul 29, 2015, 11:42:10 AM7/29/15
to Django developers (Contributions to Django itself)

Hi Loic,

Agree with having a db_default kwarg.

I am not using multiple databases so no experiance with db routers. So how would should we handle routing between different databases when the db_default value is not compatible with that backend? for example Model.save(using='mysql_db') should we than use the Field.default to generate the default value (which is not how it works today, that the default is computed at instance creation). I would prefer in such a case to rely on the database to handle the situation by omitting that field from the columns list, so mysql for example would set a non nullable varchar column to an empty string, where with other databases it would cause an IntegrityError exception. and that db_default and default kwargs cannot be used togethor.

Marc Tamlyn

unread,
Jul 29, 2015, 11:50:18 AM7/29/15
to django-d...@googlegroups.com
Debatably this situation should throw an error at model validation time - assuming the expression does error out when trying to build that database. Otherwise it will error at migrate time.

I would like this - it would be very good for UUID as PK

Michael Manfre

unread,
Jul 29, 2015, 11:51:30 AM7/29/15
to django-d...@googlegroups.com
On Wed, Jul 29, 2015 at 11:42 AM, Podrigal, Aron <ar...@guaranteedplus.com> wrote:

Hi Loic,

Agree with having a db_default kwarg.

I am not using multiple databases so no experiance with db routers. So how would should we handle routing between different databases when the db_default value is not compatible with that backend? for example Model.save(using='mysql_db') should we than use the Field.default to generate the default value (which is not how it works today, that the default is computed at instance creation). I would prefer in such a case to rely on the database to handle the situation by omitting that field from the columns list, so mysql for example would set a non nullable varchar column to an empty string, where with other databases it would cause an IntegrityError exception. and that db_default and default kwargs cannot be used togethor.


If db_default is defined for a field, then it should always be used. If the db_default is invalid for the configured backend, then it will and should break. We shouldn't automatically change db_default to a regular default or do any other magic.

Andrew Godwin

unread,
Jul 29, 2015, 11:54:41 AM7/29/15
to django-d...@googlegroups.com
My main problem with "defaults in the database" was that the current "default" kwarg is too expressive for things we can put into a database and doesn't allow representation of "NOW()", etc. I'd be happy with a default_db arg with defined semantics and that would just blow up if you tried to pass a function.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

Podrigal, Aron

unread,
Jul 29, 2015, 11:58:41 AM7/29/15
to Django developers (Contributions to Django itself)
OK, that makes implementation a lot easier. As the model validation can be done with Model.check() based on the backend given at that time either when running the server or makmigrations.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

For more options, visit https://groups.google.com/d/optout.



--
Aron Podrigal
-
'1000001', '1110010', '1101111', '1101110'   '1010000', '1101111', '1100100', '1110010', '1101001', '1100111', '1100001', '1101100'

P: '2b', '31', '33', '34', '37', '34', '35', '38', '36', '30', '39', '39'

Podrigal, Aron

unread,
Jul 29, 2015, 12:05:59 PM7/29/15
to Django developers (Contributions to Django itself)
What about db_default allows either a constent eg. db_default=contrib.postgres.functions. or a string "MyCustomPostgresFunction()" and optionaly allowing arguments to be passed to the db function optionally also allowing F() expressions by giving a tuple db_default=('MyPostgresqlFunction()', F('column1'), 'NOW()').

Loïc Bistuer

unread,
Jul 29, 2015, 12:24:17 PM7/29/15
to django-d...@googlegroups.com
You could already achieve that by making MyPostgresqlFunction a subclass of models.Func, I don't think it's worth supporting alternate syntaxes.

Cheers,
Loïc
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANJp-yjOW72mXpKHiq-MCWkSHxSC%3DnBA_42VFuBVJyKFHMXxpg%40mail.gmail.com.

Podrigal, Aron

unread,
Jul 29, 2015, 12:45:33 PM7/29/15
to Django developers (Contributions to Django itself)
indeed that's it.


For more options, visit https://groups.google.com/d/optout.

Anssi Kääriäinen

unread,
Aug 3, 2015, 3:36:25 AM8/3/15
to Django developers (Contributions to Django itself)
On Wednesday, July 29, 2015 at 8:05:10 AM UTC+3, Aron Podrigal wrote:

I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some  constants like fields.CURRENT_TIMESTAMP, etc.  The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.

Any thoughts?


A couple of questions about this:
  - What should happen when you instantiate a new model with DB default, and show that model to the user in a form. Is the field shown to the user, and if so, what value should it have?
  - How to implement actual fetching of the values after save() - PostgreSQL supports RETURNING, but this is not the case for all databases.
  - When is the default applied? On first save of the model, or on model init time?
  - Do we want to allow extending this to defaults that are applied on every save (automatic database backed modified timestamps for example)?

 - Anssi

Podrigal, Aron

unread,
Aug 3, 2015, 9:25:28 AM8/3/15
to Django developers (Contributions to Django itself)


On Aug 3, 2015 3:36 AM, "Anssi Kääriäinen" <akaa...@gmail.com> wrote:
>
> On Wednesday, July 29, 2015 at 8:05:10 AM UTC+3, Aron Podrigal wrote:
>>
>> I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some  constants like fields.CURRENT_TIMESTAMP, etc.  The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.
>>
>> Any thoughts?
>
>
> A couple of questions about this:
>   - What should happen when you instantiate a new model with DB default, and show that model to the user in a form. Is the field shown to the user, and if so, what value should it have?

The field would have no value, so when displayed on the form it would be blank. A field with blank=False which is included in a form, having a db_default would be useless.

>   - How to implement actual fetching of the values after save() - PostgreSQL supports RETURNING, but this is not the case for all databases.

The value would be fetched on demand on its first access, saving a trip to the database if the field is not accessed. Subsequent updates on that instance would leave out that field from the list of fields to be updated, unless its value has been changed on the instance. Maybe we can allow to handle this based on some parameter provided to save() ?

>   - When is the default applied? On first save of the model, or on model init time?

Of course on first save, as we want the default value atomically within the same transaction.

>   - Do we want to allow extending this to defaults that are applied on every save (automatic database backed modified timestamps for example)?

+1 for this one too.

>
>  - Anssi


>
> --
> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f1a7cc68-de1a-49f8-bdf8-ebfaade59955%40googlegroups.com.

Michael Manfre

unread,
Aug 3, 2015, 2:35:33 PM8/3/15
to django-d...@googlegroups.com
On Mon, Aug 3, 2015 at 9:25 AM, Podrigal, Aron <ar...@guaranteedplus.com> wrote:

>   - Do we want to allow extending this to defaults that are applied on every save (automatic database backed modified timestamps for example)?

+1 for this one too.


This behavior would be a nice step toward computed (readonly) database fields.



--
GPG Fingerprint: 74DE D158 BAD0 EDF8

Owais Lone

unread,
Feb 6, 2016, 6:31:12 AM2/6/16
to Django developers (Contributions to Django itself)
Hi all, I've a working PR that implements this but in a much inferior way https://github.com/django/django/pull/5904

I'm very interested in this and would love to contribute. Has anyone started work on this? If not, I'd like to pick it up.

--
Owais

Podrigal, Aron

unread,
Feb 8, 2016, 1:37:36 AM2/8/16
to Django developers (Contributions to Django itself)

Hi Owais,

I did not have the time to start any work on this. I'm very much interested in this and I'd be happy to contribute to this in any way. I'm following along on the other thread you started [1]. 


[1] https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/django-developers/BDAlTyJwQeY/tTRKkE_fBwAJ


--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Shai Berger

unread,
Apr 4, 2016, 5:08:39 AM4/4/16
to django-d...@googlegroups.com
Hi everybody,

Waking this up again because, going over some older stuff, I found a ticket
asking for this feature:

https://code.djangoproject.com/ticket/470

It was closed wontfix, but if anybody likes to put a 3-digit-numbered Django
bug under their belt, it's there. The discussion in this thread would indicate
that it should be re-opened.

Shai.

Paul Tiplady

unread,
Mar 29, 2018, 12:12:24 PM3/29/18
to Django developers (Contributions to Django itself)
Closing the loop here -- Tim just reopened https://code.djangoproject.com/ticket/470.

This feature would also be useful for achieving zero-downtime migrations, as discussed in https://code.djangoproject.com/ticket/29266.

Sounds like the design has mostly been agreed -- would a PR be accepted that just implements the simple `db_default` field without attempting PostgreSQL RETURNING or database functions (CURRENT_TIMESTAMP), and just provides static defaults?

Cheers,
Paul

Michael Grijalva

unread,
Mar 29, 2018, 1:28:39 PM3/29/18
to Django developers (Contributions to Django itself)
Yeah seems like `db_default` was the felling, at least last time I scanned through this long-going discussion :D 

I've had to tweak Django's migration handling a bit to handle our zero-downtime requirements.

SQLalchemy has had this for awhile (http://docs.sqlalchemy.org/en/latest/core/defaults.html#server-side-defaults), so I see no reason why it can't be done in Django.

Marcin Nowak

unread,
May 16, 2018, 8:14:41 PM5/16/18
to Django developers (Contributions to Django itself)


W dniu czwartek, 29 marca 2018 19:28:39 UTC+2 użytkownik Michael Grijalva napisał:
Yeah seems like `db_default` was the felling, at least last time I scanned through this long-going discussion :D 

I've had to tweak Django's migration handling a bit to handle our zero-downtime requirements.

This is not the only one issue with Django migrations.
Reply all
Reply to author
Forward
0 new messages