default values on database level

[Podrigal, Aron]

I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some  constants like fields.CURRENT_TIMESTAMP, etc.  The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.

Any thoughts?

[Loïc Bistuer]

Hi Aron,

+1 on db defaults as well, I've actually started experimenting on this last week. It's a bit tricky because migrations do use db defaults in some operations (and drop them once they are done) so we have to ensure that the new feature doesn't interfere with them.

However I don't think we should create them by default, instead I propose the introduction of a db_default kwarg, which should be either a constant, or an expression (e.g. db_default=contrib.postgres.functions.TransactionNow).

Cheers,
Loïc

> On Jul 29, 2015, at 12:04, Podrigal, Aron <ar...@guaranteedplus.com> wrote:
>
> I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some constants like fields.CURRENT_TIMESTAMP, etc. The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.
>
> Any thoughts?
>
>

> --
> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANJp-yi5%2Bu%3DW8PdUXtcot%2BO8%3Df35dVLbStwVArJhU7gDnaNXoA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Carl Meyer]

On 07/28/2015 11:55 PM, Loïc Bistuer wrote:
> +1 on db defaults as well, I've actually started experimenting on
> this last week. It's a bit tricky because migrations do use db
> defaults in some operations (and drop them once they are done) so we
> have to ensure that the new feature doesn't interfere with them.
>
> However I don't think we should create them by default, instead I
> propose the introduction of a db_default kwarg, which should be
> either a constant, or an expression (e.g.
> db_default=contrib.postgres.functions.TransactionNow).

+1 to this approach. We should not try to conflate Python-level defaults
and db-level defaults into a single kwarg (that way lies madness), nor
implicitly introduce db-level defaults for existing models where Django
never used them historically.

A separate `db_default` kwarg is clear, explicit, and matches precedent
from SqlAlchemy.

Carl

[Podrigal, Aron]

Hi Loic,

Agree with having a db_default kwarg.

I am not using multiple databases so no experiance with db routers. So how would should we handle routing between different databases when the db_default value is not compatible with that backend? for example Model.save(using='mysql_db') should we than use the Field.default to generate the default value (which is not how it works today, that the default is computed at instance creation). I would prefer in such a case to rely on the database to handle the situation by omitting that field from the columns list, so mysql for example would set a non nullable varchar column to an empty string, where with other databases it would cause an IntegrityError exception. and that db_default and default kwargs cannot be used togethor.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/125BCDAB-728B-4BAA-A8AB-38BA0842B709%40gmail.com.

[Marc Tamlyn]

Debatably this situation should throw an error at model validation time - assuming the expression does error out when trying to build that database. Otherwise it will error at migrate time.

I would like this - it would be very good for UUID as PK

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANJp-yjsEthdkNQUaUKW3DfgyeigtjyZMYQoGteX%2BQ4MM4uPAw%40mail.gmail.com.

[Michael Manfre]

On Wed, Jul 29, 2015 at 11:42 AM, Podrigal, Aron <ar...@guaranteedplus.com> wrote:

Hi Loic,

Agree with having a db_default kwarg.

I am not using multiple databases so no experiance with db routers. So how would should we handle routing between different databases when the db_default value is not compatible with that backend? for example Model.save(using='mysql_db') should we than use the Field.default to generate the default value (which is not how it works today, that the default is computed at instance creation). I would prefer in such a case to rely on the database to handle the situation by omitting that field from the columns list, so mysql for example would set a non nullable varchar column to an empty string, where with other databases it would cause an IntegrityError exception. and that db_default and default kwargs cannot be used togethor.

If db_default is defined for a field, then it should always be used. If the db_default is invalid for the configured backend, then it will and should break. We shouldn't automatically change db_default to a regular default or do any other magic.

[Andrew Godwin]

My main problem with "defaults in the database" was that the current "default" kwarg is too expressive for things we can put into a database and doesn't allow representation of "NOW()", etc. I'd be happy with a default_db arg with defined semantics and that would just blow up if you tried to pass a function.

Andrew

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAGdCwBsHa%2BGXJSB6fay%3DbM%2B-3qV0y5oSuDNd4m05QM1PPdfKrw%40mail.gmail.com.

[Podrigal, Aron]

OK, that makes implementation a lot easier. As the model validation can be done with Model.check() based on the backend given at that time either when running the server or makmigrations.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAGdCwBsHa%2BGXJSB6fay%3DbM%2B-3qV0y5oSuDNd4m05QM1PPdfKrw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Aron Podrigal

-

'1000001', '1110010', '1101111', '1101110'   '1010000', '1101111', '1100100', '1110010', '1101001', '1100111', '1100001', '1101100'

P: '2b', '31', '33', '34', '37', '34', '35', '38', '36', '30', '39', '39'

[Podrigal, Aron]

What about db_default allows either a constent eg. db_default=contrib.postgres.functions. or a string "MyCustomPostgresFunction()" and optionaly allowing arguments to be passed to the db function optionally also allowing F() expressions by giving a tuple db_default=('MyPostgresqlFunction()', F('column1'), 'NOW()').

[Loïc Bistuer]

You could already achieve that by making MyPostgresqlFunction a subclass of models.Func, I don't think it's worth supporting alternate syntaxes.

Cheers,
Loïc

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANJp-yjOW72mXpKHiq-MCWkSHxSC%3DnBA_42VFuBVJyKFHMXxpg%40mail.gmail.com.

[Podrigal, Aron]

indeed that's it.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/6708A498-F62B-4651-B52D-B40FD1E47DED%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Anssi Kääriäinen]

On Wednesday, July 29, 2015 at 8:05:10 AM UTC+3, Aron Podrigal wrote:

I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some  constants like fields.CURRENT_TIMESTAMP, etc.  The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.

Any thoughts?

A couple of questions about this:

  - What should happen when you instantiate a new model with DB default, and show that model to the user in a form. Is the field shown to the user, and if so, what value should it have?

  - How to implement actual fetching of the values after save() - PostgreSQL supports RETURNING, but this is not the case for all databases.

  - When is the default applied? On first save of the model, or on model init time?

  - Do we want to allow extending this to defaults that are applied on every save (automatic database backed modified timestamps for example)?

 - Anssi

[Podrigal, Aron]

On Aug 3, 2015 3:36 AM, "Anssi Kääriäinen" <akaa...@gmail.com> wrote:
>
> On Wednesday, July 29, 2015 at 8:05:10 AM UTC+3, Aron Podrigal wrote:
>>
>> I would like to propose making Fields.default to rely on the database generated defaults where possible. This could be implemented by having some  constants like fields.CURRENT_TIMESTAMP, etc.  The tricky part here is to detect database backend specific capabilities and have a python equivalent fallback.
>>
>> Any thoughts?
>
>
> A couple of questions about this:
>   - What should happen when you instantiate a new model with DB default, and show that model to the user in a form. Is the field shown to the user, and if so, what value should it have?

The field would have no value, so when displayed on the form it would be blank. A field with blank=False which is included in a form, having a db_default would be useless.

>   - How to implement actual fetching of the values after save() - PostgreSQL supports RETURNING, but this is not the case for all databases.

The value would be fetched on demand on its first access, saving a trip to the database if the field is not accessed. Subsequent updates on that instance would leave out that field from the list of fields to be updated, unless its value has been changed on the instance. Maybe we can allow to handle this based on some parameter provided to save() ?

>   - When is the default applied? On first save of the model, or on model init time?

Of course on first save, as we want the default value atomically within the same transaction.

>   - Do we want to allow extending this to defaults that are applied on every save (automatic database backed modified timestamps for example)?

+1 for this one too.

>
>  - Anssi

>
> --
> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f1a7cc68-de1a-49f8-bdf8-ebfaade59955%40googlegroups.com.

[Michael Manfre]

On Mon, Aug 3, 2015 at 9:25 AM, Podrigal, Aron <ar...@guaranteedplus.com> wrote:

>   - Do we want to allow extending this to defaults that are applied on every save (automatic database backed modified timestamps for example)?

+1 for this one too.

This behavior would be a nice step toward computed (readonly) database fields.

--

GPG Fingerprint: 74DE D158 BAD0 EDF8

keybase.io/manfre

[Owais Lone]

Hi all, I've a working PR that implements this but in a much inferior way https://github.com/django/django/pull/5904

I'm very interested in this and would love to contribute. Has anyone started work on this? If not, I'd like to pick it up.

--
Owais

[Podrigal, Aron]

Hi Owais,

I did not have the time to start any work on this. I'm very much interested in this and I'd be happy to contribute to this in any way. I'm following along on the other thread you started [1]. 

[1] https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/django-developers/BDAlTyJwQeY/tTRKkE_fBwAJ

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9fd48e92-d35f-4fea-9f27-95b91db895af%40googlegroups.com.

[Shai Berger]

Hi everybody,

Waking this up again because, going over some older stuff, I found a ticket
asking for this feature:

https://code.djangoproject.com/ticket/470

It was closed wontfix, but if anybody likes to put a 3-digit-numbered Django
bug under their belt, it's there. The discussion in this thread would indicate
that it should be re-opened.

Shai.

[Paul Tiplady]

Closing the loop here -- Tim just reopened https://code.djangoproject.com/ticket/470.

This feature would also be useful for achieving zero-downtime migrations, as discussed in https://code.djangoproject.com/ticket/29266.

Sounds like the design has mostly been agreed -- would a PR be accepted that just implements the simple `db_default` field without attempting PostgreSQL RETURNING or database functions (CURRENT_TIMESTAMP), and just provides static defaults?

Cheers,

Paul

[Michael Grijalva]

Yeah seems like `db_default` was the felling, at least last time I scanned through this long-going discussion :D 

I've had to tweak Django's migration handling a bit to handle our zero-downtime requirements.

SQLalchemy has had this for awhile (http://docs.sqlalchemy.org/en/latest/core/defaults.html#server-side-defaults), so I see no reason why it can't be done in Django.

[Marcin Nowak]

W dniu czwartek, 29 marca 2018 19:28:39 UTC+2 użytkownik Michael Grijalva napisał:

Yeah seems like `db_default` was the felling, at least last time I scanned through this long-going discussion :D 

I've had to tweak Django's migration handling a bit to handle our zero-downtime requirements.

This is not the only one issue with Django migrations.