should blank POST data fallback to use model field defaults?

[Tim Graham]

A change in Django 1.10 inadvertently caused the following behavior change:

class M(models.Model):
    f = models.CharField(max_length=255, default='default_value')

class MF(forms.ModelForm):
    f = forms.CharField(required=False)
    
    class Meta:
        model = M
        fields = ['f']

>>> mf = MF({})
>>> m = mf.save()
>>> m.f
'' (Django 1.10+)
'default_value' (Django < 1.10)

On Django < 1.10, {'f': ''} as the form data (rather than empty form data) also results in a model instance with the model field default rather than an empty string. Claude and I agree that for this latter case, transforming an empty POST value to a model field default doesn't seem correct (that is, the new behavior seems better).

However, the desired behavior in the empty POST data case isn't so clear. For example, I could imagine an API call where it would be useful for the model to use the default if a user doesn't submit any data for that field. An issue with the idea that values not present in POST data fallback to their default is for HTML widgets like checkboxes. If a checkbox isn't checked, the key won't appear in POST data. In such a case, it's inappropriate for an unchecked checkbox to fallback to the model field's default if it's True. A draft patch to restore the Django < 1.10 behavior [1] special cases this with isinstance(form[f.name].field.widget, CheckboxInput) to fix some test failures but that hardly seems ideal.

Do you feel we should try to restore the Django < 1.10 behavior or keep the new behavior and document that model field defaults are used to populate initial blank forms but not to fill missing data from the form input?

Ticket: https://code.djangoproject.com/ticket/27039
[1] https://github.com/django/django/pull/7068

[Tom Christie]

For the case of fields that haven't been included in the form POST, I'd be surprised if they didn't end up with the model default.

Personally I'd treat that as a regression, and revert to the older behavior.

Agree that the checkbox special-casing isn't completely ideal, but I don't see any way around that.

  - Tom

[Yo-Yo Ma]

@Tom

> I'd be surprised if they didn't end up with the model default. Personally I'd treat that as a regression, and revert to the older behavior.

I think Tim is saying the opposite, that Django 1.10 accidentally introduced "fields missing from POST are set to model defaults" behavior.

@Tim

> I could imagine an API call where it would be useful for the model to use the default if a user doesn't submit any data for that field

I think that the way to allow a key to be left out of POST entirely is either:

  1. Create a dict with defaults, then update it to the values in POST, using the result to instantiate the form
  2. Remove the field from the form (this could even be dynamic with modelform_factory based on keys missing from POST)

In a web page, # 1 is basically what's happening, but the updating is taking place in the UI.

For other cases (e.g., an API), a form method to get the same data used to populate the HTML form values would be very helpful:

>>> class Egg(Model):
>>>    temperature = Charfield(max_length=255, default='over medium')
>>> EggForm = modelform_factory(Egg, exclude=())
>>> form = EggForm()
>>> form.initial  # This is only going to output the value given as `initial`
{}
>>> form.proposed_new_property  # <--- This lil guy right here
{'temperature': 'over medium'}

That would of course be helpful in creating an initial dict for my # 1 case above.

>>> data = form.proposed_new_property
>>> data.update(request.POST)
>>> form = EggForm(data)
>>> form.is_valid()
>>> form.save()

I realize the form is now being instantiated twice, but it's necessary because of kwargs like `initial`, and it's no different than the current load-form-in-GET --> validate-form-in-POST workflow.

Thoughts?

[Tim Graham]

No, Tom's understanding is correct. Django 1.10 removed "fields missing from POST are set to model defaults."

[Florian Apolloner]

On Friday, August 12, 2016 at 4:41:21 PM UTC+2, Tom Christie wrote:

For the case of fields that haven't been included in the form POST, I'd be surprised if they didn't end up with the model default.

Personally I'd treat that as a regression, and revert to the older behavior.

that, nothing more to add.

[Tobias McNulty]

On Fri, Aug 12, 2016 at 9:58 AM, Tim Graham <timog...@gmail.com> wrote:

A change in Django 1.10 inadvertently caused the following behavior change:

class M(models.Model):
    f = models.CharField(max_length=255, default='default_value')

class MF(forms.ModelForm):
    f = forms.CharField(required=False)
    
    class Meta:
        model = M
        fields = ['f']

>>> mf = MF({})
>>> m = mf.save()
>>> m.f
'' (Django 1.10+)
'default_value' (Django < 1.10)

On Django < 1.10, {'f': ''} as the form data (rather than empty form data) also results in a model instance with the model field default rather than an empty string. Claude and I agree that for this latter case, transforming an empty POST value to a model field default doesn't seem correct (that is, the new behavior seems better).

That certainly sounds incorrect to me and I would not want to see pre-1.10 behavior restored for the blank ('') value case.

 

However, the desired behavior in the empty POST data case isn't so clear. For example, I could imagine an API call where it would be useful for the model to use the default if a user doesn't submit any data for that field. An issue with the idea that values not present in POST data fallback to their default is for HTML widgets like checkboxes. If a checkbox isn't checked, the key won't appear in POST data. In such a case, it's inappropriate for an unchecked checkbox to fallback to the model field's default if it's True. A draft patch to restore the Django < 1.10 behavior [1] special cases this with isinstance(form[f.name].field.widget, CheckboxInput) to fix some test failures but that hardly seems ideal.

Do you feel we should try to restore the Django < 1.10 behavior or keep the new behavior and document that model field defaults are used to populate initial blank forms but not to fill missing data from the form input?

I tend to prefer the more cautious approach, i.e., allow the application to supply default values to the form if they're not included in the POST, but it sounds like I might be in the minority.

Tobias

--

Tobias McNulty
Chief Executive Officer

tob...@caktusgroup.com
www.caktusgroup.com

[Tim Graham]

While falling back to model defaults for empty data seems reasonable to me, documenting that it works "except for BooleanFields where you have to change the form field or widget to get this behavior" seems sad. If we go that route, how would you suggest to document handling that case (what custom form field or widget should be used)?

[Tom Christie]

It's not perfect but given that it's an inherent limitation of HTML checkboxes, I don't think that documenting that limitation is that big a deal.

forms.BooleanField(required=False) feels like an odd combination to me in any case. One way we could potentially help users would be to error in that case unless an alternate widget is used (We could recommend select or whatever it is we use for NullBooleanField in this case)

We have the same issue in REST framework serializers FWIW. As with Django<1.10 we treat omission as omission, not blank. And yes, we have to special-case boolean omission.

[Michael Foster]

I believe Tom is right. The documentation is trivial but there is a need to raise an error or warning to the user when specifying required=False for boolean fields, as it is nonsensical.

[Tim Graham]

I think there's a misunderstanding. As the documentation says, forms.BooleanField(required=False) is the common case because required=True means the checkbox must be checked. It's only confusing in the context of a non-browser request where empty POST data might have a different meaning.

https://docs.djangoproject.com/en/stable/ref/forms/fields/#booleanfield

[Tim Graham]

I completed the pull request to restore the fallback for empty data, except for CheckboxInput. If you have any input, please speak up. https://github.com/django/django/pull/7068