ListField fails when used with multipart/formdata

[Andrew Backer]

I have an endpoint that accepts an image and a list of integers.  The serializer looks something like this:

class ImageWithDesiredSize(serializers.Serializer)
    image = serializers.ImageField(required=True)
    sizes = serializers.ListField(child=serializers.IntegerField())

When posting to this from my test cases, the `size` field ends up empty.  The data posted to the view contains the correct json, looking like this:

MergeDict(
    <QueryDict: {
       u'sizes': [u'200', u'100'],
       <MultiValueDict: {u'image': [<InMemoryUploadedFile: abc.png (image/png)>]
    }>
)

But the serializer's validated_data has no values for 'sizes' (it shows up as []).  If I post as JSON (after removing the image field), then it works fine.

I've worked around this with a simple subclass of ListField, but I am not sure if this is the best way to do this or if I'm opening myself up to issues that I don't see yet.  Here is the serializer, I'd appreciate any guidance:

class FormDataListField(serializers.ListField):
    """
    Slight hack around the ListField failing when used inside a multipart/formdata post, for
    example when posting an image along with other data
    """
    def get_value(self, dictionary):
        if html.is_html_input(dictionary):
            # don't do this (from base ListField)? it always returns empty
            # return html.parse_html_list(dictionary, prefix=self.field_name)
            return dictionary.getlist(self.field_name)
        return dictionary.get(self.field_name, empty)

[Greg Kempe]

You may want to check this pull request, I think it might be related: https://github.com/tomchristie/django-rest-framework/pull/2796

[Andrew Backer]

I saw that, actually, but didn't really understand if it applied.  Not embarrassed to say that I don't know exactly what it addresses; i've never deeply considered form parsing for lists (but I know it's not so easy).   That one looked like it applied to returning "empty" instead of a list or dict, rather than the value just not parsing.  

Usually I just apply whatever little workarounds exist, or replicate the autogenerated code from the framework.  For example, I have some interesting custom binding for asp.net mvc and knockout.  I'll go post a comment on that thread, in case this is meant to address that.  

--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Backer]

I've ended up with a serializer field that handles this situation.  Haven't heard any advice on if this is the right way or not, but here it is.

1) `ApiClient` sends the values with the name "sizes"
2) `iOS` sends them as `sizes[]`  

I still can't get the default `ListField` to work right in a `multipart/form-data` scenario.  Hope this can help someone else:

class ListFieldFormData(serializers.ListField):

    def get_value(self, dictionary):
        if html.is_html_input(dictionary):

            # try out both "name" and "name[]"
            field_names = [self.field_name, self.field_name + '[]']
            for name in field_names:
                if name in dictionary:
                    return dictionary.getlist(name)
        return dictionary.get(self.field_name, empty)

[Mark Mikofski]

This gave me an idea for a fix that seems to work just fine for both HTML and JSON. I put this in my serializers.py module and then just use the the CustomListField everywhere instead of serializers.ListField.

from rest_framework.utils import html
from rest_framework.fields import empty
import logging

logger = logging.getLogger(__name__)


class CustomListField(serializers.ListField):
    def get_value(self, dictionary):
        if self.field_name not in dictionary:
            if getattr(self.root, 'partial', False):
                return empty
        # We override the default field access in order to support
        # lists in HTML forms.
        if html.is_html_input(dictionary):

            # BEGIN NEW CODE - MARK MIKOFSKI - 2015/10/14
            logger.debug('*************** entering get_value() ***************')
            logger.debug('dictionary:\n\t%r', dictionary)
            logger.debug('field name: %r', self.field_name)
            logger.debug('is html input: %r', html.is_html_input(dictionary))
            listval = dictionary.getlist(self.field_name)
            logger.debug('field name MultiValueDict value list:\n\t%r', listval)
            if len(listval) == 1 and isinstance(listval[0], basestring):
                logger.debug('val list is a single string:\n\t%r', listval[0])
                listval = listval[0]  # get single string in value list
                if listval.startswith('[') and listval.endswith(']'):
                    # remove any brackets from string
                    listval = listval[1:-1]
                listval = listval.split(',')
                dictionary.setlist(
                    self.field_name,
                    [self.child.to_representation(v) for v in listval]
                )
                listval = dictionary.getlist(self.field_name)
                logger.debug('new dict val:\n\t%r', listval)
                logger.debug('length of new dict val: %r', len(listval))
            logger.debug('*************** leaving get_value() ***************')
            # BEGIN NEW CODE - MARK MIKOFSKI - 2015/10/14

            val = dictionary.getlist(self.field_name, [])
            if len(val) > 0:
                # Support QueryDict lists in HTML input.
                return val
            return html.parse_html_list(dictionary, prefix=self.field_name)
        return dictionary.get(self.field_name, empty)

[Mark Mikofski]

FYI: instead of calling:

dictionary.setlist(
    self.field_name,
    [self.child.to_representation(v) for v in listval]
)

you can use the `ListField.to_representation()` method directly which does exactly the same thing:

dictionary.setlist(self.field_name, self.to_representation(listval))

which is more concise.

Also this might not currently work for nested lists of lists, but if called recursively, I think it would.