working ImageField upload example

[ggi...@gmail.com]

I'm ready to tear my hair out. I've been searching and searching for a working example of uploading to django using rest apis.

I've looked at a lot of examples I've found searching, but none of them seem to work.

Do multipart forms work in the current version? I never get any of the parameters across. JSON seems to work fine, but file uploading doesn't work with JSON.

Any help would be greatly appreciated.

      Greg

[Jeff Tchang]

Here is what I have inside viewsets.py

from rest_framework.parsers import MultiPartParser, FormParser, FileUploadParser

class CustomMultiPartParser(MultiPartParser):

    media_type = 'multipart/*'

class PictureViewSet(viewsets.ModelViewSet):

    queryset = Picture.objects.all().order_by('-created')

    serializer_class = PictureSerializer

    permission_classes = [PicturePermission]

    parser_classes = (CustomMultiPartParser, FormParser, FileUploadParser)

    def create(self, request):

        response = super(PictureViewSet, self).create(request)

        return response

> --
> You received this message because you are subscribed to the Google Groups "Django REST framework" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Andrew Backer]

Greg,


Yes, file uploading doesn't work in JSON.  It is theoretically possible, but don't try :D

Multipart works fine here, and we have many several ViewSets and Views that receive images.  We haven't  even customized anything except by adding a function to de-nest the json values (e.g. user { name } =>  user.name) for the field values when posting between services.

What is the exact issue you are having?  Perhaps the response json or a picture of the crash? 

ggi...@gmail.com

August 24, 2016 at 12:45 PM

[Greg Gilley]

Okay, knowing that its working for someone is helpful. It’s possible it’s a problem with my tests.

I get errors back:

response.data= {'name': ['This field is required.'], 'image': ['No file was submitted.'], 'recipe_as_image': ['No file was submitted.']}

If I print out what the view is getting:

request= <rest_framework.request.Request object at 0x7fde1813e710>

request.full_data <class 'rest_framework.request.Empty'>

request.content_type multipart/form-data; boundary=BoUnDaRyStRiNg; charset=utf-8

request.data= <QueryDict: {}>

request.FILES= <MultiValueDict: {}>

The current state of the code is the result of trying a lot of different things and looking at a lot of different examples.

Help is greatly appreciated!

Greg

class TestRecipeViewUploads(APITestCase):
    def setUp(self):
        self.user = get_user_model().objects.create_user(
            username='jacob', first_name='jacob', email='ja...@foo.com', password='top_secret')
        self.client.login(username='jacob', password='top_secret')
        self.client.force_authenticate(self.user)
        self.banana_bread = Recipe.objects.create(name="Banana Bread", author=self.user)
        self.banana_bread_step1 = Recipe_Step.objects.create(step_number=1, description="step 1", recipe=self.banana_bread, user=self.user)
        self.banana_bread_ingredient1 = Recipe_Ingredient.objects.create(ingredient_number=1,
                                                                          name = "bananas",
                                                                          user=self.user,
                                                                          recipe=self.banana_bread)
        self.banana_bread_comment = Comment.objects.create(recipe=self.banana_bread, user=self.user, body="yum!")
        self.spaghetti = Recipe.objects.create(name="Spaghetti", author=self.user)
        self.media_folder = mkdtemp()

    def tearDown(self):
        rmtree(self.media_folder)

    @mock.patch('storages.backends.s3boto.S3BotoStorage', FileSystemStorage)
    def test_create_view_creates_and_uploads_image(self):
        image = Image.new('RGB', (100, 100))
        tmp_file = tempfile.NamedTemporaryFile(suffix='.jpg')
        image.save(tmp_file, format="JPEG")
        with open(tmp_file.name, 'rb') as fp:
            with override_settings(MEDIA_ROOT=self.media_folder):
                response = self.client.post('/api/recipes/image/',
                                            data={'name': 'Biscuits and Gravy', 'image': fp},
                                            format='multipart')
                print("response=", response)
                print("response.data=", response.data)
        created_recipe = Recipe.objects.get(name="Biscuits and Gravy")
        self.assertTrue(created_recipe.image)

class RecipeImageView(APIView):
    authentication_classes = (BasicAuthentication,)
    permission_classes = (IsAuthenticated,)
    queryset = Recipe.objects.all()
    parser_classes = (MultiPartParser, FormParser,)
    serializer_class = RecipeImageSerializer

    def post(self, request, format=None):
        print("request=", request)
        print("request.full_data", request._full_data)
        print("request.content_type", request.content_type)
        print("request.data=", request.data)
        print("request.FILES=", request.FILES)
        serializer = RecipeImageSerializer(data=request.data)
        print("serializer=", serializer)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

class RecipeImageSerializer(TaggitSerializer, serializers.ModelSerializer):
    image = serializers.ImageField(max_length=None, use_url=True)
    recipe_as_image = serializers.ImageField(max_length=None, use_url=True)

    class Meta:
        model = Recipe
        fields = ('id', 'name', 'slug', 'description', 'image', 'recipe_as_image',)
        read_only_fields = ('id','slug',)

You received this message because you are subscribed to a topic in the Google Groups "Django REST framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-rest-framework/UiKioNuavcE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-rest-fram...@googlegroups.com.

[Andrew Backer]

Try posting something like this for the image:

    SimpleUploadedFile(name="my-file.png", content=open(file_name).read())

* what does use_url do?  never needed to use that, perhaps thats the issue
* try not specifying the format=, if you are using SimpleUploadedFile

Some other notes:

Look at dj-inmemorystorage, to avoid hitting the file system

# check for ./manage.py test, jenkins, or others on the commandline
IS_TESTING = not {'testserver', 'test', 'jenkins'}.isdisjoint(set(sys.argv[1:2]))
# elsewhere in settings
if IS_TESTING:
    DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'

Greg Gilley

August 24, 2016 at 11:30 PM

[Greg Gilley]

On Aug 24, 2016, at 9:01 AM, Andrew Backer <awba...@gmail.com> wrote:

Try posting something like this for the image:

    SimpleUploadedFile(name="my-file.png", content=open(file_name).read())

The same pattern I’m using works fine for regular django forms, but I will try this (I’m open to anything at this point :-) — just tried it, doesn’t work either.

* what does use_url do?  never needed to use that, perhaps thats the issue

Don’t know. Was one of the many working examples I looked at that didn’t work for me. I removed it and still get the same failure.a

* try not specifying the format=, if you are using SimpleUploadedFile

Tried that. (Just tried it again to be certain :-)

Some other notes:

Look at dj-inmemorystorage, to avoid hitting the file system

# check for ./manage.py test, jenkins, or others on the commandline
IS_TESTING = not {'testserver', 'test', 'jenkins'}.isdisjoint(set(sys.argv[1:2]))
# elsewhere in settings
if IS_TESTING:
    DEFAULT_FILE_STORAGE = ‘inmemorystorage.InMemoryStorage'

Cool. I had not seen that I’ll check it out.

Greg

Greg Gilley

August 24, 2016 at 11:30 PM

Okay, knowing that its working for someone is helpful. It’s possible it’s a problem with my tests.

I get errors back:

response.data= {'name': ['This field is required.'], 'image': ['No file was submitted.'], 'recipe_as_image': ['No file was submitted.']}

[Andrew Backer]

Can you reproduce it in a small project, if you aren't willing to share the one you have there?  If you don't necessarily need to have models, but it might be easier.  Post to github/bitbucket or whatnot, or email it over to awba...@gmail.com and I'll see if anything stads out.   Also, try to get it using sqlite ;)  Make life easier for sharing.

~ Andrew

Greg Gilley

August 25, 2016 at 1:46 AM

The same pattern I’m using works fine for regular django forms, but I will try this (I’m open to anything at this point :-) — just tried it, doesn’t work either.

Don’t know. Was one of the many working examples I looked at that didn’t work for me. I removed it and still get the same failure.a

Tried that. (Just tried it again to be certain :-)

Cool. I had not seen that I’ll check it out.

Greg

--

You received this message because you are subscribed to the Google Groups "Django REST framework" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Andrew Backer

August 25, 2016 at 12:01 AM

Try posting something like this for the image:

    SimpleUploadedFile(name="my-file.png", content=open(file_name).read())

* what does use_url do?  never needed to use that, perhaps thats the issue

* try not specifying the format=, if you are using SimpleUploadedFile

Some other notes:

Look at dj-inmemorystorage, to avoid hitting the file system

# check for ./manage.py test, jenkins, or others on the commandline
IS_TESTING = not {'testserver', 'test', 'jenkins'}.isdisjoint(set(sys.argv[1:2]))
# elsewhere in settings
if IS_TESTING:
    DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'

Greg Gilley

August 24, 2016 at 11:30 PM

Okay, knowing that its working for someone is helpful. It’s possible it’s a problem with my tests.

I get errors back:

response.data= {'name': ['This field is required.'], 'image': ['No file was submitted.'], 'recipe_as_image': ['No file was submitted.']}

--

You received this message because you are subscribed to the Google Groups "Django REST framework" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.

[Greg Gilley]

Okay, I created a small project and it works :-(

Now to figure out why it works and my larger project doesn’t…

Greg

You received this message because you are subscribed to a topic in the Google Groups "Django REST framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-rest-framework/UiKioNuavcE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-rest-fram...@googlegroups.com.

[Greg Gilley]

It appears to be a conflict with django cms.

If I have this middleware installed, it fails.

'cms.middleware.toolbar.ToolbarMiddleware',

Any ideas what is going on?

Greg

[Andrew Backer]

Sorry, but not really.  One small thing to try is changing `api/` in yours to `myapi/` just to see if they are doing their on processing based on the path.

A quick look at their code and it seems alright.  You might post an issue to their github and see if they respond.

https://github.com/divio/django-cms/issues

Be sure to include which pip packages you have installed, your INSTALLED_APPS and MIDDLEWARE variables in full.  Good luck!

Greg Gilley

August 25, 2016 at 6:20 AM

It appears to be a conflict with django cms.

If I have this middleware installed, it fails.

'cms.middleware.toolbar.ToolbarMiddleware',

Any ideas what is going on?

Greg

--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Greg Gilley

August 25, 2016 at 4:49 AM

Okay, I created a small project and it works :-(

Now to figure out why it works and my larger project doesn’t…

Greg

--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Andrew Backer

August 25, 2016 at 1:49 AM

[Andrew Backer]

Spoke too soon, this might be a known issue with how django + multiple request-hungry middleware/apps. 

https://github.com/tomchristie/django-rest-framework/issues/3951
https://github.com/divio/django-cms/issues/5446
https://github.com/divio/django-cms/issues/5013

https://github.com/divio/django-cms/issues?utf8=%E2%9C%93&q=is%3Aissue%20rest%20framework

Greg Gilley

August 25, 2016 at 6:20 AM

It appears to be a conflict with django cms.

If I have this middleware installed, it fails.

'cms.middleware.toolbar.ToolbarMiddleware',

Any ideas what is going on?

Greg

--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Greg Gilley

August 25, 2016 at 4:49 AM

Okay, I created a small project and it works :-(

Now to figure out why it works and my larger project doesn’t…

Greg

--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-rest-fram...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Andrew Backer

August 25, 2016 at 1:49 AM

[Greg Gilley]

Switching back to 3.2 worked. That’s not a solution for when Django 1.10 comes out though.

Thanks for your help!

Greg

You received this message because you are subscribed to a topic in the Google Groups "Django REST framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-rest-framework/UiKioNuavcE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-rest-fram...@googlegroups.com.