How can i use ImageField to store uploaded image file from request.FILES

[Young]

I need a good example on how you can store image file directly to mongoengine ImageField.

So here my model has

class Group(Document):

    profile_image = ImageField(size=(250, 250, True))

and in my view...

i have 

profile_image = request.FILES.get('profile_image')

if profile_image:

    group.profile_image.put(profile_image)  # here i get "Invalid Image" error

I need a good example how i can accomplish this...I'm just beginner in file uploads...

if I print 'profile_image', I get

<InMemoryUploadedFile: abc.JPG (image/jpeg)>

Please help!.... couldn't find good sample from document.

Young

[Jan Schrewe]

Do you have PIL [1] (or better Pillow [2]) installed? Your error comes from this code in mongoengine [3]. It only gets raised when the upload can't be processed as image. Since the upload is an image I would suspect that there is an underlying error. Otherwise what you are doing should work as far as using request data without validation is a good idea. There are as far as I know two projects that try to provide a forms API for mongo and Django: django-mongonaut [4] (which I don't know at all) and django-mongodbforms [5] (which I maintain).

HTH

Jan

---

[1] http://www.pythonware.com/products/pil/

[2] http://python-imaging.github.io/

[3] https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/fields.py#L1287-L1291 

[4] https://github.com/pydanny/django-mongonaut

[5] https://github.com/jschrewe/django-mongodbforms

--
You received this message because you are subscribed to the Google Groups "MongoEngine Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongoengine-us...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Young]

Okay. thanks for your help. I've got ImageField field to save.. now i need to read <ImageGridFsProxy: 5240fc4871acde37eb9b4d44> object from my template

I was expecting something like

<img src="%(url)s" /> % {'url', self.profile_image.url}

however, couldn't find url to get from, do I need to use some libraries like PIL? appreciate for any examples. 

Thanks!

Young

[Jan Schrewe]

Webservers don't support gridfs by default. So you need to do something on your own. Most people seem to use some form of Django view at least for development (see below), there is a gridfs module for nginx [1] which I haven't used or tried. I do have some very unfinished stuff to build a full blown gridfs http server (including support for byte ranges, etags, streaming support for very large files etc.) in Erlang if you need something that can be build to scale. For a normal website or blog a Django view should be fine though.

A simple Django view might look like this:

# views.py

import time

from mongoengine.connection import get_db

from gridfs import GridFS, NoFile

from bson.objectid import ObjectId

from django.http import Http404, HttpResponse

from django.utils.http import http_date

def serve_file(request, file_id):

    fs = GridFS(db)

    try:

        f = fs.get(ObjectId(file_id))

    except NoFile:

        fs = GridFS(db, collection='images') # mongoengine stores images in a separate collection by default

        try:

            f = fs.get(ObjectId(file_id))

        except NoFile:

            raise Http404

    response = HttpResponse(f.read(), content_type=f.content_type)

    timestamp = time.mktime(gridout.upload_date.timetuple())

    response["Last-Modified"] = http_date(timestamp)

    # add other header data like etags etc. here

    return response

#urls.py

urlpatterns += patterns('',

    url(r'^gridfs/(?P<file_id>[0-9a-f]{24})/$', 'gridfs.views.serve_file'),

)

Note that this view mostly ignores everything from mongoengine and will load the file completely into memory, so if you need to serve large files this is a bad way to do it. To get a URL in your template you can do something like {% url  'gridfs.views.serve_file' mydoc.gridfile_attr %} where gridfile_attr is something like

@property

def gridfile_attr(self):

    return str(self.the_file.grid_id) # otherwise you will get ObjectId('5227a0c157a020b07eddc4aa') as parameter for url which will never match

If you want to do more with your file before serving it, you can in most cases use the file attribute from your document, that is for most cases file like enough to be used in image libraries for example and if you generate some kind of normal file you can use the normal Django stuff. There is also a storage engine provided with mongoengine. I admit that I mostly try to avoid that though, so I'm not a lot of help there.

The function above uses pymongo directly, see the gridfs docs [2] if you want to use names instead of object ids. Or load the complete document holding the file first and then serve the file from there if you need more checks.

regards,

Jan

[1] https://github.com/mdirolf/nginx-gridfs

[2] http://api.mongodb.org/python/current/api/gridfs/

[Jan Schrewe]

Oh, this

def serve_file(request, file_id):

    fs = GridFS(db)

should be

def serve_file(request, file_id):

    db = get_db()

    fs = GridFS(db)

[Young]

awesome.. that should be a good start for me. BTW, im using gunicorn as server.

You saved my butt! Thanks a lot!!!!! :-)

[Young]

Oh. one thing. what's that gridout object?

On Tuesday, September 24, 2013 10:58:59 AM UTC-7, Jan Schrewe wrote:

[Jan Schrewe]

That's what happens if you copy & paste from real code and are sloppy with renaming. It's f in the function.

[Young]

Thanks. all works, except now I get when doing some updates to the object which has ImageField (gridfs).

'Collection' object is not callable. If you meant to call the '__getnewargs__' method on a 'Collection' object it is failing because no such method exists.

i'm using mongoengine 0.7.5 and made my own patch to add missing __deepcopy__ function in GridFSProxy class.

[Jan Schrewe]

It would help if show the code that triggers the error and a bit of the preceding part.

For what do you a __deepcopy__? Really just curios never had anything complain about it missing.

I personally think that working with gridfs and mongoengine isn't all that pleasant and takes some getting used to. Especially because the proxy objects are not completely transparent and tend to get in the way sometimes. 

[Young]

I just investigated and realized the Celery tasks failed to use those objects.

(Pdb) social_action.delay(object=some_object)

*** TypeError: 'Collection' object is not callable. If you meant to call the '__getnewargs__' method on a 'Collection' object it is failing because no such method exists.

this some_object is the document object where ImageField exists. 

and it works if i don't make that object with ImageField populated with gridfs.

I found similar question to mine, but not sure how to solve.

http://stackoverflow.com/questions/6558143/celery-error-str-object-is-not-callable-getstate-function-is-a-string

[Jan Schrewe]

AFAIK passing around the actual objects to celery tasks is a terrible idea for several reasons (and I think it is strongly discouraged in the celery docs). First if you do that with data from a database there is no guarantee that you don't operate on stale data when your task gets processed. Second you send quite a bit of overhead (as in the complete object which in some cases can become quite large) over the wire to your storage backend and then load it back. Third pickling of objects in Python does not always work. I think for objects that come straight out of a C library it doesn't work (ctypes? C api? I'm never sure what it's called). And PyMongo is just a python wrapper around some C(++) code (in most cases).

How to do it? Pass the object id and maybe a couple more arguments like thumbnail size to the task. Then reload the object and don't worry about the whole pickle mess :) If you don't want it to show up on the website before it's processed add a simple boolean flag to the document and filter unprocessed objects in your view.

[Young Park]

Yeah. I just realized that. Holy cow... i gotta change lots of code. Thanks brother! I really appreciate your constant reply back!

--
You received this message because you are subscribed to a topic in the Google Groups "MongoEngine Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongoengine-users/n_cITLxpihg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongoengine-us...@googlegroups.com.