Upload path broken since 2.9.11 update

[Louis Amon]

Hi,

My website has been using web2py for a few months now, and we handle pictures using Amazon S3 and Cloudfront for storage and distribution.

The corresponding code in web2py looks like this :

Field('picture_1',

      type='upload',

      requires=IS_EMPTY_OR(IS_IMAGE(extensions=('jpeg', 'png'))),

      uploadfolder='offer/picture_1',

      uploadfs=settings.amazon_fs,

      autodelete=True,

      represent=lambda value, row: cloud_download(value) if value is not None else None,

      )

When a picture was uploaded through a form, it was uploaded to our Amazon bucket at a path similar to this : {bucket_name}/offer/picture_1/offer.picture_1.{timestamp_hash}.{filename_hash}

I recently updated to 2.9.11-stable+timestamp.2014.09.15.23.35.11 but now it seems that the uploadfolder does not get prefixed to the path.

i.e. : my uploaded pictures now go to : {bucket_name}/offer.picture_1.{timestamp_hash}.{filename_hash}

Was there something in the 2.9.11 patch that modified the behaviour of uploads through a filesystem ?

Is there a quick fix for mend these broken paths ?

[Anthony]

Which version of web2py were you using prior to the upgrade?

[Louis Amon]

Hi Anthony,

I was using 2.9.5 before the upgrade.

In the end, I found that it may be that the part of the code than handles uploads through a filesystem never did work with the « uploadfolder » attribute (i.e. it might have been a patch I made a few months ago and lost in the process of the upgrade).

In any case, here is the solution to my issue if anyone ever meets it :

In dal.py, line 10211:

                 os.makedirs(path)

             pathfilename = pjoin(path, newfilename)

             if self.uploadfs:

-                dest_file = self.uploadfs.open(newfilename, 'wb')

+                dest_file = self.uploadfs.open(pathfilename, 'wb')

             else:

                 dest_file = open(pathfilename, 'wb')

             try:

In dal.py, line 10249:

             stream = StringIO.StringIO(data)

         elif self.uploadfs:

             # ## if file is on pyfilesystem

-            stream = self.uploadfs.open(name, 'rb')

+            fullname = pjoin(file_properties['path'], name)

+            stream = self.uploadfs.open(fullname, 'rb')

         else:

             # ## if file is on regular filesystem

             # this is intentially a sting with filename and not a stream

With this, the DAL should be able to both write and retrieve files through a filesystem whilst still taking in consideration the prefixed folder (i.e. the « uploadfolder » attribute).

That’s very handy if you need to specify multiple upload locations in your bucket.

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/f4LoEF9FiiM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Leonel Câmara]

Why don't you make a pull request so you don't have to remember this in your next update? 

[Louis Amon]

I really don’t have any time to setup my environment for proper testing & git contributions.

I still consider myself to be a novice and I’m the only dev in my startup right now…

But I do intend to contribute to the framework in a few weeks’ time, when my startup will have raised funds :)

Le 13 nov. 2014 à 15:43, Leonel Câmara <leonel...@gmail.com> a écrit :

Why don't you make a pull request so you don't have to remember this in your next update?