manual file upload using SQLFORM.factory

[Alex]

Hi,

I've already spent quite some time with the following problem which I
think should be fairly easy. I hope someone can help me.

# model
db.define_table('admin_setting',
Field('name', 'string', notnull=True),
Field('value', 'string', notnull=True))

in the controller I'm creating a form for various admin settings.
form = SQLFORM.factory(
Field('invoice_logo', 'upload'), ...)

the view works well and displays all fields.

When uploading a file for the logo the file should be handled like
always (file uploaded to uploads folder, renamed to uuid filename). in
the table admin_setting I want to store the filename of the uploaded
file in a row where name='invoice_logo' (the filename should be stored
in the value field).

How can I achieve this? currently I have this code (the update is
performed later and not shown here):
if form.accepts(request.vars, formname='admin_setting_form',
dbio=False):
if request.vars.invoice_logo != None:
if type(request.vars.invoice_logo) != str:
request.vars.invoice_logo_filename =
request.vars.invoice_logo.filename
field = Field('invoice_logo', 'upload')
# field.store fails because field does not have a _tablename
uploaded_file = field.store(request.vars.invoice_logo.file,
request.vars.invoice_logo.filename)
else:
del request.vars.invoice_logo # do not delete existing logo

[TheSweetlink]

Hello Alex,

Two things I've found when manually uploading via SQLFORM.factory:

1) You need to specify a table_name='...' to avoid the
no_table_newfilename.extension issue like this:

form = SQLFORM.factory(...Field definitions...,
table_name='some_table_name')

2) Additionally you must specify an uploadfolder in your upload Field
definition similar to this:

form = SQLFORM.factory(...,
Field('invoice_logo', type='upload',
uploadfolder=os.path.join(request.folder,'static/uploads/')),
..., table_name='whatever_you_like')

**NOTE** 'static/uploads' is just an example, you can upload to
wherever it will be appropriate.

In this case the newly uploaded and renamed file to
your_application's_dir/static/uploads/your_new_filename_here

One gotcha to look out for following your field name as an example
without the quotation marks:

In your form.accepts(...):

"request.vars.invoice_logo" will contain the original filename of your
upload whereas

"form.vars.invoice_logo_newfilename" will contain the newly renamed
file like yourtablename.9203842903.thaoeu09gu023hgda3p.ext

No need to call store() directly as SQLFORM.factory will take care of
that for you.

I hope that this helps you.

-David Bloom

[Massimo Di Pierro]

Suggestions to make the behaviour better?

[Alex]

Hi David,

works great, thanks!

I'm now manually uploading with these few lines:
field = Field('invoice_logo', 'upload',
uploadfolder=os.path.join(request.folder,'uploads/'))
field._tablename = 'admin_setting'
uploaded_invoice_logo_filename =
field.store(request.vars.invoice_logo.file,
request.vars.invoice_logo.filename)

@Massimo: now that I know, it's quite easy so I don't know much to
make it better. It would be good if this would be mentioned in the
documentation (there is already a short 'Manual Uploads' section),
although this is probably not a common use case. And maybe it would be
a good idea to have default values for uploadfolder and tablename?

On 6 Okt., 03:46, Massimo Di Pierro <massimo.dipie...@gmail.com>
wrote:

[TheSweetlink]

My pleasure Alex. I am glad that worked out for you. Now that I look
at your code you got it to work in a way I didn't even think of. May
we see a more complete example of your SQLFORM.factory?

Massimo,
I don't think I would know how to make it better other than Alex's
suggestion of adding some more info to the Book.

Especially when Alex came up with another way to do the manual upload
I think it's cool that web2py is that flexible.

I know it must be difficult to keep it all up to date with all the new
features cropping up left and right. Nice work by the way to all that
did the most recent updates.

Viva web2py and its magnificent community,
David

[Alex]

it turned out that I could not do it as shown above because of
form.accepts. This already performs the upload so now I'm doing it
more like you suggested which is even more straightforward.

form = SQLFORM.factory(
Field('invoice_logo', 'upload', label=T('as.invoice_logo'),
uploadfolder=os.path.join(request.folder,'uploads/')),
Field('invoice_logo_filename',
label=T('as.invoice_logo_filename')),
... many other fields ...
table_name='admin_setting')

if form.accepts(request.vars, formname='admin_setting_form',
dbio=False):
if request.vars.invoice_logo != None:
if type(request.vars.invoice_logo) != str:
request.vars.invoice_logo_filename =
request.vars.invoice_logo.filename

invoice_logo_newfilename =
form.vars['invoice_logo_newfilename']
else:
del request.vars.invoice_logo # do not clear current logo
filename

for setting in request.vars:
if not setting.startswith('_'):
old_file = None
if setting == 'invoice_logo':
old_file = db(db.admin_setting.name ==
'invoice_logo').select(db.admin_setting.value).first().value
value = invoice_logo_newfilename
else:
value = request.vars[setting]
if old_file:
try:
os.remove(os.path.join(request.folder,
'uploads/', old_file))
except OSError:
pass
db(db.admin_setting.name ==
setting).update(value=value)



one question about the download:
def download():
import os
filename = request.args(0)
original_filename = request.vars.filename
filename = os.path.join(request.folder, 'uploads/', filename)
ret = open(filename, 'rb')
response.headers['Content-Type'] = 'application/octet-stream'
response.headers['Content-Disposition'] = 'inline; filename="%s"'
% original_filename
return ret.read()

do I have to take care about directory traversals and other security
risks? What's the easiest way to do this?

[TheSweetlink]

I'm happy the upload works for you Alex.

> do I have to take care about directory traversals and other security
> risks? What's the easiest way to do this?

As I understand it the store() renaming of the file is what takes care
of the dir traversal protection.

I'm not a web2py dev so don't quote me on that.

Perhaps you lot can confirm?

David

[Alex]

Upload should be save since its handled by web2py. But with the
download the user possible could pass any path for the filename and
download files also from other folders. Should I check for '..' in the
filename? Would it be sufficient?

btw, the community is great here. as is web2py :)

Alex

[TheSweetlink]

Yes a user can by default can download() but how would the user know
the renamed filename though? I cannot say as I do not have much
detail behind your app. Depending on where you're saving what will
dictate what you should do better than any advice I can give. web2py
enables a great deal of security enhancements by default so generally
speaking you should be just fine with store() renaming your file.

Yes, I too have found web2py to be an invaluable tool as well as this
community being one of the most helpful and nicest around.

Best,
David

[Alex]

the files in the uploads folder should be save since they are all
renamed. But what happens if the user passes e.g. '../models/db.py' as
parameter? (the slashes would have to be encoded though, is this
possible?) Then he would get access to the data model which would not
be good at all. I'm now testing for '..' in the filename, I hope
that's sufficient and there is no way to circumvent this.

[TheSweetlink]

I don't think that's how it works. I get 404 error when trying to
manually download a model. I'm not sure how but I believe web2py
handles this internally and prevents simple attacks like you're
thinking of. It may be worth asking the experts though.

[Massimo Di Pierro]

The url is validated. .. is not allowed in the URL.

[TheSweetlink]

Full stack strikes again.

On Oct 12, 10:46 am, Massimo Di Pierro <massimo.dipie...@gmail.com>
wrote:

> The url is validated. .. is not allowed in the URL.
>

I commend you Massimo and happy to see you respond so level headed to
such an emotionally driven thread (elsewhere btw)...again.

[Alex]

right, I completely forgot that the url is validated by web2py before
anything else is called.

[Tiana A. Ralijaona]

Hi everybody,

I manually uploaded file via SQLFORM.factory too and the file has been uploaded using the excerpt below :

forma = SQLFORM.factory(
                            Field("image_1", "upload", uploadfield=True, uploadfolder=os.path.join(request.folder,'uploads/'), ...), ..., table_name="imagebiens")
...
if forma.validate():
        if forma.vars.image_1:
            db.imagebiens.insert(**dict(image=forma.vars.image_1, bienid=bien.id))
...
It works. I can find the file renamed in the uploads folder.
But now, I want to render the file in a view like this :

<p>
    {{for sary in limagebien:}}
    <img src="{{=URL("default", "download", args=sary.image)}}" width="96" height="96">
    {{pass}}
</p>


which is rendered in html like this :

<p>
	
	<img src="/maloca/default/download/imagebiens.image_1.a8e26134c9014d3a.6372756e636862616e672e706e67.png" width="96" height="96">
	
	<img src="/maloca/default/download/imagebiens.image_1.b24410c2dae27017.6372756e636862616e672d666c616d65732d776964652d62792d6f6d6e732e6a7067.jpg" width="96" height="96">
	
</p>

The problem is that the image doesn't appear. The download action seems not to be capable to render it. Has anyone an idea where in this code the problem comes from?

Thanks in advance

[Anthony]

The problem is that in your factory form, you named the field "image_1", which then gets included in the filename, but in your actual db.imagebiens table, the field name is "image". When the download function receives the filename, it assumes the "imagebiens.image_1" prefix refers to table and field name. It then checks that Field object in order to figure out its associated uploadfolder. Of course, because there is no db.imagebiens.image_1 field, this lookup fails.

Anthony

[Tiana A. Ralijaona]

Actually, I've created this form factory to upload 1 to 4 images at the same time and that's why I named the field "image_1", "image_2", ... After form passes validation, I manually insert the file in db.imagebiens table. The new file name being stored in image field.
So how can I achieve this functionality and render the image file with download function ? Should I go for multiple form to upload each file or give the view the direct link to the image ?

Tiana

Le mardi 20 mai 2014 16:53:15 UTC+3, Anthony a écrit :

The problem is that in your factory form, you named the field "image_1", which then gets included in the filename, but in your actual db.imagebiens table, the field name is "image". When the download function receives the filename, it assumes the "imagebiens.image_1" prefix refers to table and field name. It then checks that Field object in order to figure out its associated uploadfolder. Of course, because there is no db.imagebiens.image_1 field, this lookup fails.

Anthony

On Tuesday, May 20, 2014 7:22:58 AM UTC-4, Tiana A. Ralijaona wrote: