How to upload a file into web2py?

[Joe Barnhart]

I have a problem. I want to upload a file into my web2py site which
does NOT go into a database, but rather will be processed (it is a CSV
file).

The proposed solution is to use form_factory to create the form as
though a database exists (using SQLField objects). When I create the
form with only an "upload" field it seems to make the form correctly,
but submitting it does not trigger the form.accepts() block in the
controller. I can see that
"request.var.my_field_name" has a value, but it is a FieldStorage
object and I'm not sure what to do with it. (FieldStorage is in the
cgi module, not a web2py object.)

Has anyone done this? I know this is described in the web2py manual,
but the section on this topic has errors and is not sufficiently
developed for me to answer this on my own.

[Yarko Tymciurak]

On Thu, Mar 12, 2009 at 6:48 PM, Joe Barnhart <joe.ba...@gmail.com> wrote:

I have a problem.  I want to upload a file into my web2py site which
does NOT go into a database, but rather will be processed (it is a CSV
file).

If you are going to use the sql.SQLTable.import_from_csv_file, then see how appadmin.py does it;

If you want to save it for some later time, then upload as a blob field - it will go into your apps uploads directory.

Hope that helps

- Yarko

[Joe Barnhart]

Actually, I need to process the CSV file before it can be inserted
into the database. I was hoping to upload it to disk (or even memory
as the files are small), do the processing, and commit the changes to
db.


On Mar 12, 5:07 pm, Yarko Tymciurak <yark...@gmail.com> wrote:

[Wes James]

On Thu, Mar 12, 2009 at 6:20 PM, Joe Barnhart <joe.ba...@gmail.com> wrote:
>
> Actually, I need to process the CSV file before it can be inserted
> into the database.  I was hoping to upload it to disk (or even memory
> as the files are small), do the processing, and commit the changes to
> db.
>

How about uploading it to the static files folder in web2py admin then
process if from there? Not sure what you are trying to do though??
upload via browser or copy over via sftp? then process??
-wj

[Joe Barnhart]

Let me preface this by explaining that I am a noob at creating
websites. This is probably so simple that anyone who's been around
the block knows the answer and is thus confused at the empty-
headedness of my question.

I wanted to take advantage of web2py's built-in handling of update
fields -- it pops up a file chooser and allows the user to pick the
the file to upload on his local computer. The file is then streamed
up to the server running web2py and usually placed in the "uploads"
directory and linked into a database table. In my case, rather than
storing the file or a reference to it in a database, I wish to process
the file immediately in the controller and store the processed
results.

I can create a form using form_factory with a single SQLField
definition for a field of type "upload". When I invoke this form it
correctly displays a file chooser and lets me pick a file. After that
-- nothing. The form.accepts() part of the controller is never
triggered. I can look at the contents of the
request.vars.my_field_name and see that it now contains an object of
FieldStorage, but I'm not sure what it is good for.

Thank you for your patience and responses.

On Mar 12, 5:23 pm, Wes James <compte...@gmail.com> wrote:

[Yarko Tymciurak]

could you share your code?

[mdipierro]

you can get the file content from

request.vars.my_field_name.file.read()

and the file name from

request.vars.my_field_name.filename

hope this helps.

[Wes James]

On Thu, Mar 12, 2009 at 6:55 PM, Joe Barnhart <joe.ba...@gmail.com> wrote:
>
> Let me preface this by explaining that I am a noob at creating
> websites.  This is probably so simple that anyone who's been around
> the block knows the answer and is thus confused at the empty-
> headedness of my question.
>

If you look in the archives for this list for the t3 foto galleria I
uploaded you will find an example of processing an uploaded file after
it has been uploaded. In this case it is a function that uses PIL
(python imaging library) to make thumbnails of uploaded images. Look
in the Manage/Procedures menu and look at the code there. If you have
not used t3, ask google "web2py t3" and you will find some movies on
vimeo that Massimo has posted on t3 or a link to t3 wiki with some
links to the videos. One other fix for my app is to change the css
for the folder for the t3 foto galleria. I renamed the app t3fg and
for got to make the changes in the css for for this change. If you
rename t3fg to t3, it will work fine.

hth,

-wj

[Joe Barnhart]

I must be doing something very wrong. The field I get back does not
seem to have any of the components it should have.

Here is my controlller:

def import_csv():
from gluon.sqlhtml import form_factory
form=form_factory(SQLField('import_csv','upload'))
if form.accepts(request.vars,session):
request.flash='Received: %s'%request.vars.import_csv
#do more processing here
return dict(form=form)

And here is the view:

{{extend 'layout.html'}}
<h1>This is the meets/import_csv.html template</h1>
{{=form}}
{{=BEAUTIFY(request.vars.import_csv.filename if
request.vars.import_csv else '')}}

The if clause above was to see if the field contained anything under
"import_csv". It did not.

I was trying to follow the information in the manual on page 177 but
there seemed to be missing information (like the input field itself).

I am currently using version 1.56.3. I plan to update as soon as the
current swim meet is over and I can take the site down for awhile as I
work on it.

[mdipierro]

I tried this:

def import_csv():
from gluon.sqlhtml import form_factory
form=form_factory(SQLField('import_csv','upload'))
if form.accepts(request.vars,session):
request.flash='Received: %s'%request.vars.import_csv

print request.vars.import_csv.filename
print len(request.vars.import_csv.file.read()),'bytes'
return dict(form=form)


and it works great for me. Are you using python 2.6 by any chance?

[Joe Barnhart]

Thanks Massimo --

I don't know what my problem was other than the view was confusing
me.

Your code works perfectly on my end as well (Python 2.5.2 on Mac
here). The view still reports request.var.import_csv as a
"FieldStorage" object, but the form.accepts() processing gets
triggered properly and the file is read.

I'm a happy camper and this pattern is going into my website! I'll
have a link to it here soon so everyone can take a look. It's not
beautiful to anyone but me so far, but it makes pretty extensive use
of web2py's DAL power. The users of the site (high school swim
coaches) are ecstatic to have it.

Regards,

-- Joe B.

[mdipierro]

Glad to hear it.

[fastcard lastname]

I have same issue, but I changed controller .py to following, it works

def uploading_file():

    from gluon.sqlhtml import form_factory

    form=form_factory(SQLField('import_csv','upload', uploadfolder = 'c:\\temp\\uploading'))

    if form.accepts(request.vars,session):

        request.flash='Received: %s'%request.vars.import_csv

        print (request.vars.import_csv.filename)

        print (len(request.vars.import_csv.file.read()),'bytes')

    #return dict(form=form)

    return locals()

view file here...

{{extend 'layout.html'}}

<h1>Option Exercise CMC file uploading</h1>

<p>{{=form}}</p>

{{=BEAUTIFY(request.vars.import_csv.filename if request.vars.import_csv else '')}}