How to upload a file without the Forms Object

[Greg_O]

Hello,

How can I upload a file without using the form. I use front end tools like vue and axios to pass and save data via the controller in the back end. There are no form submits and it makes it very fast.

This works very well except for the upload field. For file uploads I do not know how to save without submitting a form.

See my code below. Can someone help me with sample code in the controller to also save a file upload? 

#______________model__________________#

def get_download_url(file):

    return f"download/{file}"

    

db.define_table('documents',

     Field('document_date', type='date', label='Document Date', required=True),

     Field('document_title', type='string', label='Document Title', required=True),

     Field('document_file', 'upload', label='Document', autodelete=True, 

        required=True, uploadfolder=settings.UPLOAD_PATH,   download_url=get_download_url),

     Field('document_file_name', type='string', label='Document File Name', required=True),

     Field('memo', type='text', label='Memo'),

     auth.signature,

)

#________________controller_____________#

#API to Add or Update a document record

@action("add_edit_document/<id:int>", method="POST")

@action.uses(session)

@action.requires(user_in(session))

@action.uses(db)

def add_edit_document(id):

    updated=0

    inserted_id=-1

    status="error"

    code=500

    message="<p>Problem encountered.</p>"

    try:

        table = db.documents

        if id and id>0:

            updated=db(table.id==id).update(**request.json)

            status="success"

            message= "<p>Document record updated successfully.</p>"

            db.commit()

            code=200

            

        else:

            inserted_id = table.insert(**request.json)

            if inserted_id and inserted_id>0:

                status="success"

                message= "<p>Document record created successfully.</p>"

                code=200

                db.commit()

            else:

                message += '<p>Unable to create a new document record</p>'

                db.rollback()

    except Exception as e:

        print(e)

        message +=  str(os.sys.exc_info()[:2])

    return dict(status=status, code=code, message=message, inserted_id=inserted_id, updated=updated)

[Val K]

At the client side use FormData 

In the controller you can access files using request.POST or request.files

пятница, 7 мая 2021 г. в 16:53:52 UTC+3, Greg_O:

[Massimo]

The problem is that if you use vue+axios than you probably submit the form in JSON.

In this case you have to jump some hoops to include the file in json. One way to do it to store its name and base64 encoded content since you cannot have blobs in json. The backend will have to parse and convert that.

No matter what this is not a standard protocol so no sure it should be part of py4web. There is a function in utils.js Q.upload_helper that helps link an <input type="file"> to a callback which is passed the base64 encoded value. I usually just make the callback do a POST of that content and insert the blob into a pre-existing record.

[Val K]

store file in JSON is not a good idea because at backend side,  json is parsed and stored in memory +  there is limitation  bottle.BaseRequest.MEMFILE_MAX = 16 * 1024 * 1024  

axios works with FormData object as well:

//client side

let frmdata = new FormData();

frmdata.append('somefile', file);  // assume file is a file-object 

axios.post('somewhere', frmdata)

#py4web

@action('somewhere') 

def somewhere():

    file = request.POST.get('somefile')

    # file.filename - client side file name

    file.save('path/to/file.xxx', overwrite=True)  # https://github.com/bottlepy/bottle/blob/152219818378926f20a5ee3d339118d7cf50a8f6/bottle.py#L2397-L2414

воскресенье, 9 мая 2021 г. в 18:09:02 UTC+3, Massimo:

[Greg_O]

Thanks very much Val and Massimo. I think I'm almost there.  I took Val's advice and using FormData(), I was able to save the file.

//client side

let frmdata = new FormData();

frmdata.append('somefile', file);  // assume file is a file-object 

frmdata.append('somefile_name', this.file.name)

frmdata.append('document_date', datefield)

frmdata.append('document_title', titlefield)

frmdata.append('memo', memofield)

axios.post('somewhere', frmdata)

#py4web

@action('somewhere') 

def somewhere():

        import os, secrets

        document_dict={

            document_date':request.POST.get('document_date'),

            'document_title':request.POST.get('document_title''),

            'memo':request.POST.get('memo')

        }

        file=request.POST.get('somefile', None)

        if file:

                file_ext=request.POST.get('somefile_name').split('.')[-1] # the the extension of the original file name

                filename= 'documents.filename.' + secrets.token_hex(8) + '.' + secrets.token_hex(30) + '.' + file_ext #build a 'safe' file name - auto generated hex strings

                fileuploadpath=os.path.join(os.getcwd(), settings.APP_FOLDER, 'uploads',  filename)

                file.save(fileuploadpath, overwrite=True) 

                document_dict['document_file']=filename

         db.documents.insert(**document_dict)

From my MS Windows OS, the uploaded file opens and I can read it.  But, when I try to display or download the image on the front end of the app using the standard p4web download method ('/download/filename'), I get the encoding error - "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdb in position 0: invalid continuation byte." 

How can I resolve the error?

ERROR:root:Traceback (most recent call last):

  File "C:\Coding\Python\lib\site-packages\py4web\core.py", line 785, in wrapper

    ret = func(*func_args, **func_kwargs)

  File "C:\Coding\Python\lib\site-packages\py4web\core.py", line 739, in wrapper

    ret = func(*args, **kwargs)

  File "C:\Coding\py4web\apps\my_doc_app\common.py", line 170, in download

    return downloader(db, settings.UPLOAD_PATH, filename)

  File "C:\Coding\Python\lib\site-packages\py4web\utils\downloader.py", line 40, in downloader

    (original_name, stream) = field.retrieve(filename, path, nameonly=True)

  File "C:\Coding\Python\lib\site-packages\pydal\objects.py", line 2109, in retrieve

    file_properties = self.retrieve_file_properties(name, path)

  File "C:\Coding\Python\lib\site-packages\pydal\objects.py", line 2140, in retrieve_file_properties

    filename = to_unicode(

  File "C:\Coding\Python\lib\site-packages\pydal\_compat.py", line 172, in to_unicode

    return obj.decode(charset, errors)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdb in position 0: invalid continuation byte

ERROR:tornado.access:500 GET / my_doc_app/download/documents.filename.4c45e5ab3af76cb7.dbf1631adf60e57ac4971f9f4d91cfc2b3ae5d8e517ed857d556016c8207.png (127.0.0.1) 46.82ms