Problem with upload field

[Константин Комков]

Part of users and me don't have problem, but another part of users can't load files. On server I see error:

<class 'FileNotFoundError'> [Errno 2] No such file or directory: 'C:\\inetpub\\wwwroot\\applications\\test\\databases\\..\\uploads\\doc_images.file.b520635e438be8d7.646f635f696d616765732e66696c652e383062646532666332326635616663382e3331333533393332333333393330333133383339333733363338333633303333333733343333333433303335333133343336333333393333333533383334333132653661373036372e6a7067.jpg'

my tables.py

db_app.define_table('doc_images', Field('person', label='ФИО', writable=False), Field('doctype', 'integer'),

                    Field('file', 'upload', label='Документ',

                          requires=[IS_UPLOAD_FILENAME(extension='^(pdf|tiff|png|bmp|jpeg|jpg)$'),

                                    IS_LENGTH(10485760, 1024)]), format='%(title)s')

line with error in controller:

load = db_app.doc_images.validate_and_insert(person=f'{f} {i} {number} {email}',

                                                                 doctype=document.doctype, file=document[key])

What I need to do in that case?

[Константин Комков]

I thought that there is problem f'{f} {i} {number} {email}' here, but on my computer I can send document without problem.

I and another user (with the same data and loading file) having different result (:

Why in error I see databases\\..\\uploads?

What means two points?

That folder have next structure:

- test

    - databases

    - uploads

[Dave S]

On Thursday, June 18, 2020 at 6:26:32 AM UTC-7, Константин Комков wrote:

I thought that there is problem f'{f} {i} {number} {email}' here, but on my computer I can send document without problem.

I and another user (with the same data and loading file) having different result (:

Why in error I see databases\\..\\uploads?

What means two points?

("dot dot") -- that's the usual syntax (in Windows, Unix, and Linux) for "parent directory". 

The code has a variable that tells it where the databases directory is located for this application, and it constructs a path to the uploads directory by saying "go up one level from here and look for uploads".

That folder have next structure:

- test

    - databases

    - uploads

 

The file name that is Not Found follows the typical format for an uploaded file:

 [nameoftable].[nameoffieild[.[randomnumber].[encodedoriginalfilename].[ext]

When I decode the originalfilename part, it looks like:

doc_images.file.80bde2fc22f5afc8.31353932333930313839373638363033373433343035313436333933353834312e6a7067.jpg

and that's the same format!  You seem to have uploaded a file that was copied from the uploads folder (rather than downloaded); that's easiest to do when the client (browser) and server are on the same machine.

[If you're wondering what the original original name was, it looks to be

15923901897686037434051463935841.jpg)

Do either filename exist in the uploads directory?   It would seem the one in the error message doesn't, because the file open couldn't find it.

/dps

[Константин Комков]

Dave S, hello and thank you for attention!

You are right my user (my colleague) load file with name in web2py format and don't tell me about it (:

But I have error in any way becouse there is users which can't load file with web2py format and I don't know how they are doing that.

I have examples filenames of that users from errors:

doc_images.file.b293994694c6d81d.32303230303631375f31393231343920d09fd0b0d181d0bfd0bed180d18220d09fd0bed0b4d0bed181d0b8d0bdd0bdd0b8d0bad0bed0b2d0b020d09bd0b5d0bdd0b020d181d182d18020d0be20d0b7d0b0d0b3d180d0b0d0bdd0bfd0b0d181d0bfd0bed180d182d0b52e6a7067.jpg

doc_images.file.aefe38bc2064002c.d0a3d0b4d0bed181d182d0bed0b2d0b5d180d0b5d0bdd0b8d0b520d0ba20d0b7d0bed0bbd0bed182d0bed0b920d0bcd0b5d0b4d0b0d0bbd0b820d09fd0bed0b4d0bed181d0b8d0bdd0bdd0b8d0bad0bed0b2d0b020d09bd0b5d0bdd0b02e706466.pdf

That filenames looks like previous, but I don't sure.

There is file with name (doc_images.file.80bde2fc22f5afc8.31353932333930313839373638363033373433343035313436333933353834312e6a7067) in upload folder.

That file with name (doc_images.file.b520635e438be8d7.646f635f696d616765732e66696c652e383062646532666332326635616663382e3331333533393332333333393330333133383339333733363338333633303333333733343333333433303335333133343336333333393333333533383334333132653661373036372e6a7067.jpg) are absent.

Current situation 137 users with success and 5 users can't load files (For that 5 users > 100 tikets was created on server with one type of error: No such file or directory)

[Константин Комков]

I am very happy today! I can find user who can't load file and talk with him. User can't load file with name:

ФГБОУ ВО Липецкий государственный педагогический университет им. П.П. Семенова-Тян-Шанского

filename in error for that file is 

doc_images.file.9bbb3bdd72dc6f61.d0a4d093d091d09ed0a320d092d09e20d09bd0b8d0bfd0b5d186d0bad0b8d0b920d0b3d0bed181d183d0b4d0b0d180d181d182d0b2d0b5d0bdd0bdd18bd0b920d0bfd0b5d0b4d0b0d0b3d0bed0b3d0b8d187d0b5d181d0bad0b8d0b920d183d0bdd0b8d0b2d0b5d180d181d0b8d182d0b5d18220d0b8d0bc2e20d09f2ed09f2e20d0a1d0b5d0bcd0b5d0bdd0bed0b2d0b02dd0a2d18fd0bd2dd0a8d0b0d0bdd181d0bad0bed0b3d0be2e6a7067.jpg

How can I validate it? As I understand I need to set maximal length for filename, yes?

[Dave S]

In the book,

<URL:http://web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#Field-types>

it says there is no default validator for upload fields, and there is a remark that "default length is 512 bytes" , but the name-encoding process is likely to cross the 512 byte boundary somewhere around 200-240 original characters (depending on how many multi-byte characters there are).  In addition to the DB limits, the OS is likely to have a maximum size of file name (I don't remember what it is for NTFS, but it is likely smaller than the maximum permissable path; a quick goo-search indicates both are 260 unless you have W10 and you have changed a registry setting)..  If the filesystem isn't allowing the encoded name because of length, I'd have expected the ticket to occur before the not-found error, but I haven't checked out how that try-except is set up.

I haven't tried setting a validator, but you've already done that.  IS_UPLOAD_FILENAME() seems not to have a filename length check.  You could try making your regex not fit names that exceed a limit, or you could do a ON_VALIDATE hook to check the length directly.

/dps

[Константин Комков]

I tried to load file with long filename until that file was loaded. So maximal length for filename is 34 symbols.

I do like that:

last_symbol = len(document[key].filename) - 1

while document[key].filename[last_symbol] != '.':

    last_symbol -= 1

without_ext = document[key].filename[:last_symbol]

if len(without_ext) < 35:

    load = db_app.doc_images.validate_and_insert(person=f'{f} {i} {number} {email}',

                                                 doctype=document.doctype, file=document[key])
else:
    # return message about filename < 35 sybols.