Crash when upload a file which name contains non-ascii characters

[Flier Lu]

Its seems tornado force to use Unicode when parsing the HTTP request
MIME body.

2011-05-23 11:19:24,450 [11056:11060] root ERROR Exception in callback
<tornado.
stack_context._StackContextWrapper object at 0x022E1210>
Traceback (most recent call last):
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\ioloop.py", line 356, in _run_callback
callback()
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\stack_context.py", line 156, in wrapped
callback(*args, **kwargs)
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\iostream.py", line 234, in wrapper
callback(*args)
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\stack_context.py", line 156, in wrapped
callback(*args, **kwargs)
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\httpserver.py", line 409, in _on_request_body
self._parse_mime_body(v, data)
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\httpserver.py", line 427, in _parse_mime_body
parts = data[:-footer_length].split("--" + boundary + "\r\n")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position
609: ordinal not in range(128)

[Michael Wilson]

I think this was just fixed on on the trunk. You'll have to git clone
the branch to get the code (I think).

[Ben Darnell]

Actually, I think it was probably broken recently by the python3 merge.  I've just committed a change that should fix things.  Please let me know if you're still having problems.

https://github.com/facebook/tornado/commit/a4d4c2379af058632428b5942cc8a683fff09639

-Ben

[Flier Lu]

it caused another crash, why decode it with latin1?

2011-05-24 14:05:02,759 [15244:18372] root ERROR Exception in callback
<tornado.stack_context._StackContextWrapper object at 0x02488900>

Traceback (most recent call last):
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\ioloop.py", line 356, in _run_callback
callback()
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\stack_context.py", line 156, in wrapped
callback(*args, **kwargs)
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\iostream.py", line 234, in wrapper
callback(*args)
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg
\tornado\stack_context.py", line 156, in wrapped
callback(*args, **kwargs)
File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg

\tornado\httpserver.py", line 410, in _on_request_body
self._parse_mime_body(utf8(v), data)

File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg

\tornado\httpserver.py", line 435, in _parse_mime_body
headers = httputil.HTTPHeaders.parse(part[:eoh].decode("latin1"))

File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg

\tornado\httputil.py", line 104, in parse
h.parse_line(line)

File "C:\PROGRA~1\Python\lib\site-packages\tornado-1.2.1-py2.6.egg

\tornado\httputil.py", line 90, in parse_line
name, value = line.split(":", 1)
ValueError: need more than 1 value to unpack

>>>>>>>>>>>>>>>>>>>request payload<<<<<<<<<<<<<<<<<<<<<
Origin: http://localhost:8080
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.36 (KHTML,
like Gecko) Chrome/13.0.767.1 Safari/534.36
Content-Type: multipart/form-data; boundary=----
WebKitFormBoundaryUetNY85NB6tMcx4H
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8
Referer: http://localhost:8080/


------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="_xsrf"

c9042e643f2b4b6caa1d986eb2f11006
------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="type"

law
------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="categorySelected"

é‡‘èž >ä¿ é™©
------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="categoryInput"

é‡‘èž >ä¿ é™©
------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="tags"


------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="name"


------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="content"


------WebKitFormBoundaryUetNY85NB6tMcx4H
Content-Disposition: form-data; name="file"; filename="ä¿ é™©å…
¬å ¸ä¿¡æ ¯ç³»ç»Ÿå®‰å…¨ç®¡ç †æŒ‡å¼•.xls"
Content-Type: application/vnd.ms-excel


------WebKitFormBoundaryUetNY85NB6tMcx4H--

On May 24, 1:11 pm, Ben Darnell <b...@bendarnell.com> wrote:
> Actually, I think it was probably broken recently by the python3 merge.
>  I've just committed a change that should fix things.  Please let me know if
> you're still having problems.
>

> https://github.com/facebook/tornado/commit/a4d4c2379af058632428b5942c...
>
> -Ben

[Ben Darnell]

On Mon, May 23, 2011 at 11:11 PM, Flier Lu <flier.lu@gmail.com> wrote:

it caused another crash, why decode it with latin1?

Hmm.  HTTP headers are specified as latin1, but it looks like multipart/form-data is utf8 (http://www.w3.org/TR/html5/association-of-controls-and-forms.html#multipart-form-data).  I've committed a fix.

-Ben

[Flier Lu]

verified, thanks

On May 24, 2:24 pm, Ben Darnell <b...@bendarnell.com> wrote:

> On Mon, May 23, 2011 at 11:11 PM, Flier Lu <flier...@gmail.com> wrote:
> > it caused another crash, why decode it with latin1?
>
> Hmm.  HTTP headers are specified as latin1, but it looks like

> multipart/form-data is utf8 (http://www.w3.org/TR/html5/association-of-controls-and-forms.html#mul...).