Issue 280 in pyftpdlib: Unable to complete directory listing with invalid UTF8 characters

[pyft...@googlecode.com]

Status: New
Owner: d...@devicenull.org
Labels: Type-Defect Priority-Medium Component-Library

New issue 280 by d...@devicenull.org: Unable to complete directory listing
with invalid UTF8 characters
http://code.google.com/p/pyftpdlib/issues/detail?id=280

We had a user manage to get an invalid UTF8 byte sequence in one of their
file names (I believe this was done through external software, not
uploading a file via FTP). After that, they were no longer able to
retrieve a directory list, the FTP server threw the following exception:

ERROR unhandled exception in instance <pyftpdlib.handlers.TLS_DTPHandler
object at 0x8f3ca2c>
Traceback (most recent call last):
File "pyftpdlib.handlers", line 1658, in push_dtp_data
File "pyftpdlib.handlers", line 599, in push_with_producer
File "asynchat", line 190, in push_with_producer
File "pyftpdlib.handlers", line 605, in initiate_send
File "asynchat", line 226, in initiate_send
File "pyftpdlib.handlers", line 992, in more
File "pyftpdlib.filesystems", line 564, in format_mlsx
File "encodings.utf_8", line 16, in decode
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 43:
invalid continuation byte


I'm afraid I don't have any way of determining which filename was the
issue, but our workaround for this was this patch:

--- a/pyftpdlib-1.2.0/pyftpdlib/filesystems.py
+++ b/pyftpdlib-1.2.0/pyftpdlib/filesystems.py
@@ -448,7 +448,7 @@ class AbstractedFS(object):
# http://bugs.python.org/issue683592
file = os.path.join(bytes(basedir), bytes(basename))
if not isinstance(basename, unicode):
- basename = unicode(basename, 'utf8')
+ basename = unicode(basename, 'utf8', 'ignore')
else:
file = os.path.join(basedir, basename)
try:
@@ -561,7 +561,7 @@ class AbstractedFS(object):
# http://bugs.python.org/issue683592
file = os.path.join(bytes(basedir), bytes(basename))
if not isinstance(basename, unicode):
- basename = unicode(basename, 'utf8')
+ basename = unicode(basename, 'utf8', 'ignore')
else:
file = os.path.join(basedir, basename)
# in order to properly implement 'unique' fact (RFC-3659,


--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[pyft...@googlecode.com]

Updates:
Status: FixedInSVN
Labels: -Priority-Medium Priority-High Milestone-1.3.1

Comment #1 on issue 280 by g.rodola: Unable to complete directory listing

with invalid UTF8 characters
http://code.google.com/p/pyftpdlib/issues/detail?id=280

Committed in r1245. Thanks.