| FileSystemEncodingChanged exceptions | Doug Evenhouse | 11/25/12 1:51 PM | I'm spinning my wheels a bit with an FileSystemEncodingChanged exception that is thrown whenever I try to browse media files. I was happily uploading images and all was well for awhile. Now I cannot open the file browser at all without this exception being thrown. I don't believe I have created any folders nor uploaded any files with unusual encoding or names. As far as I can nothing would have changed with regard to locale settings. I removed the entire project and recreated it to no avail. The exceptions occur on both a staging server and a production server running Ubuntu 12.04, and deployed using vanilla fabric settings with Mezzanine 1.2.4. LC_ALL appears to be set correctly to "en_US.UTF-8". Anyone have advice as to possible root causes and/or troubleshooting steps to take? Traceback and request settings from the exception email - this is running on our internal staging server): File "/home/doug/virtualenvs/hccs_website/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 105, in get_response response = middleware_method(request, callback, callback_args, callback_kwargs) File "/home/doug/virtualenvs/hccs_website/local/lib/python2.7/site-packages/mezzanine/pages/middleware.py", line 45, in process_view return view_func(request, *view_args, **view_kwargs) File "/home/doug/virtualenvs/hccs_website/local/lib/python2.7/site-packages/django/contrib/admin/views/decorators.py", line 16, in _checklogin return view_func(request, *args, **kwargs) File "/home/doug/virtualenvs/hccs_website/local/lib/python2.7/site-packages/django/views/decorators/cache.py", line 89, in _wrapped_view_func response = view_func(request, *args, **kwargs) File "/home/doug/virtualenvs/hccs_website/local/lib/python2.7/site-packages/filebrowser_safe/views.py", line 116, in browse elif fileobject.filetype == 'Folder' and fileobject.is_empty: File "/home/doug/virtualenvs/hccs_website/local/lib/python2.7/site-packages/filebrowser_safe/base.py", line 134, in _is_empty raise FileSystemEncodingChanged() FileSystemEncodingChanged: Access was attempted on a file that contains unicode characters in its path, but somehow the current locale does not support utf-8. You may need to set 'LC_ALL' to a correct value, eg: 'en_US.UTF-8'. <WSGIRequest path:/admin/media-library/browse/, POST:<QueryDict: {}>, 'messages': 'COOKIES:{'__utma': '74311563.372776308.1351892605.1351898182.1351966440.4', '__utmz': '74311563.1351892605.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)', 'csrftoken': 'l754gpeXgo6WrMKmH5gQ0UpsG96acn41', 81375d9cdb262e44aff2f71e2a66e74c9742ea2a$[["__json_message",40,"The requested Folder does not exist."]]', 'mezzanine-admin-toolbar': '', 'HTTP_COOKIE': '__utma=74311563.372776308.1351892605.1351898182.1351966440.4; __utmz=74311563.1351892605.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); sessionid=4fbd7d0d1511a2c157fe2bdc60aed3cf; mezzanine-admin-toolbar=; csrftoken=l754gpeXgo6WrMKmH5gQ0UpsG96acn41; messages="81375d9cdb262e44aff2f71e2a66e74c9742ea2a$[[\\"__json_message\\"\\05440\\054\\"The requested Folder does not exist.\\"]]"','sessionid': '4fbd7d0d1511a2c157fe2bdc60aed3cf'}, META:{'CSRF_COOKIE': 'l754gpeXgo6WrMKmH5gQ0UpsG96acn41', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_ACCEPT_CHARSET': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3', 'HTTP_ACCEPT_ENCODING': 'gzip,deflate,sdch', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.8', 'HTTP_CONNECTION': 'close', 'HTTP_HOST': 'hccs-website.evenhouseconsulting.local', 'HTTP_REFERER': 'http://hccs-website.evenhouseconsulting.local/admin/people/person/add/', 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac
OS X 10_8_2) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64
Safari/537.11', 'QUERY_STRING': 'type=Image&pop=1','HTTP_X_FORWARDED_FOR': '192.168.30.53', 'HTTP_X_FORWARDED_PROTOCOL': 'http', 'HTTP_X_REAL_IP': '192.168.30.53', 'PATH_INFO': u'/admin/media-library/browse/', 'RAW_URI': '/admin/media-library/browse/?type=Image&pop=1', 'REMOTE_ADDR': '192.168.30.53', 'gunicorn.socket': <socket._socketobject object at 0x304a8a0>,'REMOTE_PORT': '80', 'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': u'', 'SERVER_NAME': 'hccs-website.evenhouseconsulting.local', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'SERVER_SOFTWARE': 'gunicorn/0.16.1', 'wsgi.errors': <open file '<stderr>', mode 'w' at 0x7fcdddc80270>, 'wsgi.input': <gunicorn.http.body.Body object at 0x2fa16d0>,'wsgi.file_wrapper': <class gunicorn.http.wsgi.FileWrapper at 0x1df10b8>, 'wsgi.multithread': False, 'wsgi.run_once': False, 'wsgi.url_scheme': 'http', 'wsgi.version': (1, 0)}> |
| Re: FileSystemEncodingChanged exceptions | Doug Evenhouse | 11/26/12 11:23 AM | Thank you Pat for pointing me in (what appears to be so far) the right direction! Much appreciated! I confirmed that starting the gunicorn processes directly from the command line corrected the issue. I don't have a gunicorn.sh script but could run the gunicorn_django script after stopping the supervisor service. I attempted to add LC_ALL=en_US.UTF-8 to my supervisor startup script as you did but for some reason that did not change anything for me. What I ended up doing that worked for me was to add the following line to the supervisor.conf template in Mezzanine's deploy folder: environment=LANG="en_US.UTF-8",LC_ALL="en_US.UTF-8",LC_LANG="en_US.UTF-8" Then after a "fab deploy" and then manually stopping and starting the supervisor service I was able to browse media files again. By the way this post was particularly helpful to me: http://albertoconnor.ca/blog/2012/Jul/21/unicodeencodeerror-when-uploading-files-django-usi Thanks again Pat. On Sunday, November 25, 2012 5:51:29 PM UTC-6, Pat James wrote: I had the same problem today. What a headache! |
| Re: [mezzanine-users] Re: FileSystemEncodingChanged exceptions | Stephen McDonald | 11/26/12 11:24 AM | Doug would you like to send a pull request with this change? Sounds like it's quite necessary.-- Stephen McDonald http://jupo.org |
| Re: [mezzanine-users] Re: FileSystemEncodingChanged exceptions | Doug Evenhouse | 11/26/12 11:42 AM | Steve, I'd like to have a little more time to be sure I'm out of the woods on this but, yes, I can send a pull request with this change. This will be my first pull request ever so be prepared to be patient :) |
| Re: [mezzanine-users] Re: FileSystemEncodingChanged exceptions | Kenneth Bolton | 11/26/12 11:42 AM | like! |
| Re: [mezzanine-users] Re: FileSystemEncodingChanged exceptions | Stephen McDonald | 11/26/12 12:28 PM | No rush at all Doug, better to take the time to get it right. Thank you!
|
| Re: [mezzanine-users] Re: FileSystemEncodingChanged exceptions | Maciek Gryka | 3/17/13 11:15 AM | Just wanted to say thanks as well - this solved a long-annoying problem for me! Would be good to see the change deployed :) |
| Re: [mezzanine-users] Re: FileSystemEncodingChanged exceptions | Stephen McDonald | 3/18/13 3:04 PM | Not sure if you're referring to this fix being added to Mezzanine proper, but Doug contributed this and it was merged in 4 months ago: https://github.com/stephenmcd/mezzanine/commit/bbfd1382e8666340058cb9d2be5989ee39cb9d44
--
|
| Re: FileSystemEncodingChanged exceptions | Bartosz Wiklak | 9/18/13 1:17 PM | I had similar problem, but I wasn't using supervisor. A file name in /media/uploads/gallery has strange character encoding (the folder contains jspeg Austin, Athens, Amsterdam ...), I wasn't uploading anything there, suppose this files are uploaded by mezzanine. My locale is pl_PL.UTF8. The exception description is a little bit misleading. |
| Re: FileSystemEncodingChanged exceptions | Terry Brown | 2/25/14 8:54 AM | +10 on this comment - spent most of a day setting LC_* to en_US.UTF-8 all over the place, didn't help. Deleted ./static/media/uploads/gallery/Ávila, Spain.jpg ./static/media/uploads/gallery/Düsseldorf, Germany.jpg ./static/media/uploads/gallery/.thumbnails/Ávila, Spain-60x60.jpg ./static/media/uploads/gallery/.thumbnails/Düsseldorf, Germany-60x60.jpg and all worked again. Note that the system always worked on the Python 3.3 dev. environment, the problem with the above was on the Python 2.7 deploy environment. Thanks Bartosz! |
| Re: [mezzanine-users] Re: FileSystemEncodingChanged exceptions | Kenneth Bolton | 2/25/14 9:41 AM | Hi Terry, If you look at the fabfile.py, https://github.com/stephenmcd/mezzanine/blob/master/mezzanine/project_template/fabfile.py#L339-L354, you can see how the system should be configured to handle the demonstrative gallery images included in Mezzanine.
hth, ken
|
| Re: FileSystemEncodingChanged exceptions | Radek Wojcik | 3/5/14 7:49 PM | I ran into this problem today, and I don't use supervisor, nor do I use gunicorn. Today I switched to gunicorn though cuz I couldn't figure out what was going on. There were no issues in dev mode when I ran the server using runserver. The issues weren't obvious even when running the uwsgi server from command line, it "seemed" to work, but there was a complete issue when trying to run it as a service using upstart... I should probably use the same stack everyone else is using and learn the fab file that comes with.. But anyways, I switched to gunicorn and it worked as a service so I thought ok problem solved. But then I got the Local Changed error. What fixed it for me was exporting all these locale variables as described above using export. Now even uwsgi is working great now.. I'm still sticking to sockets but I'll use gunicorn just to be safe. I thought uWSGI was supposed to be faster thats why I was using it. I don't know much about supervisor so I use upstart, and I use sockets because its slightly faster (no TCP/IP overhead). export LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8 exec gunicorn -k eventlet ${DJANGO_WSGI_MODULE}:application \ --name $NAME \ --workers $NUM_WORKERS \ --log-level=debug \ --bind=unix:$SOCKFILE |