Japanese named File - FileUpload django admin throws error

[Raaj Supramanian]

class FileUpload(models.Model):
    cstr = models.CharField(max_length=50)
    file_nm = models.CharField(max_length=250,editable=False,verbose_name='Name')
    file_type_cd = models.CharField(max_length=10,editable=False, verbose_name='Type')
    file_size_val = models.CharField(max_length=25,editable=False, verbose_name='Size in MB')
    file_upld_dts = models.DateTimeField(default =datetime.datetime.now() ,editable=False, verbose_name='Uploaded Date & Time')
    fileupload = models.FileField(upload_to=fs,verbose_name='File to Upload')
    file_title = models.CharField(max_length=250,verbose_name='Title')
    fileretention = models.IntegerField(default=18,verbose_name = '# months of Retention')
    def __unicode__(self):
        return unicode(self.file_nm)

This is my django model for File upload functionality, when a try to upload a file which has a japanese name i'm getting below error.


Environment:


Request Method: POST
Request URL: https://10.0.2.13/myapp/admin/allinone/fileupload/add/

Django Version: 1.4.2
Python Version: 2.6.6
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'allinone')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Traceback:
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/contrib/admin/options.py" in wrapper
  366.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/utils/decorators.py" in _wrapped_view
  91.                     response = view_func(request, *args, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  89.         response = view_func(request, *args, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/contrib/admin/sites.py" in inner
  196.             return view(request, *args, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/utils/decorators.py" in _wrapper
  25.             return bound_func(*args, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/utils/decorators.py" in _wrapped_view
  91.                     response = view_func(request, *args, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/utils/decorators.py" in bound_func
  21.                 return func(self, *args2, **kwargs2)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/transaction.py" in inner
  209.                 return func(*args, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/contrib/admin/options.py" in add_view
  955.                 self.save_model(request, new_object, form, False)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/contrib/admin/options.py" in save_model
  709.         obj.save()
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/base.py" in save
  463.         self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/base.py" in save_base
  551.                 result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/manager.py" in _insert
  203.         return insert_query(self.model, objs, fields, **kwargs)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/query.py" in insert_query
  1593.     return query.get_compiler(using=using).execute_sql(return_id)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/sql/compiler.py" in execute_sql
  909.         for sql, params in self.as_sql():
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/sql/compiler.py" in as_sql
  872.                 for obj in self.query.objs
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/fields/files.py" in pre_save
  249.             file.save(file.name, file, save=False)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/db/models/fields/files.py" in save
  86.         self.name = self.storage.save(name, content)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/core/files/storage.py" in save
  44.         name = self.get_available_name(name)
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/core/files/storage.py" in get_available_name
  70.         while self.exists(name):
File "/opt/venv/django-cmdb/lib/python2.6/site-packages/django/core/files/storage.py" in exists
  230.         return os.path.exists(self.path(name))
File "/usr/lib64/python2.6/genericpath.py" in exists
  18.         st = os.stat(path)

Exception Type: UnicodeEncodeError at /admin/allinone/fileupload/add/
Exception Value: 'ascii' codec can't encode character u'\u524d' in position 76: ordinal not in range(128)


Can someone please help on how to overcome this.

Thanks,
Raaj

[François Schiettecatte]

Hi

I suspect there is a disconnect somewhere between the encoding used by the browser and the encoding used by django to decode the file name.

François

> --
> You received this message because you are subscribed to the Google Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/19d893fd-23dc-4d1c-a2f2-36b7800093cb%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Xavier Ordoquy]

Hi,

I've run into similar issues while saving files with non ascii file names.

In my case, it turns out that supervisor was removing the environment variables leave Django left with a C locale (ie, ascii only filenames).

It worked once I added - for example - LANG="en_US.UTF-8", LC_ALL="en_US.UTF-8", LC_LANG="en_US.UTF-8".

Make sure you correctly set your locales for the process that runs Django.

Best regards,

Xavier,

Linovia.