Issues with french accents

[Samuel Brunel]

Hi,

Context:

Django 1.11.x
Python 3.4.5
PostgreSQL 9.6

Database informations :

data=> \l
                                 List of databases
   Name    |   Owner   | Encoding |  Collate   |   Ctype    |   Access privileges  
-----------+-----------+----------+------------+------------+-----------------------
 data      | data_user | UTF8     | en_US.utf8 | en_US.utf8 |


data=> select * from enumeration_choice;
 id |           name           | enumeration_id
----+--------------------------+----------------
  1 | Je suis la génèse        |              1
  2 | Je suis la 2eme version  |              1
  3 | Je suis la xieme version |              1
  4 | énuméréStatique1         |              2
  5 | énuméréStatique2         |              2
  6 | énuméréStatique3         |              2
...
 12 | Inactif                  |              4
(12 rows)

Try to dump data...

./manage.py dumpdata mymodel.enumerationchoice

...
{
  "model": " mymodel.enumerationchoice",
  "pk": 5,
  "fields": {
    "name": "\u00e9num\u00e9r\u00e9Statique2",
    "enumeration": 2
  }
},
{
  "model": " mymodel.enumerationchoice",
  "pk": 6,
  "fields": {
    "name": "\u00e9num\u00e9r\u00e9Statique3",
    "enumeration": 2
  }
},
...

énuméréStatique2   --> "\u00e9num\u00e9r\u00e9Statique2"

I've got commands in management where I use

"json_str = json.dumps(data, ensure_ascii=False, indent=4)"

to get french accents but I've no idea to do this with manage.py.

How to get french accents conversion with "./manage.py dumdata ..." ?

Regards,

Samuel

[James Bennett]

JSON is a subset of JavaScript's object-literal syntax.

In JavaScript, as in Python 3, string literals are defined to be Unicode. JSON is defined as specifically using UTF-8 as its encoding of Unicode.

However, not all systems can handle Unicode passing through them. Luckily, JavaScript, like Python, allows the use of escape sequences to express any Unicode code point using only characters legal in ASCII (which is important for things like email -- email is built on an assumption of 7-bit ASCII!).

And in both Python and JavaScript, the escape sequence for a Unicode code point in the Basic Multilingual Plane begins with "\u" followed by the code point in hexadecimal.

The code point of "é" in Unicode, expressed in hexadecimal, is 00E9:

http://www.fileformat.info/info/unicode/char/e9/index.htm

Therefore, "\u00e9" -- in both Python and JavaScript string literals -- is interpreted as being exactly equivalent to "é". Try it yourself by opening a Python interpreter and typing this:

print("\u00e9")

In other words, Django isn't changing the contents of your string here. It's just making sure to serialize it in a way that's as safe as possible for transmitting the data in JSON format.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJVNoBcTbW3N3Pdn3imkKX3e0Sibnn4zuLRR7wd8gk%3D6%2BG8Lqg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Samuel Brunel]

James,

Thanks for this explaination.

Samuel

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAL13Cg956MaW_dZkTHzznq60Ow%2B7Pa6D1JgzjLU85iT-HZTwdw%40mail.gmail.com.

[Haven McConnell]

I've never heard of a sandwich flicker