[Django] #27219: Emoji don't work in TextFields using Oracle backend
Django
----------------------------------------------+----------------------------
Reporter: dmedvinsky | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.10
Severity: Normal | Keywords: oracle,
| unicode
Triage Stage: Unreviewed | Has patch: 1
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+----------------------------
When using cx_Oracle as a backend, I can't use Emoji in TextFields, which
are stored as NCLOBs in the database. When inputting an emoji and saving
them in the database, upon retrieving I get four replacement charaters (�)
per emoji instead.
Django
[https://github.com/django/django/blob/18c72d59e0807dae75ac2c34890d08c1e0972d0a/django/db/backends/oracle/base.py#L44
uses hardcoded NLS_LANG=".UTF8"] variable, which is outdated, only
supports Unicode standard version 3 and cannot properly store 4-byte
characters. [https://docs.oracle.com/database/121/NLSPG/ch6unicode.htm
#NLSPG-GUID-EB57AB68-A390-4814-81DD-6B78D33310CC The Oracle documentation
recommends] using the AL32UTF8 encoding instead, which supports latest
Unicode standard and stores 4-byte characters properly.
The change that fixes the emoji for me:
https://github.com/django/django/pull/7241
I recognize that it needs some tests, so I seek the help from those with
better knowledge of Django test cases than me. If someone can kindly
direct me, I can create the test for this case.
--
Ticket URL: <https://code.djangoproject.com/ticket/27219>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: dmedvinsky | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: oracle, unicode | Triage Stage:
| Unreviewed
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by dmedvinsky):
* needs_better_patch: => 0
* needs_tests: => 1
* needs_docs: => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27219#comment:1>
Django
Reporter: dmedvinsky | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Changes (by timgraham):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/27219#comment:2>
Django
Reporter: dmedvinsky | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: oracle, unicode | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by dmedvinsky):
* needs_tests: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27219#comment:3>
Django
Reporter: dmedvinsky | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: oracle, unicode | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: 0 => 1
Comment:
The new tests don't pass on MySQL due to #18392. We can either fix that
issue first, or skip the test on MySQL.
--
Ticket URL: <https://code.djangoproject.com/ticket/27219#comment:4>
Django
Reporter: dmedvinsky | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: oracle, unicode | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27219#comment:5>
Django
Reporter: dmedvinsky | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 1.10
(models, ORM) |
Keywords: oracle, unicode | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Changes (by Tim Graham <timograham@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"1a9f6db5ffd2d5e71d73340ab59476572e05a728" 1a9f6db5]:
{{{
#!CommitTicketReference repository=""
revision="1a9f6db5ffd2d5e71d73340ab59476572e05a728"
Fixed #27219 -- Changed cx_Oracle client encoding to AL32UTF8 to allow
4-byte characters.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27219#comment:6>