[Django] #19606: Oracle unicode insert fails.

Django

unread,

Jan 14, 2013, 3:42:37 AM1/14/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.
----------------------------------------------+----------------------------
Reporter: jtiai | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.4
Severity: Normal | Keywords: oracle orm
Triage Stage: Unreviewed | unicode
Easy pickings: 0 | Has patch: 0
| UI/UX: 0
----------------------------------------------+----------------------------
smart_str (which detection selects) does not work.

using force_unicode works (except for connection string).

Depending on what OCI client 10.2.0.5 or instant client 11.2 is used when
compiling cx_Oracle causes variation. 10.2.0.5 doesn't work with smart_str
while 11.2 does work.

Both can take plain unicode (u'<some unicode stuff here>') when using just
cx_Oracle directly without any problems.

Note:

If I add manually some unicode to database Django can read it without any
problems.

--
Ticket URL: <https://code.djangoproject.com/ticket/19606>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Jan 14, 2013, 3:52:36 AM1/14/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------

Reporter: jtiai | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: 1.4
(models, ORM) | Resolution:
Severity: Normal | Triage Stage:
Keywords: oracle orm unicode | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by anonymous):

* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0

Comment:

Original story - https://groups.google.com/forum/?fromgroups=#!topic
/django-users/fXB4wReYth0

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:1>

Django

unread,

Jan 14, 2013, 11:03:15 PM1/14/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------

Reporter: jtiai | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: 1.4
(models, ORM) | Resolution:

Severity: Normal | Triage Stage: Accepted
Keywords: oracle orm unicode | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by akaariai):

* stage: Unreviewed => Accepted

Comment:

I haven't haven't been able to verify this (no 10.2.0.5 here). The report
seems well researched in the django-users thread so accepted based on
that.

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:2>

Django

unread,

Jan 15, 2013, 3:45:25 AM1/15/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------

Reporter: jtiai | Owner: nobody
Type: Bug | Status: new

Comment (by jtiai):

Problem is that Django makes incorrect detection what to string encoding
system to use:

I did also some source code diving to make sure how this works:

In version 5.0.x cx_Oracle.UNICODE is only defined if code was compiled
without flag WITH_UNICODE.

In version 5.1+ cx_Oracle.UNICODE is always defined if Python major
version compiled against is < 3.

According to changelog of cx_Oracle:
>Changes from 5.0.4 to 5.1
> 1) Remove support for UNICODE mode and permit Unicode to be passed
through in
> everywhere a string may be passed in. This means that strings will be
> passed through to Oracle using the value of the NLS_LANG environment
> variable in Python 3.x as well. Doing this eliminated a bunch of
problems
> that were discovered by using UNICODE mode and also removed an
unnecessary
> restriction in Python 2.x that Unicode could not be used in connect
strings
> or SQL statements, for example.

So behaviour should be: If cx_Oracle version is greater or equal to 5.1
real unicode should be used always. With pre-5.1 current behaviour is
sufficient.

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:3>

Django

unread,

Jan 19, 2013, 2:29:07 AM1/19/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------
Reporter: jtiai | Owner: nobody
Type: Bug | Status: closed

Component: Database layer | Version: 1.4

(models, ORM) | Resolution: fixed

Changes (by Anssi Kääriäinen <akaariai@…>):

* status: new => closed
* resolution: => fixed

Comment:

In [changeset:"d194f290571f7e9dda7d2fd7a6f2b171120f2f14"]:
{{{
#!CommitTicketReference repository=""
revision="d194f290571f7e9dda7d2fd7a6f2b171120f2f14"
Fixed #19606 -- Adjusted cx_Oracle unicode detection.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:4>

Django

unread,

Jan 19, 2013, 3:18:43 AM1/19/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------
Reporter: jtiai | Owner: nobody

Comment (by Anssi Kääriäinen <akaariai@…>):

In [changeset:"457290326fa38297467922d827c630ae2026e88b"]:
{{{
#!CommitTicketReference repository=""
revision="457290326fa38297467922d827c630ae2026e88b"
[1.5.x] Fixed #19606 -- Adjusted cx_Oracle unicode detection.

Backpatch of d194f290571f7e9dda7d2fd7a6f2b171120f2f14
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:5>

Django

unread,

Jan 19, 2013, 3:20:16 AM1/19/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------
Reporter: jtiai | Owner: nobody

Comment (by akaariai):

I felt very uncomfortable about backporting this as I can't actually test
the combo this is supposed to fix - that is, cx_oracle 5.1 + instantclient
10.2 as that wont compile on my machine. Based on the patch, cx_oracle 5.0
users should no get any change and I have tested 5.1 + instantclient 11.

I think this falls into the category of patches we can still backpatch to
1.5 at this stage - the combination of cx_oracle 5.1 + instantclient 10.2
is supported, but it doesn't work with unicode values currently.

So, after using too much time trying to tests this, I did backpatch this
on the grounds that instanclient 11 works, cx_oracle 5.0 works as always,
and the research done by jtiai shows that this is the right thing to do
for 5.0 + instantclient 10.

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:6>

Django

unread,

Jan 22, 2013, 1:24:26 AM1/22/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------
Reporter: jtiai | Owner: nobody

Comment (by jtiai):

According to my research it's all about version of cx_Oracle 5.0.x series
does behave have slightly different definition for cx_Oracle.UNICODE than
5.1.x

It has nothing to do with OCI version linked against even I though that
was the reason. But I think 1.5 is sufficient at this point.

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:7>

Django

unread,

Feb 8, 2013, 3:23:05 AM2/8/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------
Reporter: jtiai | Owner: nobody

Comment (by Anssi Kääriäinen <akaariai@…>):

In [changeset:"457290326fa38297467922d827c630ae2026e88b"]:
{{{
#!CommitTicketReference repository=""
revision="457290326fa38297467922d827c630ae2026e88b"
[1.5.x] Fixed #19606 -- Adjusted cx_Oracle unicode detection.

Backpatch of d194f290571f7e9dda7d2fd7a6f2b171120f2f14
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/19606#comment:8>

Django

unread,

Jun 5, 2013, 6:48:13 AM6/5/13

to django-...@googlegroups.com

#19606: Oracle unicode insert fails.

-------------------------------------+-------------------------------------
Reporter: jtiai | Owner: nobody