Is improving Django's communication with Oracle databases on the roadmap?

[Daniel Porter]

Hi list,

I recently started working in the research wing of Moffitt Cancer Center, and our shop uses django for +10 in-house applications, running almost exclusively against oracle applications. Django has enabled building some great applications, but the performance is kind of terrible when run against an oracle backend. Asking on the cx_oracle list, Shai Berger told me it was probably due to the known problem where numbers get converted to strings and then back to numbers. Shai also pointed me to a patch he wrote that might mitigate it, which I haven't had time to work with but am optimistic about.

Reading the thread he linked, it seemed like there was a decision not to make the fix - I'm hazy on the technical details, but I think it was because the fix required a loss of precision from an oracle-specific type of decimal field, and cx_oracle didn't allow a way to differentiate between this field type and other number types. I'm getting some free time towards the end of the month where I can dig more into the patch/explore causes more fully, but I wanted to ask here - has that decision been revisited, or does it stand?

If Shai's patch improves performance, then I'm sure we can just keep our own, patched version of django. I would be stoked for there to be a main branch solution, though.

Thanks!

Daniel

[Josh Smeaton]

Hi Daniel,

Can you provide links to the patches and discussion you're referring to? It'll be easier to provide some feedback if we have the detail in front of us.

I'm all for improving performance for the Oracle backend - as long as backwards compatibility or correctness doesn't suffer. If either of those are concerns, perhaps we can find a way to overcome them.

Cheers

[Marc Tamlyn]

I can't comment on the specifics of this issue, but I would like to mention that there is no roadmap for Django development in general. Oracle expertise is an area we struggle with compared to other backends, and all outside enthusiasm for improving the Oracle backend is gratefully received!

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/951372c6-2780-45a1-9082-ae356bff193b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Porter]

Josh,

Oops. Yes, here are the links:

This is the discussion Shai had about the problem with other developers: https://groups.google.com/forum/#!topic/django-developers/4BNkJyGez9A/discussion

This is the pull request that contains the patch: https://github.com/django/django/pull/393 It's coming back to me now that the patch was a little messy; part of the problem with it was that Shai had included more than the Oracle fix.

Not strictly relevant, here is my conversation with Shai on the cx-oracle-users list: http://sourceforge.net/p/cx-oracle/mailman/cx-oracle-users/thread/CACsLnW_n85PDVVueE5h4K_jHrntnNfH2rWoyOrMhPPkNfhi8vg%40mail.gmail.com/#msg33641087

As far as improving the Oracle backend: consider Research IT at Moffitt enthusiastic. They're willing to give me some development time to work on this issue, and if we can make something that is useful to django at large, I'd be thrilled.

[Shai Berger]

Hi,

For the record, I'm still here :)

Daniel, thanks for pushing this forward.

The main problem is indeed a clash between performance on integers and
accuracy on decimals, that is ultimately the result of cx_Oracle's
limitations: cx_Oracle does not return Python Decimals, only floats. To get
better precision than that of Python floats, it offers a global flag that makes
it return all numbers as strings, and then Python gets to decide what to do
with the number. Django sets this flag.

At the time of that old discussion, I was trying to solve not just the
performance issue, but some related API issues as well, where, in fact, I had
gotten things quite wrong. Having them right, and going to solve only the
perofrmance issue, I think we can make progress.

Another interesting direction, which could enable us to get the performance
improvements in Django without compromising precision (so it is available to
everyone by default), is to alleviate some of cx_Oracle limitations; but that
requires C programming. If you're up to that, talk to me (ref:
http://sourceforge.net/p/cx-oracle/mailman/message/32194748/ )

Shai.

[Josh Smeaton]

Could we selectively set the numberasstring property per query if there are any decimal fields for that particular query? I guess this could help in the interim until/if cx_oracle can fix the problem and that particular version of cx oracle becomes the minimum version django supports.