Removing Oracle from Django core in 3.0

[Johannes Hoppe]

Hi there,

I have recently refactored some bits in the database backend and came to realize that a lot of the complexity in there comes from the poor implementation of the Oracle backend.

Fun fact, did you know that Oracle tests don't run by default and that the current master, fails on oracle ;)

Anyhow, I want to come to a conclusion about the following matter:

Should we remove the Oracle database backend from Django core in the 3.0 release?

Here are a couple of reasons, why I believe this to be a good idea:

• License
  
• Oracle is  Proprietary software
• Money
• Oracle is not a sponsor of the Django Foundation, but makes 40bn in revenue
• Technical:
• Oracle does not support may features
• due to its lack of features, a lot of edge case handling to the base database backend which drives overall complexity
• Development:
• Oracle does not run in the regular CI suite, in fact master is broken right now
• entrance barrier for first time contributors is high
• one needs to accept a non open source license
• register with oracle
• go through a very complex setup process

Of course there are some users who use Oracle and I don't want to keep them hanging. I simply believe the database backend should be developed separately from Django.

This could even be helpful for the Oracle community. Since oracle is enterprise only, they usually looks for longer support cycles than what Django want's to offer.

Ok, I made my case, I am curious, what do you guys think?

Best

-Joe

[Tim Graham]

I can't find a past discussion specific to Oracle, but it's not a new proposal. See https://groups.google.com/d/topic/django-developers/O-g06EM6XMM/discussion for "Moving database backends out of the core."

I think removing Oracle from core would only increase the maintenance burden. Since Oracle has edge cases, it's useful to test those along with new Django features. If the Oracle backend is in a separate repo, then adding new features will often require commits to two repositories and I don't know how we would run the tests with pull request X for Django and pull request Y for the Oracle backend. Then we also have to release the Oracle backend separately.

djangoci.com isn't reporting any Oracle failures on master. If you've found an issue, please open a ticket with details.

We don't run the Oracle tests with pull requests because they take about an hour, while other databases take about 10 minutes. It hasn't been difficult to identify which pull requests require running the tests on Oracle and to trigger that build with the trigger phrase.

[André Luis Pereira dos Santos]

Move database backends out of the Django's core sounds great.

[Adam Johnson]

Interestingly, I didn't receive your first email Johannes, only Tim's reply. I can't even find it in spam. Maybe Gmail's filters highly associate mentions of "Oracle" with spam? :/

I agree that with Tim that it's going to be easier to keep it in core if development is going to continue. Any suggestion that unbundling it would improve its support lifecycle should be well backend by Django+Oracle users themselves, which I take it you aren't Johannes.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d354b7a9-d116-41e9-9b4c-f8335931957f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Adam

[Dan Davis]

My employer is an Oracle shop.  I would dedicate myself to Oracle specific bugs to prevent removing Oracle from core.   That said, we'll probably be off Oracle and onto the cloud and Postgresql by 3.0.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM1LxFXXusWe-mJA55%2BqoK-9pr3-ZAK%2BV6Kw-igqNkaggg%40mail.gmail.com.

[Mariusz Felisiak]

Hi

     I don't agree that the Oracle back-end is poor implemented (I probably should not treat this personally 😀). It is as well maintained as any other back-end that is in the core. We don't have much more open tickets in the Oracle back-end then in others and IMO it is easier to maintain it in the core.

• Technical:
• Oracle does not support may features
• due to its lack of features, a lot of edge case handling to the base database backend which drives overall complexity

Just like SQLite or MySQL I don't think that we should leave only PostgreSQL in the core 😀.

• Development:
• Oracle does not run in the regular CI suite, in fact master is broken right now

I don't see any failures in djangoci.com. Maybe you use an unsupported version of Oracle?

Best

Mariusz

[Johannes Hoppe]

On Monday, November 26, 2018 at 8:27:02 AM UTC+1, Mariusz Felisiak wrote:

Hi

     I don't agree that the Oracle back-end is poor implemented (I probably should not treat this personally 😀). It is as well maintained as any other back-end that is in the core. We don't have much more open tickets in the Oracle back-end then in others and IMO it is easier to maintain it in the core.

Haha, sorry, I didn't mean to offend anyone. I hope you can see past my poor choice of works. I just noticed that a lot of oracle specific behavior is implemented in the base backend, where other backends like MySQL opt to override methods to add their db specific behavior. 

• Technical:
• Oracle does not support may features
• due to its lack of features, a lot of edge case handling to the base database backend which drives overall complexity

Just like SQLite or MySQL I don't think that we should leave only PostgreSQL in the core 😀.

They are actually a lot better maintained then Oracle, which leads me to believe it's not used that often. Besides both DBs you mentioned are open source. I don't mind putting in extra work for an open source database. For a private corp that makes 4bn in revenue... not so much. Maybe a separate backend project, would also see more support from Oracle.

[Carlton Gibson]

Hi Joe! 👋

On 26 Nov 2018, at 09:05, Johannes Hoppe <in...@johanneshoppe.com> wrote:

I don't mind putting in extra work for an open source database. For a private corp that makes 4bn in revenue... not so much. 

Is the issue “How to squeeze money out of Oracle?” — On that, did anyone try asking? 🙂

(I do feel that sentiment too.) 

C.

[Florian Apolloner]

Hi,

I personally agree with Mariusz here. Oracle might have it's own quirks, but the same could be said for any database. Taking my experience with the ORM into account I do not think that Oracle requires much more work (if at all) than any other database. I think in the end it does not matter whether one works around the limitations/features of Oracle or MySQL. All in all I think having Oracle is a good thing because databases like MSSQL and Informix/DB2 quite often behave similar to Oracle and just narrowing the core scope to MySQL/Pg/Sqlite might lead to a kind of tunnel vision.

Granted, Oracle might be a bit harder to install; but with the developer VMs and docker containers that argument doesn't really hold either imo.

Cheers,

Florian

[Johannes Hoppe]

There is not official container and the one you can build from their repo, didn't work for me. Furthermore you need to register with oracle and give them your Phone number, just to download the python library bindings. So it is somewhat harder than others ;)

Cheers,

Florian

[Johannes Hoppe]

Hahaha, yes kind of :P

If they become a corporate sponsor it shut up immediately ;)

[Jani Tiainen]

I've very successfully built docker images from Oracle official repo without any major problems.

Also since cx_Oracle 6 you can build db api bindings without oracle sdk libraries with simple pip install. Using those though requires client libs.

Not sure about registration process and is it needed for instant client or xe version.

Cheers,

Florian

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be871f78-76b7-4c52-8172-748248347b82%40googlegroups.com.

[charettes]

I haven't tried it out for Oracle yet but Tom Forbes' django-docker-box seems to make it a not-too-painful process[0]

Simon

[0] https://github.com/orf/django-docker-box#oracle

[Johannes Hoppe]

To quote the documentation: https://github.com/orf/django-docker-box/blob/85780dcc81d62a4c0c1142b45eb69e825d97b074/README.md#oracle

"As usual Oracle is a bit more complex to set up.” ;)

--
Johannes Hoppe

www.johanneshoppe.com

Want to chat? Let's get a coffee!
https://calendly.com/codingjoe/coffee

Lennéstr. 19
14469 Potsdam

USt-IdNr.: DE284754038

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/dg8BUVHKOo4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ee9fa331-1606-48fb-9362-e86cd9f3c9ae%40googlegroups.com.

[Tom Forbes]

Indeed, the initial setup of the database takes a horrendous amount of time (like 30 minutes to init an empty database!).

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d638aa8b-c10a-4c3c-b6d4-a687d3d9d979%40Spark.

[Dan Davis]

Related question - how would I search for Oracle specific issues.   I found this query:

https://code.djangoproject.com/query?status=assigned&status=new&keywords=~oracle&component=Database+layer+(models%2C+ORM)&col=id&col=summary&col=status&col=component&col=owner&col=type&col=version&desc=1&order=id

However, I'm not sure how much I can rely on the keywords.

[Tim Graham]

That's the query I would use. The 'oracle' keyword might not be assigned completely but you can scan through all the "Database layers" tickets fairly easily and add it to any that are missing.

[Dan Davis]

Another related question - https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/unit-tests/#testing-other-python-versions-and-database-backends provides some terse advice for running the unit tests with different backends.   Is that essentially what is happening with a test like https://djangoci.com/job/django-oracle-1.11/, or is it more specific?

[gg wwk]

Firstly, greetings everyone I'm new here but I am looking to learn and use Django more so...

I see the edge cases being a issue but I am for Oracle going out of core.  And in order to maintain and/or expand MySQL support  how is the the implementation of MariaDB going so far?

I quick glance revealed this much: https://github.com/django/django/search?q=mariadb&unscoped_q=mariadb

Thanks,

gwk

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5bfa1fe2-38b2-4086-bc57-f9badec93489%40googlegroups.com.