Proposing the removal of Oracle from the Django supported backend databases

[Paolo Melchiorre]

Hi all,
I wanted to share the frustration of seeing yet another great new ORM
feature blocked due to Oracle compatibility:
https://github.com/django/django/pull/16417

In the past, I too have had to put a lot of effort trying to make a PR
compatible with Oracle, making the overall contributing experience
much less pleasant and holding me back from contributing, especially
in the early days.

Over the last few months, I've tried to encourage newcomers and young
users to contribute to Django and they almost always ran into the need
to provide compatibility to Oracle, so much so that they eventually
gave up contributing.

I stress that I am absolutely not criticizing the contributors (very
few) in the community who help overcome the difficulties with Oracle.

The point is that I think Oracle is a historical anomaly among the
database backends supported by Django because it is the only one that
is not Open Source, it has irrelevant usage numbers (see Django
Developers Survey 2022 Results
https://lp.jetbrains.com/django-developer-survey-2022/#horizontal-bar-chart-862)
and the company that earns from it does not contribute in any way to
its maintenance or support (see Carlton Gibson keynote at PyCon Italia
2023 https://youtu.be/AHjnGtaWDjU?t=836)

To add to all this we consider that developing for Oracle is much more
difficult than for the other Open Source databases supported by Django
and above all the new contributors to the ORM have a frustrating
experience and therefore they are less and less.

I, therefore, suggest that we start a discussion on removing Oracle
from supported databases.

Ciao,
Paolo
--
Paolo Melchiorre

https://www.paulox.net

[Christopher Jones]

We (Oracle0 would like to improve Oracle support in Django and intend to contribute. However the PR to add support for the latest Oracle driver is blocked because debugging information about a test failure isn't available, and the failure isn't seen by us. If that can be overcome, then we can make headway.

Chris 

[Jörg Breitbart]

+1 from my side for removing oracle support from django itself.

If very low usage + high maintenance burden does not qualify for feature
removal, idk what does - maybe not being supportive at all. Oh wait...

In the end such low usage numbers are prolly better served by a 3rd
party package.

Regards,
Jerch

[Tom Carrick]

I don't have a strong opinion in either direction here.

I would like to point out that every database we support is a maintenance burden, however. It's an exaggeration that Oracle alone causes problems but other backends don't. For example when I was working on collations, I ran into issues with SQLite's lack of good ways to inspect the database. I ran into problems with MySQL's lacking documentation on collations and its seeming semi-support for it. I ran into problems with Postgres as you need to create some of its more powerful collations in SQL first. I also ran into issues with Oracle, but they were actually easier to overcome than some of the others.

I also gave up on an attempt at an EnumField for models because Postgres handles them in a totally different way from other databases and it ended up raising so many questions that I just gave up.

I would say, however, that Oracle usually has perhaps more than its share of problems. And of course Oracle's lack of financial support gives impetus to care less about it, as we seemingly do for MS SQL server, which probably sees more use overall than Oracle. Bur Oracle is already established in the codebase.

It also shouldn't be forgotten that while 2% as a relative number is quite small, the absolute number of people it will affect is quite large, and I certainly wouldn't call it "irrelevant".

I wouldn't be against moving it to jazzband, say, but without people stepping up to maintain it (and there is an impetus for this when it's in Django's codebase), it may just end up abandoned.

Tom

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/0380eceb-c0bc-1a81-411b-b8e0e8588c83%40netzkolchose.de.

[Mariusz Felisiak]

Hi Paolo,

    I'm quite surprised that you've started a new thread for something that was already discussed, you could always add a comment to the existing thread e.g. https://groups.google.com/g/django-developers/c/dg8BUVHKOo4/m/5uFVmdWCAwAJ

> I wanted to share the frustration of seeing yet another great new ORM feature blocked due to Oracle compatibility: https://github.com/django/django/pull/16417

    I'm not sure how you reached this conclusion. This is not blocked due because of Oracle compatibility. I will review it and try to merge it before the Django 5.0 feature freeze. You have to be patient, it has nothing to do with Oracle. I just need more time as it's complicated feature, e.g. it took me 2 weeks to review and merge PR with database defaults. I do my best to avoid regressions and provide stable feature, unfortunately such pedantic approach takes time.

> Over the last few months, I've tried to encourage newcomers and young  users to contribute to Django and they almost always ran into the need to provide compatibility to Oracle, so much so that they eventually  gave up contributing.

    Really? Django is not only the ORM. It is easy to demonize Oracle. I'm working with contributors on daily basis, and  don't remember anyone who would resign because we have builtin Oracle backend. We don't have much more open tickets in the Oracle backend then in others. The number of unsupported features is similar to SQLite or MySQL.

> The point is that I think Oracle is a historical anomaly among the database backends supported by Django because it is the only one that is not Open Source, it has irrelevant usage numbers

It's not an anomaly. Oracle support was a conscious decision, keeping the ORM features Oracle-compatible is a good battlefield, that helps keeping the ORM friendly for 3rd-party database backends as we have more feature flags and hooks for custom behaviors.

> ... and the company that earns from it does not contribute in any way to its maintenance or support

Should be also drop support for Windows for exactly the same reason? (rhetorical question)

> I, therefore, suggest that we start a discussion on removing Oracle from supported databases.

This was already discussed. I'm still strongly against it.

Best,
Mariusz

[Carsten Fuchs]

Hello,

when I got started with Django more than 10 years ago, I had inherited a legacy project with an Oracle database for porting from PHP to Python. It might also have worked out well if Oracle support had been a 3rd party package, but I can say for sure that Oracle being a core feature of Django was a huge help in getting me started.

I'm aware that this is a weak argument from unfortunately a code non-contributor (or indirect contributor at best) who is not shouldering the work and not even using the Oracle backend any more (we migrated to MySQL). But still, I would like to express my opinion that having Oracle in Django core is a worthwhile asset.

Best regards,
Carsten


Am 03.08.23 um 10:25 schrieb Paolo Melchiorre:

[Paolo Melchiorre]

Hi Mariusz,

On Thu, Aug 3, 2023 at 8:47 PM Mariusz Felisiak
<felisiak...@gmail.com> wrote:
> I'm quite surprised that you've started a new thread for something that was already discussed, you could always add a comment to the existing thread e.g. https://groups.google.com/g/django-developers/c/dg8BUVHKOo4/m/5uFVmdWCAwAJ

I was not aware of this thread, thanks for linking it.

>> I wanted to share the frustration of seeing yet another great new ORM feature blocked due to Oracle compatibility: https://github.com/django/django/pull/16417
> I'm not sure how you reached this conclusion. This is not blocked due because of Oracle compatibility. I will review it and try to merge it before the Django 5.0 feature freeze. You have to be patient, it has nothing to do with Oracle

It was not my intention to push you into reviewing this PR given how
much you already do. But it seems to me that without Oracle
compatibility this functionality would have been ready for a long
time. However, this reflection of mine was born after seeing the
difficulty of the original contributor to add support to Oracle, at
the same time I also watch again Carlton's keynote at PyCon Italia
2023 because the videos have recently been published. Lastly, while
volunteering at the DSF booth at EuroPython 2023, I happened to meet
an Oracle Django developer for the first time, who admitted that he
had never contributed to the Oracle backend itself.

>> Over the last few months, I've tried to encourage newcomers and young users to contribute to Django and they almost always ran into the need to provide compatibility to Oracle, so much so that they eventually gave up contributing.
> Really? Django is not only the ORM. It is easy to demonize Oracle. I'm working with contributors on daily basis, and don't remember anyone who would resign because we have builtin Oracle backend. We don't have much more open tickets in the Oracle backend then in others. The number of unsupported features is similar to SQLite or MySQL.

In my personal experience, the people I collaborate with at work or
meet in the local communities where I go to talk about Django all have
experience with Open Source databases and none have ever worked with
Oracle, which makes it very difficult for them to try to contribute in
Django's ORM why should they ensure compatibility for this DB without
ever having seen it.

>> The point is that I think Oracle is a historical anomaly among the database backends supported by Django because it is the only one that is not Open Source, it has irrelevant usage numbers
> It's not an anomaly. Oracle support was a conscious decision, keeping the ORM features Oracle-compatible is a good battlefield, that helps keeping the ORM friendly for 3rd-party database backends as we have more feature flags and hooks for custom behaviors.

I think this would be the same if the Oracle backend was a third-party
package, perhaps maintained by the same company

>> ... and the company that earns from it does not contribute in any way to its maintenance or support
> Should be also drop support for Windows for exactly the same reason? (rhetorical question)

I don't know, I haven't used Windows since 2000 :-)

Anyway, tornado to talk about Database, SQL Server is a third-party package

>> I, therefore, suggest that we start a discussion on removing Oracle from supported databases.
> This was already discussed. I'm still strongly against it.

Thank you for your point of view, and also for all the extra work you
do to help Django contributors who are having trouble making their
functionality compatible with Oracle.

I reiterate that my only intention was to understand if I were the
only one to have this point of view without wanting to criticize
anyone's work or past choices, things simply change in the field of
information technology and choices could be reviewed after years.

[Carlton Gibson]

Thanks for raising this Paolo. It's good to discuss, even if we maintain the status quo.

I think this will come to a head when Mariusz steps away. It's been him that's kept it going the last few years, and I'm not at all sure that we have another currently active contributor that's an Oracle expert. (Perhaps we do... 🤔)

It's good to see Chris' comment that Oracle wish to contribute themselves, but if that doesn't materialise, we're going to need to claim *force majeure*.

Q: Are we sure we can maintain Oracle without Mariusz? What does that look like? If not, we need to make plans for something else. 


FWIW: If possible I'd like to keep Oracle support if we can — it's good for the ORM. 

(If we could see a way of it being maintainable, I'd have us add SQL Server in core too — but I don't think that is feasible given the state of play today).

The difference with Windows (of course) is that we have lots of contributors using it.

Kind Regards,

Carlton

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAKFO%2Bx6VjKOBpUtujXh-SJ3wjfO3sfLz4ekMRukiBNfy2W-FMw%40mail.gmail.com.

[Lily Foote]

I've had to deal with adding Oracle support to the features I've worked on recently and with Mariusz's pointer to the right bit of the wiki (https://code.djangoproject.com/wiki/OracleTestSetup) I was able to get a working local development setup. I think once you have this local dev setup, working on Oracle support isn't much different to MySQL or Postgres.

One thing I think might be valuable for improving the local developer experience would be adding test settings files for Oracle (and MySQL and Postgres) to the repository with instructions for those extra steps a contributor needs to do to use them (i.e. how to configure each database). I think experienced contributors tend to have a working setup, but there's a bit of a hurdle for newer contributors.

Adding this information to the main repository would I hope also mean it is better maintained - I usually forget the wiki exists and I think it has worse bitrot than other resources.

Lily

------- Original Message -------

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAJwKpySoFnSWTz65ahf7QhObKBVOeYyDrGPXntaeNU25kqubXw%40mail.gmail.com.

[David Sanders]

I posted a thread a while back about the latest version of Oracle 23c which appears to add support for a few things that would fix some of the workarounds we have in the ORM (eg boolean expressions). I know dropping 19c is a long way off but at least the future is looking brighter 🌟

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAJwKpySoFnSWTz65ahf7QhObKBVOeYyDrGPXntaeNU25kqubXw%40mail.gmail.com.