The current discussion about "Switch to database-level autocommit" (http://bit.ly/ZlVERI) reminded me of a past discussion regarding moving the database backends out of the core. I don't remember exactly where I heard the discussion, but I'd like to revive it. My proposal is to move all database backends out of Django, except for an sqlite reference implementation.
In recent years, I have been the main contributor to South's MSSQL and Oracle
backends. I am biased towards having MSSQL treated as an equal to the database
systems supported in core, but also towards support of backend-specific low-
level code in user apps. Also, I am a Linux user -- I use django-pyodbc for my
work on South, not django-mssql.
And where the dependency is weaker, being in core was not a
guarantee for fixes -- Django carried significant Oracle problems for quite
long, and they were just not deemed crucial.
I am worried that the result of such change will not be that MSSQL is treated
as well as Postgres is today, but that Oracle is treated as bad as MSSQL is
today.
I am obviously biased against postgres as my previous post indicated, but regardless of that I think that MSSQL should stay outside of core. No core-developer I know actually uses Windows as base OS and to my knowledge no one uses MSSQL. This would put the expected support for South below what we have for Oracle currently.
Full disclosure, I maintain django-mssql and am biased toward having all database backends treated as if they were 3rd party backends.
I am obviously biased against postgres as my previous post indicated, but regardless of that I think that MSSQL should stay outside of core. No core-developer I know actually uses Windows as base OS and to my knowledge no one uses MSSQL. This would put the expected support for South below what we have for Oracle currently.
- Less popular database backends may get left behind
but at work I'm restricted to corporate rules, MS SQL or Oracle, and Windows.
I have a strong feeling that by embracing Windows as a first-class platform for developing and running Django, we're enabling current and future users to easily integrate the framework in existent business infrastructure, which can only be seen as a good thing for the project.
Hi,
While I agree that moving database adapters out of core has some merit, I don't think that having sqlite as a reference implementation is a good idea: For one some features are somewhat hacky in sqlite (and people tend to copy from reference implementations, so it should be as clean as possible)
and it's lack of data validation makes it imo a nogo.
The reference implementation should imo also have strong support for GIS, which is somewhat okay on sqlite but quite hard to install. So if we were to do that I'd either vote for postgres or supporting postgres and sqlite inside of core (the later solely for fast tests).
Hi Shai,
On Tuesday, March 5, 2013 10:32:29 PM UTC+1, Shai Berger wrote:In recent years, I have been the main contributor to South's MSSQL and Oracle
backends. I am biased towards having MSSQL treated as an equal to the database
systems supported in core, but also towards support of backend-specific low-
level code in user apps. Also, I am a Linux user -- I use django-pyodbc for my
work on South, not django-mssql.
I am obviously biased against postgres as my previous post indicated, but regardless of that I think that MSSQL should stay outside of core. No core-developer I know actually uses Windows as base OS and to my knowledge no one uses MSSQL. This would put the expected support for South below what we have for Oracle currently.
And where the dependency is weaker, being in core was not a
guarantee for fixes -- Django carried significant Oracle problems for quite
long, and they were just not deemed crucial.
Now that we test on Oracle once again I am committed to have a full passing testsuite with 1.6 (whatever that means, since I don't use Oracle on a daily base I can't say what a running testsuite actually means for a project).
I am worried that the result of such change will not be that MSSQL is treated
as well as Postgres is today, but that Oracle is treated as bad as MSSQL is
today.
That might as well be true, if Oracle is outside of core I personally wouldn't put much effort in fixing stuff there (this obviously might change depending on whether I use Oracle at work or not).
If I may weigh in on the matter as an outsider, if we consider "The Django Project" as a "business", insofar as it aims to have as many users and be as ubiquitous as possible, there's considerable value in having MSSQL included in core. If it were up to me, I would develop on Linux using postgres and host everything on Heroku, but at work I'm restricted to corporate rules, MS SQL or Oracle, and Windows.
I have a strong feeling that by embracing Windows as a first-class platform for developing and running Django, we're enabling current and future users to easily integrate the framework in existent business infrastructure, which can only be seen as a good thing for the project.
The lack of data validation is definitely a nogo for production sites, but imo sqlite in production is also a nogo.
The reference implementation should imo also have strong support for GIS, which is somewhat okay on sqlite but quite hard to install. So if we were to do that I'd either vote for postgres or supporting postgres and sqlite inside of core (the later solely for fast tests).. I've never tried to install GIS for sqlite, but is the difficulty due to lack of documentation or just sheer number of steps?
On Wednesday, March 6, 2013 3:32:45 PM UTC+1, Michael Manfre wrote:The lack of data validation is definitely a nogo for production sites, but imo sqlite in production is also a nogo.
Right, but shipping Django with a non production db might send interesting signals to endusers ;)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Here's something I've been thinking about:As a rule, assuming that backends are not bug-riddled and do not have needlessly shoddy performance, nearly all of the value of an ORM is in the set of frontend features it exposes to application developers. Given a certain frontend API, then the only value in iterating the backend independently from the frontend is to fix uncaught bugs, or to improve performance, there's no value in adding features.Also as a rule, adding features to a frontend independently from the backend has little value. Without an implementation in whatever backend you are on, it only serves as a guide for future backend development, so you might as well push for backends to catch up in the same release cycle. (Or the feature doesn't require backend support, but then it doesn't hurt to reversion the backend anyways).So basically, there's not much value in independently versioning and maintaining frontends and backends to the ORM. This is in contrast to, say, localflavor, where an improvement in Mongolian localization can immediately help every Mongolian website in every Django version. So this primary motivation for independent maintenance is not a factor in the ORM. Hence I think the core team should include as many backends as it can handle in core (where "handle" means test that they function properly for each release). Oracle had been slipping, but from what I understand it's now in the CI server and back to passing most tests.So I see no reason to split off backends unless the maintenance burden is such that they are chronically broken in every Django release. And every reason to add MSSQL unless the maintenance burden is too high.
I completely agree with Jacob's analysis of the status quo, and I agree largely with his position on having MSSQL in the core.I'd have no problem seeing MSSQL in the core - it's at least as high profile as Oracle, and although the demand for a MSSQL backend is restricted to a particular subset of the community, there is at least *some* demand for it to exist. Having MSSQL in core would allow us to hold our head high and say we support Microsoft platforms. Microsoft is even a member of the DSF at this point, so they're at least notionally interested in Django as a platform.
The maintenance burden is the problem. Historically, we've already had to *remove* a MSSQL backend because of bit rot (see [1]) - this isn't a pattern I want to repeat in the future. I don't want to add a backend to core unless I'm *certain* that it will see long term maintenance.
There is, however, a possible middle ground, following the example set by Flask: we introduce to Django a list of "officially recognised" extensions. These extensions are still maintained as external projects, but the core team promise to include these projects as part of the testing regimen for a release. If the MSSQL backend were recognised like this, we would run the full Django test suite against the MSSQL backend, and if that testing revealed a bug in the test suite which was identified as being due to a change in Django's codebase, that bug would be a release blocker (Bugs in the backend itself would still be the backend's responsibility, and not release blocking on Django)
django-mssql is actively maintained and will be for at least the next few years because it's used for my employer's production site that is critical to business operations. The backend also supports stored procedures almost to the same extent as a Windows business application. The only real downside to the backend is that it only works on Windows, reducing its usefulness to a much smaller, but growing, minority of Django users.
What's the state of http://code.google.com/p/pymssql/ ?
If we have MSSQL in core I'd really like to be able to talk with it from a Linux machine too, it would also make testing easier since we'd just need a VBox with MSSQL ;) Supporting a commercially available product but requiring to run even more commercial software seems like counter intuitive to me.
If we have MSSQL in core I'd really like to be able to talk with it from a Linux machine too, it would also make testing easier since we'd just need a VBox with MSSQL ;) Supporting a commercially available product but requiring to run even more commercial software seems like counter intuitive to me.
Django needs a VBox running Windows for testing, regardless if MSSQL is in the core.
from thread "Moving database backends out of the core"
On Mar 7, 2013, at 5:13 PM, Russell Keith-Magee <rus...@keith-magee.com> wrote:
> There is, however, a possible middle ground, following the example set by Flask: we introduce to Django a list of "officially recognised" extensions. These extensions are still maintained as external projects, but the core team promise to include these projects as part of the testing regimen for a release. If the MSSQL backend were recognised like this, we would run the full Django test suite against the MSSQL backend, and if that testing revealed a bug in the test suite which was identified as being due to a change in Django's codebase, that bug would be a release blocker (Bugs in the backend itself would still be the backend's responsibility, and not release blocking on Django)
>
> Looking outside database backends, this could also apply to high-profile projects like haystack, django-registration, and so on. This would also serve to highlight projects in our community that are 'defacto standards', or good examples of reusability, without needing to expand the core team or the size of contrib, and show that the core project is explicitly interested in the broader ecosystem.
I think this is a great idea. The backend testing makes sense to me (run Django's tests). How does testing sanctioned apps work, though? You run _their_ tests, and if they are caused by bugs in Django then they are release blockers?
I really like the idea, and would like to help if I can. I imagine the leg work will be mostly in setting up the testing infrastructure, although the difficult part will be setting guidelines for and choosing apps that should be in this group.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/978f91f8-709a-48dc-b2e2-b940e19a9f7f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
On your point about non-standard backends, maybe we should focus on making it easier to add third party backends and standardize some of the internals? We could treat the backend as external (i.e no special hacks for them) but keep them in the Django package. If we make the interfaces a bit more abstract it would be a lot easier for these backends to exist without hacks perhaps.
I think this would only work if most database specific backends were maintained by the djangoproject itself, allowing for integration tests that test compatibility.To my mind, a strong ORM to backend API is a great thing, but we also need stronger backend integration tests.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/O-g06EM6XMM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/23759c70-4148-4ac7-8dfa-ca97f1dae97d%40googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/O-g06EM6XMM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20181128013339.2901e7a7.shai%40platonix.com.