Improving MSSQL and Azure SQL support on Django

[Meet Bhagdev]

Hi Django Committers,

 

My name is Meet Bhagdev, I work in the Database Systems engineering team at Microsoft in Seattle, WA. My focus is the APIs used to connect to and use Azure SQL Database and SQL Server (MSSQL). Example APIs are ODBC, JDBC, ADO.NET, etc.

 

We’d love for Django users to have a solid option to use MSSQL and Azure SQL if they wish, and it would be great to make this option a reality. Our goal is to partner with committers like yourself to bring first-class support for MSSQL to Django. We want to bring the benefits of Django to millions of existing MSSQL and Azure SQL customers as well as folks that would like to evaluate us as a database option for Django.

 

I have been in touch with Tim Graham and Russell Keith-Magee related to us building such support. We are prepared to make the engineering investment required on our end and to work with the Django community to make this happen, the right way. We are willing to do all the heavy lifting, but we do need your help and guidance.

 

At Microsoft we have an existing program for bringing partners and eco-system developers to Seattle, WA, all expenses paid, and we’d like to extend this to the Django committers. This would be a great way to get started. We would love to get a group of you here during our October workshop. The idea is to get developers from both sides to meet and learn from each other—Django and Microsoft.

 

Please reach out me at me...@microsoft.com or give me a call on +1 425 722 5342 if you would like to attend. Would love to get you more details.

 

Sincerely,

Meet Bhagdev | Linkedin | Github

 

P.S. This is not a sales pitch; we are excited about bringing the benefits of Django to our customers.:)

 

[Aymeric Augustin]

Hello Meet,

On 21 août 2015, at 19:39, Meet Bhagdev <meetb...@gmail.com> wrote:
  

We’d love for Django users to have a solid option to use MSSQL and Azure SQL if they wish, and it would be great to make this option a reality.

Even though this isn’t what you’re asking for :-) I’ll take this opportunity to summarize the landscape for combining Django and MSSQL / Azure SQL, as far as I understand it.

There’s a good option for MSSQL or Azure SQL users running Django on Windows: https://bitbucket.org/Manfre/django-mssql. (Perhaps it could be made even better, but from my perspective, it’s solid.)

There’ve been talks of making it more official. But no consensus has been found yet. The main difficulty is that very few Django committers use Windows as a primary OS, perhaps two out of fifty. Most of us don’t even own a Windows license. We face a similar issue with our continuous integration. Currently it’s running on Linux. We lack the skills and money to run Windows and SQL Server. (Sponsoring in the form of Azure SQL credits for the purpose of testing Django may solve parts of this problem, if using a remote database isn’t too slow for test suite’s workload.)

There isn’t such a clear story for running Django on Linux. This led me to write https://github.com/aaugustin/django-pymssql. Alternatives include https://github.com/denisenkom/django-sqlserver and https://github.com/lionheart/django-pyodbc.

I can’t say django-pymssql is solid. But it works and it shows that django-mssql could be made cross-platform with limited effort, subject to the quality of the underlying libraries: https://github.com/pymssql/pymssql and https://github.com/FreeTDS/freetds. However my interest has faded since I left the company where I had this use case. Furthermore testing is very painful. I'm running SQL Server Express in a VM. Tests are about 20 times slower than with PostgreSQL or MySQL. As a consequence Django’s test suite took about 2 hours. 

I appreciate the invitation. Unfortunately I live a bit too far to make the trip conveniently. I’m still interested in making Django work out of the box with SQL Server like it does with Oracle. (That’s for historical reasons: some people gathered and made it happen). I believe interoperability with SQL Server is key for using Django in the medium-sized companies where SQL Server is the obvious choice of database server.

I hope this helps. Thanks for reaching out!

-- 
Aymeric.

[Tim Graham]

I agree it would be great to get some help running the Django tests on Windows. I run them in a local virtual machine every so often, but I would love to be able to delegate fixing Windows issues. Meet, can your team provide ongoing help with fixing Windows-specific issues in Django, even if they aren't related to MSSQL/Azure? That is something where we've had a hard time finding volunteers to help with and is obviously important if we want to claim 1st-class support for Windows.

I don't have any experience with MSSQL/Azure, but I could probably attend the October 13-15 event at Microsoft if you think my participation would be valuable and if the DSF were to approve the time commitment as part of the fellow duties.

[Shai Berger]

On Saturday 22 August 2015 13:28:31 Aymeric Augustin wrote:

>

> There isn’t such a clear story for running Django on Linux. This led me to

> write https://github.com/aaugustin/django-pymssql. Alternatives include

> https://github.com/denisenkom/django-sqlserver and

> https://github.com/lionheart/django-pyodbc.

 

There's also django-pyodbc-azure, a fork of django-pyodbc (actually, the current django-pyodbc is also a fork of the original project, which has been discontinued). I took the liberty to forward the message to that project.

 

Shai.

[Tim Graham]

I guess the first step is to identify which third-party backend(s) we'll target to adopt officially (or at least borrow from heavily). For example, will we need separate backends for MSSQL and Azure? (Knowing nothing about the landscape myself, this question could be nonsensical.) Is this discussion something that should happen before the October summit? It seems to me the face-to-face time will likely be more productive if we have some of the high-level details ironed out.

By the way, is videoconferencing an option for Django developers interested in participating in the discussion at that time but unable to travel to Seattle?

[Meet Bhagdev]

Absolutely agree with Tim here. We need to start exploring all the cool open sourced third party adapters. Borrowing/adopting them definitely seems like the way to go about things instead of re inventing the wheel. I tried doing some research and came across the following: 

1. Django-mssql

2. Django-pymssql

3. Django-pyodbc-azure

4. Django-pyodbc

Am I missing any? 

Best,

Meet

[Russell Keith-Magee]

On Sat, Aug 22, 2015 at 8:28 PM, Tim Graham <timog...@gmail.com> wrote:

I agree it would be great to get some help running the Django tests on Windows. I run them in a local virtual machine every so often, but I would love to be able to delegate fixing Windows issues. Meet, can your team provide ongoing help with fixing Windows-specific issues in Django, even if they aren't related to MSSQL/Azure? That is something where we've had a hard time finding volunteers to help with and is obviously important if we want to claim 1st-class support for Windows.

I don't have any experience with MSSQL/Azure, but I could probably attend the October 13-15 event at Microsoft if you think my participation would be valuable and if the DSF were to approve the time commitment as part of the fellow duties.

I don't see why this would be a problem - I'll run it past the fellowship committee and get their OK.

Russ %-)

[Meet Bhagdev]

Thanks Russ!

[Russell Keith-Magee]

Hi Meet,

The Fellowship committee has just approved this, so go ahead and plan whatever you need to plan.

Yours,

Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be75654f-619f-4d59-ac0d-24d50de3d6f9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tim Graham]

By "we" do you mean the engineers at Microsoft who will be working on this? Will they be doing any planning for this before the meet up in October? I'd like to know more specifics about the agenda and goals for the workshop.

Also, I wanted to make sure my question about a videoconferencing option for Django developers interested in participating in the discussion at that time but unable to travel to Seattle didn't get lost.

[Meet Bhagdev]

Hi Tim,

Thanks for bringing it up. Yes, we will have developers who will engage with Django developers (like yourself) during the workshop. 

The goals of the October workshop are to:

1)      Get to know each other and begin building a relationship

2)      Get in a room with Microsoft developers and discuss the current landscape 

3)      Work on half day coding sprint(s) with Microsoft developers to get started with contributions

4)      Establish a plan for how Microsoft can best contribute to Django, and ensure we have great integration between Azure SQL and MSSQL and Django by maintaining our contributions

The key takeaway is that we want to contribute to existing solutions to improve the Django and MSSQL/Azure SQL story. To do so we want to understand the current landscape, the gaps, and the next steps to make this happen (the right way).

We are currently in the planning stages and would love to get feedback. What do you think about the goals mentioned above? Is there anything you like to add/remove?

We can definitely make attending via Skype an option for attendees unable to make it in person. 

Best,

Meet

[Tim Graham]

To arrive at the best solution, I think a lot of discussion needs to happen on this mailing list before October. As for me, besides the name of some existing packages that offer SQL Server/Azure support, I know very little about the current landscape so I really wouldn't have anything to offer in a discussion, but there are others on this list who definitely do.

The existing Microsoft database backends have come out of specific developer needs and I think it would maximize success if your team were personally invested  in the backend by building a Django application backed by SQL Server/Azure. If I were a Microsoft engineer responsible for this project, I would start by building a small Django application to solve some problem that my team has. Then I would test it out with all the third-party backends for Microsoft databases previously mentioned to get a feel for how they work. I should learn enough to at least participate in a discussion with other Django developers about the direction of this project. Ideally, I would be able to learn enough to write up a Django Enhancement Proposal [1] that summarizes the landscape and describes what an "official solution" should look like.

If you are relying on members of the Django team to do all this upfront work, then please tell us so we can see if someone will do it. If you are expecting to get all this done in a couple days in Seattle, this doesn't seem feasible to me. However, if we have a working proposal by mid-September that the community can review and give feedback on, then I think our time together in October could be productive. I don't know anything about the background of your team, but it seems pretty unlikely that any sort of coding sprint would be useful unless the work and learning I described happens well before the workshop.

[1] https://github.com/django/deps/

[Russell Keith-Magee]

Hi Tim, Meet

I know this is very late to be mentioning, but one idea that worth
raising: DjangoCon US is next week (in Austin). Thursday and Friday
are coding sprints, where there will be many people (including a good
chunk of the core team) looking for projects to hack on.

Microsoft is already a sponsor of DjangoCon itself; if we can get some
engineers from the Azure team (assuming there aren't already some
coming), we can start some of this prep work.

Yours,
Russ Magee %-)

> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/9a79aa2d-ad82-4905-8bcc-835911c21103%40googlegroups.com.

[Meet Bhagdev]

Hi Russ,Tim

That is a great idea. I am going to try to engage with the team at Microsoft who is in charge of the sponsorship. I plan on attending the conference so I will see some of you there. 

@Tim: I agree that a lot of ground work needs to be done in terms of testing out the current third party adapters and deciding which one/ones should we adopt and extend. I know Michael Manfre has a lot of expertise here. I will do my due diligence and test out the existing solutions but if Michael could share some of his findings, that would be great. If anyone else also wants to take a stab at testing and using the existing adapters for a sample app, that would be great as well. I will try to share my learnings soon.

Here are the ones I am going to test, am I missing any?

1. Django-mssql

2. Django-pymssql

3. Django-pyodbc-azure

4. Django-pyodbc

Thanks,

Meet

[Tim Graham]

Hi Meet,

Will you be the primary contributor/maintainer from Microsoft of this project or will their be other engineers at Microsoft involved? I'd like to understand the bus factor on your side and how much time resources you plan to provide.

To give you an idea of how our community works when it comes to decision making and implementing large features, I thought you might find Aymeric's log of his work to add multiple template engines support to Django helpful: https://myks.org/en/multiple-template-engines-for-django/.

[Ramiro Morales]

Hi all,

On Fri, Aug 21, 2015 at 2:39 PM, Meet Bhagdev <meetb...@gmail.com> wrote:
> Hi Django Committers,
>
>
>
> My name is Meet Bhagdev, I work in the Database Systems engineering team at
> Microsoft in Seattle, WA. My focus is the APIs used to connect to and use
> Azure SQL Database and SQL Server (MSSQL). Example APIs are ODBC, JDBC,
> ADO.NET, etc.

(sorry for possibly repeating things folks have already posted to this thread)

Meet: Is this list of APIs the final and full one?

Because there is work being done on a different stack, the one formed by :

* FreeTDS (http://www.freetds.org/ GPL licensed) which implements the
wire-level TDS protocol.
* pymssql (http://pymssql.org LGPL licensed) -- Python bindings for
FreeTDS which implements the Python DB-API 2.0. I'm part of the team
maintaining it
* django-pymssql (https://github.com/aaugustin/django-pymssql ,
MIT-licensed) which was created by Aymeric Augustin and depends on
pymssql plus ...
* django-mssql (https://django-mssql.readthedocs.org/en/latest/ ,
MIT-licensed) which was created by Michael Manfre

Michael, Aymeric and me are Django development team members.

Personally, I've been working on stabilizing the lower layers by
following a "yak shaving" non-strategy:

We (the pymssql team) realised the official pymssql Windows binaries
(in particular the FreeTDS libraries) we published when releasing
pymssql 2.1.1 don't link in a SSL implementation and so they aren't
usable to actually connect to Azure even if the pymssql code iself has
such ability.

This, plus the fact that the manual process of creating the actual
matrix of Windows deliverables is a bit tedious led me to try using
the AppVeyor.com hosted Windows CI platform (free for open source
projects) to test and actually build the binaries. The work in
progress on this can be seen at
https://github.com/ramiro/pymssql/tree/appveyor and
https://ci.appveyor.com/project/ramiro/pymssql

(When working on pymssql 2.1.1 at some point I created a free Azure
account with my credit card, tested (on Linux) the pymssql
implementation of connection changes needed to get it to work against
Azure's SQL Server and cancelled it before it started billing.)

So, this led me to start contributing to FreeTDS so to get it back to
build cleanly on Windows (work partially included with the 0.95
release back in June) and to also get it built/tested using
AppVeyor.com using the experience gained with pymssql. This has been
already merged in the current FreeTDS development code and allows the
maintainer and contributors to work without access to Windows/SQL
Server licenses (my case).

- https://ci.appveyor.com/project/freetds/freetds
- https://github.com/FreeTDS/freetds/commit/3db5caa48f281f3558d4031cb5a0f0d8e8eef28c

I know MS dropped support of the DB-Library (and hence the API it
provides, of which FreeTDS is a open source implementation) back in
the SQL Server 2005 times. That's why I ask if this stack has any
chance of getting some support from Microsoft.

Personal motivation for this is simply to get Django (running on
Linux) + SQL Server to be a viable choice, even when I currently have
no actual need of this. I was one of the two developers behind the
original django-pyodbc project which had reached "almost full Django
test suite passing" status back in the Django 1.0 & 1.1 (2008-2009)
times, but abandoned it when discoverd (by logging the traffic with
SQL Servr tools) that the combination of pyodbc + Linux ODBC stack
meant the queries were sent twice to the DB server, see
https://code.google.com/p/django-pyodbc/issues/detail?id=16 . This is
also why I started considering a FreeTDS-based solution a better
technical choice.

I'm posting a message to the FreeTDS mailing list later today pointing
to Meet's post which opened this thread.

Regards,

--
Ramiro Morales
@ramiromorales

[Meet Bhagdev]

Hi Tim,

I will definitely be driving the initial planing and will help lay out the ground work. There will be other engineer(s) who will work on contributing to third party adapter(s) and the Django project(eventualy). 

But first, we need to figure out the ask and the nature of work required. The goal of our October coding camp workshop is to figure that out the ask, by bringing developers from the Django community like yourself, and the developers from my team, to come together and better understand the technical details. 

Once we have a clear understanding of the existing solutions and the amount of work required on improving the Django-MSSQL/Azure SQL story we will be able to commit developer(s) that will contribute and do all the heavy lifting.

Also thanks for the share of Aymeric's log of his work on the template engine. I absolutely agree on having something similar for the work we are planning on doing. Once we have a clear idea I will start working on something similar. A final version of this may only be possible after the October event.

Best,

Meet

[Meet Bhagdev]

On Wednesday, September 2, 2015 at 3:05:23 PM UTC-7, Ramiro Morales wrote:

Hi all,

On Fri, Aug 21, 2015 at 2:39 PM, Meet Bhagdev <meetb...@gmail.com> wrote:
> Hi Django Committers,
>
>
>
> My name is Meet Bhagdev, I work in the Database Systems engineering team at
> Microsoft in Seattle, WA. My focus is the APIs used to connect to and use
> Azure SQL Database and SQL Server (MSSQL). Example APIs are ODBC, JDBC,
> ADO.NET, etc.

(sorry for possibly repeating things folks have already posted to this thread)

Meet: Is this list of APIs the final and full one?

 

This is definitely not a final and full one. We have actually started endorsing pymssql and FreeTDS to our Linux and Mac customers who want to use Azure SQL. 

    We have also created documentation that lets customers use pymssql on 

    1. Windows: https://azure.microsoft.com/en-us/documentation/articles/sql-database-develop-python-simple-windows/

    2. Linux: https://azure.microsoft.com/en-us/documentation/articles/sql-database-develop-python-simple-ubuntu-linux/

    3. Mac: https://azure.microsoft.com/en-us/documentation/articles/sql-database-develop-python-simple-mac-osx/

 

 Also a big thank you from everyone on our team here for working on FreeTDS and pymssql. I definitely makes our customers lives easier

 

Because there is work being done on a different stack, the one formed by :

* FreeTDS (http://www.freetds.org/ GPL licensed) which implements the
wire-level TDS protocol.
* pymssql (http://pymssql.org LGPL licensed) -- Python bindings for
FreeTDS which implements the Python DB-API 2.0. I'm part of the team
maintaining it
* django-pymssql (https://github.com/aaugustin/django-pymssql ,
MIT-licensed) which was created by Aymeric Augustin and depends on
pymssql plus ...
* django-mssql (https://django-mssql.readthedocs.org/en/latest/ ,
MIT-licensed) which was created by Michael Manfre

I am currently exploring django-pymssql, django-mssql, and django-pyodbc-azure to see which adapter(s) can we adopt to provide Django and MSSQL/Azure SQL support. Do have any opinions as to which one(s) should we contribute to eventually? 

 

Michael, Aymeric and me are Django development team members.

Personally, I've been working on stabilizing the lower layers by
following a "yak shaving" non-strategy:

We (the pymssql team) realised the official pymssql Windows binaries
(in particular the FreeTDS libraries) we published when releasing
pymssql 2.1.1 don't link in a SSL implementation and so they aren't
usable to actually connect to Azure even if the pymssql code iself has
such ability.

 I looked around and it seems like this problem was fixed on this page by Christoph Gohlke from University of California, Irvine: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pymssql. I downloaded pymssql from here and it seemed like it links the SSL implementations. I also documented the procedure here: https://azure.microsoft.com/en-us/documentation/articles/sql-database-develop-python-simple-windows/

This, plus the fact that the manual process of creating the actual
matrix of Windows deliverables is a bit tedious led me to try using
the AppVeyor.com hosted Windows CI platform (free for open source
projects) to test and actually build the binaries. The work in
progress on this can be seen at
https://github.com/ramiro/pymssql/tree/appveyor and
https://ci.appveyor.com/project/ramiro/pymssql

(When working on pymssql 2.1.1 at some point I created a free Azure
account with my credit card, tested (on Linux) the pymssql
implementation of connection changes needed to get it to work against
Azure's SQL Server and cancelled it before it started billing.)

Do you still need Azure credits to run Azure SQL for free, I can try to get some free credits for you and the FreeTDS and pymssql team depending on the need. 

 

So, this led me to start contributing to FreeTDS so to get it back to
build cleanly on Windows (work partially included with the 0.95
release back in June) and to also get it built/tested using
AppVeyor.com using the experience gained with pymssql. This has been
already merged in the current FreeTDS development code and allows the
maintainer and contributors to work without access to Windows/SQL
Server licenses (my case).

- https://ci.appveyor.com/project/freetds/freetds
- https://github.com/FreeTDS/freetds/commit/3db5caa48f281f3558d4031cb5a0f0d8e8eef28c 

 

  This is great to hear. Getting it on Appveyor definitely makes it easy to build on Windows. 

 

I know MS dropped support of the DB-Library (and hence the API it
provides, of which FreeTDS is a open source implementation) back in
the SQL Server 2005 times. That's why I ask if this stack has any
chance of getting some  support from Microsoft.

We are very much looking at providing support(as you can tell) from our documentation mentioned above. We are currently in the prioritization stage and would love to discuss some of the things you need support for. As for now, we feel Django is a good place to start, as there is customer demand and we are falling behind. Would have a document which highlights the todo feature list where you need help on? I can pass it on to my team to get some responses so that we can get the ball rolling.

 

Personal motivation for this is simply to get Django (running on
Linux) + SQL Server to be a viable choice, even when I currently have
no actual need of this. I was one of the two developers behind the
original django-pyodbc project which had reached "almost full Django
test suite passing" status back in the Django 1.0 & 1.1 (2008-2009)
times, but abandoned it when discoverd (by logging the traffic with
SQL Servr tools) that the combination of pyodbc + Linux ODBC stack
meant the queries were sent twice to the DB server, see
https://code.google.com/p/django-pyodbc/issues/detail?id=16 . This is
also why I started considering a FreeTDS-based solution a better
technical choice.

 

That is a great find. I actually was not aware of that. The good news for the Linux side of the story here is that we are releasing a new Linux ODBC driver in the next 2 months. It would be great if we can test this out and see how it goes. 

I'm posting a message to the FreeTDS mailing list later today pointing
to Meet's post which opened this thread.

I look forward to the response :) Please let me know if you have any questions 

[Tim Allen]

Sorry to have missed meeting you at DjangoCon, Meet, but I'll add my findings here to the record.

By way of background, I work at The Wharton School, where we're a Python/Django (on RHEL) and SQL Server shop. I was responsible for implementing a working configuration for Django, starting with version 1.6, including building Vagrant boxes for developers which had to be 'vagrant up plug-and-play' ready to go. We've had pretty good success with the stack we're now using, but hit quite a few pain points and found many points where things can very much be improved.

First, drivers at the Linux level.

• We've had our best luck with FreeTDS and unixODBC for reliability. We've been using it successfully throughout the process.
  
• We found the MS provided driver more performant, specifically when dealing with large numbers of inserts.
• The MS driver also offered features like bulk-loading of data, which while not Django specific, would have still been a big win for our team using Python without Django.
• FreeTDS + unixODBC runs under any Linux distro we tested (RHEL/CentOS, Ubuntu, Debian) and Solaris.
• As mentioned, we found the MSODBC Driver for RedHat (http://www.microsoft.com/en-us/download/details.aspx?id=36437) to be more performant, however, ran into quite a few issues starting with Django 1.7.
  
• There seem to be issues with multi-threading. Starting with Django 1.7, we had to run Django's runserver with the '--nothreading' option.
• We ran into further issues with SQLRowCount returns from the driver. These rendered it unusable.
• Only supports RedHat, but also works on CentOS. We'd prefer one that at least also works on Ubuntu/Debian (a definite must for the Django community).
• The driver is closed-source.
• A note about FreeTDS: while FreeTDS 0.95 support TDS Version 7.3, as does pyodbc, we had to stick with TDS Version 7.2, as any stack we tried mapped to incorrect column types when switching to 7.3, which supports new SQL Server 2008 column types. I haven't had the time to dig deeper, map correctly, and issue a P.R. on this one. Additionally, TDS version numbers are very confusing to users (see my Stack Overflow responses, heh). At one point in time, FreeTDS has named the new TDS version "8.0" before Microsoft made an official declaration; Microsoft then called it "7.2". While often this will not affect configuration, now that 7.3 is support in FreeTDS 0.95, if "8.0" is entered in configuration, it is now default to "7.3" which causes issues, and the configuration in Django, odbc.ini, and freetds.conf must all explicitly state "7.2".

Regardless of the driver chosen (FreeTDS+unixODBC or MSODBC), we ended up having to use pyodbc instead of pymssql:

• We have a test suite performing table creates / destroys, basic CRUD operations, stored procedure execution, and more against both pyodbc and pymssql.
  
• pymssql outperforms pyodbc significantly against SQL Server, especially on SELECT and INSERT operations.
• pymssql on Linux offers no stable Django options, as noted throughout this thread.
  
• pyodbc offers several options.
• we initially started using django-pyodbc (lionheart on GitHub), which worked but required quite a few tweaks to the settings.
• we moved to django-pyodbc-azure, which we found a much easier install / Django DATABASES {} configuration, and is kept up to date in a timely fashion.

To summarize, here's what we now use:

• FreeTDS 0.91 - 0.95 (dependent on RedHat/CentOS version)
  
• unixODBC (dependent on RedHat/CentOS version)
  
• pyodbc
  
• django-pyodbc-azure
  

It works well for us, but we've had to make compromises, and the promise of better performance we've seen in certain scenarios is tempting. If I were building a wish list, here's what I'd like to see, for performance and ease-of-installation:

• Native driver to replace FreeTDS + unixODBC for SQL Server connections that is supported and runs on more than just RedHat/CentOS, preferably open source!
  
• Easy, prompt free option for install: I had to hack to install to avoid having to respond to interactive prompts in Vagrantfiles, Chef recipes, etc.
  
• Eventual inclusion in EPEL, etc, for yum or apt-get installs.
  
• Python package (to replace pyodbc with one that supports SQL Server functionality and performance)
  
• Django-Python package (to replicate django-pyodbc-azure's mappings to the pyodbc replacement)
  
• Support bcp, SSIS, DATE data type, FILTERED and SPATIAL (GeoDjango, anyone?) index types, easy Stored Procedure calls, OUTPUT variables, and more I can't remember off the top of my head.

These would also be big wins for users of other languages / frameworks, such as PHP and Ruby web frameworks, Flask, etc, who use SQL Server.

So there's a quick brain dump, sorry for the wall of text, but we've done a lot of research in this area. Please let me know how I can help - and thank you to the efforts of everyone who's done some amazing work to keep Django plugged into SQL Server.

Regards,

Tim A.

[Aymeric Augustin]

Hi Tim,

Thanks for sharing your experience!

On 13 sept. 2015, at 01:48, Tim Allen <fli...@peregrinesalon.com> wrote:

• We have a test suite performing table creates / destroys, basic CRUD operations, stored procedure execution, and more against both pyodbc and pymssql.
  
• pymssql outperforms pyodbc significantly against SQL Server, especially on SELECT and INSERT operations.

Did you mean “pyodbc outperforms pymssql”? Or did you go with pyodbc despite lower performance? (Or did I misread that?)

• pymssql on Linux offers no stable Django options, as noted throughout this thread.

django-pymssql is basically django-mssql on Linux. We could debate whether django-pyodbc-azure is more stable than django-mssql. There’ve been a bunch of forks over the years.

I’m not going to argue it further because I would inevitably sound like I’m tooting my own horn which isn’t my intent. I will just say that the picture isn’t all that clear.

• pyodbc offers several options.
• we initially started using django-pyodbc (lionheart on GitHub), which worked but required quite a few tweaks to the settings.
• we moved to django-pyodbc-azure, which we found a much easier install / Django DATABASES {} configuration, and is kept up to date in a timely fashion.

From the perspective of someone who works on Django’s internals, I believe django-pyodbc-azure could use a review of how the Django database backend API is implemented.

For example, looking at the new transaction APIs I introduced in 1.6, I see that it commits or rolls back implicitly in _set_autocommit. At best that’s weird and useless, at worst a data corruption bug.

Nothing that couldn’t be fixed, but just because the code works doesn’t means it handles edge cases well. django-mssql probably fares a bit better since its author cooperated with and eventually joined the core team.

Thanks to the abstraction provided by the Python DB-API, it should be quite easy to port code implementing Django’s database backend API between django-mssql and django-pyodbc-azure anyway.

-- 

Aymeric.

[Tim Allen]

Thanks for all of your efforts, Aymeric, I've been following your project since its inception - I'm FlipperPA on GitHub.

On Sunday, September 13, 2015 at 4:59:34 AM UTC-4, Aymeric Augustin wrote:

Did you mean “pyodbc outperforms pymssql”? Or did you go with pyodbc despite lower performance? (Or did I misread that?)

We went with pyodbc, despite lower performance. I've been meaning to put the simple tests up on GitHub - making a note to do that this week.

At the time we were looking at options, we couldn't find a stable Django option for pymssql. I should have been more clear about the time frame in which we were testing as well; this was right around the time that you first started django-pymssql. 

• django-pymssql is basically django-mssql on Linux. We could debate whether django-pyodbc-azure is more stable than django-mssql. There’ve been a bunch of forks over the years.
  

I’m not going to argue it further because I would inevitably sound like I’m tooting my own horn which isn’t my intent. I will just say that the picture isn’t all that clear.

There is definitely much more to consider now, than when we were first assessing options. I will say I've been impressed with django-pyodbc-azure staying up to date with Django's releases.

From the perspective of someone who works on Django’s internals, I believe django-pyodbc-azure could use a review of how the Django database backend API is implemented.

For example, looking at the new transaction APIs I introduced in 1.6, I see that it commits or rolls back implicitly in _set_autocommit. At best that’s weird and useless, at worst a data corruption bug.

Nothing that couldn’t be fixed, but just because the code works doesn’t means it handles edge cases well. django-mssql probably fares a bit better since its author cooperated with and eventually joined the core team.

Thanks to the abstraction provided by the Python DB-API, it should be quite easy to port code implementing Django’s database backend API between django-mssql and django-pyodbc-azure anyway.

You certainly know this stuff more intimately that me, I can just relay my experience, and hope it helps! Either way, a driver provided from Microsoft that offers better performance, easier installation and configuration, and more features would be a win for us all. The amount of time I've seen people spend trying to get FreeTDS + unixODBC + pyodbc running is substantial.

Thanks again for your efforts - they're very much appreciated, and I'll check out django-pymssql again soon, it has been over a year.

Regards,

Tim

[Tim Allen]

Hey team, as promised, here are the simple tests I put together to benchmark pyodbc vs pymssql. Be kind, this was Python I wrote a long time ago!

https://github.com/FlipperPA/pyodbc-pymssql-tests

I've included example output on the README. Very basic, but useful.

[Tim Graham]

Hi Meet,

I was wondering....

1. If you have any progress updates since your last message?
2. If you have any further details on the schedule for the time in Seattle in a week and a half? (including video conference details for those unable to attend in person)
3. If myself or the other attendees should do anything to prepare for the meetings?

Thanks!

[Meet Bhagdev]

On Thursday, October 1, 2015 at 12:32:25 PM UTC-7, Tim Graham wrote:

Hi Meet,

I was wondering....

1. If you have any progress updates since your last message?

    

     Yes, engineers on my team I are currently ramping up on the three Django-SQL Server adapters

•       Django-pymssql
•          Django-pyodbc-azure
•          Django-mssql
  

     The goal is to have a thorough understanding of what’s good and what’s bad with these adapters before the event. 

2. If you have any further details on the schedule for the time in Seattle in a week and a half? (including video conference details for those unable to attend in person)

• We will have a video conference link for Day 2 and Day 3. Participants interested can join the conference stream from their browser. The conference room mics are only capable to a certain extent. Thus the quality might be a little poor. 

• We are finalizing the detailed schedule this week and will post it on this thread by next Friday.  
  

3. If myself or the other attendees should do anything to prepare for the meetings?

    Here are some things that you should prepare before coming to Seattle.

-    

•      Have a clear understanding of the things that you need from Microsoft to improve the SQL Server support on Django. We have resources to do the heavy lifting but need guidance.
  
•      Share with us the issues we can help fix (on the Django side and on the Django-ORM(database) side). 
  

[Tim Graham]

If anyone is interested in listening in on the meetings with Microsoft engineers (Wednesday and Thursday 9am-5pm Pacific), let me know and I'll send you the Skype link.

[Fabio Caritas Barrionuevo da Luz]

is there any update about the progress of this?

-- 

Fábio C. Barrionuevo da Luz
Palmas - Tocantins - Brasil - América do Sul

[Cristiano Coelho]

I'm interested in the progress of this as well :)

Sorry I didn't read through all the posts, mostly the first ones about the idea.

I would like to know, have you guys decided on which adapter to use? I have had a project where we needed to connect to SQL Server from a linux machine (actually amazon lambda) and even worse, we couldn't install any library with dependencies on C code, so we used one that was implemented in pure python that worked very well (pytds if I'm not wrong), with ofcourse, not the best performance.
Why do I tell this? Because even if you want django to run on SQL Server, it doesn't really mean you want to run it on a Windows machine, actually, that would probably be a terrible idea (no ofense), since apache works horribly bad on Windows, and Linux is atually the best OS to run a web server with python code (either nginx or apache). So please keep this in mind when chosing a connector, since if it has C dependencies (which it will probably have, since the pure python ones are quite slow).

About if you need different connectors for Azure or SQLServer, I'm 'almost' sure you don't, we use azure or other cloud based sqlserver deployments with no problem with standard sqlserver connectors.

So basically, do not aim this towards making django more Windows friendly, but rather the actual SQL Server backend.

[Michael Manfre]

There has not been much direct progress with regards to an SQL Server database backend. My personal life keeps getting in the way and the test suite takes a disturbingly long time on my computer against MSSQL. Despite me being busy, others have helped move things forward. Some have contributed code to django-mssql and others have put effort towards optimizing the Django test suite so that it doesn't take so long when connecting to a MS SQL Server.

My goal for the next MSSQL database backend will be to support running Django on Windows, Linux, and Mac. Support for Windows is one of my primary concerns because there are existing production sites that run Django on Windows.

Regards,

Michael Manfre

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8faec72f-c5ab-4913-b852-76520f3b5ac9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

GPG Fingerprint: 74DE D158 BAD0 EDF8

keybase.io/manfre

[Tim Allen]

Thanks to everyone for their efforts; my workplace has a mix of SQL Server and PostgreSQL, heavier on the SQL Server side. Due to some groups reliance on SSIS and tight SQL Server integration with data vendors, that isn't going to change any time soon, so this is project is one we're following closely as well. We've tried to contribute by way of feedback, testing various configurations with various drivers, some documentation and a minuscule amount of code contribution.

In case this anecdotal evidence helps anyone in the meantime, the stack we've found most reliable these days (from RedHat / CentOS, at least, but also partially tested on Ubuntu) is:

- FreeTDS 0.95 (supports TDS version 7.3) with unixODBC. We tried the Microsoft provided ODBC driver, but ran into quite a few issues, particularly with multi-threading.

- pyodbc 3.0.10. pyodbc just works. We get slightly better performance with pymssql, but have found pyodbc to be more frequently updated and rock-solid. The performance upgrade didn't warrant using pymssql in our case, but is worth mentioning.

- django-pyodbc-azure. This is kept up to date with Django and Python release versions, and works with the least amount of configuration tweaking that we have found.

We're on a mix of RHEL/CentOS 6 and 7, and have gotten this stack running reliably up to v7.2. YMMV, of course!

As for the C dependencies, have you considered building the C binaries necessary on another server, and then just including them as part of a wheel (or something like that) for installation were you couldn't install? This sound like a perfect use case for a temporary vagrant box you could blow away after compiling. Just a thought that might give you the performance you need without stepping on anyone's toes.

[Cristiano Coelho]

Tim Allen,

What you said about compiling the C dependencies on a similar machine and then upload it all together indeed works (it was one of the options) but caused some other issues (ie: we usually develop on Windows, and also the compiled libraries are very platform specific) and performance was really not that important in this case. But just letting you know that your idea works most of the time if you are willing to take the extra work.

[Tim Allen]

Hi Cristiano, I support a bunch of developers on Windows (and Mac and Linux). You've probably heard this before, but Vagrant has really been a game changer for us. We're a RedHat shop, so I've built a CentOS vagrant box our developers can easily spin up for development purposes. It really helps for several reasons: they're developing on a Linux flavor as close to the production environment as possible, and have root access on their local VM which allows them to test new packages, configurations, and so on. It has improved our development process by leaps and bounds. For my stuff, I've developed a Vagrant / Linux / Python 3 / Django / FreeTDS - SQL Server box. I don't know if it might help you out, but feel free to have a look here: https://github.com/FlipperPA/django-python3-vagrant

I'm still hoping this project moves forward; it'd be great to have closer-to-native support.

Regards,

Tim

[Vin Yu]

Hey Folks,

My name is Vin and I work with Meet in the Microsoft SQL Server team. Just wanted to let you all know we are still looking into how we can better improve and support MSSQL for the Django framework. We’ll continue to sync with Michael and let you know of any updates soon.

Christiano and Tim - thanks for sharing your interest and sharing how you are using Django with MSSQL. It's great to learn from your scenarios.

If you have any concerns, questions or comments feel free to reach out to me at vinsonyu[at]microsoft.com

[Meet Bhagdev]

Hi all,

On interacting with several Django developers and committers, one of the questions often came up, can I use SQL Server on non Window OS's? I wanted to share that today Microsoft announced SQL Server availibility on Linux - https://blogs.microsoft.com/blog/2016/03/07/announcing-sql-server-on-linux/. 

While there is still work needed to strengthen the MSSQL-Django story, we hope this aids more Linux developers to give SQL Server a shot. Let me know of your thoughts and questions :)

Cheers,

Meet

[Josh Smeaton]

Wow, that's really great news! I haven't used mssql for a number of years but it was always very nice to work with. Having it available to run on linux will make it much easier for the Django community to test against mssql, provided we're able to get/develop an appropriate driver and backend. 

Cheers

[Tim Allen]

[slightly off-topic] I'm wondering if this will extend to SQL Server Management Studio. While I'm mainly a command line basher, many of coworkers are married to the GUI. I've found SSMS blows the competition out of the water when it comes to DB management GUIs. I'm wondering if this means SSMS will run on Linux (or Mac) eventually.

This is certainly very big news. I wouldn't be shocked to some day see Windows itself running on the Linux Kernel.

Meet, how can we help strengthen the story for MSSQL-Django? It seems we have a chicken and egg problem here. A very small amount of Django sites use SQL Server, but is that because of the difficulty in the available stack and confusion over options? Would usage increase if provided in core?

[Vin Yu]

Hey Tim,

We've gotten lots of questions about the tools when we announced SQL Server on Linux. I am curious; what are the DB management/development tasks that are being performed by your coworkers? What are they using SSMS for? I am interested in learning more. [Perhaps we can follow up by email as this seens off-topic here :) ] 

In terms of strengthening the story for MSSQL-Django, I think there is a little bit of both difficulty and confusion over options; here are some ideas that we are working on and could solve these issues:

• Improve documentation/examples [decrease confusion]: There's already so much awesome content out there on getting started with Django (but not many are referencing MSSQL as the db of choice or why MSSQL is a great option).
• Improve getting started experience [decrease difficulty]: Getting MSSQL for development (free and easy/fast set up) is hard today;this is on MSFT to improve this experience.

We want to help provide better developer experiences for those who want to create new Django apps + MSSQL databases and if MSSQL were in the core, it would definitely help with that. This would increase usage and is something we are striving to achieve. We will continue to work with the community to make this happen.  

=) , 

Vin

[Cristiano Coelho]

"Improve documentation/examples [decrease confusion]: There's already so much awesome content out there on getting started with Django (but not many are referencing MSSQL as the db of choice or why MSSQL is a great option)."

I wouldn't think of MSSQL as a great option for django at least until it is supported natively and not through 3rd party apps which are always behind django updates.

[Tim Graham]

Yes, I wonder if Microsoft abandoned the idea of providing engineering resources to help out projects like django-mssql or if things are just moving really slowly? I'm not aware of any follow up related to that idea since Michael, Michiya, and I visited Microsoft in October.

[Michael Manfre]

The slowness of progress on django-mssql is entirely my fault due to the personal reasons I briefly mentioned during DUTH. Microsoft has offered help and in my opinion seems very eager to start pushing pull requests, but I haven't had the bandwidth to direct their efforts. I should have more bandwidth starting the end of this month when things are finally settled.

Regards,

Michael Manfre

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/adc9ef36-50da-4ca3-bd68-933f6314f7f4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Vin Yu]

Hey Tim,

We are continuing to follow up with Michael and have reached out to Michiya as well. We have not abandoned the idea of providing engineering resources either, and are still working out the logistics with Michael as he will help direct our efforts. We are syncing again in mid-April to discuss the next steps; we are hoping to start identifying a plan, come up with a proposal and start working on this shortly after. 

Regards,
Vin

[Tim Allen]

If people are planning on being in Philadelphia around the time of DjangoCon this summer (July 17th - 22nd), I'd be happy to arrange space for a meeting / code sprint and provide food. Django sprints are on July 21st / 22nd which would be ideal; or the weekend afterwards. I'm on the DjangoCon organization team and arranging for the space for the conference already, so I'd be happy to help out with logistics.

Just a thought, since several of us may be planning on being in Philly already. The Wharton School (full disclosure: I'm an employee) is hosting DjangoCon and runs Django with SQL Server, so I'm sure we'd be amenable to helping out where we can. More details: https://2016.djangocon.us

Both Tims on this thread are near Philadephia. :) Regards,

Tim A.

[uri...@gmail.com]

I wonder if there have been any updates on MS support for a official/supported MS SQL Django driver? Did the offered engineering effort from MS ever come through? Given the availability of of MS SQL on Linux, as well as support for Django in Visual Studio, it would be great if this came to fruition.

Explicit support for Django with IronPython would also be nice, if MS would really want to take their Django support to the next level...

[Michael Manfre]

Quick update of the current status of things related to MSSQL. The most actively maintained MSSQL backend that I'm aware of is django-pyodbc-azure (also works with standalone SQL Server). Michiya has been doing an amazing job supporting that backend. There has been no real movement toward there being an officially supported MSSQL backend for Django. Microsoft has continued to be involved with Django and myself related to making a better MSSQL experience for us.

Regards,

Michael Manfre

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8258f170-8f10-45fc-a1d3-9551efca6d1c%40googlegroups.com.

[Dmitriy Sintsov]

Microsoft probably will take Django support to the next level with Windows Subsystem for Linux. It's still beta but a very promising one. Not only it should make compiling Python module binaries (like lxml) much easier, it also fixes binary / nix specific support of node.js / Ruby packages.

[kunal...@williamoneilindia.com]

Hi,

I am new to Django, but planning to use Django 2.0 for my newest project. I am using MSSQL db and hence facing issues with it.

Anyone working on getting an MSSQL backend that works with Django 2.0? (Couldn't find one that's up to date including django-pyodbc-azure).

Any help would be much appreciated.

Thanks,

Kunal

[Tim Allen]

Hi Kunal, welcome to the Django community! django-pyodbc-azure is the best maintained SQL Server engine I've found, and it typically gets updated a little more quickly than this. However, Michaya who maintains the package has been extremely busy lately, and said it will take him a few more weeks to get it done. If you're starting a new project, could you develop on SQLite for now and then swap in django-pyodbc-azure when it is ready?

Regards,

Tim

[Sean Martz]

Hello,

It seems like this issue has lost momentum. Is this still something that's on anyones radar? It looks like django-pyodbc-azure is not actively maintained anymore (it looks like Michaya has taken a hiatus from GitHub). It also looks like there's a small community potentially popping up that's interested in first class MSSQL Server support for Django. (https://github.com/FlipperPA/django-mssql-backend). Is Microsoft still interested in committing resources to this goal? In my situation, it would be a lot easier to sell stakeholders and decision makers on Django if it had first class support for MSSQL Server.

For what it's worth, Django-pyodbc-azure is still working well.

Cheers,

Sean

[Tim Allen]

Hi Sean, just an update from what I know.

We are still waiting for a reply from Microsoft. They're a large company, so understandably, it takes a little while.

For now, if people need to get onto Django 2.2 for long term support (which will last until April, 2022), you can use this package:

https://github.com/ESSolutions/django-mssql-backend I've been running it in production for months without incident. Of course, YMMV.

If Microsoft and/or the DSF end up wanting to bring support under the Django umbrella, the django-mssql-backend repository is a possible starting point, IMHO.

The django-mssql-backend is currently being developed and support for Django 3.0 is being worked on: ESSolutions/django-mssql-backend#18

Regards,

Tim

[Warren Chu]

Hi All,

Microsoft has now committed ongoing resources towards improving MSSQL and Azure SQL support for Django. We're currently focused on internal compliance and forking the ESSolutions django-mssql-backend, adding testing pipelines, refactoring the Django DB engine naming convention, and addressing current test suite errors.

We'd love to hear from current mssql-backend maintainers as well as mssql-backend users about the existing issues and feature requests that we should be prioritizing.

We looking forward to engaging the community and working towards MSSQL as a first-class supported backend for Django.

-Warren

[Florian Apolloner]

Hi Warren,

given that the current django-mssql-backend maintainers might not be reading here I'd suggest you open a ticket on the repository to get in contact with them. It also might be a good idea to re-evaluate this thread (and possibly at Microsoft internally) to see why the last approaches failed and what can be done better this time. Personally I'd also suggest to work "openly" with the existing fork from ESSolutions and submit PRs there for testing pipelines etc (I assume you'd want to run them on azure or something and having that integration developed in the open would certainly provide good guidance for other projects). I am not sure what internal compliance entails, but please keep in mind that this is an open source project used already by people and compliance to some internal microsoft standards might not be something people care about or need.

I am excited to see your PRs against the repo.

Cheers,

Florian

[Chris Wedgwood]

Hi Warren

This is excellent news!

Thanks

Chris

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/188ccc39-4675-45f1-8303-ed2b93d51dfcn%40googlegroups.com.

[r...@whidbey.com]

Good news.  I've been using Django on MSSQL for about 8 years.  Couple of things:

- I've been using pyodbc, not django-mssql.  I note your messages from 2015 include it as a library to check for compatibility; what was the outcome of that?  Is it proposed that django-mssql become the "best-practices" interface for SQL Server?

- Couple of persistent pain points:

  1. Testing.  The Django code that sets up test databases fails with MSSQL, while it succeeds with PostGRE, MySQL and SQLite.  The issues seem to revolve around setting constraints as the tables are generated, rather than holding off and enabling the constraints at the end of the process.

  2. Stored Procedures.  These need to be loaded as an additional step in the creation of a database, and don't really have any representation in Django per se so migrations, etc don't generally have an idea that they exist.

I'd be happy to test out what you come up with against our system.  It's currently serving a custom REST interface with 2-million-plus rows of transactions,with clients world-wide, along with a customer-facing web app, a staff site and a suite of Tableau reports.

[Dan Davis]

I'd suggest someone talk with professional DBAs for MSSQL.  In my work, which is Federal government, the DBA told me that disconnecting from PSQL as "appuser" and attempting to connect to database "postgres" in order to create the test database violated FISMA.  I had to subclass my own postgresql backend to create the test user while connected to my actual database.  I would imagine that the original reasons for doing it this way harkens back to a time when people used the same database server for production, staging, qa, and integration, and some of these environments may not have existed. While in some organizations we have only production, staging/qa, and development/integration, I would guess there are few cases where the same database server is used for production and the other environments.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a531fb44-4a5c-48f3-b28c-d78e3419141fn%40googlegroups.com.

[Warren Chu]

Thanks for the responses.

@Florian - I've now reached out to the maintainers of ESSolutions as well as the relevant previous Microsoft engagement owners, thanks for the suggestion

@Chris - This is excellent news indeed, we are excited to make progress beginning with some small steps

@r...@whidbey.com - 

A) django-mssql is viewed as the Phase 1 focus and current preferred option with a longer term Phase 2 goal of including MSSQL as a supported backend for Django core

B) I've now logged both your pain points as issues in our private repo (which will be made public after internal compliance reviews)

C) Happy to hear about your scaled production app using Django and MSSQL. Could I ask you to send a "Hello" email to myself at v-wa...@microsoft.com, so that we can follow up with eventual testing?

@dans...@gmail.com - Your security concerns are important and noted, I've flagged this for investigation as a priority

Cheers,

Warren

[Florian Apolloner]

Hi Warren,

> A) django-mssql is viewed as the Phase 1 focus and current preferred option with a longer term Phase 2 goal of including MSSQL as a supported backend for Django core

Please keep in mind that Phase 2 is something that might never happen. We have a tendency to not bloat Django and there is no reason why a database backend cannot live outside of core. Tim Graham is currently working on a cockroach backend and is running against the builtin Django testsuite (+ submitting patches to Django where we need to alter tests due to reliance on PKs etc…). This approach works imo very well (I did the same when writing a backend for Informix). We are very supportive of 3rd party db backends in the sense that we usually quickly address issues in our testsuite etc to make testing those external backends against Django easier. In that sense there is no strong reason to include the MSSQL backend in core. Package management in Python works well enough to allow the backend to stay external.

Personally I think having the django-mssql backend with MS support behind it (be that test infra for the existing django-mssql fork etc) is already the icing on the cake.

Cheers,

Florian

[Adam Johnson]

Please keep in mind that Phase 2 is something that might never happen. We have a tendency to not bloat Django and there is no reason why a database backend cannot live outside of core.

 

I agree here with Florian. It's not *impossible* to consider merging the backend, but I think we'd want to see several years of ongoing support from MS before even considering it.

Keeping it outside of core can probably work in your favour, with the ability to update support for new features or versions of SQL Server outside of Django's release cadence.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/73a54032-b179-4c42-b16c-155c1fc3690en%40googlegroups.com.

--

Adam

[Warren Chu]

@Florian, @Adam - Thanks for the feedback. Focusing on MSSQL as a 3rd party backend sounds good, indeed even as the long-term goal. We have no directive at this point that inclusion into Django core is mandatory, so I think we're aligned in terms of practical goals. I'm sure we'll have new perspectives and opinions in 1 year's time, and we can re-evaluate or confirm this current sentiment at that time. Thanks again.

Cheers,

Warren