| Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 8/21/15 11:09 AM | Hi Django Committers,
My name is Meet Bhagdev, I work in the Database Systems engineering team at Microsoft in Seattle, WA. My focus is the APIs used to connect to and use Azure SQL Database and SQL Server (MSSQL). Example APIs are ODBC, JDBC, ADO.NET, etc.
We’d love for Django users to have a solid option to use MSSQL and Azure SQL if they wish, and it would be great to make this option a reality. Our goal is to partner with committers like yourself to bring first-class support for MSSQL to Django. We want to bring the benefits of Django to millions of existing MSSQL and Azure SQL customers as well as folks that would like to evaluate us as a database option for Django.
I have been in touch with Tim Graham and Russell Keith-Magee related to us building such support. We are prepared to make the engineering investment required on our end and to work with the Django community to make this happen, the right way. We are willing to do all the heavy lifting, but we do need your help and guidance.
At Microsoft we have an existing program for bringing partners and eco-system developers to Seattle, WA, all expenses paid, and we’d like to extend this to the Django committers. This would be a great way to get started. We would love to get a group of you here during our October workshop. The idea is to get developers from both sides to meet and learn from each other—Django and Microsoft.
Please reach out me at mebha@microsoft.com or give me a call on +1 425 722 5342 if you would like to attend. Would love to get you more details.
Sincerely, Meet Bhagdev | Linkedin | Github
P.S. This is not a sales pitch; we are excited about bringing the benefits of Django to our customers.:)
|
| Re: Improving MSSQL and Azure SQL support on Django | Aymeric Augustin | 8/22/15 3:28 AM | Hello Meet,
Even though this isn’t what you’re asking for :-) I’ll take this opportunity to summarize the landscape for combining Django and MSSQL / Azure SQL, as far as I understand it. There’s a good option for MSSQL or Azure SQL users running Django on Windows: https://bitbucket.org/Manfre/django-mssql. (Perhaps it could be made even better, but from my perspective, it’s solid.) There’ve been talks of making it more official. But no consensus has been found yet. The main difficulty is that very few Django committers use Windows as a primary OS, perhaps two out of fifty. Most of us don’t even own a Windows license. We face a similar issue with our continuous integration. Currently it’s running on Linux. We lack the skills and money to run Windows and SQL Server. (Sponsoring in the form of Azure SQL credits for the purpose of testing Django may solve parts of this problem, if using a remote database isn’t too slow for test suite’s workload.) There isn’t such a clear story for running Django on Linux. This led me to write https://github.com/aaugustin/django-pymssql. Alternatives include https://github.com/denisenkom/django-sqlserver and https://github.com/lionheart/django-pyodbc. I can’t say django-pymssql is solid. But it works and it shows that django-mssql could be made cross-platform with limited effort, subject to the quality of the underlying libraries: https://github.com/pymssql/pymssql and https://github.com/FreeTDS/freetds. However my interest has faded since I left the company where I had this use case. Furthermore testing is very painful. I'm running SQL Server Express in a VM. Tests are about 20 times slower than with PostgreSQL or MySQL. As a consequence Django’s test suite took about 2 hours. I appreciate the invitation. Unfortunately I live a bit too far to make the trip conveniently. I’m still interested in making Django work out of the box with SQL Server like it does with Oracle. (That’s for historical reasons: some people gathered and made it happen). I believe interoperability with SQL Server is key for using Django in the medium-sized companies where SQL Server is the obvious choice of database server. I hope this helps. Thanks for reaching out! -- Aymeric. |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 8/22/15 5:28 AM | I agree it would be great to get some help running the Django tests on Windows. I run them in a local virtual machine every so often, but I would love to be able to delegate fixing Windows issues. Meet, can your team provide ongoing help with fixing Windows-specific issues in Django, even if they aren't related to MSSQL/Azure? That is something where we've had a hard time finding volunteers to help with and is obviously important if we want to claim 1st-class support for Windows. I don't have any experience with MSSQL/Azure, but I could probably attend the October 13-15 event at Microsoft if you think my participation would be valuable and if the DSF were to approve the time commitment as part of the fellow duties. |
| Re: Improving MSSQL and Azure SQL support on Django | Shai Berger | 8/22/15 2:53 PM |
There's also django-pyodbc-azure, a fork of django-pyodbc (actually, the current django-pyodbc is also a fork of the original project, which has been discontinued). I took the liberty to forward the message to that project.
Shai. |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 8/24/15 11:12 AM | I guess the first step is to identify which third-party backend(s) we'll target to adopt officially (or at least borrow from heavily). For example, will we need separate backends for MSSQL and Azure? (Knowing nothing about the landscape myself, this question could be nonsensical.) Is this discussion something that should happen before the October summit? It seems to me the face-to-face time will likely be more productive if we have some of the high-level details ironed out. By the way, is videoconferencing an option for Django developers interested in participating in the discussion at that time but unable to travel to Seattle? |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 8/25/15 4:53 PM | Absolutely agree with Tim here. We need to start exploring all the cool open sourced third party adapters. Borrowing/adopting them definitely seems like the way to go about things instead of re inventing the wheel. I tried doing some research and came across the following: Am I missing any? Best, Meet |
| Re: Improving MSSQL and Azure SQL support on Django | Russell Keith-Magee | 8/25/15 6:34 PM | On Sat, Aug 22, 2015 at 8:28 PM, Tim Graham <timog...@gmail.com> wrote: I don't see why this would be a problem - I'll run it past the fellowship committee and get their OK. Russ %-) |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 8/28/15 3:21 PM | Thanks Russ! |
| Re: Improving MSSQL and Azure SQL support on Django | Russell Keith-Magee | 8/28/15 6:54 PM | Hi Meet,
The Fellowship committee has just approved this, so go ahead and plan whatever you need to plan. Yours, Russ Magee %-) -- |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 8/31/15 11:33 AM | By "we" do you mean the engineers at Microsoft who will be working on this? Will they be doing any planning for this before the meet up in October? I'd like to know more specifics about the agenda and goals for the workshop. Also, I wanted to make sure my question about a videoconferencing option for Django developers interested in participating in the discussion at that time but unable to travel to Seattle didn't get lost. |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 8/31/15 12:07 PM | Hi Tim, Thanks for bringing it up. Yes, we will have developers who will engage with Django developers (like yourself) during the workshop. The goals of the October workshop are to: 1) Get to know each other and begin building a relationship 2) Get in a room with Microsoft developers and discuss the current landscape 3) Work on half day coding sprint(s) with Microsoft developers to get started with contributions 4) Establish a plan for how Microsoft can best contribute to Django, and ensure we have great integration between Azure SQL and MSSQL and Django by maintaining our contributions The key takeaway is that we want to contribute to existing solutions to improve the Django and MSSQL/Azure SQL story. To do so we want to understand the current landscape, the gaps, and the next steps to make this happen (the right way). We are currently in the planning stages and would love to get feedback. What do you think about the goals mentioned above? Is there anything you like to add/remove? We can definitely make attending via Skype an option for attendees unable to make it in person. Best, Meet |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 8/31/15 12:46 PM | To arrive at the best solution, I think a lot of discussion needs to happen on this mailing list before October. As for me, besides the name of some existing packages that offer SQL Server/Azure support, I know very little about the current landscape so I really wouldn't have anything to offer in a discussion, but there are others on this list who definitely do. The existing Microsoft database backends have come out of specific developer needs and I think it would maximize success if your team were personally invested in the backend by building a Django application backed by SQL Server/Azure. If I were a Microsoft engineer responsible for this project, I would start by building a small Django application to solve some problem that my team has. Then I would test it out with all the third-party backends for Microsoft databases previously mentioned to get a feel for how they work. I should learn enough to at least participate in a discussion with other Django developers about the direction of this project. Ideally, I would be able to learn enough to write up a Django Enhancement Proposal [1] that summarizes the landscape and describes what an "official solution" should look like. If you are relying on members of the Django team to do all this upfront work, then please tell us so we can see if someone will do it. If you are expecting to get all this done in a couple days in Seattle, this doesn't seem feasible to me. However, if we have a working proposal by mid-September that the community can review and give feedback on, then I think our time together in October could be productive. I don't know anything about the background of your team, but it seems pretty unlikely that any sort of coding sprint would be useful unless the work and learning I described happens well before the workshop. [1] https://github.com/django/deps/ |
| Re: Improving MSSQL and Azure SQL support on Django | Russell Keith-Magee | 8/31/15 4:58 PM | Hi Tim, Meet
I know this is very late to be mentioning, but one idea that worth raising: DjangoCon US is next week (in Austin). Thursday and Friday are coding sprints, where there will be many people (including a good chunk of the core team) looking for projects to hack on. Microsoft is already a sponsor of DjangoCon itself; if we can get some engineers from the Azure team (assuming there aren't already some coming), we can start some of this prep work. Yours, Russ Magee %-) > --> https://groups.google.com/d/msgid/django-developers/9a79aa2d-ad82-4905-8bcc-835911c21103%40googlegroups.com. |
| unk...@googlegroups.com | 9/1/15 11:07 AM | <This message has been deleted.> | |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 9/1/15 7:52 PM | Hi Russ,Tim That is a great idea. I am going to try to engage with the team at Microsoft who is in charge of the sponsorship. I plan on attending the conference so I will see some of you there. @Tim: I agree that a lot of ground work needs to be done in terms of testing out the current third party adapters and deciding which one/ones should we adopt and extend. I know Michael Manfre has a lot of expertise here. I will do my due diligence and test out the existing solutions but if Michael could share some of his findings, that would be great. If anyone else also wants to take a stab at testing and using the existing adapters for a sample app, that would be great as well. I will try to share my learnings soon. Here are the ones I am going to test, am I missing any? Thanks, Meet |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 9/2/15 7:49 AM | Hi Meet, Will you be the primary contributor/maintainer from Microsoft of this project or will their be other engineers at Microsoft involved? I'd like to understand the bus factor on your side and how much time resources you plan to provide. To give you an idea of how our community works when it comes to decision making and implementing large features, I thought you might find Aymeric's log of his work to add multiple template engines support to Django helpful: https://myks.org/en/multiple-template-engines-for-django/. |
| Re: Improving MSSQL and Azure SQL support on Django | Ramiro Morales | 9/2/15 3:05 PM | Hi all,
(sorry for possibly repeating things folks have already posted to this thread) Meet: Is this list of APIs the final and full one? Because there is work being done on a different stack, the one formed by : * FreeTDS (http://www.freetds.org/ GPL licensed) which implements the wire-level TDS protocol. * pymssql (http://pymssql.org LGPL licensed) -- Python bindings for FreeTDS which implements the Python DB-API 2.0. I'm part of the team maintaining it * django-pymssql (https://github.com/aaugustin/django-pymssql , MIT-licensed) which was created by Aymeric Augustin and depends on pymssql plus ... * django-mssql (https://django-mssql.readthedocs.org/en/latest/ , MIT-licensed) which was created by Michael Manfre Michael, Aymeric and me are Django development team members. Personally, I've been working on stabilizing the lower layers by following a "yak shaving" non-strategy: We (the pymssql team) realised the official pymssql Windows binaries (in particular the FreeTDS libraries) we published when releasing pymssql 2.1.1 don't link in a SSL implementation and so they aren't usable to actually connect to Azure even if the pymssql code iself has such ability. This, plus the fact that the manual process of creating the actual matrix of Windows deliverables is a bit tedious led me to try using the AppVeyor.com hosted Windows CI platform (free for open source projects) to test and actually build the binaries. The work in progress on this can be seen at https://github.com/ramiro/pymssql/tree/appveyor and https://ci.appveyor.com/project/ramiro/pymssql (When working on pymssql 2.1.1 at some point I created a free Azure account with my credit card, tested (on Linux) the pymssql implementation of connection changes needed to get it to work against Azure's SQL Server and cancelled it before it started billing.) So, this led me to start contributing to FreeTDS so to get it back to build cleanly on Windows (work partially included with the 0.95 release back in June) and to also get it built/tested using AppVeyor.com using the experience gained with pymssql. This has been already merged in the current FreeTDS development code and allows the maintainer and contributors to work without access to Windows/SQL Server licenses (my case). - https://ci.appveyor.com/project/freetds/freetds - https://github.com/FreeTDS/freetds/commit/3db5caa48f281f3558d4031cb5a0f0d8e8eef28c I know MS dropped support of the DB-Library (and hence the API it provides, of which FreeTDS is a open source implementation) back in the SQL Server 2005 times. That's why I ask if this stack has any chance of getting some support from Microsoft. Personal motivation for this is simply to get Django (running on Linux) + SQL Server to be a viable choice, even when I currently have no actual need of this. I was one of the two developers behind the original django-pyodbc project which had reached "almost full Django test suite passing" status back in the Django 1.0 & 1.1 (2008-2009) times, but abandoned it when discoverd (by logging the traffic with SQL Servr tools) that the combination of pyodbc + Linux ODBC stack meant the queries were sent twice to the DB server, see https://code.google.com/p/django-pyodbc/issues/detail?id=16 . This is also why I started considering a FreeTDS-based solution a better technical choice. I'm posting a message to the FreeTDS mailing list later today pointing to Meet's post which opened this thread. Regards, -- Ramiro Morales @ramiromorales |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 9/2/15 4:25 PM | Hi Tim, I will definitely be driving the initial planing and will help lay out the ground work. There will be other engineer(s) who will work on contributing to third party adapter(s) and the Django project(eventualy). But first, we need to figure out the ask and the nature of work required. The goal of our October coding camp workshop is to figure that out the ask, by bringing developers from the Django community like yourself, and the developers from my team, to come together and better understand the technical details. Once we have a clear understanding of the existing solutions and the amount of work required on improving the Django-MSSQL/Azure SQL story we will be able to commit developer(s) that will contribute and do all the heavy lifting. Also thanks for the share of Aymeric's log of his work on the template engine. I absolutely agree on having something similar for the work we are planning on doing. Once we have a clear idea I will start working on something similar. A final version of this may only be possible after the October event. Best, Meet |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 9/2/15 7:15 PM |
This is definitely not a final and full one. We have actually started endorsing pymssql and FreeTDS to our Linux and Mac customers who want to use Azure SQL. We have also created documentation that lets customers use pymssql on 1. Windows: https://azure.microsoft.com/en-us/documentation/articles/sql-database-develop-python-simple-windows/ 3. Mac: https://azure.microsoft.com/en-us/documentation/articles/sql-database-develop-python-simple-mac-osx/
Also a big thank you from
everyone on our team here for working on FreeTDS and pymssql. I definitely
makes our customers lives easier
I am currently exploring
django-pymssql, django-mssql, and django-pyodbc-azure to see which adapter(s)
can we adopt to provide Django and MSSQL/Azure SQL support. Do have any
opinions as to which one(s) should we contribute to eventually?
I looked around and it seems like this problem was fixed on this page by Christoph Gohlke from University of California, Irvine: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pymssql. I downloaded pymssql from here and it seemed like it links the SSL implementations. I also documented the procedure here: https://azure.microsoft.com/en-us/documentation/articles/sql-database-develop-python-simple-windows/
Do you still need Azure credits
to run Azure SQL for free, I can try to get some free credits for you and the
FreeTDS and pymssql team depending on the need.
This is great to hear.
Getting it on Appveyor definitely makes it easy to build on Windows.
We are very much looking at providing support(as you can tell) from our documentation mentioned above. We are currently in the prioritization stage and would love to discuss some of the things you need support for. As for now, we feel Django is a good place to start, as there is customer demand and we are falling behind. Would have a document which highlights the todo feature list where you need help on? I can pass it on to my team to get some responses so that we can get the ball rolling.
That is a great find. I
actually was not aware of that. The good news for the Linux side of the story
here is that we are releasing a new Linux ODBC driver in the next 2 months. It
would be great if we can test this out and see how it goes.
I look forward to the response
:) Please let me know if you have any questions |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 9/12/15 4:53 PM | Sorry to have missed meeting you at DjangoCon, Meet, but I'll add my findings here to the record. By way of background, I work at The Wharton School, where we're a Python/Django (on RHEL) and SQL Server shop. I was responsible for implementing a working configuration for Django, starting with version 1.6, including building Vagrant boxes for developers which had to be 'vagrant up plug-and-play' ready to go. We've had pretty good success with the stack we're now using, but hit quite a few pain points and found many points where things can very much be improved. First, drivers at the Linux level.
Regardless of the driver chosen (FreeTDS+unixODBC or MSODBC), we ended up having to use pyodbc instead of pymssql:
To summarize, here's what we now use:
It works well for us, but we've had to make compromises, and the promise of better performance we've seen in certain scenarios is tempting. If I were building a wish list, here's what I'd like to see, for performance and ease-of-installation:
These would also be big wins for users of other languages / frameworks, such as PHP and Ruby web frameworks, Flask, etc, who use SQL Server. So there's a quick brain dump, sorry for the wall of text, but we've done a lot of research in this area. Please let me know how I can help - and thank you to the efforts of everyone who's done some amazing work to keep Django plugged into SQL Server. Regards, Tim A. |
| Re: Improving MSSQL and Azure SQL support on Django | Aymeric Augustin | 9/13/15 1:59 AM | Hi Tim, Thanks for sharing your experience!
Did you mean “pyodbc outperforms pymssql”? Or did you go with pyodbc despite lower performance? (Or did I misread that?) django-pymssql is basically django-mssql on Linux. We could debate whether django-pyodbc-azure is more stable than django-mssql. There’ve been a bunch of forks over the years. I’m not going to argue it further because I would inevitably sound like I’m tooting my own horn which isn’t my intent. I will just say that the picture isn’t all that clear.
From the perspective of someone who works on Django’s internals, I believe django-pyodbc-azure could use a review of how the Django database backend API is implemented. For example, looking at the new transaction APIs I introduced in 1.6, I see that it commits or rolls back implicitly in _set_autocommit. At best that’s weird and useless, at worst a data corruption bug. Nothing that couldn’t be fixed, but just because the code works doesn’t means it handles edge cases well. django-mssql probably fares a bit better since its author cooperated with and eventually joined the core team. Thanks to the abstraction provided by the Python DB-API, it should be quite easy to port code implementing Django’s database backend API between django-mssql and django-pyodbc-azure anyway. -- Aymeric. |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 9/16/15 8:27 AM | Thanks for all of your efforts, Aymeric, I've been following your project since its inception - I'm FlipperPA on GitHub.
We went with pyodbc, despite lower performance. I've been meaning to put the simple tests up on GitHub - making a note to do that this week. At the time we were looking at options, we couldn't find a stable Django option for pymssql. I should have been more clear about the time frame in which we were testing as well; this was right around the time that you first started django-pymssql.
There is definitely much more to consider now, than when we were first assessing options. I will say I've been impressed with django-pyodbc-azure staying up to date with Django's releases.
You certainly know this stuff more intimately that me, I can just relay my experience, and hope it helps! Either way, a driver provided from Microsoft that offers better performance, easier installation and configuration, and more features would be a win for us all. The amount of time I've seen people spend trying to get FreeTDS + unixODBC + pyodbc running is substantial. Thanks again for your efforts - they're very much appreciated, and I'll check out django-pymssql again soon, it has been over a year. Regards, Tim |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 9/17/15 12:38 PM | Hey team, as promised, here are the simple tests I put together to benchmark pyodbc vs pymssql. Be kind, this was Python I wrote a long time ago! I've included example output on the README. Very basic, but useful. |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 10/1/15 12:32 PM | Hi Meet, I was wondering.... 1. If you have any progress updates since your last message? 2. If you have any further details on the schedule for the time in Seattle in a week and a half? (including video conference details for those unable to attend in person) 3. If myself or the other attendees should do anything to prepare for the meetings? Thanks! |
| unk...@googlegroups.com | 10/2/15 11:47 AM | <This message has been deleted.> | |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 10/2/15 11:53 AM | On Thursday, October 1, 2015 at 12:32:25 PM UTC-7, Tim Graham wrote:
Yes, engineers on my team I are currently ramping up on the three Django-SQL Server adapters
The goal is to have a thorough
understanding of what’s good and what’s bad with these adapters before the
event.
Here are some things that you should prepare before coming to Seattle. -
|
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 10/13/15 2:12 PM | If anyone is interested in listening in on the meetings with Microsoft engineers (Wednesday and Thursday 9am-5pm Pacific), let me know and I'll send you the Skype link. |
| Re: Improving MSSQL and Azure SQL support on Django | Fabio Caritas Barrionuevo da Luz | 1/25/16 5:59 PM | is there any update about the progress of this? -- Fábio C. Barrionuevo da Luz Palmas - Tocantins - Brasil - América do Sul |
| Re: Improving MSSQL and Azure SQL support on Django | Cristiano Coelho | 1/26/16 9:15 PM | I'm interested in the progress of this as well :) Sorry I didn't read through all the posts, mostly the first ones about the idea. I would like to know, have you guys decided on which adapter to use? I have had a project where we needed to connect to SQL Server from a linux machine (actually amazon lambda) and even worse, we couldn't install any library with dependencies on C code, so we used one that was implemented in pure python that worked very well (pytds if I'm not wrong), with ofcourse, not the best performance. Why do I tell this? Because even if you want django to run on SQL Server, it doesn't really mean you want to run it on a Windows machine, actually, that would probably be a terrible idea (no ofense), since apache works horribly bad on Windows, and Linux is atually the best OS to run a web server with python code (either nginx or apache). So please keep this in mind when chosing a connector, since if it has C dependencies (which it will probably have, since the pure python ones are quite slow). About if you need different connectors for Azure or SQLServer, I'm 'almost' sure you don't, we use azure or other cloud based sqlserver deployments with no problem with standard sqlserver connectors. So basically, do not aim this towards making django more Windows friendly, but rather the actual SQL Server backend. |
| Re: Improving MSSQL and Azure SQL support on Django | Michael Manfre | 1/27/16 10:34 AM | There has not been much direct progress with regards to an SQL Server database backend. My personal life keeps getting in the way and the test suite takes a disturbingly long time on my computer against MSSQL. Despite me being busy, others have helped move things forward. Some have contributed code to django-mssql and others have put effort towards optimizing the Django test suite so that it doesn't take so long when connecting to a MS SQL Server. My goal for the next MSSQL database backend will be to support running Django on Windows, Linux, and Mac. Support for Windows is one of my primary concerns because there are existing production sites that run Django on Windows. Regards, Michael Manfre
GPG Fingerprint: 74DE D158 BAD0 EDF8 |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 1/28/16 7:48 AM | Thanks to everyone for their efforts; my workplace has a mix of SQL Server and PostgreSQL, heavier on the SQL Server side. Due to some groups reliance on SSIS and tight SQL Server integration with data vendors, that isn't going to change any time soon, so this is project is one we're following closely as well. We've tried to contribute by way of feedback, testing various configurations with various drivers, some documentation and a minuscule amount of code contribution. In case this anecdotal evidence helps anyone in the meantime, the stack we've found most reliable these days (from RedHat / CentOS, at least, but also partially tested on Ubuntu) is: - FreeTDS 0.95 (supports TDS version 7.3) with unixODBC. We tried the Microsoft provided ODBC driver, but ran into quite a few issues, particularly with multi-threading. - pyodbc 3.0.10. pyodbc just works. We get slightly better performance with pymssql, but have found pyodbc to be more frequently updated and rock-solid. The performance upgrade didn't warrant using pymssql in our case, but is worth mentioning. - django-pyodbc-azure. This is kept up to date with Django and Python release versions, and works with the least amount of configuration tweaking that we have found. We're on a mix of RHEL/CentOS 6 and 7, and have gotten this stack running reliably up to v7.2. YMMV, of course! As for the C dependencies, have you considered building the C binaries necessary on another server, and then just including them as part of a wheel (or something like that) for installation were you couldn't install? This sound like a perfect use case for a temporary vagrant box you could blow away after compiling. Just a thought that might give you the performance you need without stepping on anyone's toes. |
| Re: Improving MSSQL and Azure SQL support on Django | Cristiano Coelho | 1/28/16 10:09 AM | Tim Allen, What you said about compiling the C dependencies on a similar machine and then upload it all together indeed works (it was one of the options) but caused some other issues (ie: we usually develop on Windows, and also the compiled libraries are very platform specific) and performance was really not that important in this case. But just letting you know that your idea works most of the time if you are willing to take the extra work. |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 1/30/16 6:20 AM | Hi Cristiano, I support a bunch of developers on Windows (and Mac and Linux). You've probably heard this before, but Vagrant has really been a game changer for us. We're a RedHat shop, so I've built a CentOS vagrant box our developers can easily spin up for development purposes. It really helps for several reasons: they're developing on a Linux flavor as close to the production environment as possible, and have root access on their local VM which allows them to test new packages, configurations, and so on. It has improved our development process by leaps and bounds. For my stuff, I've developed a Vagrant / Linux / Python 3 / Django / FreeTDS - SQL Server box. I don't know if it might help you out, but feel free to have a look here: https://github.com/FlipperPA/django-python3-vagrant I'm still hoping this project moves forward; it'd be great to have closer-to-native support. Regards, Tim |
| Re: Improving MSSQL and Azure SQL support on Django | Vin Yu | 2/22/16 4:54 PM | Hey Folks,
My name is Vin and I work with Meet in the Microsoft SQL Server team. Just wanted to let you all know we are still looking into how we can better improve and support MSSQL for the Django framework. We’ll continue to sync with Michael and let you know of any updates soon. Christiano and Tim - thanks for sharing your interest and sharing how you are using Django with MSSQL. It's great to learn from your scenarios. If you have any concerns, questions or comments feel free to reach out to me at vinsonyu[at]microsoft.com |
| Re: Improving MSSQL and Azure SQL support on Django | Meet Bhagdev | 3/7/16 2:37 PM | Hi all, On interacting with several Django developers and committers, one of the questions often came up, can I use SQL Server on non Window OS's? I wanted to share that today Microsoft announced SQL Server availibility on Linux - https://blogs.microsoft.com/blog/2016/03/07/announcing-sql-server-on-linux/. While there is still work needed to strengthen the MSSQL-Django story, we hope this aids more Linux developers to give SQL Server a shot. Let me know of your thoughts and questions :) Cheers, Meet |
| Re: Improving MSSQL and Azure SQL support on Django | Josh Smeaton | 3/7/16 3:03 PM | Wow, that's really great news! I haven't used mssql for a number of years but it was always very nice to work with. Having it available to run on linux will make it much easier for the Django community to test against mssql, provided we're able to get/develop an appropriate driver and backend. Cheers |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 3/8/16 10:13 AM | [slightly off-topic] I'm wondering if this will extend to SQL Server Management Studio. While I'm mainly a command line basher, many of coworkers are married to the GUI. I've found SSMS blows the competition out of the water when it comes to DB management GUIs. I'm wondering if this means SSMS will run on Linux (or Mac) eventually. This is certainly very big news. I wouldn't be shocked to some day see Windows itself running on the Linux Kernel. Meet, how can we help strengthen the story for MSSQL-Django? It seems we have a chicken and egg problem here. A very small amount of Django sites use SQL Server, but is that because of the difficulty in the available stack and confusion over options? Would usage increase if provided in core? |
| Re: Improving MSSQL and Azure SQL support on Django | Vin Yu | 3/8/16 6:20 PM | Hey Tim, We've gotten lots of questions about the tools when we announced SQL Server on Linux. I am curious; what are the DB management/development tasks that are being performed by your coworkers? What are they using SSMS for? I am interested in learning more. [Perhaps we can follow up by email as this seens off-topic here :) ] In terms of strengthening the story for MSSQL-Django, I think there is a little bit of both difficulty and confusion over options; here are some ideas that we are working on and could solve these issues:
We want to help provide better developer experiences for those who want to create new Django apps + MSSQL databases and if MSSQL were in the core, it would definitely help with that. This would increase usage and is something we are striving to achieve. We will continue to work with the community to make this happen. =) , Vin |
| Re: Improving MSSQL and Azure SQL support on Django | Cristiano Coelho | 3/9/16 6:43 PM | "Improve documentation/examples [decrease confusion]: There's already so much awesome content out there on getting started with Django (but not many are referencing MSSQL as the db of choice or why MSSQL is a great option)."I wouldn't think of MSSQL as a great option for django at least until it is supported natively and not through 3rd party apps which are always behind django updates. |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Graham | 3/11/16 10:54 AM | Yes, I wonder if Microsoft abandoned the idea of providing engineering resources to help out projects like django-mssql or if things are just moving really slowly? I'm not aware of any follow up related to that idea since Michael, Michiya, and I visited Microsoft in October. |
| Re: Improving MSSQL and Azure SQL support on Django | Michael Manfre | 3/11/16 11:31 AM | The slowness of progress on django-mssql is entirely my fault due to the personal reasons I briefly mentioned during DUTH. Microsoft has offered help and in my opinion seems very eager to start pushing pull requests, but I haven't had the bandwidth to direct their efforts. I should have more bandwidth starting the end of this month when things are finally settled. Regards, Michael Manfre
|
| Re: Improving MSSQL and Azure SQL support on Django | Vin Yu | 3/22/16 3:52 PM | Hey Tim, Regards, |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 3/23/16 6:08 AM | If people are planning on being in Philadelphia around the time of DjangoCon this summer (July 17th - 22nd), I'd be happy to arrange space for a meeting / code sprint and provide food. Django sprints are on July 21st / 22nd which would be ideal; or the weekend afterwards. I'm on the DjangoCon organization team and arranging for the space for the conference already, so I'd be happy to help out with logistics. Just a thought, since several of us may be planning on being in Philly already. The Wharton School (full disclosure: I'm an employee) is hosting DjangoCon and runs Django with SQL Server, so I'm sure we'd be amenable to helping out where we can. More details: https://2016.djangocon.us Both Tims on this thread are near Philadephia. :) Regards, Tim A. |
| Re: Improving MSSQL and Azure SQL support on Django | uri...@gmail.com | 4/28/17 12:55 PM | I wonder if there have been any updates on MS support for a official/supported MS SQL Django driver? Did the offered engineering effort from MS ever come through? Given the availability of of MS SQL on Linux, as well as support for Django in Visual Studio, it would be great if this came to fruition. Explicit support for Django with IronPython would also be nice, if MS would really want to take their Django support to the next level... |
| Re: Improving MSSQL and Azure SQL support on Django | Michael Manfre | 4/28/17 3:01 PM | Quick update of the current status of things related to MSSQL. The most actively maintained MSSQL backend that I'm aware of is django-pyodbc-azure (also works with standalone SQL Server). Michiya has been doing an amazing job supporting that backend. There has been no real movement toward there being an officially supported MSSQL backend for Django. Microsoft has continued to be involved with Django and myself related to making a better MSSQL experience for us.
Regards, Michael Manfre --You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8258f170-8f10-45fc-a1d3-9551efca6d1c%40googlegroups.com. |
| Re: Improving MSSQL and Azure SQL support on Django | Dmitriy Sintsov | 4/29/17 1:02 AM | Microsoft probably will take Django support to the next level with Windows Subsystem for Linux. It's still beta but a very promising one. Not only it should make compiling Python module binaries (like lxml) much easier, it also fixes binary / nix specific support of node.js / Ruby packages. |
| Re: Improving MSSQL and Azure SQL support on Django | kunal...@williamoneilindia.com | 1/18/18 5:39 AM | Hi, I am new to Django, but planning to use Django 2.0 for my newest project. I am using MSSQL db and hence facing issues with it. Anyone working on getting an MSSQL backend that works with Django 2.0? (Couldn't find one that's up to date including django-pyodbc-azure). Any help would be much appreciated. Thanks, Kunal |
| Re: Improving MSSQL and Azure SQL support on Django | Tim Allen | 1/19/18 5:20 PM | Hi Kunal, welcome to the Django community! django-pyodbc-azure is the best maintained SQL Server engine I've found, and it typically gets updated a little more quickly than this. However, Michaya who maintains the package has been extremely busy lately, and said it will take him a few more weeks to get it done. If you're starting a new project, could you develop on SQLite for now and then swap in django-pyodbc-azure when it is ready? Regards, Tim
|