Yet another SoC introduction: Getting multi-db done
Casper Jensen
I am applying to this years Google Summer of Code where I would like
to get multiple databases working in Django. I hope you can give me
some feedback on my proposal so I can get it refined before submitting
it officially.
First, a quick introduction:
My name is Casper Jensen and I am 20 years old. I am a student at
Aalborg University in Denmark, where I study software engineering at
the computer science department. I have been using Django exclusively
the last seven months, using it to build a mid-sized community web
site. (I could use multi-db support in this project).
The proposal:
From what I can tell the multi-db branch has been inactive for over a
year now. Even if it seems like it is close to completion based on the
branch page_.
I propose to finish the implementation, make it compatible with the
queryset-refactor branch if needed and to add some additional
improvements to the current implementation.
One part I plan to improve is based on the following quote from the branch page:
> Add methods to Manager to support per-model installation. This will enable each > model to be installed using the connection that it specifies. It causes some > complications, mainly in determining the correct order in which to install. My > current solution is to depend on the developer already having figure that out by > defining her models in a sensible order;
I will replace this behavior with an algorithm capable of determining
the dependencies enabling the manager to install the models in the
correct order, thus reducing the complexity for the developer.
The second part I would like to work on is the possibility to make
references to tables placed in other databases.
I looked in the branch code and got the impression that is not
possible in the current implementation. This feature has both good and
bad implications. It is impossible to enforce a reference constraint
on the database level if the tables resides in different databases.
But on the other hand it would be much more intuitive and convenient
for the developer if the mapping between table and database where
transparent.
What I am most concerned about is the scope of this proposal. It seems
like a large part of the multi-db support is done, but then again, in
order to finish it, implement the two previous mentioned features and
merge it with queryset-refactor could be enough work for a summer.
I will continue to research the current implementation, so I can give
a more detailed description of how the extra features should be
implemented and what needs to be done before the branch can be
considered commit ready.
Any suggestions to the proposal, critique or encouragement are welcome
(and looked forward to).
Best regards
- Casper
Ben Ford
I did some work on upping multi-db to HEAD some months ago (see ticket #4747). I also started work on building a bit more of a flexible method of relating models to database backends. I've also been in contact with Daryl Spitzer about trying to make another start on getting multi-db implemented. To that end I've given Daryl a patch of the stuff I've done against HEAD with a view to starting to get it working against qsrf.
I've got a lot of work commitments at the moment, but multi-db is something I'm really keen on too, so I'd be glad to help as much as I can. Feel free to contact me off list and I'd be happy to send you the patch as well.
Cheers,
Ben
Daryl Spitzer
I've been working on multiple databases at the Django sprint at PyCon
2008. See http://groups.google.com/group/django-developers/browse_thread/thread/3646f4c5284daffa,
http://groups.google.com/group/django-developers/browse_thread/thread/160d100a47b24930
and http://code.djangoproject.com/ticket/1142.
I came to the conclusion (after reading some old threads and some
online discussions with Ben Ford--who appears to be the last person to
attempt to update the branch) that the multiple-db-support branch and
the scheme described in
http://code.djangoproject.com/wiki/MultipleDatabaseSupport should be
abandoned.
I proposed an API to Jacob and Adrian during the spring that was
inspired by reading some code that Ben put in dpaste--they seemed to
like it. I've been working on implementing a prototype of that API
for the last three days. I think I'm close to having something
working. I just got a patch from Ben, which will hopefully answer
some of my questions.
I just read Ben's reply--you may want to take him up on his offer to
get his patch. And keep an eye on ticket #1142. I'll attach a patch
there as soon as I have something working.
Even if I do get a prototype working reasonable soon, there will still
be plenty of work for you to do before it will be ready. It will need
to be integrated with the changes implementing the new INSTALLED_APPS
scheme that I believe Adrian will be documenting. The code will
likely need some cleaning. It will need tests, and have to be tested
with all the database backends. It's likely some issues will turn up
from that. And it will need to be documented. I'll welcome the help.
--
Daryl
Casper Jensen
Sounds like you are on the right path. I will contact Ben and look at
his patch until I can get my hands on your prototype.
I will be looking forward to helping getting this feature into Django,
either as a SoC project or in my spare time.
Best regards
Casper