Multi Database use with statement

[wei guangjing]

Hi,

I write a patch for using with statement in python 2.5 and up for multi database access, example code like this:

# test.py
from __future__ import with_statement
from django.contrib.auth.models import User
from django.db import using

with using('db1'):
print User.objects.count() # db1 user count
with using('db2'):
print User.objects.count() # db2 user count
print User.objects.count() # default db user count

#settings.py

# default db
DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'test.db'

# others
DATABASES = {
'db1': dict(
DATABASE_ENGINE = 'sql_server.pyodbc',
DATABASE_NAME = 'db1',
DATABASE_USER = 'user',
DATABASE_PASSWORD = 'password',
DATABASE_ODBC_DSN = 'DSN1',
DATABASE_COLLATION = 'Chinese_PRC_CI_AS',
),
'db2': dict(
DATABASE_ENGINE = 'sqlite3',
DATABASE_NAME = 'db2.db',
),
}

Please see patch in the attchments, comment welcome.

Regards,
Wei guangjing

--
http://code.google.com/p/django-pyodbc

[mtrier]

Hi Wei,

On Dec 27, 6:23 am, "wei guangjing" <v...@163.com> wrote:
> Hi,
>
> I write a patch for using with statement in python 2.5 and up for multi database access, example code like this:
>
> # test.py
> from __future__ import with_statement
> from django.contrib.auth.models import User
> from django.db import using
>
> with using('db1'):
>     print User.objects.count() # db1 user count
>     with using('db2'):
>         print User.objects.count() # db2 user count
> print User.objects.count() # default db user count
>

A couple of ideas to throw in the mix. I'd like to see a Meta
option. Like:

Meta:
using = 'db2'

So you have a default for your model.

Additionally what about just using the generative syntax for the
using. It doesn't seem that a lot is to be gained by using the
context manager. For instance:

User.objects.using('db2').all()

Also when thinking about third-party plugable applications you have to
consider how you're going to handle that. If you have a third party
app and you want it to all be directed to a particular database, how
does that get handled in your scenario above?

Just a couple of things to consider.

Michael Trier

[alex....@gmail.com]

Personally I think I like Michael's suggestion of a using method on a
queryset, and a default in Meta(pretty much the way ordering works).
To his question of reusable applications I think the solution to that
is actually something outside the scope of Django itself, a reusable's
apps views should take a db kwarg callback that takes whatever the
appropriate params and returns the db to be used.

Alex

[wei guangjing]

Please see the attached patch for using in models's Meta option, I also like this option.

This patch export some helper function from django.db,

get_connection(name) get database connection by name
get_current_connection() get current connection

you can use follow function to ensure use special db connection:
enter_connection_management(connection)
leave_connection_management()

Wei guangjing

[Ludvig Ericson]

On Dec 27, 2008, at 12:23, wei guangjing wrote:
> from __future__ import with_statement
> from django.contrib.auth.models import User
> from django.db import using
>
> with using('db1'):
> print User.objects.count() # db1 user count
> with using('db2'):
> print User.objects.count() # db2 user count
> print User.objects.count() # default db user count

A few thoughts:
- How will Django know which database is "chosen"? It would need
some kind of thread-global storage to know. That's, IMO, not very nice.
- Is "using" really such a great name? If I look at it, I don't
really associate to databases or anything like it.
- Isn't the QuerySet object responsible for knowing what to query?
Like:
db1_count = User.objects.at("db2").count()

As for reusability, it becomes sort of a tough situation for app
developers if they're responsible entirely for defining which database
takes care of what - if you'd use the QuerySet-centered way, you'd be
able to pass that to generic views, for example.

These are just my thoughts, though.

[wei guangjing]

I setup a git repository http://github.com/weigj/django-multidb/tree/master for it.

I patched fields, query and manager, so Model.objects.using('db1') is work now, like this:

users = User.objects.using('db1').filter(....)
for user in users:
user.save(using='db2')