Django ManyToMany performance?

[Ruturaj Dhekane]

Hi all,

I have a particular datastructure where there are two objects

1. A document - and a lot of its properties - like content, version, timestamp etc.

2. Contributors - basically people represented by unique IDs

A document can have many contributors and a contributor can author many documents. A typical manyTomany scenario

Is there a design where I can find how ManyToMany is implemented (I know I can go through the code - but existing design document/wiki/blog might help).

Also what are the performance implications? 

How many SQL queries does it really make per request on django? 

Can it scale to say 1million documents and a few thousand users?

What is a good SQL backend I can use? MYSQL or SQLite?

Has anyone tried with MondoDB etc for such relationships?

Any examples where you have used ManyToMany and scaled it up for many entries?

Thank you for your answers. Would give me a lot of confidence in going ahead with choosing M2M as a field. The whole project might hinge on it!

Ruturaj

[Daniel Roseman]

This question is a bit too vague to answer. A many-to-many relationship in a Django database is implemented just as it would be in any other SQL db - with an intermediate linking table containing foreign keys to both sides of the relationship - and queried via JOINs between the three tables. The only difference is that the linking table isn't exposed unless you specifically ask it to be.

There's nothing specifically efficient or inefficient about the m2m field: it just is what it is. If it fits your data model - which, given your description, I'd say it does - you should use it.

--

DR.

[Robin St.Clair]

First of all , any DBMS that is misused by design or in operation will give poor performance.. There is an excellent mature database available for Django developers, PostgreSQL. It works fine on the same machine as you are developing on.

MongoDB is NOT a suitable DB for general purposes. It is fine for documents, less good for being updated, particularly as it locks tables when updating or inserting.

Make sure your generated database is checked by somebody who knows about databases.

R+C

[Ruturaj Dhekane]

Thanks Daniel and Robin.

Have we seen the scales where Many-to-many DBs through Django work well? Like 50K articles in 5K publications. 

The aim of this question was to make a design choice too - whether I should use Django constructs/calls directly or should i write my own SQL to make queries - as my schemas are not just many-to-many but a lot more than that. 

Thanks for the suggestion on getting the DB schemas reviewed by a DB expert. I realized that is going to help me a lot for sure!

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/6007019b-b007-4d46-adab-4dcbf16b0e7d%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[Tom Evans]

On Mon, Oct 28, 2013 at 8:23 PM, Ruturaj Dhekane <rutu...@gmail.com> wrote:
> Thanks Daniel and Robin.
>
> Have we seen the scales where Many-to-many DBs through Django work well?
> Like 50K articles in 5K publications.

Numbers are really meaningless. One of our legacy products has a
database with 300 million rows in it, it works as well in Django as it
does in C++, because what the application connecting to the database
"is" is largely irrelevant, what matters is what it does to the
database.

> The aim of this question was to make a design choice too - whether I should
> use Django constructs/calls directly or should i write my own SQL to make
> queries - as my schemas are not just many-to-many but a lot more than that.
>
> Thanks for the suggestion on getting the DB schemas reviewed by a DB expert.
> I realized that is going to help me a lot for sure!

Almost anything you can express in a RDBMS DDL can be encapsulated as
a django model - probably the only major outstanding issue is multi
column primary keys. Using django or using direct SQL on the same
schema will produce identical results.

Besides which, django does not restrict you from using SQL, which is
sometimes necessary - whilst django covers most of SQL as DDL, there
are many many DML SQL queries that you cannot express in django's ORM.

(DDL = Data Definition Language, DML = Data Manipulation Language, in
case any of those terms are unclear)

Cheers

Tom