Django-nonrel patches

565 views
Skip to first unread message

Jonas H.

unread,
Dec 3, 2011, 5:36:57 PM12/3/11
to django-developers
Hello list!

On Jacob's suggestion in this thread [1] back in April, I split the diff
between Django trunk and Django-nonrel into logically separated patches.

I uploaded most of them to the ticket tracker (a few things are still
missing but these are the most important changes):

https://code.djangoproject.com/ticket/17335
https://code.djangoproject.com/ticket/17336
https://code.djangoproject.com/ticket/17337
https://code.djangoproject.com/ticket/17338
https://code.djangoproject.com/ticket/17339
https://code.djangoproject.com/ticket/17340
https://code.djangoproject.com/ticket/14881 (not exactly a new, has been
around for a while)

Happy reviewing! :-)

Jonas

[1]
http://groups.google.com/group/django-developers/browse_thread/thread/2c52d074b5a0f614/8610a64348e6f216?

--
http://jonas.lophus.org | http://django-mongodb.org

Want to become a Django MongoDB Engine/Django-nonrel core developer?
Get in touch: jo...@lophus.org

Adrian Holovaty

unread,
Dec 5, 2011, 6:13:39 PM12/5/11
to django-d...@googlegroups.com
On Sat, Dec 3, 2011 at 4:36 PM, Jonas H. <jo...@lophus.org> wrote:
> On Jacob's suggestion in this thread [1] back in April, I split the diff
> between Django trunk and Django-nonrel into logically separated patches.
>
> I uploaded most of them to the ticket tracker (a few things are still
> missing but these are the most important changes):

Thanks for splitting these up, Jonas! I can commit to reviewing and
committing these in the next few days. I just took care of
https://code.djangoproject.com/ticket/17335 as a first step.

Adrian

Aymeric Augustin

unread,
Dec 7, 2011, 3:53:37 PM12/7/11
to django-d...@googlegroups.com
On 3 déc. 2011, at 23:36, Jonas H. wrote:
> On Jacob's suggestion in this thread [1] back in April, I split the diff between Django trunk and Django-nonrel into logically separated patches.
>
> I uploaded most of them to the ticket tracker (a few things are still missing but these are the most important changes):

Hi Jonas,

I just came across your patches while I was triaging the "unreviewed" queue. Unfortunately, I found it difficult to assess them, since they don't include tests or docs.

Generally, all changes should come with tests. If you're adding new features — for instance "Support for non-integer primary keys" sounds an interesting one — those should be documented too.

That would help people who aren't familiar with django-nonrel (like me) to review your patches, and in particular to understand their consequences.

That said, I believe NoSQL support in a very interesting experiment for Django's ORM, and I really hope we can move forward with these tickets!

Best regards,

--
Aymeric.

Jonas H.

unread,
Dec 8, 2011, 4:31:39 PM12/8/11
to django-d...@googlegroups.com
On 12/07/2011 09:53 PM, Aymeric Augustin wrote:
> I just came across your patches while I was triaging the "unreviewed" queue. Unfortunately, I found it difficult to assess them, since they don't include tests or docs.
>
> Generally, all changes should come with tests. If you're adding new features — for instance "Support for non-integer primary keys" sounds an interesting one — those should be documented too.
>
> That would help people who aren't familiar with django-nonrel (like me) to review your patches, and in particular to understand their consequences.

So there are multiple reasons why these patches don't include tests nor
docs:

1.) I didn't write the code, I'm just submitting the patches in their
current state

2.) I don't like wasting my time, so I won't write extensive
documentation before the patch gets accepted

3.) Many changes can't be tested in a useful manner using unittest. For
instance the non-integer primary key patch should really be tested using
integration tests -- but that requires complex mocking which takes a lot
of time. Again I'd like to not waste my time. I'm willing to write tests
and docs once the patch is accepted.

To make things clear, I don't refuse to do *informal* documentation
(like in this thread or on the ticket tracker) because that doesn't take
so much time and brain power :-)

So if you need help understanding the patches, shoot!

Jonas

Want to become a Django MongoDB Engine/Django-nonrel core developer?

Get in touch: django-non...@googlegroups.com

Jacob Kaplan-Moss

unread,
Dec 8, 2011, 5:39:33 PM12/8/11
to django-d...@googlegroups.com
On Thu, Dec 8, 2011 at 3:31 PM, Jonas H. <jo...@lophus.org> wrote:
> 2.) I don't like wasting my time, so I won't write extensive documentation
> before the patch gets accepted

I'm sorry, but that's just not how we do things around here.
Documentation isn't an afterthought; is as important -- more even --
then the code itself. In order for a patch to be considered "done", it
needs documentation.

If you don't feel the same that's fine, but you'll need to connect
with someone who doesn't see documentation as a waste of time.
Otherwise with a million patches to review I'm going to prioritize the
ones that have documentation, and yours'll get lost.

> 3.) Many changes can't be tested in a useful manner using unittest. For
> instance the non-integer primary key patch should really be tested using
> integration tests -- but that requires complex mocking which takes a lot of
> time. Again I'd like to not waste my time. I'm willing to write tests and
> docs once the patch is accepted.

Like documentation, I don't see tests as a "waste of time", either.
Again, there are many other patches to reivew, so I'm going to pick
the ones that're closer to done.

I *really* want to see this work merged so that django-nonrel can
become a plugin and not a fork. But without tests or docs it's just
not gonna happen.

Jacob

Carl Meyer

unread,
Dec 8, 2011, 6:19:43 PM12/8/11
to django-d...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mmm, this reads to me like a bit of a mis-understanding. Hinges on what
Jonas means by "before the patch gets accepted." The phrase is slightly
confusing because we have an Accepted state in Trac, but that applies to
tickets, in concept, not to specific patches.

Declining to do additional patch work before the _ticket_ is marked
Accepted is reasonable and consistent with our usual process. Marking
the ticket accepted only means that the general idea is accepted in
principle, it does not validate the specific approach in any existing patch.

Docs and tests are, of course, required before any patch is actually
committed. In my reading of this thread, I think that's already understood.

Aymeric mentioned that tests and/or docs would clarify the intent of
some of the tickets, which isn't clear enough for him to even accept or
decline the ticket in concept. In that case, I think it would make sense
to close the ticket "needsinfo" with specific comments on what about the
feature request is not clear enough to evaluate. (Patch review shouldn't
even be necessary in order to move a ticket from Unreviewed to Accepted
or closed, IMO - the ticket description itself ought to provide clear
enough motivation for that).

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7hRg8ACgkQ8W4rlRKtE2flDACdFPDijhay53JXoFHAIobjFCRP
0FwAoI81yprWk359Tzb3XQKSsEng8P+a
=4YyI
-----END PGP SIGNATURE-----

Jonas H.

unread,
Dec 8, 2011, 7:37:16 PM12/8/11
to django-d...@googlegroups.com
On 12/08/2011 11:39 PM, Jacob Kaplan-Moss wrote:
> On Thu, Dec 8, 2011 at 3:31 PM, Jonas H.<jo...@lophus.org> wrote:
>> 2.) I don't like wasting my time, so I won't write extensive documentation
>> before the patch gets accepted
>
> I'm sorry, but that's just not how we do things around here.
> Documentation isn't an afterthought; is as important -- more even --
> then the code itself. In order for a patch to be considered "done", it
> needs documentation.
>
> If you don't feel the same that's fine, but you'll need to connect
> with someone who doesn't see documentation as a waste of time.
> Otherwise with a million patches to review I'm going to prioritize the
> ones that have documentation, and yours'll get lost.

I don't think documentation is a waste of time and tests aren't either.

I just don't like to write docs and tests in the first of thirteen
iterations. I don't think the patches are ready for checking yet; what
I'm currently looking for is general feedback on the approach taken by
the modifications.

Jannis Leidel

unread,
Dec 9, 2011, 5:07:59 AM12/9/11
to django-d...@googlegroups.com

I'd like to mention that I also tried to review those tickets but didn't have an idea what they were supposed to accomplish. In other words, I simply wasn't able to give a general feedback since there wasn't enough information to base it on. Docs and tests are well established practices to achieve an understanding of a problem in the life time of a patch, nobody is asking for a perfect patch, really.

Jannis

Wilfred Hughes

unread,
Dec 12, 2011, 9:53:31 AM12/12/11
to Django developers
Following up on #17338 ([nonrel] supports_deleting_related_objects
database feature flag):

carljm said that:

> [Django] is (sort of) _emulating_ SQL cascading deletes, but it does so in a way that doesn't assume anything at all from the backend.

I'm not sure this is correct. If I define the following models:

from django.db import models

class Person(models.Model):
name = models.CharField(max_length=100)


class Car(models.Model):
name = models.CharField(max_length=100)
owner = models.ForeignKey(Person)

Related objects are indeed deleted on a sqlite Django project:

In [1]: from vortaro.models import Person, Car

In [2]: p = Person.objects.create(name='bob')

In [3]: c = Car.objects.create(name='robin reliant', owner=p)

In [4]: p.delete()

In [5]: Car.objects.all()
Out[5]: []

In [6]: from django.db import connection

In [7]: connection.queries
Out[7]:
[{'sql': u'INSERT INTO "vortaro_person" ("name") VALUES (bob)',
'time': '0.001'},
{'sql': u'INSERT INTO "vortaro_car" ("name", "owner_id") VALUES
(robin reliant, 1)',
'time': '0.001'},
{'sql': u'SELECT "vortaro_car"."id", "vortaro_car"."name",
"vortaro_car"."owner_id" FROM "vortaro_car" WHERE
"vortaro_car"."owner_id" IN (1)',
'time': '0.001'},
{'sql': u'DELETE FROM "vortaro_car" WHERE "id" IN (1)', 'time':
'0.000'},
{'sql': u'DELETE FROM "vortaro_person" WHERE "id" IN (1)', 'time':
'0.000'},
{'sql': u'SELECT "vortaro_car"."id", "vortaro_car"."name",
"vortaro_car"."owner_id" FROM "vortaro_car" LIMIT 21',
'time': '0.000'}]

This isn't making a second query to deleting the Car model, it is
depending on the database to do this. For comparison, here's the same
on App Engine:

In [1]: from test_app.models import Person, Car

In [2]: p = Person.objects.create(name='bob')

In [3]: c = Car.objects.create(name='robin reliant', owner=p)

In [4]: p.delete()

In [5]: Car.objects.all()
Out[5]: [<Car: Car object>]

In [6]: c = Car.objects.get()

In [7]: c.owner
---------------------------------------------------------------------------
DoesNotExist Traceback (most recent call
last)
/home/wilfred/bleeding_edge/django-testapp/<ipython-input-7-
f94c13ea1b0a> in <module>()
----> 1 c.owner

/home/wilfred/bleeding_edge/django-testapp/django/db/models/fields/
related.pyc in __get__(self, instance, instance_type)
313 rel_obj = rel_mgr.using(db).get(**params)
314 else:
--> 315 rel_obj =
QuerySet(self.field.rel.to).using(db).get(**params)
316 setattr(instance, cache_name, rel_obj)
317 return rel_obj

/home/wilfred/bleeding_edge/django-testapp/django/db/models/query.pyc
in get(self, *args, **kwargs)
349 if not num:
350 raise self.model.DoesNotExist("%s matching query
does not exist."
--> 351 % self.model._meta.object_name)
352 raise self.model.MultipleObjectsReturned("get()
returned more than one %s -- it returned %s! Lookup parameters were
%s"
353 % (self.model._meta.object_name, num, kwargs))

DoesNotExist: Person matching query does not exist.

Have I horribly misunderstood?

Wilfred

Karen Tracey

unread,
Dec 12, 2011, 11:46:30 AM12/12/11
to django-d...@googlegroups.com

This SELECT:
 
 {'sql': u'SELECT "vortaro_car"."id", "vortaro_car"."name",
"vortaro_car"."owner_id" FROM "vortaro_car" WHERE
"vortaro_car"."owner_id" IN (1)',
 'time': '0.001'},
 {'sql': u'DELETE FROM "vortaro_car" WHERE "id" IN (1)', 'time':
'0.000'},

And the above subsequent delete of a votaro_car are Django finding the cars referencing the to-be-deleted Person and deleting them, before deleting the Person, which comes next.
 
 {'sql': u'DELETE FROM "vortaro_person" WHERE "id" IN (1)', 'time':
'0.000'},
 {'sql': u'SELECT "vortaro_car"."id", "vortaro_car"."name",
"vortaro_car"."owner_id" FROM "vortaro_car" LIMIT 21',
 'time': '0.000'}]

This isn't making a second query to deleting the Car model, it is
depending on the database to do this.

Django did explicitly find-and-delete the Car. It did not rely on the underlying database to do that.
 

It seems the problem here is that Django's attempt to find the Cars referencing the to-be-deleted person is failing here, so it did not delete the associated Car. It would if it could find it though, really it's not using database cascade delete for this.

Karen

Anssi Kääriäinen

unread,
Dec 12, 2011, 11:52:20 AM12/12/11
to Django developers

On Dec 12, 4:53 pm, Wilfred Hughes <wilf...@potatolondon.com> wrote:
> > [Django] is (sort of) _emulating_ SQL cascading deletes, but it does so in a way that doesn't assume anything at all from the backend.
>
> I'm not sure this is correct. If I define the following models:

<SNIP>


> In [1]: from vortaro.models import Person, Car
>
> In [2]: p = Person.objects.create(name='bob')
>
> In [3]: c = Car.objects.create(name='robin reliant', owner=p)

At this point you have a person, and a related car in DB.


> In [4]: p.delete()
>
> In [5]: Car.objects.all()
> Out[5]: []

<SNIP>
Delete starts here:


>  {'sql': u'SELECT "vortaro_car"."id", "vortaro_car"."name",
> "vortaro_car"."owner_id" FROM "vortaro_car" WHERE
> "vortaro_car"."owner_id" IN (1)',
>   'time': '0.001'},

We are deleting person with id 1. Here we fetch related cars.


>  {'sql': u'DELETE FROM "vortaro_car" WHERE "id" IN (1)', 'time':
> '0.000'},

Now this is the emulated cascade. Above we found one car with id 1.
Now we are deleting it.


>  {'sql': u'DELETE FROM "vortaro_person" WHERE "id" IN (1)', 'time':
> '0.000'},

All related objects have been deleted, so now we delete the person.


>  {'sql': u'SELECT "vortaro_car"."id", "vortaro_car"."name",
> "vortaro_car"."owner_id" FROM "vortaro_car" LIMIT 21',
>   'time': '0.000'}]

Here nothing is found, as excepted.


> This isn't making a second query to deleting the Car model, it is
> depending on the database to do this.

It did make a delete to the Car model, but _before_ the person was
deleted. That is the correct order to delete the objects.


> For comparison, here's the same on App Engine:

<SNIP>
> In [7]: c.owner
<SNIP>


> DoesNotExist: Person matching query does not exist.

I believe AppEngine could have executed the above queries, there are
no joins. Instead it has deleted the Person, but left the related car
in the DB.

- Anssi

Wilfred Hughes

unread,
Dec 13, 2011, 10:17:06 AM12/13/11
to Django developers
On Dec 12, 4:46 pm, Karen Tracey <kmtra...@gmail.com> wrote:>
>
> This SELECT:
>
> >  {'sql': u'SELECT "vortaro_car"."id", "vortaro_car"."name",
> > "vortaro_car"."owner_id" FROM "vortaro_car" WHERE
> > "vortaro_car"."owner_id" IN (1)',
> >  'time': '0.001'},
> >  {'sql': u'DELETE FROM "vortaro_car" WHERE "id" IN (1)', 'time':
> > '0.000'},
>
[...]

>
> Django did explicitly find-and-delete the Car. It did not rely on the
> underlying database to do that.

Ah, absolutely right. I'll file a bug our end for this. Cheers.

Wilfred

Emil Stenström

unread,
Jan 8, 2012, 11:39:02 AM1/8/12
to django-d...@googlegroups.com


On Thursday, December 8, 2011 10:31:39 PM UTC+1, jo...@lophus.org wrote:

1.) I didn't write the code, I'm just submitting the patches in their
current state

Jonas: Do you have information on who wrote the code that you submitted? I guess a good way forward would be to find those people, and have them contribute a brief explanation of what problem their patch was trying to solve. For a "solution" to be accepted, you need to know the problem.

Jonas H.

unread,
Jan 8, 2012, 12:45:08 PM1/8/12
to django-d...@googlegroups.com

Waldemar Kornewald wrote most of the code but he's pretty busy these
days.

Cal Leeming [Simplicity Media Ltd]

unread,
Jun 23, 2012, 3:13:20 PM6/23/12
to django-d...@googlegroups.com
Hi Andres,

Afaik, there's currently some compatibility issues with Django 1.4 - so it's not currently stable.

Also, in my own personal opinion - after having a chance to use the mongo models with Django, in my personal opinion, it just didn't "feel right". Not entirely sure how to explain what that means, but for whatever reason the approach used just didn't feel like it was the best approach that could be used.

Would be interesting to hear what conclusions other people came too, especially any core devs involved in the ORM.

Cal

On Sat, Jun 23, 2012 at 5:30 AM, Andres Douglas <andres....@gmail.com> wrote:
Hey All,

I've been trying to use django with mongo and it seems like django-nonrel is still the best option out there. The only sad fact is that it's still not part of the official django. Looking for an answer as to why that was the case, I came across this thread. Are tests and docs all that are missing? Can someone provide pointers to what might be expected and considered acceptable?

It seems like a lot of the hard work has already been done figuring out what needed to change and the only thing left shouldn't be as hard - hopefully these are not famous last words.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/PmkY0RQA248J.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Cal Leeming [Simplicity Media Ltd]

unread,
Jun 24, 2012, 9:13:38 AM6/24/12
to django-d...@googlegroups.com
Hi Daniel,

Thanks for giving some feedback on this.

I completely agree that one of its biggest downfalls is that it tries to treat MongoDB as a relational store, and I think this is what I meant by it just didn't "feel right".

Also +1 on the comments made about it feeling hacky, and I suspect that if Mongo ever made it into the Django core, it would be redesigned and re-implemented from scratch.

I'd love to see Mongo in the Django core one day, but not as django-nonrel.

Cal

On Sun, Jun 24, 2012 at 5:47 AM, Daniel Greenfeld <pyd...@gmail.com> wrote:


We evaluated django-nonrel for use in projects and looked again at
django-nonrel for our talk at DjangoCon Europe. To summarize our
findings and opinions:

1. django-nonrel is stuck on Django 1.3, which has some security implications.
2. django-nonrel is unsupported. It switched maintainers and the
current maintainer is not working on it.
3. [pydanny opinion warning] django-nonrel wasn't adopted in Django
core because it lacked adequate documentation and tests.
4. [pydanny opinion warning] django-nonrel treats MongoDB as a
relational store, which it most certainly is not.
5. [pydanny opinion warning] django-nonrel smells like a giant hack
done by very well intentioned people..

Hope that helps,

--
'Knowledge is Power'
Daniel Greenfeld
http://pydanny.com


--
You received this message because you are subscribed to the Google Groups "Django developers" group.

Adam "Cezar" Jenkins

unread,
Jun 24, 2012, 10:31:41 AM6/24/12
to django-d...@googlegroups.com
I'm only lightly involved in the project, but there is some misinformation going around about it.

On Sun, Jun 24, 2012 at 5:47 AM, Daniel Greenfeld 
<pyd...@gmail.com> wrote:


We evaluated django-nonrel for use in projects and looked again at
django-nonrel for our talk at DjangoCon Europe. To summarize our
findings and opinions:

1. django-nonrel is stuck on Django 1.3, which has some security implications.

1.4 support is implemented and being tested in a beta mode right now.
 
2. django-nonrel is unsupported. It switched maintainers and the
current maintainer is not working on it.

Django-nonrel is very much maintained. Look at the github https://github.com/django-nonrel/
 
3. [pydanny opinion warning] django-nonrel wasn't adopted in Django
core because it lacked adequate documentation and tests.

This is true. 
 
4. [pydanny opinion warning] django-nonrel treats MongoDB as a
relational store, which it most certainly is not.

The idea is to use the same ORM as you would any other DB, so yes the ORM does make some assumptions. Though you get Dict/ListFields, Embedded Models, etc.
 
5. [pydanny opinion warning] django-nonrel smells like a giant hack
done by very well intentioned people..

I have to agree with you here. The nature of the Django ORM makes anything like this a hack. Not much choice in that regard.

I've wondered if it's possible to make an ORM that API compatible with forms/admin/etc that doesn't hack onto the existing ORM?

Russell Keith-Magee

unread,
Jun 24, 2012, 7:33:34 PM6/24/12
to django-d...@googlegroups.com
On Sun, Jun 24, 2012 at 10:31 PM, Adam "Cezar" Jenkins
<empero...@gmail.com> wrote:
> I'm only lightly involved in the project, but there is some misinformation
> going around about it.
>
> On Sun, Jun 24, 2012 at 5:47 AM, Daniel Greenfeld <pyd...@gmail.com> wrote:
>>
>>
>>
>> We evaluated django-nonrel for use in projects and looked again at
>> django-nonrel for our talk at DjangoCon Europe. To summarize our
>> findings and opinions:
>>
>> 1. django-nonrel is stuck on Django 1.3, which has some security
>> implications.
>
>
> 1.4 support is implemented and being tested in a beta mode right now.
>
>>
>> 2. django-nonrel is unsupported. It switched maintainers and the
>> current maintainer is not working on it.
>
>
> Django-nonrel is very much maintained. Look at the
> github https://github.com/django-nonrel/
>
>>
>> 3. [pydanny opinion warning] django-nonrel wasn't adopted in Django
>> core because it lacked adequate documentation and tests.
>
>
> This is true.

It's not the only reason that django-nonrel wasn't adopted. Another
big reason is that django-nonrel made/makes a number of major
structural changes to the internals of Django's ORM. Some of these
changes would be backwards incompatible, and some are major changes
that haven't been adequately explained.

For example, the last time I looked at the problem, there was *at
least* one *major* problem that appeared to require a backwards
incompatible change in order to work. Specifically, Django currently
makes the assumption that AutoFields are integers, and on many
non-relational stores (IIRC MongoDB is one on this list), the
automatically allocated key is a UUID. This poses a problem because
Django needs to have a way to work out which key to use, and despite
the efforts of django-nonrel and Alex Gaynor's GSoC project, I don't
believe an elegant solution has been proposed.

>> 4. [pydanny opinion warning] django-nonrel treats MongoDB as a
>> relational store, which it most certainly is not.
>
>
> The idea is to use the same ORM as you would any other DB, so yes the ORM
> does make some assumptions. Though you get Dict/ListFields, Embedded Models,
> etc.

Many of these are available on good relational stores, anyway
(PostgreSQL has plenty of exotic column types).

The issue is more about the fact that relational stores assume that
joins are relatively cheap; non-relational stores (broadly speaking)
don't. As a result, the way you frame your problem will be different.
A third normal form deconstruction of a data space works really well
with a relational database, but will not perform well if you try to
implement it on a non-relational store. There are things that
relational stores do well, and there are things that non-relational
stores do well; trying to make a non-relational store behave like a
relational store means you're going to get the advantages of neither.

>> 5. [pydanny opinion warning] django-nonrel smells like a giant hack
>> done by very well intentioned people..
>
>
> I have to agree with you here. The nature of the Django ORM makes anything
> like this a hack. Not much choice in that regard.
>
> I've wondered if it's possible to make an ORM that API compatible with
> forms/admin/etc that doesn't hack onto the existing ORM?

There's an extent to which "Does Django support non-rel" is really
just asking "Can I use the admin and Django Forms with non-rel"; so
rather than trying to plug a non-relational store in as an ORM
backend, it may be better to find a higher level point of API
compatibility.

However, I'm not sure there would be that much difference between a
"Mongo ORM Backend" and a "Django ORM API compatible replacement".
The ORM is already split into a frontend/backend API. The frontend is
all the public facing filter/exclude type calls, which any
ORM-replacement would need to emulate, so if you were to write
something that was API compatible, you're just going to end up
duplicating a lot of front-end code.

That said, it would get you around some of the backwards
incompatibility problems, and it would allow you to introduce API
points that aren't "relational" in their intention.

Yours,
Russ Magee %-)

Adam "Cezar" Jenkins

unread,
Jun 24, 2012, 9:35:40 PM6/24/12
to django-d...@googlegroups.com
This is my biggest issue with nonrel at this point. The API is generic for multiple backends (mongo and appengine). To do so there are layers of abstraction. 

In general, I think if you have a backend that's only 95% compatible you can lose a lot of good things. I have an app that is helped out a lot by something like mongo. For instance an app that allows students to apply to a university. Storing someone's application as a single document has much less maintenance headaches than a much more normalized schema that includes a dozen models. This isn't a high scalability scenario though. So I'd also like to use other 3rd party apps like django-socialauth without have to do too much hacking onto it. 

If I wanted to use a good bit of the 3rd party apps out there, I'd have to do a non-trivial amount of hacking to get it done. Django-nonrel helps with most of this, but since dealing with a nonrel db backend against the django ORM is a mess out of necessity, it's really hard to go in and impliment something like manytomany emulation. Which is quite possible in theory and useful in non-scalable scenarios.

It would be great to have something that really only requires exchanging the import of django.db.model with nonrel.db.models. If the only interaction you have to deal with is a higher level API, then maybe the backend code could be more flexible and maintainable?
 
Yours,
Russ Magee %-)

Daniel Greenfeld

unread,
Jun 26, 2012, 4:34:01 PM6/26/12
to django-d...@googlegroups.com


On Sunday, June 24, 2012 10:31:41 PM UTC+8, Cezar Jenkins wrote:
I'm only lightly involved in the project, but there is some misinformation going around about it.

On Sun, Jun 24, 2012 at 5:47 AM, Daniel Greenfeld 
<pyd...@gmail.com> wrote:


We evaluated django-nonrel for use in projects and looked again at
django-nonrel for our talk at DjangoCon Europe. To summarize our
findings and opinions:

1. django-nonrel is stuck on Django 1.3, which has some security implications. 
1.4 support is implemented and being tested in a beta mode right now.

I don't mean to pick bones, but looking at the github account, BETA apparently means being a port with stern warnings about not using it right now.
 
2. django-nonrel is unsupported. It switched maintainers and the
current maintainer is not working on it.
Django-nonrel is very much maintained. Look at the github https://github.com/django-nonrel/
 
This does not match what someone claiming to be the current lead maintainer said to me personally after my talk in front of dozens at DjangoCon Europe. 

While I understand this sort of misinformation I may have spread is annoying to you, imagine how it comes to the rest of us when project leadership is not clear. 

1. http://django-nonrel.readthedocs.org lacks any sort of team/contributor listings.
2. https://github.com/django-nonrel/mongodb-engine/blob/master/AUTHORS.rst has people in it who may no longer maintain it.

What's going on? What can django-nonrel do to make it much more clear to everyone who is involved and who is the actual leadership?

I have to agree with you here. The nature of the Django ORM makes anything like this a hack. Not much choice in that regard.

I've wondered if it's possible to make an ORM that API compatible with forms/admin/etc that doesn't hack onto the existing ORM?

In my humble opinion, the answer is no. ORM is for mapping objects onto relational databases, and the entire API is designed around that concept. MongoDB is a Document store and should be handled differently.  Django-nonrel adds a lot of magic, not necessarily in code, but in API design. It turns non-relational databases into relational databases, IMO removing many of the advantages of those systems for little gain except for in early speed of development.

The trick is, as I believe others have raised, is how to get Admin and Forms and other things in Django to work with MongoDB. :-)

Danny Greenfeld

Adam "Cezar" Jenkins

unread,
Jun 26, 2012, 5:07:42 PM6/26/12
to django-d...@googlegroups.com
Note that I've talked about the issues that nonrel has, and there are many. That said, I have some responses below. I'm not trying to be the nonrel spokesperson.

On Tue, Jun 26, 2012 at 3:34 PM, Daniel Greenfeld <pyd...@gmail.com> wrote:

I don't mean to pick bones, but looking at the github account, BETA apparently means being a port with stern warnings about not using it right now.

It's not tested enough for production yet, no. Though there are those testing it right now, at least on AppEngine. On the thread I link later on you can see the discussion.
 
 
2. django-nonrel is unsupported. It switched maintainers and the
current maintainer is not working on it.
Django-nonrel is very much maintained. Look at the github https://github.com/django-nonrel/
 
This does not match what someone claiming to be the current lead maintainer said to me personally after my talk in front of dozens at DjangoCon Europe. 

While I understand this sort of misinformation I may have spread is annoying to you, imagine how it comes to the rest of us when project leadership is not clear. 

Jonas is hesitant to lead the project, and I understand where he's coming from. For a long time there were only Mongo people maintaining the project. In a project where 80% of the users are using it for AppEngine, that can be a problem for hammering out bugs.

If you look at the thread at https://groups.google.com/forum/?fromgroups#!searchin/django-non-relational/1.4/django-non-relational/_OFjwuPjc88/iqLqnEGauNwJ you'll see there are people doing active development on non-relational. 

Last I heard Jonas wanted to do a new release of mongodb-engine. He might have meant to say he was not maintaining the project anymore, but he's not the only one with commit access.

 

1. http://django-nonrel.readthedocs.org lacks any sort of team/contributor listings.
2. https://github.com/django-nonrel/mongodb-engine/blob/master/AUTHORS.rst has people in it who may no longer maintain it.

What's going on? What can django-nonrel do to make it much more clear to everyone who is involved and who is the actual leadership?

There's a big documentation issue. It's known. Migration for All Button Pressed to the new "rag tag" team is slow, but I'm sure it'll get there.
 

I have to agree with you here. The nature of the Django ORM makes anything like this a hack. Not much choice in that regard.

I've wondered if it's possible to make an ORM that API compatible with forms/admin/etc that doesn't hack onto the existing ORM?

In my humble opinion, the answer is no. ORM is for mapping objects onto relational databases, and the entire API is designed around that concept. MongoDB is a Document store and should be handled differently.  Django-nonrel adds a lot of magic, not necessarily in code, but in API design. It turns non-relational databases into relational databases, IMO removing many of the advantages of those systems for little gain except for in early speed of development.

The trick is, as I believe others have raised, is how to get Admin and Forms and other things in Django to work with MongoDB. :-)

I have to agree, that's the big one. Though I the ecosystem of 3rd party apps is what makes using Django so great. If there is one ORM for  99% of the apps out there, and only one that works Mongo, then the only real use case of Django is going to be a system where you're using a RDBM
in combination with Mongo.
 

Danny Greenfeld

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/VktFJetdkWIJ.

Chris Northwood

unread,
Jun 26, 2012, 5:11:45 PM6/26/12
to django-d...@googlegroups.com
On 26 June 2012 22:07, Adam "Cezar" Jenkins <empero...@gmail.com> wrote:

> I have to agree, that's the big one. Though I the ecosystem of 3rd party
> apps is what makes using Django so great. If there is one ORM for  99% of
> the apps out there, and only one that works Mongo, then the only real use
> case of Django is going to be a system where you're using a RDBM
> in combination with Mongo.

Surely that's what it needs though, an 'ORM' and an 'Object-Document
Mapper' at some point. What's the advantage of trying to shoehorn
MongoDB to work with an ORM, when it's not relational, and as it has
to be hidden behind the same abstraction layer as a RDBMS, you lose
the benefits of it being non-relational?

Chris

Luciano Pacheco

unread,
Jun 26, 2012, 5:44:59 PM6/26/12
to django-d...@googlegroups.com

I agree that they are 2 different concepts (ORM and documents), however the namings in Django API looks like it might work well even for handling documents <=> objects.

User.objects.all()
User.objects.filter(email=email)

Looking at this there is nothing telling about the relational concepts behind it.

Of  course for all relations operations (foreings, many to many, etc) the api might start to differ .

If Django has a support for the basic api but with admin and loging working I think is a good step forward. After that we can wait for a better common sense about relations between documents emerge. We have been waiting this common sense to emerge without any builtin support, maybe with the basic api would have more people involved with.

[].

Luciano Pacheco

Adam "Cezar" Jenkins

unread,
Jun 26, 2012, 5:46:38 PM6/26/12
to django-d...@googlegroups.com
On Tue, Jun 26, 2012 at 4:11 PM, Chris Northwood <cnort...@gmail.com> wrote:

Surely that's what it needs though, an 'ORM' and an 'Object-Document
Mapper' at some point. What's the advantage of trying to shoehorn
MongoDB to work with an ORM, when it's not relational, and as it has
to be hidden behind the same abstraction layer as a RDBMS, you lose
the benefits of it being non-relational?


In my experience, a good bit of your document will map over to relational just fine, and vice versa. There is a lot of similarities between a ManyToMany and a list of foreign keys. The intersection of relational and non-relational, at least with Mongo, is pretty large. Of course with other non-relational DBs (admittedly a large variety of DBs) you milage will vary. 

Even if you look at the libraries like mongoengine, the documents and API are similar to the ORMs. 

Remember that mongo has foreignkeys in the form of objectids.

 
Chris

--
You received this message because you are subscribed to the Google Groups "Django developers" group.

Jonas H.

unread,
Jun 26, 2012, 5:58:21 PM6/26/12
to django-d...@googlegroups.com
On 06/26/2012 11:07 PM, Adam "Cezar" Jenkins wrote:
> Last I heard Jonas wanted to do a new release of mongodb-engine. He might
> have meant to say he was not maintaining the project anymore, but he's not
> the only one with commit access.

The main thing is that I've lost interest in maintaining this huge
project on my own -- and to be honest, there haven't been too many
code/documentation contributions since the GitHub migration.

I tried to do a new MongoDB Engine release for a few times but every
time something got in my way (like Travis not working, University, ...).
Not too much support from other people here either.

I'm not blaming anyone here. It's just the current state of things.


In terms of the fundamental "MongoDB vs. ORM" debate, here are my 2
cents: It's okay if we only get to work 70% of the ORM and apps --
that's already a huge benefit for people working with MongoDB and
Python. Most apps don't use JOINs or could get rid of them with only a
handful of simple changes.

Cheers :-)
Jonas
Reply all
Reply to author
Forward
0 new messages