Issue 218 in couchdb-python: couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by

[couchdb...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 218 by j.goodno...@gmail.com:
couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

What steps will reproduce the problem?
1. Make several ViewDefinitions, with at least two of them in the same
design document.
2. Call ViewDefinition.sync_many (this is called by Flask-CouchDB's
CouchDBManager's sync method)

What is the expected output? What do you see instead?

Expected Output:
- I would expect that views that belong to the same design document should
be merged together, with one update call for each design document.

Actual Output:
- It makes a doc for each view (even if they're in the same design
document), which ultimately leads to conflicts.


What version of the product are you using? On what operating system?
CouchDB 0.8

Please provide any additional information below.
This is happening because in sync_many, there is a itertools.groupby call
that is supposed to combine these together, However, as mentioned here:
http://docs.python.org/2/library/itertools.html#itertools.groupby

"The operation of groupby() is similar to the uniq filter in Unix. It
generates a break or new group every time the value of the key function
changes (which is why it is usually necessary to have sorted the data using
the same key function). That behavior differs from SQL’s GROUP BY which
aggregates common elements regardless of their input order."

So, because there is no sort call beforehand, it does not group as you
would expect it to.

The fix is to place this line:
views = sorted(views, key=attrgetter('design'))

right before:
for design, views in groupby(views, key=attrgetter('design')):

Diff:
--- venv/lib/python2.7/site-packages/couchdb/design.py 2013-01-29
00:03:30.000000000 -0800
+++ /otherdirectory/venv/lib/python2.7/site-packages/couchdb/design.py
2013-01-28 23:51:26.000000000 -0800
@@ -160,6 +160,7 @@
"""
docs = []

+ views = sorted(views, key=attrgetter('design'))
for design, views in groupby(views, key=attrgetter('design')):
doc_id = '_design/%s' % design
doc = db.get(doc_id, {'_id': doc_id})

I would have gladly have submitted a pull request, but it's sadly not
github. I did however confirm that this works locally. If you need my
actual views, let me know.

[couchdb...@googlecode.com]

Comment #1 on issue 218 by j.goodno...@gmail.com:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Also, since this is showing up as a bug in an application I'm building and
the fix is quick, getting this into pypi soon would be pretty sweet.

Thanks,

Johnny

[couchdb...@googlecode.com]

Comment #2 on issue 218 by djc.ochtman:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Sorry, I don't think your application is enough motivation for us to do
quick release (particularly since we haven't done a release in quite a
while). It would be useful if you can provide your patch with Mercurial
metadata (via hg export). It seems you might be reluctant to use Mercurial,
but it's really not hard.

[couchdb...@googlecode.com]

Comment #3 on issue 218 by j.goodno...@gmail.com:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Not a problem; I'll attach it later this evening.

On Tuesday, January 29, 2013, wrote:

[couchdb...@googlecode.com]

Comment #4 on issue 218 by j.goodno...@gmail.com:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

I've attached my patch. And I don't dislike Mercurial (I like git better,
but I started with Mercurial); I just wish Google Code had a "pull
request"-like mechanism such as what Github and Bitbucket has...and you can
always host Mercurial projects at Bitbucket + Docs at Read The Docs :D.

Thanks!

Attachments:
sync_many_fix.diff 1.1 KB

[couchdb...@googlecode.com]

Updates:
Status: Fixed

Comment #5 on issue 218 by djc.ochtman:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

This issue was closed by revision c8a11a9a2a59.

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[couchdb...@googlecode.com]

Comment #6 on issue 218 by djc.ochtman:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Sorry, that took slightly longer than planned.

It would be really cool if you could contribute a test for this behavior,
to make sure we don't regress it.

[couchdb...@googlecode.com]

Comment #7 on issue 218 by j.goodno...@gmail.com:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Sure; I'll look at the test code you already have and will try and submit
something this week.

[couchdb...@googlecode.com]

Comment #8 on issue 218 by j.goodno...@gmail.com:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Attached is the test; thanks!

[couchdb...@googlecode.com]

Comment #9 on issue 218 by j.goodno...@gmail.com:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Attached is the test; thanks!

Attachments:
218_test.txt 1.1 KB

[couchdb...@googlecode.com]

Comment #10 on issue 218 by djc.ochtman:

couchdb.design.ViewDefinition.sync_many doesn't sort views before group_by
http://code.google.com/p/couchdb-python/issues/detail?id=218

Very nice, thank you! This will (finally) be released shortly.