post_init() hook broken in 0.6.13?

[Roy Smith]

We have some bad data in our database (pre-existing our use of mongoengine).  Some documents have an integer for a field which should be a string.  I have a post_init() method which corrects these on the fly.

Between 0.6.12 and 0.6.13, this broke.  The attached script demonstrates the problem.  It runs correctly under 0.6.12.  On 0.6.13, it fails with:

python  bug.py

doc = {'_id': ObjectId('509a77ace14c4e6a7fbb94ff'), 'year': 2000}

Traceback (most recent call last):

  File "bug.py", line 34, in <module>

    song = Song.objects.get(id=doc['_id'])

  File "/home/roy/production/python/local/lib/python2.7/site-packages/mongoengine/queryset.py", line 783, in get

    result1 = self.next()

  File "/home/roy/production/python/local/lib/python2.7/site-packages/mongoengine/queryset.py", line 964, in next

    return self._document._from_son(self._cursor.next())

  File "/home/roy/production/python/local/lib/python2.7/site-packages/mongoengine/base.py", line 974, in _from_son

    Invalid data to create a `%s` instance.\n%s""".strip() % (cls._class_name, errors))

mongoengine.base.InvalidDocumentError: Invalid data to create a `Song` instance.

year - 'int' object has no attribute 'decode'

The problem is that my post_init() script doesn't get called in 0.6.13 (or perhaps the exception is thrown before post_init() gets a chance to run).

I'm running:

Python-2.7.3

Ubuntu 12.04.1 LTS

pymongo 2.3

MongoDB 2.2.0

---------------------------------------------------------------------

from pymongo import Connection

from mongoengine import *

from mongoengine import signals

connect('test')

class Song(Document):

    meta = {'collection': 'test_songs',

            'allow_inheritance' : False,

            }

    year = StringField(required=True)

    def __unicode__(self):

        return "id=%r, year=%r" % (self.id, self.year)

    @classmethod

    def post_init(cls, sender, document, **kwargs):

        print "post_init()"

        if isinstance(document.year, int):

            print "fixing %r" % document

            document.year = str(document.year)

# Hook-up post_init hook                                                                                                                                           

signals.post_init.connect(Song.post_init, sender=Song)

# Create invalid document through raw pymongo connection                                                                                                           

doc = {"year": 2000}

mongo = Connection()

mongo.test.test_songs.insert(doc)

print "doc = %s" % doc

# Retrieve that document via mongoengine                                                                                                                           

song = Song.objects.get(id=doc['_id'])

print "song = %s" % song

---------------------------------------------------------------------

--

Roy Smith

r...@panix.com

[Ross Lawley]

Hi Roy, 

I can see the reason:  https://github.com/MongoEngine/mongoengine/compare/v0.6.12...v0.6.13#L5R51

We try to decode it - this was fixed in the 0.7 branch - see: https://github.com/MongoEngine/mongoengine/blob/v0.7.5/mongoengine/fields.py#L49

Any reasons your not upgrading to 0.7?

Ross

[Roy Smith]

Hi Ross,

Thanks for the quick response.  I just tried 0.7.6, and confirm that it fixes this problem.

As for the upgrade, we tend to not upgrade stuff in production unless there's a good reason to.  We've been running 0.6.9 for a while.  I only tried something newer because we're playing with tastypie-mongoengine.  That needs 0.6.20 or newer, so that's what we tried.

--

Roy Smith

r...@panix.com