[mongodb-user] Python and SON Manipulator...

[Martin]

Hi,

I have problems getting a manipulator right.

This is the object I want to store:

class Model(object):
"""Data Object representing a Model in the databse"""

def __init__(self, name, shortname, price=None, vendor=None, guid=None):
if guid == None:
self.guid = uuid.uuid4()
self.name = name
self.shortname = shortname
if price == None:
self.price = decimal.Decimal(0)
else:
self.price = decimal.Decimal(price)
self.vendor = vendor

self.created = datetime.utcnow()
self.last_modified = datetime.utcnow()

Ok that won't work, since I have a custom object:

>>> from assetkit.model.model import Model
>>> from pymongo import Connection
>>> connection = Connection()
>>> connection.drop_database("custom_type_example")
>>> db = connection.custom_type_example
>>> c = db.model_collection
>>> m = Model(name="Model Name", shortname="MN")
>>> c.insert(m)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/martin/Projects/python-assetkit/lib/python2.5/site-packages/pymongo/collection.py",
line 242, in insert
docs = [self.__database._fix_incoming(doc, self) for doc in docs]
File "/home/martin/Projects/python-assetkit/lib/python2.5/site-packages/pymongo/database.py",
line 212, in _fix_incoming
son = manipulator.transform_incoming(son, collection)
File "/home/martin/Projects/python-assetkit/lib/python2.5/site-packages/pymongo/son_manipulator.py",
line 73, in transform_incoming
son["_id"] = ObjectId()
TypeError: 'unicode' object does not support item assignment
>>>


Let's create a transformer and try again:

class Transformer(SONManipulator):
def transform_incoming(self, son, collection):
"To my understanding son should be the instance of the model
class I want to save, right?"
print "SON %r" % (son, )
for elem in son:
print "ELEM: %r" % (elem, )
key = elem
value = son[elem]
if isinstance(value, Model):
son[key] = value
elif isinstance(value, dict):
pass
return son
def transform_outgoing(self, son, collection):
pass # yes this cannot work...


>>> from pymongo import Connection
>>> connection = Connection()
>>> connection.drop_database("custom_type_example")
>>> db = connection.custom_type_example
>>> db.add_son_manipulator(Transformer())
>>> c = db.model_collection
>>> m = Model(name="Model Name", shortname="MN")
>>> c.insert(m)
SON u'name'
ELEM: u'n'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/martin/Projects/python-assetkit/lib/python2.5/site-packages/pymongo/collection.py",
line 242, in insert
docs = [self.__database._fix_incoming(doc, self) for doc in docs]
File "/home/martin/Projects/python-assetkit/lib/python2.5/site-packages/pymongo/database.py",
line 212, in _fix_incoming
son = manipulator.transform_incoming(son, collection)
File "<stdin>", line 9, in transform_incoming
TypeError: string indices must be integers


obiously the SONs in my transformer are the names of the attributes in
my custom class...I have no idea how to proceed, I have a feeling I'm
on the completely wrong track here :)

Finally some version info:

$ mongo # running Debian/Squeeze
MongoDB shell version: 1.2.3
url: test
connecting to: test
type "help" for help
> db.version()
1.4.0
>


any help?

thanks,
Martin

--
http://www.xing.com/profile/Martin_Marcher
http://www.linkedin.com/in/martinmarcher

You are not free to read this message,
by doing so, you have violated my licence
and are required to urinate publicly. Thank you.

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Flaper87]

Hi Martin,

This[0] is an example showing how to do what you need.

The son attr could be anything, string, int, dict, list, tuple and so on, so you have to check the son type before transforming it. 

Take a look to the link bellow ;)

[0] http://github.com/FlaPer87/django-mongodb-engine/blob/master/django_mongodb_engine/mongodb/mongodb_serializer.py#L9

2010/5/6 Martin <mar...@marcher.name>

--
Flavio Percoco Premoli, A.K.A. [Flaper87]
http://www.flaper87.org
Usuario Linux registrado #436538
Geek by nature, Linux by choice, Archer of course.
Key Fingerprint: CFC0 C67D FF73 463B 7E55  CF43 25D1 E75B E2DB 15C7
The Solution to everything:
python -c "from struct import pack; print  pack('5b', (41*len('99')), pow(8,2)+20, 4900**0.5, range(78)[-1], 10)"

[Michael Dirolf]

Is your model class iterable? If so I think that would explain the
behavior you're seeing. Try inserting a list of instances instead
(even if it's just a single item list: insert([m])). If I'm
understanding things correctly I'd expect that to behave more like how
you expect.

The problem is that insert() checks for a dict instance. Otherwise it
tries to iterate whatever it's argument was and do a bulk insert.

[Martin]

Hi,

On Thu, May 6, 2010 at 17:17, Michael Dirolf <mi...@10gen.com> wrote:
> Is your model class iterable?

Yup. Copy/Paste error here's the full code:

class Model(object):
"""Data Object representing a Model in the databse"""

def __init__(self, name, shortname, price=None, vendor=None, guid=None):
if guid == None:
self.guid = uuid.uuid4()
self.name = name
self.shortname = shortname
if price == None:
self.price = decimal.Decimal(0)
else:
self.price = decimal.Decimal(price)
self.vendor = vendor
self.created = datetime.utcnow()
self.last_modified = datetime.utcnow()

def __iter__(self):
for elem in (u"name", u"shortname", u"price", u"vendor",
u"guid", u"created", u"last_modified"):
yield elem
def __getitem__(self, key):
return getattr(self, key)
def __repr__(self):
"""Internal representation of a Vendor object"""
return u"<Model('%s', '%s', '%s')>" % (
self.name,
self.shortname,
self.price)

> If so I think that would explain the
> behavior you're seeing. Try inserting a list of instances instead
> (even if it's just a single item list: insert([m])). If I'm
> understanding things correctly I'd expect that to behave more like how
> you expect.

Hmm No :) - The goal is to have a working

db.col.insert(m) # _and_
db.col.insert([m, m, m])

> The problem is that insert() checks for a dict instance. Otherwise it
> tries to iterate whatever it's argument was and do a bulk insert.

I had a quick look at:

http://github.com/FlaPer87/django-mongodb-engine/blob/master/django_mongodb_engine/mongodb/mongodb_serializer.py#L9

and I think I got part of my problem. Regarding the dict: Does that
mean if I implement the dict protocol things will "just work". If so
it should be mentioned in the docs. My personal feeling is that if I
can just subclass dict, or implement the dict protocol it's a lot
nicer than using a SONManipulator since that would keep the necessary
transformation inside my "Data Objects" - to me it feels like that is
where this stuff should be, but I'm just starting I may as well change
my opinion "real soon now"(TM) :)

regards,
Martin

[Michael Dirolf]

Well the tricky part is what happens around line 240 in collection.py:

> if isinstance(docs, dict):
> return_one = True
> docs = [docs]
>
> if manipulate:

> docs = [self.__database._fix_incoming(doc, self) for doc in docs]

We check for instances of dict and if we get one we then try to save
by itself. Otherwise we try to iterate (assuming we have a
list/iterable of documents). So the SONManipulator stuff is really
meant to be used with subclasses of dict.

One option (if you're implementing a model class already) would be to
just add a save / insert method on that model class. So instead of
doing db.collection.save(my_model) you could do my_model.save() and
have that do the appropriate transformation and save. I think that
would probably end up being cleaner than any other solution.