Validators not called for unhashable items when replacing the collection

[Pedro Werneck]

I have a relationship with a validator to automatically convert dicts appended to the collection, so I can do something like this:

my_obj.my_collection.append({"rel_type_id": x})

Instead of this:

my_obj.my_collection.append(RelType(rel_type_id=x))

That works exactly as expected, but when I try to replace the whole collection at once:

my_obj.my_collection = [{"rel_type_id": x}]

That results in a TypeError: unhashable type: 'dict', and the validator method is never called. Apparently that happens when the orm.collection.bulk_replace function uses sets to find the difference between the old and the new collection. I don't see an straightforward fix for that, it feels more like a limitation of the current implementation than a bug.

It looks like I could do what I want with a custom collection and the collection.converter decorator. Any other ideas?

Thanks.

[mike bayer]

On 01/24/2017 09:55 PM, Pedro Werneck wrote:
>
> I have a relationship with a validator to automatically convert dicts
> appended to the collection, so I can do something like this:
>
> my_obj.my_collection.append({"rel_type_id": x})
>
> Instead of this:
>
> my_obj.my_collection.append(RelType(rel_type_id=x))
>
> That works exactly as expected, but when I try to replace the whole
> collection at once:
>
> my_obj.my_collection = [{"rel_type_id": x}]
>
> That results in a TypeError: unhashable type: 'dict', and the validator
> method is never called. Apparently that happens when the
> orm.collection.bulk_replace function uses sets to find the difference
> between the old and the new collection. I don't see an straightforward
> fix for that, it feels more like a limitation of the current
> implementation than a bug.

that's kind of beyond bug and more a design flaw. The bulk replace
wants to hit the event listener only for "new" items, but we can't
decide on the "new" items without running the event handler. The whole
bulk replace idea would need to be changed to run the event listeners up
front which suggests new events and whatnot.


So here you'd need to use the "converter" implementation as well
(http://docs.sqlalchemy.org/en/latest/orm/collections.html#sqlalchemy.orm.collections.collection.converter),
here's a demo, unfortunately we need to mix both styles for complete
coverage:

class MyCollection(list):

@collection.converter
def convert(self, value):
return [B(data=v['data']) for v in value]


class A(Base):
__tablename__ = 'a'
id = Column(Integer, primary_key=True)

bs = relationship("B", collection_class=MyCollection)

@validates('bs')
def _go(self, key, value):
if not isinstance(value, B):
value = B(data=value['data'])
return value


class B(Base):
__tablename__ = 'b'
id = Column(Integer, primary_key=True)
a_id = Column(ForeignKey('a.id'))
data = Column(String)


I think in the future, what might be nice here would be a new attribute
event so that "converter" doesn't need to be used, and then @validates
can include @validates.collection_validate or similar to handle this
case. The collection hooks are generally assuming that they are
dealing with how an incoming object should be represented within the
collection, not how to coerce an incoming value (e.g. I tried to use
@collection.appender here for the individual appends, no go), so
"converter" being where it is, and not at value reception time, is
inconsistent. The amount of collection hooks present compared to how
not possible this use case is is kind of a disaster.

I've added
https://bitbucket.org/zzzeek/sqlalchemy/issues/3896/bulk_replace-assumes-incoming-values-are.

>
> It looks like I could do what I want with a custom collection and the
> collection.converter decorator. Any other ideas?
>
>
> Thanks.
>
>

> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google
> Groups "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sqlalchemy+...@googlegroups.com
> <mailto:sqlalchemy+...@googlegroups.com>.
> To post to this group, send email to sqlal...@googlegroups.com
> <mailto:sqlal...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

[Pedro Werneck]

Yes, I noticed the collection.decorator would imply doing the same
validation in two different places. That will have to work for now,
but I'll keep an eye on the issue for 1.2.

Thanks Mike.

> --- You received this message because you are subscribed to a topic in the
> Google Groups "sqlalchemy" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sqlalchemy/Utkrott4e0g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> sqlalchemy+...@googlegroups.com.
> To post to this group, send email to sqlal...@googlegroups.com.

> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

--
---
Pedro Werneck