ext.hybrid.Comparator.property not implemented in SA2

[Lele Gaifax]

Hi all,

while testing one of my tools[1] against SA2 I found that one test is
failing, and couldn't figure out a proper workaround.

TL;DR

Why is the sqlalchemy.ext.hybrid.Comparator.property explicitly
raising a NotImplementedError?

The tool is a generic data loader, and the failing test exercises its
ability to load a variant of the so-called "generic foreign key"[2],
where one entity may be associated to any other.

Although I tend to dislike the approach, I've used it in the past, and
that is why the tool grown the ability to deal with them.

The difference between the SA example and my variant is very minimal,
basically that it uses a custom hybrid.Comparator to "reach" the related
object (the "parent" in the original example).

I verified that it (still) work as a standalone script[3], so I think
that the problem is in how my tool triggers the machinery behind the
"related_object" property.

Simply speaking, what to tool does is looping over a list of
dictionaries, that for the failing test[4] is the following YAML:

- entity: model.Customer
key: name
data:
- &customer
name: Best customer

- entity: model.Supplier
key: company_name
data:
- &supplier
company_name: ACME

- entity: model.Address
key:
- related_object
- street
data:
- related_object: *customer
street: 123 anywhere street
- related_object: *supplier
street: 321 long winding road

For each dictionary, it resolves the "entity" class name and then loops
over the "data" sequence: first it tries to locate an existing instance
using the "key" as filter, then either updating it or inserting a new
one.

Enabling echo I can thus see:

BEGIN (implicit)
SELECT customer.name AS customer_name, customer.id AS customer_id
FROM customer
WHERE customer.name = ?
[generated in 0.00013s] ('Best customer',)
INSERT INTO customer (name) VALUES (?)
[generated in 0.00009s] ('Best customer',)
data.yaml [model.Supplier]: 1 Elapsed Time: 0:00:00
SELECT supplier.company_name AS supplier_company_name, supplier.id AS supplier_id
FROM supplier
WHERE supplier.company_name = ?
[generated in 0.00010s] ('ACME',)
INSERT INTO supplier (company_name) VALUES (?)
[generated in 0.00009s] ('ACME',)
data.yaml [model.Address]: 2 Elapsed Time: 0:00:00
SELECT address.street AS address_street, address.city AS address_city, address.zip AS address_zip, address.object_kind AS address_object_kind, address.object_id AS address_object_id, address.id AS address_id
FROM address
WHERE (address.object_kind, address.object_id) = (?, ?) AND address.street = ?
[generated in 0.00011s] ('Customer', 1, '123 anywhere street')

At that point, given that the particular address does not exist, it
tries to create a new instance and the following error occurs:

Traceback (most recent call last):
File ".../src/metapensiero/sqlalchemy/dbloady/load.py", line 223, in workhorse
load(fname, session, dry_run, delete, save_new_instances,
File ".../src/metapensiero/sqlalchemy/dbloady/load.py", line 124, in load
for e in entity(session, idmap, adaptor):
File ".../src/metapensiero/sqlalchemy/dbloady/entity.py", line 85, in __call__
yield instance(session, self.delete, self.loadonly)
File ".../src/metapensiero/sqlalchemy/dbloady/entity.py", line 189, in __call__
attr_prop = getattr(model, f).property
File "...lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 645, in property
return self.comparator.property
File "...lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 1127, in __get__
return self.fget(obj)
File "...lib/python3.10/site-packages/sqlalchemy/ext/hybrid.py", line 1191, in property
raise NotImplementedError()
NotImplementedError

As you can see here[5], my code tries to obtain the "property" attribute
of the comparator to determine whether it should assign a scalar value
or an instance to the target.

In SA 2, the ext.hybrid.Comparator explicitly[6] defines the
"property" property as "not defined" (sorry for the pun), and I could
not figure out the rationale.

While I keep digging and trying, maybe you can shed some light, or
hinting about a different approach?

Thanks and bye, lele.

[1] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.dbloady
[2] https://docs.sqlalchemy.org/en/20/_modules/examples/generic_associations/generic_fk.html
[3] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.dbloady/-/snippets/2501165
[4] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.dbloady/-/tree/master/tests/generic_fk
[5] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.dbloady/-/blob/master/src/metapensiero/sqlalchemy/dbloady/entity.py#L189
[6] https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/ext/hybrid.py#L1190
--
nickname: Lele Gaifax | Dire che Emacs è "conveniente" è come
real: Emanuele Gaifas | etichettare l'ossigeno come "utile"
le...@etour.tn.it | -- Rens Troost

[Mike Bayer]

On Thu, Feb 16, 2023, at 7:16 AM, Lele Gaifax wrote:

Hi all,

while testing one of my tools[1] against SA2 I found that one test is

failing, and couldn't figure out a proper workaround.

TL;DR

  Why is the sqlalchemy.ext.hybrid.Comparator.property explicitly

  raising a NotImplementedError?

the .property attribute in the base PropComparator refers to an ORM instrumented attribute, which in the case of a custom Comparator does not exist.   This was None previously, but for typing purposes, now raises NotImplementedError since it should not be accessed.

    Traceback (most recent call last):

      File ".../src/metapensiero/sqlalchemy/dbloady/load.py", line 223, in workhorse

        load(fname, session, dry_run, delete, save_new_instances,

      File ".../src/metapensiero/sqlalchemy/dbloady/load.py", line 124, in load

        for e in entity(session, idmap, adaptor):

      File ".../src/metapensiero/sqlalchemy/dbloady/entity.py", line 85, in __call__

        yield instance(session, self.delete, self.loadonly)

      File ".../src/metapensiero/sqlalchemy/dbloady/entity.py", line 189, in __call__

        attr_prop = getattr(model, f).property

      File "...lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 645, in property

        return self.comparator.property

      File "...lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 1127, in __get__

        return self.fget(obj)

      File "...lib/python3.10/site-packages/sqlalchemy/ext/hybrid.py", line 1191, in property

        raise NotImplementedError()

    NotImplementedError

As you can see here[5], my code tries to obtain the "property" attribute

of the comparator to determine whether it should assign a scalar value

or an instance to the target.

In SA 2, the ext.hybrid.Comparator explicitly[6] defines the

"property" property as "not defined" (sorry for the pun), and I could

not figure out the rationale.

While I keep digging and trying, maybe you can shed some light, or

hinting about a different approach?

dont use getattr(), use methods like inspect(cls).attrs, inspect(cls).relationships and inspect(cls).all_orm_descriptors to get a view for what ORM things are associated with the class and which are relationships.    that all works in 1.4 also.

[Lele Gaifax]

"Mike Bayer" <mike_not_...@zzzcomputing.com> writes:

> On Thu, Feb 16, 2023, at 7:16 AM, Lele Gaifax wrote:
>>
>> TL;DR
>>
>> Why is the sqlalchemy.ext.hybrid.Comparator.property explicitly
>> raising a NotImplementedError?
>
> the .property attribute in the base PropComparator refers to an ORM
> instrumented attribute, which in the case of a custom Comparator does
> not exist. This was None previously, but for typing purposes, now
> raises NotImplementedError since it should not be accessed.

Ok, thank you for the clarification.

>
> dont use getattr(), use methods like inspect(cls).attrs,
> inspect(cls).relationships and inspect(cls).all_orm_descriptors to get
> a view for what ORM things are associated with the class and which are
> relationships. that all works in 1.4 also.

Perfect, I will find my way!

Thanks again,
ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
le...@metapensiero.it | -- Fortunato Depero, 1929.