Adapting a library to SA 1.4b1

[Lele Gaifax]

Hi all,

I started to take a closer look at the new not-yet-released 1.4b1, adapting
one of my libraries, metapensiero.sqlalchemy.proxy [1].

As expected only one "side" feature, the async support provided by
metapensiero.sqlalchemy.asyncio [2], appears to be problematic, but as I do not
currently use in production I deferred the effort to a later time.

A very few other tiny problems arose, but one is tricking me, so I'm asking
advice.

An helper function, col_by_name [3], tries to find a particular column in a
selectable, looking first in the set of "exported_columns" (as it is called in
modern SA), failing that in the "froms" collection. It's main purpose is to
apply sorters and filters (typically coming from an external source such as a
web request, so expressed in terms of strings representing field names,
ordering directions and filter expressions) to a base query.

With SA 1.4b1 I have one single failing test, where the problem is in the way
the function walks into the internals of the selectable to look for a given
column. The test is very very simple, and basically wants to distill and
execute a query like

query([Person.firstname, Person.lastname])
.filter(or_(Person.firstname.like('%Lele'),
Person.lastname.like('%Lele')))

given an entity, "Person", a string to search and a subset of the columns that
shall be considered and extracted.

So, this is the test [4]:

def test_query():
proxy = ProxiedEntity(Person)

sas = Session()

> res = proxy(sas, query=u"Lele", fields="firstname,lastname,nickname")

where "firstname" and "lastname" are valid columns on the entity [5], while
"nickname" does not exist and shall be ignored. It is this unknown name that
causes the col_by_name() function to dig further into the set of "joined"
tables, should there be any. In this case of course there aren't, since the
query is on a single entity, but this is the traceback of an AttributeError:

tests/test_orm.py:174:
----------------------------
.tox/py38-sa14/lib/python3.8/site-packages/metapensiero/sqlalchemy/proxy/base.py:191: in __call__
start, limit) = self.prepareQueryFromConditionsAndArgs(session, conditions, args)
.tox/py38-sa14/lib/python3.8/site-packages/metapensiero/sqlalchemy/proxy/base.py:118: in prepareQueryFromConditionsAndArgs
query = self.filterQueryWithArgs(session, conditions, args)
.tox/py38-sa14/lib/python3.8/site-packages/metapensiero/sqlalchemy/proxy/orm.py:104: in filterQueryWithArgs
query, self.only_cols = apply_filters(query, args)
.tox/py38-sa14/lib/python3.8/site-packages/metapensiero/sqlalchemy/proxy/filters.py:402: in apply_filters
column = col_by_name(stmt, f)
.tox/py38-sa14/lib/python3.8/site-packages/metapensiero/sqlalchemy/proxy/utils.py:59: in col_by_name
for f in query.froms:
.tox/py38-sa14/lib/python3.8/site-packages/sqlalchemy/sql/selectable.py:4548: in froms
return self._compile_state_factory(self, None)._get_display_froms()
.tox/py38-sa14/lib/python3.8/site-packages/sqlalchemy/sql/selectable.py:3622: in _get_display_froms
froms = self.froms
.tox/py38-sa14/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:1045: in __getattr__
return self._fallback_getattr(key)
----------------------------

self = <sqlalchemy.orm.context.ORMSelectCompileState object at 0x7fa8574e8160>, key = 'froms'

def _fallback_getattr(self, key):
> raise AttributeError(key)
E AttributeError: froms

.tox/py38-sa14/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:1019: AttributeError

Is this something wrong the function is doing, or is some regression in 1.4?

Thanks a lot for any hint,
ciao, lele.

[1] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.proxy
[2] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.asyncio
[3] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.proxy/-/blob/master/src/metapensiero/sqlalchemy/proxy/utils.py#L33
[4] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.proxy/-/blob/master/tests/test_orm.py#L169
[5] https://gitlab.com/metapensiero/metapensiero.sqlalchemy.proxy/-/blob/master/tests/fixture.py#L35
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
le...@metapensiero.it | -- Fortunato Depero, 1929.

[Mike Bayer]

Hi Lele -

It looks like .froms on a simple select() that is against ORM entities is just broken here, so we can fix that.   at the moment it's a one liner but there are some performance implications that might make it more tricky but you can look at https://github.com/sqlalchemy/sqlalchemy/issues/5614 where I am tracking this.

-- 

SQLAlchemy - 

The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full description.

--- 

You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/87h7rj1q83.fsf%40metapensiero.it.

[Lele Gaifax]

"Mike Bayer" <mik...@zzzcomputing.com> writes:

> Hi Lele -
>
> It looks like .froms on a simple select() that is against ORM entities is just broken here, so
> we can fix that. at the moment it's a one liner but there are some performance implications
> that might make it more tricky but you can look at
> https://github.com/sqlalchemy/sqlalchemy/issues/5614 where I am tracking this.

Thank you Mike!

ciao, lele.

[Mike Bayer]

I pushed the fix for that, give it another go.   this is good testing you're doing as you have a lot of SQL Core metaprogramming going on.  things are REALLY different on the inside :)

-- 

SQLAlchemy - 

The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full description.

--- 

You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/87d0271bbk.fsf%40metapensiero.it.

[Lele Gaifax]

"Mike Bayer" <mik...@zzzcomputing.com> writes:

> I pushed the fix for that, give it another go. this is good testing you're
> doing as you have a lot of SQL Core metaprogramming going on. things are
> REALLY different on the inside :)

Great, thanks again Mike: this indeed fixed my test suite!