Disabling implicit lazy loading

[Yegor Roganov]

Hi all!
Is there a way to disable implicit loading of relationships?
For example, I want an exception to be thrown if I try to access 'address.user' unless user was explicitly loaded via options

address = query(Address).options(joinedload(Address.user)).filter_by(id=id).first(); address.user   # OK
address = query(Address).get(id); address.user # should throw

At first I thought that `noload` option is what I need, but it seems it disables event explicit loading.
If you are intrested why I need this kind of functionality, that's because I found it hard to manage which relationships were loaded and which not, and it may result in dozens of unwanted DB queries.

[Mike Bayer]

On 7/23/15 1:24 AM, Yegor Roganov wrote:

Hi all!
Is there a way to disable implicit loading of relationships?
For example, I want an exception to be thrown if I try to access 'address.user' unless user was explicitly loaded via options

address = query(Address).options(joinedload(Address.user)).filter_by(id=id).first(); address.user   # OK
address = query(Address).get(id); address.user # should throw

At first I thought that `noload` option is what I need, but it seems it disables event explicit loading.

noload is how you'd disable implicit loading.   As far as throwing on a lazyload, the easiest way is just to detach the objects from their parent Session so they no longer have any connectivity using session.expunge(object), but then you're no longer in the session.

Otherwise, it seems the problem you are actually trying to solve is raising on unexpected SQL.   lazy loading of relationships is not the only thing that goes on, there are loads of unloaded columns, columns that had server defaults emitted on the last flush, loads of joined-inheritance rows, all kinds.     this is why the best approach is to just do real profiling of your applications using SQL logging, or perhaps using SQL events like before_execute()  / before_cursor_execute() so that you can build yourself a "with assert_no_sql(session):" -style context manager for critical blocks that should have no SQL emitted.


Guessing that's not what you want.     Feel free to write your own NoLoader that just raises, example:

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import properties
from sqlalchemy.orm import strategies


@properties.RelationshipProperty.strategy_for(lazy="raise")
class RaiseLoader(strategies.NoLoader):
    """note: this is *very SQLAlchemy 1.0 specific*!!   it will need to be reviewed for 1.1"""

    def create_row_processor(
            self, context, path, loadopt, mapper,
            result, adapter, populators):
        def invoke_no_load(state, dict_, row):
            raise Exception("boom")
        populators["new"].append((self.key, invoke_no_load))

Base = declarative_base()


class A(Base):
    __tablename__ = 'a'
    id = Column(Integer, primary_key=True)
    bs = relationship("B", lazy="raise")


class B(Base):
    __tablename__ = 'b'
    id = Column(Integer, primary_key=True)
    a_id = Column(ForeignKey('a.id'))

e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)

s = Session(e)
s.add(A(bs=[B(), B()]))
s.commit()

a1 = s.query(A).first()
a1.bs   # boom


send me a PR that includes tests and I can consider this for 1.1.

If you are intrested why I need this kind of functionality, that's because I found it hard to manage which relationships were loaded and which not, and it may result in dozens of unwanted DB queries.

--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.
To post to this group, send email to sqlal...@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

[Mike Bayer]

On 7/23/15 10:09 AM, Mike Bayer wrote:

On 7/23/15 1:24 AM, Yegor Roganov wrote:

Hi all!
Is there a way to disable implicit loading of relationships?
For example, I want an exception to be thrown if I try to access 'address.user' unless user was explicitly loaded via options

address = query(Address).options(joinedload(Address.user)).filter_by(id=id).first(); address.user   # OK
address = query(Address).get(id); address.user # should throw

At first I thought that `noload` option is what I need, but it seems it disables event explicit loading.

noload is how you'd disable implicit loading.   As far as throwing on a lazyload, the easiest way is just to detach the objects from their parent Session so they no longer have any connectivity using session.expunge(object), but then you're no longer in the session.

Otherwise, it seems the problem you are actually trying to solve is raising on unexpected SQL.   lazy loading of relationships is not the only thing that goes on, there are loads of unloaded columns, columns that had server defaults emitted on the last flush, loads of joined-inheritance rows, all kinds.     this is why the best approach is to just do real profiling of your applications using SQL logging, or perhaps using SQL events like before_execute()  / before_cursor_execute() so that you can build yourself a "with assert_no_sql(session):" -style context manager for critical blocks that should have no SQL emitted.


Guessing that's not what you want.     Feel free to write your own NoLoader that just raises, example:

whoops.  Let's try that again, this one actually works:

from sqlalchemy.orm import properties
from sqlalchemy.orm import strategies

from sqlalchemy.orm import state

@properties.RelationshipProperty.strategy_for(lazy="raise")
class RaiseLoader(strategies.NoLoader):
    """note: this is *very SQLAlchemy 1.0 specific*!!
    it will need to be reviewed for 1.1"""

    def create_row_processor(
            self, context, path, loadopt, mapper,
            result, adapter, populators):

        def invoke_no_load(state, passive):
            raise Exception("boom")
        set_lazy_callable = state.InstanceState.\
            _instance_level_callable_processor(
                mapper.class_manager,
                invoke_no_load,
                self.key
            )
        populators["new"].append((self.key, set_lazy_callable))

[Yegor Roganov]

Thanks, you are as always very helpful. I hope I'll be able to compose a PR soon.