Using __getattr__ and __setattr__ with sqlalchemy

[Jacob Biesinger]

Hi all,

Are there any best practices/caveats for overloading attribute access with declarative sqlalchemy?  I feel like the checks in setattr and getattr are very hacky and are occasionally giving me strange surprises...

from sqlalchemy import Integer, String, Column, create_engine, PickleType

from sqlalchemy.orm import Session

from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class Foo(Base):

    __tablename__ = 'foo'

    id = Column(Integer, primary_key=True)

    _instance_dict = Column(PickleType(mutable=True))

    name = Column(String)

    def __init__(self, name):

        self._instance_dict = {}

        self.name = name

    def __setattr__(self, key, value):

        if (key in self.__dict__ or key in self.__class__.__dict__ or

                not hasattr(self, '_sa_instance_state') or

                'AssociationProxy' in key):

            # handle class-level attrs (like Columns) using default behavior

            print 'setattr normal', key, value

            Base.__setattr__(self, key, value)

        else:

            # regular python objects attached to as an attribute go to pickled instance dict

            print 'setattr other', key, value

            self._instance_dict[key] = value

    def __getattr__(self, key):

        # use default behavior for looking up the dicts themselves

        if key == '_instance_dict' and key not in self.__dict__:

            self._sa_instance_state.initialize(key)

            return getattr(self, key)

        # check for the key in regular python object dict and sqlobjects dict

        try:

            return self._instance_dict[key]

        except KeyError:

            return Base.__getattr__(self, key)

engine = create_engine('sqlite://', echo=True)

Base.metadata.create_all(engine)

session = Session(engine)

# create a new animal

a = Foo('aardvark')

a.test = 10

session.add(a)

session.commit()

session.close()

for foo in session.query(Foo):

    print foo, foo.name, foo.test

[Michael Bayer]

you can do what you're doing here though I might want to make a more succinct system of determining which attributes go to __dict__ and which go into the special _instance_dict you have there.

A good system for that kind of thing often includes things like this:

1. if the name has an underscore (_somename), it's not meant for the _instance_dict.     This allows all the special things like _sa_instance_state and whatnot to not be interfered with.

2. to determine if the given key is "mapped", a quick and dirty way to check that is via the mapping:

mapper = self.__mapper__

is_mapped = mapper.has_property(key)

3. The system for determining if a key is meant for _instance_dict or not is just in one place (like def _is_instance_dict_key()) so that __setattr__ and __getattr__ don't repeat themselves

4. the _is_instance_dict_key thing could also cache the yes/no answers in a dictionary tied to the class.

hope this helps !

[Tolstov Sergey]

Before update it worked. But today i have a exception AttributeError: type object 'Base' has no attribute '__getattr__'

[Mike Bayer]

send a complete MCVE please

On Thu, Mar 29, 2018 at 2:08 AM, Tolstov Sergey <whist...@gmail.com> wrote:
> Before update it worked. But today i have a exception AttributeError: type
> object 'Base' has no attribute '__getattr__'
>

> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+...@googlegroups.com.
> To post to this group, send email to sqlal...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

[Tolstov Sergey]

I'm sorry, not it work.

I cannot undestand how work this commands:

1)if (key in self.__dict__ or key in self.__class__.__dict__ or not hasattr(self, '_sa_instance_state') or 'AssociationProxy' in key):

    Base.__setattr__(self, key, value)

2) self._sa_instance_state.initialize(key)

Can you tell me, please

[Mike Bayer]

On Fri, Mar 30, 2018 at 3:14 AM, Tolstov Sergey <whist...@gmail.com> wrote:
> I'm sorry, not it work.
> I cannot undestand how work this commands:
> 1)if (key in self.__dict__ or key in self.__class__.__dict__ or not
> hasattr(self, '_sa_instance_state') or 'AssociationProxy' in key):
> Base.__setattr__(self, key, value)

the original author is attempting to allow all attributes that seem to
originate from SQLAlchemy's instrumentation to pass through without
any alternate behavior. It appears that by looking for "key in
self.__class__.__dict__" they are testing to see if this attribute
name is an exisitng Python descriptor
(https://docs.python.org/3/howto/descriptor.html). "not hasattr(self,
'_sa_instance_state')" I guess is trying to see if the object hasn't
been instrumented yet. in any case, the script way at the top of
this thread still works in SQLAlchemy 1.2 as long as you take out the
"mutable=True" from the PickleType.

Here's the original script simplified. perhaps you can work from this
for whatever it is you're trying to do.

from sqlalchemy import Integer, String, Column, create_engine, PickleType
from sqlalchemy.orm import Session
from sqlalchemy.ext.declarative import declarative_base

from sqlalchemy.ext.mutable import MutableDict

Base = declarative_base()


class Foo(Base):
__tablename__ = 'foo'
id = Column(Integer, primary_key=True)

_instance_dict = Column(MutableDict.as_mutable(PickleType()))

name = Column(String)

def __init__(self, name):
self._instance_dict = {}
self.name = name

def __setattr__(self, key, value):

if key == '_sa_instance_state' or key in dir(self.__class__):

# handle class-level attrs (like Columns) using default behavior

print('setattr normal', key, value)

Base.__setattr__(self, key, value)
else:
# regular python objects attached to as an attribute go to

# pickled instance dict
print('setattr other', key, value)

self._instance_dict[key] = value

def __getattr__(self, key):

try:

# check for the key in regular python object dict

# and sqlobjects dict

return self._instance_dict[key]
except KeyError:

# normal lookup
return object.__getattribute__(self, key)

engine = create_engine('sqlite://', echo=True)
Base.metadata.create_all(engine)

session = Session(engine)


# create a new animal
a = Foo('aardvark')
a.test = 10

session.add(a)
session.commit()
session.close()

for foo in session.query(Foo):

print(foo, foo.name, foo.test)

assert hasattr(foo, 'test')
assert hasattr(foo, 'name')
assert not hasattr(foo, 'blah')

> 2) self._sa_instance_state.initialize(key)
>
> Can you tell me, please
>