reflected declarative versus single table inheritance
Chris Withers
I'm hitting some issues with single table inheritance and the reflected
declarative example interacting badly...
I'm not sure I have my head around this 100% and certainly no idea what
to do about it, so thought I'd see what others (Mike ;-) ) think...
IIUC, the following breaks:
class GeneralType(Reflected):
__tablename__ = 'the_table'
__mapper_args__ = dict(polymorphic_on='col')
class TypeOne(GeneralType):
__mapper_args__ = dict(polymorphic_identity='one')
...firstly because we don't have a column object to pass to
polymorphic_on (shame that and primary_key don't take a string(s)), but
in my interpretation of the recipe I've worked around that.
However, the one that's stumped me is an InvalidRequestError saying
'the_table' is already defined for this MetaData instance.
I believe that this because there's some dancing around using
_is_mapped_class that gets confused when declarative reflection is used,
because the actual mapping is deferred until later.
Any suggestions on this front?
My thinking was leaning towards actually abandoning the full declarative
base and coming up with a light weight one (or maybe even class
decorator) that basically added recorded the class in a sequence (much
like the declarative reflection does) and then calls
instrument_declarative with each class at the point of relfection.
Does that sound sane?
cheers,
Chris
--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
Chris Withers
> My thinking was leaning towards actually abandoning the full declarative
> base and coming up with a light weight one (or maybe even class
> decorator) that basically added recorded the class in a sequence (much
> like the declarative reflection does) and then calls
> instrument_declarative with each class at the point of relfection.
Well, it's a bit vom-tastic but I got this working. Here it is minus the
imports:
# The registries
meta = MetaData()
registry = {}
# The declarative bases
class ReflectedMeta(type):
_classes_to_process = []
def __new__(meta, classname, bases, classDict):
cls = type.__new__(meta, classname, bases, classDict)
meta._classes_to_process.append(cls)
return cls
@classmethod
def prepare(meta, engine, metadata):
for cls in meta._classes_to_process:
tablename = vars(cls).get('__tablename__')
if tablename is not None:
try:
table = Table(
tablename,
MetaData(),
autoload=True,
autoload_with=engine,
)
except NoSuchTableError:
continue
for col in table.c:
if getattr(cls, col.name, None) is None:
col = col.copy()
if col is None:
raise Exception()
setattr(cls, col.name, col)
kw = vars(cls).get('__mapper_args__')
if kw is not None:
# deal with cases where we need to specify the primary key
# structure
primary_key_names = kw.pop('primary_key_names', None)
if primary_key_names:
kw['primary_key'] = pk = []
for name in primary_key_names:
pk.append(getattr(cls, name))
# Likewise for polymorphic_on
polymorphic_on_name = kw.pop('polymorphic_on_name', None)
if polymorphic_on_name:
kw['polymorphic_on'] = getattr(cls,
polymorphic_on_name)
instrument_declarative(cls, registry, metadata)
# The this one for reflected models:
class Reflected(object):
__abstract__ = True
__metaclass__ = ReflectedMeta
# Use this one for concretely mapped models:
Base = declarative_base(metadata=meta, class_registry=registry)
Then, once I have a connection and I'm ready to reflect, I do:
ReflectedMeta.prepare(engine, meta)
Is the above sane?
Michael Bayer
On Feb 9, 2012, at 1:33 PM, Chris Withers wrote:
> On 09/02/2012 15:47, Chris Withers wrote:
>> My thinking was leaning towards actually abandoning the full declarative
>> base and coming up with a light weight one (or maybe even class
>> decorator) that basically added recorded the class in a sequence (much
>> like the declarative reflection does) and then calls
>> instrument_declarative with each class at the point of relfection.
OK I'm not sure if you are on 0.7.5 but the string name of the column is accepted for "polymorphic_on". The recipe just needs to fill in the 'inherits' keyword within prepare(), and that's it. See below:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.orm.util import _is_mapped_class
from sqlalchemy.ext.declarative import declarative_base, declared_attr
class DeclarativeReflectedBase(object):
_mapper_args = []
@classmethod
def __mapper_cls__(cls, *args, **kw):
"""Declarative will use this function in lieu of
calling mapper() directly.
Collect each series of arguments and invoke
them when prepare() is called.
"""
cls._mapper_args.append((args, kw))
@classmethod
def prepare(cls, engine):
"""Reflect all the tables and map !"""
while cls._mapper_args:
args, kw = cls._mapper_args.pop(0)
klass = args[0]
# autoload Table, which is already
# present in the metadata. This
# will fill in db-loaded columns
# into the existing Table object.
if args[1] is not None:
table = args[1]
Table(table.name,
cls.metadata,
extend_existing=True,
autoload_replace=False,
autoload=True,
autoload_with=engine,
schema=table.schema)
for c in klass.__bases__:
if _is_mapped_class(c):
kw['inherits'] = c
break
klass.__mapper__ = mapper(*args, **kw)
Base = declarative_base()
class Reflected(DeclarativeReflectedBase, Base):
__abstract__ = True
class GeneralType(Reflected):
__tablename__ = 'single_inherits'
__mapper_args__ = dict(polymorphic_on='type')
class TypeOne(GeneralType):
__mapper_args__ = dict(polymorphic_identity='one')
class TypeTwo(GeneralType):
__mapper_args__ = dict(polymorphic_identity='two')
e = create_engine('sqlite://', echo=True)
e.execute("""
create table single_inherits(
id integer primary key,
type varchar(30)
)
""")
Reflected.prepare(e)
s = Session(e)
s.add_all([
TypeOne(type="one"), TypeOne(type="one"), TypeTwo(type="two")
])
s.commit()
s.close()
print s.query(GeneralType).all()
Chris Withers
>
> OK I'm not sure if you are on 0.7.5
Yep, this project is.
> but the string name of the column is accepted for "polymorphic_on".
Cool, any plans for primary key?
> The recipe just needs to fill in the 'inherits' keyword within prepare(), and that's it. See below:
What I'm using is below:
class DeclarativeReflectedBase(object):
_mapper_args = []
@classmethod
def __mapper_cls__(cls, *args, **kw):
"""Declarative will use this function in lieu of
calling mapper() directly.
Collect each series of arguments and invoke
them when prepare() is called.
"""
cls._mapper_args.append((args, kw))
@classmethod
def prepare(cls, engine):
"""Reflect all the tables and map !"""
while cls._mapper_args:
args, kw = cls._mapper_args.pop(0)
klass = args[0]
# autoload Table, which is already
# present in the metadata. This
# will fill in db-loaded columns
# into the existing Table object.
if args[1] is not None:
table = args[1]
try:
Table(
table.name,
cls.metadata,
extend_existing=True,
autoload_replace=False,
autoload=True,
autoload_with=engine,
schema=table.schema
)
except NoSuchTableError:
continue
for c in klass.__bases__:
if _is_mapped_class(c):
kw['inherits'] = c
break
# deal with cases where we need to specify the primary key
# structure
primary_key_names = kw.pop('primary_key_names', None)
if primary_key_names:
kw['primary_key'] = pk = []
for name in primary_key_names:
pk.append(getattr(table, name))
klass.__mapper__ = mapper(*args, **kw)
# Use this one for concretely mapped models:
Base = declarative_base()
# The this one for reflected models:
class Reflected(DeclarativeReflectedBase, Base):
__abstract__ = True
Attempting to import the module containing the single table inheritance
models gives:
Traceback (most recent call last):
File "sqlalchemy/ext/declarative.py", line 1336, in __init__
_as_declarative(cls, classname, cls.__dict__)
File "sqlalchemy/ext/declarative.py", line 1235, in _as_declarative
**table_kw)
File "sqlalchemy/schema.py", line 290, in __new__
"existing Table object." % key)
sqlalchemy.exc.InvalidRequestError: Table 'the_table' is already defined
for this MetaData instance. Specify 'extend_existing=True' to redefine
options and columns on an existing Table object.
NB: This is before any call to prepare...
Michael Bayer
On Feb 10, 2012, at 4:33 AM, Chris Withers wrote:
> On 09/02/2012 19:47, Michael Bayer wrote:
>>
>> OK I'm not sure if you are on 0.7.5
>
> Yep, this project is.
>
>> but the string name of the column is accepted for "polymorphic_on".
>
> Cool, any plans for primary key?
not sure specifically what the issue is there but perhaps you're dealing with http://www.sqlalchemy.org/trac/ticket/2402 ? there's a fix to primary key overrides there.