DeclarativeBase and multiple inheritance

328 views
Skip to first unread message

Daishy

unread,
Dec 13, 2009, 12:09:16 PM12/13/09
to sqlalchemy
Hi together,

I'm sorry if this is a rather stupid question, but i havent found a
solution yet :/
I have a few models build with the DeclarativeBase-Class. Now each of
these models has a few columns that they have in common (created_by,
created_at, updated_by, updated_at). Rather than putting the code i
need for these columns and the columns itself in each model, i wanted
to put it into a seperate class and inherit from it (or include it).
But if i query ModelA i just get '(no name)' if i print the created_X
columns. Is there any way to extract columns each model should have
into a seperat class?


class Data:
created_by = Column(Integer)
created_at = Column(DateTime)


class ModelA(DeclarativeBase, Data):
id = Column(Integer, primary_key=True)
somethingelse = Column(Integer)


Thanks very much!
Daishy

Michael Bayer

unread,
Dec 13, 2009, 8:06:45 PM12/13/09
to sqlal...@googlegroups.com

On Dec 13, 2009, at 12:09 PM, Daishy wrote:

> Hi together,
>
> I'm sorry if this is a rather stupid question, but i havent found a
> solution yet :/
> I have a few models build with the DeclarativeBase-Class. Now each of
> these models has a few columns that they have in common (created_by,
> created_at, updated_by, updated_at). Rather than putting the code i
> need for these columns and the columns itself in each model, i wanted
> to put it into a seperate class and inherit from it (or include it).
> But if i query ModelA i just get '(no name)' if i print the created_X
> columns. Is there any way to extract columns each model should have
> into a seperat class?

The Column objects you declare within declarative become members of the underlying Table object, so its not as simple as just having those members present on a mixin - what would really be needed would be some sort of copying of each column object when declarative comes across them.

The mechanism of declarative is also to look at the "dict" of only the endpoint class itself when picking up the attributes to add to the Table/mapper, so at the moment any mixin would be ignored in any case.

There is an alternative approach to the "place these columns on every class" problem, which involves making a subclass of the metaclass that adds the desired columns when a new class is created. I don't prefer this method since we're usually talking about just a few columns, the metaclass itself is slightly awkward, and at the end of the day I do like to see the columns listed out on each class (hence "declarative"...).

What I usually do is to reduce the overhead by creating callables for the desired columns (usually the same created_at etc. columns you're using here):

def created_at():
return Column(DateTime, default=func.now)

so you only need to add a little bit to each class:

class Foo(Base):
...

created_at = created_at()
updated_at = updated_at()

The "metaclass subclass" approach is somewhere in the list archives if you want to search for it.

Also, it wouldn't be impossible for declarative to someday be enhanced to accept "mixins", which probably would subclass a specific class to mark them, and would be allowed to declare a limited set of "copyable" attributes, namely Column objects and perhaps deferred()s also. I haven't tried working out such a feature as of yet, however.



>
>
> class Data:
> created_by = Column(Integer)
> created_at = Column(DateTime)
>
>
> class ModelA(DeclarativeBase, Data):
> id = Column(Integer, primary_key=True)
> somethingelse = Column(Integer)
>
>
> Thanks very much!
> Daishy
>
> --
>
> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> To post to this group, send email to sqlal...@googlegroups.com.
> To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>
>

Daishy

unread,
Dec 14, 2009, 3:58:22 AM12/14/09
to sqlalchemy
Hi,

I think you are right, messing with the metaclass for 4 columns is not
really what i want :)
Instead i will use your approach. Put those Columns in the function
within the mixin along with all other functions and then call them
within the column-definition.

Thanks for fast answer!
Greetings,
Daishy

Chris Withers

unread,
Dec 17, 2009, 1:59:42 PM12/17/09
to sqlal...@googlegroups.com
(sorry, missed this earlier)

Michael Bayer wrote:
> The Column objects you declare within declarative become members of
> the underlying Table object, so its not as simple as just having
> those members present on a mixin - what would really be needed would
> be some sort of copying of each column object when declarative comes
> across them.

Indeed, how about a marker on "abstract" base classes like the Django ORM?

> The mechanism of declarative is also to look at the "dict" of only
> the endpoint class itself when picking up the attributes to add to
> the Table/mapper, so at the moment any mixin would be ignored in any
> case.

That's a shame, are there plans to change that approach and iterate over
dir() of the endpoint class instead?

> There is an alternative approach to the "place these columns on
> every class" problem, which involves making a subclass of the
> metaclass that adds the desired columns when a new class is created.

What about the cls parameter to declarative_base? Someone on #sqlalchemy
recommended that and linked to some working code:

http://progetti.arstecnica.it/sol/browser/sol/model/base.py

Is there a problem with having more than one Base in a project?

> I don't prefer this method since we're usually talking about just a
> few columns, the metaclass itself is slightly awkward, and at the
> end of the day I do like to see the columns listed out on each class
> (hence "declarative"...).

Yeah, but duplicate typing is always bad... That's why python allows
inheritance...

> def created_at():
> return Column(DateTime, default=func.now)
>
> so you only need to add a little bit to each class:
>
> class Foo(Base):
> ...
>
> created_at = created_at()
> updated_at = updated_at()

Yeah, I've used this approach too, but when it's several columns, and a
few utility methods, a class really seems like the right (dare I say
pythonic?) place for it to be?

How about something like:

class Foo(Base):

__abstract__ = True

...

...which would provide the desired functionality here?

> The "metaclass subclass" approach is somewhere in the list archives
> if you want to search for it.

Advising end users to write metaclasses is always bad in my books :-(
(In fact, this all came up 'cos a colleague of mine was reaching for one
of our python books, I asked why, she replied "to remind myself how
metaclasses work", to which my reply was "you *never* want to do that" ;-) )

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk

Daishy

unread,
Dec 17, 2009, 2:35:34 PM12/17/09
to sqlalchemy
Hi again,

I took the approach with creating a Mixin with 4 staticmethods and
call them in each of my models, which works quite good. But now i face
another, i think related, problem.
i get the following columns produced:

created_by_id = Column(Integer, ForeignKey('worker.id',
onupdate="cascade", ondelete="restrict"))
updated_by_id = Column(Integer, ForeignKey('worker.id',
onupdate="cascade", ondelete="restrict"))
created_by = relation('worker')
updated_by = relation('worker')

Now sqlalchemy is throwing an exception, because the join-conditions
are not clear. Ok, so i have to add a primaryjoin-expression on the
relations. But how?
They should be specified like relation('worker',
primaryjoin=worker.c.id==???.c.created_by_id), but in my Mixin-Class i
dont know what ??? is, because it changes depending on the class using
the mixin. I could specify a parameter for these two static methods,
but thats another step making things more complicated, so i want to
avoid that if possible. Any advice?

Thanks!
Daishy

Michael Bayer

unread,
Dec 17, 2009, 8:32:35 PM12/17/09
to sqlal...@googlegroups.com

On Dec 17, 2009, at 1:59 PM, Chris Withers wrote:

> (sorry, missed this earlier)
>
> Michael Bayer wrote:
>> The Column objects you declare within declarative become members of
>> the underlying Table object, so its not as simple as just having
>> those members present on a mixin - what would really be needed would
>> be some sort of copying of each column object when declarative comes
>> across them.
>
> Indeed, how about a marker on "abstract" base classes like the Django ORM?

im sorry, did you say you were feeling slightly sick earlier ? :)


>
>> The mechanism of declarative is also to look at the "dict" of only
>> the endpoint class itself when picking up the attributes to add to
>> the Table/mapper, so at the moment any mixin would be ignored in any
>> case.
>
> That's a shame, are there plans to change that approach and iterate over
> dir() of the endpoint class instead?

probably not. I can see that causing more problems than it solves.

I noticed you cut out the part of my reply with the likely fix here, to provide a special class that is specifically for declarative mixins. It would only allow elements on it that make sense for copying to many subclasses.

Michael Bayer

unread,
Dec 17, 2009, 8:33:47 PM12/17/09
to sqlal...@googlegroups.com

On Dec 17, 2009, at 2:35 PM, Daishy wrote:

> Hi again,
>
> I took the approach with creating a Mixin with 4 staticmethods and
> call them in each of my models, which works quite good. But now i face
> another, i think related, problem.
> i get the following columns produced:
>
> created_by_id = Column(Integer, ForeignKey('worker.id',
> onupdate="cascade", ondelete="restrict"))
> updated_by_id = Column(Integer, ForeignKey('worker.id',
> onupdate="cascade", ondelete="restrict"))
> created_by = relation('worker')
> updated_by = relation('worker')
>
> Now sqlalchemy is throwing an exception, because the join-conditions
> are not clear. Ok, so i have to add a primaryjoin-expression on the
> relations. But how?
> They should be specified like relation('worker',
> primaryjoin=worker.c.id==???.c.created_by_id), but in my Mixin-Class i
> dont know what ??? is, because it changes depending on the class using
> the mixin. I could specify a parameter for these two static methods,
> but thats another step making things more complicated, so i want to
> avoid that if possible. Any advice?

right. see why I recommend just placing the Column objects on each class explicitly ?

Kevin Ar18

unread,
Dec 17, 2009, 8:56:27 PM12/17/09
to sqlal...@googlegroups.com
Basic question:
I have 1 app with multiple processes and threads.  Each thread and/or process may end up trying to do something to the database at the same time.  What is the solution to threading?  How do web frameworks solve it?  Is there some inherent design in databases and/or SQLAlchemy that makes this not a actual problem?   What do I do if I have 2 separate apps?  3?

I know there has got to be a simple answer to this (after all, every web app has to deal with it), but I can't find anywhere that explains it....

Details:
Ok, so let's say I am developing an app with the potential for many threads and processes.  The problem is what do I do when each of those threads and processes need to access the database in different ways?  read data, change data, etc...

The SQLAlchemy documentation says that Sessions are not thread safe and talk about using my own locking mechanisms.  However, if I recall correctly, many web frameworks don't even deal with threading issues, yet allow interaction with the database across many threads (http server responses).

The PostgreSQL documention on the subject didn't help much: http://www.postgresql.org/docs/8.4/static/mvcc.html except to show that I really don't want to have anything to do with managing the complexities of multiple connections to a database.
Pylons docs were equally as vague as the SQLAlchemy docs: essentially that some vague thing is not thread safe in some vague way.

Ok, so here's what I can gather.  SQLAlchemy is setup like so:

1 Session => 1 DB Connection (from connection pool) => Database

* If I setup 2 sessions, that means I will have 2 connections to the database, meaning the database is the one that will handle threading conflicts per what is described here: http://www.postgresql.org/docs/8.4/static/mvcc.html

I really don't like relying on that.
However, the SQLAlchemy docs suggest making many small Sessions for short interaction with the database.  The problem is if I have many threads/processes accessing the DB simultaneously (through separate sessions\connections), then I could get into threading problems working with the database, right?

Is the solution to use one single session for the entire application (the application consisting of many processes and threads) and making all my processes communicate with this single session when it needs to do database stuff?

Somehow, I have my doubts that it how it should be done -- and even the SQLAlchemy docs say otherwise (although again, very vague about what it is talking about).


So, I have 1 app with multiple processes and threads.  Each thread and/or process may end up trying to do something to the database at the same time.  What is the solution to threading?  How do web frameworks solve it?  Is there some inherent design in databases and/or SQLAlchemy that makes this not a actual problem?   What do I do if I have 2 separate apps?  3?


Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign up now.

Michael Bayer

unread,
Dec 17, 2009, 10:34:35 PM12/17/09
to sqlal...@googlegroups.com
On Dec 17, 2009, at 8:56 PM, Kevin Ar18 wrote:

* If I setup 2 sessions, that means I will have 2 connections to the database, meaning the database is the one that will handle threading conflicts per what is described here: http://www.postgresql.org/docs/8.4/static/mvcc.html

So, I have 1 app with multiple processes and threads.  Each thread and/or process may end up trying to do something to the database at the same time.  What is the solution to threading?  How do web frameworks solve it?  Is there some inherent design in databases and/or SQLAlchemy that makes this not a actual problem?   What do I do if I have 2 separate apps?  3?

it really is that - simultaneous transactions are isolated from one another, and their results are only made visible to other transactions after they're committed.  as far as locking,  you generally choose between an "optimistic" and a "pessimistic" approach:


as far as the in-memory state of key components like connections, sessions and cursors, the general approach is to share nothing between threads.  this is what pretty much every web framework ultimately ends up doing - all the state/etc. is per thread.

its a large subject of discussion so hopefully some other list members can expound upon some of this.







I really don't like relying on that.
However, the SQLAlchemy docs suggest making many small Sessions for short interaction with the database.  The problem is if I have many threads/processes accessing the DB simultaneously (through separate sessions\connections), then I could get into threading problems working with the database, right?

Is the solution to use one single session for the entire application (the application consisting of many processes and threads) and making all my processes communicate with this single session when it needs to do database stuff?

Somehow, I have my doubts that it how it should be done -- and even the SQLAlchemy docs say otherwise (although again, very vague about what it is talking about).




Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign up now.

--

Kevin Ar18

unread,
Dec 18, 2009, 8:45:21 PM12/18/09
to sqlal...@googlegroups.com
it really is that - simultaneous transactions are isolated from one another, and their results are only made visible to other transactions after they're committed.  as far as locking,  you generally choose between an "optimistic" and a "pessimistic" approach:

Thanks.
Ok, so I found an article that talks about the different locking types for databases:
* lock everything
* don't lock but check if the data changed on you and fix it if it did
* don't care -- only use for single db connection

Now, where does SQLAlchemy fit into all of this?  Also, I notice PostgreSQL had 2 different locking strategies you can use and the info I am looking at now suggests I am going to have do some complex database handling to check for data integrity.  Of course, the whole point of SQLAlchemy is to abstract away from such low level database interaction... is it not?

Since I'm guessing everyone who uses a db must deal with this problem that much mean there is some information I am missing or something I don't understand that everyone else out there just knows about.  :)


Hotmail: Trusted email with powerful SPAM protection. Sign up now.

Michael Bayer

unread,
Dec 18, 2009, 10:12:06 PM12/18/09
to sqlal...@googlegroups.com
The method that works 95% of the time is, just establish a connection to the database, start a transaction, do some work that stays within the thread, commit the transaction.  Then throw all loaded state in that thread away, and reload it all on the next transaction.     Do this in as many concurrent threads or processes as you like.   In typical cases, you don't have to worry about taking specific steps towards locking - PG does a very good job of isolating concurrent sessions out of the box.     Its only if you expect to have many concurrent threads/processes all contending for the same row in a table, or if you have to insert a row which other threads/processes might be trying to insert at exactly the same time that you have to consider different outcomes.   And even then, you still don't have to really "do" anything except have proper constraints on your tables - PG will ensure that invalid data doesn't enter the DB.   But the "losers" in a contention battle will get an exception.   So if you have a problem with threads/processes losing the battle for contention, and you'd like to be more graceful about it, then you can start strategizing around it.   But its not something I worry about too much until it becomes a problem.

A tool like SQLAlchemy doesn't have too much to do with any of this, except to not get in one's way when attempting to use transaction isolation.   It similarly doesn't really "do" anything regarding concurrency or isolation - PG does an incredible job at that on its own.   You get a Session, it will open a connection the first time you use it, and thats it, until you rollback or commit.    Modern versions ensure that once a brand new transaction is started, any data thats currently in the session from a previous transaction is considered to be invalid, and will be loaded again once it is accessed - this since once your transaction is complete, other transactions can now get a hold of it and change it further.    

I'd also advise going through the session docs carefully for a full overview http://www.sqlalchemy.org/docs/05/session.html .   

You also might be interested to read Hibernate's document at http://docs.jboss.org/hibernate/stable/core/reference/en/html/transactions.html, since the SQLA session was largely based off that of Hibernate.   The general idea there is the same as that of SQLAlchemy, though some specifics are different, but they do reiterate a lot of the same concepts.










Hotmail: Trusted email with powerful SPAM protection. Sign up now.

--

Kevin Ar18

unread,
Dec 19, 2009, 11:30:21 AM12/19/09
to sqlal...@googlegroups.com
The method that works 95% of the time is, just establish a connection to the database, start a transaction, do some work that stays within the thread, commit the transaction.  Then throw all loaded state in that thread away, and reload it all on the next transaction.
....

Its only if you expect to have many concurrent threads/processes all contending for the same row in a table, or if you have to insert a row which other threads/processes might be trying to insert at exactly the same time that you have to consider different outcomes.   And even then, you still don't have to really "do" anything except have proper constraints on your tables - PG will ensure that invalid data doesn't enter the DB.


Thanks for the reply.  You brought up two topics I am curious about.  Maybe you can give me a little more details, so I know what to search for to find more information on these topics?


> throw all loaded state in that thread away, and reload it all on the next transaction.
>have proper constraints on your tables - PG will ensure that invalid data doesn't enter the DB.
Where can I learn more about these topics and how they relate to threading issues?  What should I be searching for to learn about these?

Also, any suggestions on something that specifically describes how these threading db issues work as a whole (from basic to advanced so I can be sure my usage is going to be thread safe and won't fail because I happened to use one of the few weird ways that can really mess things up).

Michael Bayer

unread,
Dec 19, 2009, 12:53:00 PM12/19/09
to sqlal...@googlegroups.com
Hey guys -

I forgot about the really obvious way to do this - class decorators:

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

def common_columns(cls):
cls.created_at = Column(DateTime, default=func.now)
cls.udpated_at = Column(DateTime, default=func.now, onupdate=func.now)
return cls

@common_columns
class MyClass(Base):
__tablename__ = 'mytable'

id = Column(Integer, primary_key=True)
data = Column(String)

@common_columns
class MyOtherClass(Base):
__tablename__ = 'myothertable'

id = Column(Integer, primary_key=True)
data = Column(String)

engine = create_engine('sqlite://', echo=True)

Base.metadata.create_all(engine)

Chris Withers

unread,
Dec 30, 2009, 11:03:33 AM12/30/09
to sqlal...@googlegroups.com, petra...@glcuk.com
Michael Bayer wrote:
>> Michael Bayer wrote:
>>> The Column objects you declare within declarative become members of
>>> the underlying Table object, so its not as simple as just having
>>> those members present on a mixin - what would really be needed would
>>> be some sort of copying of each column object when declarative comes
>>> across them.
>> Indeed, how about a marker on "abstract" base classes like the Django ORM?
>
> im sorry, did you say you were feeling slightly sick earlier ? :)

An abstract marker on a base class to say "actually, this is just here
to provide information and methods, it's not table-mapped class" isn't
such a bad thing, is it? Especially given how horrible the resulting
code is...

> I noticed you cut out the part of my reply with the likely fix here, to provide a special class that is specifically for declarative mixins. It would only allow elements on it that make sense for copying to many subclasses.

I did indeed miss this, but I don't see how it's fundamentally different
from marking a base class as abstract ;-)

Anyway, here's the thought process that got us where we are (which feels
a bit icky to me), any ways to make things "nicer" would be very
gratefully received.

Okay, so, the naive approach that a normal python user might expect to
work would be:

Base = declarative_base()

class VersionedBase(Base):
id = Column(Integer, primary_key=True)
version = Column(Integer, nullable=False, index=True)
a_constant = ('some','tuple')
def aMethod(self):
do_some_stuff()

class Employee(VersionedBase):
__tablename__ = 'employee'
name = Column(String, nullable=False, index=True)

...but this results in:

sqlalchemy.exc.InvalidRequestError: Class <class
'__main__.VersionedBase'> does not have a __table__ or __tablename__
specified and does not inherit from an existing table-mapped class.

Fair enough, so why not use multi-table inheritance? Because we really
want those columns in each table:

- no overhead of joins when selecting a particular set of versions
- ability to bypass the orm layer more easilly if emergency debugging is
needed

Okay, so why not concrete or single table inheritance? Because we
*really* want one table per subclass but we don't ever want to select
from more than one table at once...

So, moving on, we tried:

Base = declarative_base()

class VersionedBase(object):
id = Column(Integer, primary_key=True)
version = Column(Integer, nullable=False, index=True)
a_constant = ('some','tuple')
def aMethod(self):
do_some_stuff(self.a_constant)

class Employee(Base, VersionedBase):
__tablename__ = 'employee'
name = Column(String, nullable=False, index=True)

...which gave a more cryptic error:

sqlalchemy.exc.ArgumentError: Mapper Mapper|Employee|employee could not
assemble any primary key columns for mapped table 'employee'.

I'm guessing this is because VersionedBase is basically ignored by the
declarative metaclass?

Okay, so metaclass time... First attempt failed:

def aMethod(self):
do_some_stuff(self.a_constant)

class VersionedMeta(DeclarativeMeta):
def __init__(cls, name, bases, dict_):
cls.id = Column(Integer, primary_key=True)
cls.version = Column(Integer, nullable=False, index=True)
cls.a_constant = ('some','tuple')
cls.aMethod = aMethod
return DeclarativeMeta.__init__(cls, name, bases, dict_)

...because DeclarativeMeta's __init__ ignores columns already set on cls
by the time it's called, which is fair enough, but means we end up with
a (working) metaclass method that's a bit ugly:

def aMethod(self):
do_some_stuff(self.a_constant)

class VersionedMeta(DeclarativeMeta):
def __init__(cls, name, bases, dict_):
dict_.update(dict(
id = Column(Integer, primary_key=True),
version = Column(Integer, nullable=False, index=True),
))
cls.a_constant = ('some','tuple')
cls.aMethod = aMethod
return DeclarativeMeta.__init__(cls, name, bases, dict_)

def versioned_declarative_base():
return declarative_base(metaclass=VersionedMeta)

Base = version_declarative_base()

class Employee(Base):
__tablename__ = 'employee'
name = Column(String, nullable=False, index=True)

Is it okay to have multiple declarative bases floating around like this?
Will they all end up using the same metadata collection or are we in for
problems down the line?

So anyway, on to your next suggestion: class decorator...

def aMethod(self):
do_some_stuff(self.a_constant)

def versioned(cls):
cls.id = Column(Integer, primary_key=True),
cls.version = Column(Integer, nullable=False, index=True),
cls.a_constant = ('some','tuple')
cls.aMethod = aMethod

@versioned
class Employee(Base):
__tablename__ = 'employee'
name = Column(String, nullable=False, index=True)

Much nicer! Except... we're on Python 2.5, so no class decorators. No
problems, thinks I, we'll just do it the old fashioned way:

class Employee(Base):
__tablename__ = 'employee'
name = Column(String, nullable=False, index=True)
Employee = versioned(Employee)

Uh oh:

sqlalchemy.exc.ArgumentError: Mapper Mapper|Employee|employee could not
assemble any primary key columns for mapped table 'employee'

...which makes me wonder if the class decorator would actually work at
all. Surely it'll only kick in after the DeclarativeMeta has already
done its thing and got upset?

Ideas welcome...

Chris

Michael Bayer

unread,
Dec 30, 2009, 1:40:42 PM12/30/09
to sqlal...@googlegroups.com
Chris Withers wrote:
> Michael Bayer wrote:
>>> Michael Bayer wrote:
>>>> The Column objects you declare within declarative become members of
>>>> the underlying Table object, so its not as simple as just having
>>>> those members present on a mixin - what would really be needed would
>>>> be some sort of copying of each column object when declarative comes
>>>> across them.
>>> Indeed, how about a marker on "abstract" base classes like the Django
>>> ORM?
>>
>> im sorry, did you say you were feeling slightly sick earlier ? :)
>
> An abstract marker on a base class to say "actually, this is just here
> to provide information and methods, it's not table-mapped class" isn't
> such a bad thing, is it? Especially given how horrible the resulting
> code is...
>
>> I noticed you cut out the part of my reply with the likely fix here, to
>> provide a special class that is specifically for declarative mixins. It
>> would only allow elements on it that make sense for copying to many
>> subclasses.
>
> I did indeed miss this, but I don't see how it's fundamentally different
> from marking a base class as abstract ;-)

its not. I was still recovering from seeing the word "django" :)

> Uh oh:
>
> sqlalchemy.exc.ArgumentError: Mapper Mapper|Employee|employee could not
> assemble any primary key columns for mapped table 'employee'
>
> ...which makes me wonder if the class decorator would actually work at
> all. Surely it'll only kick in after the DeclarativeMeta has already
> done its thing and got upset?

the mapper is mostly configured by the time the class is done, and the
primary key columns are required to be there. The overwhelming use case
we're talking about here are columns like "created_at", "updated_at",
etc., not primary keys.

But agreed, make your mixin implement a marker class from the declarative
module (MixinsDeclarative, i dunno), and as_declarative() will issue
copy() for the Column objects within. It will raise errors for relation()
since it doesn't make sense to copy those.

Chris Withers

unread,
Dec 30, 2009, 2:47:34 PM12/30/09
to sqlal...@googlegroups.com
Michael Bayer wrote:
>> Uh oh:
>>
>> sqlalchemy.exc.ArgumentError: Mapper Mapper|Employee|employee could not
>> assemble any primary key columns for mapped table 'employee'
>>
>> ...which makes me wonder if the class decorator would actually work at
>> all. Surely it'll only kick in after the DeclarativeMeta has already
>> done its thing and got upset?
>
> the mapper is mostly configured by the time the class is done, and the
> primary key columns are required to be there. The overwhelming use case
> we're talking about here are columns like "created_at", "updated_at",
> etc., not primary keys.

Hmm, well, the primary key column is, in this case, one that wants to be
shared across all classes...

> But agreed, make your mixin implement a marker class from the declarative
> module (MixinsDeclarative, i dunno),

How about just a marker attribute? How about something in
__mapper__args__, which already exists?

class MyMixin:

__mapper_args__ = {'abstract':True}

> and as_declarative() will issue
> copy() for the Column objects within. It will raise errors for relation()
> since it doesn't make sense to copy those.

I assume this is a feature waiting to be implemented?

What to do in the meantime? The class decorator felt really quite neat
except that it won't work where primary key columns are added by the
decorator, right?

Michael Bayer

unread,
Dec 30, 2009, 2:51:07 PM12/30/09
to sqlal...@googlegroups.com
Chris Withers wrote:
>> But agreed, make your mixin implement a marker class from the
>> declarative
>> module (MixinsDeclarative, i dunno),
>
> How about just a marker attribute? How about something in
> __mapper__args__, which already exists?
>
> class MyMixin:
>
> __mapper_args__ = {'abstract':True}

well mapper_args is args that go to the mapper. Its a little unclean
IMHO to reuse that same dict for declarative purposes. Id rather have
another attribute like __declarative_abstract__ = True or something like
that (ideas welcome).

>
> I assume this is a feature waiting to be implemented?
>
> What to do in the meantime? The class decorator felt really quite neat
> except that it won't work where primary key columns are added by the
> decorator, right?

yeah the class decorator can't really handle the PKs. the mapper wants
those when it starts up. as far as implementation this feature isn't on
my personal radar, so if you really want anytime soon you should provide a
patch, which I can of course help with.

>
> Chris
>
> --
> Simplistix - Content Management, Batch Processing & Python Consulting
> - http://www.simplistix.co.uk
>

Chris Withers

unread,
Jan 1, 2010, 5:09:26 PM1/1/10
to sqlal...@googlegroups.com
Michael Bayer wrote:
> well mapper_args is args that go to the mapper. Its a little unclean
> IMHO to reuse that same dict for declarative purposes. Id rather have
> another attribute like __declarative_abstract__ = True or something like
> that (ideas welcome).

__declaratice_abstract__ = True works for me :-)

>> What to do in the meantime? The class decorator felt really quite neat
>> except that it won't work where primary key columns are added by the
>> decorator, right?
>
> yeah the class decorator can't really handle the PKs. the mapper wants
> those when it starts up.

Metaclass it is then. I asked a while back but I think it got lost in
the melee: is it okay to have several declarative bases knocking around
or will they each end up with their own metadata collection causing
problems when models from one have relations to models from another?

> as far as implementation this feature isn't on
> my personal radar, so if you really want anytime soon you should provide a
> patch, which I can of course help with.

Cool, it is on my radar, but sadly pretty far away :-(

Michael Bayer

unread,
Jan 1, 2010, 10:51:12 PM1/1/10
to sqlal...@googlegroups.com

you know, doing the "metaclass" approach is probably the first part of doing the "demarcated mixin" approach.

here it is: http://www.sqlalchemy.org/trac/wiki/UsageRecipes/DeclarativeMixins


Chris Withers

unread,
Jan 12, 2010, 1:57:41 PM1/12/10
to sqlal...@googlegroups.com
Michael Bayer wrote:
>>> as far as implementation this feature isn't on
>>> my personal radar, so if you really want anytime soon you should provide a
>>> patch, which I can of course help with.
>> Cool, it is on my radar, but sadly pretty far away :-(
>
> you know, doing the "metaclass" approach is probably the first part of doing the "demarcated mixin" approach.
>
> here it is: http://www.sqlalchemy.org/trac/wiki/UsageRecipes/DeclarativeMixins

Indeed, it'd just be great if the default declarative base mixin handled
this demarcation package already ;)

I argue about the comment on relations on that page, if the relation is
identical, it might well make sense to abstract it out and put it in the
mixin...

Michael Bayer

unread,
Jan 12, 2010, 2:34:59 PM1/12/10
to sqlal...@googlegroups.com
Chris Withers wrote:
> Michael Bayer wrote:
>>>> as far as implementation this feature isn't on
>>>> my personal radar, so if you really want anytime soon you should
>>>> provide a
>>>> patch, which I can of course help with.
>>> Cool, it is on my radar, but sadly pretty far away :-(
>>
>> you know, doing the "metaclass" approach is probably the first part of
>> doing the "demarcated mixin" approach.
>>
>> here it is:
>> http://www.sqlalchemy.org/trac/wiki/UsageRecipes/DeclarativeMixins
>
> Indeed, it'd just be great if the default declarative base mixin handled
> this demarcation package already ;)

maybe someday. not until users actually use the recipes first and provide
feedback.

>
> I argue about the comment on relations on that page, if the relation is
> identical, it might well make sense to abstract it out and put it in the
> mixin...

just another complicated and error prone core function I'd rather not
support.

>
> Chris
>
> --
> Simplistix - Content Management, Batch Processing & Python Consulting
> - http://www.simplistix.co.uk
>

Reply all
Reply to author
Forward
0 new messages