declarative SQLAlchemy
Jonathan LaCour
I have been playing around with SQLAlchemy lately, and I really like
it, but I feel like it needs a more declarative way to define object/
table maps. I have started on such a project, and I am calling it
ActiveMapper.
I have my current efforts so far available in a SVN repository, If
you are interested in helping out, please take a look at get back to me:
http://cleverdevil.org/svn/activemapper/
Currently, it allows you to do most of the basic mapping in the
following basic form:
class Address(ActiveMapper):
class mapping:
__table__ = 'address'
id = column(Integer, primary_key=True)
type = column(String)
address_1 = column(String)
city = column(String)
state = column(String)
postal_code = column(String)
person_id = column(Integer, foreign_key=ForeignKey
('person.id'))
class Preferences(ActiveMapper):
class mapping:
__table__ = 'preferences'
id = column(Integer, primary_key=True)
favorite_color = column(String)
personality_type = column(String)
class Person(ActiveMapper):
class mapping:
__table__ = 'person'
id = column(Integer, primary_key=True)
full_name = column(String)
first_name = column(String)
middle_name = column(String)
last_name = column(String)
birth_date = column(DateTime)
ssn = column(String)
gender = column(String)
home_phone = column(String)
cell_phone = column(String)
work_phone = column(String)
prefs_id = column(Integer,
foreign_key=ForeignKey('preferences.id'))
addresses = onetomany('Address',
colname='person_id', backref='person')
preferences = onetoone('Preferences', colname='pref_id',
backref='person')
Anyway, I hope that I find some people interested in helping move
this forward, and sorry for starting an off-topic thread.
--
Jonathan LaCour
http://cleverdevil.org
Karl Guertin
SQLObject compatibility. It looks like you've started in that
direction, but it isn't a priority.
I've switched my new TG development from SQLObject to SQLAlchemy
because most of my development is on legacy databases and SQLObject
just doesn't work for that usage scenario. I'd like to see this
happen sooner rather than later, so I'll see what I can do with this
tonight.
Justin Johnson
Excellent! I think you're right about SQLAlchemy needing a declarative
way of defining objects.
Couple of suggestions:
'onetomany' and 'onetoone' would be more readable as 'one_to_many' and
'one_to_one'. Or because, they're classes, 'OneToMany' and 'OneToOne'
in accordance with PEP8 standards.
__table__ could default to a lowercase version of the classname if it's
not specified in the mapping class. That would be a useful convention
which could be overridden in those few cases where the table name
couldn't be derived like that.
BTW, there's some interesting examples of meta programming for this kind
of thing here:
http://blog.ianbicking.org/more-on-python-metaprogramming.html
Something you didn't mention - is it necessary to call
'process_relationships()' and 'create_tables()' once you have all your
classes defined?
Jeff Watkins
I have been playing around with SQLAlchemy lately, and I really like it, but I feel like it needs a more declarative way to define object/table maps. I have started on such a project, and I am calling it ActiveMapper.
Shuying Wang
I tried accessing your repository, except that it appears to be passworded. :D
--Shuying
Jonathan LaCour
> declarative way of defining objects.
Cool, I think I am right too, ironically enough ;)
> Couple of suggestions:
>
> 'onetomany' and 'onetoone' would be more readable as 'one_to_many'
> and 'one_to_one'. Or because, they're classes, 'OneToMany' and
> 'OneToOne' in accordance with PEP8 standards.
Done, I changed it to one_to_one and one_to_many. Mike Bayer had
originally suggested the names, as lowercase names tend to look more
like a DSL, which is kind of what we want. However, I agree with you
that the underscores make it easier to read.
> __table__ could default to a lowercase version of the classname if
> it's not specified in the mapping class. That would be a useful
> convention which could be overridden in those few cases where the
> table name couldn't be derived like that.
Great idea. Done.
> BTW, there's some interesting examples of meta programming for this
> kind of thing here: http://blog.ianbicking.org/more-on-python-
> metaprogramming.html
I have read this before, and I really liked what I saw. However, I
am not really out to clone the DSL that Rails' ActiveRecord is using,
or clone SQLObject. I am trying to make something that fits in well
with the SQLAlchemy style.
Its interesting that Ian says that implementing something like this
would be hard:
class Person(ActiveRecord):
belongs_to('project_manager')
has_many('milestones')
has_and_belongs_to_many('categories')
This is very similar to what I have implemented, actually. Its not
very difficult with metaclasses, I don't think. Of course, this has
a lot to do with how exceptionally well Mike Bayer has designed
SQLAlchemy, and the fact that my implementation ignores some of the
more complicated things.
Plus, I think explicit relationship management is more pythonic
anyway. I think that:
class Person(ActiveMapper):
milestones = one_to_many('Milestone', colname='person_id')
is much more clear than the magical ActiveRecord way to do things.
> Something you didn't mention - is it necessary to call
> 'process_relationships()' and 'create_tables()' once you have all
> your classes defined?
It is _currently_ necessary because of my implementation. The
problem is that process_relationships needs to have access to all
related classes in order to do its job, so right now I do the easy
thing and wait until all classes are defined before calling it manually.
However, I have a plan to make this automatic. I will put a call to
process_relationships into the metaclass, so I can do the work
progressively. But, I will check to make sure that all needed
classes have been defined at the beginning of process_relationships,
and if they haven't, defer the processing of relationships to the
next call, etc. If someone wants to take a crack at this, please
feel free. It shouldn't be all that hard.
As for create_tables, this is a similar, but I think more complicated
problem. At some point, there will need to be logic for creating
tables, foreign keys constraints, etc. that is smart enough to manage
dependencies. I have no idea how to solve this problem, and would
love some help.
Thanks for the feedback!
Jonathan LaCour
> (specifically with this particular task). I'm glad someone was able
> to do it.
Then, get involved! I would love to have someone else work on this
code with me. Anyone who would like to, just ask, and I will give
you commit access to the repository.
> I'm definitely keen to try this out, because I keep banging my head
> against SQLObjects' severe limitations.
Indeed. I find that SQLObject is a brilliant piece of work, and I
don't want to knock it really. It works like a champ for lots of
cases, and I think its a good choice for a decent number of projects.
However, SQLAlchemy can do a lot more, and hopefully soon it can be
just as pretty and simple to get started with.
> I think what impressed me most is that Michael Bayer wants
> SQLAlchemy to work for everyone not just people who do databases
> *exactly* like he does. My personal hot button: SQLObject is
> basically worthless accessing any database that doesn't use integer
> primary keys. That leaves out GUID primary keys. That leaves out
> composite keys. That leaves out dependant keys. That leaves out
> *the majority* of the databases I want to access.
I totally understand your complaints, and hopefully this declarative
layer will help make SQLAlchemy an even better choice for you.
> Thanks for getting this underway, Jonathan.
No problem!
Michele Cella
> This is a bit off topic, but I thought some of you might find it useful.
>
> I have been playing around with SQLAlchemy lately, and I really like
> it, but I feel like it needs a more declarative way to define object/
> table maps. I have started on such a project, and I am calling it
> ActiveMapper.
>
> I have my current efforts so far available in a SVN repository, If
> you are interested in helping out, please take a look at get back to me:
>
> http://cleverdevil.org/svn/activemapper/
>
That's very cool. ;-)
Just two questions:
- is the inner "mapping" class absolutely needed?
- why not AlchemyMapper? DHH may accuse you of copying ActiveRecord.
:D
Keep in mind that I never tried SQLAlchemy so the first question is
probably a stupid question.
Keep the great work.
Ciao
Michele
Jonathan LaCour
> That's very cool. ;-)
Thanks :)
> Just two questions:
> - is the inner "mapping" class absolutely needed?
Absolutely needed? Not really. However, I really like it, for a few
of reasons:
* I think it makes it clearer that the attributes on the
instance are
a *result* of the properties in the mapping class, not the actual
properties themselves.
* It is good for namespacing things a bit. Less of a risk of
collisions
and the like with the scope of the class.
* It still maintains a little bit of SQLAlchemy's philosophy of
keeping
the mapping separate from the class. I look at it as
basically inlining
the once separate mapping directly into the class definition.
However, I am willing to listen to any arguments to remove the
mapping class, or to rename it to something else. I also considered
using the name "meta" and "schema".
> - why not AlchemyMapper? DHH may accuse you of copying
> ActiveRecord.
Actually, the term "Active Record" describes a design pattern by the
same name. Ruby on Rails' ActiveRecord library is simply
implementing the "Active Record" pattern. Ideally, I would like the
base class to be called ActiveRecord, however I think that a horde of
uninformed rubyites would probably accuse me of copying Rails. So,
thats why I picked ActiveMapper, which was actually a suggestion of
Mike Bayer's. DHH knows the pattern, and so probably wouldn't accuse
me of copying them, and if he does, I can mercilessly shoot him down :)
I would be open to changing it to something else to avoid the issue
altogether, but I don't like AlchemyMapper. In many ways, I am also
attempting to implement the "Active Record" pattern, and I thought
that ActiveMapper was both close enough to the pattern name, and far
enough from the Rails' base class to cause confusion.
Any other ideas?
> Keep the great work.
Will do!
Michele Cella
>
> > Just two questions:
> > - is the inner "mapping" class absolutely needed?
>
> Absolutely needed? Not really. However, I really like it, for a few
> of reasons:
>
> * I think it makes it clearer that the attributes on the
> instance are
> a *result* of the properties in the mapping class, not the actual
> properties themselves.
>
> * It is good for namespacing things a bit. Less of a risk of
> collisions
> and the like with the scope of the class.
>
> * It still maintains a little bit of SQLAlchemy's philosophy of
> keeping
> the mapping separate from the class. I look at it as
> basically inlining
> the once separate mapping directly into the class definition.
>
> However, I am willing to listen to any arguments to remove the
> mapping class, or to rename it to something else. I also considered
> using the name "meta" and "schema".
Well regarding "meta" vs "schema" vs "mapping", keep mapping as is the
best IMHO.
As I said I don't know enough about SQLAlchemy to understand every
detail you talked about, anyway you may be interested to see (but you
already seen it probably) how we solved the same problem in TG for
declarative forms.
That's what you can do:
class CommentFields(WidgetsDeclaration):
name = TextField()
comment = TextArea()
comment_form = TableForm(fields=CommentFields())
This is probably not doable with SQLAlchemy tough, does it use
instances like TG for a Form?
Anyway I'm just trying to give you some inspiration since you asked for
comments. :D
>
> > - why not AlchemyMapper? DHH may accuse you of copying
> > ActiveRecord.
>
> Actually, the term "Active Record" describes a design pattern by the
> same name. Ruby on Rails' ActiveRecord library is simply
> implementing the "Active Record" pattern. Ideally, I would like the
> base class to be called ActiveRecord, however I think that a horde of
> uninformed rubyites would probably accuse me of copying Rails. So,
> thats why I picked ActiveMapper, which was actually a suggestion of
> Mike Bayer's. DHH knows the pattern, and so probably wouldn't accuse
> me of copying them, and if he does, I can mercilessly shoot him down :)
>
> I would be open to changing it to something else to avoid the issue
> altogether, but I don't like AlchemyMapper. In many ways, I am also
> attempting to implement the "Active Record" pattern, and I thought
> that ActiveMapper was both close enough to the pattern name, and far
> enough from the Rails' base class to cause confusion.
>
> Any other ideas?
Don't worry really, I was just just joking.
I knowed about the ActiveRecord pattern (read it somewhere on DDH blog
probably) but since you said you're also trying to implement it (and
that's great) it makes perfect sense the ActiveMapper name, it's nice
and also blessed by Mike. ;-)
>
> > Keep the great work.
>
> Will do!
Great.
Ciao
Michele
Jorge Godoy
> That's what you can do:
>
> class CommentFields(WidgetsDeclaration):
> name = TextField()
> comment = TextArea()
> comment_form = TableForm(fields=CommentFields())
Sorry for hijacking this thread, but is it possible to do something like:
comment_form = TableForm(fields=[CommentFields(), special_consideration =
TextArea()])
I mean, is it possible to combine the declaration with something else (or two
declarations, or ...)?
--
Jorge Godoy <jgo...@gmail.com>
Karl Guertin
> comment_form = TableForm(fields=[CommentFields(), special_consideration =
> TextArea()])
>
> I mean, is it possible to combine the declaration with something else (or two
> declarations, or ...)?
That won't work, but...
comment_form = TableForm(fields=[CommentFields(),
TextArea('special_consideration')])
should if my understanding of the rewrite is correct. The fields
attribute is just a dumb list, the declarative magic only occurs in
classes that subclass WidgetsDeclaration. I think you can even adjust
the fields after you've instantiated the TableForm:
comment_form = TableForm(fields=[CommentFields()])
comment_form.fields += TextArea('special_consideration')
Michele Cella
Exactly as Karl said, a WidgetsDeclaraion instance is just a list and
you can anything you can do with a list with it.
Example:
>>> from turbogears import widgets as w
>>> from turbogears.widgets import WidgetsDeclaration as WD
>>> class CommentFields(WD):
... name = w.TextField()
... comment = w.TextArea()
...
>>> class OtherFields(WD):
... special = w.TextArea()
...
>>> fields = CommentFields() + OtherFields()
>>> for field in fields:
... print field.name
...
name
comment
special
>>> fields.append(w.CheckBox(name="jorge"))
>>> for field in fields:
... print field.name
...
name
comment
special
jorge
>>>
Ciao
Michele
Jonathan LaCour
> 'process_relationships()' and 'create_tables()' once you have all
> your classes defined?
Replying to myself: I just checked in some code to make it
unnecessary to call process_relationships() manually. I am not
positive that the code is 100% correct, so if people could try this
out, and maybe double-check my code, that would be great :)
Baby steps to a finished product :)
Justin Johnson
I'll give your code a whirl as soon as I get chance. I'm using
SQLObject at the moment but Alchemy's rich feature set and sensible
defaults for saving objects is luring me. Which is s shame in a way -
SQLObject is a great package that just needs a bit extra. Wonder if Mr
Bicking will get chance to do any more with it considering the colossal
number of things he's up to? :-) Given that it's the out-of-the-box ORM
for Turbogears it's pretty important that it's an incredibly solid
product equal to Rails ActiveRecord.
BTW, with regards to naming. Here are Martin Fowler's definitions of
ActiveRecord and DataMapper:
http://www.martinfowler.com/eaaCatalog/activeRecord.html
and
http://www.martinfowler.com/eaaCatalog/dataMapper.html
SQLAlchemy seems to be fulfilling the definition of DataMapper rather
than ActiveRecord.
ActiveMapper kind of falls somewhere inbetween but why not claim the
original DataMapper pattern name as Rails did with ActiveRecord? ;-)