new elixir extensions
Jonathan LaCour
A little treat for your weekend: I just committed two new elixir
extensions!
The first, elixir.ext.encrypted, provides automatic encryption using
blowfish encryption for any unicode field in your elixir entities. It
requires the excellent PyCrypto (http://www.amk.ca/python/code/crypto).
The second, elixir.ext.versioned, provides a fairly complete
implementation of versioning for your entities - as you update
instances, the history of that instance is tracked, with version numbers
and timestamps, including the ability to look at old versions, compare
versions, and revert to previous versions.
Here is a short example of both extensions in use, along with an example
of the new event code that we integrated into Elixir earlier in the
week:
from elixir import *
from elixir.events import before_update
from elixir.ext.versioned import acts_as_versioned, after_revert
from elixir.ext.encrypted import acts_as_encrypted
class Person(Entity):
has_field('name', Unicode)
has_field('email', Unicode)
has_field('password', Unicode)
has_many('articles', of_kind='Article', inverse='author')
acts_as_encrypted(for_fields=['password'],
with_secret='s3cr1t')
class Article(Entity):
has_field('title', Unicode, primary_key=True)
has_field('description', Unicode)
has_field('content', Unicode)
belongs_to('author', of_kind='Person', inverse='articles')
acts_as_versioned()
@after_update
def article_updated(self):
'''
Called after update, where you can access the new version
number through self.version, and the timestamp through
self.timestamp.
'''
print '-' * 80
print 'Article updated at', self.timestamp
print ' new version is ->', self.version
@after_revert
def article_reverted(self):
'''
Called after the instance is reverted
'''
print '-' * 80
print 'Article reverted!'
print '-' * 80
if __name__ == '__main__':
metadata.bind = 'sqlite:///'
metadata.create_all()
jonathan = Person(
name='Jonathan',
email='jona...@someplace.org'
)
blog_post = Article(
title='Some Blog Post',
description='A blog post on some subject',
content='Draft content for blog post',
author=jonathan
)
objectstore.flush(); objectstore.clear()
blog_post = Article.get('Some Blog Post')
blog_post.content = 'Updated content for blog post'
objectstore.flush(); objectstore.clear()
blog_post = Article.get('Some Blog Post')
blog_post.content = 'Even more updated post'
objectstore.flush(); objectstore.clear()
blog_post = Article.get('Some Blog Post')
for previous_version in blog_post.versions:
print '-' * 80
print previous_version.version
print previous_version.timestamp
print blog_post.compare_with(previous_version)
print '-' * 80
assert blog_post.version == 3
blog_post.revert()
objectstore.flush(); objectstore.clear()
blog_post = Article.get('Some Blog Post')
assert blog_post.version == 2
assert blog_post.content == 'Updated content for blog post'
I would love to get some feedback on these extensions, along with any
suggestions on how I might make them better. Have a great weekend!
--
Jonathan LaCour
http://cleverdevil.org
mattrussell
I've found a use for the versioned extension already in my project,
neatly solved a problem
that I didn't know how to get around (by accident!)
So far I've not used it anger, but preliminary testing seems good :)
Thanks,
Matt
On Aug 3, 11:42 pm, Jonathan LaCour <jonathan-li...@cleverdevil.org>
wrote:
> Greetings Elixir users!
>
> A little treat for your weekend: I just committed two new elixir
> extensions!
>
> The first, elixir.ext.encrypted, provides automatic encryption using
> blowfish encryption for any unicode field in your elixir entities. It
> requires the excellent PyCrypto (http://www.amk.ca/python/code/crypto).
>
> The second, elixir.ext.versioned, provides a fairly complete
> implementation of versioning for your entities - as you update
> instances, the history of that instance is tracked, with version numbers
> and timestamps, including the ability to look at old versions, compare
> versions, and revert to previous versions.
>
> Here is a short example of both extensions in use, along with an example
> of the new event code that we integrated into Elixir earlier in the
> week:
>
> from elixir import *
> from elixir.events import before_update
> from elixir.ext.versionedimport acts_as_versioned, after_revert
> email='jonat...@someplace.org'
Jonathan LaCour
> I've found a use for the versioned extension already in my project,
> neatly solved a problem that I didn't know how to get around (by
> accident!)
>
> So far I've not used it anger, but preliminary testing seems good :)
Excellent, please do report back on how things went for you. The
versioning extension is very very new, and I definitely need some
real-world testing to help validate or invalidate my approach.
mattrussell
that if my entity contains a relation to another entity,
for example,
I have an entity that has this defined:
class Activity(Entity):
title = Unicode()
has_activity_type("activity_types")
(where has_activity_type is a ext.associable statement)
in this case, it seems that the related entities are not versioned.
Thus:
a1 = Activity()
a1.title = "up to scotland"
a1.activity_types.append(ActivityType(name="driving"))
a1.flush()
a1.version
-> 1
a1.title = "Up to Scotland"
a1.flush()
a1.version
-> 2
a1.versions[-1].activity_types
<type 'exceptions.AttributeError'>: 'ActivityVersion' object has no
attribute 'activity_types'
Matt
On Aug 7, 3:25 pm, Jonathan LaCour <jonathan-li...@cleverdevil.org>
wrote:
> mattrussell wrote:
> > I've found a use for theversionedextension already in my project,
Jonathan LaCour
> Seems good so far.... the only snag that I seem to have hit so far is
> that if my entity contains a relation to another entity,
Yes, we currently do not provide full support for relationships. We
only version the main table for the entity, which actually *may* contain
some foreign keys (and thus relationships) but we don't handle more
complex types of relationships, and we don't even expose the supported
relationship types on the Version class yet.
These are all things we want to have a strategy for, and we have some
people who are interested in enhancing the extension to deal with
relationships, and patches are always welcome!
Jonas
1. Encryption. It would be well that the user could choose the
algorithm to encrypt althought could be left blowfish as default.
2. Versioning. Could be indicated that only been versioned any fields?
Rufus Pollock
my non-gmail account with which I subscribed (and doesn't send me mail
either). My enquiry to the support team from 4 days ago is still
unanswered and attempts to resubscribe have failed (i'm politely
informed i'm already a member of the group ...).]
Dear Jonathan (and others),
Great to see the addition of versioning support. I'd already been
working on something very similar for a while (and had meant to get
round to writing to this list about it ...).
Back in January I developed a python 'versioned domain model' (vdm)
package to allow 'versioning' of domain objects (and domain models) in a
way similar to the way subversion allows versioning of filestytem trees.
It was heavily based on Fowler's Patterns for things that change with time:
http://www.martinfowler.com/ap2/timeNarrative.html
(You can find out more in the README.txt:
http://knowledgeforge.net/ckan/svn/vdm/trunk/README.txt
)
When I wrote the first version back in January/February this year it was
focused on sqlobject (as that was what I using). However a few months
ago I started work on a port to elixir. While the sqlobject version
supports full versioning of all attributes (including many to many
relationships) the elixir version is currently at a state very similar
to your implementation with support only for versioning the basic fields
(and foreign keys of course), see:
http://knowledgeforge.net/ckan/svn/vdm/trunk/vdm/elixir/base_test.py
http://knowledgeforge.net/ckan/svn/vdm/trunk/vdm/elixir/base.py
As I'd already been wondering whether I might be able to contribute the
code into elixir as an extension, now that you already have something
I'd very much like to:
a) contribute/merge what I already have (if that would be useful).
b) contribute to developing this feature (as part of elixiir) as it
goes forward.
Regards,
Rufus Pollock
Jonathan LaCour
> [Repost from 5 days ago -- Google Groups is still rejecting email
> from my non-gmail account with which I subscribed (and doesn't send
> me mail either). My enquiry to the support team from 4 days ago is
> still unanswered and attempts to resubscribe have failed (i'm politely
> informed i'm already a member of the group ...).]
Yes, Rufus and I had a good off-list discussion, and I am glad he has
found a way to post (even if its not the preferred way). I'll try and
get the conversation rolling again here.
> As I'd already been wondering whether I might be able to contribute
> the code into elixir as an extension, now that you already have
> something I'd very much like to:
>
> a) contribute/merge what I already have (if that would be useful).
>
> b) contribute to developing this feature (as part of elixiir) as it
> goes forward.
I advised Rufus that it would be best to work off of the versioning
support that I added to SVN a few weeks ago. He agreed, and now we are
talking about how to version relationships.
One thing we both noted is that in order to version relationships, those
relationships need to be either:
1. Stored as part of the main entity's table. Or,
2. Be a mapped object as well.
This brings me to an interesting point. I think we sould deprecate
the current style of many-to-many relationships in favor of something
more powerful, that is more similar to the associable plugin. Rails'
ActiveRecord has already done this, in favor of `has_many_through`,
which uses association objects. Mike Bayer blogged a bit about the
concept here: http://techspot.zzzeek.org/?p=13. I think we should try
and integrate these ideas directly into the core. Here was one idea I
cooked up that might work:
It would be nice if the current `has_and_belongs_to_many` statement
would always generate an association object, but would hide it if
you didn't need it, or would let you add additional fields into the
relationship itself if you wanted that ability:
class Person(Entity):
has_field('name', Unicode)
has_and_belongs_to_many(
'addresses',
of_kind='Address',
with_attributes={
'address_type' : Unicode,
'address_name' : Unicode
}
)
class Address(Entity):
has_field('fulladdress', Unicode)
p = Person.query().get(1)
print p.addresses[0].address_type
print p.addresses[0].address_name
print p.addresses[0].target
p.addresses.append(
Person.addresses_association(
address_type='Home',
address_name='My Home Address',
target=Address(fulladdress='123 Main St.')
)
)
Another alternative would be to do it more in the style of Rails, and
modify our `has_many` statement:
http://wiki.rubyonrails.org/rails/pages/ThroughAssociations
Once the relationships themselves are mapped objects, it becomes easier
to version them using the existing `acts_as_versioned` Mapper Extension.
Any thoughts?
Rufus Pollock
>
> Rufus Pollock wrote:
[snip]
> > As I'd already been wondering whether I might be able to contribute
> > the code into elixir as an extension, now that you already have
> > something I'd very much like to:
> >
> > a) contribute/merge what I already have (if that would be useful).
> >
> > b) contribute to developing this feature (as part of elixiir) as it
> > goes forward.
>
> I advised Rufus that it would be best to work off of the versioning
> support that I added to SVN a few weeks ago. He agreed, and now we are
> talking about how to version relationships.
>
> One thing we both noted is that in order to version relationships, those
> relationships need to be either:
>
> 1. Stored as part of the main entity's table. Or,
> 2. Be a mapped object as well.
You are really want to go with 2 if both ends of the link are
versioned. For example, for http://www.ckan.net/ I want to have a
versioned Package and a versioned Tag object with a m2m link between
Packages and Tags.
> This brings me to an interesting point. I think we sould deprecate
> the current style of many-to-many relationships in favor of something
> more powerful, that is more similar to the associable plugin. Rails'
> ActiveRecord has already done this, in favor of `has_many_through`,
> which uses association objects. Mike Bayer blogged a bit about the
> concept here: http://techspot.zzzeek.org/?p=13. I think we should try
> and integrate these ideas directly into the core. Here was one idea I
> cooked up that might work:
The polymorphic association object seems to me to be a bit of
overkill. With the standard m2m setup that elixir already has it is
pretty easy to add associations and this polymorphic setup only
becomes useful if there is some object that is joined to every other
object (e.g. as with tags in some cases). Even then I think would be
fairly easy just to wrap the existing standard m2m relationships up
into something cleaner.
I agree that elixir should definitely support explicit intermediate
objects but I'm not sure that it might not be better to leave people
to specify them themselves rather than add it all in to one side of
the has_and_belongs_to_many. That is I'd prefer something like:
class Person(Entity):
has_field('name', Unicode)
# creates normal addresses property but addresses_association
property as well
has_and_belongs_to_many(
'addresses',
intermediary='PersonAddress',
)
class PersonAddress(Entity):
has_field('type, Unicode)
has_field('name', Unicode)
belongs_to('person', of_kind='Person')
belongs_to('address', of_kind='Address')
class Address(Entity):
has_field('fulladdress', Unicode)
has_and_belongs_to_many(
'persons',
intermediary='PersonAddress',
)
p = Person.query().get(1)
print p.addresses # list of addresses
print p.addresses_association # list of PersonAddress objects
print p.addresses_association[0].address_name
print p.addresses_association[0].target
p.addresses_assoication.append(
Person.addresses_association(
address_type='Home',
address_name='My Home Address',
target=Address(fulladdress='123 Main St.')
)
)
p.address.append(Address(fulladdress='blah')) # default values for
association attributes
> Another alternative would be to do it more in the style of Rails, and
> modify our `has_many` statement:
>
> http://wiki.rubyonrails.org/rails/pages/ThroughAssociations
The example here seems to be more about developing a simple m2m
property in the style that elxiir has already implemented. This page
seems to be more about the polymorphic stuff Mike Bayer was posting
about:
http://wiki.rubyonrails.org/rails/pages/PolymorphicAssociations
> Once the relationships themselves are mapped objects, it becomes easier
> to version them using the existing `acts_as_versioned` Mapper Extension.
To some extent but I think this is a fairly small part of doing full
versioning so I don' think this should be the main reason driving the
change.
> Any thoughts?
Regards,
Rufus
Jonathan LaCour
>> One thing we both noted is that in order to version relationships,
>> those relationships need to be either:
>>
>> 1. Stored as part of the main entity's table. Or,
>> 2. Be a mapped object as well.
>
> You are really want to go with 2 if both ends of the link are
> versioned. For example, for http://www.ckan.net/ I want to have a
> versioned Package and a versioned Tag object with a m2m link between
> Packages and Tags.
Agreed.
> The polymorphic association object seems to me to be a bit of
> overkill. With the standard m2m setup that elixir already has it
> is pretty easy to add associations and this polymorphic setup only
> becomes useful if there is some object that is joined to every other
> object (e.g. as with tags in some cases). Even then I think would be
> fairly easy just to wrap the existing standard m2m relationships up
> into something cleaner.
Again, I am with you.
> I agree that elixir should definitely support explicit intermediate
> objects but I'm not sure that it might not be better to leave people
> to specify them themselves rather than add it all in to one side of
> the has_and_belongs_to_many. That is I'd prefer something like:
>
> class Person(Entity):
> has_field('name', Unicode)
> # creates normal addresses property but addresses_association
> has_and_belongs_to_many(
> 'addresses',
> intermediary='PersonAddress',
> )
>
> class PersonAddress(Entity):
> has_field('type, Unicode)
> has_field('name', Unicode)
> belongs_to('person', of_kind='Person')
> belongs_to('address', of_kind='Address')
>
> class Address(Entity):
> has_field('fulladdress', Unicode)
> has_and_belongs_to_many(
> 'persons',
> intermediary='PersonAddress',
> )
Yes, I agree with you, the more that I look at the code I wrote before,
the less I like it. I have studied ActiveRecord a bit more, and what
you are showing above is actually very similar to what they have elected
to do, and I like their style a bit better. Its more like this:
class Person(Entity):
has_field('name', Unicode)
has_many('address_associations', of_kind='PersonAddress')
has_many(
'addresses',
of_kind='Address',
through='address_associations'
)
class PersonAddress(Entity):
has_field('type', Unicode)
has_field('name', Unicode)
belongs_to('person', of_kind='Person')
belongs_to('address', of_kind='Address')
class Address(Entity):
has_field('fulladdress', Unicode)
has_many('address_associations', of_kind='PersonAddress')
has_many('people',
of_kind='Person',
through='address_associations'
)
The `has_many` with a `through` keyword would simply create a property
on the entity proxying through the association objects, for convenience.
This should be really easy to implement, although I need to gain a
better understanding of the new autodelay technique to be really
confident about that.
>> Once the relationships themselves are mapped objects, it becomes
>> easier to version them using the existing `acts_as_versioned` Mapper
>> Extension.
>
> To some extent but I think this is a fairly small part of doing full
> versioning so I don' think this should be the main reason driving the
> change.
Well, its not the *only* consideration. Its also a much richer form of
relationship, and I think its easier to understand. I have never really
been a fan of the readability and clarify of `has_and_belongs_to_many`
and numerous people have complained to me that it didn't make immediate
sense to them. I think going with explicit association objects will
make things a bit easier to understand, with the added benefit that it
should make versioning a bit easier to do as well.