Post 1.0 Refactoring Call For Discussion
Bruce Kroeze
I've been thinking about how to approach the modularization, and some key features of it. At this point, I'm not even strongly advocating any solution, these are just initial ideas.
Documentation
I personally dislike lots of inline commenting. Sorry, but that's old-school thinking. If the function is so long or complex that it can't be understood by a decent programmer, then it is too long and should be refactored.
However, I do agree that we are missing tons of method/class documentation. For my own projects, I like to use restructured text documentation for all classes and methods. Also, I'd like to see a standard license/copyright block at the top of all modules. I think also a call-out to the primary authors would be appropriate.
I suspect that most people who are talking about documentation are really looking for better use/setup/extension documentation. That's a different beast entirely. Perhaps we should have a person who loves that kind of work head up that front? It won't be me.
Tests
I think we should get more aggressive about requiring tests. With all the refactoring I've been doing lately, the existing tests have been literally invaluable. As we step up to 2.0 work, they'll be absolutely essential.
Personally, I dislike doctests, but they're still much better than nothing. Yes UnitTests are a bit more of a pain to set up, but they are so much easier to casually extend once in place.
Pluggability
We already have *some* pluggability. The payment, shipping and tax systems all are pluggable. But that was all done "by hand", and isn't completely the same from subsystem to subsystem. Nor is the API/contract for extension clearly defined.
We do not have good pluggability in the Product model/subsystem. This is our greatest weakness at the moment. There is far too much code which tests "is this an xxx-type product" scattered around. To use a Fowler phrase, it is starting to smell a bit.
I spent some time looking at the Trac extension system. It is solid, even beautiful in many ways. I think we could use the Trac component system ( http://trac.edgewall.org/wiki/TracDev/ComponentArchitecture) almost as-is to provide a much richer and more well-defined pluggable system.
I'm not completely sure at this point how we'd marry Django and Trac at the model level for things like pluggable product models, but I think that it is quite possible to do and to do elegantly. I already see how we could extend the trac idea of an "ExtensionOption" to automatically provide satchmo Configuration registration.
I'd love to hear any other ideas people have for modularization or pluggable architectures.
Refactoring
I'd like to move around some components and make them more generic. For example, caching and configuration. They should be top-level apps, not imported from under a satchmo tree. Probably these (or at least caching) should be made pluggable via the newly-revamped pluggable architecture.
Some apps should merge, for example accounts and contact. Some apps should have functionality moved to the appropriate place. Order and friends should leave Contact and move to Shop or Payment.
All templates should live in their apps, not in one master template tree. For example, templates/product should move to product/templates/product. This is a wart and it encourages app spaghetti, imho.
Configuration
I admit bias, since I wrote it, but I love the Configuration system. One major problem with it, however, is that some config options won't take effect until the store is restarted. Either we need to fix that so that when those items are changed, the appropriate systems are reloaded, or we need to have some global flag "needs restart", and a system for restarting the store from the admin.
NewForms Admin
I think we may as well move to NewFormsAdmin almost immediately. 2.0 won't be ready before the official release, and it looks so nice ... We'll be avoiding so much rework if we start sooner rather than later.
--
Bruce Kroeze
http://solidsitesolutions.com
Dynamic Designs, Optimized
Marty Alchin
> My earlier "crossroads" thread seems to have ended up with most people
> wanting a serious refactor and modularization of the code. The thought,
> which I heartily agree with, is that this will support adding true million
> dollar class features to Satchmo. Trying to bolt on these features without
> first cleaning up our base will lead to bloat, spaghetti code, and rigidity.
I'm glad to hear this, as I think every project undergoes this at some
point anyway (magic-removal, anyone?), and it's probably better to
plan for it now that you already know you need to.
> Documentation
>
> I personally dislike lots of inline commenting. Sorry, but that's
> old-school thinking. If the function is so long or complex that it can't be
> understood by a decent programmer, then it is too long and should be
> refactored.
>
> However, I do agree that we are missing tons of method/class documentation.
> For my own projects, I like to use restructured text documentation for all
> classes and methods. Also, I'd like to see a standard license/copyright
> block at the top of all modules. I think also a call-out to the primary
> authors would be appropriate.
>
> I suspect that most people who are talking about documentation are really
> looking for better use/setup/extension documentation. That's a different
> beast entirely. Perhaps we should have a person who loves that kind of work
> head up that front? It won't be me.
I think your last point here is indeed the key. I've been talking
about integrating Satchmo into a project for a while now, but I've
found the documentation to be lacking in the area of integrating it
into an otherwise non-store site. That may be me, since I'm looking to
do some non-standard stuff, but I still found it very difficult to
even figure out where to begin.
Unfortunately, I'm much better about writing bland reference
documentation than installation docs and tutorials.
> Tests
>
> I think we should get more aggressive about requiring tests. With all the
> refactoring I've been doing lately, the existing tests have been literally
> invaluable. As we step up to 2.0 work, they'll be absolutely essential.
>
> Personally, I dislike doctests, but they're still much better than nothing.
> Yes UnitTests are a bit more of a pain to set up, but they are so much
> easier to casually extend once in place.
I'm glad to see a focus on tests, especially if you end up changing
the underlying implementation of things while maintaining an existing
API. As for doctests vs. unittests, I think it depends a lot on the
nature of the feature being tested. Doctests, in my mind, tend to work
best when developers will have to interact with the feature directly
in their own Python code. In these cases, doctests are great at
doubling as documentation, since they can quickly and easily convey
how Python code can use the feature. That's not an essential
requirement, of course, since you can document it elsewhere anyway,
but it makes it easy if you can do it all in one place.
On the other hand, things like Configuration are largely untouched by
external developers. Sure, they can define their own config options,
but they don't do so by calling many of the internal functions. In
those cases, unittests are probably better, since they're more
flexible, easier to isolate individual tests, and as you mention,
they're very easy to extend. They just don't have that documentation
feature, but for those cases where it doesn't make sense anyway, this
is hands-down the best option.
> Pluggability
>
> We already have *some* pluggability. The payment, shipping and tax systems
> all are pluggable. But that was all done "by hand", and isn't completely
> the same from subsystem to subsystem. Nor is the API/contract for extension
> clearly defined.
>
> We do not have good pluggability in the Product model/subsystem. This is
> our greatest weakness at the moment. There is far too much code which tests
> "is this an xxx-type product" scattered around. To use a Fowler phrase, it
> is starting to smell a bit.
>
> I spent some time looking at the Trac extension system. It is solid, even
> beautiful in many ways. I think we could use the Trac component system (
> http://trac.edgewall.org/wiki/TracDev/ComponentArchitecture)
> almost as-is to provide a much richer and more well-defined pluggable
> system.
>
> I'm not completely sure at this point how we'd marry Django and Trac at the
> model level for things like pluggable product models, but I think that it is
> quite possible to do and to do elegantly. I already see how we could extend
> the trac idea of an "ExtensionOption" to automatically provide satchmo
> Configuration registration.
>
> I'd love to hear any other ideas people have for modularization or pluggable
> architectures.
I admit, I haven't delved quite enough into Satchmo's internals to
know what tactics to suggest, but I can say that what i've seen of
Trac so far doesn't sit very well with me. It's probably just a
personal thing, since I can't really put my finger on any technical
*problems* with it, but it just seems like an over-engineered system
that makes Python do too much error-checking at run-time. The whole
idea of interfaces seems like a way to programmatically enforce that
programmers don't screw up. While screwups generally aren't good, I'm
just not convinced that such things should be handled at run-time.
But, like I said, that's really more just personal preference and
philosophy, so I don't have any real convincing reasons for stating
one way or another. Trac-style componentization isn't very
Django-esque, and maybe I'm just too much of a Django fanboy to
appreciate it. I do think you'd be able to marry it with Django, it
just seems like doing so might be unnecessary added time and energy.
My jury's still out on this one, so I might have a stronger opinion at
some other point.
> Refactoring
>
> I'd like to move around some components and make them more generic. For
> example, caching and configuration. They should be top-level apps, not
> imported from under a satchmo tree. Probably these (or at least caching)
> should be made pluggable via the newly-revamped pluggable architecture.
>
> Some apps should merge, for example accounts and contact. Some apps should
> have functionality moved to the appropriate place. Order and friends should
> leave Contact and move to Shop or Payment.
>
> All templates should live in their apps, not in one master template tree.
> For example, templates/product should move to product/templates/product.
> This is a wart and it encourages app spaghetti, imho.
I think this is a great idea. In working on one of my more sizable
projects, I found a variety of needs that I thought could be factored
out in a way that didn't depend on the project itself, and I was able
to release those and maintain them as separate apps. I wouldn't
necessary tell anyone they should do this, but as long as you're
already consider it, I'll certainly agree with the idea.
As for merging, I think that would help non-store projects integrate a
Satchmo store, since it would likely involve fewer additions to
INSTALLED_APPS, and the process could be documented much more easily.
Pluggability is good even for some internal things, such as Contacts,
since people might want to integrate their own existing Contact system
into their Satchmo installation. I don't know how easily that could be
done, but I'd certainly support any move that tries to make that
possible.
> Configuration
>
> I admit bias, since I wrote it, but I love the Configuration system. One
> major problem with it, however, is that some config options won't take
> effect until the store is restarted. Either we need to fix that so that
> when those items are changed, the appropriate systems are reloaded, or we
> need to have some global flag "needs restart", and a system for restarting
> the store from the admin.
As we've discussed before, I very much enjoyed seeing the work you did
on the Configuration system. I'm still not 100% thrilled about the
public API for it, but so far I haven't come up with anything better,
so that's a minor issue that's really just a matter of my own
preference.
The biggest thing I'll say here (and I'm sure you're expecting it) is
that I think you should break the Configuration system into its own
app, so that it can be maintained separately from Satchmo itself, and
can be used in non-Satchmo apps.
I say this because I'm not fond of having Satchmo's configuration and
dbsettings filling such similar roles in fairly different ways, with
different feature sets. I'd very much like to see a single
Configuration system that's available to all Django apps, and can
fulfill the needs of a variety of different types of sites. I've mused
in the past about where dbsettings is headed, after seeing the work
you've done, and I'd like to see us work together to make something
that can serve a greater audience.
I should note that I'm *not* suggesting that your Configuration system
be folded into dbsettings. dbsettings has something of a flawed
history, and since you've already done such a good job at overcoming
many of its pitfalls, I think it'd be better to start an all-new app,
based on your work, that we can refine to make it as good as possible.
> NewForms Admin
>
> I think we may as well move to NewFormsAdmin almost immediately. 2.0 won't
> be ready before the official release, and it looks so nice ... We'll be
> avoiding so much rework if we start sooner rather than later.
I haven't worked enough with newforms-admin to have much of an opinion
yet, but I do agree with you that if you're going to deal with it
eventually, it might as well be now, where you're already touching a
significant amount of Satchmo anyway.
I can't promise much in the way of time or code, but I'd be happy to
contribute wherever I can, especially if you do decide to break out
your Configuration work into a separate app. Keep up the good work!
-Gul
Chris Moffitt
I suspect that most people who are talking about documentation are really looking for better use/setup/extension documentation. That's a different beast entirely. Perhaps we should have a person who loves that kind of work head up that front? It won't be me.
I definitely don't mind working some more on documentation. In truth, I know that my coding skills are not as strong as many of the others here. Doing documentation might be something more my speed ;) One thing I need help with is figuring out what type of documentation people need. If anyone has suggestions or examples for what they think it missing, it would be helpful. As you mention we probably need more info about integration and enhancing. I don't know if we need a whole lot of tutorials. I'm interested in what people think.
Tests
Personally, I dislike doctests, but they're still much better than nothing. Yes UnitTests are a bit more of a pain to set up, but they are so much easier to casually extend once in place.
I've been guilty of not always putting in tests. I agree that we need to continue to emphasize the need for good unit tests. IMHO, part of the reason Satchmo has relatively few bugs is that the test suite is pretty strong.
Pluggability
We already have *some* pluggability. The payment, shipping and tax systems all are pluggable. But that was all done "by hand", and isn't completely the same from subsystem to subsystem. Nor is the API/contract for extension clearly defined.
Agreed. I think shipping, tax and payment are pretty decent but there are options to improve. Fixing the product models should be priority #1 from a plugin standpoint.
We do not have good pluggability in the Product model/subsystem. This is our greatest weakness at the moment. There is far too much code which tests "is this an xxx-type product" scattered around. To use a Fowler phrase, it is starting to smell a bit.
The current product model format is the second iteration which is better but still has a lot of room for improvement. In my mind, this is the main thing we need to spend some serious design/discussion on so we can come up with a clean, robust and scaleable solution. I would like to hear some ideas about how to do it. The Trac extension might be one method. I'm not familiar enough with it to know if that makes the most sense. Obviously some sort of OO model makes sense but Django's current implementation doesn't support it yet. Once again, I think we need to really think this through and come up with some good options and evaluate them.
Refactoring
I'd like to move around some components and make them more generic. For example, caching and configuration. They should be top-level apps, not imported from under a satchmo tree. Probably these (or at least caching) should be made pluggable via the newly-revamped pluggable architecture.
Some refactoring is in order. One of the goals should be to make the installation simpler. After the product model changes and admin improvements, ease of installation is my third most important improvement.
Configuration
I admit bias, since I wrote it, but I love the Configuration system. One major problem with it, however, is that some config options won't take effect until the store is restarted. Either we need to fix that so that when those items are changed, the appropriate systems are reloaded, or we need to have some global flag "needs restart", and a system for restarting the store from the admin.
I agree that the config system is great. This is a minor wart and we should try to fix but I think it's a smaller priority than the other things.
NewForms Admin
I think we may as well move to NewFormsAdmin almost immediately. 2.0 won't be ready before the official release, and it looks so nice ... We'll be avoiding so much rework if we start sooner rather than later.
Improving the admin interface is my #2 priority. In truth, improving this will also be necessary for improving management of the product models. I also agree that all of our work should be focused on using newforms admin. I've been hoping that it would make it to trunk soon but if we're going to do a major rework for 2.0 we might as well start now.
In summary, here's what I think we should do:
1. Hash out the new product model structure
2. Move Satchmo 2.0 to using newforms admin and start making the required migration changes
3. Highlight the refactoring/reorganizing we'd like to do. Get alignment and start doing it
4. Discuss our pluggability ideas and settle on a design.
5. Agree on documentation and testing standards as we move through the overhaul.
Should be lots of fun. Hopefully we can make some good progress and maybe we can have Satchmo 2.0 when Django 1.0 hits the streets ;)
-Chris
STemplar
it) the things you point to are really important.
If Satchmo was more of a framework than a regular app I would start
using it right away.
I'll respond to some of your topics here (in a different order
though).
I know this reply is a bit late but I hope it's relevant anyway.
> Refactoring
Looking at the code I think the models are nice thought out, but the
general framework is a bit confusing. To me it would make more sense
to have a couple of separate applications (contact, products, cart,
checkout/payment) that work together in a more loosely coupled and
clearly defined way. Preferrably making it possible to put custom
modules in and affecting behaviour through different hooks. For
example it should be possible to use the checkout module without the
regular cart and use an existing user/contact database for buyer
information.
Is this what you are heading for?
Anyway, I agree with the things you mention. Only question is: Can I
help out here? As I'm about to build a webshop it would be nice to
build it on Satchmo, but since I haven't followed the development I'm
not sure where to go.
> NewForms Admin
This is another main issue why I'm not already using Satchmo. Just
seems like waste of time to develop anything for oldforms-admin. And
it feels a bit difficult as a newbie to port it over to newforms.
> Pluggability
For adapters and interfaces you might want to look at Zope interfaces
as well. It seems well accepted; for example the the Twisted framework
replaced their own interface code with it and it seems to work well.
Have a look at:
http://www.stereoplex.com/two-voices/adapters-in-django-and-the-revenge-of-zope
Also found a doc regarding different Python plug-ins at
http://www.pitivi.org/wiki/Design_Docs_Plugins
The text may or may not be useful but there's a lot of links to
different plug-in systems at the end.
> Tests
> I think we should get more aggressive about requiring tests.
myself so some document covering coding guidelines and test examples
would be highly valuable.
Btw, as this topic is "Post 1.0". How far into the future are these
things?
Best regards,
Simon
Bruce Kroeze
> Refactoring
To me it would make more sense
to have a couple of separate applications (contact, products, cart,
checkout/payment) that work together in a more loosely coupled and
clearly defined way.
Agreed, just today I took a step toward that goal by removing newsletter app code from the contact app - instead using a signal which the satchmo newsletter (or some other implementation) can pick up or not. I think there is a long way we can go down that road, unweaving some of the code and using signals to handle it in a hopefully clearer and more well-defined way.
Preferrably making it possible to put custom
modules in and affecting behaviour through different hooks. For
example it should be possible to use the checkout module without the
regular cart and use an existing user/contact database for buyer
information.
Ouch. Maybe. I agree it would be nice but ouch. That would be quite an achievement. I think it more likely that we would have a base model which would be required, and then signals for other models/apps who want to stay in sync or be notified about changes. At least initially.
Anyway, I agree with the things you mention. Only question is: Can I
help out here? As I'm about to build a webshop it would be nice to
build it on Satchmo, but since I haven't followed the development I'm
not sure where to go.
You can help, of course. We can always use programming help. I think we are quite open to patches or apps.
> NewForms Admin
This is another main issue why I'm not already using Satchmo. Just
seems like waste of time to develop anything for oldforms-admin. And
it feels a bit difficult as a newbie to port it over to newforms.
That is at nearly the top of my personal todo list for post-1.0 .
> Pluggability
For adapters and interfaces you might want to look at Zope interfaces
as well. It seems well accepted; for example the the Twisted framework
replaced their own interface code with it and it seems to work well.
Zope makes me instinctively want to run screaming. It is one of the few Python apps that I will never accept another paying gig to work with. Frankly, I'd rather use PHP/Drupal, and that's saying a lot. But that is more from the nice-idea-but-not-in-the-real-world object database, the crazy admin aspect (now where do I set that setting? Hmmm. Here? No here. No ... here! Darn, it doesn't seem to be right. Let me look again ...) and the absurdly overblown, wordy and intrusive xml templating, not so much the programming. I guess the interfaces are worth looking at.
Btw, as this topic is "Post 1.0". How far into the future are these
things?
I think we are ready to release 1.0 any day now. Chris is working on a UPS module, and when it is done I think we are likely to release.
---
Nicola Larosa
> I guess the interfaces are worth looking at.
On the other hand, I distinctly remember a discussion where several
Django developers expressed displeasure with Zope interfaces, seeing
them as overkill.
Unfortunately I can't seem to find the discussion on django-developers
right now, but I mean to try again.
--
Nicola Larosa - http://www.tekNico.net/
We should take our cue from what happens when we fall in love: time
stops, goes away. We are there in the moment, and it lasts almost
forever. So the change over this infinite time is so gradual it is
no noticeable change at all. Love change, embrace it, slow it down,
make it last, and the stress disappears. -- Dave Pollard, August 2007
Marty Alchin
>
> Bruce Kroeze wrote:
> > I guess the interfaces are worth looking at.
>
> On the other hand, I distinctly remember a discussion where several
> Django developers expressed displeasure with Zope interfaces, seeing
> them as overkill.
>
> Unfortunately I can't seem to find the discussion on django-developers
> right now, but I mean to try again.
I don't know if I've spoken up on this, but I'm one of the Django
developers who extremely dislikes Zope interfaces. I just did some
more thinking about it today, actually, and I think I know the biggest
issue I have with it: what we're really after are components, not
interfaces (going by Zope terminology), but with Zope, components
require the use of interfaces. At least, as far as I've been able to
tell. So to get one potentially useful feature, we're stuck with an
extremely useless feature and a painful API.
So, what would be optimal is if we could implement a pluggable
component system without the overhead of interfaces. I did that this
afternoon in about 35 lines of code. I still want to look it over a
few more times before I'm sure that it'll actually do the job as well
as it needs to, but I think it should be quite possible to implement a
much simpler "framework" for dealing with pluggability in a much more
Django-friendly manner.
I'll write it up tomorrow if it looks promising, but regardless of
whether it's my code or not, I think there's a lot of potential for
coming up with something that doesnt rely on Zope's interface system.
-Gul
Bruce Kroeze
On 1/9/08, Nicola Larosa <nicola...@gmail.com> wrote:
>
> Bruce Kroeze wrote:
> > I guess the interfaces are worth looking at.
>
> On the other hand, I distinctly remember a discussion where several
> Django developers expressed displeasure with Zope interfaces, seeing
> them as overkill.
I don't know if I've spoken up on this, but I'm one of the Django
developers who extremely dislikes Zope interfaces.
So, what would be optimal is if we could implement a pluggable
component system without the overhead of interfaces. I did that this
afternoon in about 35 lines of code.
I'll write it up tomorrow if it looks promising, but regardless of
whether it's my code or not, I think there's a lot of potential for
coming up with something that doesnt rely on Zope's interface system.
I'd *love* to see that. I am quite suspicious of these straitjacket, overkill interfaces when really I just want a way of declaratively stating a contract. In fact, that's why I like signals so much. No overkill (though some do argue differently), ignore them if you like, but they provide useful features in a clear and straightforward manner.
STemplar
> Ouch. Maybe. I agree it would be nice but ouch. That would be quite an
> achievement. I think it more likely that we would have a base model which
> would be required, and then signals for other models/apps who want to stay
> in sync or be notified about changes. At least initially.
just based on that looking at for example the checkout/payment module
it seems to me like it could be a quite self-contained app. It's
pretty much only the order data and contact details that need to be
there and the contact details are as it is already being copied there
from Contact objects. So I imagined that you could have a similar
approach where the cart data is transferred to a order through an API
of some sort.
This may or may not be feasible in real life, I'm not sure. I might be
missing out on some factors.
> Zope makes me instinctively want to run screaming...
I'm causing too much pain here. I'm not voting for it, but since you
considered Tracs variant I just wanted to point it out.
If there's a solution as short and sweet as 35 locs, that seems like a
winner.
Looking forward to the release and hopefully I can contribute
something in the future.
/Simon
Marty Alchin
I've written it up on my blog[1], and I've added it to
djangosnippets[2] as well, since it ended up being just 6 lines for
the essentials. It's not much, but it does exactly what needs to be
done, without any of the unnecessary overhead. You're welcome to add
utilities to it, of course, but ultimately, anything added to it would
just be helper utilities. There's nothing essential missing (that I
can see, anyway). Feedback is welcome!
-Gul
[1] http://gulopine.gamemusic.org/2008/jan/10/simple-plugin-framework/
[2] http://www.djangosnippets.org/snippets/542/
Bruce Kroeze
On Jan 9, 2008 6:35 PM, Bruce Kroeze <bkr...@gmail.com> wrote:
> On Jan 9, 2008 2:52 PM, Marty Alchin < gulo...@gamemusic.org> wrote:> > I'll write it up tomorrow if it looks promising, but regardless ofI've written it up on my blog[1], and I've added it to
> > whether it's my code or not, I think there's a lot of potential for
> > coming up with something that doesnt rely on Zope's interface system.
>
> I'd *love* to see that. I am quite suspicious of these straitjacket,
> overkill interfaces when really I just want a way of declaratively stating a
> contract. In fact, that's why I like signals so much. No overkill (though
> some do argue differently), ignore them if you like, but they provide useful
> features in a clear and straightforward manner.
djangosnippets[2] as well, since it ended up being just 6 lines for
the essentials. It's not much, but it does exactly what needs to be
done, without any of the unnecessary overhead.
Great write-up Marty. I think it does just about exactly what we would want to do for more robust and documentable plugin management. As small as needed to accomplish what we need with not a wasted function in sight.
I've been adding a lot of signals and listeners to Satchmo lately, which are helping to untangle some of the code, but they feel awkward for some purposes. I think this is much more maintainable moving forward for those purposes.
I'm curious to see what other Satchmo devs think about it. Should we use Marty's minimal framework?
Chris Moffitt
I'm curious to see what other Satchmo devs think about it. Should we use Marty's minimal framework?
I like the fact that the framework is really small and should be fairly easy to add to Satchmo. I'm still trying to wrap my head around how we actually use it. Could we walk through some use cases to show how it would be integrated and how it would help us out? Please understand that it may be painfully obvious to everyone else and that I'm just a little slow ;)
-Chris
Bruce Kroeze
OK, here's an example.
When a new shop registration is created, we fire various signals at various times
satchmo_registration = object()
satchmo_registration_verified = object()
satchmo_registration_initialdata = object()
These are pretty opaque. To see how they are used, you have to go look at where they are fired and what payload is getting sent with the signal, also what is coming back from the signal.
It is not clear how someone would write a plugin and make sure it gets registered so that his custom call gets made to tweak the data on the user object - or possibly to sync with an outside DB - when a new person is registering.
For an example of this in action - let's use satchmo.newsletter.
You can see the current signal thrown in satchmo.accounts.forms.RegistrationForm.save()
And it caught in satchmo.newsletter.__init__.py
If instead we were to use Marty's framework, we'd declare something like this:
class RegistrationPlugin:
__metaclass__ = PluginMount
def handle_new_registration(self, contact, data={}):
"""Update for a new contact, not necessarily verified."""
pass
def handle_verified_registration(self, contact):
"""Update for a fully verified registration"""
pass
def validate_registration_data(self, formdata):
"""Possibly veto a registration based on some plugin-specific criteria"""
return True
In satchmo.accounts.forms.RegistrationForm.save, instead of:
dispatcher.send(signal=signals.satchmo_registration, contact=contact, subscribed=subscribed, data=data)
The code would say:
for p in self.plugins :
p.handle_new_registration(contact, data)
Then, to port my newsletter example, I'd do something like this:
class NewsletterRegistrationListener(RegistrationPlugin):
def handle_new_registration(self, contact, subscribed=False):
"""Update newsletter subscriptions."""
update_subscription(contact, data['subscribed'])
I think it is much more self-documenting and obvious to use.
Chris Moffitt
- Where do we place the plugin calls? (I realize this isn't necessarily difficult but it should be thought out)
- Do we have any limits to what a plugin could do? If we have a plugin to do some contact updating, what if we have another plugin that interacts negatively with another one in the stream? This might just be another documentation issue but we need to be clear about it.
- Finally, is there a performance (speed/memory) ramification of sprinkling plugin calls all over the place? I thought that signals were relatively expensive, do we have similar concerns here?
-Chris
Bruce Kroeze
Ok. The example helped me out a lot. I think we would need to think through a couple of things:
- Where do we place the plugin calls? (I realize this isn't necessarily difficult but it should be thought out)
The plugin calls would go wherever needed. I've been doing some replacement of existing code interwingling by using the signal method for a while now, and it really helps untangle things. The example I gave above is a perfect one. Satchmo reg doesn't need to know a thing about newsletters, so it shouldn't have to import it. A signal (or a plugin) is much cleaner.
Generally, I find the signals or plugin calls go best when attached to methods on Forms and Models.
- Do we have any limits to what a plugin could do? If we have a plugin to do some contact updating, what if we have another plugin that interacts negatively with another one in the stream? This might just be another documentation issue but we need to be clear about it.
There's no built-in limit. I think plugins just need to be careful about testing assumptions before acting. Just because the contact was valid when the plugin method got called doesn't mean it still is by the time the specific plugin method gets called.
- Finally, is there a performance (speed/memory) ramification of sprinkling plugin calls all over the place? I thought that signals were relatively expensive, do we have similar concerns here?
The performance cost of an calling a plugin list which had no registered plugins would be the same as iterating over an empty list. Very very small.
BTW, I think that is a common misconception about signals being slow/expensive. They are extremely quick by themselves. What is slow, and what people have complained about on the main Django list is having them integrated into models by default. Every save, and every instantiation of a model emits a signal. That gets expensive. And the complaint is that it is expensive with no "opt-out" option, since they are baked into Django. Just manually calling a signal on some interesting hook point is very cheap.
--
Bruce Kroeze
http://solidsitesolutions.com (also done in Django, using a custom multilingual-aware CMS app)
Dynamic Designs, Optimized
Marty Alchin
> The performance cost of an calling a plugin list which had no registered
> plugins would be the same as iterating over an empty list. Very very small.
>
> BTW, I think that is a common misconception about signals being
> slow/expensive. They are extremely quick by themselves. What is slow, and
> what people have complained about on the main Django list is having them
> integrated into models by default. Every save, and every instantiation of a
> model emits a signal. That gets expensive. And the complaint is that it is
> expensive with no "opt-out" option, since they are baked into Django. Just
> manually calling a signal on some interesting hook point is very cheap.
I wouldn't call signals "slow" on their own, exaclty, but there is
definite concern. As you mention, Django by default makes extensive
use of signals, so any speed concerns there might be are greatly
amplified. The real problem, though, is how PyDispatcher locates
listeners to call. It has a fairly complicated data structure it has
to look through, containing all listeners registered for all signals
(even though you're only dispatching one) and then it has to inspect
each listener to figure out which arguments to send it.
That all adds flexibility, of course, since you can register a
listener for *any* signal, or for *any* sender (or theoretically
both). So PyDispatcher really doesn't know which listeners to call
under the call itself is made. That whole location process takes time.
It's not earth-shattering for most projects, but for some, that time
really adds up. There are some enhancements being considered, since
Django doesn't really make use of the "any signal" dispatch, but those
haven't been finalized yet.
But you're right, the time it takes to cycle through plugins is
directly related to how many are registered for that particular mount
point. If there aren't any, it's an empty list; very, very quick. Even
if there are dozens though, a single list can be traversed much more
quickly than the dispatcher's nested dictionary structure. And since
there's no need to inspect arguments or anything, it's all that much
quicker still.
-Gul