Best practices for functional encapsulation
BrianTheLion
I mentioned some time ago that I was working on an essay about
functional encapsulation in pylons. As a point of reference I'm
interested to know how other people are dealing with the following FE
problem, one that I also encounter in my code:
You're building an application that's composed of a number of distinct
capabilities, say a wiki, instant messaging, and shopping cart
functionality. You plan to wrap the whole application with a single
authentication mechanism. How do you do it? Which facilities in the
pylons suite do you use to build and test each of the capabilities
independently? Which do you use to integrate those independent
capabilities into the final application?
I'd love to hear, especially, from folks who've been doing this for a
while. There's an issue of *process* that I'd like to treat as
thoroughly as the issue of code. What's your process?
Cheers! And happy New Year!
~br
BrianTheLion
Marius Gedminas
> Bump?
Yeah.
> On Jan 1, 9:59 am, BrianTheLion <rossa...@hotmail.com> wrote:
> > You're building an application that's composed of a number of distinct
> > capabilities, say a wiki, instant messaging, and shopping cart
> > functionality. You plan to wrap the whole application with a single
> > authentication mechanism. How do you do it? Which facilities in the
> > pylons suite do you use to build and test each of the capabilities
> > independently? Which do you use to integrate those independent
> > capabilities into the final application?
That question always interested me (in the abstract -- I didn't have any
concrete problems that needed to be solved that way).
I get the impression that the answer is "want modular applications?
Switch to Pyramid."
Marius Gedminas
--
"Nothing ever goes missing that they don't look at me, ever since that
time I lost my horse. As if that could be helped. He was white and it
was snowing, what did they expect?"
-- Dolorous Edd in "A Storm of Swords" by George R. R. Martin
Daniel Holth
Corollary: WSGI is not the right layer for composing applications.
Tony Theodore
I haven't actually done anything with pyramid yet, still working
through much the same questions you ask. This snippet:
def __call__(self, view):
return self.attr_wrapped_view(
self.predicated_view(
self.authdebug_view(
self.secured_view(
self.owrapped_view(
self.decorated_view(
self.rendered_view(
self.mapped_view(view))))))))
from this blog post:
http://plope.com/pyroptimization
has been guiding my thoughts since I read it.
Cheers,
Tony
Joost Moesker
section their application into distinct packages. Recently i have been
working with drupal and magento, and I must say i really appreciate
how these project separate functionality into packages. For a large
project this really is a must have, not only for reuse but also
maintainability and developer sanity. Keep it small, keep it simple.
So how does pyramids help in this respect? I don't see how the object
dispatch mechanism really makes a difference here (been there done
that with cherrypy & turbogears). The primary issues is how one wire's
together the different api's. Eg. a mini-app requires a user model
which has a permissions attribute. How does this mini-app publish this
requirement? And how can I wire up an app that supplies this api up
with this mini-app?
Secondly, how can i hook up parts of one mini-app into an another app.
For example, an e-commerce site might have 'productpage' showing
product details. This would be part of the catalog app. Yet i would
also like to show a mini/cart in the template. This mini-cart is part
of the shoppingcart app. I don't want shoppingcart logic in my
productpage controller, so how do i integrate these apps. Django´s
signals seems an interesting solution yet also obscure. I got a gut
feeling the problem can be simply solved with plain polymorphism and
ducktyping. We don´t need to create a new dispatch mechanism, it´s
already build into the language.
Honestly, I have been fighting with these issues in various MVC
frameworks for years and haven't found a satisfactory pattern.
Currently i'm trying to grok how magento does this with the Zend
framework. At the moment it still seems overly complex to me, to much
configuration. But that might be naivety on my side. I´m really
looking forward how other developers are handling this.
On 10 jan, 08:37, Marius Gedminas <mar...@gedmin.as> wrote:
> ...
> Switch to Pyramid."
Mike Orr
> I'm also interested in the patterns used by other developers to
> section their application into distinct packages.
I have a Pylons application with several autonomous parts under a
common auth mechanism. I wanted to split them up into packages or
controller-routes-templates bundles but it was so difficult under
Pylons I gave up. You can mount a WSGI app as a pseudo-controller, or
have an action forward to one. The problem is how the Pylons
environment is tightly integrated:
- the magic globals
- the 'full_stack' flag exists in middleware.py but nobody has
mentioned using it, so I don't know how well a nested app + middleware
would work in practical terms
- there's only one place for templates in the application template,
and no clear guidance on how to structure subapp templates, whether
they should have a separate TemplateLookup, etc.
- unclarity on whether you can have controllers in subdirectories.
(You can but the 'controller' variable has to be slash delimited
rather than dot delimited, but again there was little user feedback on
how well this worked.)
- what if you want to share 'app_globals' variables with the subapp,
or conversely you want to prevent them from leaking to the subapp?
This is a case where the much-maligned StackedObjectProxies are
necessary, and would they be reliable?
- What to do about INI settings for sub apps?
- We don't have a local PyPI mirror so we can't easy_install local
dependencies; we have to install them by hand from our Subversion
repository. So a lot of separate packages is not necessarily a good
thing in this environment.
Some of these I have tentative answers to, but it was such an
undocumented and underchartered territory that I didn't want to risk
it in our most critical production app. So instead I put everything in
one application, making a template subdirectory for each part, a
separate controller (or three), and put all the routes together.
> So how does pyramids help in this respect?
One of Pylons 2's goals was to make it more friendly to nested apps;
e.g., provide an alternative to the magic globals via instance
variables in the controller. The BFG architecture is much more
friendly to nested apps, which is one reason why it was chosen.
There's also a cultural difference with Pyramid: a willing to deviate
from application templates. There are fewer interdependencies between
parts of the application, and those that exist are well-known
containment patterns; i.e., attributes and keys. The "component
architecture" is magic to some of us, but ordinary app developers
don't have to see it, and we know it's running well in every other
Pyramid app because those aren't breaking. Required middleware doesn't
exist in Pyramid, and optional middleware is in the INI file rather
than in a middleware.py. *mypyramidapp/__init__.py* is shorter and
more straightforward than *mypylonsapp/config/*. 'app_globals' doesn't
exist so all its complications disappear. All state data is under the
'request' object in straightforward attributes. Pyramid has documented
ways of doing authentication, traversal lookup, and component piecing
that Pylons never had. Etc.
So all of these things mean Pyramid should work better with nested
apps, but i don't know how well these ideas have been tested. The BFG
developers among us may have more experience with nested apps.
--
Mike Orr <slugg...@gmail.com>
Mike Orr
> WSGI is not the right layer for composing applications.
Most WSGI framework developers have come to the conclusion that
middleware is not all it's cracked up to be. The WSGI protocol is
clunky and you have to do weird things to share data between layers.
Route middleware, session middleware, and database middleware were
good experiments but they showed this is not the best place for those.
With Pyramid's event system you can plug in code at the beginning and
end of every request and at other times, which can perform tasks
otherwise done by Pylons BaseController.__call__ or middleware.
The developers tried for three or four years to streamline the WSGI
protocol ("WSGI 2") but could never get consensus, and now there's a
realization that just turning away from WSGI may be better. Not to
eliminate WSGI in its basic purpose (connecting to a Python
webserver), but to use other protocols for other stuff: WebOb, Pyramid
components, etc.
--
Mike Orr <slugg...@gmail.com>
Chris McDonough
> One of Pylons 2's goals was to make it more friendly to nested apps;
> e.g., provide an alternative to the magic globals via instance
> variables in the controller. The BFG architecture is much more
> friendly to nested apps, which is one reason why it was chosen.
> There's also a cultural difference with Pyramid: a willing to deviate
> from application templates. There are fewer interdependencies between
> parts of the application, and those that exist are well-known
> containment patterns; i.e., attributes and keys. The "component
> architecture" is magic to some of us, but ordinary app developers
> don't have to see it, and we know it's running well in every other
> Pyramid app because those aren't breaking. Required middleware doesn't
> exist in Pyramid, and optional middleware is in the INI file rather
> than in a middleware.py. *mypyramidapp/__init__.py* is shorter and
> more straightforward than *mypylonsapp/config/*. 'app_globals' doesn't
> exist so all its complications disappear. All state data is under the
> 'request' object in straightforward attributes. Pyramid has documented
> ways of doing authentication, traversal lookup, and component piecing
> that Pylons never had. Etc.
>
> So all of these things mean Pyramid should work better with nested
> apps, but i don't know how well these ideas have been tested. The BFG
> developers among us may have more experience with nested apps.
It is "Pyramidic" to compose multiple external sources into the same
configuration using "config.include" as documented (poorly) here:
<http://docs.pylonsproject.org/projects/pyramid/dev/narr/advconfig.html#including-configuration-from-external-sources>
Any number of includes can be done to compose an application; includes
can even be done from within other includes. Any directives can be used
in an include that can be used outside of one (such as add_view, scan,
add_handler, add_renderer, etc).
Pyramid has a conflict detection system that will throw an error if two
included externals try to add "the same" configuration in a conflicting
way (such as both externals trying to add a route using the same name,
or both externals trying to add a view with the same set of predicates).
It's awful tempting to call this set of features something that can be
used to compose a system out of "pluggable applications". But in
reality, there are a number of problems with claiming this:
- The terminology is strained. Pyramid really has no notion of a
plurality of "applications", just a way to compose configuration
from multiple sources to create a single WSGI application. That
WSGI application may gain behavior by including or disincluding
configuration, but once it's all composed together, Pyramid
doesn't really provide any machinery which can be used to demarcate
the boundaries of one "application" (in the sense of configuration
from an external that adds routes, views, etc) from another.
- Pyramid doesn't provide enough "rails" to make it possible to
integrate truly honest-to-god, download-an-app-from-a-random-place
and-plug-it-in-to-create-a-system "pluggable" applications.
Because Pyramid itself isn't opinionated (it doesn't mandate a
particular kind of database, it offers multiple ways to map URLs
to code, etc), it's unlikely that someone who creates something
"application-like" will be able to casually redistribute it
to J. Random Pyramid User and have it "just work" by asking him
to config.include a function from the package.
This is particularly true of very high level components such
as blogs, wikis, twitter clones, commenting systems, etc.
The "integrator" (the Pyramid developer who has downloaded a
package advertised as a "pluggable app") will almost certainly
have made different choices about e.g. what type of persistence
system he's using, and for the integrator to appease the
requirements of the "pluggable application", he may be required
to set up a different database, make changes to his own code
to prevent his application from "shadowing" the pluggable
app (or vice versa), and any other number of arbitrary
changes.
For this reason, we claim that Pyramid has "extensible" applications,
not pluggable applications. Any Pyramid application can be extended
without forking it as long as its configuration statements have been
composed into things that can be pulled in via "config.include".
It's also perfectly reasonable for a single developer or team to create
a set of interoperating components which can be enabled or disabled by
using config.include. That developer or team will be able to provide
the "rails" (by way of making high-level choices about the technology
used to create the project, so there won't be any issues with plugging
all of the components together. The problem only rears its head when
the components need to be distributed to *arbitrary* users.
Note that Django has a similar problem with "pluggable applications"
that need to work for arbitrary third parties, even though they provide
many, many more rails than does Pyramid. Even the rails they provide
are not enough to make the "pluggable application" story really work in
reality.
Truly pluggable applications need to be created at a much higher level
than a web framework, I fear, as no web framework can offer enough
constraints to really make them "work out of the box". They really need
to plug into an application, instead. It would be a noble goal to build
an application with Pyramid that provides these constraints and which
truly does offer a way to plug in applications (Joomla, Plone, Drupal
come to mind).
- C