Two great posts from Christian
Remi Delon
I just want to share the link to a great post by Christian (dowski) showing
off the improved WSGI support on CherryPy-2.2.0-rc1:
It shows how you can run multiple WSGI-CherryPy apps using other tools like
wsgutils or Paste (or do the opposite: run a PyBloxsom or MoinMoin WSGI app
within CherryPy):
http://blog.dowski.com/2006/03/08/cherrypy-and-wsgi-can-play-nice/
Christian has also put together a great screencast showing how to run
CherryPy and interact with it directly from the prompt:
http://blog.dowski.com/2006/03/05/cherrypy-and-the-interactive-interpreter/
Note that we've also been working on a new documentation system (one of the
weakest
points of CP right now ;) ) that will be live this week-end.
Remi.
Jason Earl
> Hello everyone,
>
> Note that we've also been working on a new documentation system (one
> of the weakest points of CP right now ;) ) that will be live this
> week-end.
Perfect, I can't wait.
Jason
gog...@gmail.com
been working with CP for a while now and know what it can do, I just
have no idea how the internal stuff works, rather than I know address
get mapped to objects and callables and stuff. If you guys dont mind
can you explain in simple terms how this is an improvement ?. Pretty
please, i crave to know more.!
Kevin Dangoor
This is, I think, part of the reason that Ian Bicking has had trouble
selling Paste. Describing this stuff to someone who's not doing a web
framework has sometimes seemed to be difficult.
The next major TurboGears release is entirely based on making good use
of WSGI. Here are the real-world benefits:
1) Application composition: imagine that you've written a CherryPy
blog and a separate CherryPy wiki. These changes will let you attach
both the blog and the wiki to a single site and even do some
interesting cross-app integration.
2) Nifty "middleware": a couple months ago, Ian Bicking did a
screencast of his "eval exception" package. That package would trap an
exception (during development) and let you evaluate expressions on the
server side in the context of the exception to help you figure out
what went wrong. Very nifty. WSGI middleware is similar to CherryPy
filters, but a bit lower level and applied a bit differently.
3) including other apps - as Christian points out, you can toss a
MoinMoin into your app if you wish, even though it has nothing to do
with CherryPy! You have more ability to build up an entire site in
Python without worrying as much about how each part is implemented.
4) code sharing among frameworks - this is the least tangible one for
end users, but the results can still be impressive. The middleware is
applicable to any web framework that uses WSGI. Each server doesn't
*have* to implement their own session handling or caching or exception
handling or transaction handling. If a component is designed to work
within the WSGI realm, the possibiltiy to reuse it between frameworks
is much greater. Ultimately, this should result in better software,
because there are more people looking at a single bit of code.
Kevin
Uche Ogbuji
This is a nice summary, Kevin. I think it should be worked in, with
Christian's posts and more into an executive summary of
WSGI^H^H^H^HPortable Web components for CherryPy. As I've been
stressing on #cherrypy IRC I think there is a lot of confusion among
users, and that this doesn't help meet anyone's goals. As an example,
I think it needs to be made crystal clear somewhere that WSGI != Paste,
preferably with an overview of of what optional things Paste builds on
top of WSGI. My sense as an observer is that CherryPy will move to
embrace WSGI, but not necessarily Paste, and someone should be as clear
as possible as to the reasons.
And for you, Keven, it would be very valuable to get your perspective
in trems of TurboGears direction. You say "The next major TurboGears
release is entirely based on making good use of WSGI." Are you already
sold on Paste as the basis for this? Either way, how does that affect
your view of CherryPy as the server framework for TG? If you really
are more interested in WSGI than full on paste, what are the remaining
pain points you're aware of in using WSGI with CherryPy? Does
Christian's wsgi_filter make a difference? Please forgive me if some
of these questions are inane. I'm pretty much trying to represent the
confusion of the typical user. I've spent some time trying to
understand this whole CherryPy/WSGI/Paste SOAP Opera, and I'm lost, so
I can imagine how others feel. I think clear communication of
motivations, goals and resolutions will really help everyone avoid
duplication of work, impenetrable arguments and even bewildering forks.
Robert suggested a wIki for the discussion. Sounds good to me.
But in the meantime, do you have any nuggets of further illumination to
share?
Thanks much.
--
Uche Ogbuji Fourthought, Inc.
http://uche.ogbuji.net http://fourthought.com
http://copia.ogbuji.net http://4Suite.org
Articles: http://uche.ogbuji.net/tech/publications/
Kevin Dangoor
> And for you, Keven, it would be very valuable to get your perspective
> in trems of TurboGears direction. You say "The next major TurboGears
> release is entirely based on making good use of WSGI." Are you already
> sold on Paste as the basis for this? Either way, how does that affect
> your view of CherryPy as the server framework for TG? If you really
> are more interested in WSGI than full on paste, what are the remaining
> pain points you're aware of in using WSGI with CherryPy? Does
> Christian's wsgi_filter make a difference? Please forgive me if some
> of these questions are inane.
These questions aren't inane at all.
My perspective always starts with the user (in this case, a developer
using TurboGears). I really wanted TurboGears 0.8 to allow you to take
multiple TurboGears apps and join them together in a site. That didn't
quite work out. A whole bunch of other stuff came up for 0.9, but I've
always had this in the back of my mind.
I also understand that sometimes you really want to combine things at
deployment time in a config file, rather than in a Python file. I have
additionally recognized some use cases for which something like Routes
works better than CherryPy's dispatch (a great example of this is
backwards compatibility with existing URLs).
Either way, I've wanted for people to be able to download and install
a blog, for instance, and have it running trivially on their site
alongside the rest of their code.
Since TG 0.8, I've also seen interesting WSGI middleware cropping up
from time to time, like Ian's evalexception. I only expect that to
increase.
So, the above couple of paragraphs really summarize what I'm after for
that next major release of TG (codename "First Class"). My goal is to
get these things in the quickest and cleanest way possible. It pays to
not get too attached to a given piece of code, because that can
sometimes be blinding when a better alternative comes along.
I'm not 100% fond of the syntax, but PasteDeploy's config files are
very powerful ways to hook things up. And the syntax can be smoothed
over. So, I do expect PasteDeploy to be involved. First Class already
adds a tg-admin serve command that fires up a server based on a
configuration. That will almost certainly remain in First Class. But,
there's no reason that CP's WSGI server can't be handling the requests
in that command!
I'm also leaning toward a name-based config rather than a path-based
config. The idea is that you can give applications and controllers
names and refer to them by those names in the config file. Those apps
can then be attached to whichever URLs you need. While this is
different from CherryPy's configuration, I don't think it's
incompatible. Doing a translation from name to path shouldn't be too
difficult. (TurboGears would likely be doing cherrypy.tree.mount()s
behind the scenes on behalf of the user.)
The thing that is a question is whether CherryPy does the dispatch to
a method or RhubarbTart does. There are plusses and minuses on both
sides (isn't that always the way of it?). Right now, First Class uses
RhubarbTart, because that enabled the "Python-style" of application
embedding. Christian's wsgi_filter may do that as well, though.
Right now, there's no final answer on what does the dispatch in TG First Class.
Kevin
gog...@gmail.com
applications I have been working on are very simple and only run on the
desktop, I have 1 that I'm about to complete and 1 still on the drawing
board. This would be a great way to build big applications by building
small parts first and then glueing all together. Thanks for the
explanation.
Ben Bangert
really illustrates a key thing WSGI is bringing about, and Paste
somewhat capitalizes on.
The need for something like CherryPy is not what it used to be.
Consider this great summary of where cherrypy is mentioned:
CherryPy currently stands on its own as a robust, simple,
doesn't-make-decisions-for-you web framework. It provides a Pythonic
way to write applications that utilize HTTP. It is also a key part in
the TurboGears framework (for now, at least). It is approaching a
stable 2.2 release.
>From Christians post:
http://blog.dowski.com/2006/03/28/ye-olde-future-of-cherrypy/
Now, given how WSGI makes it easy to write web applications on a very
low level, and its rather easy to write basic object abstractions on
top (like a request API object, response object, etc.). What else do
you need? A dispatcher? Plug one in, there's Routes, and RhubarbTart
implemented a object-path dispatch similar to CP's.
If filters are dropped, and WSGI is embraced more openly in CherryPy,
CherryPy itself is essentially destroying that which makes it *not*
WSGI. In the end, what will be left is a light layer on top of WSGI
that isn't as clean as something designed as such, since CP has a whole
bunch of stuff in it thats essentially pre-WSGI legacy stuff.
As RhubarbTart demonstrates, you can have a simple, light layer on top
of WSGI, thats as minimal as possible, and as powerful as possible
without a lot of work or code.
Where does this leave CherryPy and frameworks in general? That's a
great question that I think we're just going to have to find out.
Pylons and RhubarbTart are the first, and there's many, many little
frameworks popping up. Most of the time people don't even bother
releasing them, they're so trivial to make.
I've had more than a dozen emails from people using Routes to dispatch
in their own little WSGI-based frameworks that they created solely for
their applications. The WSGI components out there continue to lower the
bar... until soon even complete novices will be able to put together
exactly what they want. Since these parts can very concise and
contained, I'd consider them more reliable, robust, and easier to have
comprehensive unit tests. How can a hulking, hard-to-test, less
flexible 'framework' compete with that?
Personally, I think CherryPy is for the most part obsolete in the face
of WSGI driven web development. I know other people who are into WSGI
have said similar things, though not out in the open. Hopefully I won't
catch too much flak for what a bunch of people have known for awhile.
Pylons is merely a configuration of WSGI parts. TurboGears FirstClass
looks like it will be the same thing, with a slightly different
configuration. It's quite likely a good amount of those WSGI parts will
be the same code and packages.
PasteDeploy isn't necessary to deploy an app using other Paste parts.
You can in just a few lines of Python code start-up a WSGI app using
Paste parts just like you can start up a Cherry-Py server.
There's a lot of great code in CherryPy, the CP WSGI server is very
fast, and I'm sure there's a bunch of other stuff I'm not aware of as
well. These are the things CP should harp more on, and I'd like to hear
more about them so I know what CP is adding that WSGI + a few objects
and a dispatcher don't give me.
Cheers,
Ben
Ben Bangert
prolly Wareweb or one of Ian's creations. :)
Sylvain Hellegouarch
Before that my message gets wrongly interpreted I would like to say that
even though I have not produced any code WSGI-compliant yet I have
indeed read carefully the PEP333 and see how a few web frameworks such
as Pylons use it.
WSGI is a very nice interface that has a great potential and it is true
that today it is impossible to avoid it. That being said WSGI also has
unanswered questions because it has not really been used in large
production or pushed to its limits AFAIK.
Many people keep talking about the "plugability" brought by WSGI
middleware and applications.
Well WSGI has a couple of issues on this one that I would be happy if
anyone could clarify.
1. Imagine I write a self-contained WSGI application using a couple of
middlewares I made on my own. Say for example a gzip filter and one that
changes the Content-Type based on content negotiation with the UA (eg.
returning application/xhtml+xml when supported). Now imagine I release
my application as a bundle so that users could simply drop it on their
server (this is basically what many people have marketed about until
now). But wait, due to the design of their server, they might already
have several middlewares running just on top of their servers because
they want to ensure that they only serve text/html or they do a
different kind of compression by default which is not gzip. Well you
will tell me they can simply configure my application so that they get
rid of my middlewares right. True indeed, but then whose to say my
application will work the same? Can you ensure that? Well you could come
back to me and check it out. Everything is possible right... I'm being
stupid here I know but the point is the whole
WSGI-plug-it-in-as-you-wish will lead to issues that might be minors but
could also be more annoying and I find a bit strange no one raises those
potential issues. Pluggability is never that transparent.
2. Another point that worries me regarding PEP 333 is that it leaves
many decisions unreplied. Things such as:
"""Applying these principles to a variety of HTTP features, it should be
clear that a server *may* handle cache validation via the If-None-Match
and If-Modified-Since request headers and the Last-Modified and ETag
response headers. However, it is not required to do this, and the
application *should* perform its own cache validation if it wants to
support that feature, since the server/gateway is not required to do
such validation."""
Well there is something annoying here as it means an application
developer which needs HTTP caching on will need to add it by default to
its application as it won't be sure the underlying HTTP WSGI server will
support it. So much for less duplication of code... Besides how is an
HTTP server supposed to make good HTTP caching as it has no knowledge of
the application it serves? It will be a very basic caching and won't
really match the needs of the WSGI app.
Besides, even if you do caching as a middleware, you may decide that for
some parts of the application that middleware has to be the first one to
be reached by the request, but some other time you may decide it not to
be the first one. I don't find middleware that flexible in that case.
3. The "Buffering and Streaming" section of the PEP is also confusing to
me has it says that application and middleware should never wait for
content to be produced and returned ASAP, if no content is available
then they must return an empty string which the server will serve
immediatly. However if the HTTP server uses Transfer-Encoding: Chunked,
it will have to serve an empty string with a Content-Length of 0 which
is a flag indicating the end of the chunked data and would confuse the
UA. That also means that the server will be limited in it chunked data
length to whatever the application/middlewares can provide when it
should not be that way as the server is the only responsible for such
choices. My understanding of that chapter might be entirely wrong but
even after reading it many times, I find it confusing to the least.
These were just a few points that poped into my head while thinking of
WSGI because we have met such issues with CherryPy design as well and
which are not easily solved.
You say that CherryPy is becoming obsolete in regards of the power of
WSGI. In many ways you are right and I am not surprised WSGI has gained
such support among the community. However does it mean we have to stop
being different? Are we doing this to rule the world of Python Web
Framework? Maybe the real challenge for CP today is to follow the path
it has taken so far and still be enjoyable as a self-contained, pythonic
HTTP toolkit. CP has lots of changes to face of course but it does not
mean WSGI is the unique path to follow.
Again this might sound as someone who is against WSGI when I am really
not. However, I acknowledge the fact things have worked so far without
WSGI and that there are many people who don't care at all about being
able to plug other pieces of software to their application. In that case
they can choose between a WSGI aware framework or not. As long as they
get productive with any I don't see anything wrong, do you?
If every framework out there goes WSGI, there will be a dozen ones doing
the same work the same way which will lead of course to a point where
there will be the need for only one WSGI framework and no more. Why
would someone would write a library such as Paste again when it is
already done? What would someone write a new web framework again when
Pylons already exists? Maybe the fact the Python Web Dev has evolved so
much in last couple of years is actually because people have done things
differently so that users could simply choose what suit them best.
Django is not better than TG but you would not make a DJango user at
ease with TG design and vice versa. WSGI is technically a nice interface
but forgets about the people using it IMO. For instance I am a big fan
of Amara, the Python library for XML made by Uche, but I'm not fan of
ElementTree. I don't say the first one is better per se but simply that
the way it is design is more natural to me. Should I drop Amara simply
because ElementTree becomes part of the Python stdlib? I don't think so.
Anyway, many people have had a go at the CP dev in the last few months
because we haven't embraced WSGI as of yet and we have not really always
been very clear in our message (uche reminded me that point recently
;)). Ego and communication issues I guess :(
This is normal that users would like to keep CP while being also
attracted to WSGI; as a CP dev there are many little ideas I appreciate
in WSGI. I believe that even though our current implementation looks
like a hack it is still feasible to use WSGI in CP as Christian has
shown. However, I also believe some developers would like their little
pet being used in a wider audience such as the one brought by TurboGears
and WSGI is the key to that.
CP 3.0 is about to start and many things should change and it is quite
clear that WSGI will be much more present in our heads from the start so
it is likely to have a better support for it in CP 3. Nonetheless I
think it is also important to say that we also want to make CP works as
it has always worked without WSGI. Then we would make everyone happy ;)
- Sylvain
Ben Bangert
> WSGI is a very nice interface that has a great potential and it is true
> that today it is impossible to avoid it. That being said WSGI also has
> unanswered questions because it has not really been used in large
> production or pushed to its limits AFAIK.
Several rather large production applications have been deployed using
WSGI, its such a minimal interface that there's generally little
overhead.
> Well WSGI has a couple of issues on this one that I would be happy if
> anyone could clarify.
Sure, I'll see if I can answer some of these.
> 1. Imagine I write a self-contained WSGI application using a couple of
> middlewares I made on my own. Say for example a gzip filter and one that
> changes the Content-Type based on content negotiation with the UA (eg.
> returning application/xhtml+xml when supported). Now imagine I release
> my application as a bundle so that users could simply drop it on their
> server (this is basically what many people have marketed about until
> now). But wait, due to the design of their server, they might already
> have several middlewares running just on top of their servers because
> they want to ensure that they only serve text/html or they do a
> different kind of compression by default which is not gzip. Well you
> will tell me they can simply configure my application so that they get
> rid of my middlewares right. True indeed, but then whose to say my
> application will work the same? Can you ensure that? Well you could come
> back to me and check it out. Everything is possible right... I'm being
> stupid here I know but the point is the whole
> WSGI-plug-it-in-as-you-wish will lead to issues that might be minors but
> could also be more annoying and I find a bit strange no one raises those
> potential issues. Pluggability is never that transparent.
Indeed. First, this is one of the reasons that the PEP actually says
middleware should never alter the transfer-encoding. While I've seen
several gzip middleware's out there, the WSGI PEP actually outlaws
these (the end of the content-length section):
http://www.python.org/dev/peps/pep-0333/#handling-the-content-length-header
Pluggability of applications is not guaranteed by WSGI, but its much
easier to have pluggable WSGI applications as a result. Plugabbility of
WSGI middleware is more of the plug-it-in-as-you-wish because
middleware by its nature is intended to be mostly ambivilant of what
happens up or down the middleware stack.
There's always ways for an application to be screwed up by the person
using it, or ways for someone to make an application in an inflexible
manner. WSGI doesn't change that. Well designed applications using WSGI
will be easier to plug-in without worrying much about other middleware
causing problems.
When plugging in a WSGI app, you will always need to take a look at
what the app does, and how, to ensure that your middleware doesn't
screw it up. Middleware should not be blindly transforming the
content-type of all data returned into xhtml, there's a dozen different
problems with middleware like that.
> 2. Another point that worries me regarding PEP 333 is that it leaves
> many decisions unreplied. Things such as:
>
> """Applying these principles to a variety of HTTP features, it should be
> clear that a server *may* handle cache validation via the If-None-Match
> and If-Modified-Since request headers and the Last-Modified and ETag
> response headers. However, it is not required to do this, and the
> application *should* perform its own cache validation if it wants to
> support that feature, since the server/gateway is not required to do
> such validation."""
>
> Well there is something annoying here as it means an application
> developer which needs HTTP caching on will need to add it by default to
> its application as it won't be sure the underlying HTTP WSGI server will
> support it. So much for less duplication of code... Besides how is an
> HTTP server supposed to make good HTTP caching as it has no knowledge of
> the application it serves? It will be a very basic caching and won't
> really match the needs of the WSGI app.
I don't see what the problem is here, it reads rather clearly to me.
The WSGI server doesn't need to 'support' those features since you can
get the HTTP headers yourself in your WSGI app, and send out your own
headers.
What that section is saying is that your application should be doing
the caching checks, not the server, and that you shouldn't rely on the
server/gateway to be doing caching checks. So if you want to use those
HTTP caching features, do it in your WSGI application, don't rely on
the server/gateway doing it.
> Besides, even if you do caching as a middleware, you may decide that for
> some parts of the application that middleware has to be the first one to
> be reached by the request, but some other time you may decide it not to
> be the first one. I don't find middleware that flexible in that case.
You probably wouldn't want middleware to do the HTTP caching unless
there was some way your app could inform the middleware of when it
should let your app respond vs when it should reply with a 304 message.
> 3. The "Buffering and Streaming" section of the PEP is also confusing to
> me has it says that application and middleware should never wait for
> content to be produced and returned ASAP, if no content is available
> then they must return an empty string which the server will serve
> immediatly. However if the HTTP server uses Transfer-Encoding: Chunked,
> it will have to serve an empty string with a Content-Length of 0 which
> is a flag indicating the end of the chunked data and would confuse the
> UA. That also means that the server will be limited in it chunked data
> length to whatever the application/middlewares can provide when it
> should not be that way as the server is the only responsible for such
> choices. My understanding of that chapter might be entirely wrong but
> even after reading it many times, I find it confusing to the least.
I unfortunately don't know enough about this scenario to comment much
on it. If you asked on the Web-SIG list, I'm sure someone who's worked
on this case would be able to supply a more useful reply.
> You say that CherryPy is becoming obsolete in regards of the power of
> WSGI. In many ways you are right and I am not surprised WSGI has gained
> such support among the community. However does it mean we have to stop
> being different? Are we doing this to rule the world of Python Web
> Framework? Maybe the real challenge for CP today is to follow the path
> it has taken so far and still be enjoyable as a self-contained, pythonic
> HTTP toolkit. CP has lots of changes to face of course but it does not
> mean WSGI is the unique path to follow.
No, I don't think it does mean that. However, since there are
super-flexible frameworks built with WSGI from scratch, it means that
its significantly less likely someone desiring flexibility and a simple
WSGI environment would want to use CherryPy. Mainly because CherryPy is
embracing WSGI, while the other frameworks were built with it.
At that point, rather than emphasizing that CP plays well with WSGI, it
will be more important to emphasize what CP is handling that the other
WSGI-based framework is not.
> Again this might sound as someone who is against WSGI when I am really
> not. However, I acknowledge the fact things have worked so far without
> WSGI and that there are many people who don't care at all about being
> able to plug other pieces of software to their application. In that case
> they can choose between a WSGI aware framework or not. As long as they
> get productive with any I don't see anything wrong, do you?
Not at all. The fact that more and more powerful WSGI components are
coming out would to me indicate that its more likely they'll be more
productive using a WSGI-based framework. This isn't the case now for
everything, there is no WSGI-based framework that can toss up an Admin
interface quite like Django for example.
Again, noting the difference between CP and wsgi-based stacks becomes
an important differentiator.
> If every framework out there goes WSGI, there will be a dozen ones doing
> the same work the same way which will lead of course to a point where
> there will be the need for only one WSGI framework and no more.
I doubt it. If someone wants the same exact way to do something as a
framework provides, they'll use that framework.
> Why
> would someone would write a library such as Paste again when it is
> already done?
They probably wouldn't. They'd use Paste WSGI components instead of
writing their own or if they wrote a better version those of us using
the Paste component could easily switch to the better one.
> What would someone write a new web framework again when
> Pylons already exists?
If they wanted something that operated in a very different manner,
they'd either tweak Pylons to provide that, or write their own stack
that works exactly how they like. If they write their own, its still
quite likely however that they would use similar components to build it
so that they can save time.
> Maybe the fact the Python Web Dev has evolved so
> much in last couple of years is actually because people have done things
> differently so that users could simply choose what suit them best.
> Django is not better than TG but you would not make a DJango user at
> ease with TG design and vice versa. WSGI is technically a nice interface
> but forgets about the people using it IMO. For instance I am a big fan
> of Amara, the Python library for XML made by Uche, but I'm not fan of
> ElementTree. I don't say the first one is better per se but simply that
> the way it is design is more natural to me. Should I drop Amara simply
> because ElementTree becomes part of the Python stdlib? I don't think so.
I think there's a better way to explain where WSGI fits into this.
Consider the Python standard library. Imagine that when using the
Python standard library, you were forced to *only* use modules from it.
If you want to use other modules, you had to use other people
libraries, at which point you could only use modules they supplied.
This resulted in everyone rewriting the same code, because you couldn't
share individual modules. Until WSGI, this scenario is how web
development with Python was for the most part. WSGI lets you share
individual modules without being forced into the entire library.
You can use the best session middleware/module you like, or the best
request object API, etc. You can use the dispatcher that meets your
needs perfectly. This is where WSGI is truly useful and powerful, not
merely pluggability of WSGI applications, but re-usability of
individual components that make up traditional non-WSGI frameworks.
Because of WSGI, you can pick and choose each part you want and make
your own framework very easily. Just like when making any Python app,
you can import whatever modules you want from any package you want that
gets your job done. If you don't want to hunt down the best parts to
make up your stack, you can choose from pre-assembled ones. This is
what Pylons and RhubarbTart are. It's what TurboGears First-Class will
be.
As I mentioned before, its quite likely there'll be many of the same
exact components used between these three and more. Extremely useful
middleware that fits in with the design of each framework will likely
be in each one, as its so easy to add them.
> This is normal that users would like to keep CP while being also
> attracted to WSGI; as a CP dev there are many little ideas I appreciate
> in WSGI. I believe that even though our current implementation looks
> like a hack it is still feasible to use WSGI in CP as Christian has
> shown. However, I also believe some developers would like their little
> pet being used in a wider audience such as the one brought by TurboGears
> and WSGI is the key to that.
Yes, this is also a big appeal to making solid WSGI components. If I
make a really good session middleware, besides for having it used in
Pylons, it can easily be used in RhubarbTart or TurboGears FirstClass,
or dozens of other little frameworks people create for themselves.
Competition is fundamentally different because of WSGI. Pylons and
RhubarbTart don't compete, they use many of the same parts in a
different configuration. They can both easily survive and get upgrades
as long as the parts they use get upgraded, and since those parts have
a broader audience there's more feedback, bug fixes, updates, etc.
Competition is no longer at the 'framework' level, but its at the WSGI
component level. The best session middleware will be used by everyone,
the best caching system, etc. I have heard people say on many occasion
about Framework X, "Yes, I really like how it does this and that, but
unfortunately it can't do other-thing nearly as well as Framework Z."
With those parts using WSGI middleware or components made for re-use
between WSGI stacks, you can have exactly what you want.
There won't be a single WSGI framework, because given the various
choices at all the different levels, there's hundreds or thousands of
possible combinations. Instead there will likely be a few best of breed
session layers, or a few best of breed dispatchers (that handle the
styles that most people like).
> CP 3.0 is about to start and many things should change and it is quite
> clear that WSGI will be much more present in our heads from the start so
> it is likely to have a better support for it in CP 3. Nonetheless I
> think it is also important to say that we also want to make CP works as
> it has always worked without WSGI. Then we would make everyone happy ;)
Yes, the question is, what will using CP 3.0 give you, that your own
WSGI-based framework that does *exactly* what you want won't give you?
If you make a little WSGI based framework, can you use anything from CP
3.0?
Why not make the best parts of CP 3.0 into WSGI components, and have CP
3.0 be a default configuration of those parts? This way regardless of
whether people want the whole CP 3.0 setup, they can still use parts
that are excellent regardless of their framework. If CP 3.0 is going to
be thinking more about WSGI, it should also think about what it brings
to the WSGI world. If parts are usable without the whole, there's more
opportunity to lure people into the entire setup as well, while still
giving them flexibility to use as much or as little as they want.
Really, what'd help the most, is if the CP developers sat down, and
made a small framework from scratch with WSGI and some middleware. Ian
Bicking has a how-to on making a small framework which is a good
starting point. That way rather than thinking about the WSGI PEP, the
real-world uses and applications of the PEP would become obvious.
Cheers,
Ben
Sylvain Hellegouarch
> Several rather large production applications have been deployed using
> WSGI, its such a minimal interface that there's generally little
> overhead.
>
>
> Indeed. First, this is one of the reasons that the PEP actually says
> middleware should never alter the transfer-encoding. While I've seen
> several gzip middleware's out there, the WSGI PEP actually outlaws
> these (the end of the content-length section):
> http://www.python.org/dev/peps/pep-0333/#handling-the-content-length-header
>
> Pluggability of applications is not guaranteed by WSGI, but its much
> easier to have pluggable WSGI applications as a result. Plugabbility of
> WSGI middleware is more of the plug-it-in-as-you-wish because
> middleware by its nature is intended to be mostly ambivilant of what
> happens up or down the middleware stack.
>
> There's always ways for an application to be screwed up by the person
> using it, or ways for someone to make an application in an inflexible
> manner. WSGI doesn't change that. Well designed applications using WSGI
> will be easier to plug-in without worrying much about other middleware
> causing problems.
>
> When plugging in a WSGI app, you will always need to take a look at
> what the app does, and how, to ensure that your middleware doesn't
> screw it up. Middleware should not be blindly transforming the
> content-type of all data returned into xhtml, there's a dozen different
> problems with middleware like that.
>
about the somewhat constant marketing around WSGI when we all know that
practice often show that things won't go as smoothly as we want them to.
WSGI is a very nice solution for recurring problems, well it's not a
solution but it's less worse than many other systems. That being said, I
want to focus on the fact there is also people who appreciate the,
almost atomic, design of CP where things are self-contained. Not
everyone looks after plugins. I'm not saying either side is bad simply
that the concept of middleware might not be as transparent and simple
for some users and CP design is sometimes more appealing to those users.
A matter of taste and requirements :)
>
> I don't see what the problem is here, it reads rather clearly to me.
> The WSGI server doesn't need to 'support' those features since you can
> get the HTTP headers yourself in your WSGI app, and send out your own
> headers.
>
> What that section is saying is that your application should be doing
> the caching checks, not the server, and that you shouldn't rely on the
> server/gateway to be doing caching checks. So if you want to use those
> HTTP caching features, do it in your WSGI application, don't rely on
> the server/gateway doing it.
>
Sure but it also says the server may decide to do it, on what grounds?
It does not say. Anyway this is not really a showstopper in our
discussion. Let's not get stopped by this one.
>
> I unfortunately don't know enough about this scenario to comment much
> on it. If you asked on the Web-SIG list, I'm sure someone who's worked
> on this case would be able to supply a more useful reply.
>
Might do yes.
>
> No, I don't think it does mean that. However, since there are
> super-flexible frameworks built with WSGI from scratch, it means that
> its significantly less likely someone desiring flexibility and a simple
> WSGI environment would want to use CherryPy. Mainly because CherryPy is
> embracing WSGI, while the other frameworks were built with it.
>
Very good point and again it's a matter of taste and requirements. Both
can totally co-exist.
>
>> Again this might sound as someone who is against WSGI when I am really
>> not. However, I acknowledge the fact things have worked so far without
>> WSGI and that there are many people who don't care at all about being
>> able to plug other pieces of software to their application. In that case
>> they can choose between a WSGI aware framework or not. As long as they
>> get productive with any I don't see anything wrong, do you?
>>
>
> Not at all. The fact that more and more powerful WSGI components are
> coming out would to me indicate that its more likely they'll be more
> productive using a WSGI-based framework. This isn't the case now for
> everything, there is no WSGI-based framework that can toss up an Admin
> interface quite like Django for example.
>
Well I agree but this is because WSGI has really been implemented for
less than a year. Many frameworks and libraries will pop out and then we
will see the community wondeing which one is the best and eventuall only
a couple will survive. Better to extend Paste than rewriting it.
>
>
> I doubt it. If someone wants the same exact way to do something as a
> framework provides, they'll use that framework.
>
Well I disagree here because you forget the human factor. Hackers have
their ways for reinventing the wheels all the time or simply because
they don't like the way a piece of software has been designed. I mean
many people could have said Django was good enough when it poped out and
yet TG hit the road along web.py, Colubrid, RhubartTart and you released
Pylons. Why would it be different with WSGI? People like doing things
their own way.
> I think there's a better way to explain where WSGI fits into this.
> Consider the Python standard library. Imagine that when using the
> Python standard library, you were forced to *only* use modules from it.
> If you want to use other modules, you had to use other people
> libraries, at which point you could only use modules they supplied.
>
> This resulted in everyone rewriting the same code, because you couldn't
> share individual modules. Until WSGI, this scenario is how web
> development with Python was for the most part. WSGI lets you share
> individual modules without being forced into the entire library.
> You can use the best session middleware/module you like, or the best
> request object API, etc. You can use the dispatcher that meets your
> needs perfectly. This is where WSGI is truly useful and powerful, not
> merely pluggability of WSGI applications, but re-usability of
> individual components that make up traditional non-WSGI frameworks.
>
Well I think I am more interested in a weblib part of the Python stdlib
totally independant from any higher interface or toolkits a bit like
Routes is in a way.
Then if peple want to build WSGI atop of that library then greta, if
they prefer filters great as well. Robert laid out the idea and it seems
just more Pythonic and more effective on the long run because it let
people work the way they want.
> Because of WSGI, you can pick and choose each part you want and make
> your own framework very easily. Just like when making any Python app,
> you can import whatever modules you want from any package you want that
> gets your job done. If you don't want to hunt down the best parts to
> make up your stack, you can choose from pre-assembled ones. This is
> what Pylons and RhubarbTart are. It's what TurboGears First-Class will
> be.
>
> As I mentioned before, its quite likely there'll be many of the same
> exact components used between these three and more. Extremely useful
> middleware that fits in with the design of each framework will likely
> be in each one, as its so easy to add them.
>
And again I do see the benefit of WSGI.
>
>
> Yes, this is also a big appeal to making solid WSGI components. If I
> make a really good session middleware, besides for having it used in
> Pylons, it can easily be used in RhubarbTart or TurboGears FirstClass,
> or dozens of other little frameworks people create for themselves.
>
> Competition is fundamentally different because of WSGI. Pylons and
> RhubarbTart don't compete, they use many of the same parts in a
> different configuration. They can both easily survive and get upgrades
> as long as the parts they use get upgraded, and since those parts have
> a broader audience there's more feedback, bug fixes, updates, etc.
>
> Competition is no longer at the 'framework' level, but its at the WSGI
> component level. The best session middleware will be used by everyone,
> the best caching system, etc. I have heard people say on many occasion
> about Framework X, "Yes, I really like how it does this and that, but
> unfortunately it can't do other-thing nearly as well as Framework Z."
> With those parts using WSGI middleware or components made for re-use
> between WSGI stacks, you can have exactly what you want.
>
> There won't be a single WSGI framework, because given the various
> choices at all the different levels, there's hundreds or thousands of
> possible combinations. Instead there will likely be a few best of breed
> session layers, or a few best of breed dispatchers (that handle the
> styles that most people like).
>
Well I assume there will be someone keeping start of the WSGI-art
framework with those best of breeds middleware modules leading to only
one. This will high quality and this is what matters I guess but I am
just careful at how the community will react.
For so long people have assumed Zope was the best web framework for
Python and the only allowing for enough high level quality code.
> Yes, the question is, what will using CP 3.0 give you, that your own
> WSGI-based framework that does *exactly* what you want won't give you?
> If you make a little WSGI based framework, can you use anything from CP
> 3.0?
>
> Why not make the best parts of CP 3.0 into WSGI components, and have CP
> 3.0 be a default configuration of those parts? This way regardless of
> whether people want the whole CP 3.0 setup, they can still use parts
> that are excellent regardless of their framework. If CP 3.0 is going to
> be thinking more about WSGI, it should also think about what it brings
> to the WSGI world. If parts are usable without the whole, there's more
> opportunity to lure people into the entire setup as well, while still
> giving them flexibility to use as much or as little as they want.
>
I do agree on that one and I assume this might be our lead.
> Really, what'd help the most, is if the CP developers sat down, and
> made a small framework from scratch with WSGI and some middleware. Ian
> Bicking has a how-to on making a small framework which is a good
> starting point. That way rather than thinking about the WSGI PEP, the
> real-world uses and applications of the PEP would become obvious.
>
We shall see :D
- Sylvain
Robert Brewer
Ben Bangert wrote:
> Indeed. First, this is one of the reasons that the PEP
> actually says middleware should never alter the
> transfer-encoding. While I've seen several gzip
> middleware's out there, the WSGI PEP actually outlaws
> these (the end of the content-length section):
> http://www.python.org/dev/peps/pep-0333/#handling-the-content-length-header
Not true. The PEP "outlaws" only transfer-codings (http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.5), not content-codings (http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6). The reason it does so has nothing to do with namespace or feature collisions--it's the fact that only the origin server deals with HTTP "hops", and the WSGI conversation is not an HTTP conversation. I haven't seen a single piece of middleware that sets transfer-codings. The ones that gzip as a content-coding are just fine and aren't "outlawed" at all.
Robert Brewer
System Architect
Amor Ministries
fuma...@amor.org
Robert Brewer
> If filters are dropped, and WSGI is embraced more openly
> in CherryPy, CherryPy itself is essentially destroying
> that which makes it *not* WSGI. In the end, what will be
> left is a light layer on top of WSGI that isn't as clean
> as something designed as such, since CP has a whole bunch
> of stuff in it thats essentially pre-WSGI legacy stuff.
Are you saying you heard filters are being dropped, or are you hoping they will? If it's the former, let me set your mind at ease and refute that. Filters will remain. Not everything can be done in middleware. If the latter, your argument boils down to "nice car, but if you remove the engine, brakes, and body, you've got a wagon just like everybody else, so what are you really adding?" *boggle*
I'm disappointed by your phrase "WSGI embraced more openly", both generally and specifically. In general, I'm saddened that people continue to use "WSGI" as a buzzword to support their architecture-du-jour. WSGI is an interface, not a component architecture. It facilitates a wide variety of architectures, and the constant (if recent) misapplication of the term "WSGI" to mean "you may only compose stacks MY way" is hurting WSGI's overall potential. There is room enough with WSGI for monarchies, republics, and communes. My money's not on the communes, but neither am I up-in-arms about other people's way of doing things. Be tolerant.
Specifically, I understand you mean "WSGI" in the sense of "more code in middleware, less in the framework", but in CP's case, what could you possibly do with a future stripped-down version of CP that you can't do today? You want to use WSGI middleware for sessions? Go for it. Want to use middleware for gzipping? Great! CP is not stopping you. CherryPy doesn't need to hemorrhage code before you can wrap it. Play nice.
CherryPy 3 will most likely abstract more of the basic, reusable logic into libraries where it belongs. We'll then decide which features work best out-of-the-box as filters, which work best as middleware, and which work best as bare library code, and provide wrappers for the first two. So CherryPy WILL be broken down a bit, just not in a "everything must be middleware" sort of way, and not in a panic. Have patience.
I'm going to be in Mexico for a week, so forgive me if I don't respond again right away.
Ben Bangert
> Are you saying you heard filters are being dropped, or are you hoping they will? If it's the former, let me set your mind at ease and refute that. Filters will remain. Not everything can be done in middleware. If the latter, your argument boils down to "nice car, but if you remove the engine, brakes, and body, you've got a wagon just like everybody else, so what are you really adding?" *boggle*
Neither. I'm not making arguments either, just observations.
> I'm disappointed by your phrase "WSGI embraced more openly", both generally and specifically. In general, I'm saddened that people continue to use "WSGI" as a buzzword to support their architecture-du-jour. WSGI is an interface, not a component architecture. It facilitates a wide variety of architectures, and the constant (if recent) misapplication of the term "WSGI" to mean "you may only compose stacks MY way" is hurting WSGI's overall potential. There is room enough with WSGI for monarchies, republics, and communes. My money's not on the communes, but neither am I up-in-arms about other people's way of doing things. Be tolerant.
And I'm saddened that people read the WSGI spec, look at some WSGI
code, but don't really *make* their own stack using WSGI components so
they can get a solid feel for what its changing. It's not just about
plugging apps into other apps, or having middleware. It goes beyond
that.
No one using WSGI would be silly enough to say "you may only compose
stacks MY way". Heck, the whole point of my last email was to try and
show that people are assembling a variety of stacks because its so easy
for *everyone* to assemble their own stack *their* way. This is why I
continue to see more and more interest in components that don't dictate
how your stack needs to be composed.
I should note though, that the CP approach so far is more of the
commune approach. That is, CP is happy to try and use WSGI to enhance
itself, but it gives nothing back outside of the CP commune. CP has
none of its own middleware that other people using other stacks can
plug-in. Notice that in all the talks about WSGI and CP, no one is ever
saying, "What great thing in CP can we put into WSGI middleware so that
everyone can use it in addition to us?". Instead its always, "How can
we use so-and-so to help us".
I'd like to think I'm exceptionally tolerant by putting what fits well
into WSGI middleware whenever possible, which means people can use the
code without any other choices I've made in Pylons.
> Specifically, I understand you mean "WSGI" in the sense of "more code in middleware, less in the framework", but in CP's case, what could you possibly do with a future stripped-down version of CP that you can't do today? You want to use WSGI middleware for sessions? Go for it. Want to use middleware for gzipping? Great! CP is not stopping you. CherryPy doesn't need to hemorrhage code before you can wrap it. Play nice.
Sure, but go farther and CP isn't really doing much that most people
need to use. Request object, response system, dispatcher, and some
filters... that's a very small amount of code and RhubarbTart's sparse
code-base shows that off. If I use session middleware, some caching
middleware, a object-path dispatcher and some handy request/response
object's, what do I need CP for?
I don't mean WSGI in the sense of more code in middleware. In some of
the stacks I've been using, I don't even know what you would call the
"framework", or if there is one, or even any point in having one. WSGI
components, and middleware, make the concept of a framework somewhat
obsolete entirely.
If you can put together the objects and dispatch you want in 50 lines
of code, that works *exactly* like you want, why would you want to use
some "framework" that you're highly unlikely to be able to understand
the intricacies of, that does excessively more than you want or need?
> CherryPy 3 will most likely abstract more of the basic, reusable logic into libraries where it belongs. We'll then decide which features work best out-of-the-box as filters, which work best as middleware, and which work best as bare library code, and provide wrappers for the first two. So CherryPy WILL be broken down a bit, just not in a "everything must be middleware" sort of way, and not in a panic. Have patience.
The general impression I get is that despite a lot of the talk, there's
a remarkably small amount of actual hands-on experience building
WSGI-based stacks amongst the Cherry-Py developers. This is usually
rather apparent in some of the odd list messages about WSGI that
reflect notions that just don't apply with real-world WSGI stacks.
I've only built 2 or 3 myself, the largest and most full featured being
Pylons, while I've helped out and worked on various WSGI middlware
thats in Paste, Beaker (session/cache middleware), and some in Pylons.
Building what used to be called a "framework" has become so amazingly
easy, that a lot of the hard stuff CP does, just doesn't apply anymore.
Rather than being adversarial and trying to fight against this, why not
whole-heartedly try it out and see how its changing things? Spend a few
hours and make a WSGI-based 'stack' that has the parts of CP you use,
see what components are in Paste that make it even easier, look at some
middleware that will help out, etc.
The point I think I'm trying to make, is that its significantly easier,
to work from a small WSGI code-base and add-on the features that make
it comparable to CP; rather than hacking away at a huge legacy CP
code-base to make it as agile as starting directly with WSGI
components. The result will also be cleaner, more flexible, and more
component-driven. If you want CP 3 to be a step into the future of
Python web development, a step like this might be necessary.
Cheers,
Ben
Mathieu Rouleau
Ben Bangert
here we are....
thakadu
discussed here will have an impact on the shape of CP 3 which in turn
will impact how I develop applications. I hope that this discussion
continues to be publicly accessible somewhere.
Christian Wyglendowski
Ben Bangert wrote:
[liberal snipping throughout]
> I should note though, that the CP approach so far is more of the
> commune approach. That is, CP is happy to try and use WSGI to enhance
> itself, but it gives nothing back outside of the CP commune. CP has
> none of its own middleware that other people using other stacks can
> plug-in. Notice that in all the talks about WSGI and CP, no one is ever
> saying, "What great thing in CP can we put into WSGI middleware so that
> everyone can use it in addition to us?". Instead its always, "How can
> we use so-and-so to help us".
That is partly a "consequence" of not being a bleeding-edge
no-user-base-to-worry-about non flavor-of-the-week framework. Since CP
has been around for a while and has a user base to worry about, we
haven't been able to simply turn on a dime and become everyones' WSGI
dream boat. Since Paste, Pylons and RhubarbTart are relatively new and
have been designed from the ground up with WSGI, they are obviously
going to be able to offer more in the WSGI department.
Regarding the commune idea, don't take not having our filter code as
middleware as not contributing to the Python web community. Earlier,
you mentioned the CP WSGI server which is not bound to CP and is
available for reuse. The webtest module is available for writing unit
tests that don't depend on CP. Obviously, Kevin found CP appealing
enough to include it in TurboGears and that community has done a great
job using projects from across the Python world to make a full featured
web app framework. And since CP applications *are* WSGI applications,
you get stuff like Robert's HTTPREPL, which gives you a nice ajaxy
Python interpreter in the browser.
I haven't been with the project from the beginning, but as far as I
know, CP has always tried to stay as open as possible and to allow the
developer to have as much freedom as they want. By providing a simple
framework that tries not to make choices for you, I think the project
has given a lot to the Python web community, and not just catered to a
commune.
> The general impression I get is that despite a lot of the talk, there's
> a remarkably small amount of actual hands-on experience building
> WSGI-based stacks amongst the Cherry-Py developers. This is usually
> rather apparent in some of the odd list messages about WSGI that
> reflect notions that just don't apply with real-world WSGI stacks.
Well, seeing that projects like Pylons and RhubarbTart are pretty fresh
out of the gates, I imagine that folks with lots of WSGI-based stacks
under their belts are hard to find. Regarding CP, some on the team are
more well versed than others. Robert has done a bunch for CP regarding
WSGI. I am a relative new-comer to CP and WSGI. Sylvain has been
getting into it lately.
So *if* being totally whiz-bang WSGI componentized is essential for
Python web frameworks (I said framework, ah!), then maybe we have some
catching up to do. But when it comes to creating a great framework (I
said it again!), the metaphorical question of the day is - is this a
sprint or a marathon?
> The point I think I'm trying to make, is that its significantly easier,
> to work from a small WSGI code-base and add-on the features that make
> it comparable to CP; rather than hacking away at a huge legacy CP
> code-base to make it as agile as starting directly with WSGI
> components. The result will also be cleaner, more flexible, and more
> component-driven. If you want CP 3 to be a step into the future of
> Python web development, a step like this might be necessary.
*sigh* CP isn't huge. At least it has never felt that way to me. As
far as the light-weight name drop, RhubarbTart, it's built on Paste,
right? Doesn't that have to be taken into account? I know, that could
digress into "well, CherryPy is built using the xzy module". My point
being, what RhubartTart does is simple and it is cool. But a lot of
other complexity likely lies beneath.
As Robert already mentioned, we are certainly going to componentize what
needs componentizing in CP3. And middleware - sure, probably. But I
guess since Pylons and RhubarbTart will rule the galaxy by then, what's
the point? ;-)
Long live Python, long live CP,
Christian
http://www.dowski.com
Robert Brewer
> I'm not making arguments either, just observations.
OK. I must have inserted my own "CherryPy needs to change" insinuation.
> And I'm saddened that people read the WSGI spec,
> look at some WSGI code, but don't really *make*
> their own stack using WSGI components so they
> can get a solid feel for what its changing.
> It's not just about plugging apps into other apps,
> or having middleware. It goes beyond that.
Really? The spec mentions "components" 19 times, and 17 of those are immediately preceded by the word "middleware". What do you mean by "WSGI component" if it's not a WSGI application, server, or both (=middleware)?
My point is not that WSGI is evil, or to be avoided, or even discouraged. As I said, there's no reason you can't use CP with any other WSGI "component" right now. Rather, it's a question of API design, and therefore program composition styles. None of the "components" in CherryPy are new or secret. The value that CP adds is its API design, including that part of the API that constrains how various "components" are discovered/declared, enabled, and configured.
Given the design choice of whether to provide a given feature as a library call:
import lib
def page(self):
lib.feature(params)
...as a filter:
[/page]
feature_filter.on = True
...or as middleware:
import lib
wrapped_app = lib.featureMiddleware(simple_app, global_conf)
...I tend to prefer one of the first two options. The library call allows per-call configuration right next to the call itself. The filter and middleware allow a feature to apply more broadly. I fully expect CP 3 will offer all such features in libraries. A "filters" module will wrap many of them so that they can be used as filters. A "wsgi" module will offer wrappers for many of them so they can be used as WSGI apps or middleware components.
> Rather than being adversarial and trying to fight
> against this, why not whole-heartedly try it out
> and see how its changing things? Spend a few hours
> and make a WSGI-based 'stack' that has the parts
> of CP you use, see what components are in Paste
> that make it even easier, look at some middleware
> that will help out, etc.
I'm sorry, but that's not going to happen. If you want CherryPy to change more than I've outlined above, you have the burden of proof in this area. I hardly have enough time to tell you I don't have that much time. I think _your_ time would be far better spent building something better than trying to convince us to do the work for you.
I have lots of questions about the rest of your post, but I'm not going to ask them because I wouldn't have the time to read your replies. I'm convinced CherryPy is on a good track for the future. If you disagree, then use something else. If that sounds short and snippy, it's because I really don't have time to drag this out. I've already spent 2 hours answering you (yes, I'm that careful in my replies) instead of doing the coding I had planned for today. That's a couple hundred dollars worth of my time, which I think you deserve because you're smart and have contributed good code to the community. But I see this conversation easily turning into days of my time, and I can't afford to sacrifice that for potential benefits even you have a hard time describing in any detail.
re...@cherrypy.org
> of WSGI driven web development. I know other people who are into WSGI
> have said similar things, though not out in the open. Hopefully I
> won't catch too much flak for what a bunch of people have known for
> awhile.
Thanks for taking the time to write to us and tell us that you think CP
is doomed.
Personally, I think that WSGI will never deliver the "revolution" that
"WSGI everywhere people" claim it will.
Let's look at the two main miracles that WSGI promises to deliver:
1- "we will have re-usable components that people can choose from and
use as building blocks (or "components") like legos; this will add more
flexibility and will lead to better quality of each component since
more people will use them".
Well, part of this might be true but I don't think the reality is quite
as perfect ... First of all, who are the "people" supposed to mix&match
these components ? If we're talking about framework developers then
fine. If we're talking about users (ie: website developers) then I
don't think they should have to care or know about these components.
Sure, there will always be a few people who want to choose the
components themselves and put them together themselves but most people
just want something that works and don't care about what's happening
under the hood. I don't think that shouting to the Java/PHP people
"come use our WSGI middlewares" is the best way to attract them ...
Secondly, breaking up things into components usually comes at a cost:
you can see the bolts and screws everywhere ... Even with a
"convention" like WSGI, the components don't always play as nicely as
some people claim they do. Sometimes you really want one component to
be able to access the internals of another component. Sometimes one
component really needs to know which other components it is being
combined with and how they work. Breaking up things into smaller pieces
only works up to a point ... At some point it's just not worth it
anymore ... What's next, breaking up each "WSGI middleware" into even
smaller pieces ?
Thirdly, while more people using some piece of code is usually a good
thing for the quality of that code, I wouldn't be surprised if WSGI
actually lead to *even more* implementations of the same things, giving
even more choices to people; the quality of these choices probably
won't always be very good, which will once again hurt users and python.
The second miracle that WSGI promises to deliver is:
2- "we will have plug&play applications that people can just take and
"mount" on their websites"
Again, part of this is probably true ... for trivial applications. But
for most real-life "interesting" applications I don't think it works as
nicely ... This is because when you want applications to work together
they have to be built to work together, *at the application level* (not
at the WSGI level) ... For instance, let's say you have a web forum, a
blog and a wiki application. Let's say they all allow users to log in
before using them. If you want to use all of them within your website
you probably want them to share the same user database and if a user is
logged in for the web forum you want them to be logged in for the blog
and the wiki as well ... Well, this means that the 3 applications have
to be designed to work together *at the application level* (ie: use the
same schema for the user table, use the same cookie for user data,
...). This has nothing to do with WSGI and the 3 applications could
actually be implemented in different languages (Python, Ruby and PHP
for instance) as long as they used the same user table and cookie !
So I think that if you want plug&play applications that work together
WSGI is neither enough nor necessary !
All I'm trying to say is that I don't buy this "WSGI is the new holy
grail of python web development" propaganda ... It is a nice interface
for what it's trying to achieve but I don't really believe in what it's
trying to achieve in the first place.
Yes, it will allow *some* component reuse and *some* application reuse
but it can only go so far.
It will work in some cases for some people but I think it'll will just
be *one* way of doing certain things and it will have its share of
problems and won't be compelling enough to replace other ways ...
CherryPy already plays nice with WSGI: if you want to use WSGI
middleware with CP you can; if you want to use CP WSGI apps from
another framework you can; if you want to use another WSGI app from
within CP you can; I think it's good enough and I don't want to break
CP into WSGI pieces and yes, filters are staying ...
Remi.
Sylvain Hellegouarch
> pardon the intrusion, but what does this have to do with cherrypy users?
>
>
choices are decided or not.
Sorry if this discussion sounds a bit too religious though :(
- Sylvain
Sylvain Hellegouarch
>
> And I'm saddened that people read the WSGI spec, look at some WSGI
> code, but don't really *make* their own stack using WSGI components so
> they can get a solid feel for what its changing. It's not just about
> plugging apps into other apps, or having middleware. It goes beyond
> that.
>
> No one using WSGI would be silly enough to say "you may only compose
> stacks MY way". Heck, the whole point of my last email was to try and
> show that people are assembling a variety of stacks because its so easy
> for *everyone* to assemble their own stack *their* way. This is why I
> continue to see more and more interest in components that don't dictate
> how your stack needs to be composed.
>
But WSGI does dictate it though due to its interface. As simple as it
is, you are still obliged to follow it.
> I should note though, that the CP approach so far is more of the
> commune approach. That is, CP is happy to try and use WSGI to enhance
> itself, but it gives nothing back outside of the CP commune. CP has
> none of its own middleware that other people using other stacks can
> plug-in. Notice that in all the talks about WSGI and CP, no one is ever
> saying, "What great thing in CP can we put into WSGI middleware so that
> everyone can use it in addition to us?". Instead its always, "How can
> we use so-and-so to help us".
>
> I'd like to think I'm exceptionally tolerant by putting what fits well
> into WSGI middleware whenever possible, which means people can use the
> code without any other choices I've made in Pylons.
>
>
That's very kind and as Christian said, CP is transparent as well and
all our code is available for everyone to grab and reuse. I agree it's
not packaged as a WSGI middleware but do not say we don't give it back
to the community.
>
> Sure, but go farther and CP isn't really doing much that most people
> need to use. Request object, response system, dispatcher, and some
> filters... that's a very small amount of code and RhubarbTart's sparse
> code-base shows that off. If I use session middleware, some caching
> middleware, a object-path dispatcher and some handy request/response
> object's, what do I need CP for?
>
This is very simplistic. Why would I need Pylons for? Why would I need
web.py for? etc. and yet people tend to choose different frameworks
because they fit their mind.
I'm being stupid again but your point is just too simplistic IMO.
>
> Rather than being adversarial and trying to fight against this, why not
> whole-heartedly try it out and see how its changing things? Spend a few
> hours and make a WSGI-based 'stack' that has the parts of CP you use,
> see what components are in Paste that make it even easier, look at some
> middleware that will help out, etc.
>
The fact we look defensive is because we have been repeating for a while
now that although we do see the value of WSGI, we will not make CP only
a WSGI stack. That being said we have long agreed to design future
version of CP well enough so that using it in a WSGI context makes it
more transparent for WSGI hardcore user. My problem is that it does not
sound enough for you and you push us to get defensive instead of
accepting our choices.
We want both worlds because we believe there is room for both ways of
doing web development in Python, with and without WSGI.
What more do you need?
We look bad ad I really don't appreciate that as we have proved our will
to improve the situation.
As Christian said, sure some CP devs are more keen to WSGI and some less
but in the end we have said we would design CP to make sure it plays
even nicer with WSGI. I believe it is a good decision.
Maybe your best options would be to follow the development of CP3 and
help us in the WSGI department to make sure it is well designed. That
would be much more productive for everyone.
- Sylvain
Uche Ogbuji
> Ben Bangert wrote:
>
> > I'm not making arguments either, just observations.
>
> OK. I must have inserted my own "CherryPy needs to change" insinuation.
Well, first of all thanks to Ben for his message, adversarial as it is.
It's touched off a very useful, and needed discussion. The main point
is that CP *does* need to change. It would need to change regardless
of WSGI, but WSGI is an important catalyst for needed change. I think
a lot of the needed change is exactly along the lines of Robert's
statement later on...
> Given the design choice of whether to provide a given feature as a library call:
>
> import lib
> def page(self):
> lib.feature(params)
>
> ...as a filter:
>
> [/page]
> feature_filter.on = True
>
> ...or as middleware:
>
> import lib
> wrapped_app = lib.featureMiddleware(simple_app, global_conf)
I'm not sure why your top option is not also middleware.
> ...I tend to prefer one of the first two options. The library call allows per-call configuration right next to the call itself. The filter and middleware allow a feature to apply more broadly. I fully expect CP 3 will offer all such features in libraries. A "filters" module will wrap many of them so that they can be used as filters. A "wsgi" module will offer wrappers for many of them so they can be used as WSGI apps or middleware components.
This improved modularization for CP 3 is essential. This is what needs
to change. CP is a bit too gnomic right now. The core engine does a
lot of cool things, but much off it is not readily exposed for the
developer. A good example is the long IRC thread I had recently about
re-using the path/object mapping. I I want to use deault() for one
path segment but then forward the processing back to CP for segments
farther to the right, I have no idea how to do so. The path traversal
algoritms ahould be expsed as modularized, dependency-free (as much as
possible), well-documented components. If not, CP *is* forcing the
user to do things its way.
Note that I never mentioned WSGI in the above para. CP's need for
componentization is independent of WSGI. However, I think WSGI is
*very* important. It represents the thoughts of *many* smart people on
how to compoentize Python Web apps, given experience with the many
frameworks that preceded it. It boggles my mind that anyone would
minimize this effort, as Remi does. I think Christian's and Robert's
attitide towards WSGI is understandable: some skepticism, but a
willingness to learn lessons and build bridges. I think Remi's
attidude on the other hand is really detrimental to CP. Remi attacks
straw man "miracles" of WSGI, but I've never seen anyone claim thet
WSGI will do anythign miraculous. It makes it look as if CP's lead
developer has his head stuck firmly in the sand. I worry a bit that
this might get in the way of the CP refactoring that is required in CP
3.0.
In short: WSGI is not a matter of miracles: it's a matter of concerted
thinking by the community of how Python Web frameworks can be carefully
componentized. As such, it provides some benefits but no panacea. I
think any observer understands that, and understanding that, would look
at some of the deficiencies of CP 2.2 and expect commitment to improved
modularization in CP 3.0. If some of the ideas and code along that
path comes from the WSGI crowd, I think most observers would feel that
the CP crowd would do well to gratefully accept it, and if there is a
perception of stonewalling instead, people *will* certainly begin to
drift away.
--
Uche Ogbuji
uc...@ogbuji.net http://uche.ogbuji.net http://copia.ogbuji.net
Work: uche....@fourthought.com http://Fourthought.com
Ian Bicking
> > Rather than being adversarial and trying to fight
> > against this, why not whole-heartedly try it out
> > and see how its changing things? Spend a few hours
> > and make a WSGI-based 'stack' that has the parts
> > of CP you use, see what components are in Paste
> > that make it even easier, look at some middleware
> > that will help out, etc.
>
> I'm sorry, but that's not going to happen. If you want CherryPy to change more than I've outlined above, you have the burden of proof in this area. I hardly have enough time to tell you I don't have that much time. I think _your_ time would be far better spent building something better than trying to convince us to do the work for you.
A concrete implementation exists: RhubarbTart (nobody is trying to get
you to do work we haven't done ourselves). Also, this document covers
the basics of how RhubarbTart is constructed, without actually using
Paste in any significant way:
http://pythonpaste.org/do-it-yourself-framework.html
Ben was only suggesting that, as the design for CP3 is considered, that
people try a little experimentation on their own, try using WSGI alone
and see how it feels. I also think this is a good suggestion. Rather
than debating the abstract benefits of different architectures, it is
much easier to understand the choices that are open to you by exploring
them -- clearly just talking about them isn't going to convince anyone
of anything. In the context of starting another major version in CP3,
trying this experiment does not seem like a lot of work.
Now, you might say, if we're so pleased with ourselves, why don't we
just use RhubarbTart and keep to ourselves? We'd really rather web
developers not have to make that choice, and if CP3 used a lighter more
WSGI-based approach that made RhubarbTart obsolete, that would be
ideal.
Ian
re...@cherrypy.org
> to change. CP is a bit too gnomic right now. The core engine does a
> lot of cool things, but much off it is not readily exposed for the
> developer. A good example is the long IRC thread I had recently about
> re-using the path/object mapping. I I want to use deault() for one
> path segment but then forward the processing back to CP for segments
> farther to the right, I have no idea how to do so. The path traversal
> algoritms ahould be expsed as modularized, dependency-free (as much as
> possible), well-documented components. If not, CP *is* forcing the
> user to do things its way.
I completely agree here. Some improved modularization is indeed needed.
It is already possible to change the dispatching algorithm and to call
the object mapping function on your own but it could be made easier and
it's not well documented.
Well, what I'm basically saying is "I don't really believe that WSGI
will help python web development. But CP is a team effort with lots of
users so we're doing our best to accomodate those of us who might want
to take advantage of what WSGI brings". I don't see anything wrong with
that position ...
What I don't like is people coming here and basically saying "rewrite
CP using WSGI everywhere, *or else* ..."
As for the "miracles" that they claim WSGI will deliver, I'm the one
who called them "miracles" because I don't think that WSGI will deliver
what they claim it will.
But as Ian said, there is no point in having some long/abstract
discussion over the benefits/drawbacks of WSGI.
Let's talk again in 6 months or a year and see if WSGI has helped us
come up with better tools for python web development and if these tools
have attracted more web developers to python ...
I doubt it and like I already said, I think that all this time and
energy would be better spent in helping some existing tools like
TurboGears, Django or CherryPy directly, because these are the tools
that will make everybody's life easier and attract more people to
python web development.
Remi.