A modest proposal for the future of TurboGears
Mark Ramm
and among other things, that got me looking at various options for
our future.
I decided we could:
* Build on top of pylons on top of bfg,
* Build on top of bfg directly
* Take the microframework (which had been being taken by pylons) and
move it back on as a core part of TG.
Jonathan Lacour did a great little experament at the last TG sprint on
where he investigated the last option:
http://github.com/cleverdevil/pecan
Chris Perkins showed me something he and Allice have been working on
which is pretty similar, but more mature:
These approaches help in some ways, they reduce or shift our
dependencies, and provide possible futures. But they would both be
TG3 type events. They'd be much smaller than the tg1 to tg2
transition, but there would still be a number of small
incompatibilities. I'm more than willing to do that kind of thing,
but am a bit hesitant about them if we're not absolutely sure that
this is the right thing to do for our users. So I was also been
seriously thinking about forking Pylons 1 to keep tg2 going long term
on that platform.
Pylons/bfg sprint
As I was thinking all that through, Ben Bangert contacted me to let me
know that there was going to be a bfg/pylons sprint, and I flew out at
the last minute to check it out.
There was a lot of discussion about how bfg and pylons work, a bunch
of discussion about how we can all work together better.
The point I emphasized a bit was that Django is doing a great job of
marketing, and the current situation is something like this:
#1 Python web Framework
---> Django
#2 Python web framework
---> TurboGears or Pylons or repoze.bfg or flask or...
Given the number of choices for the second position, people are
defaulting to Django because they don't know how to choose beyond
that.
All three frameworks have been talking about how to build higher level
abstractions (oauth support, admin tools, a widget system, etc) so
that we can better compete with Django feature for feature.
So, it was proposed by Ben, that we just all go ahead and figure out a
way to do what Rails and Merb did and stop building on top of one
another and find a way to actually merge our three communities.
Merger Talk
The merger discussion boiled down to a couple of questions:
Is there a common core that we can all agree on, and build around?
Are we willing to make the kinds of adjustments and compromises to
actually make this happen?
So, we did a lot of talking, a quite a bit of experimentation, and
more talking, and after a bunch of fits and starts, I think we came to
the conclusion that, if our communities were interested, yes we could
make this happen.
We decided that this experimental package would be created with
several core TG philosophies in mind:
We aim to create a full stack framework that gets you started quickly
We will take advantage of reusable components, and to use the best
tools available
And would use several common turbogears idioms by default:
* expose decorators
* returning a dictionary
* simple controller methods
We also got at least started on support for sqlalchemy, genshi,
turbojson/jsonify, and many other very common turbogears tools.
There's a new experamental package called Pyramid which is the result
of a lot of that talk and experimentation:
http://github.com/Pylons/pyramid
It has support for us to build in object dispatch, already supports
pylons style routes, and a third kind of dispatch which I'm going to
call site-tree traversal, all of which are backwards compatable with
our existing frameworks.
And better than that we created a "legacy" handler that allows you to
mount existing pylons or turbogers (2 for sure, and probably 1)
applications directly inside of a Pyramid app. People can then
migrate bit by bit as the feel the need.
My thoughts on what to do
----------------------------------------
I went in thinking a merger would require giving up too much.
But ultimately I found that the repoze.bfg and pylons folks recognize
the same need to build a strong framework for application code re-use,
and have begun looking at a more full-stack approach on their own.
And given all of our agreement about the best approach, I quickly
changed my mind.
Chris McDonough and Ben are very willing to work with us to achieve
the very goals that TurboGears was founded on. In a real way they
are being assimilated into Kevin's original vision for TurboGears --
just in a new package. If we join with them I think we can and should
take the lead in creating the next generation of high level tools that
will directly compete with the best that the django community has to
offer.
I left the sprint excited about the new blood that this would bring to
the TG developer community And I'm not sure how much we can afford
not to band together to build something awesome. It will require us
all to make some sacrifices, let go of some code that we love, and
embrace new approaches to some of the problems we've been facing.
But in the end, I think we will find that we're working with a much
larger group of talented python web experts to build a framework with
greater potential and greater reach than what we can do on our own.
And perhaps most importantly, it will be an opportunity for us to take
control of a much broader spectrum of our dependencies, so we don't
get left behind by abandoned projects, etc. And that control will
give us a much better ability to document, support, and prove the
value of our component based approach to web development, and in the
end, that's what I care about. Well, that and getting things done,
which I think will be a lot easier with Ben, Chris, and all the other
pylons and BFG developers on board.
No decision has yet been made, and I'm very interested in hearing what
you all think about all of this. Let the fireworks begin! ;)
--
Mark Ramm-Christensen
email: mark at compoundthinking dot com
blog: www.compoundthinking.com/blog
Antoine Pitrou
Hello Mark,
From a former TG2 user (not because I switched to anything else, but
because I don't do Web stuff anymore), the big annoyance has been the
lack of polish, which manifests itself in three areas:
- software quality
- documentation
- release management / packaging
The latter two being, in my humble (!) opinion, a disaster.
So whatever you choose to do should make it a priority to improve
massively on those points.
(Just my 3 cents)
Regards
Antoine.
Enrico
almost abandoned it till now. We all know the doco was bad but I think
there is another underlying problem. The pure MVC advantage is
undermined by having too many ways to skin a cat which makes doco
harder. Didn't someone once say, "there should be one and only one
obvious way to do it." Hopefully in the course of merging some
consesus will emerge on the best way to do a few things.
Christoph Zwerschke
> The pure MVC advantage is undermined by having too many ways to skin
> a cat which makes doco harder. Didn't someone once say, "there should
> be one and only one obvious way to do it." Hopefully in the course of
> merging some consesus will emerge on the best way to do a few
> things.
The problem is though, that the "it" in "one obvious way to do it" is
not well defined. We want to be able to do many things with TurboGears,
not only one thing. Many different kinds of applications, clients,
deployments etc. require different ways to do them. For some developers
or apps, NoSQL seems more appropriate, others want SQL. So yes, there
should be one recommended and well documented way to develop standard
web apps, but it should be easy to switch components. After all that's
one of the big advantages of TG over Django.
-- Christoph
Christoph Zwerschke
> From a former TG2 user (not because I switched to anything else, but
> because I don't do Web stuff anymore), the big annoyance has been
> the lack of polish, which manifests itself in three areas: - software
> quality - documentation - release management / packaging
>
> The latter two being, in my humble (!) opinion, a disaster. So
> whatever you choose to do should make it a priority to improve
> massively on those points.
Could not agree more. The main problem here being the lack of volunteers
who actively and continually engage as documentation writers, ticket
managers, release managers, server admins. And developers who can take
the time for writing and running tests, discussing options, doing peer
reviews etc.
In the last years, everybody expected these things to improve, but
nobody really stepped up and helped out, despite several appeals on the
mailing list. Please don't forget that this is an open source project
and it only works if people contribute.
As far as I see, our only chance to improve this is to join forces and
even then, it will only work if people not just hope that the current
crew will somehow handle it, but start to join and actively participate.
I also hope that the merge will force us to have more serious project
management and communication.
-- Christoph
Mark Ramm
> then, it will only work if people not just hope that the current crew will
> somehow handle it, but start to join and actively participate.
>
> I also hope that the merge will force us to have more serious project
> management and communication.
Should this happen, I'd expect that my role would become to focus
almost exclusively on
1) Project management communication
2) Marketing effort
We would gain folks like Ben Bangert and Paul Everest to help on the
marketing front, and Chris McDonough to help with managing the pyramid
codebase, so I think there will be significant movement forward on
those fronts.
--Mark Ramm
Mark Ramm
> lack of polish, which manifests itself in three areas:
> - software quality
> - documentation
> - release management / packaging
Pyramid already has:
* very strong test coverage (much better than tg2 did)
* several hundred pages of documentation, and we'd be planning to add
more for TG specific things like object dispatch.
* and we can work together with a larger group of people to solve the
packaging/release problem -- but I think that will be solved too.
So, we're aware of those issues, and my belief is that we will get
further with pyramid on each of those fronts than we would on our own.
--Mark Ramm
Diez B. Roggisch
Let me begin by saying that I deeply respect the efforts various
people have put into TG over the years. And successfully so in many
respects. I myself have so to varying degrees, and it was a good
experience.
Also I'm very in general very satisfied with the technical aspects of
TG2. I personally don't care about documentation (use the source,
luke), and thus have always been able to unleash it's powers.
But since the advent of TG2, TurboGears as whole has been declining in
community and overall adoption. And I believe this is due to 3 major
factors:
- the early announcment caused anxiety in the community. If you plan
on starting a new project, you don't want to invest into "old"
technology. But the new, shiny TG2 arrived later than expected.
- splitting the development force. Some of us were committed to TG1,
as they had large installations. Others jumped onto the TG2 bandwagon
- the overall tendency of developers of TG2 not to stick with
existing dependencies or approaches, but instead to go for the "next
big thing". See for example
KID -> genshi -> mako -> jinja, TGWidgets -> ToscaWidgets ->
ToscaWidgets2 -> Mark's Widgets (sorry, forgot the name), TG Admin ->
Sprockets -> SPROX -> TG Admin again (I'm unsure with the last list,
never cared much about these things)
As a result of this, I think we couldn't create enough of an eco-
system of 3rd-party contributions in terms of pluggable apps,
documentation writers, community and so forth.
Another "issue" is built-in, and partially reflected above already:
the best-of-breed-and-lots-of-dependencies approach has benefits - but
also downsides. Documentation is split, bugs span across frameworks
and packages, releases need careful testing and packaging, whole
packages depending on one single developer - who might lose interest
or can't contribute anymore - and so forth.
And now this new route. I understand it. I see chances for having a
larger development force behind it, and thus to mitigate quite a few
aforementioned problems.
But it will provoke the same anxiety, it will leave those of us behind
that have existing TG2 installations (let alone TG1). And please don't
pretend there will be a reasonable stable way to make the next big
thing backwards compatible. TG1 vs. TG2 has proven enough that this is
harder than one might think, and it's debatable if it is a worthy goal.
So, to wrap it up: I won't and can't speak out against this move - you
guys do, what you do. It's a technically sound decision. I will keep
an eye on what comes out of it. I might even move my existing projects
over to it (but then, there are 10 developers working with TG2 now -
that's a *lot* of brain to shift around)
But IMHO selling it under the turbo-gears brand is a mistake, or at
least misleading. Call it something fresh, something new, attract a
community - and try & stick with it, instead of moving faster than a
large group of people can.
From a technical POV, I strongly hope that you stick with WSGI as the
glue that binds things together. To me, it's a close-to-perfect
abstraction that cuts the various aspects of a web-app on clear, yet
still powerful interacting components.
Also, composable apps would be a great thing I guess.
And last but not least, keep or even ramp up the focus on testing, and
testability. TG2 is already much better in that respect from a user's
perspective, but the last sprint showed that testing the core is
harder than one might want it.
All in all - no hard feelings. But no excitement either - to much good
work is down the drains due to this IMHO.
Diez
Mark Ramm
> have existing TG2 installations (let alone TG1). And please don't pretend
> there will be a reasonable stable way to make the next big thing backwards
> compatible. TG1 vs. TG2 has proven enough that this is harder than one might
> think, and it's debatable if it is a worthy goal.
Well, the backwards compatibility story will be done completely
differently than the tg1 to tg2 approach.
The current code has support for putting an existing TG2 app *as is*
into the pyramid, and letting URL's be overridden by whatever you do
in the Pyramid app.
That way there's no need to change stuff that works, and if you want
to make use of the new stuff in pyramid you can do that at the same
time. And if there's value to you in the migration of controller
code to pyramid, then you can do that too.
Ultimately we plan to support TG1 to Pyramid migrations in exactly the
same way (via the legacy handler).
And obviously we will continue to provide bugfixes and support for TG2
and TG1 applications in the wild.
--Mark Ramm
P.S. Nothing is yet decided, this is the time to raise warning
flags, issues, and objections so that we can decide what we want to do
as a community.
Christoph Zwerschke
> Also I'm very in general very satisfied with the technical aspects of
> TG2. I personally don't care about documentation (use the source, luke),
> and thus have always been able to unleash it's powers.
Yes, but we need to care anyway since without proper docs you can't
attract new people and establish a larger community. Nor can you
convince coworkers or managers in your company to use TurboGears. In the
hand of Yedis TG is a great tool, but it has been just too hard for
Padawans to learn and master.
> And I believe this is due to 3 major factors:
> - the early announcment caused anxiety in the community. ...
> - splitting the development force. ...
> - the overall tendency of developers of TG2 not to stick with existing
> dependencies or approaches, but instead to go for the "next big thing".
I'll subscribe to that.
> Another "issue" is built-in, and partially reflected above already: the
> best-of-breed-and-lots-of-dependencies approach has benefits - but also
> downsides. Documentation is split, bugs span across frameworks and
> packages, releases need careful testing and packaging, whole packages
> depending on one single developer - who might lose interest or can't
> contribute anymore - and so forth.
That's true. We should learn to choose projects and libs we build upon
more carefully. They must not only be technically exciting, but also
well documented, mature, and have an active community. SQLAlchemy is a
good example.
> So, to wrap it up: I won't and can't speak out against this move - you
> guys do, what you do. It's a technically sound decision.
Most of all I think it's *organizationally* sound.
Of course, we should also discuss how we can make sure existing TG1 and
TG2 projects can be migrated to Pyramid or happily continue to run with
TG1 or TG2 for many years. I'm all for finishing TG 2.1 (properly, not
in a hurry) and providing bugfix releases in the future. I'll also
continue to provide bugfix releases for TG 1.1 and there will be a TG
1.5 alpha release soon for people who want to stick with CherryPy.
-- Christoph
Mark Ramm
>> guys do, what you do. It's a technically sound decision.
>
> Most of all I think it's *organizationally* sound.
>
> Of course, we should also discuss how we can make sure existing TG1 and TG2
> projects can be migrated to Pyramid or happily continue to run with TG1 or
> TG2 for many years. I'm all for finishing TG 2.1 (properly, not in a hurry)
> and providing bugfix releases in the future. I'll also continue to provide
> bugfix releases for TG 1.1 and there will be a TG 1.5 alpha release soon for
> people who want to stick with CherryPy.
So, I think we *absolutely must* make sure we continue to support the
current code that's out there both 2.x and 1.x. And I think we need
to make it super trivial to make your existing 1.x or 2.x application
a part of a pyramid project without any modification whatsoever, so
that migrations can be done in a controlled and sane way as real
requirements come in that make the migration work worth doing.
--Mark Ramm
Todd Blanchard
Mako/GeoAlchemy (because we are doing location based stuff) and after
the initial prototype, abandoned it for Rails 3 primarily because of
sketchy documentation, and not particularly helpful responses to our
questions on the list. I think there is real power in TG but some of
the frameworks are just too hard to get your head around (ToscaWidgets
- yes we wrote one - no we don't think it was worth it). That stupid
zope transaction manager thing needs to go as well. It causes lots of
problems and I don't think hardly anybody needs it. It should be an
opt-in.
I like python, love GeoAlchemy (will have to add its geo capabilities
to ActiveRecord in Ruby world) but it was just too hard to get off the
ground with any kind of critical momentum using TG2. We did try.
Just sharing our $.02
Mark Ramm
> zope transaction manager thing needs to go as well. It causes lots of
> problems and I don't think hardly anybody needs it. It should be an
> opt-in.
It will not be the default for pyramid apps, but will be available.
Mark Ramm
> well defined. We want to be able to do many things with TurboGears, not only
> one thing. Many different kinds of applications, clients, deployments etc.
> require different ways to do them. For some developers or apps, NoSQL seems
> more appropriate, others want SQL. So yes, there should be one recommended
> and well documented way to develop standard web apps, but it should be easy
> to switch components. After all that's one of the big advantages of TG over
> Django.
One thing which I would like to see is something like DataMapper in
ruby, where there's a standard way to map data returned by some
persistence layer into objects. We've been talking about this as SAL
your friendly Storage Abstraction Layer. If we can do a good job
with building some kind of SAL, you'll be able to plug sessions, and
other "framework" features into that abstraction layer, and still use
higher level tools even when you swap out lower level components.
This kind of dream was just impossible to imagine tackling on our own,
but seems like it is something we can do together, and that's what
excites me about the possibilities of pyramid.
Seth
that?
Mengu
i'm not using turbogears for real world applications but there were
some in-house applications that i have built with tg2. so, i'm not an
active turbogears user but i have sympathy for it and i really like
it.
i have picked pylons for my start-up instead of turbogears because
turbogears is built on pylons so i have thought why not use the parent
directly when they both do the same and considered the possibility of
encountering performance issues as it is layer over layer. the main
difference between pylons and turbogears is the widgets for
turbogears, like admin and toscawidgets.
the question here is, when developers and managers would pick
turbogears for their next web application?
- when it performs really good and scales
- when it is easy to start
- when it is extendable
- when it is still easy when things are a lot more complex than the
beginning
- when the app developed with it is re-usable
- when anything searched on the documents can be found
- when anything asked in the mailing list or the irc room is answered
now, lets take a look at those.
- turbogears performs really good and scales. how did i learn that?
because it is being used on sourceforge.net. from a slide of mark
ramm. not in the turbogears web site.
- turbogears is easy to start. how did i learn that? from the
quickstart.
- turbogears is extendable. that is primarily because its python. :)
- turbogears is still easy when more complex things needed to be done.
how did i learn that? because it doesn't bother me working with
turbogears when i build some kind of social network with it. (yes, i
have tried it.)
- turbogears apps are re-usable. still, python.
- turbogears documents are complex. they are not organized. many
things are out there in the docs but they don't even show up in the
search, let alone the document organization. hopefully this would
change with the sprint.
- any turbogears related question is replied in the mailing list or
irc. i had no questions left unanswered.
as you can see the only thing, as a developer, i did complain about is
the documentation. turbogears documentations exactly should be like
django's, as in the organization schema i mean.
those were "when would one choose turbogears" but we also have another
question. it is "why would one choose turbogears over django, pylons
or web2py". the answer of that question, i believe, is simple. it is
all about being better. having better options, having better tools,
having better support. so in order to be chosen and used in real world
applications, in my opinion, turbogears doesn't need to merge with
other frameworks or opt-in any other. unless of course, the new base
is better than the old.
thanks for reading. :)
Alice Bevan-McGregor
One of the reasons why I abandoned TurboGears was due to, at the time, an insufficient number of knowledgeable devs able or willing to assist with three specific problems I was having; one, a feature regression, which prevented me from using TurboGears for my big CMS project. Other issues had been niggling at me, and interestingly, documentation was only barely one of them.
I’ll summarize my issues, most, if not all of which have been resolved since I left nearly 2 years ago, and then describe WebCore’s philosophy:
1. Easy to get started, mind-blasting to debug. I had a problem in the dispatch mechanism of TurboGears, and it was literally easier to write my own from scratch and create a new framework in the process (~300 lines) than it was to try to figure out where in TurboGears my problem originated.
2. Dependancy graphs. TurboGears, as mentioned by several others, depends on some packages which don’t make sense for all applications. The transaction manager from zope (with the extra package for SQLAlchemy), specific repoze components for authentication/authorization which I have never used, etc. The last time I checked TurboGears 2.0 had 42 hard dependancies. Chris tells me this has been improved in 2.1 or 2.2.
3. Flat is better than nested. I never really liked the idea of TurboGears being based on another ‘megaframework’ like Pylons, which is in turn based on something else, etc. If a framework brings together separate components, they should be separate, not inherited from. That’s the wrong type of dependancy.
4. As voiced, some of the choices for ‘best of breed’ have been less than perfect. (Kid→Genshi→Mako→Jinja2, etc.)
5. Overriding defaults was devilishly difficult; changing the template from within the method, due to the passing of the template to @expose was hard. Changing the mime type required (at the time) the use of “CUSTOM/LEAVE” as the mime type in the @expose decorator.
When I wrote WebCore I wanted to write what, in my mind, was the “perfect web framework”:
1. One that does not demand much, if anything, of the client application. This means no hard dependancies on /anything/ that is not utterly required for operation. WebCore doesn’t install any database layer, templating engine, etc. as a direct dependancy. (WebError uses Tempitia though.) Core dependancies are: Paste, PasteDeploy, PasteScript, WebOb, WebError, and alacarte.
2. Any built-in feature should be optional (authentication/authorization) and minimal. WebCore’s authorization predicate system only declares Anonymous / Authenticated predicates and some helper classes so developers can easily write their own. It does not care what database or other storage mechanism you use, it uses callbacks into your own code. The login form (if used) is up to you. Etc.
3. Hello world performance is important. I was ridiculed for my hello world comparisons between frameworks set to their minimal configuration. However, hello world tests do provide useful information: this is as fast as the framework can go. Applications built on top of the framework can only get slower than this.
4. Smart dependancies and automatic feature negotiation. If you have Beaker installed in your virtual environment, Beaker becomes available for sessions and caching. Middleware that requires other middleware automatically negotiate for presence in the WSGI stack; e.g. if authentication requires a database session, authentication will be inserted ‘deeper’ into the WSGI stack automatically. I stole this idea from the uses/provides/needs initialization script system from Gentoo Linux.
5. Simple projects. When I first left TurboGears and saw WebPy, I loved the simplicity of getting started. A single file, no package, etc. WebCore follows WebPy’s example here. The core WSGI application is in WebCore itself, not generated from a number of files using callbacks into the framework in the client application. A freshly quickstarted TG application has numerous files in numerous folders, most of which you can not touch until you know exactly what you are doing without risking breaking your app badly. (A framework should not require effectively unalterable code be present in multiple client apps, it should be in the framework.)
6. Simplified, Pythonic code. Decorators are nice, but hard for new programmers to grok, especially when the decorator doesn’t decorate, it registers callbacks or data in a central registry. That’s weird. ;^) WebCore follows Python in automatically preventing access to underscore-preficed methods. Don’t want something exposed? Mark it “soft-private” by prefixing it. (An optional @template decorator is provided, however.)
7. Returning tuples is a great idea. (template_name, data, engine_options) or (template_name, data) as return values (without the brackets, even) looks nice and solves the dynamic template selection issue I had.
WebCore 2.0 will be Py3K compatible without automated code modification. It’ll straight-up install on any version of Python greater than or equal to 2.5. It’ll also support “my flavor” of WSGI2, a standard which doesn’t technically exist yet, while maintaining backwards compatibility. I already have a WSGI2 development HTTP server working. WC2 will switch away from WebOb, WebError, and Paste to marrow.* packages for the few features WebCore uses from these. (Web server, configuration files, simple command-line scripts, request/response, and error handling.) Each of these packages should be < 50KB and have few to no external dependancies, and all work on Py3K. They will all follow the same “simplicity, minimalism, and soft dependancies” style.
TurboGears needs to simplify, organize (esp. documentation), and pimp itself. In terms of legacy 2.0 code, forking Pylons and maintaining an in-house version may be required, but it’s hardly optimal. That’s a large code base currently maintained by a differing circle of developers. There may be overlap, but it’ll still make TG dev’s lives harder. :(
Chris Perkins has made some interesting packages from the TurboGears codebase; at last year’s ConFoo he extracted the object dispatch mechanism from TG into a package called Crank. This, I think, is a good direction to head in. Small, light-weight, task-specific, 100% unit tested, well documented dependancies would make working with, and more importantly /on/ TurboGears much nicer and easier.
My $2.15, (Tank Girl reference)
— Alice.
P.s. I don’t mean to troll or inflame anyone; I still actively use TurboGears on several ongoing support projects including one that is quite significant in size. The above is just my, perhaps outdated, personal thoughts. Standard disclaimers apply. ;)
Uwe Schroeder
Well said.
I have a major TG1 app running which I tried to convert to TG2 - and now I'm
happy I didn't.
There was a huge, huge issue with porting from TG1 to TG2. Some the fault of
TG2 being based on completely different principles, some the fault of
dependencies which sure did their share of abandoning code and moving forward
too.
For me the SLQAlchemy dependency and switch from 3.x to later versions was one
of the big killers (not a TG fault), the other one was the majorly different
handling of sessions, identities and authentication. If you have a code-base
of close to a million lines, you don't want to sift through every file and
replace an identity - the chance you miss one is too big and has disastrous
results if someone now can do things he shouldn't be able to.
So I'm still on a patched version of TG 1.0.3 simply because I wasn't able to
port it with a reasonable effort to a later 1.x version or even a 2.x version.
Porting to 2.x would basically have been a re-write for something that took me
12 months to write and test in the first place.
I know you can't do anything about the SQLAlchemy project releasing new,
backward incompatible versions (although that was a major point in me not
moving to TG2, because it would have required to rewrite and test every query
in the system. Personally I don't like ORM's and you might, rightfully so, say
"don't use one" - if it wasn't for hundreds of templates that use a
tablename.column syntax - which is very convenient.)
There is however the whole session and identity issue. If you run a website
that heavily depends on access rights and identities in general, you don't
want to change thousands of lines using a "py:if="tg.identity.anonymous" type
of syntax.
I have no grief about this because I've been in the "open source" business
even before linux was a word. I do however think that a backward compatible
layer on top of the new repoze.who and repoze.what packages would have made
transition much much easier. I abandoned the move from TG1 to TG2 exactly for
that reason - at the point when I tried (2.1) it was just not worth the effort
to either write a compatibility layer or change tons of code. Members crave
new features, not new technology providing the same features.
My perspective may be a bit scewed, because my livelihood depends on that
website. Knowing that users have no respect for new and improved "technology
behind the scenes" - which basically doesn't provide anything new for the
average user to toy with - I decided to rather continue using heavily patched
old versions of TG1.0.3 to achieve what I need on the frontend side.
This is a "user" perspective of things. I have no problems with tons of
dependencies etc.. Installation never was an issue for me. Backward
compatibility or at least a clean, documented, path of transition however is.
So I guess I'll see what pyramid brings along. In the end I'd love to use new
or better features that perform better. On the other hand I can't mess with
15.000 plus paying members in favor of a "technically better" technology the
members don't see. So I may or may not be stuck with old 1.0.3 for the next
years.
I know, this doesn't sound very community / open source - friendly. I'm too
constrained on staying afloat to put in big contributions in time and code, but
I'm happy to give a migration to a new TG type environment a shot and report
back on the issues. So count me in as a tester...
Uwe
Shane
Saw this post and felt that I should put in a word. I started
learning and working on TG2 almost a year ago. Coming from an
embedded development background, my last experience with Web
development were some C cgi's written for a class assignment about 15
years ago. I had chosen TG2 because I liked Python and figured that
TG, with the "best-of-breed" choices made for me had fewer possible
ways to go wrong than Pylons. I tried Rails, but went with TG because
I liked Python and SQLAlchemy.
The idea was to start development of what was then just a quick
prototype of a business Web application. Like many quick prototypes,
this one (somewhat unintentionally) expanded into into an alpha then a
beta project, which is now actually generating some revenue. Between
the alpha and beta, I completely ripped up everything I had done and
essentially started over, as the alpha had taught me much about how
NOT to write a TG app. I wrote my first widget, a wrapper for the
jqGrid library which, though it does work features some incredibly
ugly code that just barely gets the job done. As someone who very
much started from scratch (twice), I think I can second a few things
that have already been mentioned:
1) Documentation: Has come up, but let me emphasize the point a bit
more. I have actually gotten excellent help from the groups and the
quality of the quick start docs I think is rather good. The problem
for me is that the docs and examples don't go very far beyond the
basic of application. For example, the fairly simple matter of saving
form data to the SA object (uniqueness checking, URL mangling) gets
fairly complicated if you have a large number of fields and can't/
don't want to use something like Sprox (because I have so many jqGrids
for data entry). Making the leap from the quick start case to the
real-world case isn't well documented and many solutions found on the
groups provide multiple ways of doing the same thing, which brings us
to #2...
2) The issue of "One way" vs flexibility raised earlier in Christoph's
post I think could have a fairly simple solution that can be addressed
with documentation rather than code. Retain and add whatever
flexibility the expert developers need, but provide very in-depth
examples and cookbook best-approach methods that push the new
developer along a well trodden (and tested) path. When they get
better, they can then experiment more with alternative and more
complex solutions. For example, there must be about ten different
ways presented in the docs and newsgroups to fill a
SingleSelectionField. For something so common, there really should be
a single, obvious method.
3) I think a lot of you guys - and this is said with the utmost
respect - may be too friggin smart for your own good. Stop and think
for a second about the range of knowledge that you keep wrapped up in
your head. When I do an embedded project, I have a 200 page data
sheet for the chip, a copy of "The C Programming Language", some
liquor, and I'm ready to go. Since starting this project, I have
bought a pile of books at least three feet tall sitting next to my
desk - JavaScript for widgets, SQL, SQLAlchemy, Apache, Python, the TG
book, the Pylons guide, HTML/CSS layout plus online resources for
cherrypy, Genshi, etc and most of which have virtually no relation to
each other in function. The learning curve for moving into Web apps
is steeper than anything I think I have ever tried. Most of the
people I know that are looking for a Web development platform aren't
Web developers at all but capable people from outside fields (medicine
and microbiology seem popular right now) that have an idea and want to
develop a prototype before hiring a specialist.
In my opinion, TG2 is quite good technically, it "just" needs more
documentation which unfortunately can't be done in a weekend sprint.
Like Uwe, I have some pretty severe time constraints on assisting the
community as I try to get this project off the ground, but I think it
is certainly in my interest to try. I can help to write some
CookBooks, but my hesitation has always been a concern for food
poisoning -- that I may be giving a bad recipe.
- Shane
ca1...@bellsouth.net
Sent from my Verizon Wireless BlackBerry
You received this message because you are subscribed to the Google Groups "TurboGears" group.
To post to this group, send email to turbo...@googlegroups.com.
To unsubscribe from this group, send email to turbogears+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/turbogears?hl=en.
ca1...@bellsouth.net
Mark Ramm
saturday's IRC meeting turned out to be quite interesting and
productive as well. I'd love to take time and talk to every single
point, and will actually try to do that more this week, but I thought
it would be smart to give a quick higher level update on my current
activity and thoughts regarding the whole pyramid/merger thing.
I wasn't clear enough in saying that long term the merger would create
two packages as part of the same project, with a single coherent
strategy, and unified leadership.
One package (pyramid) will fill the base-microframework role, and one
(name needed) will be adding on a strong full stack approach with high
quality, documented defaults. Over time we'll be exploring
abstraction layers in the bottom level that will allow for greater
flexibility in the top layer. For example we've been talking about an
storage abstraction layer that sessions, caching, and other
"framework" pieces can use in place of choosing a specific technology
(rdbms tables, key value stores, document databases, LDAP, etc).
I'm personally at the point where I'll be spending quite a bit of my
open source free time on figuring out how that all is going to work
with Ben, Chris, and whoever wants to join us. But, as BDFL I'm not
yet making any announcements about the future of TurboGears -- other
than the fact that TG 2.1 will be released as planned, and that we
will continue to support applications built with it.
I'll also be working to create a structure to wrap up this discussion
in the next week or two, and make an official roadmap announcement.
--Mark Ramm
Alice Bevan-McGregor
There has been some great discussion in the last few days about perceived issues, future possibilities, re-inventing the wheel, community, and joining forces. Unfortunately what I’m seeing so far is a repetition of the 1.x → 2.x migration. A significant problem with “best of breed” is the criteria. Too often in the past I have seen things jumped on too hastily and three months later seen the regret.
> One package (pyramid) will fill the base-microframework role
30KLoC in 149 modules a microframework does not make. Especially when it re-implements template engine adaptors (alacarte), session handling (Beaker), configuration file parsing (PasteDeploy), thread locals (Paste), has a large amount of authentication/authorization code to adapt repoze.{who,what} (a package I recall people saying was a mistake to make required for TG2.0), object dispatch (crank), regex-based routing (Routes), static file serving (Paste), etc. The majority of these are modules contained within one package and thoroughly blended.
All of this needs testing and documentation, all of which would be more useful as reusable, and more importantly, optional components. Most are already available as reusable components. From the standpoint of a template engine developer I have to write adapters for two widget systems, at least one framework (probably more), and still have my own API to deal with. As a framework user, if I want to use a templating engine that’s new -I- have to write the adapters. The duplication of work is horrendous.
It’s one thing to decide that a Django-inspired framework design (everything and the kitchen sink available with no dependancies) is the way to go, in which case rigid structure and organization is necessary, another to want a microframework base, and a third to have a modular best-of-breed framework. Pyramid seems to me to be none of these, and has quite the intimidating codebase.
TurboGears 3, or the future project with no name yet, doesn’t need Pyramid from a technical standpoint. It didn’t need Pylons. WebCore showed that utilizing the same components as TurboGears (PasteScript, PasteDeploy, Genshi/Mako, ToscaWidgets, Beaker, SQLAlchemy, etc.) without a parent framework can be trivially easy, modular, light-weight, and fast. Relying on another ‘base’ framework seems to me to be a repeat of past mistakes, with more pitfalls than benefits.
I look forward to seeing what the future will bring for all of our projects.
— Alice.
Ben Bangert
> 30KLoC in 149 modules a microframework does not make.
more in what is accomplished (or what is not done) during a request.
Chris McDonough wrote a post covering this he called "What's it
doing?" referring to how much a framework does during a request:
http://plope.com/whatsitdoing2
The individual stats can be seen under each dir in results.txt here:
http://svn.repoze.org/whatsitdoing/
BFG has become pyramid, and the numbers haven't changed. As can be
seen from the numbers, the call-stack is much shorter than supposed
'micro-frameworks'. From my own run of it, WebCore comes in a little
better than Pylons, but not as good as Django. So there's small as in
LoC, and small as in, how many functions/calls actually run during a
request. There's also small as in scope of the framework itself.
> Especially when it re-implements template engine adaptors (alacarte)
who re-implemented who...
> , session handling (Beaker),
pyramid has a session plug-point, and there's already a pyramid_beaker
package to plug in Beaker sessions. there was no "re-implementation"
of beaker
http://github.com/Pylons/pyramid_beaker
> configuration file parsing (PasteDeploy),
PasteDeploy is still used for config file parsing, as is PasteScript
for creating a project template
> thread locals (Paste),
I believe that is used from Paste, though Paste had those because back
in the day we wanted threading.local and ppl still used Python
pre-2.4.
> has a large amount of authentication/authorization code to adapt repoze.{who,what} (a package I recall people saying was a mistake to make required for TG2.0),
not. It has 250 LoC in the auth modules combined.
> object dispatch (crank),
It has a different scheme called traversal, which functions in a
different manner to ensure there's a separation of URL resolution,
from the actual code that will get called. The hard binding that
object dispatch has between URL traversing and the method that is
actually called was part of the reason BFG folks sought to split that
up. By not having the URL bound so closely to the actual code
(controller method) that is called, it becomes much easier to extend
them from other packages allowing for a level of re-use that is
currently unattainable by this method.
> regex-based routing (Routes),
As the author of Routes, I can't really blame pyramid for not using
it. I spent a weekend doing some re-writing of Routes functionality,
based on the implementation from pyramid, and easily replicated almost
all of its functionality that I was considering for Routes 2 in about
160 lines of code... vs the thousand-plus in Routes. The
implementation in pyramid is about 120 lines of code or so, and uses
methods I wish I had done in Routes originally, this implementation
was more or less copied from bobo.
Would I have appreciated having some patches to Routes that cut down
on all that crufty code? Sure, absolutely. But at the same time, if
someone wants a quick Routes-like system and can write the
functionality they need in literally a hundred lines of code... vs the
behometh the Routes code-base is... its hard to blame them for not
putting in the rather huge amount of effort it would've been to
rewrite Routes to use that code.
> static file serving (Paste), etc.
> All of this needs testing and documentation,
the TG and Pylons docs pale in comparison...
http://docs.pylonshq.com/pyramid/dev/index.html
> all of which would be more useful as reusable, and more importantly, optional components.
Pylons, is that sometimes people want to introduce a feature into
Routes or Beaker, that adversely affects Pylons... and its hard to
clearly say why it can't be done... since they're supposed to be
separate projects.
> Most are already available as reusable components. From the standpoint of a template engine developer I have to write adapters for two widget systems, at least one framework (probably more), and still have my own API to deal with. As a framework user, if I want to use a templating engine that’s new -I- have to write the adapters. The duplication of work is horrendous.
adapter is easy, I'd be happy to kick one out for a missing template
engine. Right now there's ones for Jinja2, and Mako. And maybe a few
others floating about.
> It’s one thing to decide that a Django-inspired framework design (everything and the kitchen sink available with no dependancies) is the way to go, in which case rigid structure and organization is necessary, another to want a microframework base, and a third to have a modular best-of-breed framework. Pyramid seems to me to be none of these, and has quite the intimidating codebase.
picture of what its doing just by walking into it. I'm not sure how
much that hurts/helps, as oddly enough the worst code-bases I've had
(Routes) seems to get more ppl fixing it than Pylons does. I haven't
had too much of a problem going through it, as it is well documented,
but obviously I'm coming at it from the perspective of a framework
developer myself.
Cheers,
Ben
chrism
> Chris convinced me at last year’s ConFoo that I should re-join the TurboGears community, voice my opinions, and try to help out. Unfortunately I no longer feel comfortable camping here, as my role as direct competitor (aka “The Enemy™”) gives me a not-so-minor conflict of interest. Additionally, the pervasive belief in the “not invented here” mentality blinds people to the reasoning behind some of my decisions and ensures I get very little respect.
>
> There has been some great discussion in the last few days about perceived issues, future possibilities, re-inventing the wheel, community, and joining forces. Unfortunately what I’m seeing so far is a repetition of the 1.x → 2.x migration. A significant problem with “best of breed” is the criteria. Too often in the past I have seen things jumped on too hastily and three months later seen the regret.
>
> > One package (pyramid) will fill the base-microframework role
>
> 30KLoC in 149 modules a microframework does not make. Especially when it re-implements template engine adaptors (alacarte), session handling (Beaker), configuration file parsing (PasteDeploy), thread locals (Paste), has a large amount of authentication/authorization code to adapt repoze.{who,what} (a package I recall people saying was a mistake to make required for TG2.0), object dispatch (crank), regex-based routing (Routes), static file serving (Paste), etc. The majority of these are modules contained within one package and thoroughly blended.
I'm not writing this to convince anyone that TG should use Pyramid
(that's a decision best left to y'all), but I do have to respond to
the above because almost all of it is just not true. I had really
hoped to not have to do this, it's a huge time sink.
With respect to "30KLoC":
[chrism@thinko pyramid]$ pwd
/home/chrism/projects/pyramid
[chrism@thinko pyramid]$ cd pyramid
[chrism@thinko pyramid]$ sloccount .
... output snipped ...
SLOC Directory SLOC-by-Language (Sorted)
13830 tests python=13830
5096 top_dir python=5096
170 fixers python=170
156 paster_templates python=156
40 compat python=40
20 personality python=20
0 includes (none)
Totals grouped by language (dominant language first):
python: 19312 (100.00%)
Total Physical Source Lines of Code (SLOC) = 19,312
Development Effort Estimate, Person-Years (Person-Months) = 4.48
(53.74)
(Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Schedule Estimate, Years (Months) = 0.95
(11.36)
(Basic COCOMO model, Months = 2.5 * (person-months**0.38))
Estimated Average Number of Developers (Effort/Schedule) = 4.73
Total Estimated Cost to Develop = $
605,007
(average salary = $56,286/year, overhead = 2.40).
SLOCCount, Copyright (C) 2001-2004 David A. Wheeler
SLOCCount is Open Source Software/Free Software, licensed under the
GNU GPL.
SLOCCount comes with ABSOLUTELY NO WARRANTY, and you are welcome to
redistribute it under certain conditions as specified by the GNU GPL
license;
see the documentation for details.
Please credit this data as "generated using David A. Wheeler's
'SLOCCount'."
The output line to pay attention to is:
5096 top_dir python=5096
That's 5000 lines of code that has a chance to be executed at
runtime. The *rest* of the code is:
- Tests.
- Paster templates.
- Tutorial code in documentation.
Note another thing, please. There are *13KLoC* of tests. That's 2.6
lines of test code for each line of code that has a chance of running
when your application is running. I hope to not be punished for this.
See also http://docs.pylonshq.com/pyramid/dev/designdefense.html#pyramid-is-too-big
"Reimplements template engine adapters"... alacarte.. first time I've
ever heard of it. It appears to have been created in September of
2009. The template abstraction code in Pyramid has been around since
August of 2008. It's also reasonably well documented:
http://docs.pylonshq.com/pyramid/dev/narr/views.html#adding-a-new-renderer
"Reinvents thread locals"... have you ever *seen* the Paste
StackedObjectProxy code? This is what replaces it in Pyramid:
http://github.com/Pylons/pyramid/blob/master/pyramid/threadlocal.py
Not rocket science. Not that there *should* be any thread locals to
start with, but that's a different thing. Certainly, at least, the
idea that you don't *import* a thread local has merit.
Session handling... we dont replace Beaker, we allow people to *use*
it: http://github.com/Pylons/pyramid_beaker .. we also have a very
basic implementation that doesn't require Beaker, and we allow
alternate implementations:
http://docs.pylonshq.com/pyramid/dev/narr/sessions.html#creating-your-own-session-factory
Configuration file parsing.. gotta leave this one to
http://docs.pylonshq.com/pyramid/dev/designdefense.html#pyramid-uses-zcml-zcml-is-xml-and-i-don-t-like-xml
Large amount of code to adapt r.who/r.what.. no.. it doesn't require
r.who or r.what at all. See the red "Warning" box near the top of
http://docs.repoze.org/bfg/1.3/narr/security.html (which i actually
removed from the Pyramid docs because I couldn't imagine someone
seriously making this assertion anymore). Pyramid has its own
authorization policy and authentication policy plugpoints: see
http://docs.pylonshq.com/pyramid/dev/narr/security.html for an
overview.
Object dispatch.. Crank wasn't released until early 2010. Pyramid's
dispatch (written first in 2008) is based on Zope's, which has been
around since 1998. The traversal is pluggable, though. I've actually
even helped try to make TG-style dispatch work on top of BFG
(Pyramid's predecessor): http://svn.repoze.org/repoze.bfg.metatg/trunk/README.txt
Replacing Routes: Ben Bangert, the author of Routes, has signed on to
Pyramid's implementation. BFG actually *used* Routes for some time,
but didn't need most of its functionality. The routes code was very
large, and it was replaced by less than 4K of code in BFG (now
Pyramid).
Static file serving.. Pyramid actually *does* use the Paste static
file application.
> All of this needs testing and documentation, all of which would be more useful as reusable, and more importantly, optional components. Most are already available as reusable components. From the standpoint of a template engine developer I have to write adapters for two widget systems, at least one framework (probably more), and still have my own API to deal with. As a framework user, if I want to use a templating engine that’s new -I- have to write the adapters. The duplication of work is horrendous.
isn't perfect, but good lord, it's 600+ pages printed in PDF format.
*Nothing* is undocumented. Even the add-ons have similar docs. I'm
unclear as to what more could be done.
As far as templating system add ons:
- http://github.com/Pylons/pyramid_jinja2
(see also http://docs.pylonshq.com/pyramid_jinja2/dev/index.html)
That's an example of what it takes to add new templating system
bindings. Seems pretty straightforward to me. Pyramid supports Mako
and Chameleon ZPT/text templates out of the box, Jinja2 with the
extension above, and I plan to write at least one more for Genshi.
>
> It’s one thing to decide that a Django-inspired framework design (everything and the kitchen sink available with no dependancies) is the way to go, in which case rigid structure and organization is necessary, another to want a microframework base, and a third to have a modular best-of-breed framework. Pyramid seems to me to be none of these, and has quite the intimidating codebase.
>
> TurboGears 3, or the future project with no name yet, doesn’t need Pyramid from a technical standpoint. It didn’t need Pylons. WebCore showed that utilizing the same components as TurboGears (PasteScript, PasteDeploy, Genshi/Mako, ToscaWidgets, Beaker, SQLAlchemy, etc.) without a parent framework can be trivially easy, modular, light-weight, and fast. Relying on another ‘base’ framework seems to me to be a repeat of past mistakes, with more pitfalls than benefits.
was just required to defend Pyramid.
Mark Ramm
30KLoC in 149 modules a microframework does not make.
Especially when it re-implements template engine adaptors (alacarte), session handling (Beaker), configuration file parsing (PasteDeploy), thread locals (Paste), has a large amount of authentication/authorization code to adapt repoze.{who,what} (a package I recall people saying was a mistake to make required for TG2.0), object dispatch (crank), regex-based routing (Routes), static file serving (Paste), etc. The majority of these are modules contained within one package and thoroughly blended.
All of this needs testing and documentation, all of which would be more useful as reusable, and more importantly, optional components. Most are already available as reusable components. From the standpoint of a template engine developer I have to write adapters for two widget systems, at least one framework (probably more), and still have my own API to deal with. As a framework user, if I want to use a templating engine that’s new -I- have to write the adapters. The duplication of work is horrendous.
TurboGears 3, or the future project with no name yet, doesn’t need Pyramid from a technical standpoint. It didn’t need Pylons. WebCore showed that utilizing the same components as TurboGears (PasteScript, PasteDeploy, Genshi/Mako, ToscaWidgets, Beaker, SQLAlchemy, etc.) without a parent framework can be trivially easy, modular, light-weight, and fast. Relying on another ‘base’ framework seems to me to be a repeat of past mistakes, with more pitfalls than benefits.
Mark Ramm
>
> PS. I think the TG mission is clear, but the goal is not. I think getting a stable framework that can be distributed with ubuntu amd do it within a year would be a worthy goal.
I want to work something that is:
* Technically superior to TG2 and other web frameworks in significant ways
* Faster than TG2
* Better tested than TG2
* More extensible than TG2
* Has centralized easy to read, easy to navigate API docs
* Has better narative/tutorial documentaiton including docs specific to folks who already know other technologies
* More third party extensions written
* More patches, bugfixes, and other core development particupation
Christoph Zwerschke
> IMHO, what we NEED is to keep and maintain a critical mass of committed
> developers that will move the framework forward, and will help us to be
> competitive in the market.
>
> We've had lots of developers go on to create their own frameworks, or do
> other things, and IMHO that's good. It shows the options, and helps us
> see ourselves better, but it doesn't make TG more viable. What makes
> us more viable is a core team of people who work together to improve the
> framework, and are committed to the project and the team.
Right, that's a crucial point in the whole discussion. I heard people
say "I don't care if the framework is popular, it just needs to work for
me". Sounds reasonable, but I beg to differ, for various reaons. First,
it is very difficult to convince managers, coworkers or clients to use a
framework that nobody has ever heared about. Hiring people and getting
hired is also difficult. Second, a popular framework will likely not go
away so quickly. Third, and most importantly, a popular framework will
attract more users and developers who contribute in various ways, and
make it even better, not only in terms of features, but also stability
and documentation, or who blog about it and make it even more popular
(the "Matthew effect" kicks in), soon you may even have your own
conferences. I.e. once a certain popularity is reached, a
self-reinforcing loop is created. Another example for this
self-reinforment: Django is now so popular that Apatana (Pydev) comes
with special Django integration which makes it even more attractive
(http://pydev.org/manual_adv_django.html) for developers. People will
create a whole ecosystem around the framework, providing apps, plug-ins
etc. that you can reuse or build upon, again reinforcing the loop.
So a popular framework has many benefits. Of course we all know that the
most popular ("industry standard") technology is not always the best;
actually often the opposite is true. The badness and uglyness of the
technology can hurt so much that people like us may deliberately step
down and set the benefits of popularity aside, just to be able to use
the better technology. Nevertheless, becoming a popular and vivid
project and community is still highly desirable, not only for the sake
of it, but because of the additional benefits I pointed out.
So what we need is a project with a critical mass to get this kind of
chain reaction started that is able to contend with Django. That's the
thing we may be able to achieve by getting BFG, Pylons and TG under one
umbrella and joining forces.
-- Christoph
Martin Eliasson
While it's nice and important to have lot's of developers for the moment, the core teams long term commitment is most important. Example: a few years ago lots of pluggins where created for Eclipse, today, activity seems to be quite lower. As if the fun-to-do-pluggin crowd moved on to PHP, android or whatever.
Don't look to much at Django. Django is successfull at being Django. TG need to get really successfull at being TG and we will know when this has happened. It has happened when we have the one big killer web-app built using TG. If you ask me, TG is for web-applications rather than web-sites. Unfortunately, that makes Java-solutions the real hard competition, not Django or PHP.
Anyway, from a happy TG user to all developers.
Big Thanks.
--
You received this message because you are subscribed to the Google Groups "TurboGears" group.
To post to this group, send email to turbo...@googlegroups.com.
To unsubscribe from this group, send email to turbogears+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/turbogears?hl=en.
--
Martin Eliasson
+46 (0) 739 97 87 33
http://asplunden.org
Mark Ramm
It all has 100% unit tests with documentation that unfortunately makes
> All of this needs testing and documentation,
the TG and Pylons docs pale in comparison...
http://docs.pylonshq.com/pyramid/dev/index.html
AD.
caught my attention back in the 0.8 days. The truth is that I don't
get much opportunity to do much real life Python web development. I
still hope that changes in the future.
Recently I had been looking around at other frameworks to play around
with, and BFG was one that appealed (Flask was the other main option I
liked). Chris sounds like a smart guy and seems to be running the
project very professionally. I had also thought about plain old
Pylons, but until recently it wasn't clear what future it was going to
have (like TG). Chris and Ben teaming up sounds like a promising move.
A full stack framework without thorough docs and a large user
community is going to struggle to keep users defecting to either a
full stack framework that does have those things (eg Django), or to
micro-frameworks where the small limited codebase is simple enough
that full docs or advice from users isn't as important.
Personally while I think joining Pyramid or switching to building TG3
on top of Pyramid does risk another painful fragmenting change of
direction (although one that sounds like it can be better managed),
all the other alternatives sound like they will eventually run out of
steam.
In fact based on Chris and Bens comments and the lessons they've
learned from BFG and Pylons I'd be tempted to use Pyramid even if TG
didn't go in that direction.
--
Cheers
Anton
Derick Eisenhardt
the very first email and choose a direction right here.
>The point I emphasized a bit was that Django is doing a great job of
>marketing, and the current situation is something like this:
>#1 Python web Framework
> ---> Django
>#2 Python web framework
> ---> TurboGears or Pylons or repoze.bfg or flask or...
goal of redefining that logic as:
Python web frameworks:
#1 Django (let's just be realistic here)
#2 Turbogears (full-featured web app development stack)
#3 Pyramid, web-core, etc... (build it yourself + disparate
components)
Basically TG needs to move out of the pool of "other" and rise above
to the step new devs look at before they dig in to #3.
Personally what I'd like to see is a solid, stable stack...with pretty
much every component most users will need 99% of the time built-in and
fully integrated. As while it's good to offer choice of various
components (hence why I chose TG over Django), TG2 has a few areas
that are not very pythonic...there needs to be one obvious, preferred
way, and if you want to venture into another direction, that's up to
you...but TG's documentation will primarily focus on the "correct"
way.
One thing that has really bothered me ideologically in this area is
TG2's seeming lack of a preferred Javascript/Widget set (as opposed to
TG1's implied integration with MochiKit). Sure, there's Tosca, but
it's just so limited that it's really only good for simple web sites,
not true web apps, which as someone already said should be the real
focus of TG. I don't really care whether it's JQuery, Dojo, Pyjamas or
some other option...but this needs to be built in and standardized
upon. For the app I've been working on the past 2+ years I've had to
write everything myself to get Dojo and TG2 to work together, it could
really be much much more *automagical* in transferring data between
client and server and integrating with my templates.
What you really need to be successful is a singular set of tools, with
just about everything you should need to write a web app built in,
fully documented as if TG were a single project like Django as far as
the user is concerned. Abstract out all your dependencies too, there's
no reason I should be importing things directly from SQLAlchemy or
Genshi, I should be importing things like tg.database and tg.template,
even if they're only wrappers so that if and when those things are
changed I don't have to go back and update my code to the new
dependency. That also applies to documentation, putting "see docs on
Dependency X's web site" doesn't cut it. If you want to be the full-
featured mega framework, then I should rarely have to leave the
turbogears.org domain.
Whatever is decided, TG or whatever we call this in the future needs a
much more active community, with more developers, more users, and
probably some sort of corporate backing where at least a couple of our
core devs are getting paid full-time to work on TG if it's really
going to be sustainable and become the vibrant framework we all want
it to be. Personally I'm still leaning toward the 3 way merger of BFG/
Pyrmaid, Pylons and TG, with the 2 package concept Mark mentioned
(minimal-core vs full-stack).
John_Nowlan
I mainly lurk on tg, pylons, pyjamas and other python web framework lists. I originally chose pylons because of sqlalchemy and its ability to handle composite keys. We have had some success here promoting python (we mainly work with a big erp java oracle system) but the vendor has recently come out saying they are moving to groovy and zk. I'd much rather work with python but I see the python frameworks losing ground to java (with oracle being a wild card).
With Google releasing gwt designer there appears to be less reason (unfortunately) to use python. The java to javascript toolchain appears strong, with other players (wavemaker, titanium) also in this space. Perhaps without corporate backing it is unrealistic to expect python to be able to compete.
One exception to this is https://github.com/robertolupi/genropy, video http://blip.tv/file/3837476 but it still seems like early days there. Using dojo looks great, but not using sqlalchemy probably makes it a non-starter for us.
Pyramid seems to have purposefully left the 'client/browser' area untouched, concentrating on the server but to me a framwork these days has to address the client/browser side of the equation as well. It is where the bar is being raised to with respect to web frameworks. RIA development, while being a twisted path, seems to be converging towards the benefits that the web has brought (ease of deployment) with the benefits of client/server (RAD).
I'm curious to know what others think, if tg plans to address this piece in the pyramid puzzle.
> -----Original Message-----
> From: turbo...@googlegroups.com [mailto:turbo...@googlegroups.com]
> On Behalf Of Derick Eisenhardt
> Sent: Tuesday, November 09, 2010 12:59 PM
> To: TurboGears
> Subject: [TurboGears] Re: A modest proposal for the future of TurboGears
>
Christoph Zwerschke
> With Google releasing gwt designer there appears to be less reason
> (unfortunately) to use python. The java to javascript toolchain
> appears strong, with other players (wavemaker, titanium) also in this
> space. Perhaps without corporate backing it is unrealistic to expect
> python to be able to compete.
GWT sure makes it easier for Java developers to enter the Javascript
world, but then sooner or later they will discover they don't really
need Java any more. Server side Javascript becomes attractive again
thanks to node.js. As far as I see, "Javascript the good parts" is
actually what JavaScript has in common with Python, and "Javascript the
bad parts" is primarily the stuff that has been built in to make it look
more like Java. Building RIAs with Python and "Javascript the good
parts" looks very attractive to me since I have to do less mental
switching between languages.
But I fully agree with you that we need a strong alternative to GWT, we
should not only look at Django or Rails and building simple web forms,
but at the challenges that come with building the RIAs of the future.
Pyjamas can be one option, but there should be others e.g. for building
jQuery UI based applications.
In my opinion TurboGears 1 was trend-setting with its widget browser in
the toolbox and integration of MochiKit, but unfortunately TG did not
continue to develop into that direction. The widgets for me have been a
real added value on top of TG, but after the spin off into ToscaWidgets,
they did not get enough attention by their own inventor and other
developers. There is TW2 and EasyWidgets now, but they need to grow and
build a larger community continually building and maintaining useful
wrappers for the various Javascript GUI libaries, foremost jQuery.
Having a base widget library is not enough, we need a vivid marketplace
for useful, ready-made widgets, like the "cogbin" which unfortunately
zonked out, too.
Take for instance the super useful jqGrid, how would you use it with
TurboGears? The website http://www.trirand.net has components for
ASP.NET and PHP, we need exactly such components/warapper widgets for
Python. People need to write such wrappers on their own. Sometimes they
release them (http://pypi.python.org/pypi/tw.jqgrid), but usually only
simple alpha releases that are newer updated, and this one is for TW not
for TW2 or EasyWidgets. It cannot do the more sophisticated features
like exporting excel files like the PHP component does.
I really think TG 3 should consider re-integrating widgets and try
building an attractive "cogbin" again.
-- Christoph
Daniel Fetchinson
I completely agree with you on the importance of a well-defined bridge
between client and server within the tg framework, which mostly means
a well-defined way of sharing data between client (javascript) and
server (python) using tg itself, but there are a number of other
important issues. I wrote about exactly this point almost 3 years ago:
http://groups.google.com/group/turbogears-trunk/browse_thread/thread/c8492173ee8a473e
Cheers,
Daniel
--
Psss, psss, put it down! - http://www.cafepress.com/putitdown
Mark Ramm
What you really need to be successful is a singular set of tools, with
just about everything you should need to write a web app built in,
fully documented as if TG were a single project like Django as far as
the user is concerned. Abstract out all your dependencies too, there's
no reason I should be importing things directly from SQLAlchemy or
Genshi, I should be importing things like tg.database and tg.template,
even if they're only wrappers so that if and when those things are
changed I don't have to go back and update my code to the new
dependency. That also applies to documentation, putting "see docs on
Dependency X's web site" doesn't cut it. If you want to be the full-
featured mega framework, then I should rarely have to leave the
turbogears.org domain.
Personally I'm still leaning toward the 3 way merger of BFG/
Pyrmaid, Pylons and TG, with the 2 package concept Mark mentioned
(minimal-core vs full-stack).
Tim Black
Personally I'm still leaning toward the 3 way merger of BFG/
Pyrmaid, Pylons and TG, with the 2 package concept Mark mentioned
(minimal-core vs full-stack).
I've heard a lot of that in this thread. Sure there's been some feedback in other directions, but here and offline I'd say the majority of people are in favor of some kind of merger. And I think that the momentum that Pyramid has been gaining over the last few days since it's public announcement has been a very good thing.
* there will be "Pyramid" as the base, and "TurboPyramid" as its more full-featured partner, and the distinction is expressed in the name,
* this will also retain the essentials of each brands' name, and
* express that TurboGears is now based on Pyramid.
I suppose during TurboGears 2's development it would not have been good to move from "TurboGears" to "TurboPylons" (and now on to "TurboPyramid," needlessly multiplying names), because though Pylons and TurboGears were cooperating at that point, they were not actually merged into one. But now if they do merge, it might be best to bring both the minimal-core & full-stack names within the same general brand umbrella in this kind of fashion.
Tim Black
Andy Bierman
>
> What you really need to be successful is a singular set of tools, with
> just about everything you should need to write a web app built in,
> fully documented as if TG were a single project like Django as far as
> the user is concerned. Abstract out all your dependencies too, there's
> no reason I should be importing things directly from SQLAlchemy or
> Genshi, I should be importing things like tg.database and tg.template,
> even if they're only wrappers so that if and when those things are
> changed I don't have to go back and update my code to the new
> dependency. That also applies to documentation, putting "see docs on
> Dependency X's web site" doesn't cut it. If you want to be the full-
> featured mega framework, then I should rarely have to leave the
>
>
> Well, we aren't going to wrap the entirity of SQLAlchemy or Genshi or
> whatever, because that's a HUGE API to try to manage. But I agree we
> need to be doing a better job of managing that set of problems.
>
It would be possible to encapsulate a reasonable subset for people who
cared about stability more than incremental feature improvements.
I guess I don't really understand the goals of this group.
I guess TG is supposed to target people who want to constantly
change their code to chase the latest-and-greatest bleeding-edge opensource.
If the framework makes me code to components like dB or templates directly,
then I have 2 choices:
1) maintain the code myself when TG tires of the underlying technology
and moves on to something else
2) rewrite my code so it stays current with the framework
Andy
> I think we will likely need to create some new abstraction layers in a
> couple of key places in order to make the "full stack" approach work in
> a world where NoSQL and HTML5 local apps are busy changing the web
> framework landscape.
>
> Personally I'm still leaning toward the 3 way merger of BFG/
> Pyrmaid, Pylons and TG, with the 2 package concept Mark mentioned
> (minimal-core vs full-stack).
>
>
> I've heard a lot of that in this thread. Sure there's been some
> feedback in other directions, but here and offline I'd say the majority
> of people are in favor of some kind of merger. And I think that the
> momentum that Pyramid has been gaining over the last few days since it's
> public announcement has been a very good thing.
>
> --Mark Ramm
>
mark.mch...@gmail.com
> I guess TG is supposed to target people who want to constantly
> change their code to chase the latest-and-greatest bleeding-edge opensource.
>
> If the framework makes me code to components like dB or templates directly,
> then I have 2 choices:
> 1) maintain the code myself when TG tires of the underlying technology
> and moves on to something else
> 2) rewrite my code so it stays current with the framework
We will support the DB layer and template layer indefinitely, we're
not planning on abandoning SA or Genshi. We just aren't planning on
"wrapping" them so that they hidden from you, and you are insulated
from changes *inside* genshi or SQLAlchemy.
If that's not enough, then you really do need a monolithic framework,
and that's not going to be TG.
Andy Bierman
>> I guess I don't really understand the goals of this group.
>> I guess TG is supposed to target people who want to constantly
>> change their code to chase the latest-and-greatest bleeding-edge opensource.
>>
>> If the framework makes me code to components like dB or templates directly,
>> then I have 2 choices:
>> 1) maintain the code myself when TG tires of the underlying technology
>> and moves on to something else
>> 2) rewrite my code so it stays current with the framework
>
> That's not really fair.
>
perhaps, but for people like me that invested in TG1, we were led
to believe that SO and Kid would be supported.
For people starting from scratch, SA and genshi are good choices,
so I hope they really do get supported and not dropped for the
next greatest thing. (So why is Elixir not really supported in TG2.1 again?)
For stuff like identity, access control, session management, and deployment,
there is no good excuse for the all the churn.
> We will support the DB layer and template layer indefinitely, we're
> not planning on abandoning SA or Genshi. We just aren't planning on
> "wrapping" them so that they hidden from you, and you are insulated
> from changes *inside* genshi or SQLAlchemy.
>
> If that's not enough, then you really do need a monolithic framework,
> and that's not going to be TG.
>
I want to use TG.
If a reasonably robust and well documented branch was available, I would use it.
IMO, the 5 concurrent branch approach was a big mistake and caused lots of confusion.
The premise that users want to mix-and-match so many different ways is rather suspect.
Perhaps if SA and genshi were the only external dependencies, and the rest was solid TG
framework where the user API backward compatibility is a top priority, this would be
a winning approach.
Andy
Michael Brickenstein
On Nov 11, 3:54 am, "mark.mchristen...@gmail.com"
> We will support the DB layer and template layer indefinitely, we're
> not planning on abandoning SA or Genshi. We just aren't planning on
> "wrapping" them so that they hidden from you, and you are insulated
> from changes *inside* genshi or SQLAlchemy.
Doing a framework like TG is a very, very hard job.
After all these years of web development it is very clear to me
that too much wrapping and improving in reality makes the situation
just worse.
I appreciate the modular approach of TG and I prefer excellent
components to super integrated ones.
Moreover, I think it is important to think of TG not only as a
framework but also
as a community of developers, building webapps with a very similar set
of components and similar problems.
Even in the very rare case, that components are no longer supported by
the original author,
then I know that there is a whole community searching for the best
upgrade path.
Thanks to all TG developers!
Cheers,
Michael
ca1...@bellsouth.net
Sent from my Verizon Wireless BlackBerry
-----Original Message-----
ca1...@bellsouth.net
$covjvjvjuxyr
Mark Ramm
>
> I'm curious to know what others think, if tg plans to address this piece in the pyramid puzzle.
The pyramid puzzle is I think destined to (and was designed to)
include two layers under the same Pylons Project banner.
There's the low level, fast, flexible, powerful framework that makes
very few assumptions, and maximizes flexibility.
But then there's another related project that configures all of that
in a standard way, includes a lot more "batteries" and works to
provide lots of opt in options to the users of Pyramid layer, and lots
of defaults to this higher layer.
If we go through with a merger I expect that the TurboGears
development community is going to be much more interested and much
more active on this second layer. Which is natural because it's a
reasonable outgrowth of the TurboGears philosophy.
--Mark Ramm
Ben Bangert
> There's the low level, fast, flexible, powerful framework that makes
> very few assumptions, and maximizes flexibility.
>
> But then there's another related project that configures all of that
> in a standard way, includes a lot more "batteries" and works to
> provide lots of opt in options to the users of Pyramid layer, and lots
> of defaults to this higher layer.
>
> If we go through with a merger I expect that the TurboGears
> development community is going to be much more interested and much
> more active on this second layer. Which is natural because it's a
> reasonable outgrowth of the TurboGears philosophy.
up very clearly. No one wants to be abandoned "again" for the "next
best thing". Part of the reason I think this happened is that despite
TG choosing different best of breed components, and increasing the
user-base, the developer base very frequently did not increase in any
noticeable fashion. Some one-off patches perhaps, and that was about
it.
As such, if the chosen golden project that was best of breed failed to
keep up, it was in danger of no longer being the best just a year or
so later. That's kind of a bummer really.
Already with Pyramid, I'm trying to avoid that, because for Pylons
users, they'll be used to using Mako and SQLAlchemy. I want to ensure
these two tools *stay* best of breed for the choice of ORM and the
choice of non-XML template language. So I'm talking with Mike Bayer,
because I want to ensure that SQLAlchemy/Mako have the support/help
needed to keep them on top.
Once something is chosen as best of breed, if enough developers hop on
board, it should be pretty dang hard to de-thrown it. If Kid had a
huge dev base early on, I think Genshi might not have even been
necessary. So ensuring that the 'chosen tools' get the support they
need feels like a very critical aspect to me.
If TG merges, I'd like to ensure that with the developers that
encompasses, every part of the "full stack" has sufficient resources
to ensure there is no further hopping between core components. That
the "best of breed" *stays* the best option, and that if something
else starts to surpass it... well, there'll be enough resources
available so that we can catch up. Changing projects perpetually is
clearly not maintainable.
I know some people were worried about other projects being gobbled by
"The Pylons Project", but really, given the code/test/doc requirements
necessary to be part of the Pylons Project I can't see any reason
anyone should be anything other than thrilled that a project is
accepted to be managed under these requirements.
Cheers,
Ben