TurboGears and Crank

[percious]

Hi all,

Sorry I've been away for so long.  Needless to say things have been crazy in my life and I was unable to attend to TG in the last few months.  I very much appreciate the work that Michael and Amol have done in my stead.  It is especially encouraging to see that 4 releases have found their way out since I have been busy.

I have done some work in the last year that I would like considered in the next minor release (2.2).  I had worked last March to extract the dispatch mechanism from TurboGears and put it on it's own footing.  The name of this project is Crank.  This accomplishes a few goals:

1) By eliminating the Pylons/TG framework from the dispatch, testing was made easier.  Crank has 100% code coverage.

2) Crank can be used for URL dispatch outside of TurboGears.  I wanted this capability to be able to use it within TW2 without writing a new dispatcher, and in another WSGI-only CRUD (admin) system I have been developing.  Also, I had initially been in cooperation with Alice Bevan-McGregor of web.core to help provide an admin system for that project.  By making the dispatch work on it's own, I am able to maintain one codebase for all of these purposes, which puts a lot less strain on me as the one who has historically done most of the maintenance in this area of TG.

Amol has agreed to participate in maintenance of Crank along side me.  That way we don't have a single-source developer.  I have already done the work to integrate Crank into TG.  Amol finished up that work when it was merged into the current Git repos on sf.net. The current repo for Crank is at https://bitbucket.org/percious/crank/ but I am okay with it moving to the main TG repo, or moved to github as it's own project.  Also, I have kept up-to-date with dispatch changes that affect some of our most important constituents and have integrated them into Crank's codebase.

I am writing this message to garner support and feedback from the community before the inclusion of crank as a dispatch mechanism within TurboGears.

 

cheers.

-chris

[Christoph Zwerschke]

Chris, good to see you cranking TG stuff again ;-)

Maybe it's time to envision a more long-term roadmap, and keep that in
mind when working on TG2. E.g. how about our plans of joining with the
Pylons folks and basing on Pyramid? Are we heading into a different
direction now? The idea of TG is to be an opinionated full-stack
framework like Django, but with more freedom to exchange components.
Which are our long-term "opinions"? Will TG back out from doing
client-side stuff? In TG1 we had TG widgets and MochiKit. In my TG2
projects I use TW and tw.jquery heavily. I think they were always an
important part of the "full stack", but unfortunately somewhat
neglected. In my current projects I'm using even more of JavaScript and
even defected to Pyramid because I only need a very lightweight and fast
server component, TG2 would be overkill there. Anyway, on the Pyramid
mailing list I'm seeing many people who look for something more full
stack like TG2 or TG3 could offer.

I'd also like to know whether Mark and the SF folks are still behind TG2
or some kind of TG3.

-- Christoph

[Alessandro Molina]

About crank I'm fine using it, it might be a good improvement, I just
want to ensure that it won't cause issues to TurboGears as we are
practically outsource the core part of the framework. It should never
break compatibility with past TG and TG has to be a first citizen for
crank.
I can off course give help at maintaining it, not an issue for me.
But I would like to hear others ideas about outsourcing one of the
core parts of the framework.

> --
> You received this message because you are subscribed to the Google Groups
> "TurboGears Trunk" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/turbogears-trunk/-/YXqXzHnd3P0J.
> To post to this group, send email to turbogea...@googlegroups.com.
> To unsubscribe from this group, send email to
> turbogears-tru...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/turbogears-trunk?hl=en.

[Alessandro Molina]

The Pylons-less branch opens us the way of both moving to a different
foundation or going on our own.
I'm still not sure if we should move to pyramid or not, what I'm sure
is that we should work more tightly with the Pylons project guys
because in both the case we go on our own or use pyramid we will be
based on many of the packages that are part of the pylons project. Any
solution has to guarantee backward compatibility anyway.

What I know is:
1) From the Pylons package we were using about only 15% of the code as
most of it was overwritten or monkey-patched in TG.
2) Jumping around changing the framework tools multiple times caused
TG to get the fame of an unreliable framework
3) Many people I told them "woah, take a look at TurboGears, is cool"
just answered me "it seems Pylons bloated, I think I'll go with plain
pylons" and I want to avoid being the bloated dumb brother again :D

So right now I'm still thinking about what is the best for the TG
community, I'm slightly headed to the "go without pylons/pyramid
dependency and cooperate on all the Pylons project packages we can
share" way currently, but still not decided definitely.

> --
> You received this message because you are subscribed to the Google Groups
> "TurboGears Trunk" group.

[percious]

To bring the discussion back to crank, and relate to what Chris said.

Crank actually allows us to move over to pyramid without maintaining multiple codebases if we decide to do that.  I realize we are spinning off one of the major parts of TG, but I think in this case it makes sense.  This was a big problem with TG1.0, 1.5 and 2.0, all of which have separate codebases to maintain for dispatch.

I agree that TG should remain as a first-class citizen with Crank, and that backwards compatibility should be maintained.  I have done everything in my power to make sure that the existing 2.1.x tests work with crank in the codebase.

As a side note, I'm writing something to remove the repoze.w* mess, and I plan on spinning off the part of the code that selects templates/template engines into it's own package, or partnering with Alice of web.core to utilize alacarte, which is a replacement for buffet.  I'm tired of maintaining multiple dotted template lookup code too ;-)

cheers.

-chris

[Michael Pedersen]

Here's a question to consider in this discussion:

Alessandro has put together a branch that removes Pylons as a requirement. What happens if we go forward with that branch? How does it affect Crank? Are the two of them compatible, or will we have to make a choice between them?

cheers.

-chris

--

You received this message because you are subscribed to the Google Groups "TurboGears Trunk" group.

To view this discussion on the web visit https://groups.google.com/d/msg/turbogears-trunk/-/7F_CCpa9aZgJ.

To post to this group, send email to turbogea...@googlegroups.com.
To unsubscribe from this group, send email to turbogears-tru...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/turbogears-trunk?hl=en.

--
Michael J. Pedersen

My Online Resume: http://www.icelus.org/ -- Google+ http://plus.ly/pedersen

Google Talk: m.ped...@icelus.org -- Twitter: pedersentg

[Alessandro Molina]

I still have to look how crank integrates in TG, but it should be
framework independent it shouldn't be an issue.

As I worked hard at optimizing the pylons-less branch in the last
week, using profiler middleware and a set of some benchmark testing tg
applications, I have been able to push the performances from
~800req/sec up to ~1700req/sec on my machine (most simple app without
templating on gevent, genshi quickstart app on paste got from 58r/s
to 77r/s) some of the changes I did apply to the dispatching and so
should probably be ported to crank in the case we switch to it.

[Mark Ramm]

I think crank should be strongly considered if we can achieve a few things: 

1) a strong commitment to maintain backwards compatibility with tg2.1

2) no major pain for applications upgrading from 2.1 to 2.2+crank

3) crank is actively maintained by chris perkens + at least one other active person, and all core TG folks commit to it's maintenance 

4) crank becomes part of the turbogears/pylons project

I think all those things are actually pretty reasonable, and are pretty close to being in place right now.   So, I'm definitely in support of exploring this further.     

We had a lot of dispatch related bugs moving fromt he 2.0 to 2.1 code that broke a lot of people's applications, and I want to make sure we're moving to a more maintainable platform.    So, before we commit to merge this in we should test the branch with some existing codebases.   

--
Mark Ramm-Christensen
email: mark at compoundthinking dot com
blog: www.compoundthinking.com/blog

[Mark Ramm]

That's great news. 

Gevent has very high concurency performance, but in my experience there can be very large standard deviation in latency, so I still think we should benchmark paste vs paste for most applications (which are going to have blocking events like database access, etc). 

But I expect that all the work you've been doing to optimize the tg2 stack and python code will make a big difference, even without the gevent foo!

--Mark Ramm

On Wed, Dec 21, 2011 at 4:28 AM, Alessandro Molina <alessand...@gmail.com> wrote: