how to open up our code, possible pyramid extension project

[Iain Duncan]

Hey folks, so we ( xornot studios ) have been using BFG for a couple of years now, and ported all our work to pyramid. We have what I would say is quite an extensive and useful in house framework on top of pyramid that enables us to build db apps and resource oriented CMS/CRM type things really quickly. We have a lot of really useful tools done and have arrived at some great templates for large applications that need to extend our central app without touching code. We have focused on the the 'big-app' advantages of pyramid, we're using the registry and zope interfaces a lot, we're using zope 3 style views as classes for easy overriding by interface registration, that sort of thing. Templating is Chameleon zpt, and persistence is SQLAlchemy wrapped in an abstract model proxy to keep SA specific code out of anything other than the model for easy persistence swapping. We have a multi-view overridable widgeting system for building admin like user interfaces that is really slick, including almost autogenerated file upload handling, and the ability to make nested subforms almost automatically ( as automatically as we saw fit ) for handling associations and relations ( that's how the project started actually, it eats the Django admin for breakfast in my no-so-unbiased-and-humble-opinion). Dojo is used a lot for this. We have also made some RoR like scripts for creating new resources easily with code generation of views, ui and model files, mapper entries, zcml entries, etc. We are using ZCML for configuration, and buildout for building. We have it running on some pretty large client projects now, so we know it works. We use traversal and have managed to implement a zodb like security and permission scheme in SQLAlchemy with row level security and permission checking. ( I love pyramid's ACL handling, so slick! ) 

I think it's time for us as developers to take the big step of cleaning this up and starting to release parts of it, and I think it could be useful to the community as it seems like there is a lot of talk and demos out there on how Pyramid is great for minimal code, but less in the way of how the ZCA is great for big projects where the key is extensibility without modifying your main code base. I think we have that, but as a small shop of largely self taught python developers, some of it is really messy, underdocumented, and undertested. It's time for our company to grow big enough to take on subcontractors and I think releasing as open source could be a great forcing function for us to do the clean up that never seems to be highest priority when clients are paying for new features. ;-)

I'm thinking I ought to:

- find some other folks for whom this approach sounds interesting/useful who might want to team up with their similar stuff

- see if parts of what we're doing can merge with similar efforts or compliment them

- start to clean up and release bits of it, using this process to fully document and test the code

I guess this is just a shout out to see if anyone is interested in this kind of thing and might like to join in mentoring/advising us on how to make sure this is up to snuff for releasability. (Mike Orr, I'm looking at you? it sounds like it might be a good cousin to your project? )

Some questions:

- should I start a mailing list for it? discuss here or on pylons-user?

- does anyone have a suggestion for what level of functionality one should bite off as a first release? Ultra minimal with full docs? more functional with docs and testing in progress?

- can anyone comment on which of the above sound exciting? I think the admin app thing is the best feature, but it's also the hardest to get cleaned up, with some meta-class hackery in there that I'm somewhat frightened of figuring out how to test better

- do you think we need 100% code coverage, or do we do something like multiple branches of release? (ie: covered vs upcoming vs stuff we use and you can use at your own risk?)

- should we switch from SVN to git to do this?

- should this be here or on pylons discuss? ;-)

thanks!

Iain

[Andrzej Mleczko]

Hi Iain!

> Some questions:

> - can anyone comment on which of the above sound exciting? I think the admin
> app thing is the best feature, but it's also the hardest to get cleaned up,
> with some meta-class hackery in there that I'm somewhat frightened of
> figuring out how to test better

We did similar thing (at least a part of what you have describe) in
repoze.bfg - admin crud app and we get to similar doubts/questions
(http://blog.redturtle.it/redturtle-blog/2011/02/10/pyramid-crud).
Now we are trying to merge our work with pyramid_formalchemy during
"Pyramid CRUD sprint" -
http://coactivate.org/projects/pyramid-crud-sprint/project-home

However - what you are saying is very exciting, especially admin panel
stuff. I don't know how your stuff is similar to what we have and what
FormAlchemy is offering but at least I would love to share experience
and to get known how you solved certain problems ;-)

best,
Andrew

[md]

Hi Iain

Would very much welcome more code (just started looking).
Particularly interested in context of Sqlalchemy,traversal and acl.

[Rocky Burt]

This is pretty awesome.  I can't wait to see some code.

- Rocky

[Iain Duncan]

Thanks for the interest guys. I'm trying to figure out what to reasonably release first. I think the most sensible maybe a paster template to a very simple sample app that shows how we lay out the code and views, handle configuration via zcml, and take advantage of ZCA interfaces for big projects, mostly because that seems to be the secret weapon that is glossed over in the docs a bit in the ( understandable ) desire to not make the learning curve for pyramid any worse. But the thing is, ZCA interfaces and adapters are actually awesome for big extensible projects once you realize your project is big enough to warrant the extra typing. I've heard it likened to 'dependency injection on steroids'. =)

I'm debating whether this first release ( which may be a while, depending on the vagaries of clients ) should be bound to SQLAlchemy or not. I'm leaning to yes, because on of the first things I did when learning BFG was look up how to initialize SQLAlchemy code better in such a way that I could easily re-use models in different apps.

open to suggestions.

iain

On Fri, Jun 10, 2011 at 6:36 AM, Rocky Burt <rocky...@gmail.com> wrote:

This is pretty awesome.  I can't wait to see some code.

- Rocky

--
You received this message because you are subscribed to the Google Groups "pylons-devel" group.

To view this discussion on the web visit https://groups.google.com/d/msg/pylons-devel/-/bvwKH3JRHMcJ.

To post to this group, send email to pylons...@googlegroups.com.
To unsubscribe from this group, send email to pylons-devel...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-devel?hl=en.

[Iain Duncan]

BTW, if anyone has any recos on how to test paster script templates, I'd love to hear them.

iain

[Rocky Burt]

Biased opinion ahead:

I use https://github.com/serverzen/DistChecker

- Rocky