CP newbie looking for project structure recommendations

[Dave B.]

Hi,

I am relatively new to Python (2.7) and to CP (3.2), and really like
what I've learned from the CP docs so far. I am currently planning to
develop a fairly straightforward app using CP, hookbox and Dojo, and
am looking for some "best-practices" guidance on how to structure the
project. I know CP seemingly lets you structure things however you
want, but given what I have:

- hookbox "WebHooks.py" script (Q: should this be part of the
"index.py" script below?)

- main "main.py" script to quickstart CP and serve index.html
- index.html

- additional scripts to serve content in response Dojo AJAX requests
- other content (HTML) served by the above scripts

- CSS files, image files, etc.

Just wondering what more experienced folks might recommend. For
example, in the static docs portion of the current CP site tutorial, I
see one example where __init__.py is used as the main Python script in
the topleve project directory, with other data files in
subdirectories. But elsewhere I've read that __init__.py is really
supposed to be used to allow a module to be imported by some other
application, which isn't what my web app is designed for really.
Unless it's somehow a good idea to put web apps under the Python
"scripts" area (where CP, hookbox, other products etc.) are...?

Anyway, thanks very much for any guidance, and again for what has been
proving to be a great product to use,

Dave B.

[Eric Larson]

Here is a general overview of what I do. This is by no means a prescription for success, but merely a slightly more generic way of how we lay out CP projects at my job that I have used on personal projects as well.

I should start by saying that we distribute our apps for deployment as eggs. If you're unfamiliar with eggs, then I'd take a look at setuptools, pip and virtualenv (google!) to see what that is about.

For example's sake, we'll call this project "Jack". Here is how I typically layout my package:

# $home/projects/Jack
setup.py
jack/
control/
view/
static/
static/css/
static/js/
etc/
lib/
testing/
tests/unit
tests/func
scripts/
bin/

The control dir is for my main CP handler classes. I typically will create base classes in control/__init__.py that set up things like templating and returning common formats such as JSON (there is a tool for this as well). The control dir typically also has a "root.py" which is where the actual cherrypy configuration ends up. This is where we would include the configuration details like the host, port, setup static resources, etc. Often times I fit most my controller objects in one file called main.py in control dir, but sometimes I use different files for things like API handlers vs. traditional web pages.

The view directory is where I put template files. Personally, I use Mako, but use whatever works for you (if you even need it). I usually try to organize it similarly to the structure of the app. For example, if I had a URL like foo/bar/, there might be a foo and bar directory in the view directory for those templates.

The static directory is for static resources. I typically have a js and css directory to keep things separated.

The etc directory is for configuration details. As a package I like to have a base configuration that is helpful for development that gets overridden in production. There is usually a "startup.py" file as well that helps to set up things like database connection pools. Basically it acts as a place to put bootstrapping needs.

The lib directory is where I keep most of the actual application classes. Usually I'll create a store.py file that has general database connection details. Sometimes that will involve a "model" directory with specific models defined as .py files. Typically if there is specific functionality the application can implement as a library of sorts it will get its own directory under lib but at the very least it has its own file. For example if "Jack" project does some audio processing, you might have a lib/audio/downsample.py.

The testing directory is for helpers used in testing. These could mocks, test runners, stubs, base classes used in testing, etc.

The test directory is where I keep the tests. Some people like to have a test directory for each directory in the package such that things are mirrored directly (ie lib/foo/bar.py and lib/foo/test/test_bar.py). Personally, I like having all the tests in one place, but that is just me. For running tests, I use py.test (http://pytest.org) but others might have other suggestions such as nose or unittest2.

The scripts directory is for project specific scripts. These could be shell scripts for running tests or python scripts for loading test data or helpers for building a release. Anything really.

The bin directory is for putting scripts that will end up in the python bin directory. Again, this is a setuptools feature where in your setup.py you can define console scripts that can be called from the command line. A simple example would be if you had a simple script that started up your server.

All of this is considered part of the "package" that in our case is an setuptools package. This lets us build eggs for deployment. If you do something similar be sure to consider things like package data and console scripts when writing your setup.py otherwise you might find that certain files are not present when you run the application.

Again, this is just an example of what I've been doing. It is also worthwhile to see what the larger frameworks are doing. Most can scaffold a directory structure that might be helpful to emulate if it makes sense.

Hope that helps!

Eric
> --
> You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
> To post to this group, send email to cherryp...@googlegroups.com.
> To unsubscribe from this group, send email to cherrypy-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cherrypy-users?hl=en.
>

[Sylvain Hellegouarch]

Hi Dave,

Like Eric says, CherryPy doesn't prescribe a specific way to layout your project but you usually end up something equivalent to what Eric describes (which follows what other framework use as well). You may also be interested in a snippet I've setup a while ago:

https://bitbucket.org/Lawouach/twiseless/src

--
- Sylvain
http://www.defuze.org
http://twitter.com/lawouach

[Eric Larson]

Great example app Sylvain! The templates as an engine plugin is a really helpful example.

--
Eric Larson

[Sylvain Hellegouarch]

On Thu, May 12, 2011 at 8:45 AM, Eric Larson <er...@ionrock.org> wrote:

Great example app Sylvain! The templates as an engine plugin is a really helpful example.

Cheers Eric.
Once you've started using CherryPy's bus plugins, you cannot look back. It's just way too handy and powerful :)

[Anders Langworthy]

On Thu, May 12, 2011 at 2:45 AM, Eric Larson <er...@ionrock.org> wrote:
> Great example app Sylvain! The templates as an engine plugin is a really helpful example.
>
> --
> Eric Larson
>
> On Thursday, May 12, 2011 at 1:03 AM, Sylvain Hellegouarch wrote:
>> Hi Dave,
>>
>> Like Eric says, CherryPy doesn't prescribe a specific way to layout your project but you usually end up something equivalent to what Eric describes (which follows what other framework use as well). You may also be interested in a snippet I've setup a while ago:
>>
>> https://bitbucket.org/Lawouach/twiseless/src

I'm not the OP, but there's a lot of good stuff in both of these
examples. Thanks!

[Dave B.]

I am the OP :^), and I agree with Anders and appreciate you taking the
time - thank you so much,
guys!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Dave B.

On May 12, 10:08 am, Anders Langworthy <lagrang...@gmail.com> wrote:

[Guido Kollerie]

On Thu, May 12, 2011 at 08:03, Sylvain Hellegouarch <s...@defuze.org> wrote:

Sylvain,

> You may also be interested
> in a snippet I've setup a while ago:
>
> https://bitbucket.org/Lawouach/twiseless/src

That was a very helpful example to get me started. I copied your
template plugin and tool and modified it slightly to use Jinja2
instead.

However I have found it not to be working. Reason: any dict() returned
by an exposed function is converted to a list() before it reaches the
template tool. What converts a dict() to a list() is the
lib.encoding.ResponseEncoder.encode_string() method that runs on
behalf of ResponseEncoder.find_acceptable_charset().

Any idea how to fix your template tool so that it does get a dict()
instead of a list() to work with?

--
Guido Kollerie

[Sylvain Hellegouarch]

Any idea how to fix your template tool so that it does get a dict()
instead of a list() to work with?

Off the top my head, aside from disabling the encoder tool, I'd say perhaps trying to attach the template tool to the 'before_handler' hook with a greater priority (below 70) so that it runs before the encoder tool. However that means you need to call the page handler yourself like the encoder tool does:

http://www.cherrypy.org/browser/trunk/cherrypy/lib/encoding.py#L186

This makes it a bit more complicated.

[Laurent]

Hi Sylvain,

I'm a Python and Cherrypy newbie too for a month now. To get Mako working with decorators in Cherrypy I had to use the tool described in the Cherrypy wiki: http://tools.cherrypy.org/wiki/Mako

This code is totally different and much longer than the code you gave us as an example (I slightly modified this wiki code to use Mako exceptions and it's even longer). That's a bit confusing. Can you tell me which one I should choose for a project using most of Mako features? What are the differences? Does your code support Mako inheritance like the wiki code does?

Thanks for your answer,

Laurent

[Laurent]

PS : when I talk about the code you gave us as an example for Mako integration I talk about this file: https://bitbucket.org/Lawouach/twiseless/src/d171fde9e454/lib/tool/template.py

[Sylvain Hellegouarch]

Hi Laurent,

On Sat, Aug 20, 2011 at 1:50 AM, Laurent <lauren...@gmail.com> wrote:

PS : when I talk about the code you gave us as an example for Mako integration I talk about this file: https://bitbucket.org/Lawouach/twiseless/src/d171fde9e454/lib/tool/template.py

Both are fine. It's perhaps a matter of taste more than anything else. I usually prefer custom tools rather than custom dispatchers because I consider that dispatchers are meant to locate and call the appropriate resource handler, not perform any work on the request or response itself, which is why tools are for.

So I will usually code as I show in twiseless rather than what the wiki suggests. But again, both solutions are fine by CherryPy's standards.

[Laurent]

Ok. Thanks Sylvain.

[EuGeNe]

Hi Sylvain,

When I first started using CP I was creating a serve_template function
that would load a template and render it with a given context so the
last line of my controllers was usually looking like:

return serve_template(template_name, context)

For the past couple of years I switched to using a simple decorator
wrapping the call to my controller in serve_template (using the name
of the controller as the template name, a rails-like convention)

def render(handler)
def wrap(*args, **kvargs):
return serve_template(handler.__name__, handler(*args,
**kvargs)
return wrap

@cherrypy.expose
@render
def controller(self):
return context_dict

Which has been working fine for me and is very simple.

What is the advantage of registering a tool for template rendering?

Cheers,

EuGeNe

[EuGeNe]

> Any idea how to fix your template tool so that it does get a dict()
> instead of a list() to work with?

You can return an itemize dict in your controller and call dict on
data in your tool see the tool example (my_tool attached as pystache
on controller tool) in https://gist.github.com/1185683

EuGeNe -- follow me on http://twitter.com/3kwa

[Sylvain Hellegouarch]

Hey Eugene,

I realise I never answered your question. For a simple functionality like that, the decorator idea is great. CherryPy always favours the most straightforward solution anyway. I guess I'm used to create tools so that I write them as fast as I would have for a decorator. Also it means there's consistency across my utility functions. That's about it :)

- Sylvain

[Eugene Van den Bulke]

Thanks Sylvain!

> --
> You received this message because you are subscribed to the Google Groups
> "cherrypy-users" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/cherrypy-users/-/XYQpKPLLVkwJ.

> To post to this group, send email to cherryp...@googlegroups.com.
> To unsubscribe from this group, send email to
> cherrypy-user...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/cherrypy-users?hl=en.
>

--
EuGeNe -- follow me http://twitter.com/3kwa