web.py review/questions for best practices

[Justin Bayer]

Hello group,

I am quite new to web.py and after spending only half of a day to
rewrite the major parts of my blog with web.py some issues came up.

(1) Object publishing, url dispatch
The url dispatch is å quite nice thing. It lets me write new pages so
fast. I have some questions though.

For instance, my blog. The blog is divided into two parts:
administration and presentation. What I am doing at the moment:

urls = ( '/admin/(.*)', 'Admin',
'/view/(\d+)', 'View',
#...
)

class Admin(object):
def GET(self, one_single_variable):
print "Administration"
# look up further administration logic

What I have to do now is let Admin.GET dispatch further. I would like
to dispatch in the urls tuple. Not onto different classes like
AdminNewBlogEntry and AdminEditBlogEntry but to, e.G.
Admin.GET_new_blog_entry.

What I am asking: Is there a best practice how to achieve this? Is
there a way of letting web.py dispatch
"/(?P<package>)/(?P<module>.*?)/(?P<class>.*?)/(?P<method>.*)" to
package.module.class.GET_method?

Or: what is the best way of moving the logic of different parts of the
code into seperate files?

(2) Base template, Cheetah templates, Containers
I have a "base design" of my blog. That is a page where the blog
title is, where my eMail adress is and so on. I do not want to hardcode
this into the template. Instead I want to define it in my index.py. At
the moment I am using cheetah's #BLOCK, which I do not like that much
due to the necessity (?) to use #def in the included template.

What's the best pracitce for realizing containers in web.py? Containers
in the sense of "Putting together multiple applications", like a
calendar, a login form and a blogpost listing. I find it difficult.

(3) Repetition
I find myself repeating myself a lot in web.py. For instance I often
have to write

'/somebase/(.*?)', 'somebase'

I can understand that web.py mabe is not meant to be that high level.
It's not a problem to write this myself. But I would like to now what
others are doing.

(4) POST Variables
As far as I know, Post Variables should NOT overwrite each other if
they have the same key.

key=value1&key=value2

should not return {"key":"value2"} * but (('key', 'value1'), ('key',
'value2')). The current situation is very unhandy. For example when
creating multiselect selects.

* I know that should be a storage object, but writing a dict here is
okay, too.

(5) Cookies
Do I have to take care of cookie security myself or will web.py (try
to) make sure that the cookies send are really cookies that web.py set?

(6) After all
web.py is definately the web library to end all web frameworks. ;) Just
kidding, I think the others have their positive sides, too. (But I
don't like letting a web-framework creating my sql tables for me...)

(7) Documentation
Documentation is not that good. After I worked a few things out, I
would like to contribute to that.

In hope to be enlightend ;)
-Justin

[Aaron Swartz]

> What I have to do now is let Admin.GET dispatch further. I would like
> to dispatch in the urls tuple. Not onto different classes like
> AdminNewBlogEntry and AdminEditBlogEntry but to, e.G.
> Admin.GET_new_blog_entry.

You can use the `web.autodelegate` function for this.

> What I am asking: Is there a best practice how to achieve this? Is
> there a way of letting web.py dispatch
> "/(?P<package>)/(?P<module>.*?)/(?P<class>.*?)/(?P<method>.*)" to
> package.module.class.GET_method?

You can't currently pick the method name, although that might be a
good idea for the future. You can however pick different files and
modules and so on.

'(.*?)/(.*?)', r'\1.\2' should generally work (although be careful;
this will allow users to initialize arbitrary classes and call their
GET and POST methods, so make sure you don't have any that do anything
dangerous).

> (2) Base template, Cheetah templates, Containers

All this stuff becomes a lot clearer in template.py, I think.

> (3) Repetition
> I find myself repeating myself a lot in web.py. For instance I often
> have to write
>
> '/somebase/(.*?)', 'somebase'
>
> I can understand that web.py mabe is not meant to be that high level.
> It's not a problem to write this myself. But I would like to now what
> others are doing.

The url list is just a list, so you can certainly generate it dynamically:

for x in ['somebase', 'anotherbase']:
urls += ('/%s/(.*?)' % x, x)

> (4) POST Variables
> As far as I know, Post Variables should NOT overwrite each other if
> they have the same key.

The plan was that you could give web.input a list to indicate you
wanted back a list of values (instead of overwriting), but it doesn't
look like I ever implemented that. In any event, would that be OK:

web.input(f=[]).f == ['1', '2', '3']

> (5) Cookies
> Do I have to take care of cookie security myself or will web.py (try
> to) make sure that the cookies send are really cookies that web.py set?

I'm not sure what you mean.

> (7) Documentation
> Documentation is not that good. After I worked a few things out, I
> would like to contribute to that.

That would be great; I wish I had more time to write documentation.

[_max]

Hi,

Here's my experience with the issue.

I have been working on a fairly complex app web.py , formed of a few
sub applications including a blog.

The main web.py file contains only the urls of my apps db connection
definition and the run method call

the main urls look a bit like :
urls = apps.blog.public.urls + apps.blog.admin.urls +
apps.p2tgmap.public.urls

Where apps.*.urls are defined in apps/*.py and so on.

All the app code is in their respective python file.

I am not using autodelegate (lazyness), though i might in the future.

It s a bit like django does i think, a main url dispatcher where you
(can) include you apps dispatchers wich are defined deeper in the apps
folder.

I think this "freedom" in how you can layout your application is one of
the greatest features of web.py, it gives you the smallest (without
beeing a cgi++ module) set of tools to build webapps with python
without hassle (no need to worry about the common web dev problems,
like escaping, sql injection, etc), the rest is up to you.

The documentation is a bit small i agree, i'd have liked to find
something about uploads in it for example, even though given the small
codebase of web.py it s a matter of minutes to dig in it and get an
answer from the code (yet another huge benefit from web.py).

[Justin Bayer]

Hello,

Thanks for the response. I looked into template.py, which is a lot
better than cheetah I think. I do have on problem though with its
automatic escaping of HTML Tags. I like to go by the Zen of Python
which says "Explicit is better than implicit."
I have yet find out how I can prevent the websafe function to be used
on the input if the extension of the template is html.

> web.input(f=[]).f == ['1', '2', '3']

Looks complicated to me but I do not think that I can come up with
something better. Perhaps an iterator that evaluates to its first value
if used as a string.

>>> str(web.input().multiple)
1
>>> web.input().multiple

['1', '2', '3']

But that's quite a lot of magic, too.

> I'm not sure what you mean.

I guess the best is if I take some extra-lessons on cookies, then ask
again. :)

[Justin Bayer]

Hey max,

> the main urls look a bit like :
> urls = apps.blog.public.urls + apps.blog.admin.urls +
> apps.p2tgmap.public.urls

Neat approach. Can you give an example how those urls-tuples in the
modules look like and what your imports of the applications look like
in the main script? If I am correct, there should be some namespace
issues. (like app.blog.public needing to know what main.py's path to
the module is)

-Justin

[Aaron Swartz]

> Thanks for the response. I looked into template.py, which is a lot
> better than cheetah I think. I do have on problem though with its
> automatic escaping of HTML Tags. I like to go by the Zen of Python
> which says "Explicit is better than implicit."

That's probably usually true, but not when it comes to something as
important as security. You shouldn't have to explicitly turn of
security holes.

> I have yet find out how I can prevent the websafe function to be used
> on the input if the extension of the template is html.

$: instead of just $ -- this should definitely be more clear in the
docs, though.

> something better. Perhaps an iterator that evaluates to its first value
> if used as a string.

I think that'd break a lot of existing code.

[_max]

You have to design you urls as if they would be in your man script,
something like this.

in /apps/blog.py :

urls = (
'/blog/(.*)','apps.blog.index',
)

in /main.py :

from apps import blog
...
...
urls += blog.urls

[Kamal Mustafa]

This is how I layout my web.py app:-

$ ls myproject
conf db lib public templates wrapper

all python modules which contain actual my web.py code goes into lib.
For now, lib still contain single modules python code but I plan to
break it up into few packages.

$ ls lib
acl.py customer.py filters.py models.py order_item.py page.py
stock.py user.py
ajax.py database.py item.py myform.py order.py payment.py
tags.py validators.py

in public, I have index.py which contain a long list of urls tuple and
a static folder. when developing in web.py, I'd prefer to think of
resource instead of action so basically all my modules just contain
class definition to display, edit and delete the resource. for
instance, in customer.py I have:-

class Customers:
def GET(self): pass

class Customer:
def GET(self, customer_id): pass
def POST(self, customer_id): pass

class CustomerEditor:
def GET(self, customer_id): pass
def POST(self, customer_id): pass

class CustomerDelete:
def GET(self, customer_id): pass

ideally, CustomerDelete should use a POST method to handle the request
but because of laziness, I use GET. I might switch to POST or even
better DELETE soon.

btw, _max approach in handling the urls dispatch look's good,
especially if your app going to grow. what about other people ? I'd
love to see their approach.

--
kamal, www.k4ml.com

[Justin Bayer]

Here is what I did this morning. What I want is that I can design
applications independently. Only my main script should know where they
are, not the script itself.

First I made a package `apps`, in which I put two modules: `blog`and
`wiki`.

# apps/__init__.py
import blog
import wiki

My main script looked like this:

# index.py
import web
import apps

urls = (
'/wiki(.*)', 'apps.wiki.dispatch',
'/blog(.*)', 'apps.blog.dispatch',
)

if __name__ == "__main__": web.run(urls, web.reloader)

As you can see, it takes a part of the url and dispatches it to the
modules dispatch class. This one looks like this:

# wiki.py (replace wiki with blog -> blog.py)
import web

class dispatch(object):

urls = (
'', lambda x: 'wiki!'
)

def GET(self, dispatch):
for url, func in web.group(self.urls, 2):
if url == dispatch:
print func(dispatch)
break

If I would design a large application now, I would make it this way.
The url dispatching in dispatch.GET is not based on regexes and I call
a lambda and stuff, because I was to lazy to implement that for the
test.
I guess it makes clear what I mean, though.

What I would like is the following: web.py introspects to what it
dispatches the results of a url handling. If it is a class it calls
GET/POST whatever, but if it is a module it looks for a dispatch
function which returns the class on which to run GET/POST whatever.

I think that would be a great way of putting together multiple
applications.

[_max]

Nice alternative. The dispatch class feels a little overkill to me
though.

I went for yet another solution.
I settled for a deeper file structure with a separation of the urls
from the controllers, like that :

/apps
/apps/blog/controller.py : contains all the classes
/apps/blog/view.py : contains the definition for urls

in main.py i do "from apps.blog. import urls" for every app i want
use.
then :

urls += apps.blog.view.urls
urls += apps.another.view.urls

so main.py isn't importing tons of unnecessary stuff anymore, and i am
happy :)

I don't mind having the apps path in the urls definitions much,
actually it s kind of a feature, i can call external applications in
the current context, consider this example :

a comment url in my blog app, but the comment app is separate from the
blog.
i can do.

urls = (
'/blog/','apps.blog.controler.index',
'/blog/post/comment','apps.comment.controler.add_comment',
)