cherrypy and toscawidgets for dummies?
Jim Jones
i'm a django refugee, currently sharpening my teeth
on cherrypy. i like the minimal weight and that i get
to choose pretty much everything.
on my quest to find a sane way to deal with html forms
(mapping from/to entities, generation, validation) i found
pointers to toscawidgets in many places - apparently that's
what you big guys use and recommend these days. :-)
unfortunately integrating TW with plain cherrypy
(no turbogears here!) has turned out quite challenging -
partly due to the total lack of documentation.
could anyone be bothered to write up a short
tutorial/overview for dummies like me?
basic usage (of the forms stuff) seems to be outlined here:
http://wiki.pylonshq.com/display/toscawidgets/Using+twForms+with+Pylons.+Part+1
i also found a list post with code snippets that seem to be necessary
to get CP and TW work together at all:
http://groups.google.com/group/toscawidgets-discuss/browse_thread/thread/85e891ac023ef4d1/ff61267adcf10dbc?lnk=gst&q=cherrypy#ff61267adcf10dbc
but how to i put the pieces together?
my humble attempts so far have only led to more or less
obscure exceptions, so any help would by highly appreciated!
best regards,
jim
percious
working on a set of documentation which might help. Included are a
number of tutorials aimed at helping developers get a grasp on what is
really going on with TW. I will bug alberto about documentation for
CherryPy, or start working on it myself. I mean, if I can get it
working on Grok, how hard can it be to get it working on CherryPy?
By the way, which version of CP are you using?
-chris
On Apr 3, 5:09 pm, Jim Jones <jjimjjo...@googlemail.com> wrote:
> hi guys,
>
> i'm a django refugee, currently sharpening my teeth
> on cherrypy. i like the minimal weight and that i get
> to choose pretty much everything.
>
> on my quest to find a sane way to deal with html forms
> (mapping from/to entities, generation, validation) i found
> pointers to toscawidgets in many places - apparently that's
> what you big guys use and recommend these days. :-)
>
> unfortunately integrating TW with plain cherrypy
> (no turbogears here!) has turned out quite challenging -
> partly due to the total lack of documentation.
>
> could anyone be bothered to write up a short
> tutorial/overview for dummies like me?
>
> i also found a list post with code snippets that seem to be necessary
percious
http://docs.turbogears.org/2.0/RoughDocs/ToscaWidgets
cheers.
-chris
Jim Jones
thanks for the quick reply.
Actually I have looked "everywhere" incl. the TG-Domain,
I spent almost a day on google. ;-)
Nonetheless, thanks for the pointer!
The problem is that most bits & pieces that I found were about
general usage of TW when it's already properly integrated.
But I'm stuck a bit earlier in the process; I have
yet to make TW play properly with CP at all.
It's CP 3.0.3 here from cheeseshop. Since you're asking
"how hard can it be" I guess I'll just outline what I've
done so far:
I installed TW and all dependencies via easy_install.
The first attempt failed (some deps were broken, I don't
remember the exact problem) but the instructions on
http://code.google.com/p/dbsprockets/wiki/Installation
under "Common Problems" eventually helped me around that.
Now I can do this:
$ easy_install toscawidgets
Searching for toscawidgets
Best match: ToscaWidgets 0.2
Processing ToscaWidgets-0.2-py2.5.egg
ToscaWidgets 0.2 is already the active version in easy-install.pth
Which makes me think TW is properly installed?
For integration with cherrypy I first put this in my app.cfg
under [/] and under [global] because I wasn't sure where it
really belongs:
wsgi.pipeline = [('tw', toscawidgets.middleware.make_middleware)]
That only gave me an Exception on CP startup, complaining that
"middleware was not found in toscawidgets". I'm too lazy to
reproduce the exact error right now but I couldn't resolve
it using all kinds of crazy import statements in my
main python code (where the CP engine is started).
Since I *knew* toscawidgets.middleware is there (through the
power of 'ls') I decided to forget about app.cfg and move
my cfg into the python code and got to this:
--snip--
import toscawidgets
...
conf = {
'/': {
'tools.staticdir.on': False,
'tools.encode.on': True,
'tools.decode.on': True,
'tools.encode.encoding': 'utf-8',
'tools.staticdir.dir': 'static',
'request.dispatch': d.dispatch,
'tools.trailing_slash.on': True,
'server.request_queue_size':128,
'wsgi.pipeline': [('tw', toscawidgets.middleware.make_middleware)],
}
}
cherrypy.quickstart( None, '/', config=conf )
--snip--
Which bails out on startup with this Exception:
Traceback (most recent call last):
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/wsgiserver/__init__.py", line 624, in communicate
req.respond()
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/wsgiserver/__init__.py", line 357, in respond
response = self.wsgi_app(self.environ, self.start_response)
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/_cptree.py", line 74, in __call__
return self.wsgiapp(environ, start_response)
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/_cpwsgi.py", line 268, in __call__
head = callable(head, **conf)
TypeError: make_middleware() takes at least 2 arguments (1 given)
After this point my memory fades a bit.
I know that at one point I *did* get it to initialize properly
(I think I did something wierd like "import middleware from toscawidgets"),
but on first usage it would drop out with:
TypeError: No object (name: ToscaWidgets per-request storage) has been
registered for this thread
During investigation of that new problem I found that mailing list post
(second link in my first e-mail) which seems to suggest that I need
to first wire up a "ToscaWidgetsTool" with CP to get a proper TW-Environment
with the threading stuff sorted out. I spent some hours trying to figure out
how to integrate or use that code but failed. That's where I eventually
gave up and rolled everything back (after 15 hours of fun ;)).
So, to sum it up and answer your question: I think it is *really damn hard*,
at least for someone with medicore python knowledge who's just starting with
cherrypy, WSGI and TW.
My problems:
* There's zero useful documentation for TW. Everything I had were bits & pieces
from blogs and mailing list posts. I'm still not even sure that it will
eventually really do what I want but I firmly believe into risk-vs-reward. ;-)
* The CP docs are better but not optimal either. Lots of very basic stuff is
unclear to newbies, many of the docs still refer to old CP versions (without
saying so) and fail in wierd ways with CP3.
For example: Pretty much every doc/tutorial uses a different way to
initialize and startup cherrypy. Some use cherrypy.quickstart(), some use
cherrypy.server.quickstart(), some use cherrypy.engine.start()/cherrypy.engine.block(),
some use an app.cfg, some don't, some use cherrypy.tree.mount(), some don't.
I know that lots of that can be used interchangably but dealing with the
subtle differences ("No webserver was started" etc.) all the time is painful
while you're actually trying to solve a different problem.
I don't blame anyone for this. I know that both CP and TW are primarily aimed
at framework-builders. Still, it would be very nice to have a simple step-by-step
guide for vanilla CP/TW integration that stubborn "roll-my-own-users" like me can use. :-)
regards,
Jim
Jim Jones
I'm sure this will help once I got TW to run :-)
Alberto Valverde
http://beta.toscawidgets.org/trac/tw/browser/examples/cherrypy_app.py
(If you're searching the list and the above link is broken try removing
the beta prefix)
Unfortunately, I've stumped into a bug which I'm too tired to chase
right now. In my machine the HTTP response is truncated when I turn
inject_resources on. Strangely enough, wsgi_app, which uses wsgiref's
WSGI validator from stdlib, works fine and no errors are reported.
Can anyone please confirm this to single out a platform dependent issue?
My dev machine is an Ubuntu 7.10 running linux 2.6.22 and python2.5.
Alberto
Jim Jones
the app starts but I get this exception for every request:
Traceback (most recent call last):
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/wsgiserver/__init__.py", line 624, in communicate
req.respond()
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/wsgiserver/__init__.py", line 357, in respond
response = self.wsgi_app(self.environ, self.start_response)
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/_cptree.py", line 74, in __call__
return self.wsgiapp(environ, start_response)
File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/_cpwsgi.py", line 270, in __call__
return head(environ, start_response)
File "/opt/proj7/trunk/sys/opt/python-2.5.2/lib/python2.5/site-packages/ToscaWidgets-0.2-py2.5.egg/toscawidgets/middleware.py", line 40, in __call__
return self.wsgi_app(environ, start_response)
File "/opt/proj7/trunk/sys/opt/python-2.5.2/lib/python2.5/site-packages/Paste-1.6-py2.5.egg/paste/registry.py", line 334, in __call__
app_iter = self.application(environ, start_response)
File "/opt/proj7/trunk/sys/opt/python-2.5.2/lib/python2.5/site-packages/ToscaWidgets-0.2-py2.5.egg/toscawidgets/middleware.py", line 43, in wsgi_app
self.host_framework.start_request(environ)
TypeError: unbound method start_request() must be called with HostFramework instance as first argument (got dict instance instead)
I have CP 3.0.3, ToscaWidgets-0.2-py2.5.egg, FormEncode-1.0.1-py2.5.egg, tw.forms-0.2dev_r74-py2.5.egg,
all from cheeseshop.
greets,
Jim
Alberto Valverde
> Hi Alberto,
>
> the app starts but I get this exception for every request:
>
> Traceback (most recent call last):
> File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/wsgiserver/__init__.py", line 624, in communicate
> req.respond()
> File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/wsgiserver/__init__.py", line 357, in respond
> response = self.wsgi_app(self.environ, self.start_response)
> File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/_cptree.py", line 74, in __call__
> return self.wsgiapp(environ, start_response)
> File "/opt/proj7/x/sys/python/lib/python2.5/site-packages/cherrypy/_cpwsgi.py", line 270, in __call__
> return head(environ, start_response)
> File "/opt/proj7/trunk/sys/opt/python-2.5.2/lib/python2.5/site-packages/ToscaWidgets-0.2-py2.5.egg/toscawidgets/middleware.py", line 40, in __call__
> return self.wsgi_app(environ, start_response)
> File "/opt/proj7/trunk/sys/opt/python-2.5.2/lib/python2.5/site-packages/Paste-1.6-py2.5.egg/paste/registry.py", line 334, in __call__
> app_iter = self.application(environ, start_response)
> File "/opt/proj7/trunk/sys/opt/python-2.5.2/lib/python2.5/site-packages/ToscaWidgets-0.2-py2.5.egg/toscawidgets/middleware.py", line 43, in wsgi_app
> self.host_framework.start_request(environ)
> TypeError: unbound method start_request() must be called with HostFramework instance as first argument (got dict instance instead)
>
> I have CP 3.0.3, ToscaWidgets-0.2-py2.5.egg, FormEncode-1.0.1-py2.5.egg, tw.forms-0.2dev_r74-py2.5.egg,
> all from cheeseshop.
Please try the latest TW development snapshot:
easy_install -U
http://beta.toscawidgets.org/hg/ToscaWidgets-dev/archive/tip.zip
Alberto
Jim Jones
Thank you!
Works now, that got me started.
cheers,
Jim
Jim Jones
For me that test app works without a problem
(gentoo linux 2.6.23.12, py2.5).
cheers,
Jim
Jim Jones
On Sat, 2008-04-05 at 03:32 +0200, Alberto Valverde wrote:
Sorry for the long latency but I had to set this project
aside for a while. I'm now seeing the same problem that you
describe: my output gets truncated after exactly 10169 bytes.
Have you had time to look into this, yet?
regards,
Jim
Alberto Valverde
Graham Higgins
> Unfortunately, I've stumped into a bug which I'm too tired to chase
> right now. In my machine the HTTP response is truncated when I turn
> inject_resources on. Strangely enough, wsgi_app, which uses wsgiref's
> WSGI validator from stdlib, works fine and no errors are reported.
>
> Can anyone please confirm this to single out a platform dependent issue?
> My dev machine is an Ubuntu 7.10 running linux 2.6.22 and python2.5.
the light is a sickly greenish-purple :-)
It's not platform- or server- specific, I see it on OS X (10.5.2)
under Pylons 0.9.7dev.I can get a fully rendered page using wget, lynx
or FF 2.0.0.13 but, strangely, not in Safari 3.1 (5525.13) or in Opera
9.27. This suggests that it might be a JS-related issue.
I'm on OS X 10.5.2, using Pylons hg clone and working from the latest
Toscawidgets-dev tip and twForms-dev hg clone.
(incidentally, I don't seem to be able to "hg clone" the TW-dev repos,
I get HTTP 500 status errors:
% hg clone http://beta.toscawidgets.org/hg/ToscaWidgets-dev/
destination directory: ToscaWidgets-dev
requesting all changes
abort: HTTP Error 500: Internal Server Error
)
I've discovered that the truncation doesn't necessarily occur at the
same point in the serialization of the response. In the example I am
using, the truncation always occurs during the rendering of
CalendarDatePicker, wherever that element occurs in the serialization.
In both Safari and Opera, the truncation occurs at exactly the same
place:
<label id="mill_category_form_release_da
FF renders it fully as:
<li class="even"
id="mill_category_form_release_date.container">
<label id="mill_category_form_release_date.label"
for="mill_category_form_release_date" class="fieldlabel
required">Release Date</label>
<div>
<input type="text" id="mill_category_form_release_date"
class="calendardatepicker required" name="release_date"
value="08-04-14" />
<input type="button" class="date_field_button"
id="mill_category_form_release_date_trigger" value="Choose" />
</div>
</li>
The ticket referred to by Alberto declares the issue to be CP-related,
so I've refrained from adding to it but perhaps when the cross-
platform nature of the issue is confirmed, the ticket should be closed
and reopened as a broader issue.
That's about the limit of my contribution, except to observe that when
the injection does work, the injected link markup seems to remain in
the request when it gets passed on to the dispatcher, (Routes, in this
instance):
No route matched for GET /toscawidgets/resources/
toscawidgets.widgets.forms.calendars/static/calendar/calendar.js
[...]
No route matched for GET /toscawidgets/resources/
toscawidgets.widgets.forms.calendars/static/calendar/calendar-
system.css
(and so on, ad nauseam)
Is that the result of Real Processing(tm) being done by Routes? In
which case, how do I persuade Routes not to waste cycles trying to
find a match for injected TW resources?
Graham Higgins
> In both Safari and Opera, the truncation occurs at exactly the same
> place:
>
> <label id="mill_category_form_release_da
calendardatepickers, both were rendered fully by Safari, the
truncation occurred later :-(
However, Opera and Safari both truncate the rendering at /exactly/ the
same point in the page. There's something they agree about on which FF
differs.
On Vista Home Premium (spit) IE7 agrees with Safari/Opera and
truncates, whereas FF 2.0.0.9 agrees with its OS X counterpart and
renders the complete page.
Time to dig out the old "This web site is best viewed using Firefox"
badge :-)
Alberto Valverde
Thanks for the detailed investigation. Comments follow between quotes.
Graham Higgins wrote:
> On Apr 5, 2:32 am, Alberto Valverde <albe...@toscat.net> wrote:
>> Unfortunately, I've stumped into a bug which I'm too tired to chase
>> right now. In my machine the HTTP response is truncated when I turn
>> inject_resources on. Strangely enough, wsgi_app, which uses wsgiref's
>> WSGI validator from stdlib, works fine and no errors are reported.
>>
>> Can anyone please confirm this to single out a platform dependent issue?
>> My dev machine is an Ubuntu 7.10 running linux 2.6.22 and python2.5.
>
> I can throw a little more light on this. Unfortunately, the colour of
> the light is a sickly greenish-purple :-)
Indeed :(
>
> It's not platform- or server- specific, I see it on OS X (10.5.2)
> under Pylons 0.9.7dev.I can get a fully rendered page using wget, lynx
> or FF 2.0.0.13 but, strangely, not in Safari 3.1 (5525.13) or in Opera
> 9.27. This suggests that it might be a JS-related issue.
>
> I'm on OS X 10.5.2, using Pylons hg clone and working from the latest
> Toscawidgets-dev tip and twForms-dev hg clone.
>
> (incidentally, I don't seem to be able to "hg clone" the TW-dev repos,
> I get HTTP 500 status errors:
>
> % hg clone http://beta.toscawidgets.org/hg/ToscaWidgets-dev/
> destination directory: ToscaWidgets-dev
> requesting all changes
> abort: HTTP Error 500: Internal Server Error
> )
I've managed to reproduce this and fixed it (by hgpull'ing from tg.org a
fresh checkout). The error on the server was:
[Mon Apr 14 18:13:18 2008] [error] [client 192.168.0.100] File
'/usr/local/lib/python2.5/
site-packages/mercurial/revlog.py', line 97 in decompress
[Mon Apr 14 18:13:18 2008] [error] [client 192.168.0.100] return
_decompress(bin)
[Mon Apr 14 18:13:18 2008] [error] [client 192.168.0.100] error: Error
-5 while decompres
sing data
Which I *believe* to be related to the fact that I dumbly rsynced my
local repository to beta.tw.org where I have different hg versions and
there might be some file-format discrepancies between hg versions.
TW's middleware should have intercepted those requests and serve the
files instead of calling the wrapped pylons app. I think the problem
you're seeing is probably because Routes is stacked above TW (closer to
the server), is it? If so then try stacking TW above Routes to have a
wsgi stack similar to the way it was in pylons 0.9.6.
Back to the truncation issue, it seems it is neither:
- A platform issue (since the problem is reproduced in MacOS too)
- A cherrypy issue (since Pylons and wsgi_app.py, which I've just tried,
exhibit the same problem)
- A JS issue (I get the truncated response with wget)
Given that the problem appeared with the new injector_middleware I
believe it must be related to something dumb I wrote there :(
One interesting thing I've noticed is that it works fine with FF
3.0beta2 on linux, which according to the log outputted by wsgiref's
server (used by wsgi_app.py) is using HTTP 1.1 as a transport protocol
but the same FF breaks with cherrypy_app.py using 1.1 as well. To sum up:
FF (HTTP/1.1) + wsgiref's server -> Good
wget (HTTP/1.0) + wsgiref's server -> Bad
FF (HTTP/1.1) + cherrypy's server -> Bad
wget (HTTP/1.0) + cherrypy's server -> Bad
This makes me suspect a that something to do with... ahh, got it! Darn,
Middleware Is Hard, resource_injector was not updating Content-Length
with the new length hence well-behaved clients/servers/proxies were
ignoring everything past the original document length (before injection)
Fixed in latest tip. :) Please confirm.
Alberto
Alberto
Graham Higgins
> > (incidentally, I don't seem to be able to "hg clone" the TW-dev repos,
>
> > No route matched for GET /toscawidgets/resources/
>
> files instead of calling the wrapped pylons app. I think the problem
> you're seeing is probably because Routes is stacked above TW (closer to
> the server), is it? If so then try stacking TW above Routes to have a
> wsgi stack similar to the way it was in pylons 0.9.6.
> with the new length hence well-behaved clients/servers/proxies were
> ignoring everything past the original document length (before injection)
>
> Fixed in latest tip. :) Please confirm.
attribute 'close':
File '.../WebError-0.8a-py2.5.egg/weberror/evalexception/
middleware.py', line 364 in respond
app_iter = self.application(environ, detect_start_response)
File '.../Beaker-0.9.3-py2.5.egg/beaker/middleware.py', line 74 in
__call__
return self.app(environ, start_response)
File '.../Beaker-0.9.3-py2.5.egg/beaker/middleware.py', line 145 in
__call__
return self.wrap_app(environ, session_start_response)
File '.../Routes-1.8-py2.5.egg/routes/middleware.py', line 99 in
__call__
response = self.app(environ, start_response)
File '.../ToscaWidgets-dev/toscawidgets/middleware.py', line 39 in
__call__
return self.wsgi_app(environ, start_response)
File '.../Paste-1.6-py2.5.egg/paste/registry.py', line 334 in __call__
app_iter = self.application(environ, start_response)
File '.../ToscaWidgets-dev/toscawidgets/middleware.py', line 54 in
wsgi_app
app_iter = self.application(environ, start_response)
File '.../ToscaWidgets-dev/toscawidgets/resource_injector.py', line 51
in _injector
app_iter.close()
AttributeError: 'list' object has no attribute 'close'
I threw caution to the wind and edited resource_injector.py, changing
app_iter.close() to:
if hasattr(app_iter, 'close'):
app_iter.close()
Which seemed to work. At any rate, the form renders happily and is no
longer truncated. I need to get better WSGI-fu.
Alberto Valverde
Fixed in latest SVN/tip. Thanks
Alberto