Pylons 0.10rc1 and ToscaWidgets 0.9.9 problem with Trackbacks

[cd34]

After upgrading Pylons from 0.9.7 to 0.10rc1, any undefined variable
in a template no longer displays the web trackback, but, returns a
console trackback. I installed a fresh 0.10rc1 to see if the problem
was related to the configuration changes in 0.10rc1 over 0.9.7 but the
problem persisted. Downgrading to 0.9.7 allows the web trackback to
work. I created a very basic virtualenv for testing that included
only Mako and ToscaWidgets, created a very simple controller that
rendered a single file with ${asdf}. When ToscaWidgets is enabled,
the trackback instantly goes to the console trackback.

The related thread in the pylons-discuss group:

http://groups.google.com/group/pylons-discuss/t/ede16f830f41fa1

Since the configuration method changed slightly in 0.10rc1, I'm
wondering if the changes made to the config system may have affected
the wsgi_app configuration stage.

Here's the error prior to ToscaWidgets being installed:

Error !
NameError: Undefined

1 asdf
2 ${asdf}
3
/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-py2.5.egg/mako/
runtime.py, line 135:
raise NameError("Undefined")
/var/www/pylons/lib/python2.5/site-packages/WebHelpers-1.0b5-py2.5.egg/
webhelpers/html/builder.py, line 412:
return literal(cgi_escape(unicode(val), True))
/var/www/pylons/cp/cp/templates/test_index.mako, line 2:
${asdf}
/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-py2.5.egg/mako/
runtime.py, line 450:
callable_(context, *args, **kwargs)
/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-py2.5.egg/mako/
runtime.py, line 462:
result = template.error_handler(context, error)
/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-py2.5.egg/mako/
runtime.py, line 452:
_render_error(template, context, e)
/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-py2.5.egg/mako/
runtime.py, line 434:
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-py2.5.egg/mako/
runtime.py, line 403:
_render_context(template, callable_, context, *args,
**_kwargs_for_callable(callable_, data))
/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-py2.5.egg/mako/
template.py, line 198:
as_unicode=True)
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/templating.py, line 271:
return literal(template.render_unicode(**globs))
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/templating.py, line 249:
return render_func()
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/templating.py, line 274:
cache_type=cache_type, cache_expire=cache_expire)
/var/www/pylons/cp/cp/controllers/test.py, line 20:
return render('/test_index.mako')
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/controllers/core.py, line 60:
return func(**args)
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/controllers/core.py, line 108:
result = self._perform_call(func, args)
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/controllers/core.py, line 176:
response = self._inspect_call(func)
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/controllers/core.py, line 225:
response = self._dispatch_call()
/var/www/pylons/cp/cp/lib/base.py, line 18:
return WSGIController.__call__(self, environ, start_response)
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/wsgiapp.py, line 327:
return controller(environ, start_response)
/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-py2.5.egg/
pylons/wsgiapp.py, line 124:
response = self.dispatch(controller, environ, start_response)
/var/www/pylons/lib/python2.5/site-packages/Routes-1.12.1-py2.5.egg/
routes/middleware.py, line 131:
response = self.app(environ, start_response)
/var/www/pylons/lib/python2.5/site-packages/Beaker-1.5.3-py2.5.egg/
beaker/middleware.py, line 152:
return self.wrap_app(environ, session_start_response)
/var/www/pylons/lib/python2.5/site-packages/WebError-0.10.2-py2.5.egg/
weberror/evalexception.py, line 431:
app_iter = self.application(environ, detect_start_response)

Pylons version 0.10rc1

Here's the error from the console after ToscaWidgets is enabled by
adding the following lines to config/middleware.py

import tw.api as twa

# CUSTOM MIDDLEWARE HERE (filtered by error handling middlewares)
app = twa.make_middleware(app, {
'toscawidgets.framework': 'pylons',
'toscawidgets.framework.default_view': 'mako',
})

serving on 0.0.0.0:8080 view at http://127.0.0.1:8080
Debug at: http://pch1.mia.colo-cation.com:8080/_debug/view/1269285807
----------------------------------------
Exception happened during processing of request from ('72.153.151.78',
58225)
Traceback (most recent call last):
File "/var/www/pylons/lib/python2.5/site-packages/Paste-1.7.2-
py2.5.egg/paste/httpserver.py", line 1062, in
process_request_in_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.5/SocketServer.py", line 254, in
finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.5/SocketServer.py", line 522, in __init__
self.handle()
File "/var/www/pylons/lib/python2.5/site-packages/Paste-1.7.2-
py2.5.egg/paste/httpserver.py", line 436, in handle
BaseHTTPRequestHandler.handle(self)
File "/usr/lib/python2.5/BaseHTTPServer.py", line 316, in handle
self.handle_one_request()
File "/var/www/pylons/lib/python2.5/site-packages/Paste-1.7.2-
py2.5.egg/paste/httpserver.py", line 431, in handle_one_request
self.wsgi_execute()
File "/var/www/pylons/lib/python2.5/site-packages/Paste-1.7.2-
py2.5.egg/paste/httpserver.py", line 287, in wsgi_execute
self.wsgi_start_response)
File "/var/www/pylons/lib/python2.5/site-packages/Paste-1.7.2-
py2.5.egg/paste/cascade.py", line 130, in __call__
return self.apps[-1](environ, start_response)
File "/var/www/pylons/lib/python2.5/site-packages/Paste-1.7.2-
py2.5.egg/paste/registry.py", line 350, in __call__
app_iter = self.application(environ, start_response)
File "/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-
py2.5.egg/pylons/middleware.py", line 200, in __call__
self.app, environ, catch_exc_info=True)
File "/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-
py2.5.egg/pylons/util.py", line 91, in call_wsgi_application
app_iter = application(environ, start_response)
File "/var/www/pylons/lib/python2.5/site-packages/WebError-0.10.2-
py2.5.egg/weberror/evalexception.py", line 235, in __call__
return self.respond(environ, start_response)
File "/var/www/pylons/lib/python2.5/site-packages/WebError-0.10.2-
py2.5.egg/weberror/evalexception.py", line 483, in respond
return debug_info.content()
File "/var/www/pylons/lib/python2.5/site-packages/WebError-0.10.2-
py2.5.egg/weberror/evalexception.py", line 545, in content
result = tmpl_formatter(self.exc_value)
File "/var/www/pylons/lib/python2.5/site-packages/Pylons-0.10rc1-
py2.5.egg/pylons/error.py", line 43, in mako_html_data
css=False)
File "/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-
py2.5.egg/mako/template.py", line 189, in render
return runtime._render(self, self.callable_, args, data)
File "/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-
py2.5.egg/mako/runtime.py", line 403, in _render
_render_context(template, callable_, context, *args,
**_kwargs_for_callable(callable_, data))
File "/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-
py2.5.egg/mako/runtime.py", line 434, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File "/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-
py2.5.egg/mako/runtime.py", line 457, in _exec_template
callable_(context, *args, **kwargs)
File "memory:0xb66726ccL", line 54, in render_body
File "/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-
py2.5.egg/mako/exceptions.py", line 88, in __init__
self.records = self._init(traceback)
File "/var/www/pylons/lib/python2.5/site-packages/Mako-0.3.2-
py2.5.egg/mako/exceptions.py", line 166, in _init
line = line.decode('ascii', 'replace')
AttributeError: 'NoneType' object has no attribute 'decode'
----------------------------------------

So, it appears to be related to ToscaWidgets.

Beaker-1.5.3-py2.5.egg Pylons-0.10rc1-py2.5.egg
decorator-3.1.2-py2.5.egg Routes-1.12.1-py2.5.egg
easy-install.pth setuptools-0.6c11-py2.5.egg
Mako-0.3.2-py2.5.egg setuptools.pth
nose-0.11.3-py2.5.egg Tempita-0.4-py2.5.egg
Paste-1.7.2-py2.5.egg ToscaWidgets-0.9.9-py2.5.egg
PasteDeploy-1.3.3-py2.5.egg tw.forms-0.9.9-py2.5.egg
PasteScript-1.7.3-py2.5.egg WebError-0.10.2-py2.5.egg
pip-0.6.3-py2.5.egg WebHelpers-1.0b5-py2.5.egg
Pygments-1.3.1-py2.5.egg WebTest-1.2-py2.5.egg

[cd34]

This can be ignored, appears to have been addressed in:

http://toscawidgets.org/trac/tw/ticket/48

[Diez B. Roggisch]

> This can be ignored, appears to have been addressed in:
>
> http://toscawidgets.org/trac/tw/ticket/48

I see a few problems with this ticket/patch:

- there is no recipe on how to create a valid test for this, let
alone a test itself.
- the patch seems to be broken, as it relies on the following to be
True where in fact it's False:

>>> a = '0.9.7'
>>> b = '0.10rc1'
>>> a < b
False

Are you willing to put some effort into this? I don't use Pylons as
such, "just" TG2, so for me it's quite an effort to try & set things up.

Diez

[cd34]

Is the code that is executed called frequently? Perhaps something
like this would enable the version comparison to work as intended?

#!/usr/bin/python

import re

def versionmath(version):
version=re.sub(r'[a-zA-Z]+','.',version).split('.')
version.reverse()

result=0
for (power,val) in enumerate(version):
result+=pow(int(val),power+1)
return result

print versionmath('0.9.7') < versionmath('0.10rc1')
print versionmath('0.9.7') < versionmath('0.10.1')
print versionmath('0.9.7') < versionmath('0.9.6')
print versionmath('0.9.7') < versionmath('0.8.1')
print versionmath('0.9.7') < versionmath('0.0.0')

[Diez B. Roggisch]

On Monday 29 March 2010 18:08:06 cd34 wrote:
> Is the code that is executed called frequently? Perhaps something
> like this would enable the version comparison to work as intended?

Certainly very useful. The second problem I have is the Pylons-setup. There
are already examples for various systems, such as plain WSGI environs and the
like. Can you come up with one for pylons? Or write down how to create &
integrate TW into a pylons-app?

Then I'd be willing to adapt the patch & check it in.

Diez

[cd34]

Steps to install ToscaWidgets:

python go-pylons.py toscatest
cd toscatest
source bin/activate
easy_install -U toscawidgets (0.9.4 is installed by default)
easy_install -U tw.forms
paster create -t pylons tosca
(selected mako=True, sqlalchemy=False)

edit

config/middleware.py: (two changes)

import tw.api as twa

# CUSTOM MIDDLEWARE HERE (filtered by error handling middlewares)
app = twa.make_middleware(app, {
'toscawidgets.framework': 'pylons',
'toscawidgets.framework.default_view': 'mako',
})

resulting diff:

--- pylonshf.py.orig 2010-03-30 15:48:16.592840863 -0400
+++ pylonshf.py 2010-03-30 16:13:50.433492487 -0400
@@ -1,4 +1,5 @@
import logging
+import re
from decorator import decorator

from tw.api import retrieve_resources
@@ -8,12 +9,26 @@
import pylons
from pylons.util import AttribSafeContextObj, ContextObj
from pylons.i18n import ugettext
-from pylons.templating import render
+try:
+ from pylons.templating import render
+except ImportError:
+ from pylons.templating import render_mako as render
+
+def versionmath(version):
+ version=re.sub(r'[a-zA-Z]+','.',version).split('.')
+ version.reverse()
+ result=0
+ for (power,val) in enumerate(version):
+ result+=int(val)*pow(10,power+1)
+ return result

from formencode import Invalid

-__all__ = ["PylonsHostFramework", "validate", "render_response",
"render",
- "valid"]
+if versionmath(pylons.__version__) > versionmath('0.9.7'):
+ __all__ = ["PylonsHostFramework", "validate", "render", "valid"]
+else:
+ __all__ = ["PylonsHostFramework", "validate", "render_response",
"render",
+ "valid"]

log = logging.getLogger(__name__)

@@ -116,8 +131,15 @@
controller.form_result = {}
for field, validator in validators.iteritems():
try:
- controller.form_result[field] = \
- validator.to_python(decoded[field] or None,
state)
+ if versionmath(pylons.__version__) >
versionmath('0.9.7'):
+ controller.form_result[field] = \
+ validator.to_python(
+ field.decode() or None, state)
+ else:
+ controller.form_result[field] = \
+ validator.to_python(
+ decoded[field] or None, state)
+
except Invalid, error:
errors[field] = error
if errors:
@@ -139,8 +161,13 @@
if len(args) > 1 and args[0] in framework.engines:
framework.default_view = args[0]

- global_widgets = getattr(pylons.g, 'w', None)
- request_widgets = getattr(pylons.c, 'w', None)
+ if versionmath(pylons.__version__) > versionmath('0.9.7'):
+ global_widgets = getattr(pylons.app_globals, 'w', None)
+ request_widgets = getattr(pylons.tmpl_context, 'w', None)
+ else:
+ global_widgets = getattr(pylons.g, 'w', None)
+ request_widgets = getattr(pylons.c, 'w', None)
+
other_resources = kargs.pop('resources', None)
pylons.c.resources = retrieve_resources(
[global_widgets, request_widgets, other_resources]
@@ -163,5 +190,8 @@
wrapper.__dict__ = func.__dict__
return wrapper

-render_response =
_render_func_wrapper(pylons.templating.render_response)
-render = _render_func_wrapper(pylons.templating.render)
+if versionmath(pylons.__version__) > versionmath('0.9.7'):
+ render = _render_func_wrapper(pylons.templating.render_mako)
+else:
+ render_response =
_render_func_wrapper(pylons.templating.render_response)
+ render = _render_func_wrapper(pylons.templating.render)

Using the original patch allows paster/pylons to start and pages load
when using the 0.10rc1 on pypi. Using the original patch doesn't work
when using 1.0rc1. The modified patch allows Toscawidgets to
function, forms are generated properly, but, the original issue still
remains with ${asdf} resulting in a console trackback and 'Internal
Server Error' in the browser.

I wouldn't say this patch fixes anything, but, does allow ToscaWidgets
to at least function in 1.0rc1 (and 0.10rc1) and the conditionals are
at least executed properly.

[cd34]

Actually, slight correction in going back over my virtualenv.

The original patch does work with 1.0rc1. I didn't notice that I had
applied the patch incorrectly. The original patch does not work
properly with 0.10rc1 which the versionmath function fixes.

[Diez B. Roggisch]

Hi,

> Steps to install ToscaWidgets:
>
> python go-pylons.py toscatest
> cd toscatest
> source bin/activate
> easy_install -U toscawidgets (0.9.4 is installed by default)
> easy_install -U tw.forms
> paster create -t pylons tosca
> (selected mako=True, sqlalchemy=False)
>
> edit
>

I don't know where to get go-pylons - instead I did this:

- create virtualenv, activate it
- easy_install Pylons -> Pylons 0.10rc1 is installed
- easy_install decorator routes # they were missing apparently
- installed TW from my local working-copies
- quickstarted a pylons-tosca-app, as you show above
- started it
- it runs. With an unpatched TW. But it seems to only render a
static HTML-page, so that might
be part of the problem.

I don't know how to create a Pylons controller & stuff. How about you
provide a full example as tar.gz-file?

Diez

[cd34]

http://pylonshq.com/download/1.0rc1/go-pylons.py will download
1.0rc1. easy_install will download 0.10rc1. By using easy_install,
you have actually installed 0.10rc1 which includes the deprecation
warnings for buffet. The patch was written to handle 1.0rc1 where the
comparison would have worked, but, the easy_install version uses non-
decimal versions which makes math comparisons break.

http://cd34.com/tosca.tar.bz2

url to hit is /test/index

Contents of mako template:

asdfa
{asdf}
${tmpl_context.form(value=tmpl_context.value)|n}

forms work, validation works. If you replace {asdf} with ${asdf},
pylons gets an Internal Server Error and logs to the console. If
Toscawigdets is removed from the middleware, undefined tokens get the
Mako trackback.

--- packages as listed

Beaker-1.5.3-py2.5.egg Pylons-1.0rc1-py2.5.egg

[Diez B. Roggisch]

[cd34]

pylonshf.py was included in the .bz2

And yes, with either solution, the original bug remains. The patch
gets ToscaWidgets to function for forms, but, with 1.0rc1 or 0.10rc1,
undefined values in a .mako template result in a console trackback
rather than the mako trackback.

Two distinct issues.

Without the patch you've proposed (which works fine on both 0.10rc1
and 1.0rc1), ToscaWidgets doesn't allow Pylons to start. And that is
a much better method - I didn't know about pkg_resources. Actually,
ToscaWidgets does start with 0.10rc1 and works, but throws a lot of
deprecation warnings since it still called buffet.

With ToscaWidgets middleware working, the undefined value in the
template raises an exception which is the second issue. I'll repatch
pylonshf.diff and forward that over to Mike Orr to see if he has any
insights.

[cd34]

http://groups.google.com/group/mako-discuss/msg/f06184b0ee507c63

Fixes the trackback error. With the pylonshf.diff fix, ToscaWidgets
appears to work 100% for me in both 0.10rc1 and 1.0rc1

--
You received this message because you are subscribed to the Google Groups "ToscaWidgets-discuss" group.
To post to this group, send email to toscawidge...@googlegroups.com.
To unsubscribe from this group, send email to toscawidgets-dis...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/toscawidgets-discuss?hl=en.