ANN: Pylons 0.9.4 Released

[Ben Bangert]

It's with great pleasure that I announce the release of Pylons 0.9.4.
This release has quite a few bug fixes and enhancements, the most
since the 0.9 milestone. It's also likely one of the last big updates
before a 1.0 release candidate (there will be some small changes in
0.9.5 and possibly a 0.9.6).

First, the most important changes for those upgrading from an
existing Pylons application:

* WARNING: Removed the lang_extract and lang_compile commands. They used
pygettext.py and its associated msgfmt.py, which lacked the
ability to
extract ngettext style function calls and had issues with unicode
strings.
The new I18NToolBox project aims to provide this functionality
(and more)
via the gettext command line utilities. http://i18ntoolbox.ufsoft.org
* WARNING: Myghty's allow_globals config var has changed, causing the
following when running pre-compiled templates:
Error(TypeError): do_run_component() takes exactly 13 non-keyword
arguments (5 given)
Delete the compiled Myghty templates directory (specified by
cache_dir or
myghty_data_dir in the config file) to resolve the error.
* WARNING: The localization function '_' now uses ugettext (returns
unicode
strings) instead of gettext. To preserve the old behavior, append the
following line to your project's lib.base and lib.helpers imports:
from pylons.helpers import gettext as _
* WARNING: Removed 0.8.x legacy code and backwards compatibility
functions.

Please note that since some i18n functions have moved, your
helpers.py will need to be updated to import _, and ungettext from
pylons.i18n.

Also:

- The XMLRPC Controller got a significant update so that it now
provides the full range of XML-RPC Introspection facilities for your
service methods.
- SQLAlchemy convenience functions have been added to pylons.database
for use with the SessionContext plugin, and to create and retain SA
engines.
- Paste dependency was updated to 1.1.1, Routes to 1.6.1 (important
update for map.resource functionality)
- Pylons special objects (g, c, h, request, session) now available in
interactive debugger without _attach_globals.
- Controller actions can now be generators
- Pylons base WSGI app uses wsgi.org routing_args spec for easier
swapping of URL resolvers.

== Install ==

Please see http://pylonshq.com/docs/0.9.4/install for installation
details.

== Full Changelog ==

0.9.4 (Dec. 29th, 2006)
* WARNING: Removed the lang_extract and lang_compile commands. They used
pygettext.py and its associated msgfmt.py, which lacked the
ability to
extract ngettext style function calls and had issues with unicode
strings.
The new I18NToolBox project aims to provide this functionality
(and more)
via the gettext command line utilities. http://i18ntoolbox.ufsoft.org
* All Pylons special objects are now available within paster shell
(not just
h and g).
* WARNING: Myghty's allow_globals config var has changed, causing the
following when running pre-compiled templates:
Error(TypeError): do_run_component() takes exactly 13 non-keyword
arguments (5 given)
Delete the compiled Myghty templates directory (specified by
cache_dir or
myghty_data_dir in the config file) to resolve the error.
* Changed i18n functions in templates to use proxy objects so that using
set_lang in a template works right. Fixes #153.
* Now allowing any template plugin to overwrite global PYLONS_VARS
(such as c,
g), not just pylonsmyghty.
* Adding SQLAlchemy support to the database.py file. Saves the
session engine
to g to maintain it during the apps lifetime. Uses SessionContext
plugin for
management of the current session.
* Updated config object so that init_app can take an optional
template engine
argument to declare the default template engine.
* Updated Myghty plugin to use extra_vars_func when passed in.
* Fixed Buffet to use extra_vars_func properly.
* Fixed the validate decorator when there are validation errors and
variable_decode=True: now passing the original params to
htmlfill.render
instead of the varable_decode'd version. Patch by FlimFlamMan.
* Added ungettext function for use with pluralized i18n, and the N_
function
(gettext_noop) to mark global strings for translation. Added
ungettext, N_
and translator objects to be globals for templates. Refs #126.
* WARNING: The localization function '_' now uses ugettext (returns
unicode
strings) instead of gettext. To preserve the old behavior, append the
following line to your project's lib.base and lib.helpers imports:
from pylons.helpers import gettext as _
* Pylons special objects are now available within the interactive
debugger
(deprecating _attach_locals).
* Added setup-app run before unit tests run so that webapp has proper
setup
tasks handled. Fixes #113.
* Added paste.deploy.CONFIG setup to middleware.py, websetup.py and
testing
files in the Pylons project templates. Closes #112.
* Added security policy doc to index for use as Pylons security policy.
Closes #91.
* Improved the repr() of the c context object to show attributes.
* Set environ['paste.testing_variables'] whenever that key is
available, not
just in testing mode.
* Added capability to have an action be a generator function.
* Added introspection capability to XMLRPCController and signature
checking.
* Updated Controller to use additional arg lookup scheme so that the
source of
the function args for _inspect_call can be easily overridden.
* Updated RPCController, renamed to XMLRPCController.
XMLRPCController now
functions properly and will automatically return proper xmlrpc
responses.
* Added test configuration ini file to default template. Closes #114.
* Fixed problem with pylons.database.PackageHub.__get__ raising errors
other than AttributeError when the database isn't configured. Added
new UnconfiguredConnectionError exception, instead of just KeyError
or TypeError (depending on what part of the configuration failed).
* Fixed default g init, since bare object has no init method.
Reported by Ian
Bicking.
* Fixed issue with SQLObject method override having wrong name.
Reported by
climbus with patch. Fixes #133.
* Moved log function to pylons.helpers and translation functions to
pylons.i18n. using pylons.util purely for Pylons internal util
functions.
* WARNING: Removed 0.8.x legacy code and backwards compatibility
functions.
* PylonsApp now has option to not use Routes middleware, default
resolving
uses new wsgi.org routing_args spec.
* Refactored routes dispatching to use new Routes middleware.
* Fixed paster shell command to properly acquire mapper object without
relying on the template being configured in a specific manner.
* Added keyword argument pool_connection to
pylons.database.PackageHub; if set to false then SQLObject
connections
won't use pooled database connections (a new connection will be
opened for each request).

Many thanks to Phil Jenvey, Ian Bicking, James Gardner, and all the
other active members of the Pylons community!

Cheers,
Ben

[Jose Galvez]

Dear Ben
Any chance we could get a short example showing how to use the new
sqlalchemy features
Jose

[Ben Bangert]

On Dec 29, 2006, at 11:08 PM, Jose Galvez wrote:

> Any chance we could get a short example showing how to use the new
> sqlalchemy features

Sure, the two methods of interest are in pylons.database here:
http://pylonshq.com/docs/0.9.4/module-pylons.database.html#create_engine
http://pylonshq.com/docs/0.9.4/module-pylons.database.html#make_session

Since a SQLAlchemy engine is a pooled connection, only one of them
needs to be created for your application. The create_engine stores
the connection pool in your 'g' object.

I primarily use SQLAlchemy with the SessionContext plugin, which is
what the QuickWiki example uses. I also use the assign_mapper plugin
so that I don't need to use methods off the session directly, and can
just query my model classes like so:

c.violation = model.Violation.get_by(code=code)

I'm using the ORM capability of SA, if you're not, the create_engine
function should still come in handy.

So first, here's how I setup my models, it's pretty easy to deviate
from this as desired. I've put the SA metadata in its own module to
avoid circular import issues since the other models all need the
app's metadata object. It also makes it easier to cut-down on the
imports in the individual model modules. I rather like having each
model with its table in its own module as my ORM classes grow rather
large with various methods to retrieve different sets of data. If you
have more basic models, I'd suggest throwing them all in one module.

models/
__init__.py
metadata.py
restaurants.py
violations.py

# __init__.py
# import the ctx and meta here so its in the controllers under model.
from metadata import ctx, meta

from restaurants import Restaurant
from violations import Violation

# metadata.py
from sqlalchemy import *
from sqlalchemy.ext.assignmapper import assign_mapper
from pylons.database import create_engine, session_context

meta = MetaData()
ctx = session_context

# restaurants.py
from metadata import *

from inspections import Inspection, inspection_table,
inspectionviolations_table
from violations import Violation, violation_table

restaurant_table= Table('restaurants', meta,
Column('id', Integer, primary_key=True),
Column('name', String(200), nullable=False),
Column('address1', String, nullable=False),
...
Column('franchise_id', Integer, ForeignKey('franchises.id')),
)

class Restaurant(object):
def __init__(self, name, in_store=False, **kargs):
self.name = name
self.in_store = in_store
for k, v in kargs.iteritems():
if k in self.c.keys():
setattr(self, k, v)

assign_mapper(ctx, Restaurant, restaurant_table, properties=dict(
inspections=relation(Inspection, lazy=False, cascade="all,
delete-orphan",
order_by=desc(Inspection.c.date), backref="restaurant"),
)
)

# violations.py
from metadata import *

violation_table = Table('violations', meta,
Column('id', Integer, primary_key=True),
Column('code', String(10), nullable=False),
Column('description', String, nullable=False),
Column('correction', String),
Column('county', String),
)

class Violation(object):
def __init__(self, code, description, correction, county):
self.code = code
self.description = description
self.correction = correction
self.county = county

assign_mapper(ctx, Violation, violation_table)

So that's all the model files, with the assign_mapper plugin. This
works with the assumption that you define your SA connection in your
config ini file as 'sqlalchemy.dburi', also note that you can set
'sqlalchemy.echo = True' in your ini file to have all the SQL echoed
to the console while you're running the app (rather handy).

Several people noted on the mail list that a SQLAlchemy session is
persistent and needs to be cleared every request. Alternatively, you
can create a new SA session every request, which according to Mike
Bayer is slightly faster. This is what my BaseController looks like:

# add to the imports:
from pylons.database import make_session

class BaseController(WSGIController):
def __call__(self, environ, start_response):
c.model = model
model.ctx.current = make_session()
return WSGIController.__call__(self, environ, start_response)

At this point, you're ready to go with accessing your models while
your app is running. For example, here's one of my controllers:
class RestaurantController(BaseController):
def detail(self, id):
c.restaurant = model.Restaurant.get_by(id=id)
if not c.restaurant:
return render_response('/restaurant/notfound.html')
return render_response('/restaurant/detail.html')

For reference, here are the 2 SQLAlchemy plugins being used:
http://www.sqlalchemy.org/docs/plugins.myt#plugins_sessioncontext
http://www.sqlalchemy.org/docs/plugins.myt#plugins_assignmapper

Finally, you will want to create your tables during websetup.py.
Here's what my websetup.py looks like:
from sqlalchemy import *
import YOURPROJ.models as model
from paste.deploy import appconfig

def setup_config(command, filename, section, vars):
app_conf = appconfig('config:'+filename)
if not app_conf.has_key('sqlalchemy.dburi'):
raise KeyError("No sqlalchemy database config found!")
print "Connecting to database %s..."%repr(app_conf
['sqlalchemy.dburi'])
engine = create_engine(app_conf['sqlalchemy.dburi'])

model.meta.drop_all(engine)

print "Creating tables"
model.meta.create_all(engine)
print "Successfully setup"

You should be set from here on out.

Cheers,
Ben

[Jose Galvez]

Thanks Ben,
one more quick question below you mention that the SQLAlchemy session
is persistent and needs to be cleared every request, but that a new one
can be created instead. If a new one is created with each request, I'm
assuming that the old one is cleaned up, is that right? I guess I just
don't won't old sessions hanging about taking up memory
Jose

[Ben Bangert]

On Dec 30, 2006, at 12:07 AM, Jose Galvez wrote:

> one more quick question below you mention that the SQLAlchemy session
> is persistent and needs to be cleared every request, but that a
> new one
> can be created instead. If a new one is created with each request,
> I'm
> assuming that the old one is cleaned up, is that right? I guess I just
> don't won't old sessions hanging about taking up memory

Correct, they'll be deallocated and garbage collected by Python.

- Ben

[Robert Leftwich]

After upgrading, some of the links issue 301 status codes to exactly the same
url which causes the browser to report redirection limit errors or similar.
Others are working as expected. The common link seems to be that the links use
routes with '*url'

e.g. for the route:
map.connect('philosophy', 'philosophy/:subsection/*url', controller='content',
action='view', subsection='overview', section='philosophy', url='')

LiveHTTPHeaders reports:

http://localhost:8000/philosophy/overview/

GET /philosophy/overview/ HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.8) Gecko/20061115
Ubuntu/dapper-security Firefox/1.5.0.8
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Accept-Language: en-au,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: UTF-8,*
Keep-Alive: 300
Connection: keep-alive
Referer: http://localhost:8000/

HTTP/1.x 301 Moved Permanently
Server: PasteWSGIServer/0.5 Python/2.4.3
Date: Sat, 30 Dec 2006 23:46:20 GMT
Location: http://localhost:8000/philosophy/overview/
Content-Type: text/html
Connection: close

Anyone else seeing this?

Robert

[Philip Jenvey]

I don't know of anything in Pylons or Paste that sends 301s except
paste.urlparser. redirect_to sends 302s.

So I'm inclined to suggest trying Paste 1.0.1 to see if that affects
this problem (although it might not work completely with Pylons 0.9.4).

Does the 301 response include a body? Is this a Windows server?

--
Philip Jenvey

[Ian Bicking]

It's probably Paste. Is /philosophy/overview/ a directory served as
static files? Of course, without any particular support for index.html
there shouldn't be anything useful at that location...?

Anyway, there was an infinite redirect problem that I thought James
Gardner fixed for Paste 1.1.1.

--
Ian Bicking | ia...@colorstudy.com | http://blog.ianbicking.org

[Robert Leftwich]

Ian Bicking wrote:
>
> It's probably Paste. Is /philosophy/overview/ a directory served as
> static files? Of course, without any particular support for index.html
> there shouldn't be anything useful at that location...?

True.

IIRC this may be the result of a Routes issue that I've never gone back and fixed.

>
> Anyway, there was an infinite redirect problem that I thought James
> Gardner fixed for Paste 1.1.1.
>

It's definitely coming from StaticURLParser.add_slash() in Paste 1.1.1, but
nothing in my config sets up this particular path to use StaticURLParser.

Philip Jenvey wrote:
> Does the 301 response include a body? Is this a Windows server?

No and No.

Robert

[Ian Bicking]

Robert Leftwich wrote:
>> It's probably Paste. Is /philosophy/overview/ a directory served as
>> static files? Of course, without any particular support for
>> index.html there shouldn't be anything useful at that location...?
>
> True.
>
> IIRC this may be the result of a Routes issue that I've never gone back
> and fixed.
>
>>
>> Anyway, there was an infinite redirect problem that I thought James
>> Gardner fixed for Paste 1.1.1.
>>
>
> It's definitely coming from StaticURLParser.add_slash() in Paste 1.1.1,
> but nothing in my config sets up this particular path to use
> StaticURLParser.

Pylons automatically sends all requests that result in a 404 to the
public/ directory. If there's a public/philosophy/overview/ directory,
and no controller responding to public/philosophy, then it'll get passed
to StaticURLParser.

[Robert Leftwich]

I wrote:
> Philip Jenvey wrote:
>> Does the 301 response include a body? Is this a Windows server?
>
> No and No.
>

Actually, that should be yes and no.

LiveHTTPHeaders doesn't display the body, but using urlopen() returns 'The
resource has moved to http://localhost:8000/philosophy/overview/ - you should be
redirected automatically.', i.e. StaticURLParsers.add_slash() body.

Robert

[Robert Leftwich]

It looks like something is caching the 301 response. I've taken out all the
routes except

map.connect('/philosophy/overview', controller='home')

If I use urlopen() on http://localhost:####/philosophy/overview I get the 301
response (where #### is any port no.)

If I use urlopen() on any other url I get the expected 400 not found.

If I change the route to something different (even if only 1 character):

map.connect('/philosophy/verview', controller='home')

and then use urlopen() on that url I get the expected content from the 'home'
controller, but *still* get 301 on '/philosophy/overview'.

I'm only running 'paster serve dev.ini', no webservers or caches that I'm aware
of or can find.

Anyone have any clue why?

Robert

[Bruce Wang]

On 12/30/06, Ben Bangert <b...@groovie.org> wrote:

It's with great pleasure that I announce the release of Pylons 0.9.4.
This release has quite a few bug fixes and enhancements, the most
since the 0.9 milestone. It's also likely one of the last big updates
before a 1.0 release candidate (there will be some small changes in
0.9.5 and possibly a 0.9.6).

== Install ==

Please see http://pylonshq.com/docs/0.9.4/install for installation
details.

When will 0.9.4 update on Cheeseshop?
easy_install still report the 0.9.3 is the latest version

--
simple is good
http://brucewang.net
skype: number5

[Robert Leftwich]

I wrote:
>
> It looks like something is caching the 301 response.

Nope - it was my own stupidity!

A few months back there were a couple of different people creating content
off-site and they wanted to include images in the templates. In the interests of
expediency I let them use relative paths to the images as long as they were at
or below the current template directory. Then I added a StaticURLParser to my
middleware.py that pointed to the 'templates/docs' directory, which let the
images be accessed w/o requiring code changes or moving them around after
uploading from the off-site content creators (i.e. via webdav). This worked ok
until this latest Pylons update, which broke this big hack :-/ (although I'm not
sure what change actually caused the breakage).

Darn that expediency :-)

Robert

[Ben Bangert]

On Dec 30, 2006, at 8:40 PM, Bruce Wang wrote:

> When will 0.9.4 update on Cheeseshop?
> easy_install still report the 0.9.3 is the latest version

It should work right now, is easy_install -U Pylons, not working?
Notice you'll need -U to indicate you want to upgrade the package.

HTH,
Ben

[Bruce Wang]

I do add -U switch last time I checked.
But it's ok now anyway. Thanks!

BTW: Pylons is great, I'm using Pylons + Genshi + SqlAlchemy for my toy projects.