ANN: Pylons 0.9.6

[Ben Bangert]

Pylons 0.9.6 was released last night, after a mere 3 release candidates (thanks to the many who helped us work out the kinks in them). We're rather pleased with 0.9.6 and hope everyone else will be as well.

For those upgrading, please keep in mind that deprecation warnings are just that... warnings. Your application is not broken when you see them, and you should feel free to update your project to remedy them at your own pace. The easiest way to update your Pylons project is by following the directions on upgrading on the wiki here: 

http://wiki.pylonshq.com/display/pylonsdocs/Upgrading

Note that the backup option is recommended, then merge your changes into the new project files. In the future we're also looking at making it possible for 3-way merges on project templates which will mostly automate this task.

Additional Thanks

------------------------

- I'd really like to thank Brendan Arnold for kicking off the whole "people in a hurry" doc series. 

- Ches Martin did an excellent job updating both the QuickWiki docs *and* the QuickWiki Pylons project, thanks a bunch! 

- David Smith joined the Pylons developer team, and has been helping out here and there.

- Phil Jenvey for all his hard work

- Patch submissions from David Turner, David Smith, and others whose name escapes me.

- Bug reports from the active IRC channel, many thanks to all of you.

Changes

------------

Global Response object:

One of the bigger changes in 0.9.6 is the new way to return responses (Note that returning Response objects is still allowed). A global response object is now made available which can track cookies, headers, and the status code to return to clients, as well as content. This means your controller actions only need to return content, such as a string or generator. This 0.9.5 example:

return Response("Hello World")

Is now just:

return "Hello World"

The content will be merged into the global response object for you. This will make it easier to set cookies and headers in your controllers without worrying whether or not it will really make it into the response.

Uniform Config Access:

Getting to the configuration for your project during start-up, and during handling of a request was previously a little tough. It required different methods and helper functions to reliably find where the configuration was. This has been fixed in 0.9.6, and there is a single pylons.config object that is available immediately after your projects environment was loaded.

Logging:

There is now logging throughout the entire Pylons code-base. This makes it easier to track down how your request is routed, to which controller, and how it has handled it. For those wondering why a certain action was called instead of another one, what arguments were passed to it, etc. this information should really help out. Phil Jenvey has put up a nice document on how to configure logging output to Chainsaw which really makes the new logging shine:

http://wiki.pylonshq.com/display/pylonsdocs/Logging#lumberjacking-with-log4j-s-chainsaw

Default Template Language:

The default template language for Pylons 0.9.6 is now Mako. Myghty served us great, but provided Mike Bayer with a rather nasty code base filled with Perl port cruft to maintain that didn't have much of a future. Mako is an excellent update on Myghty, that provides a wonderfully fresh approach to Python templating as well as being the fastest Python templating language available.

Note that Myghty is still usable with Pylons, and people are free to continue using it as desired.

More Unicode:

The Pylons request object now returns unicode parameters by default. In 0.9.5 this was optional, while in 0.9.6 it is now on by default. Note that if you have code relying on request parameters being basic strings you will want to add the following line to your updated lib/environment.py module:

config['request_options']['charset'] = None

Or if you haven't updated your project template to 0.9.6, the following will disable unicode:

  request_settings = pylons.config.request_defaults.copy()

  request_settings['charset'] = None

  return pylons.config.Config(tmpl_options, map, paths,

                              request_settings=request_settings)

Templating Changes:

Template render statements beginning with a / mark will no longer have the dots replaced with slashes. This shouldn't affect most people using module style syntax as that generally implies you don't put a / at the front.

Database usage with SQLAlchemy:

There was quite a bit of debate about how to best handle using SQLAlchemy with Pylons. In the end however, it was decided that pylons.database is too limiting, and ties usage of SQLAlchemy to specific versions of Pylons in ways that don't benefit either project. For those using SQLAlchemy 0.3, the best choice is using SAContext and is documented in the In a Hurry docs:

http://wiki.pylonshq.com/display/pylonscookbook/SQLAlchemy+0.3+for+people+in+a+hurry

For those using SQLAlchemy 0.4, nothing besides 0.4 and Pylons 0.9.6 is needed, and using them together is documented here:

http://wiki.pylonshq.com/display/pylonscookbook/SQLAlchemy+0.4+for+people+in+a+hurry

Full Changelogs

----------------------

Of course, there's been tons of bug fixes and feature enhancements in Paste, PasteScript, Routes, Beaker, and WebHelpers as well. Too numerous to go into detail over, so here's the full changelog since the last Pylons for each project:

Pylons:

* Updated requirements for newer WebHelpers for SQLAlchemy 0.4 compatibility.

  Fixes #300.

* Fixed pylons.templating to not pull session objects if there are none in use

  for the request. Thanks Bob Ippolito.

* Catch UnicodeEncodeErrors when finding the Controller action method and fail

  gracefully. Thanks max. Fixes #298.

* Allow passing of a state keyword to the validate decorator for the

  to_python methods. Fixes #297.

* paster shell now configures logging from the config file, like paster serve

  and setup-app. This can be disabled via the -q option. Thanks Yannick

  Gingras.

* Fixed controllers.core to allow responses of None (empty bodies). Logs a

  message indicating the response was empty.

* pylons.helpers has been moved to pylons.controllers.util, to differentiate

  between controller utility functions and projects' helpers modules.

* Fixed non-basestring/generator/WSGIResponse objects returned from

  Controllers not being set as the response content. Thanks Alex Conrad.

* development.ini now configures the app's Logger level to DEBUG by default.

  Thanks Christoph Haas

* Projects now include a MANIFEST.in file: it directs distutils to recursively

  include all files in the project's public/ and templates/ dir. This fixes

  these dirs not being included in dists unless they were checked into an RCS

  recognized by setuptools. This is at the expense of dists now globbing all

  files in those dirs (even those not checked into your RCS). Thanks Christoph

  Haas.

* Fixed the validate decorator not setting c.form_errors in certain

  circumstances. Thanks max. Fixes #286.

* email_to lines commented out in development.ini and test.ini files to avoid

  emails being sent to a non-existent address by mistake. If an error occurs 

  it is logged but no email is sent unless email_to is specified.

* [paste.app_factory] entry points changed to point to the actual make_app() 

  function to make it simpler for someone to work out how Pylons works (tests 

  updated accordingly too).

* All use of the ez_setup module is now tested by an ImportError to make 

  Pylons compatible with Buildout. Note: Tags and releases should be made

  using an svn export and an svn add to ensure a real copy of the ez_setup

  module is included and not just an svn:external so that the module is tied

  to time of the release.

* More full-featured README.txt included.

* Updated beaker_cache to cache global response cookies/status/headers. 

  Fixes #280.

* Fixed missing abort name import in restrict rest decorator. Fixes #281.

* Added cheetah as a supported template language for template_engine option.

* Fixed public/ and templates/ directories not being created with paster

  create.

* Fixed cookie header addition to use add instead of append. Thanks to 

  anonymous patcher. Fixes #268, again.

* Added ability to pass _code option to specify the status code type for

  redirect_to.

* Fixed redirect_to to not copy all headers into redirect for old _response

  usage. Fixes #268.

* WARNING: By default, the Pylons request object now returns unicode

  parameter (pylons.GET/POST/params) values (and assumes those parameters

  were sent to Pylons as utf-8). Unicode parameters can cause major

  problems if your application is not setup to handle unicode. To disable

  unicode parameters (0.9.5 default behavior), add the following to your

  load_environment function (0.9.6 syntax):

  config['request_options']['charset'] = None

  or, if still using the deprecated pre-0.9.6 pylons.config syntax, add:

  request_settings = pylons.config.request_defaults.copy()

  request_settings['charset'] = None

  return pylons.config.Config(tmpl_options, map, paths,

                              request_settings=request_settings)

* WARNING: Template names beginning with a / (or the OS's path separator)

  will now result in the name not having the separator's replaced with '.'s

  for the template engine. This shouldn't affect most projects as they usually

  assume a dot notation will be used with dot notation template engines (Kid, 

  Genshi, etc.). This change allows template engines that can take filename

  paths to function properly. Fixes #233.

* WARNING: The pylons.util.get_prefix(environ) function is deprecated. Please

  use:

      environ.get('SCRIPT_NAME', '')

  instead (the get_prefix function is used in the default ErrorController).

  Fixes #243.

* WARNING: The paths dictionary's 'root_path' has moved to the less

  redundant 'root'.

* Fixed the Error Documents/EvalException css referencing non-existent images.

  Thanks Shannon -jj Behrens. Fixes #238.

* Added ability to pass _code option to specify the status code type for

  redirect_to.

* Fixed redirect_to to not copy all headers into redirect for old _response

  usage. Fixes #268.

* Added logging statements throughout Pylons code, added logging setup to

  default template. Fixes #98.

* Refactored global response to be setup in wsgiapp along with the other 

  globals. Updated WSGIController to copy in global response headers and

  cookies into a WSGI app's output.

* Added global pylons.response object. Thanks Shannon -jj Behrens and Damjan

  Georgievski. Fixes #268 and #201.

* Updated default project template files for new configuration layout. Options

  to handle config now just in environment.py, and middleware.py handling just

  middleware. Fixes #203.

* Removing mako tests, as its now the default. Default test changed from 

  Myghty to Mako.

* Changing default templating to mako.

* Added the https decorator. It requires an action to be loaded via

  https. Patch by ido. Fixes #241.

* Added upgrade instructions, and posted a copy on the wiki. Fixes #230.

* Added deprecation warnings for usage of the Pylons Controller class, all

  controllers should inherit from WSGIController instead. Fixes #239.

* Removed deprecated attach_locals function from Controller class.

* Added an authenticate_form decorator for use with WebHelpers'

  secure_form_tag functions for preventing CSRF attacks. Original patch

  by David Turner. Fixes #157.

* Fix Buffet's include_pylons_variables not being upheld. Thanks Jonathan

  LaCour.

* The validate decorator now accepts extra keyword arguments (**htmlfill_kwargs)

  to pass along to formencode's htmlfill.render function.

* Removed POST-only restriction on validate decorator, now handles GET 

  requests. No form arg required during a GET request, which will run the

  current action with c.form_errors set to the errors. Fixes #246.

* Added PylonsConfig, which gets accessed as pylons.config dict. Contains

  all the merged ini options, in addition to the Config options such as

  'routes.map', 'pylons.paths', 'buffet.template_options', etc. Check the

  pylons.config docs on PylonsConfig for dict keys populated by it.

* Split up resolution stages in wsgiapp, so that controller lookup is a

  separate function making it easier to subclass. PylonsApp now takes a 

  base_wsgi_app argument which is then used for the BaseWSGIApp instead of the

  one from wsgiapp.py.

* Added mako template render tests.

* Added storage of the action func used to handle a call, for later code that

  might need a reference to the action that originally handled the request.

  Fixes #253.

* Updated config object to optionally take a single merged conf dict, updated

  project templates to pass around the single merged conf dict.

* Changed project template to use new Beaker session keys.

* Changed default routing for raw template matching to not unicode decode the

  route argument. Fixes #242.

* Catch any exceptions raised by template engine entry points and emit a

  warning instead of crashing. Thanks markbradley. Fixes #249

* Fixed the validate decorator not working with formencode's

  CompoundValidators when variable_decode=False. Fixes #209.

* Fixed the validate decorator failing with a KeyError when no value is

  specified to validate against for separate validators (as opposed to a

  schema). Reported by Graham Stratton.

* Fixed paster shell not merging app_conf and global_conf into the main CONFIG

  dict namespace. Original patch by David Smith. Fixes #244.

* Added logging to decorators. Refs #98.

* Fixed paster restcontroller to test for lib.base and only add that import 

  statement when its present. This fixes the restcontroller template when used

  with minimal Pylons project templates. Fixes #237.

* Fixed the EvalException debugger showing broken links and buttons when the

  app's ErrorController was broken (such as when BaseController's __before__

  raises an exception). Suggested by Ian Bicking. Fixes #228.

* paster create now accepts a 'template_engine' option to setup the new

  project's default template engine. E.g. to create a new project that

  uses Genshi by default, use:

  paster create --template=pylons mygenshiproj template_engine=genshi

  Suggested by Ian Bicking. Fixes #141.

* Fixed the validate decorator triggering the following error with

  FormEncode>=0.7 and non-ascii rendered form content:

  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 10:

  ordinal not in range(128) the form was passed in as an encoded string, but

  some data or error messages were unicode strings; the form should be passed

  in as a unicode string

  Reported by Christoph Haas.

* HTTPExceptions are now converted to Response objects (for __after__),

  making the httpexceptions middleware no longer required.

* Added Warning to jsonify to warn about cross-site attacks when returning

  a list as the outer-most element to jsonify. Fixes #232.

* Fixed beaker_cache decorator to take optional keyword arguments intended

  for the backend cache container (such as url for memcached).

* Fixed paster controller assuming the minimal template was in use when

  the lib.base module existed but raised an exception.

* Fixed bug in XMLRPC Controller not setting proper Content-Type. Fixes #236.

* Added the '-d' ('--disable-ipython') option to paster shell for

  disabling IPython.

* Allow creation of controllers named 'setup' via paster controller.

  Reported by Matt Good.

* Added support for generic arguments to SQLAlchemy's create_engine of

  the form sqlalchemy.* from the PasteDeploy config file.

Routes:

* Fixed url_unquoting to only apply for strings.

* Added _encoding option to individual routes to toggle decoding/encoding on a

  per route basis.

* Fixed route matching so that '.' and other special chars are only part of the

  match should they not be followed by that character. Fixed regexp creation so

  that route parts with '.' in them aren't matched properly. Fixes #48.

* Fixed Unicode decoding/encoding so that the URL decoding and encoding can be

  set on the mapper with mapper.encoding. Fixes #40.

* Don't assume environ['CONTENT_TYPE'] always exists: it may be omitted

  according to the WSGI PEP.

* Fixed Unicode decode/encoding of path_info dynamic/wildcard parts so that

  PATH_INFO will stay a raw string as it should. Fixes #51.

* Fixed url_for (thus redirect_to) to throw an exception if a Unicode

  string is returned as that's an invalid URL. Fixes #46.

* Fixed Routes middleware to only parse POST's if the content type is

  application/x-www-form-urlencoded for a HTML form. This properly avoids

  parsing wsgi.input when it doesn't need to be.

PasteScript:

1.3.5

-----

* Initialize logging earlier in the serve command for components that want

  to utilize it. Patch from Christopher Lenz.

* Fixed Python 2.3 incompatibility (no ``string.Template``) in

  ``paste.script.appinstall``.

1.3.4

-----

* Make sure that when using ``--monitor`` or ``--reload``, if the

  parent monitoring process dies, also kill the subprocess.

* When using ``paster serve --log-file``, append to the log file (was

  truncating any previous contents).

* Read logging information from the server file, using the logging module's

  `standard configuration format

  <http://python.org/doc/current/lib/logging-config-fileformat.html>`_

* When adding files don't fail because an svn command fails.  Helpful

  particularly on Windows, when the svn command-line isn't installed

  (e.g., using TortoiseSVN).

WebHelpers:

0.3.2

* Added capability to pass pagination a SA 0.4 Session object which will be 

  used for queries. This allows compatibility with Session.mapper'd objects

  and normal SA 0.4 mapper relations.

* Updated SQLAlchemy ORM pagination for SA 0.4 Session.mapper objects.

* Updated Scriptaculous to 1.7.1 beta 3 (1.7.0 is incompatible with Prototype

  1.5.1). Thanks errcw. Fixes #288.

0.3.1

* Added the secure_form_tag helper module, for generating form tags

  including client-specific authorization tokens for preventing CSRF

  attacks. Original patch by David Turner. Fixes #157.

* current_url now accepts arguments to pass along to url_for. Fixes #251.

* Updated prototype to 1.5.1.1.

* Added image support to button_to. Patch by Alex Conrad. Fixes #184.

* Fix radio_button and submit_to_remote not handling unicode values.

  Fixes #235.

* Added support for the defer attribute to javascript_include_tag. Suggested by

  s0undt3ch. Fixes #214.

* Added a distutils command compress_resources, which can combine CSS

  and Javascript files, and compress Javascript via ShrinkSafe.  Add

  "command_packages=webhelpers.commands" in [global] in setup.cfg to

  enable this command for your package.

Cheers,

Ben