Pylons application maintenance

[Mike Orr]

Some cracks have started to appear in Pylons' documentation and
support scripts, so I thought I'd better mention them for people like
Jonathan that are maintaining Pylons programs or writing new ones.

The "go-pylons.py" script was lost in the transition from the old
Pylons sever to the Pyramid server. You can still find an old version
on Google, but it won't work because it tries to download the packages
from a directory that no longer exists on a server whose domain name
has changed. Fortunately, it's pretty easy to do the equivalent
manually:
1) Download "virtualenv" manually from PyPI.
2) Inside it you'll find a "virtualenv.py" script.
3) Run "virtualenv.py NEW_DIRECTORY_NAME". It will install a
virtual Python and Distribute.
4) Activate the virtualenv per the instructions on PyPI. ("source
NEW_DIRECTORY_NAME/bin/activate" on Unix, or run a .bat script on
Windows.)
5) Install Pylons: "pip install Pylons".
6) If you've already written your application, cd into it and
install it via "pip install -e ." . In this case, you can skip step 5
because it will automatically install Pylons as a dependency.
7) When you're finished using the virtualenv, deactivate it by
running "deactivate". (Maybe a .bat script on Windows?)

The links from some Pylons docs to third-party docs have broken. The
one I notice most is WebHelpers, which is my fault because I need to
square away its online docs and haven't yet. In the meantime the docs
are under a temporary URL
http://sluggo.scrapping.cc/python/WebHelpers/ .

If you're using the Pylons 1 series, you should be using Pylons 1.0.1,
which was released in August. It's updated for compatibility changes
in Paste, PasteDeploy, and WebOb 1.2. It also has several other small
fixes and enhancements that had accumulated since 1.0.

I'm also supporting Pylons 1 and -- eek! -- Pylons 0.9.7 applications,
because my organization has insufficient developer time to convert
them to Pyramid as fast as I'd wish. It's easier for me because I've
been with Pylons a long time so I remember when the changes were made.
But the fact remains that Pylons 1 is now 2 1/2 years old, its old
maintainers aren't focusing on it as much as they used to, and cracks
have started to appear. The cracks will doubtlessly get wider over
time. So it would be prudent to:

- Switch to Pyramid when feasable. It has better support and more
thorough documentation.

- Make a "requirements" file listing the versions of dependencies that
are known to work with your application. That way, if a dependency
makes an incompatible change in the future, you won't be stuck until
Pylons is patched. (Run "pip freeze >requirements.txt" to make a
requirements file, then optionally delete any lines that are not
critical to your application.)

- Set up a Pip "download cache", which will copy packages to a
designated directory as they are downloaded, and later install them
from that directory rather than downloading them again. That way,
you'll have a copy of all required packages in their compatible
versions. (Make a directory "~/.pip" containing a file "pip.conf" in
INI format, with an "[install]" section containing "download-cache
~/.pip/download_cache").

- Consider setting up a caching PyPI mirror. This will avoid the need
to go to the network to check current versions of packages when
installing. (I haven't done this part but there are blog articles
about it.)

--
Mike Orr <slugg...@gmail.com>

[Ben Bangert]

On Oct 14, 2012, at 11:20 AM, Mike Orr <slugg...@gmail.com> wrote:

> The "go-pylons.py" script was lost in the transition from the old
> Pylons sever to the Pyramid server. You can still find an old version
> on Google, but it won't work because it tries to download the packages
> from a directory that no longer exists on a server whose domain name
> has changed. Fortunately, it's pretty easy to do the equivalent
> manually:

Doh, the file probably still exists, but the links do not. Do you know what the link was offhand? I can take a look at ensuring it works.

> The links from some Pylons docs to third-party docs have broken. The
> one I notice most is WebHelpers, which is my fault because I need to
> square away its online docs and haven't yet. In the meantime the docs
> are under a temporary URL
> http://sluggo.scrapping.cc/python/WebHelpers/ .

Ah yes, that's unfortunately bound to happen over time, URL's are so fickle time-wise. Sigh.

> I'm also supporting Pylons 1 and -- eek! -- Pylons 0.9.7 applications,
> because my organization has insufficient developer time to convert
> them to Pyramid as fast as I'd wish. It's easier for me because I've
> been with Pylons a long time so I remember when the changes were made.
> But the fact remains that Pylons 1 is now 2 1/2 years old, its old
> maintainers aren't focusing on it as much as they used to, and cracks
> have started to appear. The cracks will doubtlessly get wider over
> time. So it would be prudent to:

I'm wondering if it might be prudent to do a 'last update' or something to freeze everything needed for Pylons under a sub-tree and mirror all docs, downloads, etc. indefinitely. This wouldn't be trivial, but hopefully would be more of a one-off type operation.

Reddit is still on Pylons, as are other large old projects that may never change, while Pylons is seeing no improvements, its worthwhile to remember that Beaker, WebOb, and WebTest have all continued to mature so components are still being maintained.

> - Set up a Pip "download cache", which will copy packages to a
> designated directory as they are downloaded, and later install them
> from that directory rather than downloading them again. That way,
> you'll have a copy of all required packages in their compatible
> versions. (Make a directory "~/.pip" containing a file "pip.conf" in
> INI format, with an "[install]" section containing "download-cache
> ~/.pip/download_cache").
>
> - Consider setting up a caching PyPI mirror. This will avoid the need
> to go to the network to check current versions of packages when
> installing. (I haven't done this part but there are blog articles
> about it.)

Some great suggestions, maybe we need to put this up on the site somewhere.

--
/////^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\\\\
|| Ben Bangert ||
|| b...@groovie.org ||
|| http://be.groovie.org/ ||
\\\\\_________________________________________________________/////

[Jonathan Vanasco]

thankfully, i'm nearly out of the deep , dark Pylons hole.  ( no offense, ben )   only one startup that i'm advising uses Pylons, my new company is on Pyramid, and all my consulting projects are now on the business or product side ( oh, my life is so much easier now! )

when helping that one company migrate from 0.9.6 -> 1.x there were a handful of problems.  

if anyone is doing a site fix, i would bring these up:

- constant issues with the Pylons book having better SEO.  Getting a header on the Pylons site noting "This is for Pylons 1.x, The Pylons Book 1.0 covers an earlier version and is incompatible" would be strongly suggested.  If someone can convince the Pylons book site to put up a header, that would solve a lot of potential problems too.

- There were several issues in package upgrades.  SqlAlchemy was generally fine, but Routes and WebHelpers were a pain -- at some points, both projects upgraded and broke compatibility with the Pylons docs/templates.  I think this happened 2x for WebHelpers.

- The migration path from 0.9.7 -> 0.9.7 -> .10 -> 1.0 -> 1.01 was a little dicey...

-- migrating to 0.9.7 from 0.9.6 required going through earlier release notes.  would have been nicer if they were on that page.  ( 0.9.6 was out for a long time )

-- the 0.10 warnings tag didn't seem to generate many warnings.    just letting everything break under 1.01 and fixing that was easier.

[Jonathan Vanasco]

Via another thread, I also remembered this issue...

Pylons' setup.py has this in install_requires:

WebOb>=1.1.1

If you do an easy_install/pip upgrade , it'll fetch the most recent WebOb -- which is in the 1.2 or 1.3 branch

Unfortunately, in WebOb 1.2a1 , there was this deprecation...

https://github.com/Pylons/webob/blob/master/docs/news.txt

1.2a1

---------

* Remove UnicodeMultiDict class; the result of ``Request.GET`` and

 ``Request.POST`` is now just a plain ``MultiDict``.

So you either need to :

a- remove UnicodeMultiDict from your code

or

b- revert to earlier webob `easy_install WebOb==1.1.1

IIRC, I also think I had an issue with routes upgrading too high during migration too.

[Mike Orr]

On Sat, Mar 2, 2013 at 11:57 AM, Ben Bangert <b...@groovie.org> wrote:
> On Oct 14, 2012, at 11:20 AM, Mike Orr <slugg...@gmail.com> wrote:
>
>> The "go-pylons.py" script was lost in the transition from the old
>> Pylons sever to the Pyramid server. You can still find an old version
>> on Google, but it won't work because it tries to download the packages
>> from a directory that no longer exists on a server whose domain name
>> has changed. Fortunately, it's pretty easy to do the equivalent
>> manually:
>
> Doh, the file probably still exists, but the links do not. Do you know what the link was offhand? I can take a look at ensuring it works.

It was something like http://pylonshq.com/downloads, but let's get
confirmation before spending time on it.

> I'm wondering if it might be prudent to do a 'last update' or something to freeze everything needed for Pylons under a sub-tree and mirror all docs, downloads, etc. indefinitely. This wouldn't be trivial, but hopefully would be more of a one-off type operation.
>
> Reddit is still on Pylons, as are other large old projects that may never change, while Pylons is seeing no improvements, its worthwhile to remember that Beaker, WebOb, and WebTest have all continued to mature so components are still being maintained.

I'm still supporting two Pylons 1 applications and three Pylons 0.9.7
applications. So yes, a "with all dependencies" distribution would be
helpful. Two of the applications I'm not exactly maintaining myself
but more advising the maintainers who don't know a lot about Pylons
and may they eventually be replaced by people who know even less about
it. I was able to help them when go-pylons.py broke or a WebOb version
became incompatible with Pylons. but if I'm not around then someday
they may not be able to get the program to run. I want to upgrade all
the applications to Pyramid, but time and budget constraints keep
pushing that to next year, then the year after that...

--
Mike Orr <slugg...@gmail.com>

[Mike Orr]

On Mon, Mar 11, 2013 at 9:06 AM, Jonathan Vanasco <jvan...@gmail.com> wrote:
>> I'm still supporting two Pylons 1 applications and three Pylons 0.9.7
>> applications. So yes, a "with all dependencies" distribution would be
>> helpful.
>
>
>

> In the case of things like WebOb and Routes, and others, it might make sense
> to at least drop the ">=" in setup.py/requirements.txt and replace with an
> '=='

Yes, that would be worth putting in a Pylons 1.0.2. We also need to
ensure compatible dependencies remain on PyPI. That may mean sending
an email to the maintainers telling them we're still using version X
and will need it "permanently".

--
Mike Orr <slugg...@gmail.com>

[Ben Bangert]

On Mar 10, 2013, at 4:04 PM, Mike Orr <slugg...@gmail.com> wrote:

> It was something like http://pylonshq.com/downloads, but let's get
> confirmation before spending time on it.

I have fixed the page:
http://pylonshq.com/download/

Now works again, the go-pylons scripts are in the appropriate directories. I notice there is no 1.0.1, I must not have made that when I released it recently, I'll look into populating it.

> I'm still supporting two Pylons 1 applications and three Pylons 0.9.7
> applications. So yes, a "with all dependencies" distribution would be
> helpful. Two of the applications I'm not exactly maintaining myself
> but more advising the maintainers who don't know a lot about Pylons
> and may they eventually be replaced by people who know even less about
> it. I was able to help them when go-pylons.py broke or a WebOb version
> became incompatible with Pylons. but if I'm not around then someday
> they may not be able to get the program to run. I want to upgrade all
> the applications to Pyramid, but time and budget constraints keep
> pushing that to next year, then the year after that...

I'll go ahead and make a 1.0.1 directory and assemble all the parts, that should help serve as a reference batch of dependencies to ease later installations.