A Proposal to Move 0.8.dev Forward

[Doug Latornell]

In lieu of tl;dr I’ll summarize for those with insufficient time to read what’s below. I’m going to take a stab at moving blogofile 0.8.dev forward in my Github fork [1]. Hopefully this will be considered to be a friendly fork. If I’m successful, I’ll work with Ryan to get my work merged into his repository, and perhaps get commit access to it so that I can maintain blogofile going forward. Please don’t expect overnight results; I have a day job and a life away from the keyboard, and blogofile isn’t my only side-project. I just really like it, and don’t like seeing it languish.

[1] https://github.com/douglatornell/blogofile

For details, please read on...

I’ve spent quite a bit of time over the past week or so reading the blogofile code, older posts in this group, and Ryan McQuire’s 0.8 preview blog post [2]. My reading was prompted by Mike Bayer’s “velocity?” post [3], Mike’s comment on my recent blog post about blogofile [4], and conversations I had with several people about blogofile at PyCon. The result is that I have a proposal regarding what I think I can do to move the 0.8.dev plugins branch to a state where it can be released.

[2] http://www.blogofile.com/blog/2011/04/30/preview-of-blogofile-0.8/
[3] https://groups.google.com/forum/?fromgroups&hl=en#!topic/blogofile-discuss/MG02xNwS8Lc
[4] http://douglatornell.ca/blog/2012/05/03/blogofile-improvements/

First though, a bit of background. I’m a latecomer to the blogofile party. I only started to use it at the very end of 2011 when I migrated my personal site to it and started a new blog on that site. I’ve only ever used the 0.8.dev plugins branch and the blogofile_blog plugin, and only ever under Python 3.2. So, until my reading this week, I was blissfully ignorant of the trials and tribulations that folks were having with 0.6 and 0.7 and upgrading, and whatever else.

The project seems pretty moribund at the moment, but some people in the Python community that I know and use blogofile and don’t want to see it fade into history. That, and the fact that blogofile is a perfect fit for the way I want to run my site and blog at present - reStructuredText markup, Mako templates, Disqus comments, and a statically generated site all powered by Python 2.7 or Python3 are a big truckload of win in my mind.

So, I propose to dive in and get blogofile moving again. Ryan wrote a list of stuff in his preview post that 0.8 would have:

• Python 3 support! Blogofile now runs on Python 2.6, 2.7, 3.1 or 3.2.
• Plugins.
• New template engines.
• blogofile build now works from any subdirectory in your source tree.
• Cross-python-version unit tests with Tox.
• Full stack testing with in-browser Selenium tests.
• HTML 5 Boilerplate enabled site templates.
• Command line completion and plugin commands.
• Blogger import script (contributed by Seth de l'Isle)
• Markdown extensions are enabled by user configuration.
• Static files can be configured to hard link rather than copy into the _site directory (Contributed by Nick Craig-Wood)

Here’s my take on those, in the order that I prioritize working on them:

Plugins
This is the core of what needs to get working so that 0.8 can be released. It looks to me like there is still lots of code that can be dropped from blogofile core, either completely, or by moving it into plugins like blogofile_blog. My goal is to get to a lean and solid static site builder core that can be augmented with controllers, filters, templates, and static content from plugins that are installed via pip or easy_install. There should be at least 1 reference plugin implementation like blogofile_blog. I think there also needs to be a well documented API for plugins so that plugin developers have something stable to work against, and the dependence on blogofile core internals that might change can be discouraged.

Testing
I’m test-infected, so improving the codebase means improving the test suite in parallel. I’ve used Selenium on other projects. For now I’m reserving judgement on whether it can add value to the blogofile test suite. Peripherally related to testing - I’m pretty sticky about following PEP8, so there’s a bunch of cleanup I want to do to make the red bars in my emacs buffers go away. As for Tox, read on...

Python 3 Support
Yep, definitely going there. But I’m not sure that the 3to2 route that Ryan started down over a year ago is the right way. I tried to build a Python 2.7 tarball of blogofile a few months ago, and the process failed (sorry, can’t recall the details). The setup.py file in the plugins branch, frankly, scares me. The impression I get from people that know more than I do about migrating from Python 2 to Python 3 is that the way to go is to write Python 2.6+ code that is forward looking and use the six library [5] to make it work in both Python 2.6+ and Python 3. As Mike pointed out [1], It looks like python-modernize [6] can help automate that process. At any rate, Tox is definitely the way to go to handle multi-version testing and packaging testing.

My experience with Python 2 to Python 3 migration is pretty limited, so I am definitely open to advice on this. And to get things started, I’ll ask: Does blogofile need to run under Python 3.0 or 3.1, or are 2.6, 2.7, 3.2 and 3.3 (coming soon) sufficient?

[5] http://packages.python.org/six/
[6] https://github.com/mitsuhiko/python-modernize

Command Line Completion
Meh. I don’t see this as a priority, especially since the argparse is in the standard library for 2.7 and 3.2, and the code comments say that the completion implementation relies on nasty monkey patching. Does blogofile really need to maintain an independent branch of argparse?

The Rest of Ryan’s List

• New template engines.
• blogofile build now works from any subdirectory in your source tree.
• HTML 5 Boilerplate enabled site templates.
• Blogger import script (contributed by Seth de l'Isle)
• Markdown extensions are enabled by user configuration.
• Static files can be configured to hard link rather than copy into the _site directory (Contributed by Nick Craig-Wood)

Sure, provided I don’t run into any unresolvable problems with these features and contributions, they’ll stay. I’ll also look at the open pull-requests for things that make sense to merge in.

As I wrote at the top, I plan to do this work in my Github fork (https://github.com/douglatornell/blogofile). Hopefully Ryan and the rest of the blogofile community will consider this to be a friendly fork. If I make decent progress I’ll work with Ryan to take over maintenance of the blogofile project. 

[Mike Pirnat]

I don't have time for a proper reply, but I figured that it's been
almost a day so I should at least say that I'm +1 in favor of this and
that I'd like to help how I can (which is sometimes not a lot due to
all the other directions that life wants to pull in).

--
Mike Pirnat
mpi...@gmail.com
http://mike.pirnat.com/

[Jason McKellar]

This sounds like a great idea to me. I use blogofile for a couple simple sites, so I haven't had a chance to look into the code very far other than to fix some syntax highlighting issues in my own site, but it would be nice to see development start again on Blogofile.

--Jason

[Ryan McGuire]

I gave Doug commit rights to the blogofile repos on github yesterday, I think he's got a good plan for moving forward. If there's anything I can do to help things along, let me know. 

Ryan

[Doug Latornell]

On Wednesday, May 16, 2012 8:00:25 PM UTC-7, Ryan McGuire wrote:

I gave Doug commit rights to the blogofile repos on github yesterday, I think he's got a good plan for moving forward. If there's anything I can do to help things along, let me know. 

Thanks for the commit access on the repos, and for the endorsement of my plan.

I'm not having very much luck getting the test suite in the plugins branch to work. I wonder if you have any recollections, suggestions, or magical incantations you can share. 

Running the tests via python setup.py test fails because 3to2 can't build the package. Initially 3to2 failed to refactor 2 of the tests in tests/browser/test_chrome.py, so I dropped those. Then it failed in setup.py, so I decided to set multi-version testing and tox aside for the moment and focus on getting the tests to run just under Python 3.2.

Using python -m unittest discover as the test runner, 17 out of 19 tests with errors. Typical traceback is https://gist.github.com/2728814

Oddly, using py.test as the test runner, I get different results, 16 failures, 3 passes, and 5 tests with errors, and the typical traceback is https://gist.github.com/2728853

I haven't tried nose as the test runner yet.

I suspect the problem in both cases is that the test aren't finding some of the fixtures, but I figured I might as well ask if you have any clues to offer before I bang my head against this too hard. I guess I can also ask the dumb question: was the plugins test suite passing for you when you left off work on it? I see several commented out tests with notes about them not working, so I presume the rest did, but maybe that's a bad assumption.

Any suggestions welcome...

Doug

[Doug Latornell]

On Wednesday, May 16, 2012 10:57:23 AM UTC-7, Mike Pirnat wrote:

I don't have time for a proper reply, but I figured that it's been
almost a day so I should at least say that I'm +1 in favor of this and
that I'd like to help how I can (which is sometimes not a lot due to
all the other directions that life wants to pull in).

Thanks for the support, Mike.

IIRC from our conversation at PyCon, you are running 0.7.1, correct? If so, perhaps when the time comes you can help me sort out and document the least troublesome migration path for folks to move from 0.7 to 0.8. Don't hold you breath waiting for the call though - I think I have some work ahead of me...

Doug 

[Mike Pirnat]

On Fri, May 18, 2012 at 11:34 PM, Doug Latornell <djlat...@gmail.com> wrote:
> IIRC from our conversation at PyCon, you are running 0.7.1, correct? If so,
> perhaps when the time comes you can help me sort out and document the least
> troublesome migration path for folks to move from 0.7 to 0.8. Don't hold you
> breath waiting for the call though - I think I have some work ahead of me...

Yes, I'm on 0.7.1. I'd be glad to help out with writing up the
migration path, as well as things like testing, cleanup, and fielding
miscellaneous issues.

[Karol Langner]

Hi,

Sorry to get in this so late, but I've been busy. I'm not a big user, but a big fan of blogofile, so +1 to this. Perhaps I'll switch to 0.8 soon (using 0.7 for now).

Cheers,

Karol