Blogofile Plugins: Request For Comments.

[Ryan McGuire]

I've been thinking a lot about how to make upgrading Blogofile
smoother as well as make sharing the cool things that Blogofile users
have been creating easier.

I'm thinking about adding a plugin system to Blogofile, and I would
appreciate all of your thoughts on the matter.

Here's what I've got so far:

https://docs.google.com/document/d/1xiK7nRWYbUK1DqkiwsnjAx7gcdRgt0-YQOD64d3FHTE/edit?hl=en&authkey=CN6hk-8F

You can either respond here or make a copy of that document with your
own modifications.

Eventually, I would see this as becoming a Blogofile 0.8

Thanks!

[Mike Pirnat]

Interesting. Would definitely make it easier to distribute things
among the community.

In terms of the set of command-line tools, what about a "blogofile
plugin search" to go with "blogofile plugin available"?

--
Mike Pirnat
mpi...@gmail.com
http://www.pirnat.com/

[Doug Hellmann]

I love the idea of a formal plugin system, but have some feedback and questions about this specific proposal.

First, you seem to be defining a lot of requirements/features based on packaging and installation. Why not let Python's normal tools take care of those things? Defining an API for loading and using plugins, but leaving the structure of the source code up to the plugin developer, and the distribution up to normal channels, means fewer details for you to worry about and fewer new details for a plugin developer to have to figure out.

If you base plugins on the pkg_resources feature of setuptools/distribute/distutils2, that would let them be installed system-wide (or in a virtualenv) via pip or easy_install, so you don't have to maintain the manifest and installation data yourself. It would still allow them to be activated/deactivated on a per-user or per-site basis through a configuration file, so installation would not activate a plugin unconditionally.

Code can be loaded with pkg_resources.iter_entry_points(), and that code can be responsible for loading its templates. If blogofile needs to access the templates directly, part of the plugin API can define a function that returns information about them so they can be loaded with pkgutil.get_data() -- that way it doesn't matter where they end up on the filesystem.

I don't understand the idea of the remote plugins on an HTTP server. What is that feature for? Why not install the plugin on the system where you're running blogofile?

Rather than hosting the plugins on blogofile.com, I would ask the pypi.python.org folks for a Trove classifier for Blogofile plugins and then use the search API to get the list. Defining a command line wrapper for listing plugins may still make sense, but why build a new special server to host them when the Python community already has a registration system for packages: PyPI. As a potential plugin author, I would much rather use the tools I'm familiar with for defining and distributing non-Blogofile packages.

Using PyPI also means you don't need "blogofile plugin install" or "blogofile plugin upgrade" commands -- just use pip or easy_install. Alternately, those commands could be re-defined to work only on the plugins actually installed on the current system, and run configuration scripts packaged within the plugins themselves.

$0.02-ly,
Doug

[Ryan McGuire]

Hey Doug, 

I think you're on to something golden. This is exactly why I wanted to get feedback before I did any work. However, I do have some obstacles in my mind about your proposal that I don't know how to resolve. Most of them can likely be resolved (or compromised satisfactorily) with a bit more thought though:

1. Plugins for Blogofile are not necessarily Python source code. The design I've proposed is actually just mini-blogofile sites that layer on top of each other, so a Blogofile plugin could theoretically just be CSS files and images that compose a theme for Blgoofile, or they could just be a collection of templates and/or javascript. Now, obviously these can technically be wrapped in a Python module and installed with setuptools no problem and retrieved with pkgutil as you suggest, but would uploading such a module to PyPI be abusing it, considering it has no actual Python code in it? Is there any precedent for that? Now it's true that not every plugin needs to be in PyPI to work with easy_install, but then that makes the trove classifier idea kind of broken.
2. Is there a way for me to query blogofile plugins that are installed on a system without first having to know which modules to query for? I'd still like to be able to maintain the "blogofile plugin list" mechanism without having to require modules imported in _config.py first. To put this in other words: Can a python module do something upon installation to declare itself as a certain type of python module and then later have another script (blogofile) ask which modules of that type exist on the system? Obviously I could hack together some kind of a database myself that modules register themselves into upon installation, but that seems kludgy.
3. The blog templates at least will need to be easily modified by the user, if they're tucked away somewhere on sys.path, that's not going to work for some users. I guess the solution there is to have the templates relocatable somewhere in the user's site directory via a configuration setting on the plugin.
4. Sites depending on deployment through git post-receive hooks will need to have a smarter build system in place because the controllers are outside the repository now and may not be installed on the host. Not difficult, just something I don't have to deal with today.
5. I still think it's useful for Blogofile.com to host a list of plugins, this is so we can create a gallery of all the plugins, links to demo sites, screenshots, provide user ratings and comments etc. Like an app store of sorts. Plus this maintains a minimal amount of control in the bizarre case that someone were to upload something malicious to PyPI.

(BTW, I wasn't proposing any thing like remote plugins, I was merely envisioning a user being able to download a plugin from a remote HTTP server and install it and run it locally. much like "easy_install <URL>")

-Ryan

--
You received this message because you are subscribed to the Google Groups "blogofile-discuss" group.
To post to this group, send email to blogofil...@googlegroups.com.
To unsubscribe from this group, send email to blogofile-disc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/blogofile-discuss?hl=en.

[Doug Hellmann]

On Feb 7, 2011, at 7:42 PM, Ryan McGuire wrote:

Hey Doug, 

I think you're on to something golden. This is exactly why I wanted to get feedback before I did any work. However, I do have some obstacles in my mind about your proposal that I don't know how to resolve. Most of them can likely be resolved (or compromised satisfactorily) with a bit more thought though:

1. Plugins for Blogofile are not necessarily Python source code. The design I've proposed is actually just mini-blogofile sites that layer on top of each other, so a Blogofile plugin could theoretically just be CSS files and images that compose a theme for Blgoofile, or they could just be a collection of templates and/or javascript. Now, obviously these can technically be wrapped in a Python module and installed with setuptools no problem and retrieved with pkgutil as you suggest, but would uploading such a module to PyPI be abusing it, considering it has no actual Python code in it? Is there any precedent for that? Now it's true that not every plugin needs to be in PyPI to work with easy_install, but then that makes the trove classifier idea kind of broken.

That's a good question, and I'm not sure. Since it is a plugin for a Python project, I would think it would be OK. If not, you could still package them so they can be installed that way, and list them on the blogofile site for download. The packaging step could be mostly automated (to the point of generating the setup.py file, at least), so users who aren't familiar with the packaging tools could still benefit from them.

2. Is there a way for me to query blogofile plugins that are installed on a system without first having to know which modules to query for? I'd still like to be able to maintain the "blogofile plugin list" mechanism without having to require modules imported in _config.py first. To put this in other words: Can a python module do something upon installation to declare itself as a certain type of python module and then later have another script (blogofile) ask which modules of that type exist on the system? Obviously I could hack together some kind of a database myself that modules register themselves into upon installation, but that seems kludgy.

Yes, querying the installed plugins is a function of pkg_resources.iter_entry_points(). Each "entry point" is a Python object (usually a function or class) defined in a Python package. They are organized in to "groups" and each entry point has a name. You can run queries to find all of the items in a group, or just all of the items with a given name in a group (having a unique name isn't enforced at the loader level, but can be at the application level if it makes sense). So you could have a group called "blogofile.plugin" and ask for all of the members, and you would get back objects that knew the name of the plugin and could load it (import it) to get more details.

Unfortunately, the documentation for pkg_resources is verbose and completely lacking in examples. The run_hooks() function from virtualenvwrapper is about as simple an example as can be constructed, through: https://bitbucket.org/dhellmann/virtualenvwrapper/src/e538c6f84010/virtualenvwrapper/hook_loader.py#cl-119

3. The blog templates at least will need to be easily modified by the user, if they're tucked away somewhere on sys.path, that's not going to work for some users. I guess the solution there is to have the templates relocatable somewhere in the user's site directory via a configuration setting on the plugin.

Yes, installing templates and then modifying them is a different problem from just installing them. The usual way to handle that is to build an override mechanism into the template language. You could also have a special "install template files into my project" command line app that copies the files from the plugin into the right directory in a local project automatically.

4. Sites depending on deployment through git post-receive hooks will need to have a smarter build system in place because the controllers are outside the repository now and may not be installed on the host. Not difficult, just something I don't have to deal with today.

Hmm, true. A virtualenv seems like an appropriate solution for that. If a requirements file is part of the project directory structure, pip could be used to update the environment in one command.

5. I still think it's useful for Blogofile.com to host a list of plugins, this is so we can create a gallery of all the plugins, links to demo sites, screenshots, provide user ratings and comments etc. Like an app store of sorts. Plus this maintains a minimal amount of control in the bizarre case that someone were to upload something malicious to PyPI.

Sure, a gallery makes sense -- that's not easy to do with PyPI. My point was mainly to take advantage of the existing ecosystem and tools as much as possible, not that you shouldn't go beyond existing features.

(BTW, I wasn't proposing any thing like remote plugins, I was merely envisioning a user being able to download a plugin from a remote HTTP server and install it and run it locally. much like "easy_install <URL>")

Ah, OK, that does make more sense. :-)

Doug

[Ryan McGuire]

I've got a first draft of how I think Blogofile plugins will work. The
core blogofile code is in a "plugin" branch here:
https://github.com/EnigmaCurry/blogofile/tree/plugins

If you install that you can instantiate the simple_blog as usual:

blogofile init simple_blog

You'll notice that the source directory it creates no longer has any
_controllers directory! You can still create one if you like, but the
blog controller is now a plugin which you need to install seperately:

https://github.com/EnigmaCurry/blogofile_blog

Once you have the plugin installed you should be able to build it:

blogofile build

From there on out, it's Blogofile as usual. The code is a bit messy
right now, and things will be moving around a lot, but I wanted to
show you guys some good progress.

-Ryan

[OJ Reeves]

Given that for the most part people use Blogofile to run blogs, would it not be good to include the blog controller plugin by default for the simple_blog? Give people the ability to opt-out via a command line parameter or something?

Just thinking out loud, but manually installing a blog controller every time could be annoying.

Cheers!

OJ

--

You received this message because you are subscribed to the Google Groups "blogofile-discuss" group.
To post to this group, send email to blogofil...@googlegroups.com.
To unsubscribe from this group, send email to blogofile-disc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/blogofile-discuss?hl=en.

--

OJ Reeves
http://buffered.io/

[Ryan McGuire]

Yes, once it's ready, I'm planning on making blogofile depend on
blogofile_blog in it's setup.py. Then when you "easy_install
blogofile" you'll get the blogofile_blog plugin for free.

[OJ Reeves]

Groovey :) Thanks for that mate.

OJ