Repository for non-core modules: sketching goals and guidelines

[Brad Olson]

Last week I said I'd be willing to vet a repository for modules that
aren't in ansible's core library, but are still useful.

The concept is simple, but when I started to sketch out a draft
README.md at https://github.com/bradobro/ansible-extras, and realize
as a community we should discuss a few things, namely, and in this
order, I think:

1. Audience and Goals: who and what is for.
2. Name: what should we call it.
3. Guidelines: quality, submission, documentation, duplication. (How
can we keep it easy yet neat, usable, inviting?)

Background and my first thoughts to follow in replies.

[Brad Olson]

==Background==

Here's the tail of the formative discussion (a drift off Art Zemon's
prototype lineinfile module at
https://groups.google.com/group/ansible-project/browse_thread/thread/bdfa3d19fdc4ecc5/00628efab9dc40f0?hl=en&lnk=gst&q=art+zemon#00628efab9dc40f0)

> Michael, I'm wondering if you would be open to someone else
> maintaining a repo for useful modules that should never make core, but
> are worth collecting? It would be a bit like ansible-contrib was, but
> 1) you wouldn't have to hassle maintaining it, 2) it would have looser
> guidelines (though still expect some basic documentation and the
> ansible license statement) for inclusion. I'm willing to start that. I
> wouldn't want it to detract from core ansible development.
> --Brad


+100.
Some one else maintaining contrib would be great for me. We can
still keep it in the ansible repo and I can grant permissions.
Start
something and I'll fork it (so it shows up on the main ansible repo
list) and give whoever needs it access.
I kind of care that playbooks and modules included look presentable,
as they are the first thing people will see when deciding whether to
use the tool... but then, I don't have the bandwidth for that. Also
other CM tools don't really review them either and it seems to be
working out ok for them.
Great idea.
--Michael

[Michael DeHaan]

There are some older threads on this too...

1. Audience and Goals: who and what is for.

Everyone.    Scripts/playbooks/modules.

 

2. Name: what should we call it.

ansible-contrib

 

3. Guidelines: quality, submission, documentation, duplication. (How

can we keep it easy yet neat, usable, inviting?)

I think we can let this evolve as it goes.  Just be inclusive -- and things that end up causing people bad experiences can be removed if it becomes a problem.    Requiring a README

with author contact info + usage info is reasonable though.   I would hate for people to have to read code to figure it out -- that's not Ansible's way.

(1) In the repo at least, all modules and scripts should be in directories named like "appname_username" so it's clear who wrote what and there can be contrasting implementations.

How user X manages apache might be totally different from user Y.   I want to make sure everyone has a chance.   

(2)  Use git sub modules so individual authors can maintain their stuff without having to submit constant pull requests.   We don't want stuff going out of date, or having to manually manage keeping hundreds of various modules at level.   This is about the best way I think we can do it without actually needing some software infrastructure for this.   Someone should be able to submit a PR to get

their submodule added.    However we should make it really clear that the github tracker for the contrib project is NOT the place to get support or fix a module -- go to the original author.

[Brad Olson]

Here's my initial proposal to get the discussion started.

==Audience and Goals==

I think we'll all benefit if this collection of non-core modules is
first directed at those Ansible evaluators or new-adopters looking to
see what can be done and how to do it.

It's second audience will be more seasoned Ansible users looking for a
somewhat stable kit of tools others have found (or are likely to find)
repeatably helpful.

==Name==

There was a repository, ansible/ansible-contrib, that did something
like this. I'm okay with that name, but prefer ansible-extras or
ansible-tools or ansible-modules.

I had thought to call it ansible-island-of-unwanted-modules, but
people would have to watch Rudolph the Red Nosed Reindeer to
understand <g>.

==Guidelines==

We don't want the guidelines so strict it's painful to offer a module,
but for it to stay useful and help promote Ansible, I suggest:

1. Module Implementation: allow other languages common to major
distros, like sh, bash, perl, ruby. If in doubt, we put it to this
group or Michael.

2. Module Documentation: single line descriptions listed
alphabetically in a file, INDEX.md. Top-of-module comments describing
purpose, usage, prerequisites.

3. Module Code: clean, clear, maintainable.

4. Module Names: clear, non-clever, functional names, open to
tailoring by this group and Michael.

5. Module Duplication: no dups. This is huge for newcomers. It's one
thing that made it hard for me coming to Chef and Puppet--there are a
thousand ways to write a module to manage apache, for instance, but a
newcomer doesn't want to sort through the pro's and con's of 10
modules. Stuff like that definitely has a place--maybe ansible-
experiments or something--but working towards some conciseness in this
tool kit will help us and Ansible.

[Michael DeHaan]

There was a repository, ansible/ansible-contrib, that did something

like this. I'm okay with that name, but prefer ansible-extras or

ansible-tools or ansible-modules.

It needs to be inclusive of playbooks/scripts/and modules. 

I had thought to call it ansible-island-of-unwanted-modules, but

people would have to watch Rudolph the Red Nosed Reindeer to

understand <g>.

I find that movie kind of disturbing after you get past the funny part about dentistry.

 

We don't want the guidelines so strict it's painful to offer a module,

but for it to stay useful and help promote Ansible, I suggest:

1. Module Implementation: allow other languages common to major

distros, like sh, bash, perl, ruby. If in doubt, we put it to this

group or Michael.

allow anything is fine with me.

 

2. Module Documentation: single line descriptions listed

alphabetically in a file, INDEX.md. Top-of-module comments describing

purpose, usage, prerequisites.

+1 

3. Module Code: clean, clear, maintainable.

+1

 

4. Module Names: clear, non-clever, functional names, open to

tailoring by this group and Michael.

namespacing 

5. Module Duplication: no dups. This is huge for newcomers. It's one

thing that made it hard for me coming to Chef and Puppet--there are a

thousand ways to write a module to manage apache, for instance, but a

newcomer doesn't want to sort through the pro's and con's of 10

modules. Stuff like that definitely has a place--maybe ansible-

experiments or something--but working towards some conciseness in this

tool kit will help us and Ansible.

Must support dups.

Having alternates is huge to maintaining a community of good modules, and modules are frequently distro or OS specific.   

--Michael

[Brad Olson]

> > 4. Module Names: clear, non-clever, functional names, open to
> > tailoring by this group and Michael.
>
> namespacing

Clarify what you mean by that.

--Brad

[Michael DeHaan]

so basically it needs to have the name of each user in the module.

First off, you want to browse it by name, like looking for, say, squid

But then you also know that you like modules written by timmy,

so in the directory, name it like squid-timmy.

It would not be as useful to have to read each other's directories, like timmy/squid, because you couldn't see what was available, and you couldn't easily tell there were two squids if it was timmy-squid and jimmy-squid.

--Michael

[Brad Olson]

Michael,

I put together a draft ansible-contrib at https://github.com/bradobro/ansible-contrib,
borrowing from old ansible-contrib and incorporating what I had time
for of the discussions so far.

I think sub-modules is a good plan, but I'll need some guidance with
that. My experience with subrepos in other VCS is that they are great
but have lot's of pitfalls unique to each system.

Let me know what need's tweaking.

[Michael DeHaan]

--Michael

On Friday, May 25, 2012 at 4:28 PM, Brad Olson wrote:

> Michael,
>
> I put together a draft ansible-contrib at https://github.com/bradobro/ansible-contrib,
> borrowing from old ansible-contrib and incorporating what I had time
> for of the discussions so far.
>

Great!

Once this acquires some content I can see about forking it for the organization page.

Hint: let Brad know about your content :)

>
> I think sub-modules is a good plan, but I'll need some guidance with
> that. My experience with subrepos in other VCS is that they are great
> but have lot's of pitfalls unique to each system.

Don't know of any pitfalls -- anyway, I want to make sure everyone has freedom
to update their stuff whenever they want. That's the main point for that, so that individual
contributors can be responsive and it's not just a snapshot that requires pull requests to keep
up to date… that can be a serious pain, and submodules alleviate most all of that.

To get included, someone would just have to tell you where your repo was, and you wouldn't
have to keep following it.

http://help.github.com/submodules/

>
> Let me know what need's tweaking.

Looks good to me -- thanks for getting this started again.

[Brad Olson]

> Don't know of any pitfalls -- anyway, I want to make sure everyone has freedom
> to update their stuff whenever they want.  That's the main point for that, so that individual

I'll have to read up on git subrepos. My bias from other VCS's makes
me cautious about these things...maybe you can tell me Git doesn't
have those problems.

1. Submodules not updating when you want them to (Mercurial is
cautious about this, ensuring a stable snapshot).
2. Submodules updating when you don't want them to (Subversion can do
this, trying to make sure you get the latest).
3. Problems cloning if the submodule URL changes or disappears.

Like I said, I'll read up as soon as I have time, but if you can
dispell these concerns, I'd love it.

[Michael DeHaan]

They are updated whenever you run git submodule update. Comparing git to other VCSes is a not the best idea -- it's LEAGUES better than hg or svn.

The most important feature of this is anyone will be able to do a checkout of the contrib project and choose to update submodules and see the latest, even if the person manning the fort hasn't updated the repo on github.com/ansible/ansible in ages. It is also trivially easy to keep things up to date.

For a community project like this, "stable snapshot" is totally uninteresting.

"no gatekeepers" is the core principle here.

Maintaining copies of everyone's modules in your tree, but merged in so that they fit into your directory structure is a non-starter… why? Because not everyone wants to keep a copy of all their stuff mixed in with a bunch of stuff they might not care about. However, it's easy for everyone to maintain their own stuff, and it's also easy to add in content from people even if they didn't ask to have it linked -- because the contrib project becomes just an index into "the best of ansible github", more or less.

Submodules encourage maximum contribution and the ability for everyone to keep everything up to date ridiculously easy.

At risk of saying this too strongly -- This is a hard requirement for me in order to have any mention of submodules on the org page, to fork the project, or to mention it here. It's that important.

--Michael

[Brad Olson]

>
> Looks good to me -- thanks for getting this started again.

Welcome. Stuck in my own mercurial (was going to call it tricycle ;) )
module. I asked art zemon about splitting off his lininfile module.

I notice with github submodules, when you grab the repo as a zip, the
submodule directory is bare.

[Michael DeHaan]

Few people on github use the download as zip option. We can include instructions for how to clone the repo.

--Michael