Giving more exposure to capistrano plugins

[Bruno Sutic]

Hi,

I'd like to start a discussion about giving more exposure to specific capistrano plugins and ways how I could get involved.

Here's a bit of the background which I think I share with a lot of capistrano users:

I'd like to setup my rails app deployment using the standard stack of postgres + unicorn + nginx. While at it, I'd like to do it with minimum config and boilerplate. And god forbid - I also do not want to ssh to the server for *any of the tasks*.

(premise here is that server if fully provisioned of course)

Now, I'm doing a research on capistrano unicorn plugins. I'm a "dumb user" and I want a plugin to handle unicorn scripts and configs for me - pretty standard stuff.

It turns out there is a bunch of plugins and their forks our there. It's totally messy, as you can see here:

https://github.com/sosedoff/capistrano-unicorn/issues/82

I'm thinking we really do not need all these plugins that all try to do one (simple?) thing.

Questions / suggestions:

- is it a good idea to give more highlight to specific plugins on the capistrano home page? Like for example here: http://capistranorb.com/documentation/frameworks/ruby-on-rails/

- or even make a step further and ask the plugin owners to move the plugins to github.com/capistrano?

That way capistrano community is committed and focused on improving small number of plugins.

Also, newcommers to capistrano are clearly pointed to what they should use and they do not have to waste time on investigation and trying out various plugins (like I did).

Involvement:

If the above suggestions are aligned with capistrano goals I would like to offer time to make this happen. Here are the tasks I'd like to do:

- do the work on investigating the best (working) capistrano plugins for unicorn, nginx and postgres

- submit pull requests (or just communicate to maintainers) so that chosen plugins are highlighted on this page: https://github.com/sosedoff/capistrano-unicorn/issues/82

Let me know how you like the idea!

Bruno

[Lee Hambley]

I'd like to start a discussion about giving more exposure to specific capistrano plugins and ways how I could get involved.

​Great!​

 

Here's a bit of the background which I think I share with a lot of capistrano users:

I'd like to setup my rails app deployment using the standard stack of postgres + unicorn + nginx. While at it, I'd like to do it with minimum config and boilerplate. And god forbid - I also do not want to ssh to the server for *any of the tasks*.

(premise here is that server if fully provisioned of course)

That's not too far off the mark!​

​​

Now, I'm doing a research on capistrano unicorn plugins. I'm a "dumb user" and I want a plugin to handle unicorn scripts and configs for me - pretty standard stuff.

It turns out there is a bunch of plugins and their forks our there. It's totally messy, as you can see here:

https://github.com/sosedoff/capistrano-unicorn/issues/82

I'm thinking we really do not need all these plugins that all try to do one (simple?) thing.

Questions / suggestions:

- is it a good idea to give more highlight to specific plugins on the capistrano home page? Like for example here: http://capistranorb.com/documentation/frameworks/ruby-on-rails/

​Absolutely, it's written as a Jekyll site ​so that plugin authors can send PRs to add themselves!

 

- or even make a step further and ask the plugin owners to move the plugins to github.com/capistrano?

​We already invited the plugin authors we came across (mostly due to v3 upgrade related questions and problems) to host their plugins with us, under the Capistrano name.​

 

That way capistrano community is committed and focused on improving small number of plugins.

Also, newcommers to capistrano are clearly pointed to what they should use and they do not have to waste time on investigation and trying out various plugins (like I did)

​.

That's a problem best solved by a plugin directory, or example videos or tutorials showing people how to get started with common stack http://roots.io/screencasts/ is a great example of that.

 

Involvement:

If the above suggestions are aligned with capistrano goals I would like to offer time to make this happen. Here are the tasks I'd like to do:

- do the work on investigating the best (working) capistrano plugins for unicorn, nginx and postgres

- submit pull requests (or just communicate to maintainers) so that chosen plugins are highlighted on this page: https://github.com/sosedoff/capistrano-unicorn/issues/82

​Absolutely, please feel free to add anything you think it useful to the documentation site, we're not precious about some arbitrary measure of purity, or worthiness for improvements, all improvements are gladly received.​ Perversely as the authors, we're the worst people to write beginner documentation!

 

Let me know how you like the idea!

​I do, it should be noted, that we'll be doing more around this area once we are launched with Harrow.io​, as common plugins/integrations will be important, and Harrow should include autodiscovery for common things.

[Bruno Sutic]

Hi Lee,

your reply is encouraging!

As I'll be working on this for my own needs, I'll try to contribute back in some way mentioned in the post above.

Thanks!

[Bruno Sutic]

Hi,

I'd like to provide an update to this incentive.

So, I've been investigating existing plugins for capistrano 2 (since my app was already working with it):

- capistrano-nginx-unicorn works great for nginx + unicorn

- I wanted to abstract postgresql management as well. Didn't find an existing plugin so I wrote my own capistrano2-postgresql.

With the above 2 plugins, deploying basic rails apps is a breeze. deploy.rb stays in ~30 lines and a user does not have to bother writing custom tasks. Most important of all: ssh-ing to the server wasn't necessary at all!

It would be great (and beneficial for others, especially newcommers) to have a similar setup for capistrano 3. Here's the progress on that:

- I wrote capistrano-postgresql cap3 plugin/gem that deals with postgresql. I was banging on it today and it seems to work ok.

- capistrano-nginx-unicorn support is on the way.

Suggestions:

- Lee, do you think it's a good idea we transfer capistrano-postgresql gem to capistrano/postgresql?

- after capistrano-nginx-unicorn is working for capistrano 3, can we do the same for that one?

I still have a couple of things to understand about cap 3, but after the above is done, I'd be glad to write an entry level tutorial for deploying rails apps with cap 3.

Let me know your feedback!

Bruno

[Lee Hambley]

> - Lee, do you think it's a good idea we transfer capistrano-postgresql gem to capistrano/postgresql?

Actually, this is a tricky one, if you're prepared to deal with it, absolutely. But the problem falls on the tricky line between infrastructure provisioning, and application deployment. For example I think I would resist officially supporting a capistrano plugin that installed apt packages according to some magic formula, but we support bundler out of the box. We support database migrations, but provisioning the database almost rubs me the wrong way.

​I'll allow it, but please forgive me if there's something else you write in the future, and my not-entirely-scientific method of deciding what to embrace, and not rejects something else.

- after capistrano-nginx-unicorn is working for capistrano 3, can we do the same for that one?

Yes, and in a perfect world, I might push you to test them when I get the capistrano test VM to a working state.

Lee Hambley

--

http://lee.hambley.name/

+49 (0) 170 298 5667

--
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/7d51990a-1b0a-4c01-97fe-179cb7e08172%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bruno Sutic]

Hi Lee,
okay, I understand!

Just to make sure we're on the same track: capistrano-postgresql does not actually *install* postgres package. It presumes postgres is installed/provisioned and:

1. creates db user and database for the app (is this database provisioning?) 2. creates appropriate database.yml

But yea, creating a database (task 1.) is in the grey zone I guess.

About testing - no problem. I presume, you don't mean manual testing, right?

If not manual, then what tool is used? Test-kitchen?

Let me know next steps for capistrano-postgresql.

Also, I'll update you when we have capistrano-nginx-unicorn working.

Thanks,

Bruno

[Lee Hambley]

Hey Bruno, replies inline:

Just to make sure we're on the same track: capistrano-postgresql does not actually *install* postgres package. It presumes postgres is installed/provisioned and:

​Yeah, I got it, except that "provisioned" is a huge grey area, if it's set up "properly' the user named in the application database.yml should have limited read/write to data, but without being able to make schema changes, or using GRANT. I know most people don't bother to set that up correctly unless they're in a strongly security audited environment, but the idea that the application user, can change schemas and grants scares me. 

 

1. creates db user and database for the app (is this database provisioning?) 2. creates appropriate database.yml

​Yeah, same as above, I don't know whether I expect database.yml to be checked in, synched from somewhere else or manage by a provisioning tool,  it depends a lot

 

But yea, creating a database (task 1.) is in the grey zone I guess.

​As are most things if we apply enough (too much?) analysis to them.​

 

About testing - no problem. I presume, you don't mean manual testing, right?

​No, we're a bit hung up because prior to the "omnibus" series of vagrant installations (when it was still a Gem) we used vagrant to script tests against virtual machines, vagrant has broken that use case now that it doesn't ship as a Gem anymore. The idea​

​is to provide a "base box' against which plugins can write integration tests, and we can be confident that nothing broke.​

 

If not manual, then what tool is used? Test-kitchen?

https://github.com/capistrano/capistrano/blob/master/Gemfile#L7

​ is what we use right now, but it's no good. Probably with the new vagrant, we'll use the ssh kit project local ./.ssh/config and mount the :deploy_to so that we can run operations, and natively assets on the filesystem state.

 

Let me know next steps for capistrano-postgresql.

Also, I'll update you when we have capistrano-nginx-unicorn working.

​Remind me, what's your GH username?​

 

To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/4e14255c-4576-4b41-9d8a-caf6a1a4bd0c%40googlegroups.com.

[Bruno Sutic]

Hi Lee,

my github username is "bruno-". Here's the profile, so there's no mistakes link

Also, thank you for letting me know about the potential issues with database user that can change schemas and grants.

If an app needs so strict environment I guess this plugin is not appropriate. I'll make sure to outline that in the README.

Thanks!

[Lee Hambley]

Hi Bruno,

All done, it's private for now, I'll help you review it before we make it public.

- Lee

Lee Hambley

--

http://lee.hambley.name/

+49 (0) 170 298 5667

To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/2ad34a72-0c6d-41d3-b890-18eba1188a71%40googlegroups.com.

[Bruno Sutic]

Okay great!

I've pushed the code.

Also, I made a not-so-big-refactor that introduces a lot of diff (moving code from one file to another).

If you want, we can use that pull request for comments and other updates.

Also, please let me know if you think we need to enforce some special git workflow, versioning or other rules..

Bruno

[Bruno Sutic]

Hi,

as promised a week or so, here's the update on the progress for capistrano-nginx-postgresql.

So we have the plugin working for capistrano 3.1 in the projects v0.1 branch.

I was manually testing it a lot over the weekend (for both rbenv, and rvm) and it seems to be fine.

There are a couple things, I don't fancy: it's also fiddling with logrotate support, there are some Cap2 leftover files in the project.

But since I don't have push privileges to the repo I'm leaving it for now.

So:

- is this plugin better aligned with capistrano non-provisioning policy? At it's core it manages 3 config files for nginx and unicorn.

- should we try to bring it to capistrano github namespace?
- if the above is yes, I'm split on the idea of collaborating with current project owner or just forking the project. I think the current project owner isn't really responsive and didn't work much with Capistrano 3 (even compared to me, and I started playing with it ~2weeks ago).

Bruno

[slowjack2k]

Hi Bruno,

what is meant by "non-provisioning policy" ?

Regards
Dieter