Howto: Extending Dradis with Rails Engines

[Stefan Streichsbier]

Hi everyone, 

I want to share a piece of information that might be useful for some of you. 

In order to boost efficiency when delivering security projects, we have written integrations for most of our process stack components interaction.

Dradis Pro is an important part of that stack, but unfortunately does not expose an API at this point. (Daniel told me that it is already on the roadmap.)

So in order to get our components to talk to Dradis, I was getting in touch with Daniel to see what the options are.

As it turns out, the way to go is writing a Rails Engine (http://guides.rubyonrails.org/engines.html).

I won't go into the details of the Rails Engine itself, but will rather explain the steps required to add an existing engine to the production environment. 

The following steps have to be taken.

1.) Modify the Gemfile and add your Rails Engine as a gem. In this case I use a private github repository.

Gemfile

  1 source 'http://rubygems.org'

  2 

  3 gem 'rails', '3.2.16'

  4 gem 'dradis_integration_api', :git => 'https://username@github.com/username/DradisIntegrationApi.git'

2.) Modify the Gemfile.lock by adding the github repository details as shown below in lines 1-6. The version number in between the parentheses is the version number described in the gemspec file and the revision hash corresponds commit hash (as listed at https://github.com/username/DradisIntegrationApi/commits/master).

Gemfile.lock 

  1 GIT

  2   remote: https://username@github.com/username/DradisIntegrationApi.git

  3   revision: 8acde25d25ec32d5d84a40e2c4241b1d9feaeb95

  4   specs:

  5     dradis_integration_api (0.0.2)

  6       rails

  7 

  8 GEM

...

200 DEPENDENCIES

...

209   dradis_integration_api!

...

3.) Mount the engine into the Dradispro application

config/routes.rb:

72  mount DradisIntegrationApi::Engine, :at => "/drapi"

73 end

4.) Run "bundle install" and "god restart dradispro-unicorn"

5.) Access the engine functionality at https://dradishost/pro/drapi

That's it folks,

Stefan

[Daniel Martin]

Hi Stefan,

First of all, thanks for sharing this with the list.

A couple of comments inline.

> So in order to get our components to talk to Dradis, I was getting in touch with Daniel to see what the options are.
> As it turns out, the way to go is writing a Rails Engine (http://guides.rubyonrails.org/engines.html).

This is excellent work. Engines are ‘the way forward’ as Stefan says, and anyone considering creating their own plugins should take this approach.

Ever since Stefan and his team started working on this I’ve been thinking about how to make this process simpler, and I think I have a couple of ideas that will be worth trying in the future. We’ll have an ‘Engine manager’ or 'Addon Manager’ in the CIC that will abstract most of the steps in Stefan’s process and present a nice interface to download, use and activate new addons. We can’t have this inside Dradis, because the way Rails works (i.e. all libraries and dependencies need to be loaded when the app starts). But having it in the CIC will be just as good for our purposes.

Then we’ll move our existing plugins to Engines so we can manage them from this new module. This means that if, for instance, you know you are never going to use the OpenVAS plugin, or the OSVDB one, you will be able to disable them, reducing the memory footprint of the app, which hopefully will improve performance as well.

So many great ideas and so little time to implement them!

Stay well,
Daniel

—
Founder
Security Roots Ltd.
Registered company no 07389856 in England and Wales

[Rory McCune]

Hi Daniel,

So on this is the idea that each plugin as it is now will become a separate engine?  I've not done much with them in the past, but looking at the documentation it looks like quite a heavyweight construct for some of the smaller plugins (e.g. something like a vulnerability database import).

If we were re-doing existing import/upload/export plugins do you have a note of what additional steps would be needed with the engine model to hook them into Dradis?

Also sounds we'd need some way of managing gemfile/gemfile.lock files?  At the moment does the upgrade process touch that side of things? It would be annoying to have them overwritten by an upgrade

Cheers

Rory

--

Founder
Security Roots Ltd.
Registered company no 07389856 in England and Wales

--
You received this message because you are subscribed to the Google Groups "Dradis Pro users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dradis-pro+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Martin]

Hi Rory,

> So on this is the idea that each plugin as it is now will become a separate engine? I've not done much with them in the past, but looking at the documentation it looks like quite a heavyweight construct for some of the smaller plugins (e.g. something like a vulnerability database import).

I don’t think they are necessarily more heavyweight. As far as I understand them in order to make a plugin into an engine, you just need to add a new file, which defines a class that inherits from Rails::Engine. That’s the bare minimum.

You can see an example of this here:

https://github.com/dradis/dradis-osvdb/blob/master/lib/dradis/plugins/osvdb/engine.rb

Which is basically the same OSVDB plugin structure with a new engine.rb file.

Of course you can get fancier than this and add your own controllers and models inside an engine, but I don’t think we’ll go that far just yet.

> If we were re-doing existing import/upload/export plugins do you have a note of what additional steps would be needed with the engine model to hook them into Dradis?

That’s one of the things I’ve been trying to think about on the Community project and the dradis3.x branch of things:

https://github.com/dradis/dradisframework/tree/dradis3.x

Right now, our import/upload/export plugins are very ‘free form’ (which is good), but 80% of the code of each import plugin is the same, and 80% of each upload plugin is the same. What I want to create is a series of Base classes or modules (like Dradis::Plugin::Export::Base, Dradis::Plugin::Import::Base, etc.) that will implement this 80% of the code for you and you’ll have to just focus on what’s specific to your plugin.

This is work in progress (mostly in my head at the moment) and there is nothing set in stone, but I’m thinking along those lines. I’ve only started doing a bit of work on the Upload front, for the code components that let the plugins interact with the Upload Manager. But I need to extend those support classes to the other plugin types and define a proper plugin architecture.

In any case, this isn’t going to happen overnight. And by the time we get there, you’ll have plenty of docs (yes, hold me accountable to this) and plenty of code for the existing plugins to look at.

> Also sounds we'd need some way of managing gemfile/gemfile.lock files? At the moment does the upgrade process touch that side of things? It would be annoying to have them overwritten by an upgrade

At the moment the upgrade process takes care of the custom plugins that you may have under your /opt/dradispro/dradispro/shared/plugins. The Addon Manager in the CIC would do something similar to that once we move on to the Engine-based architecture.

Again, I’ve been doing some research on the subject and there is not a whole lot of information about how to gem management from the user’s point of view, but in a nutshell, the way I think it will work is by having the Gemfile include a CIC-generated file. And having the Addon Manager do the necessary magic in this external file to make sure all the pieces fit together.

But again, this is all mid-to-long term to say the least, stuff I keep thinking about in my *spare* time, but nothing that I’m actively working at the moment.

For the time being a custom plugin as we traditionally had, or an Engine as described by Stefan in this thread should be a good enough solution for those wanting to do a bit of custom development.

HTH,
Daniel


—
Morris's three golden rules of computer security:
do not own a computer;
do not power it on;
and do not use one

[Rory McCune]

Hi Daniel, 

On Mon, Mar 24, 2014 at 10:41 PM, Daniel Martin <dan...@securityroots.com> wrote:

Hi Rory,

> So on this is the idea that each plugin as it is now will become a separate engine?  I've not done much with them in the past, but looking at the documentation it looks like quite a heavyweight construct for some of the smaller plugins (e.g. something like a vulnerability database import).

I don't think they are necessarily more heavyweight. As far as I understand them in order to make a plugin into an engine, you just need to add a new file, which defines a class that inherits from Rails::Engine. That's the bare minimum.

You can see an example of this here:

https://github.com/dradis/dradis-osvdb/blob/master/lib/dradis/plugins/osvdb/engine.rb

Which is basically the same OSVDB plugin structure with a new engine.rb file.

Of course you can get fancier than this and add your own controllers and models inside an engine, but I don't think we'll go that far just yet.

Ahh I see where I was getting confused.  Reading http://edgeguides.rubyonrails.org/engines.html it seemed to show that engines necessarily had full app directories (e.g. with controllers, models views etc) which seemed like overkill for a small plugin..

 

> If we were re-doing existing import/upload/export plugins do you have a note of what additional steps would be needed with the engine model to hook them into Dradis?

That's one of the things I've been trying to think about on the Community project and the dradis3.x branch of things:

https://github.com/dradis/dradisframework/tree/dradis3.x

Right now, our import/upload/export plugins are very 'free form' (which is good), but 80% of the code of each import plugin is the same, and 80% of each upload plugin is the same. What I want to create is a series of Base classes or modules (like Dradis::Plugin::Export::Base, Dradis::Plugin::Import::Base, etc.) that will implement this 80% of the code for you and you'll have to just focus on what's specific to your plugin.

This is work in progress (mostly in my head at the moment) and there is nothing set in stone, but I'm thinking along those lines. I've only started doing a bit of work on the Upload front, for the code components that let the plugins interact with the Upload Manager. But I need to extend those support classes to the other plugin types and define a proper plugin architecture.

In any case, this isn't going to happen overnight. And by the time we get there, you'll have plenty of docs (yes, hold me accountable to this) and plenty of code for the existing plugins to look at.

> Also sounds we'd need some way of managing gemfile/gemfile.lock files?  At the moment does the upgrade process touch that side of things? It would be annoying to have them overwritten by an upgrade

At the moment the upgrade process takes care of the custom plugins that you may have under your /opt/dradispro/dradispro/shared/plugins. The Addon Manager in the CIC would do something similar to that once we move on to the Engine-based architecture.

Again, I've been doing some research on the subject and there is not a whole lot of information about how to gem management from the user's point of view, but in a nutshell, the way I think it will work is by having the Gemfile include a CIC-generated file. And having the Addon Manager do the necessary magic in this external file to make sure all the pieces fit together.

But again, this is all mid-to-long term to say the least, stuff I keep thinking about in my *spare* time, but nothing that I'm actively working at the moment.

For the time being a custom plugin as we traditionally had, or an  Engine as described by Stefan in this thread should be a good enough solution for those wanting to do a bit of custom development.

One thing that might be of interest/use (although I've not read it yet) is that there's a new eBook about to come out about building ruby gems, which has a chapter on engines.  There's a discount code if you sign up to the mailing list but it only lasts a day (tomorrow) from what I understand so thought I'd mention it now... http://brandonhilkert.com/books/build-a-ruby-gem/ 

 

HTH,
Daniel

Cheers

Rory 

--

Morris's three golden rules of computer security:
do not own a computer;
do not power it on;
and do not use one