Integrating Rundeck with Salt

[John Burbridge]

Greg and folks,

I'm currently working on extending Rundeck to be able to delegate executions to Salt, but am a bit stuck with a couple of points. 

For a little background, the goal is to have Rundeck send commands and target node definitions to a Salt master, which would then dispatch those to the actual nodes running the Salt minions. The sequence would be as follows:

I'm breaking up the work to be done in three parts:

1. Adding a delegated dispatcher/executor to Rundeck
   
2. Writing custom Salt returners to post ouput back to Rundeck
   
3. Adding a new method to the Rundeck API to accept ouput
   

I'm starting to tackle item #3 - getting Salt minions to write output back to Rundeck. Given that Rundeck already has an API end-point for fetching output from an execution, I think it would be the logical place to add the ability to post output back. 

To retrieve output:

GET http://rundeck:9090/rundeck/api/${apiVersion}/execution/${executionId}/output 

And now using the same endpoint to submit/append output:

PUT http://localhost:9090/rundeck/api/${apiVersion}/execution/${executionId}/output

{ payload: 'a string containing log output to be appended' }

So far I've managed to update the Grails UrlMappings and added a new controller / action which accepts a JSON request. No problems there. 

The issue is I am having a hard time figuring out how to hook in to logging. I've created a new apiExecutionAppendOutput closure in the ExecutionController and am able to get the execution, but that only gives me the "grails" execution record and not the "actual" execution instance. 

Any words of wisdom? Also, should I be approaching this differently?

Thanks,

JB

[Greg Schueler]

Hi John,

I have actually just pushed to the development branch a new (in-progress) feature for rundeck 1.5, which is Plugins for workflow steps.  You might want to look at the updated plugin development documentation, or ping me for more information about it. I think the way that Salt works it would make a natural fit for the "Workflow Step" plugin type, which allows you to implement one action for a set of nodes instead of having Rundeck do per-node actions.

I like your the idea of having a way for systems to post output back to the execution log.  I think with some minor modifications it should be doable as you envisage.  

You can access the Quartz Scheduler's running "Job" (i.e. quartz's job) which will be of the ExecutionJob class.  This object contains the runtime state of the execution, including a reference to the "loghandler" which handles writing all of the contextualized log data to disk.  Currently the ScheduledExecutionService handles interaction with the quartzScheduler (scheduling, interrupting, deleting), so you might add a method there to post log data.  The ExecutionJob would have to be modified a bit to expose the loghandler.

The logging mechanism uses a set of contextual data (node name, loglevel, timestamp, username, etc.) when it records a log entry, so perhaps the JSON input data should contain some of those fields as well.

[John Burbridge]

Greg, 

Thanks for the insights - much appreciated! I'll be sure to pass along all of the contextual data you mentioned - which implies those details needs to be sent out to the salt master as well so the minions know what they need to report back.  The workflow step plugin definitely sounds like what we're aiming for (one action for a set of nodes). I'll take a closer look at exposing the loghandler and poking a path through to it. 

By the way, what JDK are you using for development? I started getting some odd compile errors in eclipse after importing the new code drop though Gradle compiles just fine. 

Thanks,

JB

[chengkai liang]

Where I can grab 1.5 rundeck?

[Adolfo Rodriguez]

http://build.rundeck.org/job/rundeck-development/