Rundeck - observations/questions/requests

[Andrew Steady]

Hi,

We are using Rundeck quite heavily now and getting great results. I do have some observations which perhaps people can give me some tips on, or perhaps let me know if they are issues or sensible feature requests.

1. Parameters created or modified in a step of a multi-step job, are not available in their new/updated form in later steps. It would be nice if updated values of parameters defined and/or modified during a multi-step job were available in subsequent steps.

2. Sudo only for commands (not scripts) is limiting. In the end we have work arounds for this, the best one being a script job which "cats" the script we want to run as sudo into a tmp.sh, and then two subsequent sudo commands that chmod +x it, and execute it. Given that work-arounds are so easy, surely there is little security value in preventing a script being executed as sudo, especially as it makes rundeck less usable. Could we have a sudo script option perhaps with a project level enabling property with suitable warning?

3. Lack of file transfer capability. It would be brilliant if rundeck had jobs that would do scp like behaviour using the sshPassword and sudoPassword options passed - giving access to all files on the target server if the sudoPassword is passed.

4. For us, rundeck hangs when there is a lot of text output quickly from a job. To get around this we have to redirect the output to a temporary file. If the next step of the job simply cat's the contents of that file, again, it hangs. On the other hand my rundeck jobs can tail an active log file for a very long time with lots of output trickling in slowly, that's why I think the issue is to do with the speed of the output when using cat. I'm pretty sure the issue is only there or virtual servers. Does anyone else face this? It worst on freebsd.

5. We have a similar issue to (4) using wget. Wget commands (a) create multiple sub-processes and (b) create STDERR output as well as STDOUT. The net result is that the rundeck job never returns - again our solution is to redirect to a file and it works.

6. It would be nice to have a checkbox on a job to automatically pass all its parameters through to its child job steps.

7. It would be nice when defining a child job step, to have a UI in which to enter parameters for the sub-job. Having to type the command line in is error prone and the benefits of the GUI (validation, defaults etc.) are lost.

8. When you do a "run again" values of multi-value options (lists) are not pre-populated. Would be nice if they were.

9. We would like to use Rundeck for windows also. What are the options, and latest recommendation for that?

10. Conditional logic. Every now and then I wish I could call either one job or another based on the return value or a parameter set by a previous job. I suppose this would open a world of bpel type complexity, but interested if this has been considered or if people have light-weight ways of achieving this?

Finally many thanks to all owners and contributors to Rundeck!

Kind regards,

Andy

[Greg Schueler]

Hi Andrew,

On Mon, Mar 25, 2013 at 3:52 PM, Andrew Steady <steady...@gmail.com> wrote:

Hi,

We are using Rundeck quite heavily now and getting great results. I do have some observations which perhaps people can give me some tips on, or perhaps let me know if they are issues or sensible feature requests.

 

Great! Most of your items do fall under sensible feature requests to file, so do feel free to file any and all of them.  If they already exist I'll link to the issue below.

1. Parameters created or modified in a step of a multi-step job, are not available in their new/updated form in later steps. It would be nice if updated values of parameters defined and/or modified during a multi-step job were available in subsequent steps.

I think you are talking about a feature to allow steps to modify the data context (e.g. produce some data that you want a later step to use via normal rundeck property expansion.) which we talked about a bit on this thread: https://groups.google.com/d/topic/rundeck-discuss/oWipw3qbd4o/discussion

This is a big item on our roadmap and we have a plan for the implementation.  If you like I can talk more about the solution we are contemplating.

Allowing a step to override the user input options is certainly a feature we could enable in this mechanism. 

https://github.com/dtolabs/rundeck/issues/116

 

2. Sudo only for commands (not scripts) is limiting. In the end we have work arounds for this, the best one being a script job which "cats" the script we want to run as sudo into a tmp.sh, and then two subsequent sudo commands that chmod +x it, and execute it. Given that work-arounds are so easy, surely there is little security value in preventing a script being executed as sudo, especially as it makes rundeck less usable. Could we have a sudo script option perhaps with a project level enabling property with suitable warning?

 

We could add an "invocation" parameter for Script steps, which can let you define how the script would be invoked on the remote node.  At that point, the sudo password feature could be enabled for scripts by setting it to the sudo invocation you want, and configuring your node properly for sudo authentication.

This could be added for script files/URLs as well.

3. Lack of file transfer capability. It would be brilliant if rundeck had jobs that would do scp like behaviour using the sshPassword and sudoPassword options passed - giving access to all files on the target server if the sudoPassword is passed.

Luckily Rundeck 1.5 allows for new types of steps via plugins: this sounds like a candidate for a new plugin which provides SCP capabilities using the same ssh+sudo mechanism we already use for node execution.

 

4. For us, rundeck hangs when there is a lot of text output quickly from a job. To get around this we have to redirect the output to a temporary file. If the next step of the job simply cat's the contents of that file, again, it hangs. On the other hand my rundeck jobs can tail an active log file for a very long time with lots of output trickling in slowly, that's why I think the issue is to do with the speed of the output when using cat. I'm pretty sure the issue is only there or virtual servers. Does anyone else face this? It worst on freebsd.

I haven't seen this issue. Are you sure it's the amount of output, and not something like the weird character entity bug?  I would like to find out more about this.  Do you have an example of the log output which is causing the hang? Do you have more info about the type of "hang" that you see?  If you want to file a bug we can track the details there.

 

5. We have a similar issue to (4) using wget. Wget commands (a) create multiple sub-processes and (b) create STDERR output as well as STDOUT. The net result is that the rundeck job never returns - again our solution is to redirect to a file and it works.

Also sounds like a bug.

 

6. It would be nice to have a checkbox on a job to automatically pass all its parameters through to its child job steps.

Great idea for a feature request, it has come up before but I don't see it filed yet

 

7. It would be nice when defining a child job step, to have a UI in which to enter parameters for the sub-job. Having to type the command line in is error prone and the benefits of the GUI (validation, defaults etc.) are lost.

Yes, this is a failure of the UI, and has been requested before: https://github.com/dtolabs/rundeck/issues/252

 

8. When you do a "run again" values of multi-value options (lists) are not pre-populated. Would be nice if they were.

Ok, sounds like another bug, that should work!

 

9. We would like to use Rundeck for windows also. What are the options, and latest recommendation for that?

We know that Windows support in Rundeck could be much better. There is a WinRM node executor plugin which is essentially still in "beta" condition, but has been used successfully in some integrations. But please let us know where you would like to see Windows support improved.

 

10. Conditional logic. Every now and then I wish I could call either one job or another based on the return value or a parameter set by a previous job. I suppose this would open a world of bpel type complexity, but interested if this has been considered or if people have light-weight ways of achieving this?

This is another "flow control" feature that has been requested.  (In fact, if you go back into the source we had stubbed this idea out in the 1.0 version!)

It's possible to use the Error Handler feature to do some conditional logic around the result of a step: you can access the ${result.reason} and ${result.returnCode} values within an Error handler step (see http://rundeck.org/docs/manual/job-workflows.html#context-variables )

However, I think with the addition of the feature mentioned allowing steps to add data to the context, this will enable a more robust conditional system around workflow steps.

 

Finally many thanks to all owners and contributors to Rundeck!

Thank you for the ideas and suggestions, we love to hear how people use Rundeck and how we can make it better. 

Kind regards,

Andy

--
You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Adolfo Rodriguez]

Sorry if this is too basic but I am really missing some very very very simple examples, by using the GUI, of basic operations:

* execute a local command (ok this one),

* execute a remote command.

* copy a local file,

* copy a remote file,

* invoke from the GUI the dummy examples com.dtolabs.rundeck.plugin.example.ExampleNodeStepPlugin, com.dtolabs.rundeck.plugin.example.ExampleRemoteScriptNodeStepPlugin and com.dtolabs.rundeck.plugin.example.ExampleStepPlugin

Also, the option Local Command - Run a command locally on the server is has not description here. When I fill a command echo A, it complaints with:

	Time	Message
	03:19:28	Failed: IOFailure: java.io.IOException: Cannot run program "echo": CreateProcess error=2, The system cannot find the file specified

I have read the docs and played around many times and still I have big troubles getting started. It looks that you have to start creating config files and, from my point of view (more of a developer than an admin), is harder.

Thanks.

[Moses Lei]

Hi Adolfo,

"echo" as a command does not work, because echo is not a binary but a shell built-in. If you use a "Script" step (by first deleting the default command step, then adding a new step and selecting "Script") you can use "echo" because it will be passed to bash.

"cp" is a binary, so you can use it as a "command", but you can't use wildcards (because those are expanded by a shell). If you want to use wildcards, use a script.

To copy a script to a node, set up a script and then set your job up to execute on the node you want it to be copied to. RunDeck will copy the script to the node and execute it.

There's no default way to copy other kinds of files to target nodes-- it is assumed you will use any kind of method you have at your disposal that's available on the target node. "curl" or "wget" is generally the way people fetch arbitrary files. Another way to create arbitrary files on target nodes is to send a script over to the target node that cat<<HEREDOCs any kind of file you want.

Perhaps you can tell us what kind of automation problem you're trying to solve, and we can guide you further?

Moses

--
Moses Lei
[ Professional Services | DTO Solutions, Inc. ]
[ mobile: +1 703.901.5969 | e-mail: ml...@dtosolutions.com | aim/gtalk: ml...@controltier.com | yahoo: moseslei | windows live (msn): ml...@dtosolutions.com ]

--

[Adolfo Rodriguez]

An example of issue that would allow to get started (just to clarify). Plugin Descriptions says: To define a plugin that presents custom GUI configuration properties and/or uses Project/Framework level configuration, you need to provide a Description of your plugin.

ExampleStepPlugin already implements Descriptable and defines extra properties (bunny, Lampkin, Color, Many, Cramp, rice) for the GUI. 

* How do I configure a job step in the GUI invoking ExampleStepPlugin to see these extra properties and the dummy example working? 

* Is a Command/Script/Script file or URL/Local Command or Job Reference? 

* What exactly should I write in the input field? (com.dtolabs.rundeck.plugin.example.ExampleStepPlugin?)

* Does the extra props comes up automatically or do I have to do anything? 

I am really stuck on these questions.

Thanks

[Greg Schueler]

Hi Adolfo,

You will need to build and install the example plugin jar into the "libext" directory for Rundeck before it shows up in the GUI.

I see that the example-java-step-plugin/build.gradle was out of date, I've updated it here: https://github.com/dtolabs/rundeck/blob/development/examples/example-java-step-plugin/build.gradle

To build the plugin, run "./gradlew build".  Then copy the file `build/libs/example-java-step-plugin-1.5.jar` into the `libext` directory for your Rundeck installation.

Once installed, the plugin will show up in the list of available Step types when you create or modify a Workflow.  

You should see the "Example Step" available in the list of step types.  When you click on it, you will see the fields defined in the plugin as GUI text fields and controls.

--

[Adolfo Rodriguez]

Thanks Greg, I worked like a charm. I think this bit of practice would be very helpful somewhere in the manuals.

[Andrew Steady]

Hi,

Thanks I have created issues 343-346 for some of items above.

I need to spend some time to recreate the cat issue I talked about and you were interested in. I dug a little deeper and have a few observations - the issues is on freebsd, when using sudo - I think the issue is related to multiple processes being spawned by the script being executed (much like the wget issue) and perhaps rundeck can't handle multiple streams of output coming at it when in sudo mode.

Thanks,

Andy

--
You received this message because you are subscribed to a topic in the Google Groups "rundeck-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rundeck-discuss/yMhpDW8NnN8/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to rundeck-discu...@googlegroups.com.