Issues with script-exec and script-copy plugins

[James Fillman]

I'm getting a filename discrepancy between my script-copy and script-exec config which i can't figure out. 

Is the script-copy step supposed to rename the file to something more sane that the script-exec step uses? If so, what am I missing?

Here's my node config:

<node name="sandboxserver" hostname="sandboxserver" username="rdeck" script-copy-remote-filepath="/var/tmp/${file-copy.filename}" local-file-copier="script-copy" file-copier="script-copy" local-node-executor="script-exec" node-executor="script-exec" script-copy="/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:/var/tmp/${file-copy.filename}" script-exec="/usr/bin/sudo -u ${node.username} /usr/bin/ssh ${node.username}@${node.hostname} ${exec.command}" script-exec-dir="/var/tmp"/>

Here's the debug output from a job run:

[workflow] Begin step: 2,NodeDispatch
2: Workflow step executing: com.dtolabs.rundeck.execution.ExecutionItemFactory$1@56f90fd3
preparing for sequential execution on 1 nodes
Executing command on node: sandboxserver, NodeEntryImpl{tags=[], attributes={script-copy=/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:/var/tmp/${file-copy.filename}, tags=, username=rdeck, script-copy-remote-filepath=/var/tmp/${file-copy.filename}, node-executor=script-exec, hostname=sandboxserver, nodename=vacsld02sand, local-node-executor=script-exec, script-exec=/usr/bin/sudo -u ${node.username} /usr/bin/ssh ${node.username}@${node.hostname} ${exec.command}, file-copier=script-copy, script-exec-dir=/var/tmp, local-file-copier=script-copy}, project='null'}
[workflow] beginExecuteNodeStep(vacsld02sand): NodeDispatch: com.dtolabs.rundeck.execution.ExecutionItemFactory$1@56f90fd3
[script-copy] executing: /usr/bin/sudo -u rdeck /usr/bin/scp /var/lib/rundeck/var/tmp/dispatch2994228990672541014.tmp rdeck@sandboxserver:/var/tmp/dispatch2994228990672541014.tmp
[script-exec] executing: /usr/bin/sudo -u rdeck /usr/bin/ssh rdeck@sandboxserver chmod +x /var/tmp/38-32-vacsld02sand-dispatch-script.tmp.sh
chmod: cannot access `/var/tmp/38-32-sandboxserver-dispatch-script.tmp.sh': No such file or directory
[script-exec]: result code: 1, success: false
[script-exec]: result code: 1, success: false
Failed: NonZeroResultCode: Result code was 1
[workflow] finishExecuteNodeStep(vacsld02sand): NodeDispatch: NonZeroResultCode: Result code was 1
2: Workflow step finished, result: Dispatch failed on 1 nodes: [vacsld02sand: NonZeroResultCode: Result code was 1]
[workflow] Finish step: 2,NodeDispatch

Thanks in advance, gang.

James

[James Fillman]

To further add to this post,

What my goal is with all this is to configure rundeck so that all locally executed scripts or commands(locally meaning the rundeck server) are executed as a user unique to each project as defined in the project.properties files: project.ssh.user=<user1>

To accomplish this, i have to override the behavior of the local exec plugin. Can't quite seem to figure out the details for this. I can't even seem to remove the plugin. deleting the rundeck-localexec-plugin-2.6.9.jar plugin from libext AND exp/webapp/WEB-INF/rundeck/plugins didn't seem to accomplish anything.

I'm a little surprised that this isn't really addressed within Rundeck's security policy. I can think of countless use cases where executing local code in jobs is useful or even necessary. Giving users the ability to build jobs that run code locally as the same user that rundeck runs as is problematic.

my 2 cents, anyway.

Any help or advise would be greatly appreciated.


James

[Peter Garlic]

Hi James
at a first look seems that you forget something on configuration of script copier:

workflow] beginExecuteNodeStep(

vacsld02sand): NodeDispatch: com.dtolabs.rundeck.execution.ExecutionItemFactory$1@56f90fd3
[script-copy] executing: /usr/bin/sudo -u rdeck /usr/bin/scp /var/lib/rundeck/var/tmp/dispatch2994228990672541014.tmp rdeck@sandboxserver:/var/tmp/dispatch2994228990672541014.tmp
[script-exec] executing: /usr/bin/sudo -u rdeck /usr/bin/ssh rdeck@sandboxserver chmod +x /var/tmp/38-32-vacsld02sand-dispatch-script.tmp.sh
chmod: cannot access `/var/tmp/38-32-sandboxserver-dispatch-script.tmp.sh': No such file or directory

the script you copy have a name, but when rundeck try to rename the script it find a different file....

Can you post your your job configuration?
(Default Node Executor/Default File Copier)

-Peter

[James Fillman]

Hi Peter,

Thanks for replying to my post.

The example i'm providing is from a job that simply contains an 'inline script' step and an 'script file' step.

Here's the output of the job:

[workflow] Begin execution: rundeck-workflow-node-first context: null

preparing for sequential execution on 1 nodes

Executing command on node: devldap, NodeEntryImpl{tags=[], attributes={tags=, script-copy-remote-filepath=/var/tmp/${file-copy.filename}, hostname=devserver, file-copier=script-copy, script-exec-dir=/var/tmp, script-copy=/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:${file-copy.filename}, username=rdeck, nodename=devldap, node-executor=script-exec, local-node-executor=script-exec, scipt-exec-shell=bash -c, script-exec=/usr/bin/sudo -u ${node.username} /usr/bin/ssh ${node.username}@${node.hostname} -- ${exec.command} 2>/dev/null, local-file-copier=script-copy}, project='null'}
NodeSet: MultiNodeSelector{nodenames=[devldap]}
Workflow: com.dtolabs.rundeck.core.execution.workflow.StepFirstWorkflowStrategy$stepFirstWrapper@da1c7c5a
data context:  {node={tags=, os-version=, script-copy-remote-filepath=/var/tmp/${file-copy.filename}, hostname=devserver, os-name=, file-copier=script-copy, script-exec-dir=/var/tmp, os-family=, script-copy=/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:${file-copy.filename}, username=rdeck, description=, name=devldap, os-arch=, node-executor=script-exec, local-node-executor=script-exec, scipt-exec-shell=bash -c, script-exec=/usr/bin/sudo -u ${node.username} /usr/bin/ssh ${node.username}@${node.hostname} -- ${exec.command} 2>/dev/null, local-file-copier=script-copy}, globals={}, job={loglevel=DEBUG, wasRetry=false, url=http://vacsld02sand.oss.central1.com:4440/project/Test/execution/follow/61, id=8a4feb37-41c0-406b-adb3-f71ea1a40b07, project=Test, username=admin, retryAttempt=0, user.name=admin, name=Test 2, serverUUID=null, group=null, execid=61, serverUrl=http://vacsld02sand.oss.central1.com:4440/}, option={}}
[workflow] Begin step: 1,NodeDispatch
1: Workflow step executing: com.dtolabs.rundeck.execution.ExecutionItemFactory$1@28f55e2c

preparing for sequential execution on 1 nodes

Executing command on node: devldap, NodeEntryImpl{tags=[], attributes={tags=, script-copy-remote-filepath=/var/tmp/${file-copy.filename}, hostname=devserver, file-copier=script-copy, script-exec-dir=/var/tmp, script-copy=/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:${file-copy.filename}, username=rdeck, nodename=devldap, node-executor=script-exec, local-node-executor=script-exec, scipt-exec-shell=bash -c, script-exec=/usr/bin/sudo -u ${node.username} /usr/bin/ssh ${node.username}@${node.hostname} -- ${exec.command} 2>/dev/null, local-file-copier=script-copy}, project='null'}
[workflow] beginExecuteNodeStep(devldap): NodeDispatch: com.dtolabs.rundeck.execution.ExecutionItemFactory$1@28f55e2c
[script-copy] executing: /usr/bin/sudo -u rdeck /usr/bin/scp /var/lib/rundeck/var/tmp/dispatch6671378336492311984.tmp rdeck@devserver:dispatch6671378336492311984.tmp
[script-exec] executing: /usr/bin/sudo -u rdeck /usr/bin/ssh rdeck@devserver -- chmod +x /var/tmp/0-61-devldap-dispatch-script.tmp.sh 2>/dev/null

[script-exec]: result code: 1, success: false
[script-exec]: result code: 1, success: false
Failed: NonZeroResultCode: Result code was 1

[workflow] finishExecuteNodeStep(devldap): NodeDispatch: NonZeroResultCode: Result code was 1
1: Workflow step finished, result: Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1]
[workflow] Finish step: 1,NodeDispatch
[workflow] Begin step: 2,NodeDispatch
2: Workflow step executing: com.dtolabs.rundeck.execution.ExecutionItemFactory$2@4f9afb6

preparing for sequential execution on 1 nodes

Executing command on node: devldap, NodeEntryImpl{tags=[], attributes={tags=, script-copy-remote-filepath=/var/tmp/${file-copy.filename}, hostname=devserver, file-copier=script-copy, script-exec-dir=/var/tmp, script-copy=/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:${file-copy.filename}, username=rdeck, nodename=devldap, node-executor=script-exec, local-node-executor=script-exec, scipt-exec-shell=bash -c, script-exec=/usr/bin/sudo -u ${node.username} /usr/bin/ssh ${node.username}@${node.hostname} -- ${exec.command} 2>/dev/null, local-file-copier=script-copy}, project='null'}
[workflow] beginExecuteNodeStep(devldap): NodeDispatch: com.dtolabs.rundeck.execution.ExecutionItemFactory$2@4f9afb6
[script-copy] executing: /usr/bin/sudo -u rdeck /usr/bin/scp /home/rdeck/scripts/test.sh rdeck@devserver:test.sh
[script-exec] executing: /usr/bin/sudo -u rdeck /usr/bin/ssh rdeck@devserver -- chmod +x /var/tmp/1-61-devldap-test.sh 2>/dev/null

[script-exec]: result code: 1, success: false
[script-exec]: result code: 1, success: false
Failed: NonZeroResultCode: Result code was 1

[workflow] finishExecuteNodeStep(devldap): NodeDispatch: NonZeroResultCode: Result code was 1
2: Workflow step finished, result: Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1]
[workflow] Finish step: 2,NodeDispatch
[workflow] Finish execution:  rundeck-workflow-node-first: [Workflow result: , step failures: {1=Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1], 2=Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1]}, Node failures: {devldap=[NonZeroResultCode: Result code was 1, NonZeroResultCode: Result code was 1]}, status: failed]
[Workflow result: , step failures: {1=Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1], 2=Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1]}, Node failures: {devldap=[NonZeroResultCode: Result code was 1, NonZeroResultCode: Result code was 1]}, status: failed]
Execution failed: 61: [Workflow result: , step failures: {1=Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1], 2=Dispatch failed on 1 nodes: [devldap: NonZeroResultCode: Result code was 1]}, Node failures: {devldap=[NonZeroResultCode: Result code was 1, NonZeroResultCode: Result code was 1]}, status: failed]

And here's the job definition:

- description: ''
  executionEnabled: true
  id: 8a4feb37-41c0-406b-adb3-f71ea1a40b07
  loglevel: DEBUG
  name: Test 2
  nodefilters:
    dispatch:
      excludePrecedence: true
      keepgoing: false
      rankOrder: ascending
      threadcount: 1
    filter: devldap
  nodesSelectedByDefault: true
  notification:
    onfailure:
      email:
        attachLog: true
        recipients: m...@mycompany.com
        subject: Rundeck Job
  scheduleEnabled: true
  sequence:
    commands:
    - description: Execute inline script on remote node
      script: |-
        #!/bin/bash

        echo what user is this script running as...

        id
    - description: Execute script file on remote node
      scriptfile: /home/rdeck/scripts/test.sh
    keepgoing: true
    strategy: node-first
  uuid: 8a4feb37-41c0-406b-adb3-f71ea1a40b07

[Peter Garlic]

Hi James

I see always the same problem:
.....

[script-copy] executing: /usr/bin/sudo -u rdeck /usr/bin/scp /var/lib/rundeck/var/tmp/dispatch6671378336492311984.tmp rdeck@devserver:dispatch6671378336492311984.tmp

[script-exec] executing: /usr/bin/sudo -u rdeck /usr/bin/ssh rdeck@devserver -- chmod +x /var/tmp/0-61-devldap-dispatch-script.tmp.sh 2>/dev/null

......

you copy a file with a name and then you try to execute a different name.
For this reason I think that you have a configuration error somewhere.

Most probable error location is project configuration.
Can you upload it?

the job definition doesn´t help
-Peter

[James Fillman]

Hey Peter,

As i pointed out in my first post, the only related configuration is the node parameters(as far as i can see). Specifically the ${exec.command} seems to be the culprit.

 Here's my node config:

<node name="devldap" hostname="clxd01ldap.oss.central1.com" username="rdeck" script-copy-remote-filepath="/var/tmp/${file-copy.filename}" local-file-copier="script-copy" file-copier="script-copy" local-node-executor="script-exec" node-executor="script-exec" script-copy="/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:${file-copy.filename}" script-exec="/usr/bin/sudo -u ${node.username} /usr/bin/ssh ${node.username}@${node.hostname} -- ${exec.command} 2>/dev/null" scipt-exec-shell="bash -c" script-exec-dir="/var/tmp"/>

As you can see, it's the ${exec.command} var that changes the filename from 'dispatch....tmp' to '0-61-devldap-dispatch-script.tmp.sh'.

I've messed around with various versions of the 'script-copy-remote-filepath', 'script-copy', and 'script-exec' attributes. Given these vars to work with, ${file-copy.file}, ${file-copy.filename}, ${exec.command}, and the 'script-copy-remote-filepath' node attribute, I don't seem to be in control of what the resulting filename becomes that's used in the ${exec.command} var.


Here's my Test project config:

#Tue Nov 01 09:32:39 PDT 2016
#edit below
project.description=Test project
project.name=Test
project.nodeCache.delay=30
project.nodeCache.enabled=true
project.ssh-authentication=privateKey
project.ssh-key-storage-path=keys/rdeck/rdeck private key
project.ssh.user=rdeck
resources.source.1.config.file=/var/rundeck/projects/Test/etc/resources.xml
resources.source.1.config.format=resourcexml
resources.source.1.config.generateFileAutomatically=true
resources.source.1.config.includeServerNode=true
resources.source.1.config.requireFileExists=true
resources.source.1.type=file
service.FileCopier.default.provider=jsch-scp
service.NodeExecutor.default.provider=jsch-ssh

Here's my /etc/rundeck/project.properties:
project.dir = /var/rundeck/projects/${project.name}
project.etc.dir = /var/rundeck/projects/${project.name}/etc
project.resources.file = /var/rundeck/projects/${project.name}/etc/resources.xml
project.description =
project.organization =


I really appreciate the time you're taking to offer some assistance. I've found that this mailling list can be really shy on help and the IRC channel is completely useless.

James

[Peter Garlic]

Hi James
i don´t know wich version are you using (mine is stil on 2.6.7), but something looks different (which version are you using?):

• Default Node Executor

  The Node Executor is responsible for executing commands and scripts on remote nodes.
  Script Execution Delegates command execution to an external script. Can be configured project-wide or on a per-node basis.

  Command: /usr/bin/ssh -q -tt -F /var/lib/rundeck/.ssh/config ${node.username}@${node.hostname} -- ${exec.command}

• Default File Copier

  The File Copier is responsible for copying scripts as files to remote nodes before they are executed.
  Script Execution Delegates file copying to an external script. Can be configured project-wide or on a per-node basis.

  Command: /usr/bin/scp -q -F /var/lib/rundeck/.ssh/config ${file-copy.file} ${node.username}@${node.hostname}:${file-copy.destination} Remote Filepath: ${node.destdir}/${file-copy.filename}

1) I use this these command lines to execute/copy. the syntax seem quite different.

2) inside /etc/rundeck/framework properties I´ve added:

framework.file-copy-destination-dir = ~

This because I´ve some hardenized systems that doesn´t allow script execution from /tmp, /var/tmp and so on

3) your node config seems to have a typo "....scipt-exec-shell="bash -c" script-exec-dir="/var/tmp"/>......"

I hope this will help you.

-Peter

p.s..

If still down´t work post again more info with: rundeck version, /path/filename.ext of the configurations and the list of files on /var/lib/rundeck/libext.

[James Fillman]

I'm using the latest version of rundeck: 2.6.9

I fixed that typo you pointed out. I didn't help. I'm really at a loss. At this point, i may create a bug report.

There must be a property available that i need to use.

When i copy a file or run an inline script on a node which simply uses the built in ssh and scp plugin, i see this in the debug:

copying file: '/var/lib/rundeck/var/tmp/dispatch7317851860317814628.tmp' to: 'clxd01ldap:/var/tmp/4-63-clxd01ldap-dispatch-script.tmp.sh'
or
copying file: '/home/rdeck/scripts/test.sh' to: 'clxd01ldap:/var/tmp/5-63-clxd01ldap-test.sh'

Rundeck obviously knows the name it wants to give the file it's copying to the node.

If there was a property similar to ${file-copy.file} or ${file-copy.filename} that represents that filename, i could use it here:

<node name="devldap" ...script-copy="/usr/bin/sudo -u ${node.username} /usr/bin/scp ${file-copy.file} ${node.username}@${node.hostname}:${mystery_property}" .../>

James

[Peter Garlic]

Hi James
I´m really sorry. One last try

1) check your ssh user
sudo rdeck
scp anyfile rdeck@destinationhost
should work correctly

2)
Oper your rundeck project
On properties web page:
select simple configuration

Default Node executor
select * Script Execution
Command: /usr/bin/ssh -q -tt  ${node.username}@${node.hostname} -- ${exec.command}

Default node copier
select * Script execution
Command: /usr/bin/scp -q  ${file-copy.file} ${node.username}@${node.hostname}:${file-copy.destination}

Remote Filepath: ${node.destdir}/${file-copy.filename}

create a simple script
execute the script

If doesn´t work  open an issue

-Peter

p.s.
Maybe next week I Will install the version 2.6.9  on a local VM just to check it out, but I was waiting for incoming 2.6.10 release..

[James Fillman]

I've managed to get things working. The ${file-copy.destination} var was the key. I misunderstood what it represented so I was either using it incorrectly or not at all.

Thanks for all your assistance. You're last post was what put me on the right track.

cheers,
James

[Peter Garlic]

Great James

Welcome to Rundeck fellowship.
:D

-Peter