Rundeck File Copy and read data

[javed...@gmail.com]

  Hello,

This used to work prior to upgrade to rundeck-3.3.7.20201208-1.noarch (CentOS 7) but now it no longer does..

I am using Python winRM plugin 2.0.10 as my default file copier and node executor.

I am executing on Windows Server 2012. 

My host is identified by tags. 

My job has 2 steps: 

1. Copy file

2. Read file contents and do something with it (python.exe on remote note)

The read file contents part fails with error: 

[WinRMPython] execCommand started, command: C:\python37\python.exe C:\1457-254446-xxx-dispatch-script.tmp.py

[WinRMPython] executing: [[${config.interpreter}, -u, /var/lib/rundeck/libext/cache/py-winrm-plugin-2.0.10/winrm-exec.py, ${node.hostname}]]

[DEBUG ] Using username from node definition: script-runner (winrm-exec.py:171)[root]

[DEBUG ] Using password from node (winrm-exec.py:185)[root]

[DEBUG ] ------------------------------------------ (winrm-exec.py:200)[root]

[DEBUG ] endpoint:https://xxxxx (winrm-exec.py:201)[root]

[DEBUG ] authentication:basic (winrm-exec.py:202)[root]

[DEBUG ] username:script-runner (winrm-exec.py:203)[root]

[DEBUG ] nossl:True (winrm-exec.py:204)[root]

[DEBUG ] diabletls12:False (winrm-exec.py:205)[root]

[DEBUG ] krb5config:/etc/krb5.conf (winrm-exec.py:206)[root]

[DEBUG ] kinit command:kinit (winrm-exec.py:207)[root]

[DEBUG ] kerberos delegation:False (winrm-exec.py:208)[root]

[DEBUG ] shell:cmd (winrm-exec.py:209)[root]

[DEBUG ] readtimeout:None (winrm-exec.py:210)[root]

[DEBUG ] operationtimeout:None (winrm-exec.py:211)[root]

[DEBUG ] exit Behaviour:console (winrm-exec.py:212)[root]

[DEBUG ] ------------------------------------------ (winrm-exec.py:213)[root]

[DEBUG ] Starting new HTTPS connection (1): xxxx (connectionpool.py:943)[urllib3.connectionpool]

[WinRMPython]: result code: 1, success: false


But if I change my job definition to not read the file, just scan the directory where it's copied, the job succeeds. Why? Why is reading the file the problem and it returns exit code 1?

In the attached job definition, if you uncomment the read csv part it, rundeck will fail this job.

[rac...@rundeck.com]

Hi Javed,

Did you check all requirements and steps to configure your windows node? also, are you using python 3 on the Rundeck server?

I tested your job (bit a few modifications) against windows machine and it seems to work ok:

<joblist>
  <job>
    <context>
      <options preserveOrder='true'>
        <option name='mypath' value='C:\scripts'>
          <hidden>true</hidden>
        </option>
        <option name='Upload' type='file' />
      </options>
    </context>
    <defaultTab>output</defaultTab>
    <description></description>
    <dispatch>
      <excludePrecedence>true</excludePrecedence>
      <keepgoing>false</keepgoing>
      <rankOrder>ascending</rankOrder>
      <successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
      <threadcount>1</threadcount>
    </dispatch>
    <executionEnabled>true</executionEnabled>
    <id>6df7eb26-7f0f-46ed-bb60-cec9eb21b28b</id>
    <loglevel>INFO</loglevel>
    <name>UploadFile</name>
    <nodeFilterEditable>false</nodeFilterEditable>
    <nodefilters>
      <filter>name: windows</filter>
    </nodefilters>
    <nodesSelectedByDefault>true</nodesSelectedByDefault>
    <plugins />
    <scheduleEnabled>true</scheduleEnabled>
    <sequence keepgoing='false' strategy='node-first'>
      <command>
        <node-step-plugin type='copyfile'>
          <configuration>
            <entry key='destinationPath' value='${option.mypath}\${file.Upload.fileName}' />
            <entry key='echo' value='true' />
            <entry key='recursive' value='false' />
            <entry key='sourcePath' value='${file.Upload}' />
          </configuration>
        </node-step-plugin>
      </command>
      <command>
        <fileExtension>.py</fileExtension>
        <script><![CDATA[import os
import csv

with open('@option.mypath@\@file.Upload.fileName@', "r", encoding="utf-8") as in_file:
    reader = csv.reader(in_file, delimiter=',')
    for row in reader:
        print(row)]]></script>
        <scriptargs />
        <scriptinterpreter>C:\Python37\python.exe</scriptinterpreter>
      </command>
    </sequence>
    <uuid>6df7eb26-7f0f-46ed-bb60-cec9eb21b28b</uuid>
  </job>
</joblist>

Greetings!

​

[javed...@gmail.com]

Hello,

I am not sure why but it's not working for me. I tried your job.  The file copy succeeds but when it tries to execute the script, I see it tries to do this 2 times?

[WinRMPython] executing: [[${config.interpreter}, -u, /var/lib/rundeck/libext/cache/py-winrm-plugin-2.0.10/winrm-exec.py, ${node.hostname}]]

[WinRMPython]: result code: 1, success: false

[WinRMPython] execCommand started, command: del C:\Users\Osman\Code\scripts\1508-254647-O365AEH3-dispatch-script.tmp.py

[WinRMPython] executing: [[${config.interpreter}, -u, /var/lib/rundeck/libext/cache/py-winrm-plugin-2.0.10/winrm-exec.py, ${node.hostname}]]

[WinRMPython]: result code: 0, success: true 

Failed: NonZeroResultCode: [WinRMPython] Result code: 1

[workflow] finishExecuteNodeStep(O365AEH3): NodeDispatch: NonZeroResultCode: [WinRMPython] Result code: 1 

Could you share your file copier node + node executor versions?

My requirements are all satisfied as I have this rundeck setup running for about 3 years now. 

I am using python3. 

I have around 40 jobs that work just fine on the same node, but this is the only job that doesn't work and the difference here is to copy the file first and then perform work on it. 

--
You received this message because you are subscribed to a topic in the Google Groups "rundeck-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rundeck-discuss/JFJpK0Hc_0s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rundeck-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rundeck-discuss/ba7dbf5a-854a-42dd-a82f-cf8c49561e90n%40googlegroups.com.

[rac...@rundeck.com]

Hi Osman,

I'm using Python 3 on the node executor/file copier. Running the python code directly on the windows machine runs ok? (just to discard some dependency problem on the windows-side python environment).

Regards.

[javed...@gmail.com]

Yes, that is my current workaround to run the code directly on the server.

To view this discussion on the web visit https://groups.google.com/d/msgid/rundeck-discuss/9c3d52a7-dc50-40e3-b01a-5561f38d22e4n%40googlegroups.com.

[javed...@gmail.com]

My last successful job run was on the 2nd of Feb. 

Between then and now the only thing I did was upgrade rundeck from 3.3.1 to 3.3.7...

[javed...@gmail.com]

Can I get some help? 

New jobs created are also failing now.. with same exit code 1. 

[javed...@gmail.com]

I just realized that if I print something to screen then the job fails. It's just a simple print statement..

[rac...@rundeck.com]

Hi Osman,

To discard some upgrade problem, could you test stopping the Rundeck service, erase all plugin cache folder content (libext/cache), relaunch Rundeck and try again? Also, could you test with the last plugin version? (included with Rundeck 3.3.9, last at this moment).

Regards.

[javed...@gmail.com]

Hello,

Yeap, I tried that also but still fails. 

I tried the overthere-winrm plugin and it worked fine though. 

You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rundeck-discuss/4d90d5f0-e1d9-4c1b-9d1e-7a3b18e842b9n%40googlegroups.com.