Rundeck 4.10 sshj-ssh timeout issue

[Russ Robinson]

Team,

  After trying to upgrade from 4.7 to 4.10; the linux nodes are hitting the following error after running for 5 minutes:

• net.sf.expectit.ExpectIOException: Expect operation fails (timeout: 30000000 ms) for matcher: regexp('~.*\$')
• at net.sf.expectit.ExpectImpl.expectIn(ExpectImpl.java:106)
• at net.sf.expectit.AbstractExpectImpl.expectIn(AbstractExpectImpl.java:57)
• at net.sf.expectit.AbstractExpectImpl.expect(AbstractExpectImpl.java:61)
• at com.plugin.sshjplugin.sudo.SudoCommand.runSudoCommand(SudoCommand.java:93)
• at com.plugin.sshjplugin.model.SSHJExec.execute(SSHJExec.java:96)
• at com.plugin.sshjplugin.SSHJNodeExecutorPlugin.executeCommand(SSHJNodeExecutorPlugin.java:250)
• at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeCommand(ExecutionServiceImpl.java:427)
• at com.dtolabs.rundeck.core.execution.ExecutionServiceImpl.executeCommand(ExecutionServiceImpl.java:390)
• at com.dtolabs.rundeck.core.execution.workflow.steps.node.impl.ExecNodeStepExecutor.executeNodeStep(ExecNodeStepExecutor.java:56)

My related project settings are:

• project.keep-alive-interval=5
• ...
• project.retry-counter=3
• project.retry-enable=true
• project.ssh-authentication=password
• project.ssh-command-timeout=0
• project.ssh-connect-timeout=0
• project.ssh-keypath=/var/lib/rundeck/.ssh/id_rsa
• project.ssh.user=${option.username}
• project.sudo-command-enabled=true
• ...
• service.FileCopier.default.provider=sshj-scp
• service.NodeExecutor.default.provider=sshj-ssh

The commands being ran are via sudo.  What am I overlooking on this?

[Russ Robinson]

I can reliably re-produce this error by setting up a job to run the following inline script on remote servers:

#!/bin/bash
my_start_date=`date`
my_max=10000
my_curr_try=0
my_sleep=480

while [ $my_curr_try -lt $my_max ]
do
  my_curr_date=`date`
  (( my_curr_try = my_curr_try + 1 ))
  echo "Start: ${my_start_date} Curr: ${my_curr_date} Try: ${my_curr_try} of ${my_max}"
  echo "Sleeping ${my_sleep}"
  sleep ${my_sleep}
done

Using invocation string: sudo su - root

File Extension: .sh

[rac...@rundeck.com]

Hi Russ,

Probably is related to the latest SSHJ bug reported here by you, could you try using the 4.7.0 sshj jar plugin like this?

Please open a new issue on the SSHJ github space.

Regards.

[Russ Robinson]

New issue submitted: https://github.com/rundeck-plugins/sshj-plugin/issues/15