Single quotes added to Environment Variable and breaks SSH command

[Chris Flynn]

I have a task that calls a Shell script. The Shell script is connecting to a remote server with SSH. The SSH command makes use of environment variables set on the pipeline.

from the Shell script...

echo "Stop Tomcat server on ${HOST}..."

ssh ${SSH_OPTS} ${SSH_USER}@${HOST} "sudo /sbin/service tomcat-${ENV} stop"

This task fails and when when I look at the logs I see that single quotes have been added around the ${SSH_OPTS} and where the code has double quotes.

ssh '-o StrictHostKeyChecking=no -i ~/.ssh/my.pem' my-...@myhost.com 'sudo /sbin/service tomcat-dev stop'

You can see that it does not add single quotes around my-user or myhost.com variables. I've done other testing to confirm that it appears that single quotes are only added to environment variables that contain spaces.

HOWEVER...

I've also noticed that when this Shell script calls another Shell script the ${SSH_OPTS} value does not have single quotes added to it. So the behavior does not seem to be consistent...

ssh -o StrictHostKeyChecking=no -i ~/.ssh/my.pem my-...@myhost.com 'ln -s ~/properties/my-dev.properties ~/properties/my.properties'

Both of the shell scripts run on the same server.

I've tried running the log output on the same sever and having two sets of single quotes in the command causes a problem.

Does anyone have any suggestions to help resolve this issue? Thank you for your help.

[Chris Flynn]

It is definitely only an issue if the pipeline calls the Shell script directly. There is no issue if a secondary script is used. Also, another point of information, I'm configuring my pipelines in code as JSON.

[Aravind SV]

Hello Chris,

On Fri, Jan 03, 2020 at 10:19:59 -0800, Chris Flynn wrote:
> It is definitely only an issue if the pipeline calls the Shell script
> directly. There is no issue if a secondary script is used. Also, another
> point of information, I'm configuring my pipelines in code as JSON.

Can you show a relevant snippet of the JSON, to understand how you're configuring the task (and maybe the env vars)?

Thanks,
Aravind

[Chris Flynn]

Hi Aravind,

Thank you very much for the response. Hopefully the below information will help. Let me know if you need more or something else.

I've been doing some more testing and believe I can reproduce the issue outside of GoCD with a script that defines the SSH_OPTS variable with values that have a space and then using that in my SSH command in the same script. This must be a known behavior with bash scripts, but I have yet to figure out how to get around it or find much documentation about how others work with this. I feel like this would be a pretty common scenario, but maybe I'm wrong.

JSON task...

{

  "command": "bash",

  "timeout": -1.0,

  "arguments": [

    "-c",

    "./src/main/config/scripts/wrapper.sh -h ${HOSTS_INT}"

  ],

  "run_if": "passed",

  "type": "exec"

}

Environment Variable

{

  "name": "SSH_OPTS",

  "value": "-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /path/to/.ssh/my.pem"

}

[Aravind SV]

Hello Chris,

I figured you were using it like you showed. You're also right that all of this is behavior of bash, rather than GoCD.

Your JSON is equivalent to calling a script such as this:

---

#!/bin/bash

export SSH_HOSTS="-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /path/to/.ssh/my.pem"

bash -c './src/main/config/scripts/wrapper.sh -h ${HOSTS_INT}'

---

You should see whatever behavior you're seeing when run through GoCD.

I would suggest using a script such as this one, to understand the effect of quotes and eval, etc.

---

#!/bin/bash

echo "I got: $# arguments"

for idx in `seq "$#"`; do

  echo "$idx: $1"

  shift

done

---

For instance, use it like this:

---

#!/bin/bash

SSH_USER="ssh-user"

HOST="ssh-host"

SSH_OPTS="-o StrictHostKeyChecking=no -i ~/.ssh/my.pem"

echo "Without quotes:"

./how-many-args.sh ${SSH_OPTS} ${SSH_USER}@${HOST} "sudo /sbin/service tomcat-${ENV} stop"

echo

echo "With quotes:"

./how-many-args.sh "${SSH_OPTS}" ${SSH_USER}@${HOST} "sudo /sbin/service tomcat-${ENV} stop"

echo

echo "With eval:"

eval ./how-many-args.sh "${SSH_OPTS}" ${SSH_USER}@${HOST} "sudo /sbin/service tomcat-${ENV} stop"

echo

echo "With eval and protection for the last arg:"

eval ./how-many-args.sh "${SSH_OPTS}" ${SSH_USER}@${HOST} '"sudo /sbin/service tomcat-${ENV} stop"'

echo

---

With the right usage of eval and the right number of quotes, you should be able to get what you are looking for, but it can be tricky. :) I would help, but I don't know what your wrapper.sh contains.

Cheers,

Aravind

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/5f2f25e3-6061-48a7-a1a9-5475c655cc0b%40googlegroups.com.

[Chris Flynn]

Thank you so much for all of the help. This is really interesting and a big help.

I think it is very interesting, because I believe the result I want from your scripts is the "With quotes" option, but that is what I'm already doing and SSH does not like it for some reason. It also feels like defining the variable in the same script is getting treated differently than passing them into another script, but that could be crazy.

Here is an example of what I'm doing in the wrapper.sh file that I've been using to test and find a solution.

#!/bin/bash -x

die() {

  echo >&2 "$@"

  exit 1

}

# Get HOST Name for Deployment

while getopts ":h:" opt; do

  case ${opt} in

  h)

    HOSTS=$OPTARG

    ;;

  \?)

    die "Invalid option: -$OPTARG"

    ;;

  :)

    die "Invalid option: $OPTARG requires an argument"

    ;;

  esac

done

shift $((OPTIND - 1))

if [[ -z $HOSTS ]] ; then

  die "BAD ARGUMENTS: Host was not supplied"

fi

OIFS=$IFS

IFS=";"

for HOST in $HOSTS; do

  SSH_OPTS="-o StrictHostKeyChecking=no -i /path/to/.ssh/my.pem"

  ssh -v "$SSH_OPTS" my-user@$HOST "sudo sendmail m...@my-company.com < email.txt"

  if [ $? -ne 0 ]; then

    die "Stopping Tomcat failed!"

  fi

done

IFS=$OIFS

exit 0

To unsubscribe from this group and stop receiving emails from it, send an email to go...@googlegroups.com.

[Aravind SV]

Does changing that line to the one below help?

eval ssh -v "$SSH_OPTS" my-user@$HOST '"sudo sendmail m...@my-company.com < email.txt"'

By changing the line to the one below:

eval ./how-many-args.sh -v "$SSH_OPTS" my-user@$HOST '"sudo sendmail m...@my-company.com < email.txt"'

... I find that it outputs:

I got: 7 arguments
1: -v
2: -o
3: StrictHostKeyChecking=no
4: -i
5: /path/to/.ssh/my.pem
6: my-user@HOST1
7: sudo sendmail m...@my-company.com < email.txt

... which seems to be correct. The earlier options are to the ssh client and the final command should be sent as one argument to the client so that it can be sent to the server.

It also feels like defining the variable in the same script is getting treated differently than passing them into another script, but that could be crazy.

I don't understand what you mean by that statement.

Cheers,

Aravind

To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/84a61c5a-1d01-4bb1-ab29-a251b1d4cd24%40googlegroups.com.

[Chris Flynn]

That did it. Thank you very much for all of your help! I really appreciate you taking the time to help me out even though it wasn't strictly a GoCD issue.

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/84a61c5a-1d01-4bb1-ab29-a251b1d4cd24%40googlegroups.com.