bash operator ssh problem

[Antonio Zurlo]

hi,

I'm trying to use the bash operator to ssh to a remote VM and run a command.

I get an error and this is what I see in the log:

thanks

Antonio

{bash_operator.py:80} INFO - Temporary script location: /tmp/airflowtmpDa3PDY//tmp/airflowtmpDa3PDY/downloadMD6Qm9fy
{bash_operator.py:88} INFO - Running command: gcloud compute --project antonio-zurlo ssh --zone us-east1-b hpcnode3 --command 'cd /home/zurlo && gsutil cp gs://antoniobucket/WIKI_PRICES_2018-01-01.csv c:\shared\WIKI.csv'
{bash_operator.py:97} INFO - Output:
{bash_operator.py:101} INFO - WARNING: The public SSH key file for gcloud does not exist.
{bash_operator.py:101} INFO - WARNING: The private SSH key file for gcloud does not exist.
{bash_operator.py:101} INFO - WARNING: You do not have an SSH key for gcloud.
{bash_operator.py:101} INFO - WARNING: SSH keygen will be executed to generate a key.
{bash_operator.py:101} INFO - This tool needs to create the directory [/home/airflow/.ssh] before
{bash_operator.py:101} INFO - being able to generate SSH keys.
{bash_operator.py:101} INFO - 
{bash_operator.py:101} INFO - Do you want to continue (Y/n)?
{bash_operator.py:101} INFO - Generating public/private rsa key pair.
{bash_operator.py:101} INFO - Your identification has been saved in /home/airflow/.ssh/google_compute_engine.
{bash_operator.py:101} INFO - Your public key has been saved in /home/airflow/.ssh/google_compute_engine.pub.
{bash_operator.py:101} INFO - The key fingerprint is:
{bash_operator.py:101} INFO - 89:e9:ee:fc:da:42:c4:f8:ad:8b:79:22:68:37:6d:91 airflow@airflow-worker-7ff5fcb5fc-szg86
{bash_operator.py:101} INFO - The key's randomart image is:
{bash_operator.py:101} INFO - +---[RSA 2048]----+
{bash_operator.py:101} INFO - |                 |
{bash_operator.py:101} INFO - |                 |
{bash_operator.py:101} INFO - |     o           |
{bash_operator.py:101} INFO - |    . oo .       |
{bash_operator.py:101} INFO - |     o+.S        |
{bash_operator.py:101} INFO - |     Eo .        |
{bash_operator.py:101} INFO - | .  ..o.         |
{bash_operator.py:101} INFO - |...o.B+.         |
{bash_operator.py:101} INFO - |. ..=+*=o        |
{bash_operator.py:101} INFO - +-----------------+
{bash_operator.py:101} INFO - Updating project ssh metadata...
{bash_operator.py:101} INFO - ................Updated [https://www.googleapis.com/compute/v1/projects/antonio-zurlo].
{bash_operator.py:101} INFO - done.
{bash_operator.py:101} INFO - Waiting for SSH key to propagate.
{bash_operator.py:101} INFO - ssh: connect to host 35.231.83.81 port 22: Connection timed out
{bash_operator.py:101} INFO - ERROR: (gcloud.compute.ssh) Could not SSH into the instance.  It is possible that your SSH key has not propagated to the instance yet. Try running this command again.  If you still cannot connect, verify that the firewall and instance are set to accept ssh traffic.
{bash_operator.py:105} INFO - Command exited with return code 1
{base_task_runner.py:98} INFO - Subtask: Traceback (most recent call last):
{base_task_runner.py:98} INFO - Subtask:   File "/usr/local/bin/airflow", line 27, in <module>
{base_task_runner.py:98} INFO - Subtask:     args.func(args)
{base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 392, in run
{base_task_runner.py:98} INFO - Subtask:     pool=args.pool,
{base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper
{base_task_runner.py:98} INFO - Subtask:     result = func(*args, **kwargs)
{base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1492, in _run_raw_task
{base_task_runner.py:98} INFO - Subtask:     result = task_copy.execute(context=context)
{base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 109, in execute
{base_task_runner.py:98} INFO - Subtask:     raise AirflowException("Bash command failed")
{base_task_runner.py:98} INFO - Subtask: airflow.exceptions.AirflowException: Bash command failed

[Vince Gonzalez]

Firewall issue?

{bash_operator.py:101} INFO - ssh: connect to host 35.231.83.81 port 22: Connection timed out

--
You received this message because you are subscribed to the Google Groups "cloud-composer-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-composer-di...@googlegroups.com.
To post to this group, send email to cloud-compo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-composer-discuss/CAJcv0%3D2PWrY8TfWj%2B6xx%3DEXDOvVfQ54F8y1nf66b6BDe24ZSLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

•  Vince Gonzalez   •  Customer Engineer, Big Data, Google Cloud   •  vinceg...@google.com   •  212-694-3879

[ri...@safechain.io]

In my thread on this topic a few days ago I observed that when gcloud has to generate the key it doesn't wait long enough for it to propogate.

The second time you try the task, it doesn't have to generate the key and it will work.

The alternative is to pre-generate the key for it (see my thread).

The system will autogenerate a new key for each worker-task combination, so if you have 3 tasks and 3 workers, you'll end up with 9 keys in the metadata and at least 9 failures before they are all generated.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-composer-discuss+unsub...@googlegroups.com.

To post to this group, send email to cloud-compo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-composer-discuss/CAJcv0%3D2PWrY8TfWj%2B6xx%3DEXDOvVfQ54F8y1nf66b6BDe24ZSLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.