AWX syncronize built in stopped working. No error message just stalls out

[John Davis]

After pulling Gallaxy content the synchronize built just stops and hangs at the login to the remote host. Worked perfectly previously, I have blown away AWx and reinstalled it completely and still the same results.

Using synchronize with delegated host to copy files from repos.

Any thoughts on what would cause this. 

sshpass -d10 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o User="******-admin" -o ConnectTimeout=10 -o ControlPath="/runner/cp/244a2ad7c3" -tt **********02 /bin/sh -c '/usr/bin/python3 /home/*****-admin/.ansible/tmp/ansible-tmp-1675369700.6906097-45-62275435632762/AnsiballZ_synchronize.py && sleep 0'

1000     3850516 3850515  0 14:28 pts/2    00:00:00 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o User="******-admin" -o ConnectTimeout=10 -o ControlPath="/runner/cp/244a2ad7c3" -tt ***********02 /bin/sh -c '/usr/bin/python3 /home/*****-admin/.ansible/tmp/ansible-tmp-1675369700.6906097-45-62275435632762/AnsiballZ_synchronize.py && sleep 0'

*****-a+ 3850554 3850548  0 14:28 pts/4    00:00:00 /usr/bin/rsync --delay-updates -F --compress --checksum --archive --no-recursive --rsh=/usr/bin/ssh -S none -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null --rsync-path=sudo -u ******-admin rsync --out-format=<<CHANGED>>%i %n%L /software/packages/cisco/current/ ******-admin@***********01:/tmp/software/cisco

All processes are waiting, below is the last created by the ansible. Killing this one kills all.

******-a+ 3850555 3850554  0 14:28 pts/4    00:00:00 /usr/bin/ssh -S none -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -l ******-admin **********01 sudo -u *****-admin rsync --server -logDtpcze.LsfxC --log-format=%i --delay-updates . /tmp/software/cisco
root     3851196 3847802  0 14:31 pts/3    00:00:00 grep --color=auto ssh

[AWX Project]

Hi!

do you see ansible processes (python processes that called the module) hanging on the remote node?

do your playbooks run fine outside of AWX?

"Worked perfectly previously"

what do you mean exactly? Like a previous version of AWX?

AWX Team

[D]

Hi

Processes don't even show up on the remote node...strangely.

Yes they run fine using ansible-playbook, I think this is related to an
issue that others had with issues after upgrading to latest image. This
was working fine until I installed a watch agent from Galaxy.

After making some changes I see the "you must install sshpass..." error.

I am currently rolling back to 21.11.0

[John Davis]

Rolling back didn't work same issue. Syncronize jobs hang, these appear in the process list as ssh processes,  AWX just running but not completing. Checking to see if docker vbridge could be blocking.

[John Davis]

Something has definitely change to cause this to stop working. The difference between running from AWX is substantial as oppose to ansible-playbook. BTW the same play runs fine with ansible-playbook. All playbooks work fine that are not using synchronize that I can tell or that I have tested in AWX. All work when running from the CLI

[Rowe, Walter P. (Fed)]

Are you succeeding with CLI on the same host that AWX is failing?

Walter
--
Walter Rowe, Division Chief
Infrastructure Services, OISM
Mobile: 202.355.4123

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/34507cf0-58eb-40bb-bed5-4a604f6e0ccbn%40googlegroups.com.

[Rowe, Walter P. (Fed)]

Also, running the playbook is a connection from control host to target machine. The git sync is a connection between the control host and the git repo host.

Walter
--
Walter Rowe, Division Chief
Infrastructure Services, OISM
Mobile: 202.355.4123

To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/1ED75802-2476-465C-9C5E-848DC04A72B6%40nist.gov.

[John Davis]

I am succeeding on the same host.

[John Davis]

At one point I thought maybe this is a system issue with SSH but since the CLI playbook runs fine. I have changed my thinking on that. I can see the system processes spin up by listing the processes. The last playbook process in the chain just sits and spins forever. 

[Rowe, Walter P. (Fed)]

After making some changes I see the "you must install sshpass..." error. 

Are you using ssh keys or ssh username/password?

Walter
--
Walter Rowe, Division Chief
Infrastructure Services, OISM
Mobile: 202.355.4123

To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/d1e30e23-f633-47cb-9398-d1edfaf5e542n%40googlegroups.com.