-c Paramiko hangs if stdout is large?

[fdavis]

Hello all,

I don't know if this is a paramiko problem, or our connection implementation for it, but the SSH connection seems to handle this okay.

From running a  'command: yum update -y' for CentOS 5 machines on the recent upgrade, paramiko would sometimes hang indefinitely.
Checking running processes and logs on the remote host would show that the yum update completed successfully.

I believe the problem is having a large stdout returned.
(I tested on localhost dumping big randomly generated string into a file and just catting from a playbook.)

SSH spits out a buffered amount at a time, while paramiko just hangs.
(no load on either local or remote host, no disk io happening, the process is just sitting around)

Another case that may be related is an EC2 playbook for setting up cloud9.
I believe the error started showing up after I switched back to paramiko from Fireball.
The first sourcemint install (which dumps a whole lot to stdout) tends to fail.
The error message claims mismatching quotes, but the install was successful.

I thought I should bring this up in case anyone runs into similar behaviour.

I remember something similar before in the mailing list, but I couldn't find it.
I think it was related to filesizes and paramiko??


PS: My cloud 9 playbook can be found here.
https://github.com/fdavis/ansible-ec2-env/tree/master/c9-servers/c9

[Michael DeHaan]

We have a large output test for this reason, but tests are currently
running -c local (they were not always).

How large was large?

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Daniel Hokka Zakrisson]

This should be fixed on devel. Please test it again and see if it's still
an issue.

Daniel

> PS: My cloud 9 playbook can be found here.
> https://github.com/fdavis/ansible-ec2-env/tree/master/c9-servers/c9
>

[Daniel Hokka Zakrisson]

Michael DeHaan wrote:
> We have a large output test for this reason, but tests are currently
> running -c local (they were not always).

Yeah, running this test with -c paramiko triggered the issue.

Daniel

[fdavis]

My test file for the cat playbook was 100k.
I estimated yum's output to be about that order of magnitude, but didn't get any real metrics on that.
(stdout size: 80 char lines * 400 pkgs * 4 iterations (install, download, verify, cleanup on each package) ~= 128k)

At any rate, it is fixed. So thank you :)

[Michael DeHaan]

On Mon, Jan 28, 2013 at 6:41 PM, Daniel Hokka Zakrisson
<dan...@hozac.com> wrote:
> Michael DeHaan wrote:
>> We have a large output test for this reason, but tests are currently
>> running -c local (they were not always).
>
> Yeah, running this test with -c paramiko triggered the issue.

Seems reasonable to make the tests take the connection string from an
environment variable, like

TEST_CONNECTION=<type> or similar?

The reason we did local was so you didn't need SSH setup for one, but
also when making quick changes you could execute the test battery
faster.

[Dag Wieers]

On Tue, 29 Jan 2013, Daniel Hokka Zakrisson wrote:

> fdavis wrote:
>
>> I remember something similar before in the mailing list, but I couldn't
>> find it.
>> I think it was related to filesizes and paramiko??
>
> This should be fixed on devel. Please test it again and see if it's still
> an issue.

This is great to hear. In retrospect, this probably explains hangs we have
seen from time to time, but given the fact there was no output we did not
relate this to the amount of output.

--
-- dag wieers, d...@wieers.com, http://dag.wieers.com/
-- dagit linux solutions, in...@dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]