Net::HTTPS 6.04: read_entity_body hanging

[Bill Moseley]

Some LWP requests are hanging *most* of the time. Sometimes they work. At
times it seems dependent if gzip is used or not. This behavior seems to
have started with recent LWP upgrades. If I use Net::HTTPS 6.04 (most
current) I see the problem. Reverting to Net::HTTPS 6.02 appears to fix
the issue.


I'm not clear if it's related to this ticket:
https://rt.cpan.org/Public/Bug/Display.html?id=81237

Below are example command line requests and where they are hanging in LWP.


LWP::UserAgent is up to date. (6.04)
LWP::Protocol::https is up to date. (6.03)
Net::SSL is up to date. (2.85)
openssl098e.x86_64
CentOS 6.2.

Net::HTTPS (6.04) / Net-HTTP-6.05 -- hangs.
Net::HTTPS (6.04) / Net-HTTP-6.04 -- hangs.
Net::HTTPS (6.02) / Net-HTTP-6.03 -- works ok.


The page fetched below is behind an F5 load balancer that handles the SSL
and gzip compression.


In LWP::Protocol::http I've modified the code like this so it's easy to see
that read_entity_body is hanging.

READ:
{
warn "Reading $size from $socket\n";
$n = $socket->read_entity_body($buf, $size);
warn "Read $n bytes\n";
unless (defined $n) {
redo READ if $!{EINTR} || $!{EAGAIN};
die "read failed: $!";
}
redo READ if $n == -1;
}

For example:

$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", accept_encoding => "gzip" );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x164d0c8)
Read 605 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x164d0c8)
(hangs here)
^C


$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", *accept_encoding => "nope"* );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 653 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 459 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 2896 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 1448 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 2880 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 0 bytes

Yet, it's not consistent. Here's without gzip and it does hang.

$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", accept_encoding => "nope" );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
Read 653 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
Read 471 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
Read 4096 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
( hangs here again )
^C

Then again it works:

$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", accept_encoding => "nope" );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 653 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 459 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 2896 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 1448 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 2880 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 0 bytes

Sorry, not a lot to go on.

--
Bill Moseley
mos...@hank.org

[Bill Moseley]

On Mon, Mar 11, 2013 at 3:15 PM, Gisle Aas <gi...@activestate.com> wrote:

> There are new releases of libwww-perl and Net-HTTP on CPAN now. I hope
> they resolve the issue.
>

With a few quick tests it seems fixed.

Thanks very much for this update!




>
> --Gisle

--
Bill Moseley
mos...@hank.org

[Gisle Aas]

There are new releases of libwww-perl and Net-HTTP on CPAN now. I hope
they resolve the issue.

--Gisle


On Fri, Mar 8, 2013 at 8:47 AM, Bill Moseley <mos...@hank.org> wrote: