Re: IO::Socket::INET hostname restrictions?

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:
> I'm using a program called BWwhois that uses the code
>
> my $rs = IO::Socket::INET->new(
> PeerAddr => $w_host,
> PeerPort => $port,
> Proto => $protoname,
> Timeout => $g->{timeout},
> LocalAddr => $g->{LocalAddr}
> );
>
> This normally works; however, if the hostname[1] has a CNAME RR rather
> than an A RR it fails. I couldn't find anything in the documentation
> as to how the host name is resolved.

It's not documented in detail, no.

If you don't set MultiHomed, I:S::INET calls Socket::inet_aton, which
first calls C's inet_aton(3) and then calls gethostbyname(3) if that
fails. (If perl -V:d_inetaton doesn't say 'define' then perl uses its
own implementation of inet_aton(3), since it didn't find one on your
system it could use.) Does your system's gethostbyname(3) work properly?

> Is this an oversight, or is the caller expected to look up the
> hostname in A DNS server and replace it if it finds a CNAME RR?

It works for me, here (FreeBSD). What does

perl -MSocket -E'say inet_ntoa inet_aton "www.google.com"'

(adjust quotes as necessary) say for you? If Perl's inet_aton is working
as designed it ought to properly resolve the CNAME.

Ben

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:

> In <m4ednQa8m8lbFEbT...@bt.com>, on 12/04/2011

> at 08:55 AM, Ben Morrow <b...@morrow.me.uk> said:
>
> Does your system's gethostbyname(3) work properly?
>

> AFAIK, but I can run some tests.

>
> > perl -MSocket -E'say inet_ntoa inet_aton "www.google.com"'
>

> Can't find string terminator "'" anywhere before EOF at -e line 1.

You need to change the quotes, then. I don't know how OS/2 does quoting:
everything after the -E should be passed to perl as a single argument.
Try

perl -MSocket -E"say inet_ntoa inet_aton 'www.google.com'"

or put this in a file and run it through perl:

use 5.010;
use Socket;
say inet_ntoa inet_aton "www.google.com";

Ben

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:
>

> So why does IO::Socket::INET fail?
>
> IO::Socket::INET->new(
> PeerAddr => whois.nic.mn,
> PeerPort => 43,
> Proto => tcp,
> Timeout => 60,
> LocalAddr => )
> errno 22: Invalid argument
> host whois.nic.mn not found
> 22
> IO::Socket::INET: connect: Invalid argument

*Interesting*. I believe there's at least one bug here, in IO::Socket,
but it's not entirely clear.

First, the workaround: don't use Timeout. Find some other way of timing
out the operation, like alarm(). The CNAMEs are a red herring: what
matters is that that machine doesn't accept connections on port 43. The
'Invalid argument' is just the wrong error: without Timeout, you should
see 'Connection refused', which is the right answer.

[You don't need to read the rest unless you're interested...]

That ->new call gives me exactly the same result as you: EINVAL from
connect(). Since I didn't think connect(2) was even *able* to return
EINVAL, I ran it under ktrace: this showed two calls to connect. The
first failed with EINPROGRESS, the second with EINVAL. Retrying without
Timeout consistently gave 'Connection refused' instead.

Looking at the source of IO::Socket, the logic if Timeout is specified
is approximately

set non-blocking mode on the socket
try to connect
if connect fails with EINPROGRESS
select(2) for write, with the given timeout
if the select doesn't time out
retry the connect
if that connect succeeds, return success
if it failed with EISCONN, or we are on Win32 and it failed
with WSAEINVAL, return *success*
otherwise, return the error from the second connect

The EISCONN case is the strange one. There is a comment

# Some systems refuse to re-connect() to
# an already open socket and set errno to EISCONN.
# Windows sets errno to WSAEINVAL (10022)

and it turns out that it's not just Win32 that returns EINVAL in that
case, it's at least current versions of FreeBSD and, presumably, OS/2 as
well. However, even given that, returning success is *completely wrong*.
The sequence of events is

- app calls connect(2) on a non-blocking socket
- connect(2) fails with EINPROGRESS
- app calls select(2)-for-write
- some time later, a RST packet comes in
- select(2) returns socket as ready
- app calls connect(2) again
- connect(2) fails with EISCONN/EINVAL

The EISCONN/EINVAL error is saying 'this socket isn't fit to be reused':
it doesn't tell you *anything* about whether the second connection
attempt succeeded or failed. (In fact, you get the same EINVAL if you
attempt to re-connect a TCP socket while it's in TIME_WAIT state after a
successful close.)

Looking about a bit, I found http://cr.yp.to/docs/connect.html, which
mentions this connect-twice strategy as a way of performing a non-
blocking connect and still getting the correct error. It also says that
it doesn't always work, and that you should use getpeername instead (and
for all djb's... um... charm, he's usually right about things like
this). So, I think this counts as a bug in IO::Socket, since there are
common situations on common operating systems where it doesn't work
properly.

[Oddly, it looks from the CVS log as though FreeBSD used to return
EISCONN in this situation, as the IO::Socket code was kind-of expecting,
but it was changed to EINVAL in 2006. It's not clear to me why,
especially since there now appears to be no way to get an EISCONN error
from a TCP socket.]

Ben

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:

> In <Nf-dnZw65uvr7kDT...@bt.com>, on 12/05/2011

> at 07:44 PM, Ben Morrow <b...@morrow.me.uk> said:
>
> >That ->new call gives me exactly the same result as you: EINVAL from
> >connect().
>

> Does it have the same faulty logic for multihome?

Yes. MultiHomed just makes ->new call ->connect on all the addresses in
DNS until one succeeds, so for a host with only a single address (or for
one where all addresses will refuse the connection) it makes no
difference.

I think your best way forward is either to drop Timeout and use alarm()
instead, or to just accept that that error is what you get when a
connection is refused. Assuming your kernel is behaving the same as
mine, I don't think you can get that error when you ought to have got a
successful connection, nor for situations like a hostname that doesn't
resolve.

I do intend to report this as a bug in IO::Socket, when I get round to
it.

Ben

[Ben Morrow]

Quoth Ben Morrow <b...@morrow.me.uk>:

>
> I do intend to report this as a bug in IO::Socket, when I get round to
> it.

If and when you get gcc working, would you be able to test this patch?
You would need to download and unpack the IO-1.25 tarball from CPAN,
apply the patch, and verify that it both passes 'make test' and gives a
more sensible error in the situation we've been discussing.

If you don't want to install the patched version you can use 'perl
-Mblib' from within the build directory to test the built-but-not-
installed modules. If you do want to install it, it should be perfectly
safe to install with 'make install': it will simply overwrite the IO::*
modules which came with your perl with newer versions.

Ben

diff -ur IO-1.25/lib/IO/Socket.pm IO/lib/IO/Socket.pm
--- IO-1.25/lib/IO/Socket.pm 2009-05-14 00:47:42.000000000 +0100
+++ IO/lib/IO/Socket.pm 2011-12-07 16:35:52.859688028 +0000
@@ -122,12 +122,12 @@
$err = $! || (exists &Errno::ETIMEDOUT ? &Errno::ETIMEDOUT : 1);
$@ = "connect: timeout";
}
- elsif (!connect($sock,$addr) &&
- not ($!{EISCONN} || ($! == 10022 && $^O eq 'MSWin32'))
- ) {
- # Some systems refuse to re-connect() to
- # an already open socket and set errno to EISCONN.
- # Windows sets errno to WSAEINVAL (10022)
+ elsif (!getpeername($sock)) {
+ if ($!{ENOTCONN}) {
+ # this read will fail, but will give us the error
+ # from connect(2) in $!
+ sysread $sock, my $tmp, 1;
+ }
$err = $!;
$@ = "connect: $!";
}

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:

> In <1ua5r8-...@anubis.morrow.me.uk>, on 12/07/2011
> at 11:54 PM, Ben Morrow <b...@morrow.me.uk> said:
>
> >apply the patch,
>
> What's the command, and are there any environment variables I should
> set?

Save that message (the whole message or just the patch portion, it
doesn't matter) to a file. Untar IO-1.25.tar.gz, change directory to
IO-1.25, and run

patch -p1 < saved-patch-file
perl Makefile.PL
make test
perl -Mblib -MIO::Socket::INET -e"IO::Socket::INET->new(
PeerAddr => 'whois.nic.mn', PeerPort => 43,
Timeout => 5) or die $@"

You need whatever environment variables set you need to run perl and
gcc, but nothing else. 'make test' should return quite a lot of output
followed by

All tests successful.
Files=17, Tests=180, <some more details>
Result: PASS

The last command should be all on one line, and should return

IO::Socket::INET: connect: Connection refused at -e line 1.

rather than the

IO::Socket::INET: connect: Invalid argument at -e line 1.

you get if you omit the '-Mblib'.

Ben

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:

> In <2io7r8-...@anubis.morrow.me.uk>, on 12/08/2011

> at 09:58 PM, Ben Morrow <b...@morrow.me.uk> said:
>
> > patch -p1 < saved-patch-file
> > perl Makefile.PL
> > make test
> > perl -Mblib -MIO::Socket::INET -e"IO::Socket::INET->new(
> > PeerAddr => 'whois.nic.mn', PeerPort => 43,
> > Timeout => 5) or die $@"
>

> I.e, do I need a make before the make test?

Either should work (make test implicitly performs make if it hasn't been
done yet).

> I've already installed IO-1.25; I'll test your patch but probably will
> wait for an updated version on CPAN before installing it. Thanks.

OK. Thank you.

Ben

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:

> In <9ps6s8-...@anubis.morrow.me.uk>, on 12/20/2011

> at 05:21 PM, Ben Morrow <b...@morrow.me.uk> said:
>
> >Either should work (make test implicitly performs make if it hasn't
> >been done yet).
>

> In this case I had alread done make, make test and make install. Would
> make have recognized that I ran the patch had it hit code that needed
> to be recompiled? That's not, of course, an issue for Socket.pm.

It should, yes. (Assuming OS/2 updates file timestamps in a sensible
fashion, which I expect it does.)

If you're not sure, is there any reason not to simply delete that
working directory and start again from the tarball? That's what I would
do in this situation, simply because it's quicker than thinking about
it.

(Yes, this is just the usual Unix worse-is-better brute-force approach
to the problem.)

Ben

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:

> In <9ps6s8-...@anubis.morrow.me.uk>, on 12/20/2011

> at 05:21 PM, Ben Morrow <b...@morrow.me.uk> said:
>
> >OK. Thank you.
>
> Is this outrput what you expected?
>
> patching file lib/IO/Socket.pm
<snip>
>
> [h:\vendors\cpan\io-1.25]perl -Mblib -MIO::Socket::INET

> -e"IO::Socket::INET->new

> (PeerAddr => 'whois.nic.mn', PeerPort => 43,Timeout => 5) or die $@"

> IO::Socket::INET: connect: Connection refused at -e line 1.

*Yes*. Good. Thank you. (That's the right error, rather than the
'Invalid argument' you were getting before.)

Now I've got confirmation this works properly I'll submit a bug report
(when I get to it...). I'm afraid I don't know if or when this'll make
its way into a release.

Ben

[Shmuel Metz]

In <j5eas8-...@anubis.morrow.me.uk>, on 12/22/2011

at 01:36 AM, Ben Morrow <b...@morrow.me.uk> said:

>Now I've got confirmation this works properly I'll submit a bug
>report (when I get to it...). I'm afraid I don't know if or when
>this'll make its way into a release.

Have you submitted a bug report on this? If not, should I submit one
and cite this thread, noting that your patch worked, at least in OS/2?
Thanks.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action. I reserve the
right to publicly post or ridicule any abusive E-mail. Reply to
domain Patriot dot net user shmuel+news to contact me. Do not
reply to spam...@library.lspace.org

[Ben Morrow]

Quoth Shmuel (Seymour J.) Metz <spam...@library.lspace.org.invalid>:

> In <j5eas8-...@anubis.morrow.me.uk>, on 12/22/2011
> at 01:36 AM, Ben Morrow <b...@morrow.me.uk> said:
>
> >Now I've got confirmation this works properly I'll submit a bug
> >report (when I get to it...). I'm afraid I don't know if or when
> >this'll make its way into a release.
>
> Have you submitted a bug report on this? If not, should I submit one
> and cite this thread, noting that your patch worked, at least in OS/2?
> Thanks.

Ack, no I haven't, sorry. It just kept slipping down my list of things
to do... (I was intending to properly test the patch on several
platforms before submitting it.)

Yes, you should probably submit one yourself. <per...@perl.org>.

Ben