socket.getaddrinfo() crashes

[JohnMudd]

I'm building on an old (2003) Linux and trying to run on a collection
of machines up to and including modern Ubuntu Linux. My app runs on
the build machine but crashes on the modern Linux. I tracked it down
to a call to socket.getaddrinfo(). It even crashes in this 2 line
demo program. Any suggestions?

import socket
res_list = socket.getaddrinfo("google.com", 443, 0,
socket.SOCK_STREAM)


Build PC:
Python 2.7.2, PyInstaller 1.5.1
Red Hat Enterprise Linux AS release 3 (Taroon)
Kernel \r on an \m
Linux ares 2.4.21-4.EL #1 Fri Oct 3 18:13:58 EDT 2003 i686 i686 i386
GNU/Linux

Execute machine:
Ubuntu 10.10 \n \l
Linux rx30-mudd 2.6.35-30-generic #56-Ubuntu SMP Mon Jul 11 20:00:22
UTC 2011 i686 GNU/Linux

[JohnMudd]

I tried executing an a slightly older Linux, no help.

CentOS-5.4 x86 (ks-3.03) 

Kernel \r on an \m

Not Fedora

Linux rx30.localdomain 2.6.18-194.32.1.el5PAE #1 SMP Wed Jan 5 18:43:13 EST 2011 i686 i686 i386 GNU/Linux

I tried recompiling Python on the build machine with gcc optimization turned off (-O0). Then rebuilding the test program.  No help.

[Martin Zibricky]

JohnMudd píše v Pá 16. 09. 2011 v 09:01 -0700:

> It even crashes in this 2 line
> demo program. Any suggestions?

I would say it could be caused by different kernels: your old kernel is
2.4 and newer is 2.6 series. I would assume there could be some changes
in the networking api. And the compiled libraries may not run on kernel
2.6.

How did you get runnning python 2.7 on your old build machine?

Could you try create builds on your centos 5.4 and then try the
executables on your ubuntu box if it will also crash?

[JohnMudd]

Thanks for the response.  Yes, I suppose I'm asking for too much, trying to span such a wide gap in kernel versions.  I will try building on CentOS and run on Ubuntu next. If that works then that will have to serve as my short term solution. If I need to run on older machines then I will make additional builds on older machines.

I was a little surprised but I didn't have any problem compiling python 2.7.2 from source on the old kernel machine.  I've also tried other version of python going back to 2.2.3 just in case that would help me but they did make any difference.  The old machine has python 2.2.3 as the standard python.

I googled for any reference of getaddrinfo crashing. I found this link http://goo.gl/OoOnL which mentions (near the bottom) that changing thread options might help. I just made one last attempt to build on the old machine using various python threads options.  None helped but when I used --without-threads the PyInstaller built executable produced the following message when executing on Ubuntu. That's the best clue so far.

test_app: relocation error: /lib/libnss_dns.so.2: symbol __ns_get32, version GLIBC_PRIVATE not defined in file libresolv.so.2 with link time reference

[Martin Zibricky]

JohnMudd píše v Pá 16. 09. 2011 v 11:15 -0700:

> I was a little surprised but I didn't have any problem compiling
> python 2.7.2 from source on the old kernel machine. I've also tried
> other version of python going back to 2.2.3 just in case that would
> help me but they did make any difference. The old machine has python
> 2.2.3 as the standard python.

One more thing to try:

What is your gcc version on that old machine?
Maybe you could try first try compiling any more recent gcc version (4.x
series) and then try compiling python 2.7 with this more recent version.
Maybe this would help.

[JohnMudd]

Well, the compiler is gcc version 3.2.3 20030502. But I won't need to replace it because I think it's solved now.

The problem was that PyInstaller was including libresolv.so.2 from the old machine. That's apparently incompatible with my new machine. All I had to do was add the following line to bindepend.py in the PyInstaller dir and rebuild.

   r'/lib/libresolv.so.2':1,

[Martin Zibricky]

JohnMudd píše v Pá 16. 09. 2011 v 12:18 -0700:
> r'/lib/libresolv.so.2':1

does it mean that it should be put on the exclude list of linux
libraries?

[JohnMudd]

I wondering the same thing. Let me test my full app first to see if I created any new problems.

[JohnMudd]

Well, I tested my full app and it seems to work now. Maybe libresolv should be excluded on Linux.

For reference, my app is about 500 lines of python. I use the following standard modules along with suds and tornado. suds is for creating SOAP clients. tornado is a HTTP server framework but I'm just using it in this app for logging and command line argument parsing.

    binascii datetime json logging os pprint Queue shutil string sys threading time traceback urlparse

[Hartmut Goebel]

This file belongs to glibc, so most probaply it sould be excluded.

$ rpm -qf '/lib/libresolv.so.2'
glibc-2.11.1-8.2mnb2

And perhaps quite some more which belong to glibc:

ld-linux.so.*
libBrokenLocale.so.*
libanl.so.*
libcidn.so.*
libcrypt.so.*
libnsl.so.*
libnss_*so.*
libresolv.so.*
libutil.so.*

--
Schönen Gruß - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP

Goebel Consult
Spezialist für IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de

Monatliche Kolumne: http://www.cissp-gefluester.de/
Goebel Consult ist Mitglied bei http://www.7-it.de

[Martin Zibricky]

Hartmut Goebel píše v So 17. 09. 2011 v 14:55 +0200:
> libnss_*so.*

It seems to be only the following libnss:

r'/libnss_compat.*\.so\..*':1,
r'/libnss_dns.*\.so\..*':1,
r'/libnss_files.*\.so\..*':1,
r'/libnss_hesiod.*\.so\..*':1,
r'/libnss_nis.*\.so\..*':1,
r'/libnss_nisplus.*\.so\..*':1,

[Martin Zibricky]

JohnMudd píše v Pá 16. 09. 2011 v 12:18 -0700:

> The problem was that PyInstaller was including libresolv.so.2 from the
> old machine. That's apparently incompatible with my new machine. All I
> had to do was add the following line to bindepend.py in the
> PyInstaller dir and rebuild.
>
> r'/lib/libresolv.so.2':1,

I committed the fix to 1.5 branch and to svn trunk. Could you please
test it, if it still works for you?

These are the statements for exclude list:

# glibc regex excludes
r'/ld-linux\.so\..*':1,
r'/libBrokenLocale\.so\..*':1,
r'/libanl\.so\..*':1,
r'/libcidn\.so\..*':1,
r'/libcrypt\.so\..*':1,
r'/libnsl\.so\..*':1,

r'/libnss_compat.*\.so\..*':1,
r'/libnss_dns.*\.so\..*':1,
r'/libnss_files.*\.so\..*':1,
r'/libnss_hesiod.*\.so\..*':1,
r'/libnss_nis.*\.so\..*':1,
r'/libnss_nisplus.*\.so\..*':1,

r'/libresolv\.so\..*':1,
r'/libutil\.so\..*':1,

[JohnMudd]

Yes, this works for me.  I tried both adding these excludes to 1.5.1 and also tested trunk-r1593.zip. Both worked.