Re: error importing smtplib

[Terry Reedy]

On 11/15/2012 9:38 AM, Eric Frederich wrote:
> Hello,
>
> I created some bindings to a 3rd party library.
> I have found that when I run Python and import smtplib it works fine.
> If I first log into the 3rd party application using my bindings however
> I get a bunch of errors.
>
> What do you think this 3rd party login could be doing that would affect
> the ability to import smtp lib.

I don't know what 'login' actually means,...

> This works...
>
> import smtplib
> FOO_login()
>
> This doesn't...
>
> FOO_login()
> import smtplib

but my first guess is that FOO_login alters the module search path so
that at least one of smtplib, hashlib, or the _xxx modules imported by
hashlib is being imported from a different place. To check that

import sys
before = sys.path
FOO_login()
print sys.path==before

Similar code can check anything else accessible through sys.

> Errors.....
>
> >>> import smtplib
> ERROR:root:code for hash sha224 was not found.

I am puzzled by this line before the traceback. I cannot find 'ERROR' in
either smtplib or hashlib.

> Traceback (most recent call last):
> File "/opt/foo/python27/lib/python2.7/hashlib.py", line 139, in <module>
> globals()[__func_name] = __get_hash(__func_name)
> File "/opt/foo/python27/lib/python2.7/hashlib.py", line 103, in
> __get_openssl_constructor
> return __get_builtin_constructor(name)
> File "/opt/foo/python27/lib/python2.7/hashlib.py", line 91, in
> __get_builtin_constructor
> raise ValueError('unsupported hash type %s' % name)
> ValueError: unsupported hash type sha224
[snip similar messages]

It is also unusual to get multiple tracebacks. *Exactly* how are you
running python and is 2.7 what you intend to run?

--
Terry Jan Reedy

[Terry Reedy]

On 11/15/2012 1:48 PM, Eric Frederich wrote:
> Thanks for the idea.
> sys.path was the same before and after the login

Too bad. That seems to be a typical cause of import failure.

> What else should I be checking?

No idea. You are working beyond my knowledge. But I might either look at
the foo-login code carefully, or disable (comment out) parts of it to
see what makes the import fail.

--
Terry Jan Reedy

[Dieter Maurer]

Eric Frederich <eric.fr...@gmail.com> writes:

> I created some bindings to a 3rd party library.
> I have found that when I run Python and import smtplib it works fine.
> If I first log into the 3rd party application using my bindings however I
> get a bunch of errors.
>
> What do you think this 3rd party login could be doing that would affect the
> ability to import smtp lib.
>

> Any suggestions for debugging this further. I am lost.

>
> This works...
>
> import smtplib
> FOO_login()
>
> This doesn't...
>
> FOO_login()
> import smtplib
>

> Errors.....
>
>>>> import smtplib
> ERROR:root:code for hash sha224 was not found.

> Traceback (most recent call last):
> File "/opt/foo/python27/lib/python2.7/hashlib.py", line 139, in <module>
> globals()[__func_name] = __get_hash(__func_name)
> File "/opt/foo/python27/lib/python2.7/hashlib.py", line 103, in
> __get_openssl_constructor
> return __get_builtin_constructor(name)
> File "/opt/foo/python27/lib/python2.7/hashlib.py", line 91, in
> __get_builtin_constructor
> raise ValueError('unsupported hash type %s' % name)
> ValueError: unsupported hash type sha224

>From the error, I suppose it does something bad
for hash registries.

When I have analysed problems with "hashlib" (some time ago,
my memory may not be completely trustworthy), I got the
impression that "hashlib" essentially delegates to the
"openssl" libraries for the real work and especially
the supported hash types. Thus, I suspect that
your "FOO_login()" does something which confuses "openssl".
One potential reason could be that it loads a bad version
of an "openssl" shared library.

I would use the "trace" (shell) command to find out what operating system
calls are executed during "FOO_login()", hoping that one of them
give me a clue.

[Terry Reedy]

On 11/16/2012 2:37 PM, Eric Frederich wrote:
> So I inspected the process through /proc/<pid>/maps
> That seemed to show what libraries had been loaded (though there is
> probably an easier way to do this).

> In any case, I found that if I import smtplib before logging in I see
> these get loaded...
>
> /opt/foo/python27/lib/python2.7/lib-dynload/_ssl.so
> /lib64/libssl.so.0.9.8e
>
> Then after logging in, I see this other .so get loaded...
>
> /opt/bar/lib64/libssl.so.0.9.7

That version appears to be about a decade old. Why is bar using it?

> So that is what happens when when things are well and I don't get any
> error messages.
> However, when I do the log in first I see the /opt/bar .so file loaded first
>
> /opt/bar/lib64/libssl.so.0.9.7
>
> Then after importing smtplib I see the other two show up...
>
> /opt/foo/python27/lib/python2.7/lib-dynload/_ssl.so
> /lib64/libssl.so.0.9.8e

What I know is that hashlib.py imports _hashlib (compilied .c) and that
the latter wraps libssl, or calls _ssl.so which wraps libssl. In *nix
this is expected to already be on the system, so in not distributed with
python. Furthermore, hashlib requires a version recent enough to have
the latest (best) hash functions. I suspect decade-old 9.9.7 does not
qualify.

What I don't know is how .so loading and linking works. It seems that
two version get loaded but linking gets messed up. This reminds me of
'dll hell' on Windows ;-). I don't know either if modifying the loading
of ...9.7 in for or bar code could do anything.

> So.... I'm guessing the problem is that after I log in, the process has
> a conflicting libssl.so file loaded.
> Then when I try to import smtplib it tries getting things from there and
> that is where the errors are coming from.
>
> The question now is how do I fix this?

[easy] Do the import before the function call, which is the proper order
and the one that works.

Remove ...9.7 from bar/lib64/ and have bar use the up-to-date system
version, like python does. An alternative is to replace ...9.7 with a
duplicate ...9.8e (and probably better, only load it if there is no
system version).

> What else should I be checking?

Thinking more, you can look at sys.modules, but this does not have any
info about non-module libraries wrapped by modules, even if the latter
are C-coded.

--
Terry Jan Reedy

[Dieter Maurer]

Eric Frederich <eric.fr...@gmail.com> writes:

> ...

> So.... I'm guessing the problem is that after I log in, the process has a
> conflicting libssl.so file loaded.
> Then when I try to import smtplib it tries getting things from there and
> that is where the errors are coming from.
>
> The question now is how do I fix this?

Likely, you must relink the shared object containing
your "FOO_login". When its current version was linked,
the (really) old "libssl" has been current and the version was linked
against it.
As the binary objects for your shared object might depend on the old
version, it is best, to not only relink but to recompile it as well.