[Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on Windows

[anatoly techtonik]

Could anybody reopen http://bugs.python.org/issue8766 ? I can't.
Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test).

> set PYTHONHOME=C:\
> python

BTW, what is the role of PYTHONPATH on Windows?
Is it a path for %INSTALLDIR%\Lib\site-packages?
--
anatoly t.
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com

[Christian Heimes]

Am 22.10.2012 15:31, schrieb anatoly techtonik:
> Could anybody reopen http://bugs.python.org/issue8766 ? I can't.
> Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test).
>
>> set PYTHONHOME=C:\
>> python

The segfault isn't caused by faulty code but by a call to abort() in
Py_FatalError(). The interpreter fails with an hard error because it is
unable to locate the encodings package.


$ PYTHONHOME=/ python3.2
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named encodings
Abgebrochen (Speicherabzug geschrieben)

$ gdb /usr/bin/python3.2 core
[...]
(gdb) bt
#0 0x00007fb05cd37445 in __GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007fb05cd3abab in __GI_abort () at abort.c:91
#2 0x0000000000482d0e in Py_FatalError ()
#3 0x0000000000482f14 in initfsencoding ()
#4 0x0000000000483e56 in Py_InitializeEx ()
#5 0x0000000000493e1a in Py_Main ()
#6 0x000000000041d307 in main ()

[R. David Murray]

You are more likely to get action on bugs by posting to the bug tracker.

That said, since 8766 was fixed, your issue is most likely a new one.
Please open a new bug report.

--David

> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rdmurray%40bitdance.com

[Victor Stinner]

2012/10/22 anatoly techtonik <tech...@gmail.com>:

> Could anybody reopen http://bugs.python.org/issue8766 ? I can't.
> Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test).
>
>> set PYTHONHOME=C:\
>> python

The issue #8766 is about PYTHONPATH environment variable, not
PYTHONHOME. Test on Linux with Python 3.4:

$ PYTHONHOME=/x ./python

Fatal Python error: Py_Initialize: Unable to get the locale encoding

ImportError: No module named 'encodings'
Abandon (core dumped)

It's not a segfault, but an "abort". It's different: abort() is called
by the application (Python) when something goes wrong. In this case,
Python is unable to locate the standard library. It's not a bug: I
asked Python to look in /x directory, whereas this directory doesn't
exist.

> BTW, what is the role of PYTHONPATH on Windows?
> Is it a path for %INSTALLDIR%\Lib\site-packages?

Same role than PYTHONPATH on Linux: specify where third party modules
are installed.

Victor

[Oleg Broytman]

On Mon, Oct 22, 2012 at 04:31:34PM +0300, anatoly techtonik <tech...@gmail.com> wrote:
> Could anybody reopen http://bugs.python.org/issue8766 ? I can't.

Even if you add a comment?

> BTW, what is the role of PYTHONPATH on Windows?

The same as on Unix -- it's inserted into (but not replaced)
sys.path. And nothing else.

Oleg.
--
Oleg Broytman http://phdru.name/ p...@phdru.name
Programmers don't die, they just GOSUB without RETURN.

[Barry Warsaw]

On Oct 22, 2012, at 03:40 PM, Victor Stinner wrote:

>The issue #8766 is about PYTHONPATH environment variable, not
>PYTHONHOME. Test on Linux with Python 3.4:
>
>$ PYTHONHOME=/x ./python
>Fatal Python error: Py_Initialize: Unable to get the locale encoding
>ImportError: No module named 'encodings'
>Abandon (core dumped)
>
>It's not a segfault, but an "abort". It's different: abort() is called
>by the application (Python) when something goes wrong. In this case,
>Python is unable to locate the standard library. It's not a bug: I
>asked Python to look in /x directory, whereas this directory doesn't
>exist.

We have this bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python3.2/+bug/938869

This is caused by the VMware player installer playing games with $PYTHONHOME
to try to point it at its own *Python 2* runtime, and trying to call
lsb_release (a Python 3 script on Ubuntu 12.10) while $PYTHONHOME is pointing
at the wrong runtime.

I personally think this is a bug in the VMware player installer script.

-Barry

[Christian Heimes]

Am 22.10.2012 16:18, schrieb Barry Warsaw:
> This is caused by the VMware player installer playing games with $PYTHONHOME
> to try to point it at its own *Python 2* runtime, and trying to call
> lsb_release (a Python 3 script on Ubuntu 12.10) while $PYTHONHOME is pointing
> at the wrong runtime.
>
> I personally think this is a bug in the VMware player installer script.

I think it's a bug in lsb_release, too. It should use the -E option in
its shebang:

with "#!/usr/bin/python"

$ PYTHONHOME=/invalid /usr/bin/lsb_release
ImportError: No module named site

with "#!/usr/bin/python -E"

$ PYTHONHOME=/invalid /usr/bin/lsb_release
LSB Version:
core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:core-4.0-noarch

[Barry Warsaw]

On Oct 22, 2012, at 04:30 PM, Christian Heimes wrote:

>I think it's a bug in lsb_release, too. It should use the -E option in
>its shebang:

Indeed, thanks! I'd forgotten about -E.

(bug updated btw)

-Barry

[Christian Heimes]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

You might want -s, too.

- -E : ignore PYTHON* environment variables (such as PYTHONPATH)
- -s : don't add user site directory to sys.path

- -E -s are useful for system commands and mandatory for suid commands.

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBCAAGBQJQhVx9AAoJEMeIxMHUVQ1F3YEP/RUP6jp0I73zhXKX218PTW3C
1TFqe3V+1hoItaG3wAxTtGFfhlNfiQ2P0cJxXz3XecqKR5N23tC1ESn+sjHBfCtS
rzB4d8h3F3BO1r15NzTnEKiZD6SxMAtiPksd6HQtwcAiPmiSbKded1v8WS2nQfKV
xaz05NSEkbsG09Cj4DIGx+Vb7dJb3y+j9oVxI/8EQijURfSXXqx10x3mTy3upH6c
3LK+qtPbnrC3OFdHVKnZcAcHTNXzALs3vagcye04I3Jes927y6SdboEx77kptRbm
rDmMQyRbvQgsYP+tXhJvJg8zf68t7F20pP4ato+FkYlaJJjkDXeSloGD8x8KSGvP
P5QHmeMjFCazrKIjYNH00mLLUlW7cMZzapPvJP/sUaMFSVn+eEsU7L0Wj2fxxIsu
s7kBAvso7t8Ins3OP2vn+RbFfo1wj8FV7DtBQoPo0MAjXKLZToaE7hCrqx0pBEmc
nodQxgwaJXsRQAx4n4ZpfRCzqeUTIlgbvNpf7UPTEIGaWz6J98FZe5I3Gpa9VUDR
h40lHvHoCy7UypmBYAvBh6q/EamSrpanUPIECHaxKqiehnUeChUz67A5hsIQ482z
7frVmgTcLHn8Q8+XlyXbCy6nBNU1Jpgy3yWbybPPsrDZWSJVtpdfgSURkimDZ2wG
tRSwF3gWtTTU4te2NkYP
=Suu/
-----END PGP SIGNATURE-----

[anatoly techtonik]

On Mon, Oct 22, 2012 at 4:40 PM, Victor Stinner
<victor....@gmail.com> wrote:
>>
>>> set PYTHONHOME=C:\
>>> python
>
> The issue #8766 is about PYTHONPATH environment variable, not
> PYTHONHOME. Test on Linux with Python 3.4:
>
> $ PYTHONHOME=/x ./python
> Fatal Python error: Py_Initialize: Unable to get the locale encoding
> ImportError: No module named 'encodings'
> Abandon (core dumped)
>
> It's not a segfault, but an "abort". It's different: abort() is called
> by the application (Python) when something goes wrong. In this case,
> Python is unable to locate the standard library. It's not a bug: I
> asked Python to look in /x directory, whereas this directory doesn't
> exist.

I don't know what is abort() on Linux, but I believe coredumps is not
something you want to get while setting some environment variable. On
Windows it outputs a standard crash dialog box, which immediately
raises questions about Python stability and potential exploitability
in this direction.

>> BTW, what is the role of PYTHONPATH on Windows?
>> Is it a path for %INSTALLDIR%\Lib\site-packages?
>
> Same role than PYTHONPATH on Linux: specify where third party modules
> are installed.

I meant PYTHONHOME, sorry.

[Christian Heimes]

Am 22.10.2012 18:26, schrieb anatoly techtonik:
> I don't know what is abort() on Linux, but I believe coredumps is not
> something you want to get while setting some environment variable. On
> Windows it outputs a standard crash dialog box, which immediately
> raises questions about Python stability and potential exploitability
> in this direction.

abort() is a C stdlib function that kills the current process with
SGIABRT or similar. It's designed to draw attention to a fatal error.

Are you proposing that Python should rather use _exit() than abort()
here? Both forcedly shut down the process immediately.

Christian

[anatoly techtonik]

On Mon, Oct 22, 2012 at 7:37 PM, Christian Heimes <chri...@python.org> wrote:
> Am 22.10.2012 18:26, schrieb anatoly techtonik:
>> I don't know what is abort() on Linux, but I believe coredumps is not
>> something you want to get while setting some environment variable. On
>> Windows it outputs a standard crash dialog box, which immediately
>> raises questions about Python stability and potential exploitability
>> in this direction.
>
> abort() is a C stdlib function that kills the current process with
> SGIABRT or similar. It's designed to draw attention to a fatal error.
>
> Are you proposing that Python should rather use _exit() than abort()
> here? Both forcedly shut down the process immediately.

I am not a C coder and don't have any core Unix programming
background. If Python is unable to start because it can not find its
libraries, I prefer an explanative error message with standard system
error code. Even if it is Fatal Python error - this case is still in
user land and should be fixed normally. The error message could be
improved though. Right now I get:

E:\>python
Fatal Python error: Py_Initialize: unable to load the file system codec

ImportError: No module named 'encodings'

This could be improved to:

Fatal Python error: Py_Initialize: unable to find module named 'encodings'
in 'C:\'

--
anatoly t.

[Stephen J. Turnbull]

anatoly techtonik writes:

> I am not a C coder and don't have any core Unix programming
> background. If Python is unable to start because it can not find its
> libraries, I prefer an explanative error message with standard system
> error code. Even if it is Fatal Python error - this case is still in
> user land and should be fixed normally.

Python can't know that the problem is in "user land" as you call it.
For all Python knows, the directory containing its libraries is
corrupt, or that the file system code itself experienced a mysterious
glitch.

While in the technical sense the problem here is in userland, not in
the kernel, the distinction you're making is incoherent. For most
users, it doesn't matter whether an environment variable is set
incorrectly without their knowledge, or if the kernel is buggy, or if
the disk is corrupt. And from Python's point of view, the world as a
whole no longer makes.

So it shuts down abnormally. That's what an abort means, in
programming as in rocket launches. Users should be scared if this
happens; somebody really screwed up. (Unless it's themselves, and
then they only have themselves to blame.)

> The error message could be
> improved though. Right now I get:
>
> E:\>python
> Fatal Python error: Py_Initialize: unable to load the file system codec
> ImportError: No module named 'encodings'
>
> This could be improved to:
>
> Fatal Python error: Py_Initialize: unable to find module named 'encodings'
> in 'C:\'

That may be an improvement. But in that case it might be worth
explaining where "C:\" came from (in this case PYTHONHOME, I guess?)

[Greg Ewing]

Stephen J. Turnbull wrote:
> For most
> users, it doesn't matter whether an environment variable is set
> incorrectly without their knowledge, or if the kernel is buggy, or if
> the disk is corrupt. And from Python's point of view, the world as a
> whole no longer makes.
>
> So it shuts down abnormally. That's what an abort means, in
> programming as in rocket launches.

Seems to me the only reason to use SIGABRT rather than exit()
with an error code is to provoke a core dump, and that's only
useful if we suspect a bug in Python itself. That's not the
case here -- the cause is clearly something external.

--
Greg

[anatoly techtonik]

On Tue, Oct 23, 2012 at 2:39 AM, Stephen J. Turnbull <turn...@sk.tsukuba.ac.jp> wrote:

So it shuts down abnormally.  That's what an abort means, in
programming as in rocket launches.  Users should be scared if this
happens; somebody really screwed up.  (Unless it's themselves, and
then they only have themselves to blame.)

 > The error message could be
 > improved though. Right now I get:
 >
 > E:\>python
 > Fatal Python error: Py_Initialize: unable to load the file system codec
 > ImportError: No module named 'encodings'
 >
 > This could be improved to:
 >
 > Fatal Python error: Py_Initialize: unable to find module named 'encodings'
 > in 'C:\'

That may be an improvement.  But in that case it might be worth
explaining where "C:\" came from (in this case PYTHONHOME, I guess?)

Good idea. 

Fatal Python error: Py_Initialize: unable to find module named 'encodings'

in PYTHONHOME ('C:\')