Why doesn't Python remember the initial directory?

[kj]

As far as I've been able to determine, Python does not remember
(immutably, that is) the working directory at the program's start-up,
or, if it does, it does not officially expose this information.

Does anyone know why this is? Is there a PEP stating the rationale
for it?

Thanks!

[Giacomo Alzetta]

You can obtain the working directory with os.getcwd().

giacomo@jack-laptop:~$ echo 'import os; print os.getcwd()' > testing-dir.py
giacomo@jack-laptop:~$ python testing-dir.py
/home/giacomo
giacomo@jack-laptop:~$ cd Documenti
giacomo@jack-laptop:~/Documenti$ python ../testing-dir.py
/home/giacomo/Documenti
giacomo@jack-laptop:~/Documenti$

Obviously using os.chdir() will change the working directory, and the os.getcwd() will not be the start-up working directory, but if you need the start-up working directory you can get it at start-up and save it in some constant.

[Roy Smith]

In article <k0rj38$2gc$1...@reader1.panix.com>, kj <no.e...@please.post>
wrote:

> As far as I've been able to determine, Python does not remember
> (immutably, that is) the working directory at the program's start-up,
> or, if it does, it does not officially expose this information.

Why would you expect that it would? What would it (or you) do with this
information?

More to the point, doing a chdir() is not something any library code
would do (at least not that I'm aware of), so if the directory changed,
it's because some application code did it. In which case, you could
have just stored the working directory yourself.

[Mark Lawrence]

Why would you have a Python Enhancement Proposal to state the rationale
for this?

--
Cheers.

Mark Lawrence.

[Laszlo Nagy]

When you start the program, you have a current directory. When you
change it, then it is changed. How do you want Python to remember a
directory? For example, you can put it into a variable, and use it
later. Can you please show us some example code that demonstrates your
actual problem?

[Nobody]

On Sun, 19 Aug 2012 14:01:15 -0700, Giacomo Alzetta wrote:

> You can obtain the working directory with os.getcwd().

Maybe. On Unix, it's possible that the current directory no longer
has a pathname. As with files, directories can be "deleted" (i.e.
unlinked) even while they're still in use.

Similarly, a directory can be renamed while it's in use, so the current
directory's pathname may have changed even while the current directory
itself hasn't.

[88888 Dihedral]

Immutable data can be frozen and saved in somewhere off the main memory.

Perative and imperative programming are different.

Please check Erlang.

[kj]

This means that no library code can ever count on, for example,
being able to reliably find the path to the file that contains the
definition of __main__. That's a weakness, IMO. One manifestation
of this weakness is that os.chdir breaks inspect.getmodule, at
least on Unix. If you have some Unix system handy, you can try
the following. First change the argument to os.chdir below to some
valid directory other than your working directory. Then, run the
script, making sure that you refer to it using a relative path.
When I do this on my system (OS X + Python 2.7.3), the script bombs
at the last print statement, because the second call to inspect.getmodule
(though not the first one) returns None.

import inspect
import os

frame = inspect.currentframe()

print inspect.getmodule(frame).__name__

os.chdir('/some/other/directory') # where '/some/other/directory' is
# different from the initial directory

print inspect.getmodule(frame).__name__

...............

% python demo.py
python demo.py
__main__
Traceback (most recent call last):
File "demo.py", line 11, in <module>
print inspect.getmodule(frame).__name__
AttributeError: 'NoneType' object has no attribute '__name__'



I don't know of any way to fix inspect.getmodule that does not
involve, directly or indirectly, keeping a stable record of the
starting directory.

But, who am I kidding? What needs fixing, right? That's not a
bug, that's a feature! Etc.

By now I have learned to expect that 99.99% of Python programmers
will find that there's nothing wrong with behavior like the one
described above, that it is in fact exactly As It Should Be, because,
you see, since Python is the epitome of perfection, it follows
inexorably that any flaw or shortcoming one may *perceive* in Python
is only an *illusion*: the flaw or shortcoming is really in the
benighted programmer, for having stupid ideas about programming
(i.e. any idea that may entail that Python is not *gasp* perfect).
Pardon my cynicism, but the general vibe from the replies I've
gotten to my post (i.e. "if Python ain't got it, it means you don't
need it") is entirely in line with these expectations.

[Jerry Hill]

On Sun, Aug 19, 2012 at 9:57 PM, kj <no.e...@please.post> wrote:
> By now I have learned to expect that 99.99% of Python programmers
> will find that there's nothing wrong with behavior like the one
> described above, that it is in fact exactly As It Should Be, because,
> you see, since Python is the epitome of perfection, it follows
> inexorably that any flaw or shortcoming one may *perceive* in Python
> is only an *illusion*: the flaw or shortcoming is really in the
> benighted programmer, for having stupid ideas about programming
> (i.e. any idea that may entail that Python is not *gasp* perfect).
> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.

Since you have no respect for the people you're writing to, why
bother? I know I certainly have no desire to spend any time at all on
your problem when you say things like that. Perhaps you're looking
for for the argument clinic instead?

http://www.youtube.com/watch?v=RDjCqjzbvJY

--
Jerry

[alex23]

On Monday, 20 August 2012 11:57:46 UTC+10, kj wrote:
> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__. That's a weakness, IMO.

No, it's not. It's a _strength_. If you've written a library that requires absolute knowledge of its installed location in order for its internals to work, then I'm not installing your library.

> When I do this on my system (OS X + Python 2.7.3), the script bombs
> at the last print statement, because the second call to inspect.getmodule
> (though not the first one) returns None.

So, uh, do something sane like test for the result of inspect.getmodule _before_ trying to do something invalid to it?

> I don't know of any way to fix inspect.getmodule that does not
> involve, directly or indirectly, keeping a stable record of the
> starting directory.

Then _that is the answer_. YOU need to keep "a stable record":

import inspect
import os

THIS_FILE = os.path.join(os.getcwd(), '<this_module_name>.py')

frame = inspect.currentframe()
print inspect.getmodule(frame).__name__

os.chdir('/some/other/directory')

print inspect.getmodule(frame, _filename=THIS_FILE).__name__

> But, who am I kidding? What needs fixing, right? That's not a
> bug, that's a feature! Etc.

Right. Because that sort of introspection of objects is rare, why burden the _entire_ language with an obligation that is only required in a few places?

> By now I have learned to expect that 99.99% of Python programmers

> will find that [blah blah blah, whine whine whine].

> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.

Oh my god, how DARE people with EXPERIENCE in a language challenge the PRECONCEPTIONS of an AMATEUR!!! HOW DARE THEY?!?!

[alex23]

My apologies for any double-ups and bad formatting. The new Google Groups interface seems to have effectively shat away decades of UX for something that I can only guess was generated randomly.

[alex23]

On Monday, 20 August 2012 12:57:44 UTC+10, alex23 wrote:
> a library that requires absolute knowledge of its
> installed location in order for its internals to work

That should read: "a library that requires derivation of its installed location from the current working directory in order" etc

[Steven D'Aprano]

On Mon, 20 Aug 2012 01:57:46 +0000, kj wrote:

> In <roy-CA6D77.1...@news.panix.com> Roy Smith <r...@panix.com>
> writes:
>
>>In article <k0rj38$2gc$1...@reader1.panix.com>, kj <no.e...@please.post>
>>wrote:
>
>>> As far as I've been able to determine, Python does not remember
>>> (immutably, that is) the working directory at the program's start-up,
>>> or, if it does, it does not officially expose this information.

[...]

> This means that no library code can ever count on, for example, being
> able to reliably find the path to the file that contains the definition
> of __main__.

Possibly not in the face of chmod.

> That's a weakness, IMO.

Certainly.

> One manifestation of this
> weakness is that os.chdir breaks inspect.getmodule, at least on Unix.

At the very least, it can do so under some circumstances.

By the way, inspect.currentframe is quite problematic. Although it is
given a public name, it is just a thin wrapper around sys._getframe which
is not just *named* as a private function, but also states clearly in the
documentation:

"This function should be used for internal and specialized purposes only."

So I'm not sure that fixing problems with inspect.currentframe will be
high on anyone's priority.


[...]

> I don't know of any way to fix inspect.getmodule that does not involve,
> directly or indirectly, keeping a stable record of the starting
> directory.

I'm not convinced that this is necessary for the example you give.
Perhaps modules need to store absolute pathnames rather than relative
ones?

os.chdir probably breaks a lot of things. Python operates under the
assumption that the working directory does not change. If you change it,
it's your responsibility to deal with it.

Apart from shell scripting languages, do other languages cope well with
having the working directory changed?

> But, who am I kidding? What needs fixing, right? That's not a bug,
> that's a feature! Etc.

Not so much. More of, "yes, that's a bug, but it's not an important bug.
If you have a patch for fixing it, we'll consider it, but if you don't,
we've got about two thousand more important bugs and features to get
through first."

Your chances of getting this fixed will be *much* improved if you can
demonstrate a genuine problem that can't easily be fixed by just telling
the programmer "don't change working directories, or if you do, you need
to change back again before doing X, Y or Z".

Personally, I think that trying to fix all the things that break if you
chdir is not worthwhile, but having Python record the directory you
started with in (say) sys.startingdirectory might be a worthwhile feature
request.



--
Steven

[alex23]

On Monday, 20 August 2012 12:57:44 UTC+10, alex23 wrote:

> If you've written a library that requires absolute
> knowledge of its installed location in order for its
> internals to work, then I'm not installing your library.

That should read: "requires the ability to derive absolute knowledge of its installed location for an application's current working directory".

There's nothing wrong with a library knowing where shit is. But again, that should in NO WAY be bound to the application working directory.

[alex23]

On Monday, 20 August 2012 13:38:03 UTC+10, Steven D'Aprano wrote:
> Not so much. More of, "yes, that's a bug, but it's not an important bug.
> If you have a patch for fixing it, we'll consider it, but if you don't,
> we've got about two thousand more important bugs and features to get
> through first."

I'm not really convinced it's a bug, though. The issue only happens if the module is the main application, so wrapping this behind a launcher will resolve the issue. Given that you can manually provide the filepath to inspect, I'd say it's a known limitation with a workaround.

[Mark Lawrence]

I see, I see, I get the picture. You're in the "The OS is flawed so I
expect the Python core developers to do my work for me by producing code
that gets around every known flaw in every supported OS" club. Somehow
I don't think that will happen.

--
Cheers.

Mark Lawrence.

[Mark Lawrence]

On 20/08/2012 04:04, alex23 wrote:
> My apologies for any double-ups and bad formatting. The new Google Groups interface seems to have effectively shat away decades of UX for something that I can only guess was generated randomly.
>

It's very useful for reporting spam. Otherwise Thunderbird is go :)

--
Cheers.

Mark Lawrence.

[andrea crotti]

2012/8/20 kj <no.e...@please.post>:
> In <roy-CA6D77.1...@news.panix.com> Roy Smith <r...@panix.com> writes> This means that no library code can ever count on, for example,

>..

As in many other cases the programming language can't possibly act
safely on all the possible stupid things that the programmer wants to
do, and not understanding how an operating system works doesn't help
either..

In the specific case there is absolutely no use of os.chdir, since you
can:
- use absolute paths
- things like subprocess.Popen accept a cwd argument
- at worst you can chdir back to the previous position right after the
broken thing that require a certain path that you are calling is run

[Steven D'Aprano]

On Mon, 20 Aug 2012 12:56:42 +0100, andrea crotti wrote:

> In the specific case there is absolutely no use of os.chdir, since you
> can:
> - use absolute paths
> - things like subprocess.Popen accept a cwd argument - at worst you can
> chdir back to the previous position right after the broken thing that
> require a certain path that you are calling is run

I wouldn't say so much that there's "absolutely no use", it's more that
there are other, safer, ways to get the same result.

As I understand it, os.chdir is more for the convenience of sys admins
who want to write quick scripts in Python than a function intended to be
used in libraries or major applications.

An interesting question is, what do other languages do in this case?



--
Steven

[Walter Hurry]

On Mon, 20 Aug 2012 13:14:02 +0000, Steven D'Aprano wrote:

> On Mon, 20 Aug 2012 12:56:42 +0100, andrea crotti wrote:
>
>> In the specific case there is absolutely no use of os.chdir, since you
>> can:
>> - use absolute paths - things like subprocess.Popen accept a cwd
>> argument - at worst you can chdir back to the previous position right
>> after the broken thing that require a certain path that you are calling
>> is run
>
>
> I wouldn't say so much that there's "absolutely no use", it's more that
> there are other, safer, ways to get the same result.
>
> As I understand it, os.chdir is more for the convenience of sys admins
> who want to write quick scripts in Python than a function intended to be
> used in libraries or major applications.
>

I don't disagree, but in my experience few sysadmins use Python; they use
shell scripts. And these seldom do a 'cd' anyway - they will normally
operate irrespective of the current working directory.

It is difficult to think of a sensible use for os.chdir, IMHO.

[Roy Smith]

In article <k0tf8g$adc$1...@news.albasani.net>,

Walter Hurry <walte...@lavabit.com> wrote:

> It is difficult to think of a sensible use for os.chdir, IMHO.

It is true that you can mostly avoid chdir() by building absolute
pathnames, but it's often more convenient to just cd somewhere and use
names relative to that. Fabric (a very cool tool for writing remote
sysadmin scripts), gives you a cd() command which is a context manager,
making it extra convenient.

Also, core files get created in the current directory. Sometimes
daemons will cd to some fixed location to make sure that if they dump
core, it goes in the right place.

On occasion, you run into (poorly designed, IMHO) utilities which insist
of reading or writing a file in the current directory. If you're
invoking one of those, you may have no choice but to chdir() to the
right place before running them.

[andrea crotti]

2012/8/20 Roy Smith <r...@panix.com>:

> --
> http://mail.python.org/mailman/listinfo/python-list


I've done quite a lot of system programming as well, and changing
directory is only a source of possible troubles in general.

If I really have to for some reasons I do this


class TempCd:
"""Change temporarily the current directory
"""
def __init__(self, newcwd):
self.newcwd = newcwd
self.oldcwd = getcwd()

def __enter__(self):
chdir(self.newcwd)
return self

def __exit__(self, type, value, traceback):
chdir(self.oldcwd)


with TempCd('/tmp'):
# now working in /tmp

# now in the original

So it's not that hard to avoid problems..

[Jean-Michel Pichavant]

kj wrote:
> 99.99% of Python programmers
> will find that there's nothing wrong with behavior

[snip]

> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it")
>

[snip]

Don't you find there's something wrong in applying your observation from
2 or 3 replies to your post to 99% of python programmer ? Moreover,
flaming people on their own mailing list won't do you any good, ever, in
any list.

To get back to your original question,

> inspect.getmodule?
Docstring:
Return the module an object was defined in, or None if not found.

As getmodule may return None if not found, you need too handle that
case. There's possibly a weakness in the inspect module when changing
the current directory however nothing wrong with Python having the
remember the intial directory. If you need to remember it, do it youself
(or file a bug to inspect module authors).

JM

[Grant Edwards]

On 2012-08-20, kj <no.e...@please.post> wrote:
> In <roy-CA6D77.1...@news.panix.com> Roy Smith <r...@panix.com> writes:
>
>>In article <k0rj38$2gc$1...@reader1.panix.com>, kj <no.e...@please.post>
>>wrote:
>
>>> As far as I've been able to determine, Python does not remember
>>> (immutably, that is) the working directory at the program's start-up,
>>> or, if it does, it does not officially expose this information.
>
>>Why would you expect that it would? What would it (or you) do with this
>>information?
>

> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__.

What makes you think that the file that contains the definition of
__main__ is the working directory on program startup? That's almost
never the case in my experience.

2) Why should a library expect to be able to access the file
containing the definition of __main__.

> That's a weakness, IMO.

You must be in possession of some rather odd use cases. I've been
writing Python programs for something like 13 years, and I've never
felt any need to have either an immutable record of the startup
directory or access to the file containing the definition of __main__.

> I don't know of any way to fix inspect.getmodule that does not
> involve, directly or indirectly, keeping a stable record of the
> starting directory.

If what you really want is access to the definition of __main__, what
does that have to do with the startup directory?

If you want to know where __main__ is, you can probably figure it out
from /proc/self/<something-or-other>


--
Grant Edwards grant.b.edwards Yow! Am I having fun yet?
at
gmail.com

[Piet van Oostrum]

kj <no.e...@please.post> writes:

> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__. That's a weakness, IMO.

On Unix based systems there is no reliable way to find out this
information. So how could Python reliably supply this?
--
Piet van Oostrum <pi...@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[Neil Hodgson]

Nobody:

> Maybe. On Unix, it's possible that the current directory no longer
> has a pathname.

Its also possible that you do not have permission to successfully
call getcwd. One example of this I have experienced is the OS X sandbox
where you can run Python starting in a directory where you have only
limited permissions.

getcwd works by calling readdir and lstat and looping up from the
current directory to the root to build the whole path so will break
without read permissions on directories:
http://www.opensource.apple.com/source/Libc/Libc-763.13/gen/FreeBSD/getcwd.c

Neil

[Paul Rubin]

alex23 <wuw...@gmail.com> writes:
> Oh my god, how DARE people with EXPERIENCE in a language challenge the
> PRECONCEPTIONS of an AMATEUR!!! HOW DARE THEY?!?!

+1 QOTW :)

[John Roth]

On Sunday, August 19, 2012 7:57:46 PM UTC-6, kj wrote:

> This means that no library code can ever count on, for example,
>
> being able to reliably find the path to the file that contains the
>
> definition of __main__.

If you want to find that, look up __main__ in sys.modules and then look at the __file__ attribute. You need to be a bit careful with this; the import machinery was rewritten for the upcoming 3.3 release, and there were several changes to the module information.

John Roth