As far as I've been able to determine, Python does not remember
(immutably, that is) the working directory at the program's start-up,
or, if it does, it does not officially expose this information.
Does anyone know why this is? Is there a PEP stating the rationale
for it?
Obviously using os.chdir() will change the working directory, and the os.getcwd() will not be the start-up working directory, but if you need the start-up working directory you can get it at start-up and save it in some constant.
In article <k0rj38$2g...@reader1.panix.com>, kj <no.em...@please.post> wrote:
> As far as I've been able to determine, Python does not remember
> (immutably, that is) the working directory at the program's start-up,
> or, if it does, it does not officially expose this information.
Why would you expect that it would? What would it (or you) do with this information?
More to the point, doing a chdir() is not something any library code would do (at least not that I'm aware of), so if the directory changed, it's because some application code did it. In which case, you could have just stored the working directory yourself.
> As far as I've been able to determine, Python does not remember
> (immutably, that is) the working directory at the program's start-up,
> or, if it does, it does not officially expose this information.
> Does anyone know why this is? Is there a PEP stating the rationale
> for it?
> Thanks!
Why would you have a Python Enhancement Proposal to state the rationale for this?
> As far as I've been able to determine, Python does not remember
> (immutably, that is) the working directory at the program's start-up,
> or, if it does, it does not officially expose this information.
> Does anyone know why this is? Is there a PEP stating the rationale
> for it?
> Thanks!
When you start the program, you have a current directory. When you change it, then it is changed. How do you want Python to remember a directory? For example, you can put it into a variable, and use it later. Can you please show us some example code that demonstrates your actual problem?
On Sun, 19 Aug 2012 14:01:15 -0700, Giacomo Alzetta wrote:
> You can obtain the working directory with os.getcwd().
Maybe. On Unix, it's possible that the current directory no longer
has a pathname. As with files, directories can be "deleted" (i.e.
unlinked) even while they're still in use.
Similarly, a directory can be renamed while it's in use, so the current
directory's pathname may have changed even while the current directory
itself hasn't.
In <roy-CA6D77.17031119082...@news.panix.com> Roy Smith <r...@panix.com> writes:
>In article <k0rj38$2g...@reader1.panix.com>, kj <no.em...@please.post> >wrote:
>> As far as I've been able to determine, Python does not remember
>> (immutably, that is) the working directory at the program's start-up,
>> or, if it does, it does not officially expose this information.
>Why would you expect that it would? What would it (or you) do with this >information?
>More to the point, doing a chdir() is not something any library code >would do (at least not that I'm aware of), so if the directory changed, >it's because some application code did it. In which case, you could >have just stored the working directory yourself.
This means that no library code can ever count on, for example,
being able to reliably find the path to the file that contains the
definition of __main__. That's a weakness, IMO. One manifestation
of this weakness is that os.chdir breaks inspect.getmodule, at
least on Unix. If you have some Unix system handy, you can try
the following. First change the argument to os.chdir below to some
valid directory other than your working directory. Then, run the
script, making sure that you refer to it using a relative path.
When I do this on my system (OS X + Python 2.7.3), the script bombs
at the last print statement, because the second call to inspect.getmodule
(though not the first one) returns None.
import inspect
import os
frame = inspect.currentframe()
print inspect.getmodule(frame).__name__
os.chdir('/some/other/directory') # where '/some/other/directory' is
# different from the initial directory
print inspect.getmodule(frame).__name__
...............
% python demo.py
python demo.py
__main__
Traceback (most recent call last):
File "demo.py", line 11, in <module>
print inspect.getmodule(frame).__name__
AttributeError: 'NoneType' object has no attribute '__name__'
I don't know of any way to fix inspect.getmodule that does not
involve, directly or indirectly, keeping a stable record of the
starting directory.
But, who am I kidding? What needs fixing, right? That's not a
bug, that's a feature! Etc.
By now I have learned to expect that 99.99% of Python programmers
will find that there's nothing wrong with behavior like the one
described above, that it is in fact exactly As It Should Be, because,
you see, since Python is the epitome of perfection, it follows
inexorably that any flaw or shortcoming one may *perceive* in Python
is only an *illusion*: the flaw or shortcoming is really in the
benighted programmer, for having stupid ideas about programming
(i.e. any idea that may entail that Python is not *gasp* perfect).
Pardon my cynicism, but the general vibe from the replies I've
gotten to my post (i.e. "if Python ain't got it, it means you don't
need it") is entirely in line with these expectations.
On Sun, Aug 19, 2012 at 9:57 PM, kj <no.em...@please.post> wrote:
> By now I have learned to expect that 99.99% of Python programmers
> will find that there's nothing wrong with behavior like the one
> described above, that it is in fact exactly As It Should Be, because,
> you see, since Python is the epitome of perfection, it follows
> inexorably that any flaw or shortcoming one may *perceive* in Python
> is only an *illusion*: the flaw or shortcoming is really in the
> benighted programmer, for having stupid ideas about programming
> (i.e. any idea that may entail that Python is not *gasp* perfect).
> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.
Since you have no respect for the people you're writing to, why
bother? I know I certainly have no desire to spend any time at all on
your problem when you say things like that. Perhaps you're looking
for for the argument clinic instead?
On Monday, 20 August 2012 11:57:46 UTC+10, kj wrote:
> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__. That's a weakness, IMO.
No, it's not. It's a _strength_. If you've written a library that requires absolute knowledge of its installed location in order for its internals to work, then I'm not installing your library.
> When I do this on my system (OS X + Python 2.7.3), the script bombs
> at the last print statement, because the second call to inspect.getmodule
> (though not the first one) returns None.
So, uh, do something sane like test for the result of inspect.getmodule _before_ trying to do something invalid to it?
> I don't know of any way to fix inspect.getmodule that does not
> involve, directly or indirectly, keeping a stable record of the
> starting directory.
Then _that is the answer_. YOU need to keep "a stable record":
> But, who am I kidding? What needs fixing, right? That's not a
> bug, that's a feature! Etc.
Right. Because that sort of introspection of objects is rare, why burden the _entire_ language with an obligation that is only required in a few places?
> By now I have learned to expect that 99.99% of Python programmers
> will find that [blah blah blah, whine whine whine]. > Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.
Oh my god, how DARE people with EXPERIENCE in a language challenge the PRECONCEPTIONS of an AMATEUR!!! HOW DARE THEY?!?!
My apologies for any double-ups and bad formatting. The new Google Groups interface seems to have effectively shat away decades of UX for something that I can only guess was generated randomly.
On Monday, 20 August 2012 12:57:44 UTC+10, alex23 wrote:
> a library that requires absolute knowledge of its > installed location in order for its internals to work
That should read: "a library that requires derivation of its installed location from the current working directory in order" etc
>>> As far as I've been able to determine, Python does not remember
>>> (immutably, that is) the working directory at the program's start-up,
>>> or, if it does, it does not officially expose this information.
[...]
> This means that no library code can ever count on, for example, being
> able to reliably find the path to the file that contains the definition
> of __main__.
Possibly not in the face of chmod.
> That's a weakness, IMO.
Certainly.
> One manifestation of this
> weakness is that os.chdir breaks inspect.getmodule, at least on Unix.
At the very least, it can do so under some circumstances.
By the way, inspect.currentframe is quite problematic. Although it is given a public name, it is just a thin wrapper around sys._getframe which is not just *named* as a private function, but also states clearly in the documentation:
"This function should be used for internal and specialized purposes only."
So I'm not sure that fixing problems with inspect.currentframe will be high on anyone's priority.
[...]
> I don't know of any way to fix inspect.getmodule that does not involve,
> directly or indirectly, keeping a stable record of the starting
> directory.
I'm not convinced that this is necessary for the example you give. Perhaps modules need to store absolute pathnames rather than relative ones?
os.chdir probably breaks a lot of things. Python operates under the assumption that the working directory does not change. If you change it, it's your responsibility to deal with it.
Apart from shell scripting languages, do other languages cope well with having the working directory changed?
> But, who am I kidding? What needs fixing, right? That's not a bug,
> that's a feature! Etc.
Not so much. More of, "yes, that's a bug, but it's not an important bug. If you have a patch for fixing it, we'll consider it, but if you don't, we've got about two thousand more important bugs and features to get through first."
Your chances of getting this fixed will be *much* improved if you can demonstrate a genuine problem that can't easily be fixed by just telling the programmer "don't change working directories, or if you do, you need to change back again before doing X, Y or Z".
Personally, I think that trying to fix all the things that break if you chdir is not worthwhile, but having Python record the directory you started with in (say) sys.startingdirectory might be a worthwhile feature request.
On Monday, 20 August 2012 12:57:44 UTC+10, alex23 wrote:
> If you've written a library that requires absolute > knowledge of its installed location in order for its > internals to work, then I'm not installing your library.
That should read: "requires the ability to derive absolute knowledge of its installed location for an application's current working directory".
There's nothing wrong with a library knowing where shit is. But again, that should in NO WAY be bound to the application working directory.
On Monday, 20 August 2012 13:38:03 UTC+10, Steven D'Aprano wrote:
> Not so much. More of, "yes, that's a bug, but it's not an important bug. > If you have a patch for fixing it, we'll consider it, but if you don't, > we've got about two thousand more important bugs and features to get > through first."
I'm not really convinced it's a bug, though. The issue only happens if the module is the main application, so wrapping this behind a launcher will resolve the issue. Given that you can manually provide the filepath to inspect, I'd say it's a known limitation with a workaround.
> In <roy-CA6D77.17031119082...@news.panix.com> Roy Smith <r...@panix.com> writes:
>> In article <k0rj38$2g...@reader1.panix.com>, kj <no.em...@please.post>
>> wrote:
>>> As far as I've been able to determine, Python does not remember
>>> (immutably, that is) the working directory at the program's start-up,
>>> or, if it does, it does not officially expose this information.
>> Why would you expect that it would? What would it (or you) do with this
>> information?
>> More to the point, doing a chdir() is not something any library code
>> would do (at least not that I'm aware of), so if the directory changed,
>> it's because some application code did it. In which case, you could
>> have just stored the working directory yourself.
> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__. That's a weakness, IMO. One manifestation
> of this weakness is that os.chdir breaks inspect.getmodule, at
> least on Unix. If you have some Unix system handy, you can try
> the following. First change the argument to os.chdir below to some
> valid directory other than your working directory. Then, run the
> script, making sure that you refer to it using a relative path.
> When I do this on my system (OS X + Python 2.7.3), the script bombs
> at the last print statement, because the second call to inspect.getmodule
> (though not the first one) returns None.
> import inspect
> import os
> frame = inspect.currentframe()
> print inspect.getmodule(frame).__name__
> os.chdir('/some/other/directory') # where '/some/other/directory' is
> # different from the initial directory
> print inspect.getmodule(frame).__name__
> ...............
> % python demo.py
> python demo.py
> __main__
> Traceback (most recent call last):
> File "demo.py", line 11, in <module>
> print inspect.getmodule(frame).__name__
> AttributeError: 'NoneType' object has no attribute '__name__'
> I don't know of any way to fix inspect.getmodule that does not
> involve, directly or indirectly, keeping a stable record of the
> starting directory.
> But, who am I kidding? What needs fixing, right? That's not a
> bug, that's a feature! Etc.
> By now I have learned to expect that 99.99% of Python programmers
> will find that there's nothing wrong with behavior like the one
> described above, that it is in fact exactly As It Should Be, because,
> you see, since Python is the epitome of perfection, it follows
> inexorably that any flaw or shortcoming one may *perceive* in Python
> is only an *illusion*: the flaw or shortcoming is really in the
> benighted programmer, for having stupid ideas about programming
> (i.e. any idea that may entail that Python is not *gasp* perfect).
> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.
I see, I see, I get the picture. You're in the "The OS is flawed so I expect the Python core developers to do my work for me by producing code that gets around every known flaw in every supported OS" club. Somehow I don't think that will happen.
> My apologies for any double-ups and bad formatting. The new Google Groups interface seems to have effectively shat away decades of UX for something that I can only guess was generated randomly.
It's very useful for reporting spam. Otherwise Thunderbird is go :)
> In <roy-CA6D77.17031119082...@news.panix.com> Roy Smith <r...@panix.com> writes> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__. That's a weakness, IMO. One manifestation
> of this weakness is that os.chdir breaks inspect.getmodule, at
> least on Unix. If you have some Unix system handy, you can try
> the following. First change the argument to os.chdir below to some
> valid directory other than your working directory. Then, run the
> script, making sure that you refer to it using a relative path.
> When I do this on my system (OS X + Python 2.7.3), the script bombs
> at the last print statement, because the second call to inspect.getmodule
> (though not the first one) returns None.
> import inspect
> import os
> frame = inspect.currentframe()
> print inspect.getmodule(frame).__name__
> os.chdir('/some/other/directory') # where '/some/other/directory' is
> # different from the initial directory
> print inspect.getmodule(frame).__name__
> ...............
> % python demo.py
> python demo.py
> __main__
> Traceback (most recent call last):
> File "demo.py", line 11, in <module>
> print inspect.getmodule(frame).__name__
> AttributeError: 'NoneType' object has no attribute '__name__'
>..
As in many other cases the programming language can't possibly act
safely on all the possible stupid things that the programmer wants to
do, and not understanding how an operating system works doesn't help
either..
In the specific case there is absolutely no use of os.chdir, since you
can:
- use absolute paths
- things like subprocess.Popen accept a cwd argument
- at worst you can chdir back to the previous position right after the
broken thing that require a certain path that you are calling is run
On Mon, 20 Aug 2012 12:56:42 +0100, andrea crotti wrote:
> In the specific case there is absolutely no use of os.chdir, since you
> can:
> - use absolute paths
> - things like subprocess.Popen accept a cwd argument - at worst you can
> chdir back to the previous position right after the broken thing that
> require a certain path that you are calling is run
I wouldn't say so much that there's "absolutely no use", it's more that there are other, safer, ways to get the same result.
As I understand it, os.chdir is more for the convenience of sys admins who want to write quick scripts in Python than a function intended to be used in libraries or major applications.
An interesting question is, what do other languages do in this case?
On Mon, 20 Aug 2012 13:14:02 +0000, Steven D'Aprano wrote:
> On Mon, 20 Aug 2012 12:56:42 +0100, andrea crotti wrote:
>> In the specific case there is absolutely no use of os.chdir, since you
>> can:
>> - use absolute paths - things like subprocess.Popen accept a cwd
>> argument - at worst you can chdir back to the previous position right
>> after the broken thing that require a certain path that you are calling
>> is run
> I wouldn't say so much that there's "absolutely no use", it's more that
> there are other, safer, ways to get the same result.
> As I understand it, os.chdir is more for the convenience of sys admins
> who want to write quick scripts in Python than a function intended to be
> used in libraries or major applications.
I don't disagree, but in my experience few sysadmins use Python; they use shell scripts. And these seldom do a 'cd' anyway - they will normally operate irrespective of the current working directory.
It is difficult to think of a sensible use for os.chdir, IMHO.
In article <k0tf8g$ad...@news.albasani.net>,
Walter Hurry <walterhu...@lavabit.com> wrote:
> It is difficult to think of a sensible use for os.chdir, IMHO.
It is true that you can mostly avoid chdir() by building absolute pathnames, but it's often more convenient to just cd somewhere and use names relative to that. Fabric (a very cool tool for writing remote sysadmin scripts), gives you a cd() command which is a context manager, making it extra convenient.
Also, core files get created in the current directory. Sometimes daemons will cd to some fixed location to make sure that if they dump core, it goes in the right place.
On occasion, you run into (poorly designed, IMHO) utilities which insist of reading or writing a file in the current directory. If you're invoking one of those, you may have no choice but to chdir() to the right place before running them.
> In article <k0tf8g$ad...@news.albasani.net>,
> Walter Hurry <walterhu...@lavabit.com> wrote:
>> It is difficult to think of a sensible use for os.chdir, IMHO.
> It is true that you can mostly avoid chdir() by building absolute
> pathnames, but it's often more convenient to just cd somewhere and use
> names relative to that. Fabric (a very cool tool for writing remote
> sysadmin scripts), gives you a cd() command which is a context manager,
> making it extra convenient.
> Also, core files get created in the current directory. Sometimes
> daemons will cd to some fixed location to make sure that if they dump
> core, it goes in the right place.
> On occasion, you run into (poorly designed, IMHO) utilities which insist
> of reading or writing a file in the current directory. If you're
> invoking one of those, you may have no choice but to chdir() to the
> right place before running them.
> --
> http://mail.python.org/mailman/listinfo/python-list
I've done quite a lot of system programming as well, and changing
directory is only a source of possible troubles in general.
If I really have to for some reasons I do this
class TempCd:
"""Change temporarily the current directory
"""
def __init__(self, newcwd):
self.newcwd = newcwd
self.oldcwd = getcwd()
kj wrote:
> 99.99% of Python programmers
> will find that there's nothing wrong with behavior [snip]
> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it")
[snip]
Don't you find there's something wrong in applying your observation from 2 or 3 replies to your post to 99% of python programmer ? Moreover, flaming people on their own mailing list won't do you any good, ever, in any list.
To get back to your original question,
> inspect.getmodule?
Docstring:
Return the module an object was defined in, or None if not found.
As getmodule may return None if not found, you need too handle that case. There's possibly a weakness in the inspect module when changing the current directory however nothing wrong with Python having the remember the intial directory. If you need to remember it, do it youself (or file a bug to inspect module authors).
>>> As far as I've been able to determine, Python does not remember
>>> (immutably, that is) the working directory at the program's start-up,
>>> or, if it does, it does not officially expose this information.
>>Why would you expect that it would? What would it (or you) do with this >>information?
> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__.
What makes you think that the file that contains the definition of
__main__ is the working directory on program startup? That's almost
never the case in my experience.
2) Why should a library expect to be able to access the file
containing the definition of __main__.
> That's a weakness, IMO.
You must be in possession of some rather odd use cases. I've been
writing Python programs for something like 13 years, and I've never
felt any need to have either an immutable record of the startup
directory or access to the file containing the definition of __main__.
> I don't know of any way to fix inspect.getmodule that does not
> involve, directly or indirectly, keeping a stable record of the
> starting directory.
If what you really want is access to the definition of __main__, what
does that have to do with the startup directory?
If you want to know where __main__ is, you can probably figure it out
from /proc/self/<something-or-other>
-- Grant Edwards grant.b.edwards Yow! Am I having fun yet?
at gmail.com
kj <no.em...@please.post> writes:
> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__. That's a weakness, IMO.
On Unix based systems there is no reliable way to find out this
information. So how could Python reliably supply this?
-- Piet van Oostrum <p...@vanoostrum.org>
WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4]
