os.access() under windows
Yann Leboulanger
Under Windows XP os.access has a strange behaviour:
I create a folder test under e:
then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
Now I move My Documents to this e:\test folder
Then os.access('e:\\test', os.W_OK) returns False !!
but this works:
f = open('e:\\test\\test', 'a')
f.write('abc')
f.close()
So why os.access returns False ?
--
Yann
David Tweet
permission) approach works better for this kind of thing.
try:
f = open('e:\\test\\test', 'a')
f.write('abc')
f.close()
except IOError:
print "couldn't write test file, continuing..."
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
-David
Yann Leboulanger
yep but the test file I try to create to know if I can write there may
be a folder for exemple, and the open() will fail.
So this can be considered as a bug in os module?
"Martin v. Löwis"
>
> then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
>
> Now I move My Documents to this e:\test folder
>
> Then os.access('e:\\test', os.W_OK) returns False !!
This description is, unfortunately, too imprecise to allow reproducing
that effect. What precisely do you mean by "I move My Documents to
this e:\test folder"?
Regards,
Martin
Yann Leboulanger
I Right click on "My Documents" folder, and change the path to e:\test
there.
So that "My documents" folder points to e:\test
Tim Golden
Python uses the GetFileAttributesW API call to determine
access to the path you pass in. It then tests the return
against the FILE_ATTRIBUTE_READONLY flag and returns False
if you asked for Write and the Readonly flag is set.
The problem seems to be twofold: whereas a normal directory
(such as e:\temp) will not usually have its readonly flag set,
on a special directory such as the My Documents folder the flag
seems to be set by the system; also, the READONLY flag on a
directory refers to its deleteability, not to its writeablility.
At least, according to:
http://msdn2.microsoft.com/en-us/library/aa364944.aspx
I suspect this constitutes a bug in os.access which
should do some more interpretation of the results. But
no doubt Martin von L can comment with more authority
than I on this one.
TJG
Tim Golden
> access to the path you pass in. It then tests the return
> against the FILE_ATTRIBUTE_READONLY flag and returns False
> if you asked for Write and the Readonly flag is set.
>
> The problem seems to be twofold: whereas a normal directory
> (such as e:\temp) will not usually have its readonly flag set,
> on a special directory such as the My Documents folder the flag
> seems to be set by the system; also, the READONLY flag on a
> directory refers to its deleteability, not to its writeablility.
> At least, according to:
>
> http://msdn2.microsoft.com/en-us/library/aa364944.aspx
>
> I suspect this constitutes a bug in os.access which
> should do some more interpretation of the results. But
> no doubt Martin von L can comment with more authority
> than I on this one.
>
> TJG
And just to complicate matters, it seems that readonly on
a folder has an entirely special meaning. At least, according
to this:
http://support.microsoft.com/kb/326549
Goodness knows what we're supposed to do with that.
Part of the issue here is that we're using a Unix-style
access API against Windows-style filesystem semantics.
According to the man pages, it looks as though it basically
checks the standard security bits against whatever you're
asking for for your user. On Windows, the security semantics
are quite different and you have to make some kind of
decision as to what the os.access means.
I'm happy to contribute a doc patch if I can imagine what
exactly to write.
TJG
Yann Leboulanger
>
> I'm happy to contribute a doc patch if I can imagine what
> exactly to write.
>
"Don't use it under windows, always consider it's True"?
Maybe it would be a good idea to remove support for windows for this
function, or make it always return True?
Current behaviour is worth that nothing, access() return False but
open() doesn't fail ...
--
Yann
Yann Leboulanger
>
> I'm happy to contribute a doc patch if I can imagine what
> exactly to write.
>
"Don't use it under windows, always consider it's True"?
Tim Golden
> Tim Golden a écrit :
>>
>> I'm happy to contribute a doc patch if I can imagine what
>> exactly to write.
>>
>
> "Don't use it under windows, always consider it's True"?
Well, that's not the case for files: if you set your
file's readonly attribute to True, then os.access (W_OK)
will return False and you won't be able to write to the
file:
<code>
import os
open ("temp.tmp", "w").close ()
os.access ("temp.tmp", os.W_OK)
os.system ("attrib +r temp.tmp")
os.access ("temp.tmp", os.W_OK)
open ("temp.tmp", "w").close ()
os.system ("attrib -r temp.tmp")
os.access ("temp.tmp", os.W_OK)
</code>
> Maybe it would be a good idea to remove support for windows for this
> function, or make it always return True?
The only issue (at least, the only one we're discussing here) is:
If os.W_OK on a directory returns True, that won't stop you writing
into that directory.
> Current behaviour is worth that nothing, access() return False but
> open() doesn't fail ...
To be precise: open () on a file within that directory doesn't fail.
Personally, I sympathise with you here. Python comes from a Unix
background and, unsurprisingly, it offers all the major Unix
system calls. Since Windows historically offered a Posix layer
which mapped them to *something*[1], the developers simply called
those under the covers. And the basic policy was: whatever Windows
passes back to Python, Python passes on to you.
Later (mid-2006, I think) some or all of these Posix-layer functions
were replaced by native Win32 APIs. At that point, it arguably became
Python's responsibility to define semantics. But it's a fuzzy sort of
area. I think a doc patch which said something like: "Calls
FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
might meet the case, although a bit of a cop-out.
TJG
[1] http://msdn2.microsoft.com/en-us/library/1w06ktdy(VS.80).aspx
Yann Leboulanger
> Well, that's not the case for files: if you set your
> file's readonly attribute to True, then os.access (W_OK)
> will return False and you won't be able to write to the
> file:
> The only issue (at least, the only one we're discussing here) is:
> If os.W_OK on a directory returns True, that won't stop you writing
> into that directory.
>
>
> To be precise: open () on a file within that directory doesn't fail.
Yep sorry I was a bit too expeditive :)
> Personally, I sympathise with you here. Python comes from a Unix
> background and, unsurprisingly, it offers all the major Unix
> system calls. Since Windows historically offered a Posix layer
> which mapped them to *something*[1], the developers simply called
> those under the covers. And the basic policy was: whatever Windows
> passes back to Python, Python passes on to you.
>
> Later (mid-2006, I think) some or all of these Posix-layer functions
> were replaced by native Win32 APIs. At that point, it arguably became
> Python's responsibility to define semantics. But it's a fuzzy sort of
> area. I think a doc patch which said something like: "Calls
> FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
> might meet the case, although a bit of a cop-out.
>
Ok thanks for all those information, I'll remove the call to os.access()
on folders for windows in my application.
--
Yann
Yann Leboulanger
> Well, that's not the case for files: if you set your
> file's readonly attribute to True, then os.access (W_OK)
> will return False and you won't be able to write to the
> file:
> The only issue (at least, the only one we're discussing here) is:
> If os.W_OK on a directory returns True, that won't stop you writing
> into that directory.
>
>
> To be precise: open () on a file within that directory doesn't fail.
Yep sorry I was a bit too expeditive :)
> Personally, I sympathise with you here. Python comes from a Unix
> background and, unsurprisingly, it offers all the major Unix
> system calls. Since Windows historically offered a Posix layer
> which mapped them to *something*[1], the developers simply called
> those under the covers. And the basic policy was: whatever Windows
> passes back to Python, Python passes on to you.
>
> Later (mid-2006, I think) some or all of these Posix-layer functions
> were replaced by native Win32 APIs. At that point, it arguably became
> Python's responsibility to define semantics. But it's a fuzzy sort of
> area. I think a doc patch which said something like: "Calls
> FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
> might meet the case, although a bit of a cop-out.
>
Ok thanks for all those information, I'll remove the call to os.access()
Tim Golden
> Ok thanks for all those information, I'll remove the call to os.access()
> on folders for windows in my application.
>
FWIW, I think it's worth bearing in mind what was said
earlier in this thread: it's easier to ask forgiveness
than permission. Technically, even if os.access did
exactly what you expected, there's nothing to stop the
access changing between os.access (...) and open (...).
Unless you have reason not to, it's considered perfectly
good practice in Python to try/except the open ():
<code>
try:
f = open ("e:/test/test.tst", "w")
except (IOError, WindowsError):
print "Something went wrong"
else:
"Do whatever"
</code>
TJG
"Martin v. Löwis"
Just in case it's not clear what Tim is getting at:
if a folder is marked read-only on Windows, it doesn't mean
that you can only read from it. The read-only bit is a legacy
thing, anyway, since you are supposed to use ACLs to mark
a folder as read-only (by only granting write access to those
who are supposed to write)
As the read-only bit is otherwise unused, Explorer uses it
to mark folders as being special, such a My Documents.
So by redirecting My Documents, you set the read-only bit
on the folder, causing access() to claim that write access
is not granted.
> Part of the issue here is that we're using a Unix-style
> access API against Windows-style filesystem semantics.
> According to the man pages, it looks as though it basically
> checks the standard security bits against whatever you're
> asking for for your user. On Windows, the security semantics
> are quite different and you have to make some kind of
> decision as to what the os.access means.
>
> I'm happy to contribute a doc patch if I can imagine what
> exactly to write.
It would be possible to fix this specific case, by always
returning True for directories; and perhaps I'll do that for
2.5.2.
A more general issue is whether the ACL should also be
taken into account. This would involve calling things like
OpenThreadToken, MapGenericMask, and AccessCheck. These are
all functions from the NT security API, and unavailable
on Win9x - which is the likely reason why the MS CRT did
not use them, either. Providing a proper access() implementation
for NT+ then only becomes possible for 2.6 (where W9x
is no longer supported).
Regards,
Martin
P.S. I would never guessed that "move My Documents to test"
doesn't mean "drag-and-drop My Documents into test", but
"redirect My Documents to the test folder".
Yann Leboulanger
> Just in case it's not clear what Tim is getting at:
>
> if a folder is marked read-only on Windows, it doesn't mean
> that you can only read from it. The read-only bit is a legacy
> thing, anyway, since you are supposed to use ACLs to mark
> a folder as read-only (by only granting write access to those
> who are supposed to write)
>
> As the read-only bit is otherwise unused, Explorer uses it
> to mark folders as being special, such a My Documents.
> So by redirecting My Documents, you set the read-only bit
> on the folder, causing access() to claim that write access
> is not granted.
>
> returning True for directories; and perhaps I'll do that for
> 2.5.2.
>
> A more general issue is whether the ACL should also be
> taken into account. This would involve calling things like
> OpenThreadToken, MapGenericMask, and AccessCheck. These are
> all functions from the NT security API, and unavailable
> on Win9x - which is the likely reason why the MS CRT did
> not use them, either. Providing a proper access() implementation
> for NT+ then only becomes possible for 2.6 (where W9x
> is no longer supported).
>
Great, thanks for those information!
> P.S. I would never guessed that "move My Documents to test"
> doesn't mean "drag-and-drop My Documents into test", but
> "redirect My Documents to the test folder".
Right, sorry :/
--
Yann
Tim Golden
> It would be possible to fix this specific case, by always
> returning True for directories; and perhaps I'll do that for
> 2.5.2.
Martin. Could you confirm that the outline below correctly
describes the behaviour of the os.access function under
Windows, please? If you confirm its accuracy, I'll write it
up as a docs patch against the development docs.
"""
The os.access function originates on Unix where it straightforwardly
reflects the permission-bit model of security. On Windows, that
model does not obtain so the Python function compromises as follows:
+ If the path does not exist or if access if forbidden, return False
+ Requests for access mode R_OK and X_OK will always return True
if the path can be accessed at all.
+ If the path refers to a directory, return True since the readonly
attribute on a directory does not prevent writing files.
+ If W_OK access is requested and the file's readonly flag is set,
return False.
+ If W_OK access is requested and the file's readonly flag is not
set, return True.
NB None of the checks explicitly take into account security
ACLs.
"""
> A more general issue is whether the ACL should also be
> taken into account. This would involve calling things like
> OpenThreadToken, MapGenericMask, and AccessCheck. These are
> all functions from the NT security API, and unavailable
> on Win9x - which is the likely reason why the MS CRT did
> not use them, either. Providing a proper access() implementation
> for NT+ then only becomes possible for 2.6 (where W9x
> is no longer supported).
Agreed. I think I'd like to see that happen, but I have to
down several strengthening drinks every time I approach the
Windows Security API!
TJG
"Martin v. Löwis"
> describes the behaviour of the os.access function under
> Windows, please?
It's correct for Python 2.5.2 and 2.6; for 2.5.1 (as discussed)
the test "if directory:return True" was not implemented.
Notice that the first sentence:
"If the path does not exist or if access if forbidden, return False"
is confusing. It seems to say "access() returns false if access is
forbidden". I think you are referring to the directory (in the sense
of os.path.dirname) of the path to be tested.
> Agreed. I think I'd like to see that happen, but I have to
> down several strengthening drinks every time I approach the
> Windows Security API!
There is a good tutorial example in the AccessCheck documentation.
If you leave out the impersonation part, it should be
straight-forward to apply this to Python. Contributions are
welcome.
Regards,
Martin
Tim Golden
>> Martin. Could you confirm that the outline below correctly
>> describes the behaviour of the os.access function under
>> Windows, please?
>
> It's correct for Python 2.5.2 and 2.6; for 2.5.1 (as discussed)
> the test "if directory:return True" was not implemented.
Thanks. Version dependent stuff noted.
> Notice that the first sentence:
>
> "If the path does not exist or if access if forbidden, return False"
>
> is confusing. It seems to say "access() returns false if access is
> forbidden". I think you are referring to the directory (in the sense
> of os.path.dirname) of the path to be tested.
Now, ironically, I'm confused by your recap :) What I meant to say was
that the os.access function as implemented under Windows returns False
if the path in question (say, "x:\someones-private-docs\diary.doc") was
inaccessible to the process invoking os.access by virtue of file
system permissions. (Or, even, that it simply didn't exist).
TJG
"Martin v. Löwis"
> that the os.access function as implemented under Windows returns False
> if the path in question (say, "x:\someones-private-docs\diary.doc") was
> inaccessible to the process invoking os.access by virtue of file
> system permissions. (Or, even, that it simply didn't exist).
Isn't that exactly the question that access() is trying to answer?
(whether the file is inaccessible?)
Then the reader could say "if it's inaccessible, return False,
otherwise, return True". It suggests that we already have a mechanism
to determine accessibility.
The case you are referring to is when GetFileAttributes fails, with
a permission error, right?
I'm not quite sure why it could fail in cases where the file is
present, in particular, if the user has the privilege to bypass
traversal checking.
Regards,
Martin
Tim Golden
>> Now, ironically, I'm confused by your recap :) What I meant to say was
>> that the os.access function as implemented under Windows returns False
>> if the path in question (say, "x:\someones-private-docs\diary.doc") was
>> inaccessible to the process invoking os.access by virtue of file
>> system permissions. (Or, even, that it simply didn't exist).
>
> Isn't that exactly the question that access() is trying to answer?
> (whether the file is inaccessible?)
> Then the reader could say "if it's inaccessible, return False,
> otherwise, return True". It suggests that we already have a mechanism
> to determine accessibility.
Up to a point: this meets the case where we fail to access
the file at all (for read or write or whatever). But what
about where we can read the directory entry, and the
read-only attribute isn't set? At present, we'll return
True to a W_OK access check in these circs, but this user
might in fact be denied write access by the ACLs. (In fact,
they might even be denied read access, since I imagine we
only need access to the directory entry to check the
attributes).
Have I missed something?
BTW I can't see a tutorial in the AccessCheck docs here:
http://msdn2.microsoft.com/en-us/library/aa374815.aspx
or in the SDK help file. Were you referring to a different
set of docs? I'm girding my loins to provide a patch if I
can!
TJG
"Martin v. Löwis"
> the file at all (for read or write or whatever). But what
> about where we can read the directory entry, and the
> read-only attribute isn't set? At present, we'll return
> True to a W_OK access check in these circs, but this user
> might in fact be denied write access by the ACLs. (In fact,
> they might even be denied read access, since I imagine we
> only need access to the directory entry to check the
> attributes).
That's my question exactly: what do you need to do
GetFileAttributes successfully?
In a POSIX world, you need read permission on the directory.
In Windows, with the "bypass-traversal-check" privilege,
you only need read permission on the directory if you want
to list it, not to access a file in the directory. Is it
actually possible for GetFileAttributes to ever fail for
security reasons?
> BTW I can't see a tutorial in the AccessCheck docs here:
>
> http://msdn2.microsoft.com/en-us/library/aa374815.aspx
>
> or in the SDK help file. Were you referring to a different
> set of docs?
I meant
http://msdn2.microsoft.com/en-us/library/aa379648.aspx
and thought it was linked from AccessCheck, but
it apparently isn't.
Regards,
Martin
Tim Golden
> In a POSIX world, you need read permission on the directory.
> In Windows, with the "bypass-traversal-check" privilege,
> you only need read permission on the directory if you want
> to list it, not to access a file in the directory. Is it
> actually possible for GetFileAttributes to ever fail for
> security reasons?
After a little experimentation I can confirm:
* R_OK: A process with bypass-traversal-check priv. enabled
doesn't need any access to intervening directories in order
to get the attributes of a file within them. This means that
our existing R_OK result is accurate for any file: if we can
get its attributes then you can open the file for reading.
* W_OK: If a user has *only* read permission on a file
(regardless of the intervening directories), we'll still return
True for a W_OK check, as long the file doesn't have its read-only
bit set. This means that it's possible for os.access to return True
for a W_OK check on a file which can't then be opened for, say,
appending.
* X_OK: No idea what we should do with this.
In short, no further fiddling with the existing GetFileAttributes
solution is likely to achieve anything useful. The way to go would
be to use an AccessCheck solution which mirrors the approach used
on *nix: we ask the OS to check for r/w/x and return whatever it
returns. The exact semantics of that (eg on directories) are o/s
dependent and you need to refer to the docs for more info.
I hope to make time in the next few days to put forward a patch
to implement this in posixmodule.c.
TJG
Tim Golden
"Martin v. Löwis"
> doesn't need any access to intervening directories in order
> to get the attributes of a file within them. This means that
> our existing R_OK result is accurate for any file: if we can
> get its attributes then you can open the file for reading.
That cannot be. There is still an ACL on the file itself which
is considered when the file is opened for reading, namely,
the "List Folder / Read Data" permission. It may be that you need
the "Read Attributes" permission to have GetFileAttributes
succeed, but that is independent from "read data".
> * W_OK: If a user has *only* read permission on a file
> (regardless of the intervening directories), we'll still return
> True for a W_OK check, as long the file doesn't have its read-only
> bit set. This means that it's possible for os.access to return True
> for a W_OK check on a file which can't then be opened for, say,
> appending.
Or any other kind of writing.
> * X_OK: No idea what we should do with this.
Well, there is the "Traverse Folder / Execute File" permission that
should be checked against (but currently isn't).
Regards,
Martin