Does This Scare You?

[Lawrence D’Oliveiro]

Python 3.5.2+ (default, Aug 5 2016, 08:07:14)
[GCC 6.1.1 20160724] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import PureWindowsPath
>>> PureWindowsPath("prn").is_reserved()
True
>>> PureWindowsPath("prn.doc").is_reserved()
True
>>> PureWindowsPath("com9.exe").is_reserved()
True
>>> PureWindowsPath("c:/my documents/prn.doc").is_reserved()
True

[Chris Angelico]

When was the last time you wanted to create a file with a reserved
name? Paths, drive letters, file extensions, don't matter. All that
matters is the base name.

Not a Python issue; they're reserved by Windows.

ChrisA

[Wildman]

Since I am fairly new to Python, I realize there is much that I
still don't know but I don't understand how Windows can have
reserved names on a Linux system. What am I missing?

--
<Wildman> GNU/Linux user #557453
The cow died so I don't need your bull!

[Chris Angelico]

On Sat, Aug 20, 2016 at 11:11 AM, Wildman via Python-list
<pytho...@python.org> wrote:
> Since I am fairly new to Python, I realize there is much that I
> still don't know but I don't understand how Windows can have
> reserved names on a Linux system. What am I missing?

The PureWindowsPath class is specifically working with the Windows
file system rules. Those names aren't reserved under Linux, but you
can still ask the pathlib module whether or not they *would be*
reserved under Windows. This is very handy if you're planning to
create files and don't know whether they'd work or not - you don't
need to grab a Windows machine (virtual or physical) to test stuff.

ChrisA

[Wildman]

OK. I understand. Python is after all cross-platform so that
makes perfect sense. Thanks.

--
<Wildman> GNU/Linux user #557453

The voices in my head may not be real
but, they have some good ideas.

[Michael Torrie]

Which part are you getting at? That Windows treats certain filenames as
reserved (a known gotcha that has existed for decades) or that Python
allows you to test whether a path is valid in Windows?

[eryk sun]

On Sun, Aug 21, 2016 at 8:03 PM, Michael Torrie <tor...@gmail.com> wrote:
> On 08/19/2016 05:42 PM, Lawrence D’Oliveiro wrote:

> Which part are you getting at? That Windows treats certain filenames as
> reserved (a known gotcha that has existed for decades) or that Python
> allows you to test whether a path is valid in Windows?

To me it's scary that this check misses cases because it's trying to
be cross-platform instead of simply relying on GetFullPathName to do
the work. For example, it misses at least the following cases:

Optional trailing colon:

>>> pathlib.Path('C:/foo/NUL:').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL:'))
\\.\NUL

Trailing spaces:

>>> pathlib.Path('C:/foo/NUL ').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL '))
\\.\NUL

Trailing spaces followed by a file extension:

>>> pathlib.Path('C:/foo/NUL .txt').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL .txt'))
\\.\NUL

It's also a bit disappointing that the author of this function claims
in a comment that "foo/NUL" isn't reserved yet decides to "err on the
side of caution" (obviously not enough). Of course "foo/NUL" is
reserved:

>>> print(os.path._getfullpathname('foo/NUL'))
\\.\NUL

I think what happened is that the author tested by calling open() on a
non-existing path. DOS device names are only reserved for existing
directories, in order to return an error for an invalid path. The
difference is how RtlGetFullPathName_Ustr is called. When
GetFullPathName calls the latter function it doesn't care whether or
not the path is valid. On the other hand, RtlDosPathNameToNtPathName_*
calls RtlGetFullPathName_Ustr with a parameter to check for an invalid
path, which makes the path normalization fail. For example:

Existing directory:

>>> os.path.exists('C:/Temp')
True
>>> f = open('C:/Temp/NUL')

Query the device name:

>>> hFile = msvcrt.get_osfhandle(f.fileno())
>>> ntdll.NtQueryObject(hFile, 1, byref(name), sizeof(name), None)
0
>>> print(name.Buffer[:name.Length//2])
\Device\Null

(\\.\NUL, i.e. \GLOBAL??\NUL, is a symbolic link to NT's \Device\Null.)

Non-existing directory:

>>> os.path.exists('C:/Spam')
False
>>> g = open('C:/Spam/NUL')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Spam/NUL'

[Chris Angelico]

On Mon, Aug 22, 2016 at 10:38 AM, eryk sun <ery...@gmail.com> wrote:
> To me it's scary that this check misses cases because it's trying to
> be cross-platform instead of simply relying on GetFullPathName to do
> the work.

"Trying to be" cross-platform? The point of these path modules is to
*be* cross-platform. I can't call Windows APIs on my Linux box (short
of messing around with VMs or Wine or something, which are
dependencies that Python doesn't need). So yes, these *are*
cross-platform, and that's not scary, that's deliberate design.

ChrisA

[eryk sun]

On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico <ros...@gmail.com> wrote:
> "Trying to be" cross-platform? The point of these path modules is to
> *be* cross-platform. I can't call Windows APIs on my Linux box (short
> of messing around with VMs or Wine or something, which are
> dependencies that Python doesn't need). So yes, these *are*
> cross-platform, and that's not scary, that's deliberate design.

Sorry for the wording SNAFU there. Of course it's cross-platform. But
I think it's a dubious check. On Windows itself I would never try what
it's attempting to do. I would just call GetFullPathName to see if the
path changes to use the \\.\ device namespace. I doubt that the 3
examples I gave are the only places that this simplistic
implementation of 'is_reserved" fails. It's giving a dangerous
impression to the user that the path will be safe to use on Windows
when that's not the case.

[Steve D'Aprano]

On Mon, 22 Aug 2016 10:38 am, eryk sun wrote:

> To me it's scary that this check misses cases because it's trying to
> be cross-platform instead of simply relying on GetFullPathName to do
> the work. For example, it misses at least the following cases:

Instead of shaking in your boots over a simple bug in a non-critical
library, how about reporting these cases on the bug tracker with an
explanation of the problem?



--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[Chris Angelico]

Understood, and apology accepted. But I think it should be understood
that this will *not* call any Windows APIs. It's called
"PureWindowsPath" and is an example of a pure path:

https://docs.python.org/3/library/pathlib.html#pure-paths

They're specifically documented as not touching any file system, which
means that they are cross-platform and cannot be guaranteed to be
perfect. If you know you're running on Windows, use WindowsPath
instead (trying to do so on a non-Windows system will cause an
immediate exception), and then you can test your examples on that. It
may well already probe the file system in that situation, and if not,
you could put forward the feature suggestion that it should. But the
PurePath classes won't.

ChrisA

[Terry Reedy]

On 8/21/2016 9:08 PM, eryk sun wrote:
> On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico <ros...@gmail.com> wrote:
>> "Trying to be" cross-platform? The point of these path modules is to
>> *be* cross-platform. I can't call Windows APIs on my Linux box (short
>> of messing around with VMs or Wine or something, which are
>> dependencies that Python doesn't need). So yes, these *are*
>> cross-platform, and that's not scary, that's deliberate design.
>
> Sorry for the wording SNAFU there. Of course it's cross-platform. But
> I think it's a dubious check. On Windows itself I would never try what
> it's attempting to do. I would just call GetFullPathName to see if the
> path changes to use the \\.\ device namespace. I doubt that the 3
> examples I gave are the only places that this simplistic
> implementation of 'is_reserved" fails. It's giving a dangerous
> impression to the user that the path will be safe to use on Windows
> when that's not the case.

Since the function is there, please consider opening an issue and
submitting a patch to improve it. Also, could the doc be improved to
say that is_reserved() == False might be wrong? It is possible to
broaden the test so that too many names, rather than too, are marked as
reserved? Better to say "don't use" for a good name rather than 'use'
for a reserved name.

--
Terry Jan Reedy

[eryk sun]

On Mon, Aug 22, 2016 at 1:26 AM, Chris Angelico <ros...@gmail.com> wrote:
> They're specifically documented as not touching any file system, which
> means that they are cross-platform and cannot be guaranteed to be
> perfect. If you know you're running on Windows, use WindowsPath
> instead (trying to do so on a non-Windows system will cause an
> immediate exception), and then you can test your examples on that. It
> may well already probe the file system in that situation, and if not,
> you could put forward the feature suggestion that it should. But the
> PurePath classes won't.

PureWindowsPath.is_reserved() delegates to
self._flavour.is_reserved(), and _WindowsFlavour.is_reserved() is the
common implementation. I don't like splitting this up between a pure
and possibly inaccurate version that can be used on Unix systems
versus a concrete and accurate version that can only be used on
Windows. It can certainly be done if people think that's a good idea.

Terry, I'll open an issue that includes the 3 examples I provided plus
anything else I can find. Some patterns may emerge for the handling of
space, dot, and colon that can be used to fix this in a general way.
It would help to consult a reverse-engineered implementation of
RtlGetFullPathName_Ustr and RtlIsDosDeviceName_Ustr. I'll check the
ReactOS source code.

[Jon Ribbens]

On 2016-08-22, Steve D'Aprano <steve+...@pearwood.info> wrote:
> On Mon, 22 Aug 2016 10:38 am, eryk sun wrote:
>> To me it's scary that this check misses cases because it's trying to
>> be cross-platform instead of simply relying on GetFullPathName to do
>> the work. For example, it misses at least the following cases:
>
> Instead of shaking in your boots over a simple bug in a non-critical
> library, how about reporting these cases on the bug tracker with an
> explanation of the problem?

That seems a rather unnecessarily harsh response.
Also, it's not "non-critical", this is a security bug.

[Chris Angelico]

Explain how?

ChrisA

[Jon Ribbens]

I don't know what purpose you are envisaging this function being used
for, but the only one I can think of is input sanitisation. e.g. a web
form where you receive a file from the Internet and store it somewhere,
and you want to use the filename given to you rather than choose your
own randomly-generated one.

Under Unix all you need to do is check for the filename starting with
"." or containing "/." (or "/", depending on your requirements).
Under Windows you would use this function, which apparently doesn't
work, hence: security hole.

[Steve D'Aprano]

Eryksun bought into Lawrence's over-the-top rhetorical question "does this
scare you?" by answering "Yes", and repeating the ridiculous term "scary".
He specifically said that it scares him *because* it is cross-platform
code, as if cross-platform code is a bad thing.

Now I'm sure that Eryksun isn't *actually* scared of cross-platform code.
I'm sure he is quite capable of using (say) os.listdir() without widdling
himself in terror *wink*. And I don't know if he was intentionally using
the word "scary" or whether it was just an ill-thought out choice of words.
Either way, yes, I'm making a gentle dig at Eryksun for exaggerating the
magnitude of the supposed problem and for taking something which is clearly
a mere bug and treating it as a feature that is broken by design.

There's nothing wrong with writing cross-platform code, and there's no
reason why non-Windows users shouldn't be permitted to explicitly query
whether a file name could be valid on a Windows system.

> Also, it's not "non-critical", this is a security bug.

How is this a security bug? What's the nature of the vulnerability?

[Chris Angelico]

Nope. On Windows, you would try/except it. There are myriad other ways
something could fail, and the only correct action is to attempt it.
Most of the reserved names will simply give an error; the only way
you'd actually get incorrect behaviour is if the file name, including
extension, is exactly a device name. (Caveat: My knowledge of Windows
is rusty and my testing just now was cursory. I could be wrong.) So
you can check for a few exact strings... or just slap some extra text
onto the beginning or end of the file name (beginning meaning "after
the last slash", not the beginning of the file *path*) and you're
safe.

ChrisA

[Random832]

On Mon, Aug 22, 2016, at 08:39, Chris Angelico wrote:
> Nope. On Windows, you would try/except it.

No, you can't, because the failure mode often isn't "file refuses to
open" but "data is written to a serial port".

There are myriad other ways
> something could fail, and the only correct action is to attempt it.
> Most of the reserved names will simply give an error; the only way
> you'd actually get incorrect behaviour is if the file name, including
> extension, is exactly a device name.

I think the reason you believe this can be traced back to the
"C:\con\con" trick, which crashed the system by trying to use the name
as a directory.

> (Caveat: My knowledge of Windows
> is rusty and my testing just now was cursory. I could be wrong.)

Eryk Sun already posted an example using "NUL .txt".

[eryk sun]

On Mon, Aug 22, 2016 at 12:39 PM, Chris Angelico <ros...@gmail.com> wrote:
>
> Nope. On Windows, you would try/except it. There are myriad other ways

No, I would not rely on exceptions in this case. Both \\.\CON and
\\.\NUL can be opened for both reading and writing, so you may not
detect the problem.

I think it's best to check for this via os.path.abspath, which calls
GetFullPathName, which is implemented by RtlGetFullPathName_Ustr,
which is exactly what RtlDosPathNameToNtPathName_U uses to normalize a
path before it converts it to the NT namespace.

> or just slap some extra text onto the beginning or end of the file name
> (beginning meaning "after the last slash", not the beginning of the file
> *path*) and you're safe.

Adding to the beginning is the safe bet without having to worry about
the rules for trailing spaces, dots, and colons at the end of the
name. Adding a single underscore prefix will suffice (or however many
to make a unique name) if for some reason you can't ask the user for a
new name.

[Chris Angelico]

On Mon, Aug 22, 2016 at 10:56 PM, Random832 <rand...@fastmail.com> wrote:
>> Most of the reserved names will simply give an error; the only way
>> you'd actually get incorrect behaviour is if the file name, including
>> extension, is exactly a device name.
>
> I think the reason you believe this can be traced back to the
> "C:\con\con" trick, which crashed the system by trying to use the name
> as a directory.

I tried things like "con.txt" and it simply failed (no such file or
directory), without printing anything to the console. But as Eryk
says, adding an underscore is safe; and to be honest, I wouldn't
accept file names from untrusted sources on *any* system - at very
least, I'd prefix/suffix them with something to ensure uniqueness,
which would deal with this issue as a convenient side effect. (Or
alternatively, I'd use arbitrary numbers or hashes as the file names,
and store the originally-submitted file name in some sort of metadata
repository, like a Postgres table.)

So I still don't see this as a security problem, just a practicality one.

ChrisA

[Steve D'Aprano]

That's backwards: it works under Windows, but is buggy under non-Windows.

In any case, I don't think that what you suggest is even close to
sufficient.

For starters, even on Unix I probably wouldn't want to
use "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\.txt" as a
file name, even if it is technically legal. Nor would I allow

"foo.jpg .exe"

either, especially not on Windows.

But let's suppose my application can handle newlines and other control
characters in filenames (I never `ls` the storage directory, all my scripts
are newline safe, etc.) And I don't care about users uploading malware with
disguised file names. Am I done?

No, not at all. I can't assume that just because a file name is legal that I
can write to it. Even after sanitising the name (or deciding I don't need
to bother) I still need to be prepared to catch I/O errors when writing the
file -- including attempts to write to reserved file names.

None of this makes the issue a security issue. How would you exploit it? The
attacker would have to ensure that the file name is sanitised under Linux
but then written to a file under Windows, in which case the worst that
happens is that I get an unexpected I/O error.

Technically that could be called a DoS attack, but the same could be said
for ANY bug that raises an exception. I think that it takes more than that
to be a security bug.

[Steve D'Aprano]

On Mon, 22 Aug 2016 10:56 pm, Random832 wrote:

> On Mon, Aug 22, 2016, at 08:39, Chris Angelico wrote:
>> Nope. On Windows, you would try/except it.
>
> No, you can't, because the failure mode often isn't "file refuses to
> open" but "data is written to a serial port".

Ah, that's a good point. I hadn't thought of that.

But... what are the consequences if you write to the serial port? Unless you
actually have an external device plugged into it, isn't that equivalent to
writing to /dev/null? (Bytes go into the serial port, and just disappear.)
The user uploads their file, and cleverly fools you into discarding their
file? I'm not seeing how this is an attack.


I suppose they could write to CON and display a lot of garbage on the
screen. But if you're running this on Windows, surely you've already dealt
with these issues, in which case it's a non-issue. Or you haven't dealt
with them, in which case it's an existing bug and the code Lawrence
demonstrated doesn't change anything.

>> There are myriad other ways
>> something could fail, and the only correct action is to attempt it.
>> Most of the reserved names will simply give an error; the only way
>> you'd actually get incorrect behaviour is if the file name, including
>> extension, is exactly a device name.
>
> I think the reason you believe this can be traced back to the
> "C:\con\con" trick, which crashed the system by trying to use the name
> as a directory.

\con\con hasn't been an issue since Windows 98. If you're running your web
application under Win 98, you deserve to be blue-screened :-)

[Ben Finney]

Chris Angelico <ros...@gmail.com> writes:

> […] to be honest, I wouldn't accept file names from untrusted sources
> on *any* system […]

That's one of the wiser things said in this whole thread.

> I'd use arbitrary numbers or hashes as the file names, and store the
> originally-submitted file name in some sort of metadata repository,
> like a Postgres table.)

The failure modes of using filenames from untrusted input are shockingly
diverse, as Tom Eastman describes:

The scope for abuse is eye-widening: The contents of the file, the
type of the file, the size and encoding of the file, even the *name*
of the file can be a potent vector for attacking your system.

The scariest part? Even the best and most secure web-frameworks
(yes, I'm talking about Django) can't protect you from all of it.

In this talk, I'll show you every scary thing I know about that can
be done with a file upload, and how to protect yourself from --
hopefully -- most of them.

<URL:https://2016.pycon-au.org/schedule/148/view_talk>

Tom presented to us at this year's PyCon AU
<URL:https://www.youtube.com/watch?v=HS8KQbswZkU>.

So yes, filenames from arbitrary sources should be *completely*
untrusted, and never used to access any file on the system. Throw the
entire filename away and make a filename locally, without using any part
of the original name.

--
\ “I saw a sign: ‘Rest Area 25 Miles’. That's pretty big. Some |
`\ people must be really tired.” —Steven Wright |
_o__) |
Ben Finney

[Random832]

On Mon, Aug 22, 2016, at 10:21, Ben Finney wrote:
> So yes, filenames from arbitrary sources should be *completely*
> untrusted, and never used to access any file on the system. Throw the
> entire filename away and make a filename locally, without using any part
> of the original name.

To be fair, this particular case is unique in presenting a possibility
to cause problems even for a filename that consists only of whitelisted
characters (for a reasonable-sounding whitelist such as "ASCII letters
and numbers and underscore only; all other characters to be scrubbed and
replaced with {underscore, hex escape, nothing}"). I don't think there's
any other precedent.

[Tim Chase]

On 2016-08-23 00:21, Ben Finney wrote:
> So yes, filenames from arbitrary sources should be *completely*
> untrusted, and never used to access any file on the system. Throw
> the entire filename away and make a filename locally, without using
> any part of the original name.

Sadly, this ideal advice too often conflicts with the shoddy Code
Other People Wrote In Our "Enterprise" System™. :-/

-tkc

[Chris Angelico]

On Tue, Aug 23, 2016 at 12:34 AM, Random832 <rand...@fastmail.com> wrote:

> On Mon, Aug 22, 2016, at 10:21, Ben Finney wrote:
>> So yes, filenames from arbitrary sources should be *completely*
>> untrusted, and never used to access any file on the system. Throw the
>> entire filename away and make a filename locally, without using any part
>> of the original name.
>

> To be fair, this particular case is unique in presenting a possibility
> to cause problems even for a filename that consists only of whitelisted
> characters (for a reasonable-sounding whitelist such as "ASCII letters
> and numbers and underscore only; all other characters to be scrubbed and
> replaced with {underscore, hex escape, nothing}"). I don't think there's
> any other precedent.

Windows has some other issues, including that arbitrary files can
become executable very easily (eg if %PATHEXT% includes its file
extension), and since the current directory is always at the beginning
of your path, this can easily turn into a remote code execution
exploit. And any GUI that automatically calculates thumbnails from
image files (this includes Windows, Mac OS, and more than one Linux
window manager) could potentially be attacked via a malformed file,
simply by having it appear on the file system. So the idea that some
file names are dangerous is far FAR broader than "a file called
prn.txt will get saved to the printer".

ChrisA

[Chris Angelico]

On Tue, Aug 23, 2016 at 12:21 AM, Ben Finney <ben+p...@benfinney.id.au> wrote:
>
> So yes, filenames from arbitrary sources should be *completely*
> untrusted, and never used to access any file on the system. Throw the
> entire filename away and make a filename locally, without using any part
> of the original name.

Oh, and I wish you could convince some other parts of the world about
this. When you mix file uploads with Apache+PHP web applications, you
basically get remote code execution right there. As sysadmin, I have
to constantly play whack-a-mole with stupid exploits that just
wouldn't happen if Joomla's web uploads followed this model.
Seriously, how hard is it for something that *already has a database*
to simply insert a row into jos_uploaded_files and then use that row's
ID eg "uploads/file_"+id as the file name? In one stroke, you
eliminate code execution (it can't be ".php" or any variant thereof),
collisions (two uploads with the same file name will get different
IDs), and even detritus from incomplete transactions (if you find
"uploads/file_12345" but there's no row with ID 12345, you can assume
the transaction got rolled back, and safely delete the file).

S'not that hard, folks. Be safe. Be smart.

ChrisA

[eryk sun]

On Mon, Aug 22, 2016 at 1:17 PM, Chris Angelico <ros...@gmail.com> wrote:
> I tried things like "con.txt" and it simply failed (no such file or
> directory), without printing anything to the console.

Are you using IDLE or some other IDE that uses pythonw.exe instead of
python.exe? If so, first use ctypes to allocate a console:

import ctypes
ctypes.WinDLL('kernel32').AllocConsole()

The CON device should work if the process is attached to a console
(i.e. a conhost.exe instance).

You can detect any of the classic DOS devices via isatty(), since
they're all character devices. Below I'm using Windows 10, so the
console uses a kernel device (condrv.sys was added in Windows 8), for
which the name can be queried:

>>> f = open('con.txt')
>>> h = msvcrt.get_osfhandle(f.fileno())
>>> devname = UNICODE_STRING()
>>> ctypes.resize(devname, sizeof(devname) + 32767*2)
>>> ntdll.NtQueryObject(h, 1, byref(devname), sizeof(devname))
0
>>> print(devname.Buffer)
\Device\ConDrv

> I wouldn't accept file names from untrusted sources on *any* system

There are still desktop applications that ask users to name their files.

[Random832]

On Mon, Aug 22, 2016, at 11:40, Chris Angelico wrote:
> Windows has some other issues, including that arbitrary files can
> become executable very easily (eg if %PATHEXT% includes its file
> extension), and since the current directory is always at the beginning
> of your path, this can easily turn into a remote code execution
> exploit.

I didn't include dot in my example whitelist, and there's no mechanism
for an attacker to add random extensions to your PATHEXT.

> And any GUI that automatically calculates thumbnails from
> image files (this includes Windows, Mac OS, and more than one Linux
> window manager) could potentially be attacked via a malformed file,
> simply by having it appear on the file system.

This has nothing to do with the filename, unless you additionally assume
that this will only happen if the file is called .jpg

[Marko Rauhamaa]

Random832 <rand...@fastmail.com>:

> On Mon, Aug 22, 2016, at 11:40, Chris Angelico wrote:
>> Windows has some other issues, including that arbitrary files can
>> become executable very easily (eg if %PATHEXT% includes its file
>> extension), and since the current directory is always at the
>> beginning of your path, this can easily turn into a remote code
>> execution exploit.
>
> I didn't include dot in my example whitelist, and there's no mechanism
> for an attacker to add random extensions to your PATHEXT.

Years back, my FTP server was hacked by exploiting a buffer overflow.
The anonymous input directory contained a very long filename that
apparently contained some valid x86 code.

Did you vet your whitelist so it couldn't possibly be interpreted by the
CPU as meaningful instructions?


Marko

[Chris Angelico]

On Tue, Aug 23, 2016 at 1:54 AM, eryk sun <ery...@gmail.com> wrote:
> On Mon, Aug 22, 2016 at 1:17 PM, Chris Angelico <ros...@gmail.com> wrote:
>> I tried things like "con.txt" and it simply failed (no such file or
>> directory), without printing anything to the console.
>
> Are you using IDLE or some other IDE that uses pythonw.exe instead of
> python.exe? If so, first use ctypes to allocate a console:
>
> import ctypes
> ctypes.WinDLL('kernel32').AllocConsole()
>
> The CON device should work if the process is attached to a console
> (i.e. a conhost.exe instance).

No, I used Pike (to avoid any specifically-Python issues or
protections) running in a console. Attempting to write to "Logs/con"
wrote to the console, so I know the console device is active.
Attempting to write to "Logs/con.txt" failed as described.

> > I wouldn't accept file names from untrusted sources on *any* system
>
> There are still desktop applications that ask users to name their files.

A person running a desktop application is generally considered a
trusted source. In a kiosk environment, you have a lot more to worry
about than special device names (eg someone could overwrite a key
file), so again, allowing an untrusted user to name a file in that
situation would be inappropriate. If the user owns the computer, s/he
should be allowed to attempt any name, and there'd simply be some that
fail - same as any other invalid characters (Windows won't let you put
a colon or question mark in a file name, for instance, which annoys my
brother no end when I give him files like "What's Up, Doc?.mkv" or
"Operation: Rabbit.mkv") or over-long names or anything like that.

ChrisA

[Chris Angelico]

On Tue, Aug 23, 2016 at 1:56 AM, Random832 <rand...@fastmail.com> wrote:
>> And any GUI that automatically calculates thumbnails from
>> image files (this includes Windows, Mac OS, and more than one Linux
>> window manager) could potentially be attacked via a malformed file,
>> simply by having it appear on the file system.
>
> This has nothing to do with the filename, unless you additionally assume
> that this will only happen if the file is called .jpg

It generally will (or rather, only if the file has one of a particular
set of extensions). Automatic thumbnailing is usually done only for
certain file names. I don't know of anything that opens every single
file to see if it has a JFIF signature (etc for PNG and whatever other
types).

ChrisA

[Chris Angelico]

Step 1: Don't have buffers.
Step 2: Profit!

Anyone who's using fixed-sized buffers for application-level code
deserves to be exploited. A program designed to be accessed via the
internet is never (well, hardly ever) going to need so much
performance that it can't afford to be written in a high level
language - it's going to spend most of its time waiting for the
network. The rare exceptions (*maybe* DNS, but even there, I'd be
quite happy to replace my DNS server with one written in Pike, if
BIND9 ever becomes a major threat vector) should be monitored closely
- preferably statically checked with something like Coverity - because
they're remotely-accessible and thus a major risk.

ChrisA

[eryk sun]

On Mon, Aug 22, 2016 at 3:40 PM, Chris Angelico <ros...@gmail.com> wrote:
> Windows has some other issues, including that arbitrary files can
> become executable very easily (eg if %PATHEXT% includes its file
> extension),

cmd uses PATHEXT to augment its search by appending each extension in
the list, in addition to searching for the exact filename. cmd will
always attempt to run any match, regardless of the extension.

You must be thinking of PowerShell, which for some reason reinterprets
how this environment variable has worked since probably OS/2 in the
late 80s. PowerShell only executes files found in PATH that have an
extension that's listed in PATHEXT.

CreateProcess checks the user's execute access in the file security,
which can prevent the execution of .BAT/.CMD files and PE executables,
regardless of extension. But ShellExecute(Ex) has an MS-DOS brain (so
much of the entire Explorer/shell32 implementation has an MS-DOS
brain; it's like they think they're still supporting Windows 9x), so
scripts and data files are always 'executable'. You get some help here
from cmd, which always tries CreateProcess, regardless of extension,
and won't continue to ShellExecuteEx if CreateProcess failed because
access was denied. PowerShell... not so much.

> and since the current directory is always at the beginning
> of your path, this can easily turn into a remote code execution
> exploit.

Since Vista, both CreateProcess and cmd.exe support the environment
variable NoDefaultCurrentDirectoryInExePath. If this is set, you have
to explicitly reference the current directory. PowerShell always
required this.

[eryk sun]

On Mon, Aug 22, 2016 at 4:18 PM, Chris Angelico <ros...@gmail.com> wrote:
>
>> The CON device should work if the process is attached to a console
>> (i.e. a conhost.exe instance).
>
> No, I used Pike (to avoid any specifically-Python issues or
> protections) running in a console. Attempting to write to "Logs/con"
> wrote to the console, so I know the console device is active.
> Attempting to write to "Logs/con.txt" failed as described.

What version of Windows is this? If it's Windows 7 I'll have to check
that later. If "Logs" is an existing directory, then both "Logs/con"
and "Logs/con.txt" should refer to the console. If "Logs" doesn't
exist, then both should fail. Virtual DOS devices only exist in
existing directories.

>> > I wouldn't accept file names from untrusted sources on *any* system
>>
>> There are still desktop applications that ask users to name their files.

> If the user owns the computer, s/he should be allowed to attempt any
> name, and there'd simply be some that fail - same as any other invalid
> characters

Silently writing a file to a (possibly hidden) console or NUL device
is not the same thing as failing with an error. If users explicitly
open "NUL" or "\\.\NUL", I can detect that, and I have no problem with
it (with some reservations about the former). But if they open files
like "C:\Users\JoeUser\Documents\Nul.20160822.doc", I want to make
sure they know that they just asked to save to "\\.\NUL". It's not a
common problem. I just find the system's behavior abhorrent. I'd like
to have a manifest setting that opts the process out of this stupid
DOS legacy behavior.

> Windows won't let you put a colon or question mark in a file name, for instance,
> which annoys my brother no end when I give him files like
> "What's Up, Doc?.mkv" or "Operation: Rabbit.mkv")

A colon is reserved for NTFS streams, so it can silently bite you. If
you get an error for "Operation: Rabbit.mkv", you're probably using
FAT32. On NTFS that creates a file named "Operation" with a $DATA
stream named " Rabbit.mkv".

Every NTFS file has at least the anonymous stream, e.g.
"filename::$DATA". Every NTFS directory has at least the $I30 stream,
e.g. "dirname:$I30:$INDEX_ALLOCATION". Directories can also have named
$DATA streams.

As to question mark and other wildcard characters, those are all
unambiguously handled as exceptions. No Microsoft filesystem allows
them because wildcard matching is baked into the system, such as the
FileName parameter of NtQueryDirectoryFile when listing the contents
of a directory. I guess no one on the NT I/O team wanted to bother
with the possibility of escaping these wildcards since they weren't
allowed in DOS filenames.

FsRtlIsNameInExpression [1] is the kernel-mode filesystem function
that implements wildcard matching. Note that DOS wildcard semantics
are implemented by DOS_STAR (<), DOS_QM (>), and DOS_DOT (").
FindFirstFile has to convert DOS wildcard patterns to NT semantics
before calling NtQueryDirectoryFile. It's not as simple as
substituting the corresponding DOS_ wildcard.

Pipe and ASCII control characters are also reserved, except not in
NTFS stream names. Reserving pipe is just a DOS legacy. I don't think
it's used for anything internally.

> or over-long names or anything like that.

Good news. In Windows 10, systems and applications can opt in to using
long paths with up to about 32760 characters. Python 3.6 will be
opting in.

[1]: https://msdn.microsoft.com/en-us/library/ff546850

[Chris Angelico]

On Tue, Aug 23, 2016 at 3:13 AM, eryk sun <ery...@gmail.com> wrote:
> On Mon, Aug 22, 2016 at 4:18 PM, Chris Angelico <ros...@gmail.com> wrote:
>>
>>> The CON device should work if the process is attached to a console
>>> (i.e. a conhost.exe instance).
>>
>> No, I used Pike (to avoid any specifically-Python issues or
>> protections) running in a console. Attempting to write to "Logs/con"
>> wrote to the console, so I know the console device is active.
>> Attempting to write to "Logs/con.txt" failed as described.
>
> What version of Windows is this? If it's Windows 7 I'll have to check
> that later. If "Logs" is an existing directory, then both "Logs/con"
> and "Logs/con.txt" should refer to the console. If "Logs" doesn't
> exist, then both should fail. Virtual DOS devices only exist in
> existing directories.

Yes, it was Windows 7 (running in a VM under Debian Jessie, though I
doubt that makes any difference). The Logs directory did exist (that's
why I used that otherwise-odd choice of name).

>> Windows won't let you put a colon or question mark in a file name, for instance,
>> which annoys my brother no end when I give him files like
>> "What's Up, Doc?.mkv" or "Operation: Rabbit.mkv")
>
> A colon is reserved for NTFS streams, so it can silently bite you. If
> you get an error for "Operation: Rabbit.mkv", you're probably using
> FAT32. On NTFS that creates a file named "Operation" with a $DATA
> stream named " Rabbit.mkv".

I think it was FAT32 that we were using - it was a USB stick for
sharing files between my Linux systems and his Windows. Linux was
quite happy to put files into that FS that Windows refused to touch.
(Also, due to some renamings, I ended up giving him two copies of a
file that differed only in case - something like "Name The File
Something.mkv" and "Name the File Something.mkv" - and Windows took
issue with that too.)

> As to question mark and other wildcard characters, those are all
> unambiguously handled as exceptions. No Microsoft filesystem allows
> them because wildcard matching is baked into the system, such as the
> FileName parameter of NtQueryDirectoryFile when listing the contents
> of a directory. I guess no one on the NT I/O team wanted to bother
> with the possibility of escaping these wildcards since they weren't
> allowed in DOS filenames.

Yeah. They're nice straight-forward errors, as long as you're running
on Windows. Detecting that on POSIX platforms and avoiding those names
is why we have PureWindowsPath. If it doesn't currently have a quick
little "is_valid", it could possibly benefit from one - PurePosixPath
would simply "return '\0' not in name", while PureWindowsPath can do
the more extensive checks with wildcard characters and stuff.

>> or over-long names or anything like that.
>
> Good news. In Windows 10, systems and applications can opt in to using
> long paths with up to about 32760 characters. Python 3.6 will be
> opting in.
>
> [1]: https://msdn.microsoft.com/en-us/library/ff546850

Great. So just as soon as all previous Windowses die.......

Remind me how many people are still using Python 3.4 because they're
still on Windows XP?

ChrisA

[Jon Ribbens]

On 2016-08-22, Chris Angelico <ros...@gmail.com> wrote:

> I tried things like "con.txt" and it simply failed (no such file or
> directory), without printing anything to the console.

I'm not sure how you got that to fail, but writing to "con.txt"
certainly does write to the console in Windows 10 - I just tried it:

C:\>echo hello >con.txt
hello

[Tim Chase]

On 2016-08-23 02:20, Chris Angelico wrote:
> It generally will (or rather, only if the file has one of a
> particular set of extensions). Automatic thumbnailing is usually
> done only for certain file names. I don't know of anything that
> opens every single file to see if it has a JFIF signature (etc for
> PNG and whatever other types).

How about a web server that opens arbitrary files. Compare any of

https://technet.microsoft.com/en-us/library/nonexistent.aspx
https://technet.microsoft.com/en-us/library/doesnotexist.aspx
https://technet.microsoft.com/en-us/library/asdf.aspx

vs

https://technet.microsoft.com/en-us/library/con.aspx
https://technet.microsoft.com/en-us/library/lpt1.aspx
https://technet.microsoft.com/en-us/library/com1.aspx
https://technet.microsoft.com/en-us/library/nul.aspx

This is FREAKING MICROSOFT and it breaks things. It's not like
anybody would open arbitrarily-named files...

-tkc

[Chris Angelico]

Oh, brilliant. Brilliant brilliant brilliant.

ChrisA

[Jon Ribbens]

On 2016-08-22, Steve D'Aprano <steve+...@pearwood.info> wrote:
> On Mon, 22 Aug 2016 09:50 pm, Jon Ribbens wrote:
>> I don't know what purpose you are envisaging this function being used
>> for, but the only one I can think of is input sanitisation. e.g. a web
>> form where you receive a file from the Internet and store it somewhere,
>> and you want to use the filename given to you rather than choose your
>> own randomly-generated one.
>>
>> Under Unix all you need to do is check for the filename starting with
>> "." or containing "/." (or "/", depending on your requirements).
>> Under Windows you would use this function, which apparently doesn't
>> work, hence: security hole.
>
> That's backwards: it works under Windows, but is buggy under non-Windows.

I'm not sure what you're getting at there, but you appear to be wrong:

C:\>python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:38:48) [MSC v.1900
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> pathlib.WindowsPath("con .txt").is_reserved()
False

> For starters, even on Unix I probably wouldn't want to
> use "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\.txt" as a
> file name, even if it is technically legal.

That might be your personal preference, but it's not a security hole
per se.

> No, not at all. I can't assume that just because a file name is legal that I
> can write to it. Even after sanitising the name (or deciding I don't need
> to bother) I still need to be prepared to catch I/O errors when writing the
> file -- including attempts to write to reserved file names.

As mentioned by others, this is nothing to do with lack of error-checking.

> Technically that could be called a DoS attack, but the same could be said
> for ANY bug that raises an exception. I think that it takes more than that
> to be a security bug.

It doesn't raise an exception. It interacts with the hardware of the
server.

[Tim Chase]

On 2016-08-22 22:39, Chris Angelico wrote:
> Nope. On Windows, you would try/except it. There are myriad other

> ways something could fail, and the only correct action is to
> attempt it. Most of the reserved names will simply give an error;

The problem is that when opening such a pseudo-file, you can get
unexpected behavior. In the Unix world, we're used to
files-that-aren-t-files (such as things in /dev ). But a lot of
Windows developers don't handle these cases, and so opening something
like COM1 can end up hanging a program indefinitely instead of
actually returning either an error or a file-handle. If you have a
web-server running on Windows and can manage to coerce a file to have
such a name, you might be able to hang the web-server process while
it tries to read from (or write to) the serial port. And poof,
near-instant DoS.

-tkc

[Steve D'Aprano]

On Tue, 23 Aug 2016 03:13 am, eryk sun wrote:

> But if they open files
> like "C:\Users\JoeUser\Documents\Nul.20160822.doc", I want to make
> sure they know that they just asked to save to "\\.\NUL". It's not a
> common problem. I just find the system's behavior abhorrent. I'd like
> to have a manifest setting that opts the process out of this stupid
> DOS legacy behavior.

Aren't we getting further and further away from the original topic?

I don't think this is about Window's support for legacy behaviour from
ancient DOS days. Yes, it would be nice if Windows dropped support for
these horrid magic file names, but that's nothing to do with Python.

I'm not really sure what the question is -- we've established that there's a
bug in the non-Windows implementation that tries to emulate Window's
behaviour. What else is there to argue about?

- Does anyone wish to argue that Python shouldn't provide
PureWindowsPath.is_reserved on non-Windows systems? For what reason?

- Is anyone still arguing that there's a new security vulnerability
here because of the pathlib functions? If so, how do you see this
attack working? (Existing filename-based attacks are not new.)

I don't see what the issue is. Eryksun found a bug in pathlib, well done. (I
mean that, I'm not being sarcastic.) I still don't understand why Lawrence
posed his question in the first place.

[Jon Ribbens]

On 2016-08-22, Steve D'Aprano <steve+...@pearwood.info> wrote:

> I'm not really sure what the question is -- we've established that there's a
> bug in the non-Windows implementation that tries to emulate Window's
> behaviour. What else is there to argue about?

It doesn't seem to be "the non-Windows implementation", it seems to be
"the implementation".

> - Does anyone wish to argue that Python shouldn't provide
> PureWindowsPath.is_reserved on non-Windows systems? For what reason?

It shouldn't provide it at all unless it works. The reason I'm putting
it that way is that making it work may be a very great deal of effort
(What is the actual full list of special filenames? Does it vary
between supported Windows versions? How can anyone tell test that
the function is correct?)

> - Is anyone still arguing that there's a new security vulnerability
> here because of the pathlib functions? If so, how do you see this
> attack working? (Existing filename-based attacks are not new.)

Already answered.

> I don't see what the issue is. Eryksun found a bug in pathlib, well done. (I
> mean that, I'm not being sarcastic.) I still don't understand why Lawrence
> posed his question in the first place.

Presumably because of the security implications as described.

[Wildman]

I got the same result with Windows 7, Vista, XP, 2000, NT4, 98SE
and 3.11. Yes, I have a lot VM's although I don't have 8.x. I
would, however, expect the same result.

On Linux a file is created named con.txt that contains hello/n
as expected.

--
<Wildman> GNU/Linux user #557453
"The Constitution only gives people the right to
pursue happiness. You have to catch it yourself."
-Benjamin Franklin

[Ethan Furman]

On 08/19/2016 06:11 PM, Wildman via Python-list wrote:

> Since I am fairly new to Python, I realize there is much that I
> still don't know but I don't understand how Windows can have
> reserved names on a Linux system. What am I missing?

A PureWindowsPath (and PurePosixPath and PurePath) is a theoretical -- it's still, basically, a string and hasn't been checked against the file system to see if it actually exists (or to create it, etc.).

A WindowsPath (PosixPath) does actually have a tie-in to the actual file system, and can only be instantiated on Windows (Posix) systems.

--
~Ethan~

[eryk sun]

On Mon, Aug 22, 2016 at 5:24 PM, Chris Angelico <ros...@gmail.com> wrote:
> On Tue, Aug 23, 2016 at 3:13 AM, eryk sun <ery...@gmail.com> wrote:
>> On Mon, Aug 22, 2016 at 4:18 PM, Chris Angelico <ros...@gmail.com> wrote:
>>>
>>>> The CON device should work if the process is attached to a console
>>>> (i.e. a conhost.exe instance).
>>>
>>> No, I used Pike (to avoid any specifically-Python issues or
>>> protections) running in a console. Attempting to write to "Logs/con"
>>> wrote to the console, so I know the console device is active.
>>> Attempting to write to "Logs/con.txt" failed as described.
>>
>> What version of Windows is this? If it's Windows 7 I'll have to check
>> that later. If "Logs" is an existing directory, then both "Logs/con"
>> and "Logs/con.txt" should refer to the console. If "Logs" doesn't
>> exist, then both should fail. Virtual DOS devices only exist in
>> existing directories.
>
> Yes, it was Windows 7 (running in a VM under Debian Jessie, though I
> doubt that makes any difference). The Logs directory did exist (that's
> why I used that otherwise-odd choice of name).

I discovered why "Logs/con.txt" isn't working right in Windows 7,
while "Logs/nul.txt" does get redirected correctly to r"\\.\nul".
Prior to Windows 8 the console doesn't use an NT device, so the base
API has a function named BaseIsThisAConsoleName that looks for names
such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and
returns either "CONIN$" or "CONOUT$" if there's a match. A match for
just "CON" maps to one or the other of the latter depending on whether
read or write access is desired.

If there's a match, then a CreateFile call gets routed to
OpenConsoleW, which uses the process ConsoleHandle to send the request
to the attached instance of conhost.exe, which replies with a console
pseudohandle. Note that the handle value is flagged by setting the
lower 2 bits, i.e. 3, 7, 11, which allows routing to special console
functions, such as WriteFile => WriteConsoleA. In Windows 8+ console
handles are regular File handles (e.g. 24, 28, 32).

When debugging this I observed that there's a performance hack in
BaseIsThisAConsoleName. It only calls RtlIsDosDeviceName_U, which does
a full check for "CON" in the path, if the name starts with '\\' ,
'c', or 'C', or if it ends with 'n', 'N', ':', or '$'. This means
r"C:\whatever\con.txt" works (the base path doesn't even have to
exist), but not r"D:\whatever\con.txt". In your case the name starts
with 'L', so BaseIsThisAConsoleName returns false and the code falls
through to calling DosPathNameToRelativeNtPathName_U_WithStatus. This
returns r"\??\con", which NtCreateFile fails to open.

r"\??" is a virtual directory starting in Windows XP. For this
directory the object manager first checks the logon session's DOS
devices in r"\Sessions\0\DosDevices\[Logon Id]" and then the global
DOS devices in r"\GLOBAL??". This is an improvement over the less
flexible way that Windows 2000 managed DOS devices for terminal
services and logon sessions. A DOS 'device' is an object symbolic link
to the real NT device in the r"\Device" directory. In Windows 8+,
r"\GLOBAL??\CON" is a link to r"\Device\ConDrv\Console", which is why
opening "Logs/con.txt" worked 'correctly' for me in Windows 10.

[Chris Angelico]

On Thu, Aug 25, 2016 at 7:00 AM, eryk sun <ery...@gmail.com> wrote:
> I discovered why "Logs/con.txt" isn't working right in Windows 7,
> while "Logs/nul.txt" does get redirected correctly to r"\\.\nul".
> Prior to Windows 8 the console doesn't use an NT device, so the base
> API has a function named BaseIsThisAConsoleName that looks for names
> such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and
> returns either "CONIN$" or "CONOUT$" if there's a match. A match for
> just "CON" maps to one or the other of the latter depending on whether
> read or write access is desired.

See? This is why *even after I tested it* I wasn't sure I was right!
The rules are... complicated.

ChrisA

[Jon Ribbens]

Hence my doubts about this function - it has almost no chance of not
being broken, potentially leading to whatever code was using it having
security holes. Perhaps it should be deprecated or at least attract a
large caveat in the documentation.

[eryk sun]

I agree that the docs need a warning that the behavior of paths
containing legacy DOS device names is inconsistent between versions of
Windows and that the rules that Windows uses aren't explicitly
documented.

There's another inconsistency if Python is run under ReactOS (and
probably Wine, too). The ReactOS equivalent to BaseIsThisAConsoleName
is IntCheckForConsoleFileName, which doesn't have the (buggy) speed
hack that Windows uses to gate the more expensive call to
RtlIsDosDeviceName_U:

http://preview.tinyurl.com/reactos-console-c-71210

Also note the comment on line 354 that "C:\some_path\CONIN$" should
open the console. This highlights yet another problem with the current
pathlib check: it doesn't reserve the names "CONIN$" and "CONOUT$".

Actually, the comment is technically wrong for versions prior to
Windows 8. The old implementation of RtlIsDosDeviceName_U only checks
for "CON", not "CONIN$" or "CONOUT$". The latter two devices aren't
inherited from DOS. They were added to the Windows API to allow
opening console handles with both read and write access, e.g. to be
able to call WriteConsoleInput and ReadConsoleOutput. "CON" doesn't
allow this. However, the base API does allow opening plain "CONIN$"
and "CONOUT$" without the "\\.\" device namespace prefix, so those
names are at least reserved for the current directory.

The problem is more encompassing for Windows 8+, which has a real
console device. It no longer calls BaseIsThisAConsoleName to redirect
CreateFile to OpenConsoleW. Instead it passes the regular NT paths
r"\??\CON", r"\??\CONIN$", or r"\??\CONOUT$" to the NtCreateFile
system call, which resolves the object manager symbolic links
respectively to r"\Device\ConDrv\Console",
r"\Device\ConDrv\CurrentIn", and r"\Device\ConDrv\CurrentOut". As part
of the redesign, the base API moved the check for "CONIN$" and
"CONOUT$" into the NT runtime library function
RtlpIsDosDeviceName_Ustr. Now "CON", "CONIN$", and "CONOUT$" are
reserved in every directory, just like how it's always worked for NUL,
AUX, PRN, COM1-9, and LPT1-9. For example:

Windows 10

>>> print(os.path.abspath('C:/Temp/conout$ : spam . eggs'))
\\.\conout$

Windows 7

>>> print(os.path.abspath('C:/Temp/conout$ : spam . eggs'))
C:\Temp\conout$ : spam . eggs

[Lawrence D’Oliveiro]

On Monday, August 22, 2016 at 4:18:39 PM UTC+12, eryk sun wrote:
> It would help to consult a reverse-engineered implementation of
> RtlGetFullPathName_Ustr and RtlIsDosDeviceName_Ustr. I'll check the
> ReactOS source code.

This <https://googleprojectzero.blogspot.co.nz/2016/02/the-definitive-guide-on-win32-to-nt.html> might also be useful (found from <http://www.theregister.co.uk/2016/03/01/windows_path_hacks/>).

[eryk sun]

On Thu, Sep 8, 2016 at 1:30 AM, Lawrence D’Oliveiro
<lawren...@gmail.com> wrote:
> On Monday, August 22, 2016 at 4:18:39 PM UTC+12, eryk sun wrote:
>> It would help to consult a reverse-engineered implementation of
>> RtlGetFullPathName_Ustr and RtlIsDosDeviceName_Ustr. I'll check the
>> ReactOS source code.
>

> <https://googleprojectzero.blogspot.co.nz/2016/02/the-definitive-guide-on-win32-to-nt.html>
> might also be useful (found from
> <http://www.theregister.co.uk/2016/03/01/windows_path_hacks/>).

I posted a comment there about a month and a half ago with a couple of
corrections. But at the time I wasn't concerned about the section on
legacy DOS device names. The author only tested on 8.1 and 10, so the
description of DOS device behavior is incomplete -- especially
concerning the way Windows Vista/7 handle CON, CONIN$, and CONOUT$.
For example, "CONOUT$" gets redirected to the console, but not
"C:\Temp\CONOUT$" or "\\.\CONOUT$" even though the last two path
strings start with "C" and "\", which satisfies the speed hack used by
BaseIsThisAConsoleName. The difference is that, prior to Windows 8,
RtlIsDosDeviceName_U doesn't look for "CONIN$" or "CONOUT$", yet
BaseIsThisAConsoleName relies on RtlIsDosDeviceName_U to find the
offset to the device name. Thus it can only redirect an exact match
for "CONIN$" and "CONOUT$".