How to delete a symbolic link? A possible bug in serial.sys
John
I based my dynamic serial driver on serial.sys source code from the DDK.
Basically, what this code does:
- assign a new number to each new device, for example \Device\Serial0,
\Device\Serial1, etc.
- Call IoCreateSymbolicLink to create a link with name such as
\DosDevices\COMx, where COMx name is taken from the PortName registry value
(under Enum\...\Device Parameters)
- when device is deleted: again read the COMx name from the registry and
call IoDeleteSymbolicLink on the symbolic link
I guess this code was correct under NT, but it appears to be broken in
Win2K, XP and Vista when I change port number in Device Manager.
I tried the following steps on XP SP2 and Vista with 2 physical serial
ports:
1. Observe COM1 and COM2 in Device manager. Execute "echo 123 > COM2:" on
command prompt. Should be no errors. In WinObj observe the symbolic link
under GLOBAL??
from COM2 to \Device\Serial1. Also observe that there are no symbolic links
under Sessions.
2. Now in Device Manager select Communication Port (COM2), Right click,
Properties, Port Settings, Advanced, COM port number. Change it from COM2 to
COM3 (or some other port). Click OK. You may also Invoke "Scan for hardware
changes" to have the port name updated in the view.
3. Execute "echo 123 > COM2:". Observe "System cannot find the file
specified" error. This is correct. This old port does not exist anymore.
4. Execute "echo 123 > COM3:". Should be no errors.
5. Now in Device Manager select Communication Port (COM3), Right click,
select Disable.
6. Now in Device Manager select Communication Port (COM3), Right click,
select Enable.
7. Execute "echo 123 > COM3:". Observe "System cannot find the file
specified" error.
This is the bug: you cannot use COM3 from any Win32 program anymore.
Why this happened: in WinObj observe under GLOBAL?? symbolic link from COM3
to \Device\Serial2. This is the correct link created by the driver.
But under Sessions\0\DosDevices\... I see a symbolic link from COM3 to
\Device\Serial1. This is an error. This link was created by the device
manager before the driver was disabled and reenabled. The driver calls
IoDeleteSymbolicLink, which only deletes links under GLOBAL?? (when called
by a system thread). So, the driver didn't delete this old
COM3->\Device\Serial1 link. But it is a per-session link, so it takes
precedence over the global link, so all Win32 program cannot open the com
port anymore.
Now some more experiments:
8. In Device Manager change the COM port number from COM3 to COM4. Observe
in WinObj under Sessions\0\DosDevices\... a link changed. Now it is
COM4->\Device\Serial1.
But it is wrong: there is no \Device\Serial1 anymore. That device was
deleted long ago and it is now called \Device\Serial2. But Device Manager
does not care.
9. Execute "echo 123 > COM4:". Observe "System cannot find the file
specified" error. This is because of the bad link above.
10. Execute "echo 123 > COM3:". Observe no errors. This is really crazy
(from user's perspective). After I changed COM port from COM3 to COM4, COM4
does not work, but COM3 miraculously began to work. Of course, if users
would know hot to use WinObj, they will see that there is no mystery -
simply COM3 link was deleted from Sessions and COM3 link under GLOBAL?? was
uncovered and it is now accessible. But how would you explain it to a user,
that in order to use COM3 you should rename the port to COM4?
Another related bug.
11. Again invoke Disable and then Enable for the same port.
12. Execute "echo 123 > COM3:". Observe "System cannot find the file
specified" error.
13. Execute "echo 123 > COM4:". Observe "System cannot find the file
specified" error.
Great. Now we lost our port completely. There is no way to access it.
14. In WinObj, see that there is still a bad link COM4->\Device\Serial1
under Sessions, but there is also an orphaned bad link under Globals??:
COM3->\Device\Serial2.
This is because, as I explained above, the drive reads the port name from
the registry in order to delete the link. So, if port changed, then the old
link remains orphaned until system reboot.
May be nobody cares about this bug because few people even invoke
Disable/Enable on serial ports. But my serial port is dynamic and it is
deleted and created often, so my users loose access to the ports all the
time and it is a really annoying bug. I really need to fix it, but how? I
guess that one way would be to enumerate all objects under Sessions, find
all my COM links and delete them. But I haven't seen such a code anywhere in
the sample. I am not sure this is the right way to do it. Any suggestions?
Thank you
John
David Craig
c:\WINDDK\6001.18002\src\kmdf\serial? You get the capability to handle PnP
for serial ports not embedded on the motherboard in the old fixed locations.
<John> wrote in message
news:v7qdnf-wXsf2EPvU...@speakeasy.net...
Wilhelm Noeker
> I based my dynamic serial driver on serial.sys source code from the DDK.
> Basically, what this code does:
> - assign a new number to each new device, for example \Device\Serial0,
> \Device\Serial1, etc.
> - Call IoCreateSymbolicLink to create a link with name such as
> \DosDevices\COMx, where COMx name is taken from the PortName registry value
> (under Enum\...\Device Parameters)
> - when device is deleted: again read the COMx name from the registry and
> call IoDeleteSymbolicLink on the symbolic link
>
> I guess this code was correct under NT, but it appears to be broken in
> Win2K, XP and Vista when I change port number in Device Manager.
[... observations on session-specific symlinks ...]
> May be nobody cares about this bug because few people even invoke
> Disable/Enable on serial ports. But my serial port is dynamic and it is
> deleted and created often, so my users loose access to the ports all the
> time and it is a really annoying bug. I really need to fix it, but how? I
> guess that one way would be to enumerate all objects under Sessions, find
> all my COM links and delete them. But I haven't seen such a code anywhere in
> the sample. I am not sure this is the right way to do it. Any suggestions?
I have observed problems with renumbering COM ports, too. Although I did
not arrive at a satisfactory solution, here is what I tried anyway:
Since the driver itself always creates (and deletes) global symbolic
links, I decided to avoid session links. So in my user-mode calls to
DefineDosDevice(), I would use \Global\COMx instead of just COMx. Well,
this worked under W2K, but failed miserably under XP: Apparently I could
delete global links, but was not allowed to create any! So in order to
recognize this scenario and fail gracefully, I first define a dummy
global symbolic link, and if successful, immediately delete it again.
Otherwise I decide that I just can't renumber COM ports on the fly and
defer the change until the port is reloaded.
I found another interesting "feature" under W2K terminal server: It
seems that whenever a new session is created, it receives a local copy
of the global namespace! That means that there may be session symlinks
hiding any changes to a global symlink, even if I did not create them in
my own code. So maybe your idea to walk through all sessions is not so
bad after all. (I admit though that I was too lazy to find out if any
newer versions of Windows have this unfortunate implementation of
session namespaces, too.)
One more strangeness that I found, though probably unrelated to this
problem: The DDK recommends using IoCreateUnprotectedSymbolicLink()
instead of IoCreateSymbolicLink(), "if the user needs to be able to
manipulate the symbolic link" and claims that this is what the parallel
and serial drivers do. But looking at the serial code in the DDK,
apparently they don't. Oh well.
Vetzak
First, never use DDK samples unless really needed.
Second, serial.sys uses a global variable to keep track of the
numbering of it's device name \Device\Serial. No, I'm not kinding you.
See pnp.c function SerialCreateDevObj():
static ULONG currentInstance = 0;
This variable is thus system-global ! Serial.sys increments and the
variable and sticks it to \Device\Serial. If the name is already
taken, bummer. This mighty programming technique makes use of \Device
\Serial in other driver binaries unpredictable.
So don't use \Device\Serial as a base name, use something else like
\Device\GimmeCerial or so.
Tim Roberts
That is incredibly bad advice.
>Second, serial.sys uses a global variable to keep track of the
>numbering of it's device name \Device\Serial. No, I'm not kinding you.
>See pnp.c function SerialCreateDevObj():
>
>static ULONG currentInstance = 0;
>
>This variable is thus system-global !
As opposed to what? This, after all, is a system-wide value. You could
use a registry entry, I suppose, but that's only a visible global.
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.
Vetzak
> Vetzak <ptrs...@gmail.com> wrote:
>
> >First, never use DDK samples unless really needed.
>
> That is incredibly bad advice.
I don't agree and I'll tell why. What I mean is, don't literally copy
sources from the DDK and then change bits here and there. Ofcourse you
can look at the DDK sources to grasp to big picture, but important
details like correct locking or IRP handling are often overlooked by
the programmer. Some DDK source code dates back to NT4 which
translates to problems with surprise-removal.
In my experience of writing production/stable drivers (mainly USB, pc/
sc, modem/serial, mass storage drivers) I've always started writing
code from scratch, just to understand the driver 100%. Okay, after
some drivers, you'll feel comfortable with copy-pasting your own
boiler-plate code to start off a new driver, because this code has
been debugged thoroughly by then.
Once you start running extensive WHQL-testing, a lot of details will
pop up and then you'll be glad you know your driver 100%. I've seen
colleagues spending more time on debugging DDK-based drivers than it
takes me to write a driver from scratch and pass WHQL.
Wilhelm Noeker
> In my experience of writing production/stable drivers (mainly USB, pc/
> sc, modem/serial, mass storage drivers) I've always started writing
> code from scratch, just to understand the driver 100%. Okay, after
> some drivers, you'll feel comfortable with copy-pasting your own
> boiler-plate code to start off a new driver, because this code has
> been debugged thoroughly by then.
Well, my congratulations, and I'm very impressed at the skills of a
veteran driver programmer. But suggesting that everyone should use your
approach is just not helpful.
DDK sample code is far from perfect, and I'd be happy to discuss the
shortcomings of the serial driver code in particular, with anyone who
cares. But all in all, I think that "pick the DDK sample driver that's
closest to your needs, then change it to do what you really need" is
good advice.
John
> c:\WINDDK\6001.18002\src\kmdf\serial? You get the capability to handle
> PnP for serial ports not embedded on the motherboard in the old fixed
> locations.
Well, I decided not to use KMDF for now.
My customer requires support for Win2K and XP OS and KMDF driver
does not load on these OS without some additional package
and I am scared by numerous complaints on the Web about compatibility
problems between different versions of KMDF and Windows.
So, I am sticking with WDM for now.
But, nevertheless, I tried to build c:\WINDDK\6001.18001\src\kmdf\serial
sample and installed it on Vista instead of the built-in serial.sys and,
sure enough, got the same bug as I presented before.
The following line seems to be the cause of the problem:
status = RtlUnicodeStringPrintf(&deviceName, L"%ws%d",
L"\\Device\\Serial",
currentInstance++);
I looked through KMDF source and found it worse that WDM because
it hides some important functionality (like deleting the symbolic link)
behind
the scenes, so you have to guess what it is actually doing and why it does
not work.
My current understanding that there are actually 2 bugs in the OS:
one is that pesky "currentInstance++" operation above
and another is in the Device Manager. The Device Manager does no notice
when the device name is changed from Serial1 to Serial2 and it keeps
assigning new port names to the old device, which does not exist anymore.
Even if the driver will delete the local port names under the Sessions
directory
(as I suggested earlier), the Device Manager will still continue to assign
names
to the old device (I didn't try to do that, this is just my guess).
So, my current solution is to drop this stupid "currentInstance++"
and instead assign permanent device names, that is the first physical serial
port
should always be Serial0 and the second Serial1, no matter how many times
these devices were enabled and disabled.
Actually, my driver is a USB-to-serial kind, so I will have maintain an
array
of plugged USB devices and assign Serial0 to the first one, Serial1 to the
second, etc.
John
John
> instead of just COMx.
Well, but I do not have any user mode code.
As I explained, I am using the standard Windows Device Manager from the
Control Panel.
It allows to assign port names to serial ports, and I would like my driver
work well will this scenario.
John
John
Yes, as other people already commented, this advice is almost as good
as not to use any software from Microsoft.
What is special about serial sample, is that it is not really a "sample",
like toaster,
but actual source code the for the actual serial.sys (or, at least, very
close)
that is shipped with Windows for decades and used by millions.
So, one would expect at least some minimal stability from this code.
> Second, serial.sys uses a global variable to keep track of the
> numbering of it's device name \Device\Serial. No, I'm not kinding you.
> So don't use \Device\Serial as a base name, use something else like
> \Device\GimmeCerial or so.
Yes, your observation is correct: currentInstance++ is the source of the
problem
(or, at least, one of them). However, simply renaming \Device\Serial by
\Device\GimmeCerial will not fix anything - the Device Manager will still
assign symbolic links to wrong devices.
So, as I explained in another post, my current solution is to get rid of
"currentInstance++"
and, instead, keep an array of array of plugged devices and assign MySerial0
to the first one,
MySerial1 to the second, etc.
John
John
should not be SerialX, but MySerialX or something like that, such as
MySerial0, MySerial1, etc.
John
Maxim S. Shatskih
> problem
> (or, at least, one of them). However, simply renaming \Device\Serial by
> \Device\GimmeCerial will not fix anything - the Device Manager will still
> assign symbolic links to wrong devices.
Not correct.
The COM1, COM2 etc names are invented in the COM port name arbiter which is in user mode in the class installer for the Ports class. You can write a coinstaller to customize this. The database of the existing COM port numbers is named ComDB and is in the registry somewhere, there is a user-mode API to access it, your coinstaller should work with this API.
The kernel-mode names like \Device\Serial0 are not relevant for this at all.
The process is like:
- on install, the Ports installer and your coinstaller scan the ComDB registry via API, invent the next COM port number, update the ComDB and also save this name to your device registry key, namely as a "PortName" value.
- then the kernel-mode code reads the PortName value at MN_START_DEVICE and creates the symlink to itself.
So, the numbering policy used for \Device\Serial%d numbers is not relevant for this at all.
--
Maxim S. Shatskih
Windows DDK MVP
ma...@storagecraft.com
http://www.storagecraft.com
John
I guess you didn't read my initial post, where I presented a sequence of
steps to reproduce the problem.
> - on install, the Ports installer and your coinstaller scan the ComDB
> registry via API,
> invent the next COM port number, update the ComDB and also save this name
> to your device registry key, namely as a "PortName" value.
> - then the kernel-mode code reads the PortName value at MN_START_DEVICE
> and creates the symlink to itself.
> So, the numbering policy used for \Device\Serial%d numbers is not
> relevant for this at all.
Yes, this is all plain and clear, but it has nothing to do with the nature
of the bug in serial.sys and in Device manager.
Let me repeat the bug in a nutshell:
if user changes the port name using Advanced Property of a Serial port
device in Device Manager,
then disable/reenable the device, the COM port becomes inaccessible.
For details, read my original post.
John
Vetzak
I've build a number of serial drivers and the only problem was the
\Device\Serial naming conventing. Ofcourse, you'll have to keep up
with the "serial administration" i.e. do all the registry stuff that
serial.sys does.
BTW, in the field of the "serial administration", there are some buggy
serial out in the field. For example, on some laptops I've seen
bluetooth drivers that take up a lot COMx names without any respect
for the serial admininstration.
To debug your own serial driver, I advice you to disable all serial
(COMx) devices that have a driver other then serial.sys. Doing so will
save you a lot of frustration.
Vetzak
> What is special about serial sample, is that it is not really a "sample",
> like toaster,
> but actual source code the for the actual serial.sys (or, at least, very
> close)
> that is shipped with Windows for decades and used by millions.
> So, one would expect at least some minimal stability from this code.
That wisdom doesn't work for software. Let me put it this way:
software is not a bottle of wine. Wine gets better over time, software
doesn't. ;-)
Wilhelm Noeker
> Let me repeat the bug in a nutshell:
> if user changes the port name using Advanced Property of a Serial port
> device in Device Manager,
> then disable/reenable the device, the COM port becomes inaccessible.
The deeper reason behind this is the device manager (actually the
property page provider for the serial ports, which is located in
MsPorts.dll) trying to modify a symbolic link from user mode that was
originally created from kernel mode. That is just wrong and will never work.
> Well, but I do not have any user mode code.
> As I explained, I am using the standard Windows Device Manager from the
> Control Panel.
> It allows to assign port names to serial ports, and I would like my driver
> work well will this scenario.
I am afraid that the only solution to your problem is to write your own
property page provider in replacement for the one from MsPorts.dll. This
isn't rocket science, and there is sample code provided in the DDK,
src\setup\pnpports.
What you need to do when renumbering ports (at least that's what I do,
and it works fine):
- change the PortName parameter and the friendly name
- move the ComDB allcoation flag (ComDBReleasePort, ComDBClaimPort)
- "rename" the default parameters in ...\Windows NT\CurrentVersion\Ports
But you DON'T touch the device's symbolic link and its entry in
...\Devicemap\SerialComm. Instead:
- send the device a DICS_PROPCHANGE message
That will stop and restart the device, causing it to delete and recreate
its symbolic link and devicemap entry. Voilà. (For details on sending
this message, see src\setup\devcon.)
HTH
John
> property page provider in replacement for the one from MsPorts.dll.
Not really.
It is a solution, but not the only one.
Replacing MsPorts.dll would not be convenient, since it is used by most
serial devices, like standard serial device, many Usb-2-serial, etc.,
so users do know it well.
So, if I replace MsPorts.dll, I will have to keep UI similar to the
original.
But what if the MsPorts.dll UI will change in the future?
As I explained in another post, my current solution is to get rid of
"currentInstance++" and, instead, keep an array of plugged
devices and assign MySerial0 to the first one, MySerial1 to the second, etc.
I hope MS will fix serial.sys and serial sample in DDK in a similar way.
John