Bluetooth Serial Port Profile and Tcl communications
sv
Serial port communications I thought should be fairly straightforward,
but it appears not to be - at least in what I am experiencing. Any
help rendered to get me past this would help. Thanks.
I have a BT-module (Roving Networks RN-41) connected to a processor's
UART. Data is transmitted by the processor and the RN-41 BT module
acts as a pass-through. The data is received over RF by a BT USB
dongle (Dlink DBT-120) connected to a PC running Windows XP SP3).
Under device manager in XP, I see that the RN-41 is recognized as
having a Serial Port profile and assigned a COM number.
When I open Docklight or RS232Comm, I can open the COM and talk to the
remote BT device (query status/address/baud settings, etc.) and also
transfer data bi-directionally. So all appears good there.
I think that validates all aspects of the BT stacks,drivers and
communication channel. I can do this at various baud settings and
everything works just well.
An important point to note here is that I can disconnect and connect
as many times as I want and no problems are noticed. In fact on my BT
module there's a 'link' green LED. When I connect, it comes on green
and goes off when I disconnect. So visually I can tell when the COM
port is opened or closed.
Now when I try this through Tcl ("open COMx: r+" or "open \\\\.\\COMx r
+" or variants of this), I observe a bizzare behavior:
3 out of 5 times, I get either of the following two message screens:
couldn't open serial "\\.\COMx": invalid argument (or)
couldn't reopen serial "\\.\COMx": invalid argument
And the other 2 out of 5 times the following is observed: The device
connects (green LED ON), disconnects immediately (green LED OFF),
connects again, disconnects again and finally connects (green LED ON)
and stays solid. Sometimes this toggling is not 5 times as above, but
only 3 times…ON, OFF, ON.
Now if I try to connect again, 9 out of 10 times, I get a message like
this:
couldn't reopen serial "\\.\COMx": invalid argument
There's no way for me to recover from this, except by power cycling
the BT module (leaving the dongle as is).
Interestingly, there's a very similar (I would say, identical really)
behavior observed by someone using JavaApps (not Tcl). The link is:
http://groups.google.com/group/trackbot/browse_thread/thread/89910d2c97ad42bc
and he is using a completely different BT USB adapter (dongle) and BT
module.
Any pointers on where I ought to look?
Incidentally, I had an off-line discussion with a member of this group
Rolf Schroedter and he rightly suggested that I move this discussion
to this board for getting more people involved and preserve this for
the public should there be others with similar issues. (Thanks Rolf!)
His Tcl serial port monitor ( http://www.rolf-schroedter.de/moni/)
also exhibits a similar behavior as what is detailed above.
Thanks,
-sv
Alexandre Ferrieux
> and he is using a completely different BT USB adapter (dongle) and BT
> module.
>
> Any pointers on where I ought to look?
>
> Incidentally, I had an off-line discussion with a member of this group
> Rolf Schroedter and he rightly suggested that I move this discussion
> to this board for getting more people involved and preserve this for
> the public should there be others with similar issues. (Thanks Rolf!)
> also exhibits a similar behavior as what is detailed above.
>
> Thanks,
> -sv
Not sure whether you only omitted to mention it, but when using a BT
device you must at least select the device you want to connect to
(even if it's the only one within range). I have very little
experience in this, but I seem to remember the BT USB dongle I was
using shipped with a specific GUI doing exactly that.
Once the device becomes visible in the list, you select it, and tell
the GUI to connect to it. Then, and only then, does the channel become
a COMx. You may then open and close it many times on the COM side,
this will not change the connected status at the BT level.
Sorry if this is obvious, but from your description the separation
between the two levels was not clear.
Also, check out the COM-specific settings like baudrate etc. Make sure
there is agreement between what's configured at the COM emulator level
and what your Tcl script [fconfigure]s.
One last shot in the dark: why are you trying various decorations of
"COMx:" ?
-Alex
sv
wrote:
Alex,
Yes, the dongle and BT module are paired up first over Windows 'My
Bluetooth Places'. I had left that detail out as I assumed it was
implicitly stated given that I can connect up using terminal-like
programs Docklight and RS232Comm, but thanks for asking nevertheless.
Can't overlook any detail!
And as for why various COMx? BT module comes up in Windows with high
integer COM (e.g., COM11 or COM21 or similar) and many Windows COM
related sites refer to constraints in Windows to serial ports between
COM1-COM4 or COM1-COM9. So just to be sure, I have tried reassigning
port values down to COM2/4, etc. and have also left them at the values
that widows automatically assigns.
-sv
sv
Also, I don't even get to the stage of fconfigure in Tcl. I need to
have a port opened to configure that wouldn't I? I mean, I think my
script barfs at the 'open' statement. My fconfigure follows that line.
-sv
sub...@fuh-edv.de
I am Oliver, one of the Docklight developers. Thanks for reporting
this and we are happy to share our bits of knowledge here.
> Serial port communications I thought should be fairly
> straightforward,
Serial communication programming Windows is anything but
straightforward. There are for example different ways to open and read/
write from/to a serial port (e.g. overlapped or non-overlapped i/o):
http://msdn.microsoft.com/en-us/library/ms810467.aspx
Unfortunately, today's serial port drivers, be it USB-to-RS232 device
drivers, Bluetooth virtual com port drivers, or virtual null modem
cables like http://com0com.sourceforge.net, seem to have different
ideas or approaches to implement the Windows Serial Communicartions
funcions. Or sometimes they are just plain buggy and a driver update
fixes the problem.
So even after years of development and fine tuning we are sometimes
unable to tell why a certain hardware / serial port driver is not
working with Docklight V1.6, while it might be just fine with
Docklight V1.8 or Hyperterminal. Or the other way round. Unfortunately
we do not use Bluetooth serial connections ourselves, though our
customers reportedly use Docklight within Bluetooth apps., too.
I will try the next days and find out some more specifics about the
your Bluetooth setup.
For a start, you can use following free Microsoft tool to check how
the Tcl application's COM port access is different from what Docklight
V1.8 does:
http://technet.microsoft.com/en-us/sysinternals/bb896644.aspx
Hope that can already provide some insight.
Best regards,
Oliver Heggelbacher
Flachmann und Heggelbacher
www.docklight.de
Alexandre Ferrieux
>
> > Also, check out the COM-specific settings like baudrate etc. Make sure
> > there is agreement between what's configured at the COM emulator level
> > and what your Tcl script [fconfigure]s.
>
> have a port opened to configure that wouldn't I? I mean, I think my
> script barfs at the 'open' statement. My fconfigure follows that line.
OK, I had not identified those apps as clones of Hyperterminal,
sorry :-}
Can you get them to work *also* with a single-digit (overridden)
COMx ?
Same question with the genuine Hyperterminal.
That's weird. Never seen Tcl diverge that far from HT et al.
Maybe you can hook a ProcMon/FileMon to both your working terminal
application and the failing Tcl. The CreateFile syscall (yeah, that
means "open" to some disturbed minds) might be interesting to compare.
-Alex
sv
> Hi,
>
> I am Oliver, one of the Docklight developers. Thanks for reporting
> this and we are happy to share our bits of knowledge here.
>
> > Serial port communications I thought should be fairly
> > straightforward,
>
> Serial communication programming Windows is anything but
> straightforward. There are for example different ways to open and read/
> write from/to a serial port (e.g. overlapped or non-overlapped i/o):http://msdn.microsoft.com/en-us/library/ms810467.aspx
>
> Unfortunately, today's serial port drivers, be it USB-to-RS232 device
> drivers, Bluetooth virtual com port drivers, or virtual null modem
> ideas or approaches to implement the Windows Serial Communicartions
> funcions. Or sometimes they are just plain buggy and a driver update
> fixes the problem.
>
> So even after years of development and fine tuning we are sometimes
> unable to tell why a certain hardware / serial port driver is not
> working with Docklight V1.6, while it might be just fine with
> Docklight V1.8 or Hyperterminal. Or the other way round. Unfortunately
> we do not use Bluetooth serial connections ourselves, though our
> customers reportedly use Docklight within Bluetooth apps., too.
>
> I will try the next days and find out some more specifics about the
> your Bluetooth setup.
>
> For a start, you can use following free Microsoft tool to check how
> the Tcl application's COM port access is different from what Docklight
> V1.8 does:http://technet.microsoft.com/en-us/sysinternals/bb896644.aspx
>
> Hope that can already provide some insight.
>
> Best regards,
> Oliver Heggelbacher
> Flachmann und Heggelbacherwww.docklight.de
Thanks Oliver. Your link to Sysinternals' Portmon appears to shed some
light. However, I am not an expert and certainly can't interpret what
it outputs. While I am trying to independently research that, I wanted
to share the output with you for your feedback (I have added my
comments to the log captured) so that you might be able to see what
the sequence of steps were:
-- BT module power cycled.
-- Wish85 shell opened
-- Tcl "open" command issued over the wish shell.
0 2.70472141 wish85.exe IRP_MJ_CREATE BthModem1 SUCCESS Options: Open
1 0.00000279 wish85.exe IRP_MJ_QUERY_INFORMATION BthModem1
SUCCESS -1162332132
2 0.00000475 wish85.exe IRP_MJ_CLEANUP BthModem1 SUCCESS
3 0.06210733 wish85.exe IRP_MJ_CLOSE BthModem1 SUCCESS
4 0.00102499 wish85.exe IRP_MJ_CREATE BthModem1 SUCCESS Options: Open
5 0.00000531 wish85.exe IOCTL_SERIAL_GET_BAUD_RATE BthModem1 SUCCESS
6 0.00000279 wish85.exe IOCTL_SERIAL_GET_LINE_CONTROL BthModem1
SUCCESS
7 0.00000279 wish85.exe IOCTL_SERIAL_GET_CHARS BthModem1 SUCCESS
8 0.00000223 wish85.exe IOCTL_SERIAL_GET_HANDFLOW BthModem1 SUCCESS
9 0.00000363 wish85.exe IRP_MJ_CLEANUP BthModem1 SUCCESS
10 0.00039921 wish85.exe IRP_MJ_CLOSE BthModem1 SUCCESS
11 0.20623711 wish85.exe IRP_MJ_CREATE BthModem1 * 0xC0000225 Options:
Open
-- BT module serial port profile port open failed!
-- Error reported in Wish85 shell: "couldn't reopen serial "\\.
\COM11": invalid argument"
-- A second attempt of the same:
-- BT module power cycled.
-- Wish85 shell opened
-- Tcl "open" command issued
12 0.00105851 wish85.exe IRP_MJ_CREATE BthModem1 SUCCESS Options: Open
13 0.00000196 wish85.exe IRP_MJ_QUERY_INFORMATION BthModem1
SUCCESS -1162332132
14 0.00000419 wish85.exe IRP_MJ_CLEANUP BthModem1 SUCCESS
15 0.00036569 wish85.exe IRP_MJ_CLOSE BthModem1 SUCCESS
16 1.45223597 wish85.exe IRP_MJ_CREATE BthModem1 SUCCESS Options: Open
17 0.00000503 wish85.exe IOCTL_SERIAL_GET_BAUD_RATE BthModem1
SUCCESS
18 0.00000279 wish85.exe IOCTL_SERIAL_GET_LINE_CONTROL BthModem1
SUCCESS
19 0.00000279 wish85.exe IOCTL_SERIAL_GET_CHARS BthModem1 SUCCESS
20 0.00000223 wish85.exe IOCTL_SERIAL_GET_HANDFLOW BthModem1 SUCCESS
21 0.00000447 wish85.exe IRP_MJ_CLEANUP BthModem1 SUCCESS
22 0.10786902 wish85.exe IRP_MJ_CLOSE BthModem1 SUCCESS
23 0.61481278 wish85.exe IRP_MJ_CREATE BthModem1 SUCCESS Options: Open
24 0.00000363 wish85.exe IOCTL_SERIAL_SET_QUEUE_SIZE BthModem1 SUCCESS
InSize: 4096 OutSize: 4096
25 0.00000531 wish85.exe IOCTL_SERIAL_PURGE BthModem1 SUCCESS Purge:
TXABORT RXABORT TXCLEAR RXCLEAR
26 0.00000251 wish85.exe IOCTL_SERIAL_SET_TIMEOUTS BthModem1 SUCCESS
RI:0 RM:0 RC:0 WM:0 WC:0
-- BT module serial port profile COM successfully opened
-- Issuing a 'close' command over Wish85 shell
27 0.00000587 wish85.exe IOCTL_SERIAL_PURGE BthModem1 SUCCESS Purge:
RXABORT RXCLEAR
28 0.00000391 wish85.exe IOCTL_SERIAL_PURGE BthModem1 SUCCESS Purge:
TXABORT TXCLEAR
29 0.00000419 wish85.exe IRP_MJ_CLEANUP BthModem1 SUCCESS
30 0.00022461 wish85.exe IRP_MJ_CLOSE BthModem1 SUCCESS
-- BT module disconnected successfully
Clearly, the error code "* 0xC0000225" on line 11 appears to provide a
hint. But I have no idea what that means.
In the interest of not adding more lines of probably useless
information, I have not included the Docklight sequence as registered
by Portmon. But in brief: Docklight connected up with the BT module
instantly. While the IOCTL_SERIAL_* messages were similar to the
above, what stands out as different is that Docklight did not issue
IRP_MJ_* commands until I hit the 'port close' button. Also in the
Docklight sequence, after it connects up, it read back the baud
settings from the port and then wrote in baud settings from its
configuration settings and proceeded to set a WAIT_MASK and continued
in WAIT_ON_MASK loop.
If it would be of intellectual interest, I can post the Docklight
series of events as recorded by Portmon.
Regards
-sv
Rolf Schroedter
> ...
> Also, check out the COM-specific settings like baudrate etc. Make sure
> there is agreement between what's configured at the COM emulator level
> and what your Tcl script [fconfigure]s.
>
From the error messages we see that it's Windows createFileProc()
failing to open the comport, we don't even come to configuring the port.
Tcl opens a serial channel twice.
First time in TclWinChan.c, then after detecting that this is a serial
port TclWinSerial.c needs to reopen it with the OVERLAPPED flag.
That's why we have two different error messages with "open/reopen".
In both cases it's Windows createFileProc failing.
What if there is some probability for CreateFileW() to fail with the BT
device ? (yes I know, that this is pure guessing)
Then, off course, Tcl would fail more often, opening the channel twice.
Additionally Docklight or RS232Comm as RS232-specialists could know how
to workaround it.
IMO it makes sense to trace the Windows API calls with Docklight,
RS232Comm, Tcl.
Perhaps it also makes sense to check how behaves Hyperterminal.
I played a little with "Auto Debug Professional", trial version,
and could trace all kernel32.dll API calls.
http://www.autodebug.com/
Unfortunately tracing API calls could only be done with the real BT
device...
Rolf.
Rolf Schroedter
As I mentioned above Tcl opens the serials port twice.
1. CreateFileW() -- open the channel
2. GetCommState() -- check a channel whether its a serial port.
3. CreateFileW() -- Reopen the serial port with the OVERLAPPED flag set.
Rolf.
sub...@fuh-edv.de
On Dec 9, 8:58 am, Rolf Schroedter <m...@privacy.net> wrote:
> As I mentioned above Tcl opens the serials port twice.
> 1. CreateFileW() -- open the channel
> 2. GetCommState() -- check a channel whether its a serial port.
> 3. CreateFileW() -- Reopen the serial port with the OVERLAPPED flag set.
IMHO closing and re-opening the port immediately seems to trigger a
problem within the Bluetooth driver, or rather the RN-41 Bluetooth
module, even though there should be nothing wrong with that.
Considering that it is a problem that doesn't appear every time, I
would assume it might only appear when the port is re-opened too fast,
while the bluetooth module has not returned to the original state
after being closed.
I don't see why the OVERLAPPED flag should be an issue. Docklight uses
overlapped mode only, but the simpler non-overlapped I/O mode should
be fine, too, and I also don't see why this should cause problems at
the CreateFile() stage already.
Here's what I read from sv's Portmon log:
10 0.00039921 wish85.exe IRP_MJ_CLOSE
BthModem1 SUCCESS
11 0.20623711 wish85.exe IRP_MJ_CREATE
BthModem1 * 0xC0000225 Options:
Open
---> ERROR; time between closing and re-opening 0.21 seconds
15 0.00036569 wish85.exe IRP_MJ_CLOSE
BthModem1 SUCCESS
16 1.45223597 wish85.exe IRP_MJ_CREATE
BthModem1 SUCCESS Options: Open
---> SUCCESS; time between closing and re-opening 1.45 seconds
22 0.10786902 wish85.exe IRP_MJ_CLOSE
BthModem1 SUCCESS
23 0.61481278 wish85.exe IRP_MJ_CREATE
BthModem1 SUCCESS
---> SUCCESS; time between closing and re-opening 0.61 seconds
This for me is an indication that the bluetooth module - for whatever
reason - gets stuck if it doesn't receive an additional grace period
between closing and re-opening.
Unfortunately when I use PortMon to debug Docklight on my PC, it
doesn't give me the correct time between closing a port and re-opening
it. Portmon always starts from zero when I re-open a port. I have to
check why that is, maybe we do some additional cleanup that causes
PortMon to reset. Maybe this is exactly what makes it work nicely with
the RN-41 module.
Here's now my questions / suggestions:
1. Rolf, is there any way to control the delay between closing and
reopening the serial port in Tcl?
2. Rolf, is Tcl accidently using the "dwShareMode" for CreateFile()? I
have seen this occasionally in Windows code, but I cannot think of any
serial device that would support it. I think it is a mistake to use it
and Microsoft also advises to stay away from it. See:
http://msdn.microsoft.com/en-us/library/aa450580.aspx
3. For a test, I provided a Docklight script which can be used with
the evaluation version of Docklight Scripting (
http://docklight.de/download/Docklight_Scripting.zip). This is the
script:
http://docklight.de/examples/extras/close_and_reopen_delay.zip
It closes and re-opens a serial port, and you can specify an
additional delay in milliseconds between closing and reopening. You
cannot do 'zero' delay, because our Docklight code already introduces
certain delays, but anything from 100 milliseconds to several seconds
should be no problem.
4. Which Bluetooth serial port drivers are on the PC? I've been
digging in our customer support data. There seem to be mainly two
different Bluetooth serial port drivers. One "Native Microsoft
Bluetooth Software", and a "WIDCOMM Bluetooth Software". See here:
http://www.rovingnetworks.com/documents/SPP-on-pcs.pdf
- Which one are you using, sv?
- Is it the latest version available?
I have actually ordered two Roving Networks FireFly serial adapters
today, so we are willing to investigate this in detail. We'll be happy
to share any further insight, but I need some quiet peaceful lab hours
to look into the issue.
Regards,
Oliver
www.docklight.de
Rolf Schroedter
thank you very much for your detailed analysis.
Some background info of Tcl & serial ports
(Alexander, please correct me if I'm wrong):
The unusual procedure of open-close-reopen of serial ports in Tcl is
only-and-only for setting the OVERLAPPED flag of an already open file
handle.
Due to the internal structure of Tcl there is no separate procedure for
opening a serial port (Tcl is like Unix). Tcl doesn't check the filename
"com1" for being a serial port.
Therefore Tcl opens "com1" like a normal file (NON-OVERLAPPED).
Then, having an open handle to "com1", Tcl checks whether the file
handle is a serial port. This is done by checking the result of
GetCommState(). On success the (open) handle is given to Tcl's serial
port driver.
The driver then has to close-reopen "com1" only for setting the
OVERLAPPED flag (same as you, I'm always using OVERLAPPED for serial I/O).
IMHO a NON-OVERLAPPED close() should block until closing is finished and
an immediate reopen shouldn't be a problem. So it's a driver problem and
I really don't have any clue how to resolve it without proving a
separate serial_open procedure to Tcl...
And, yes, dwShare=0.
Regards, Rolf.
Alexandre Ferrieux
Random idea: couldn't we, instead, use a "name hint", recognizing
things like {^COM[0-9]+:} ?
This would take care of something like 99.99999% of cases in a much
more efficient, and bad-driver-proof, way...
-Alex
Rolf Schroedter
Yes, that would be a solution,
or a separate serial_open procedure, like for sockets...
IMO the name hint is easier, but we should be careful not to break
existing Tcl code like
open {\\.\COM12} r+
Another option would be to seek the Windows registry for existing serial
ports. However I'm not very lucky with hard-coding registry keynames to Tcl.
Rolf.
Alexandre Ferrieux
> Alexandre Ferrieux wrote, On 09.12.2008 17:17:
>
> > On Dec 9, 8:58 am, Rolf Schroedter <m...@privacy.net> wrote:
> >> Just to add some light on Tcl's erial port Handling.
> >> As I mentioned above Tcl opens the serials port twice.
> >> 1. CreateFileW() -- open the channel
> >> 2. GetCommState() -- check a channel whether its a serial port.
> >> 3. CreateFileW() -- Reopen the serial port with the OVERLAPPED flag set.
>
> > Random idea: couldn't we, instead, use a "name hint", recognizing
> > things like {^COM[0-9]+:} ?
> > This would take care of something like 99.99999% of cases in a much
> > more efficient, and bad-driver-proof, way...
>
> > -Alex
>
> Yes, that would be a solution,
> or a separate serial_open procedure, like for sockets...
> IMO the name hint is easier, but we should be careful not to break
> existing Tcl code like
> open {\\.\COM12} r+
>
Then, does
{^COM[0-9]+:$|^\\\\[.]\\COM[0-9]+$}
cover the full space of possibilities ?
-Alex
sv
> 3. For a test, I provided a Docklight script which can be used with
> script:http://docklight.de/examples/extras/close_and_reopen_delay.zip
> It closes and re-opens a serial port, and you can specify an
> additional delay in milliseconds between closing and reopening. You
> cannot do 'zero' delay, because our Docklight code already introduces
> certain delays, but anything from 100 milliseconds to several seconds
> should be no problem.
> Oliverwww.docklight.de
Great inputs Rolf/Oliver/Alex. Thanks!
There are quite a few questions and scenarios here, so let me address
them in parts as I experiment with them. First off, Rolf's description
of the Tcl serial port opening sequence and Oliver's Portmon log
analysis, got me interested in wanting to try out the scripting
feature of Docklight.
I installed Docklight Scripting and ran Oliver's script for opening/
closing and opening the BT serial port. And here are my observations
so far:
Delay: 100ms - Fails everytime * (see below)
Delay: 150ms - Fails everytime *
Delay: 175ms - Fails everytime *
Delay: 190ms - Fails 4 out of 5 tries *
Delay: 200ms - Fails 3 out of 5 tries *
Delay: 225ms - Passes everytime
Delay: 250ms - Passes everytime
Delay: 500ms - Passes everytime
So Oliver's analysis of the delay between a close and open of the port
and resulting behavior of the RN-41 BT module is spot-on!
* The failure happens at the DL.StartCommunication (in Docklight
script) after the DL.Pause myDelay event, i.e., when re-opening the
port is attempted. And when this happens, the only way to recover is
to power cycle the BT module.
-sv
sv
> 3. For a test, I provided a Docklight script which can be used with
> script:http://docklight.de/examples/extras/close_and_reopen_delay.zip
> It closes and re-opens a serial port, and you can specify an
> additional delay in milliseconds between closing and reopening. You
> cannot do 'zero' delay, because our Docklight code already introduces
> certain delays, but anything from 100 milliseconds to several seconds
> should be no problem.
sub...@fuh-edv.de
sv, great to hear you could track down the problem.
On Dec 9, 8:16 pm, sv <nathan.sel...@gmail.com> wrote:
> And when this happens, the only way to recover is
> to power cycle the BT module.
This is what made me suspect the module itself and not the driver - if
it were a PC driver problem, a BT module reset would hardly fix the
problem.
I have seen the RN-41 module supports flash updates of its firmware.
Can you check with Roving Networks if they are aware of the problem
and provide an updated firmware?
One final question - which Bluetooth serial port drivers are you using
on your PC, the "Native Microsoft Bluetooth Software" or the "WIDCOMM
Bluetooth Software"? Or neither of them?
Rolf, you wrote:
> So it's a driver problem and
> I really don't have any clue how to resolve it without proving a
> separate serial_open procedure to Tcl...
Yes, I agree, it's really a device problem. Even the PC serial driver
seems fine in this case, but the Bluetooth device itself gets stuck.
But then again, if the re-opening causes problems with this specific
Bluetooth module, this smells like trouble with other similar gadgets,
too, like Ethernet Serial Device Servers that use a COM Port
Redirector Software on the PC side. So a specific serial port
procedure sounds like a good idea to me, simply for having less
problems with today's complicated serial COM equipment and drivers.
If there's anything we can still do here, please let me know.
Best Regards,
Oliver
www.docklight.de
Rolf Schroedter
The trailing ":" should be optional, and obviously case-insensitivity is
necessary:
regexp -nocase {^COM[0-9]+[:]?$|^\\\\[.]\\COM[0-9]+[:]?$} $name
But however I'm not completely sure, whether Windows allows serial port
names other than "com[0-9]+". I feel like I've seen this somewhere.
The cleanest would be something like:
package require registry
set key "HKEY_LOCAL_MACHINE\\HARDWARE\\DEVICEMAP\\SERIALCOMM"
set values [ registry values $serial_base ]
foreach v $values {
puts [format "port=%s" [registry get $key $v]]
}
But scanning the registry must be done "in real time" on every call to
[open] because e..g USB serial ports may appear/disappear.
I'll google around to find whether there is an "official" way to detect
whether a filename refers to a serial port in Windows.
Rolf.
Alexandre Ferrieux
>
> >>> Random idea: couldn't we, instead, use a "name hint", recognizing
> >>> things like {^COM[0-9]+:} ?
>
> necessary:
> regexp -nocase {^COM[0-9]+[:]?$|^\\\\[.]\\COM[0-9]+[:]?$} $name
> But however I'm not completely sure, whether Windows allows serial port
> names other than "com[0-9]+". I feel like I've seen this somewhere.
Hmmm, a bit of experiment with cmd.exe ("type ..\com1" etc.) shows
that the match (inside Windows) occurs on the "file tail" (in Tcl
vernacular). So that it is really impossible to open a create a file
named "com1" prefixed by any path...
Anyway, I believe the workaround need *not* cover all cases, since it
can fall back to the current behavior in case of non-match.
As a summary:
- "fast lane" for {^COM[0-9]+[:]?$}
- Current behavior otherwise
By "fast lane" of course I mean directly open with the proper settings
for a serial port (overlapped, whatever), hence no double-open and no
timing issues regardless of the underlying driver.
The advantage of the simplified regexp above is that it can be tested
without the regexp engine, hence very fast. This is important since
this check will be on the critical path for *any* [open]. (this
precludes things like real-time registry lookup ;-)
Then if someone uses one of the more complex patterns like \\machine
\foo\bar\..\..\cOm199 and complains that it fails for his specific
military-grade microwave oven, we'll see ;-)
Do you agree ?
-Alex
Rolf Schroedter
FILE_FLAG_OVERLAPPED for serial ports.
The solution is the ReOpenFile() API, available since Windows Server
2003 / Vista, but not for Win2000/XP,....
BTW, Alex, you have discussed a similar problem here:
http://www.archivum.info/comp.lang.tcl/2006-05/msg01097.html
And seems there is no ReOpenFile() equivalent for general Win32:
http://www.tech-archive.net/Archive/Development/microsoft.public.win32.programmer.kernel/2004-05/0439.html
Okay, as Alex suggested, an alternative is to check for comX filenames.
DOS special filename comX seem to be still reserved.
for {set i 1} {$i <= 9} {incr i} {
puts -nonewline [file exists ./com$i]
}
==>111111111
For my WinXP all com1..com9 are "existing", although physically I have
only com1,com3,com4.
So I'm drifting to the solution, that a name check in TclWinChan.c
before opening the serial port would be the easiest solution. That needs
adding a TclWinSerialOpen() procedure to TclWinSerial, or perhaps better
rename and change it to
HANDLE TclWinSerialOpen(handle, name, acces)
{
ThreadSpecificData *tsdPtr;
tsdPtr = SerialInit();
/*
* If the channel is already open (handle != 0)
* then reopen it with the OVERLAPPED flag set.
* Multithreaded I/O needs the overlapped flag set otherwise
* ClearCommError blocks under Windows NT/2000 until serial output is
* finished
*/
if ( handle != 0 ) {
if (CloseHandle(handle) == FALSE) {
return INVALID_HANDLE_VALUE;
}
}
handle = (*tclWinProcs->createFileProc)(name, access, 0, 0,
OPEN_EXISTING, FILE_FLAG_OVERLAPPED, 0);
return handle;
}
Rolf.
Rolf Schroedter
Alexandre Ferrieux wrote, On 10.12.2008 09:13:
> Anyway, I believe the workaround need *not* cover all cases, since it
> can fall back to the current behavior in case of non-match.
> As a summary:
>
> - "fast lane" for {^COM[0-9]+[:]?$}
> - Current behavior otherwise
Good idea, I like it.
But I would like to include the {\\.\COM10} specification, because
"problematic" serial devices (BlueTooth, mobile phones, Usb2Serial)
often tend to use even higher ports >= com10.
> By "fast lane" of course I mean directly open with the proper settings
> for a serial port (overlapped, whatever), hence no double-open and no
> timing issues regardless of the underlying driver.
>
> The advantage of the simplified regexp above is that it can be tested
> without the regexp engine, hence very fast. This is important since
> this check will be on the critical path for *any* [open]. (this
> precludes things like real-time registry lookup ;-)
>
> Then if someone uses one of the more complex patterns like \\machine
> \foo\bar\..\..\cOm199 and complains that it fails for his specific
> military-grade microwave oven, we'll see ;-)
>
> Do you agree ?
Yes, I fully agree.
Thanks a lot.
The change to TclWinSerial.c is very simple, see my other message.
Will you implemt the TclWinChan.c part, or should I ?
Backport for the Tcl8.4 tree, or 8.5 only ?
Regards, Rolf.
Alexandre Ferrieux
>
> Yes, I fully agree.
> Thanks a lot.
>
> The change to TclWinSerial.c is very simple, see my other message.
> Will you implemt the TclWinChan.c part, or should I ?
> Backport for the Tcl8.4 tree, or 8.5 only ?
I have created a Patch tracker entry (without actual patch ;-)
#2413550 at:
https://sourceforge.net/tracker/index.php?func=detail&aid=2413550&group_id=10894&atid=310894
Please upload your patch there, I will review it (with the help of the
Windows experts who are floating around c.l.t of course ;-).
-Alex
Rolf Schroedter
Okay, thanks Alex.
One question is coming up dubugging the implementation:
I see that Tcl_FSGetNativePath() at the beginning of
TclpOpenFileChannel() converts my channel name from {\\.\com12} to //./com12
Is this correct ?
That means I should compare the nativeName with "//./com" instead of
"\\\\.\\com" ?
Rolf.
Alexandre Ferrieux
> Alexandre Ferrieux wrote, On 10.12.2008 11:28:
>
> > On Dec 10, 11:02 am, Rolf Schroedter <m...@privacy.net> wrote:
> >> Yes, I fully agree.
> >> Thanks a lot.
>
> >> The change to TclWinSerial.c is very simple, see my other message.
> >> Will you implemt the TclWinChan.c part, or should I ?
> >> Backport for the Tcl8.4 tree, or 8.5 only ?
>
> > I have created a Patch tracker entry (without actual patch ;-)
> > #2413550 at:
>
>
> > Please upload your patch there, I will review it (with the help of the
> > Windows experts who are floating around c.l.t of course ;-).
>
> Okay, thanks Alex.
> One question is coming up dubugging the implementation:
> I see that Tcl_FSGetNativePath() at the beginning of
> TclpOpenFileChannel() converts my channel name from {\\.\com12} to //./com12
> Is this correct ?
> That means I should compare the nativeName with "//./com" instead of
> "\\\\.\\com" ?
In fact it is the converse: the "portable" (i.e. script-level) path
should be //./com12, and internally it would be converted to the
native form \\.\com12.
But in Tcl the spirit is to have the FS interfaces tolerant, so that
callers can also provide native strings directly. So the following
both work in Windows:
open //./com1
open \\\\.\\com1
So the bottom line is that our "name hint" should operate on the
transform string, hence after the Tcl_FSGetNativePath(). Hence it
suffices to match against the backslash-based pattern.
-Alex
Rolf Schroedter
Hmmmh, that was my first thought.
But suprisingly I see that after Tcl_FSGetNativePath()
nativeName=="//./com12", therfore comparing with "\\\\.\\com" fails...
I'm compiling and debugging with VC 6.0 using the workspace file
"tcl8.5.5/win/tcl.dsw" under WinXP.
Rolf.
Alexandre Ferrieux
Ugh ! That's peculiar. Will test as soon as I get my hands on my home
mingw.
-Alex
Rolf Schroedter
> Ugh ! That's peculiar. Will test as soon as I get my hands on my home
> mingw.
In all cases, when compiling tcl8.5.5 with
- gcc (msys)
- VC 6.0 using "tcl.dsw" or using "buildall.vc.bat"
I'm getting the same result:
Tcl_FSGetNativePath() converts {\\.\com12} to "//./com12"
and "//./com12" stays the same "//./com12"
Rolf.
sv
>
> I have seen the RN-41 module supports flash updates of its firmware.
> Can you check with Roving Networks if they are aware of the problem
> and provide an updated firmware?
Oliver,
I checked with Roving. The module I was using for test had a firmware
revison of 4.3.
This is what required a min. delay of 225ms between closing and
opening a port when I used Docklight scripting.
The latest revision is 4.71. I have another module that has the
revision on it.
With that and Docklight scripting, I still require a min. delay of
125ms to be able to consistently close and open the port using
Docklight scripting.
> One final question - which Bluetooth serial port drivers are you using
> on your PC, the "Native Microsoft Bluetooth Software" or the "WIDCOMM
> Bluetooth Software"? Or neither of them?
I was using the WIDCOMM BT software initially. I have rolled back to
the native MS BT software now. All my earlier observations (not
including tests with Docklight scripting) were all the same between
either of the above two BT software.
-sv
sv
wrote:
>
> Anyway, I believe the workaround need *not* cover all cases, since it
> can fall back to the current behavior in case of non-match.
> As a summary:
>
> - "fast lane" for {^COM[0-9]+[:]?$}
> - Current behavior otherwise
>
> By "fast lane" of course I mean directly open with the proper settings
> for a serial port (overlapped, whatever), hence no double-open and no
> timing issues regardless of the underlying driver.
Alex/Rolf,
So as I understand you are implementing the above scheme with a 'fast
lane' check, which if it fails, you will still revert to the current
behavior. I think this would be necessary because apparently, virtual
serial ports (which is what BT serial devices employ) can have any
name and not just COMx.
source:
http://en.wikipedia.org/wiki/Serial_port
excerpt:
A virtual serial port is an emulation of the standard serial port.
This port is created by software which enable extra serial ports in an
operating system without additional hardware installation (such as
expansion cards, etc.). Unlike a physical serial port the virtual one
can be assigned any name (COM255, VSP33, etc.).
I must add here though that my independent attempts to validate this
by trying to change the COM port name to something else was not
successful, but I didn't spend a lot of time on that.
-sv
Alexandre Ferrieux
OK I'm starting to get the picture. At least the first half :-}
What happens is that FSGetNativePath, when given any form of path,
first computes a normalized, forward-slash-based internal rep
(SetFsPathFromAny), *then* projects this back into the "native"
universe.
But here, and that is the fuzzier second half of the story, the
projection for some reason does nothing, seemingly happy with the
forward slashes. So in fine, the Win32 syscall CreateFile is called
with forward slashes ! Could a Windows-knowledgeable person explain ?
-Alex
sv
> that the match (inside Windows) occurs on the "file tail" (in Tcl
> vernacular). So that it is really impossible to open a create a file
> named "com1" prefixed by any path...
>
Just an FYI:
Extending that experiment, it appears that only COM1 to COM9 are
locked out by windows. All higher order COMx can still be created and
Windows (XP SP3 that I am running) does not prevent that.
-sv
Rolf Schroedter
> But here, and that is the fuzzier second half of the story, the
> projection for some reason does nothing, seemingly happy with the
> forward slashes. So in fine, the Win32 syscall CreateFile is called
> with forward slashes ! Could a Windows-knowledgeable person explain ?
Hmmmm ... seems that forward slashes are officially supported by the
Win32-API (perhaps for URL's). See the Note in
http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx
"Note File I/O functions in the Windows API convert "/" to "\"
as part of converting the name to an NT-style name, except when
using the "\\?\" prefix as detailed in the following sections.
My only question in this respect is:
Can I rely on getting forward-slashed filename for the comport
name-hint, or should I better test both: "/" and "\" conventions ?
Rolf.
Rolf Schroedter
I'm just debugging the patch avoiding double-opening of native serial
ports. You seem the understand the PORTMON output ...
Could you please enlighten me, what the following output means ?
1. I'm opening COM1 (Serial0) once with the new procure, while
2. COM3 (SsPort1) is opened with the old-style open-close-reopen.
I'm confused to see for
1. COM1 procedure:
IRP_MJ_CREATE/RP_MJ_CLOSE/IRP_MJ_CREATE/RP_MJ_CLOSE/IRP_MJ_CREATE
2. COM3 procedure:
IRP_MJ_CREATE/RP_MJ_CLOSE/IRP_MJ_CREATE/RP_MJ_CLOSE/IRP_MJ_CREATE/RP_MJ_CLOSE/IRP_MJ_CREATE
At least for the COM3 procedure there is one additional CLOSE/CREATE.
Many thanks for your help.
Regards, Rolf.
=================== 1. COM1 =====================
0.00004495 tclsh85tg.exe IRP_MJ_CREATE Serial0 SUCCESS Options: Open
0.00000065 tclsh85tg.exe IRP_MJ_QUERY_INFORMATION Serial0 INVALID
PARAMETER -1162332900
0.00000020 tclsh85tg.exe IRP_MJ_QUERY_INFORMATION Serial0 INVALID
PARAMETER -1162332132
0.00000126 tclsh85tg.exe IRP_MJ_CLEANUP Serial0 SUCCESS
0.00921132 tclsh85tg.exe IRP_MJ_CLOSE Serial0 SUCCESS
0.00006063 tclsh85tg.exe IRP_MJ_CREATE Serial0 SUCCESS Options: Open
0.00000064 tclsh85tg.exe IRP_MJ_QUERY_INFORMATION Serial0 INVALID
PARAMETER -1162332132
0.00000140 tclsh85tg.exe IRP_MJ_CLEANUP Serial0 SUCCESS
0.01548667 tclsh85tg.exe IRP_MJ_CLOSE Serial0 SUCCESS
0.00005253 tclsh85tg.exe IRP_MJ_CREATE Serial0 SUCCESS Options: Open
0.00000094 tclsh85tg.exe IOCTL_SERIAL_SET_QUEUE_SIZE Serial0 SUCCESS
InSize: 4096 OutSize: 4096
0.00000169 tclsh85tg.exe IOCTL_SERIAL_PURGE Serial0 SUCCESS Purge:
TXABORT RXABORT TXCLEAR RXCLEAR
0.00000036 tclsh85tg.exe IOCTL_SERIAL_SET_TIMEOUTS Serial0 SUCCESS RI:0
RM:0 RC:0 WM:0 WC:0
=================== 2. COM3 =====================
0.00346219 tclsh85tg.exe IRP_MJ_CREATE SsPort1 SUCCESS Options: Open
0.00000050 tclsh85tg.exe IRP_MJ_QUERY_INFORMATION SsPort1 INVALID
PARAMETER -1162332900
0.00000029 tclsh85tg.exe IRP_MJ_QUERY_INFORMATION SsPort1 INVALID
PARAMETER -1162332132
0.00000165 tclsh85tg.exe IRP_MJ_CLEANUP SsPort1 SUCCESS
0.00002127 tclsh85tg.exe IRP_MJ_CLOSE SsPort1 SUCCESS
0.01551844 tclsh85tg.exe IRP_MJ_CREATE SsPort1 SUCCESS Options: Open
0.00000040 tclsh85tg.exe IRP_MJ_QUERY_INFORMATION SsPort1 INVALID
PARAMETER -1162332132
0.00000138 tclsh85tg.exe IRP_MJ_CLEANUP SsPort1 SUCCESS
0.00001835 tclsh85tg.exe IRP_MJ_CLOSE SsPort1 SUCCESS
0.01552172 tclsh85tg.exe IRP_MJ_CREATE SsPort1 SUCCESS Options: Open
0.00000339 tclsh85tg.exe IOCTL_SERIAL_GET_BAUD_RATE SsPort1 SUCCESS
0.00000086 tclsh85tg.exe IOCTL_SERIAL_GET_LINE_CONTROL SsPort1 SUCCESS
0.00000052 tclsh85tg.exe IOCTL_SERIAL_GET_CHARS SsPort1 SUCCESS
0.00000061 tclsh85tg.exe IOCTL_SERIAL_GET_HANDFLOW SsPort1 SUCCESS
0.00000231 tclsh85tg.exe IRP_MJ_CLEANUP SsPort1 SUCCESS
0.00004364 tclsh85tg.exe IRP_MJ_CLOSE SsPort1 SUCCESS
0.01538871 tclsh85tg.exe IRP_MJ_CREATE SsPort1 SUCCESS Options: Open
0.00000152 tclsh85tg.exe IOCTL_SERIAL_SET_QUEUE_SIZE SsPort1 SUCCESS
InSize: 4096 OutSize: 4096
0.00000230 tclsh85tg.exe IOCTL_SERIAL_PURGE SsPort1 SUCCESS Purge:
TXABORT RXABORT TXCLEAR RXCLEAR
0.00000070 tclsh85tg.exe IOCTL_SERIAL_SET_TIMEOUTS SsPort1 SUCCESS RI:0
RM:0 RC:0 WM:0 WC:0
Rolf Schroedter
I'll send you an email with a zipped static tclsh.exe.
Could you please check, whether the new behavior (single open)
solves your problem ?
New behavior (single open):
open com1 r+
open //./com12 r+
Enforce Old behavior (open-close-reopen)
open /com1 r+
Alexandre Ferrieux
> Alexandre Ferrieux wrote, On 10.12.2008 22:46:
> > But here, and that is the fuzzier second half of the story, the
>
> > projection for some reason does nothing, seemingly happy with the
> > forward slashes. So in fine, the Win32 syscall CreateFile is called
> > with forward slashes ! Could a Windows-knowledgeable person explain ?
>
> Hmmmm ... seems that forward slashes are officially supported by the
>
> "Note File I/O functions in the Windows API convert "/" to "\"
> as part of converting the name to an NT-style name, except when
> using the "\\?\" prefix as detailed in the following sections.
Credit where credit is due :-)
Thanks do David Gravereaux for whispering the above in our ears by
private e-mail.
> My only question in this respect is:
> Can I rely on getting forward-slashed filename for the comport
> name-hint, or should I better test both: "/" and "\" conventions ?
Yes indeed. As David puts it, it sounds like Tcl_FSGetNativePath()
isn't being all that native !
I will bring up the issue to other maintainers; with luck I'll find
the person who _decided_ that :)
In the meantime, the best is probably to check both...
Hmmm, I see you're proposing an executable to sv. Can you upload the
corresponding patch to "our" SF entry ?
-Alex
Rolf Schroedter
>
>> My only question in this respect is:
>> Can I rely on getting forward-slashed filename for the comport
>> name-hint, or should I better test both: "/" and "\" conventions ?
>
> Yes indeed. As David puts it, it sounds like Tcl_FSGetNativePath()
> isn't being all that native !
> I will bring up the issue to other maintainers; with luck I'll find
> the person who _decided_ that :)
>
> In the meantime, the best is probably to check both...
Done.
> Hmmm, I see you're proposing an executable to sv. Can you upload the
> corresponding patch to "our" SF entry ?
When I open
https://sourceforge.net/tracker2/?func=detail&aid=2413550&group_id=10894&atid=310894
I can "Add a comment", but I can't attach a patch file.
"Adding a new artifact" is probably not what is desired.
Should I copy-paste the patch into "Add a comment" field ?
Rolf.
Alexandre Ferrieux
> Alexandre Ferrieux wrote, On 11.12.2008 12:03:
>
>
>
> >> My only question in this respect is:
> >> Can I rely on getting forward-slashed filename for the comport
> >> name-hint, or should I better test both: "/" and "\" conventions ?
>
> > Yes indeed. As David puts it, it sounds like Tcl_FSGetNativePath()
> > isn't being all that native !
> > I will bring up the issue to other maintainers; with luck I'll find
> > the person who _decided_ that :)
>
> > In the meantime, the best is probably to check both...
>
> Done.
>
> > Hmmm, I see you're proposing an executable to sv. Can you upload the
> > corresponding patch to "our" SF entry ?
>
> I can "Add a comment", but I can't attach a patch file.
> "Adding a new artifact" is probably not what is desired.
Yes, this is the behavior for anonymous connections. Create yourself
an SF login to get the upload privilege.
> Should I copy-paste the patch into "Add a comment" field ?
If you prefer :) (But creating an SF login is cheap)
-Alex
Rolf Schroedter
> Yes, this is the behavior for anonymous connections. Create yourself
> an SF login to get the upload privilege.
I have a SF account and I'm logged in, logname=schroedter.
Seems, that by default only the author has the right to upload a file.
So I had to create a new patch artifact, and -surprise- I'm able to
upload, but I guess, you don't ...
https://sourceforge.net/tracker2/?func=detail&aid=2417238&group_id=10894&atid=310894
Regards, Rolf.
Alexandre Ferrieux
OK, re-uploaded your patch to the initial report... pheeew !
Thanks.
-Alex
sv
> Hi SV,
> I'll send you an email with a zipped static tclsh.exe.
> Could you please check, whether the new behavior (single open)
> solves your problem ?
Works like a charm, everytime!
I tried the case where the SPP of the BT module of mine gets assigned
a high COM number (e.g., COM22).
Using the syntax "open //./COM22" works everytime.
And then I tried the case, where I reassigned the SPP (through the
Device Manager) to COM2.
This again worked by using either "open COM2" or "open //./COM2"
Rolf/Oliver/Alex...you guys have been great! Thanks for helping me get
past that.
-sv