Repeatable bug where beaglebone black loses bytes from it's serial connection.
1,068 views
Skip to first unread message
David Howlett
Apr 21, 2017, 12:18:37 PM4/21/17
to BeagleBoard
I have spent about two days tracking down an issue in one of my machines. I am streaming pressure readings from an i2c pressure sensor to a PC at high speed. I have reduced the problem to a small test case. To replicate the fault I connect the beaglebone black to a PC, connect to it with serial over the micro USB port, login and launch a command that streams data fast. On the PC run a program that collects the streamed data more slowly than it is being created.
import serial
x = serial.Serial('COM15')
to_send = b'\x03yes HelloWorld\n'
x.write(to_send)
while True:
received = x.readline()
if received != b'HelloWorld\r\n':
print(received)
The python program above prints lines like:
b'ld\r\n'
b'HelloWoroWorld\r\n'
b'HelloWorldorld\r\n'
b'HelloWorldorld\r\n'
b'ld\r\n'
b'ld\r\n'
b'ld\r\n'
b'ld\r\n'
b'ld\r\n'
b'ld\r\n'
b'HelloWoroWorld\r\n'
b'ld\r\n'
b'HelloWlloWorld\r\n'
b'ld\r\n'
b'HelloWorldorld\r\n'
At about 4 lines per second. If I set the program on the beagle bone to stream slowly then this issue goes away. It appears to be an issue with a buffer filling up somewhere between the output of the yes command and reading bytes from the windows API. The correct behaviour is that the yes command should be stopped when the buffer is full to prevent the loss of data, then allowed to continue when there is space in the buffer again. I observe that the streaming command (in this case yes) is stopped but there is still data loss happening somewhere.
I wrote this in python because it is the language I am most comfortable in but if you needed me to I could rewrite the example in some other language. To run the program shown above you will need pyserial and you will need to change 'COM15' to be whatever the port is called on your computer.
David Howlett
Apr 21, 2017, 12:26:16 PM4/21/17
to BeagleBoard
I should mention I am using:
- Windows 10
- Drivers from about 2 weeks ago
- A beaglebone image from about 2 weeks ago
Przemek Klosowski
Apr 21, 2017, 1:17:56 PM4/21/17
to beagl...@googlegroups.com
It looks like the serial output buffer overflow. I think you're using the USB serial port, so it's probably not lost in the USB link, but on the Windows side the USB serial driver is presumably overwriting the output buffer. Would you have a Linux system somewhere to cross-check it by using a different OS/serial driver combination?
Can you change the settings in the Windows serial port, asking for hardware flow control? To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/f4c86058-1fee-41fb-bc6f-d41f8eccb3b6%40googlegroups.com.--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard+unsubscribe@googlegroups.com.
arsi
Apr 21, 2017, 2:29:18 PM4/21/17
to beagl...@googlegroups.com
Hi,
You have to create a state machine, at start you can have something in the serial buffer and you will lose receiver sync..
Java example:
ArSi
You have to create a state machine, at start you can have something in the serial buffer and you will lose receiver sync..
Java example:
private enum Stage {
FIND_STAGE1,
FIND_STAGE2,
FIND_STAGE3,
FIND_STAGE4,
FIND_STAGE5
}
public void test() {
Stage currentStage = Stage.FIND_STAGE1;
ByteBuffer buffer = ByteBuffer.allocate(255);
while (true) {
buffer.clear();
int count = readCom(buffer);
if (count < 1) {
continue;
}
for (int i = 0; i < count; i++) {
int j = buffer.get(i) & 0xff;
switch (currentStage) {
case FIND_STAGE1:
if (j == 0x10) {
currentStage = Stage.FIND_STAGE2;
}
break;
case FIND_STAGE2:
if(j == 0x11){
currentStage = Stage.FIND_STAGE3;
}else{
currentStage = Stage.FIND_STAGE1;
}
break;
case FIND_STAGE3:
if(j == 0x12){
currentStage = Stage.FIND_STAGE4;
}else{
currentStage = Stage.FIND_STAGE1;
}
break;
case FIND_STAGE4:
if(j == 0x13){
currentStage = Stage.FIND_STAGE5;
}else{
currentStage = Stage.FIND_STAGE1;
}
break;
case FIND_STAGE5:
System.out.println("received header: 0x10,0x11,0x12,0x13");
System.out.println("received data: "+j);
currentStage = Stage.FIND_STAGE1;
break;
}
}
}
}ArSi
--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/d63ea012-b3ea-49b6-9b13-f3e3be9c51ca%40googlegroups.com.
Charles Steinkuehler
Apr 21, 2017, 6:10:16 PM4/21/17
to beagl...@googlegroups.com
On 4/21/2017 11:18 AM, David Howlett wrote:
> I have spent about two days tracking down an issue in one of my machines. I am
> streaming pressure readings from an i2c pressure sensor to a PC at high speed. I
> have reduced the problem to a small test case. To replicate the fault I connect
> the beaglebone black to a PC, connect to it with serial over the micro USB port,
> login and launch a command that streams data fast. On the PC run a program that
> collects the streamed data more slowly than it is being created.
This is expected behavior unless you are using hardware flow control
> I have spent about two days tracking down an issue in one of my machines. I am
> streaming pressure readings from an i2c pressure sensor to a PC at high speed. I
> have reduced the problem to a small test case. To replicate the fault I connect
> the beaglebone black to a PC, connect to it with serial over the micro USB port,
> login and launch a command that streams data fast. On the PC run a program that
> collects the streamed data more slowly than it is being created.
or something like XON/XOFF to throttle the Tx side.
Otherwise, if you transmit data faster than a receiver can process it,
you will loose data.
...looks like you need a faster PC! ;-)
--
Charles Steinkuehler
cha...@steinkuehler.net
Graham
Apr 21, 2017, 6:26:20 PM4/21/17
to BeagleBoard
What is the current speed you are using for the serial ports?
If you are putting streaming character data into the transmit buffer faster than
the serial port can send it, you can also lose data.
One way to fix this is to set the serial port to a higher speed, on both ends.
I think the BBB and most PCs are well behaved up to 256000 bits per second.
--- Graham
==
William Hermans
Apr 21, 2017, 8:23:05 PM4/21/17
to beagl...@googlegroups.com
On Fri, Apr 21, 2017 at 3:26 PM, Graham <gra...@flex-radio.com> wrote:
What is the current speed you are using for the serial ports?There is also a transmit buffer inside the serial port transmitter in the BBB.If you are putting streaming character data into the transmit buffer faster thanthe serial port can send it, you can also lose data.One way to fix this is to set the serial port to a higher speed, on both ends.I think the BBB and most PCs are well behaved up to 256000 bits per second.
I recall our local serial expert stating that the beaglebone's UART is capable of up to at least 4Mbit.
William Hermans
Apr 21, 2017, 8:27:38 PM4/21/17
to beagl...@googlegroups.com
I think Charles' suggestion is probably the best way to go about dealing with this issue. Enable some kind of hardware flow control. XON / XOFF is probably a good bet.
Dennis Lee Bieber
Apr 21, 2017, 9:22:46 PM4/21/17
to beagl...@googlegroups.com
On Fri, 21 Apr 2017 17:27:23 -0700, William Hermans
<yyr...@gmail.com> declaimed the following:
>I think Charles' suggestion is probably the best way to go about dealing
>with this issue. Enable some kind of hardware flow control. XON / XOFF is
>probably a good bet.
>
Technically, XON/XOFF is not hardware flow control -- it is software
flow control, relying on the receiving end sending a character to the
sending side which has to be read and interpreted (hopefully by the driver
and not the application) to start/stop transmission.
RTS/CTS (ready to send/clear to send) is hardware flow control, since
it uses extra signal wires to actually affect the other side UART. DTR/DSR
(data terminal ready/data set [modem] ready) is the other hardware set as I
recall.
Of course, once you go through a USB<>serial converter (as I recall,
the OP is using the USB connection, not the debug or other wired UART),
anything goes -- since USB doesn't have hardware serial lines running end
to end; the adapter has to convert hardware flow control to special USB
packets which are then converted back on the other side. It should probably
also be handling some of the buffering -- perhaps by setting RTS/CTS itself
when /it/ is nearing full before the other end polls the USB serial
connection, and the other end probably doesn't have a physical UART either,
just USB packet buffer.
--
Wulfraed Dennis Lee Bieber AF6VN
wlf...@ix.netcom.com HTTP://wlfraed.home.netcom.com/
<yyr...@gmail.com> declaimed the following:
>I think Charles' suggestion is probably the best way to go about dealing
>with this issue. Enable some kind of hardware flow control. XON / XOFF is
>probably a good bet.
>
flow control, relying on the receiving end sending a character to the
sending side which has to be read and interpreted (hopefully by the driver
and not the application) to start/stop transmission.
RTS/CTS (ready to send/clear to send) is hardware flow control, since
it uses extra signal wires to actually affect the other side UART. DTR/DSR
(data terminal ready/data set [modem] ready) is the other hardware set as I
recall.
Of course, once you go through a USB<>serial converter (as I recall,
the OP is using the USB connection, not the debug or other wired UART),
anything goes -- since USB doesn't have hardware serial lines running end
to end; the adapter has to convert hardware flow control to special USB
packets which are then converted back on the other side. It should probably
also be handling some of the buffering -- perhaps by setting RTS/CTS itself
when /it/ is nearing full before the other end polls the USB serial
connection, and the other end probably doesn't have a physical UART either,
just USB packet buffer.
--
Wulfraed Dennis Lee Bieber AF6VN
wlf...@ix.netcom.com HTTP://wlfraed.home.netcom.com/
David Howlett
Apr 27, 2017, 12:54:42 PM4/27/17
to BeagleBoard
Thanks for all the replies.
Reply to Przemek Klowoski
=========================
I agree it is almost certainly a serial buffer overflow somewhere. By filling and emptying the buffers between my code on the BBB and my code on the PC I can detect a 12KB buffer. I believe that this 12KB buffer is in the BBB driver on the PC. I tried to set the driver's buffer to be bigger but the driver appears to ignore all calls to the WINAPI SetupComm system call.
I have tried OSX but I still get data loss.
I have tried to use a second BeagleBone as PC replacement but the first beagle bone does not show up on the list of available serial ports.
I have tried:
x = serial.Serial('COM10', xonxoff=True)
x = serial.Serial('COM10', rtscts=True)
x = serial.Serial('COM10', dsrdtr=True)
None of these cause any change in behaviour.
I have also tried sending stop commands directly:
x.write(b'\x13') # byte 0x13 is XOFF
This does not cause any change in behaviour either.
Reply to arsi
=============
If I understand you correctly, you are recommending adding a 4 byte sequence "0x10 0x11 0x12 0x13" on a regular basis in the streaming data to allow the receiving PC to sync up with the stream. Is that what you mean? For my current application the data looks like:
10392.61424 125 1299
10392.61469 125 1242
10392.61514 125 1357
10392.61558 125 1385
10392.61628 125 1357
10392.61674 125 1242
10392.61718 125 1328
10392.61765 125 1473
10392.61810 125 1212
10392.61880 125 1299
It is delimited by "/r/n" which allows easy syncing up. The trouble is that bytes can go missing to make the above data look like:
10392.61424 125 1299
10392.61469 125 1242
10392.61514 125 1357
10392.61558 125 1385
103921628 125 1357
10392.61674 125 1242
10392.61718 125 1328
10392.61765 125 1473
10392.61810 125 1212
10392.61880 125 1299
I could make my format checking stricter and I could add checksums to every line. This would cut down the error rate but I would prefer to fix the cause of the data loss if possible.
Reply to Graham
===============
I have tried baud rates on the PC from 10Hz to 100GHz including common baud rates like 9600. The serial connection works at all baud rates tested. There appears to be no change in the data rate or the error rate. To rule out a bug in the pyserial library I have also changed the baud rate with a GUI terminal called "Termite 3.1". This also does not change the data rate. I believe that the serial port is a virtual device and all commands to change the baud rate are ignored by the driver.
x = serial.Serial('COM10', baudrate=10)
x = serial.Serial('COM10', baudrate=9600)
x = serial.Serial('COM10', baudrate=100_000_000_000)
My current ideas
=============
I could add checksums. I could switch from a virtual serial port to using a pair of physical USB-RS232 adapters. This would go: BBB USB->hardware RS232->PC USB. I only mention this because I have plenty of experience with physical RS232 and I have USB-RS232 adapters which respond to the win32 SetupComm call to change the buffer size. I could also switch to something else entirely like SSH over the ethernet port.
Dennis Lee Bieber
Apr 27, 2017, 8:00:36 PM4/27/17
to beagl...@googlegroups.com
On Thu, 27 Apr 2017 09:54:42 -0700 (PDT), David Howlett
<david.h...@gmail.com> declaimed the
following:
>
>x = serial.Serial('COM10', xonxoff=True)
>x = serial.Serial('COM10', rtscts=True)
>x = serial.Serial('COM10', dsrdtr=True)
>
These will only be of use if both sides of the link are set for the
same protocol. The above look to be Windows COM port names -- you need to
have the BBB also set to use those protocols (I suspect the default is for
no flow control).
>I have tried baud rates on the PC from 10Hz to 100GHz including common baud
>rates like 9600. The serial connection works at all baud rates tested.
>There appears to be no change in the data rate or the error rate. To rule
>out a bug in the pyserial library I have also changed the baud rate with a
>GUI terminal called "Termite 3.1". This also does not change the data rate.
>I believe that the serial port is a virtual device and all commands to
>change the baud rate are ignored by the driver.
>
If you are going through the USB port, that would tend to be correct as
the driver is just moving USB packet data (which are polled at USB rates)
into a "serial" buffer for reading.
wouldn't have any affect on baudrate on the BBB => though if one were using
real serial ports and not going through a USB translation (at both ends),
getting a mismatch in serial port speeds will result in garbage (seen at my
former employer when using USB->Serial dongles to connect to equipment with
physical serial ports; the serial end of the dongle has to match baudrate
of the physical equipment, otherwise garbage is seen and sent via USB).
<david.h...@gmail.com> declaimed the
following:
>
>x = serial.Serial('COM10', xonxoff=True)
>x = serial.Serial('COM10', rtscts=True)
>x = serial.Serial('COM10', dsrdtr=True)
>
same protocol. The above look to be Windows COM port names -- you need to
have the BBB also set to use those protocols (I suspect the default is for
no flow control).
>I have tried baud rates on the PC from 10Hz to 100GHz including common baud
>rates like 9600. The serial connection works at all baud rates tested.
>There appears to be no change in the data rate or the error rate. To rule
>out a bug in the pyserial library I have also changed the baud rate with a
>GUI terminal called "Termite 3.1". This also does not change the data rate.
>I believe that the serial port is a virtual device and all commands to
>change the baud rate are ignored by the driver.
>
the driver is just moving USB packet data (which are polled at USB rates)
into a "serial" buffer for reading.
>
>x = serial.Serial('COM10', baudrate=10)
>x = serial.Serial('COM10', baudrate=9600)
>x = serial.Serial('COM10', baudrate=100_000_000_000)
>
And as mentioned above -- changing baudrate on the Windows side
>x = serial.Serial('COM10', baudrate=10)
>x = serial.Serial('COM10', baudrate=9600)
>x = serial.Serial('COM10', baudrate=100_000_000_000)
>
wouldn't have any affect on baudrate on the BBB => though if one were using
real serial ports and not going through a USB translation (at both ends),
getting a mismatch in serial port speeds will result in garbage (seen at my
former employer when using USB->Serial dongles to connect to equipment with
physical serial ports; the serial end of the dongle has to match baudrate
of the physical equipment, otherwise garbage is seen and sent via USB).
arsi
Apr 28, 2017, 2:47:00 AM4/28/17
to beagl...@googlegroups.com
Hi,
I use a similar algorithm on 76800 baud for RS485 Honeywell C-bus token ring protocol and it works without problems from java on BBB and PC..
And I noticed that you nowhere call a flush when sending data, that can be the problem..
Test it with the state machine, It must work..
Here is my reading algorithm from c lib on linux it detects a pause in the transfer:
timeout - read timeout in seconds
usReadPause - after no incomming data on port for x micro seconds, return from read sequence.. (return one data packet)
jint user_com_apli_jni_Cbridge_serialReceiveInterSpacingReadPause(JNIEnv * env, jobject object,jint handle,jobject buffer,jint usReadPause,jint timeout){
size_t capacity = env->GetDirectBufferCapacity(buffer);
fd_set readfs;
timeval tv;
tv.tv_sec = timeout / 1000;
tv.tv_usec = 0;
void *buff = env->GetDirectBufferAddress(buffer);
int size = 0;
int rd = 0;
FD_ZERO(&readfs);
FD_SET(handle, &readfs);
select(handle + 1, &readfs, NULL, NULL, &tv );
if (FD_ISSET(handle, &readfs)) {
size = read(handle, buff, capacity);
rd = size;
do {
usleep(usReadPause);
buff = ((char*) buff) + rd;
rd = read(handle, buff, capacity - size);
if (rd > 0) {
size += rd;
}
} while (rd > 0 && capacity - size > 0);
env->SetIntField(buffer, limitFieldId, size);
return size;
}
return 0;
}
From: David Howlett
Sent: Thursday, April 27, 2017 6:54PM
To: Beagleboard
Subject: [beagleboard] Re: Repeatable bug where beaglebone black loses bytes
from it's serial connection.
David Howlett
May 2, 2017, 9:26:01 AM5/2/17
to BeagleBoard, ar...@chello.sk
I couldn't easily find documentation on how to enable flow control on the beaglebone side so I chose to add a CRC32 checksum to each line. The data now looks like:
102.04541 125 1186 3665238047
102.04756 125 1245 943190513
102.04803 125 1303 526161833
102.04850 125 1274 3970916767
102.04898 125 1273 472341404
102.04970 125 1100 2928958186
102.05018 125 1244 2418030567
102.05113 125 1157 1456340796
102.05161 125 1273 258216438
102.05234 125 1012 3972342546
102.05282 125 1214 2310087250
102.05329 125 1274 2579294272
102.05377 125 1215 1325398437
This means that corrupted lines are easy to detect. I then deal with data being missing later on in the analysis pipeline.
>> Here is my reading algorithm from c lib on linux it detects a pause in the transfer
Unfortunately the code I am writing on the PC side is called too infrequently for this to help. The 12k buffer can occasionally fill up and lose data in the gap between two calls. I could make a seperate thread regularly poll the windows API to see if there was new data but a checksum was simpler to implement as I am not familiar with threading.
William Hermans
May 2, 2017, 5:06:00 PM5/2/17
to beagl...@googlegroups.com
On Tue, May 2, 2017 at 6:26 AM, David Howlett <david.h...@gmail.com> wrote:
I couldn't easily find documentation on how to enable flow control on the beaglebone side so I chose to add a CRC32 checksum to each line. The data now looks like:
This is not really so much a beaglebone thing, and more of a Linux thing. The problem is documentation. As in it exists, but one part of the equation does not necessarily have anything to do with the other. As such, it's hard to find information if you know nothing about what you're looking for. Trust me, I've been using Debian nearly since it's first release, and I personally do not know how this is to be done from hands on experience. However, I can shed some light on the subject whcih should allow you to experiment on your own, and have something working fairly quickly if you have a decent amount of experience with Linux in general.
First, you're going to need to learn how to use systemd, and specifically, you're going ot need to learn how to create, and use a startup service.
Second, you need a tool that is capable of working with serial devices at a "low level". Such as: https://manpages.debian.org/jessie/coreutils/stty.1.en.html and specifically this setting:
- [-]ixon
- enable XON/XOFF flow control
Then it's just a matter of using this tool properly form within the service, or perhaps calling a script from the service, that sets the serial port exactly how you want. You'll have to experiment to get things exactly how you want / need.
David Howlett
May 4, 2017, 12:32:04 PM5/4/17
to BeagleBoard
>> First, you're going to need to learn how to use systemd, and specifically, you're going to need to learn how to create, and use a startup service.
I happen to have used systemd startup services in a previous project so I can see how it would be useful for setting settings on startup.
>> Second, you need a tool that is capable of working with serial devices at a "low level". Such as: https://manpages.debian.org/jessie/coreutils/stty.1.en.html and specifically this setting:
This is the tool I was looking for and could not find, thank you. The default settings for the serial connection I am using are:
root@beaglebone:~/reusable# stty --all
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff
-iuclc -ixany imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke
I can now use the stty command to set ixon on or off.
root@beaglebone:~/reusable# stty -ixon
root@beaglebone:~/reusable# stty --all
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl -ixon -ixoff
-iuclc -ixany imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke
Unfortunately the data loss is the same regardless of whether ixon is enabled or disabled. Changing ixoff does not affect the data loss either.
I tried setting crtscts and cdtrdsr but this is not permitted.
root@beaglebone:~/reusable# stty crtscts
stty: standard input: unable to perform all requested operations
root@beaglebone:~/reusable# stty cdtrdsr
stty: invalid argument `cdtrdsr'
There is flow control regardless of what I do
=============================================
There does appear to be flow control of some sort. I realised I can get debug information with the CPU LED. I can make the CPU light on the BBB light up for 2.4 seconds by running the following program on the PC. The fact that the CPU light turns off again indicates that there is some kind of flow control that pauses the computation when the buffer fills up.
import time
import serial.tools.list_ports
# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
x.read_all()
x.write(b'python3.6 -c "while True: print(sum(i for i in range(1000)))"\n')
# pause forever
time.sleep(1_000_000)
If I slow down the CPU wasting command like this:
import time
import serial.tools.list_ports
# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
x.read_all()
x.write(b'python3.6 -c "while True: print(sum(i for i in range(10_000)))"\n')
# pause forever
time.sleep(1_000_000)
Then the CPU LED stays lit for about 10x as long. This demonstrates that streaming on the beagle bone is stopped by a buffer filling up rather than a timer.
Data loss only happens when the length of the data being written is not a multiple of 4 bytes
=============================================================================================
As an example if I run the following program there is no data loss:
import serial.tools.list_ports
# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
x.write(b'yes "123456"\n')
while True:
received = x.readline()
if received != b'123456\r\n':
print(received)
But there is data loss if I run this:
import serial.tools.list_ports
# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
x.write(b'yes "1234567"\n')
while True:
received = x.readline()
if received != b'1234567\r\n':
print(received)
It prints:
b'yes "1234567"\r\n'
b'11234567\r\n'
b'11234567\r\n'
b'12234567\r\n'
b'12334567\r\n'
b'12345677\r\n'
b'\n'
b'\n'
b'\n'
b'\n'
b'\n'
b'\n'
b'12345567\r\n'
b'\n'
b'12234567\r\n'
b'\n'
b'12334567\r\n'
b'12344567\r\n'
b'12344567\r\n'
b'\n'
b'\n'
b'12344567\r\n'
b'12234567\r\n'
b'12345567\r\n'
b'12345667\r\n'
b'\n'
b'\n'
b'12234567\r\n'
b'12334567\r\n'
b'12234567\r\n'
b'12345677\r\n'
b'12334567\r\n'
b'\n'
b'12345677\r\n'
b'1234567\r\r\n'
b'\n'
b'11234567\r\n'
b'12234567\r\n'
b'123234567\r\n'
b'12234567\r\n'
b'12334567\r\n'
b'\n'
b'12345677\r\n'
b'1234567\r\r\n'
b'12344567\r\n'
b'\n'
b'11234567\r\n'
b'\n'
b'12334567\r\n'
b'12345667\r\n'
b'12234567\r\n'
Note that every error printed only contains single byte errors
Conclusions
===========
- The virtual serial port provided is a poor imitation of a real serial port. Every command to change any property of either end of the connection either fails or is ignored.
- There is undocumented flow control somewhere behind the scenes outside my control. It is probably in a driver.
- The flow control mechanism loses data if the data is not a multiple of 4 bytes long.
Graham
May 4, 2017, 2:50:49 PM5/4/17
to BeagleBoard
David:
I am confused as to what your actual hardware configuration is, with respect to the "serial communications."
Are you using a COM port in Windows to talk to a USB to serial cable, which is talking to a hardware serial port in the BBB?
Or, are you doing something else?
If so, please describe.
I have never seen these kinds of problems when dealing with hardware serial ports.
--- Graham
==
William Hermans
May 4, 2017, 3:55:03 PM5/4/17
to beagl...@googlegroups.com
On Thu, May 4, 2017 at 9:32 AM, David Howlett <david.h...@gmail.com> wrote:
>> First, you're going to need to learn how to use systemd, and specifically, you're going to need to learn how to create, and use a startup service.I happen to have used systemd startup services in a previous project so I can see how it would be useful for setting settings on startup.
That good, something less to get in your way to fixing your problem.
Conclusions===========- The virtual serial port provided is a poor imitation of a real serial port. Every command to change any property of either end of the connection either fails or is ignored.- There is undocumented flow control somewhere behind the scenes outside my control. It is probably in a driver.- The flow control mechanism loses data if the data is not a multiple of 4 bytes long.
So, you've actually delved further into using the UARTs on this platform than I ever have. Even with other( bare metal ) platforms, I only really use UARTs for a printf() style serial debug output. With this platform, we do not need it, unless you're troublshooting the boot process.
Anyway, my first instinct wants to say baud rate is involved somehow. It's very unlikely a baud rate difference between systems, because usually when this occurs, your transmission will be garbled. At least a few characters here and there will be different between send, and receive. So this leads me to speculate that you're somehow exceeding your maximum baud rate.
Another thing I'm noticing from what your code is outputting. is that your "data loss" is not consistent. Which immediately makes me want to jump to the conclusion that you code is somehow being preempted. Almost as if you're data is on the stack, something preempts your code, and by the time the part of your code is given control back, the data on the stack is no longer there. This sort of situation is what I refer to as "stepping all over the stack". e.g. some of those routines you're using could be stack unfriendly. Granted, I know very little about Python, or the libraries Python uses. But I've personally experienced this effect first hand, when using a third party web server API, when dealing with a lot of data fast. CANBUS at 1Mbit, while decoding PGNs in real time, and attempting to send this data in real time using this third part web server API.
I wonder if you run the application that failing for you there is you pipe that data to a file, instead of to stdout, if you would be experiencing the same problem ? Why don't you humor me, and give that a try ? e.g. run your application as such:
$ myapp > some_log_file.txt
Run it for a few hundred iterations and see if you get any blank transmission, and if you do, if they are fewer. If this stops your blank tramissions, or lessens them a fair bit. Then your problem could either be python not being able to keep up( I'm kind of doubting that ), or Windows for whatever reason is preempting your code mid transmission. Or even something else not yet considered.
Dennis Lee Bieber
May 4, 2017, 5:41:06 PM5/4/17
to beagl...@googlegroups.com
WARNING -- LONG POST FOLLOWS (Lots of cut&paste output)
On Thu, 4 May 2017 09:32:03 -0700 (PDT), David Howlett
<david.h...@gmail.com> declaimed the
following:
I'll have to confess I'm still not sure which serial port is being used
here...
The debug serial located on the 6-pin header midway between the
USB-host port and the 5V power jack (I don't have a 3.3V USB/Serial for
that -- though I could maybe use jumper wires with the RaspPi adapter I
have [the pin-out is different]); or
The emulated serial port available when connecting to the USB-client
port (connecting a BBB via that port created two COM ports on my Win10
machine -- Gadget Serial/COM3 [triangle warning in device manager] and USB
Serial Device/COM4); or
Some other port/UART via some other serial/USB converter.
>This is the tool I was looking for and could not find, thank you. The
>default settings for the serial connection I am using are:
>
>root@beaglebone:~/reusable# stty --all
>speed 9600 baud; rows 0; columns 0; line = 0;
>intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
>eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt =
>^R;
>werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
>-parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
>-ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon
by the remote end to pause the local end.
>-ixoff
But the local end will NOT send X-on/X-off flow control to tell the
other end to pause.
-=-=-=-=-
[-]ixoff
enable sending of start/stop characters
[-]ixon
enable XON/XOFF flow control
(as I understood it, the - means the option is turned off)
Putty serial-mode via COM4 9600 8N1 Xon/Xoff (default) gave:
debian@beaglebone:~$ stty -a
speed 9600 baud; rows 24; columns 80; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt =
^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal -crtscts
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt =
^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon
ixoff
-iuclc -ixany -imaxbel iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0
ff0
isig icanon -iexten echo echoe echok -echonl -noflsh -xcase -tostop
ff0
-echoprt
echoctl echoke
debian@beaglebone:~$
So Xon/Xoff is already fully enabled on the BBB, and Putty is set to
use it (though device manager default is no flow control -- with the
receive FIFO buffer set to interrupt at 14-bytes [transmit at 16]). Have
you tried reducing the receive FIFO level? {Device manager, pick out the
proper COM#, <right-click> Properties/Port Settings [Advanced] -- reduce
the FIFO settings to half or less of the defaults}
>I can now use the stty command to set ixon on or off.
>
>root@beaglebone:~/reusable# stty -ixon
>
>Unfortunately the data loss is the same regardless of whether ixon is
>enabled or disabled. Changing ixoff does not affect the data loss either.
Remember that X-on/X-off protocol has to send the control character to the
other end -- which often means it has to be sent early enough to be
processed before that other end has overflowed the local end... (The only
time I've seen X-on/X-off used was on 1200baud terminals in the late 70s --
wherein the USER was typing <ctrl-s>/<ctrl-q> in order to pause the output
of long listings -- ie; the sending application is what responded to the
characters; not to control serial port buffers which were still RTS/CTS).
Suspect I'd start with
stty ixon ixoff -ixany
and ensure the other end is set for equivalent.
>I tried setting crtscts and cdtrdsr but this is not permitted.
>
>root@beaglebone:~/reusable# stty crtscts
>stty: standard input: unable to perform all requested operations
you are controlling the current console standard input -- and unless that
is the same serial port you are trying to send stuff too, it probably
doesn't have hardware flow control.
>root@beaglebone:~/reusable# stty cdtrdsr
>stty: invalid argument `cdtrdsr'
>
http://beaglebone.cameon.net/home/serial-ports-uart only RTS/CTS is
available, and even that is not on UART0 (which appears to be the debug
console jumper) UART1_5 do have RTS/CTS pins associated. If you are using
the USB jack, I have nothing useful (the presence of polled USB packets
means both sides are emulating UARTs and hardware flow control only applies
between the USB and Serial emulation at each end -- flow control between
the USB<>USB ends is USB packet polling, and how those packets encapsulate
the serial data is something I'm not familiar with)
integers takes 10 times as long as generating 1000 integers; only when the
integers have been generated and summed is anything produced as output to
the stdout. The only serial output is the final sum, which is not available
until the end...
BTW --
If you want to fill a buffer with different amounts of data, try
print([i for i in range(10000)])
which should dump the representation of a list of 10000 integers. Granted,
you'll see a 10x factor if comparing 1000 vs 10000, along with the longer
string being sent.
I can't duplicate your results as
debian@beaglebone:~$ python3.6 -c "while True: print(sum(i for i in
range(10000)))"
-bash: python3.6: command not found
debian@beaglebone:~$
Using just python at the shell prompt
debian@beaglebone:~$ python -c "while True: print(sum(i for i in
range(10000)))"
...
49995000
49995000
49995000
49995000
49995000
49995000
49995000
49995000
49995000
49995000
^CTraceback (most recent call last):
File "<string>", line 1, in <module>
File "<string>", line 1, in <genexpr>
KeyboardInterrupt
I let it run for quite a few seconds, and did not spot any point at
which the output had drop-outs. Neither do I suspect any flow control was
needed -- the time to generate the integers is long enough to allow the
output buffer to flush (it's only 10 bytes if <cr><lf> is included)
Didn't see any blatant drop-outs after changing to
debian@beaglebone:~$ python -c "while True: print([i for i in
range(10000)])"
...
9937, 9938, 9939, 9940, 9941, 9942, 9943, 9944, 9945, 9946, 9947, 9948,
9949, 9950, 9951, 9952, 9953, 9954, 9955, 9956, 9957, 9958, 9959, 9960,
9961, 9962, 9963, 9964, 9965, 9966, 9967, 9968, 9969, 9970, 9971, 9972,
9973, 9974, 9975, 9976, 9977, 9978, 9979, 9980, 9981, 9982, 9983, 9984,
9985, 9986, 9987, 9988, 9989, 9990, 9991, 9992, 9993, 9994, 9995, 9996,
9997, 9998, 9999]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,
59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,
78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96,
97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141,
142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156,
157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171,
172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186,
187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201,
202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216,
217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229,
...
I'd let that run long enough for four or five cycles of output.
At this point I would have to suspect the problem is on the program
being used on the receiving end. The drivers may /respond/ to Xon/Xoff --
but they may not /send/ them, relying upon the application to tell the
other end to pause/continue.
Tricky to do the next phase, since I had to log-in via Putty, then
close the Putty connection without logging out, to allow a Python script to
open the same port and write to the active BASH shell. (Using Python 2.7)
-=-=-=-=-
import serial
# the BBB USB-client port comes up as COM4 on my system
bbb = serial.Serial("COM4", 9600, xonxoff=True)
#flush input buffer (where did .read_all() come from?)
bbb.reset_input_buffer()
bbb.write('python -c "while True: print(sum(i for i in range(10000)))"\n')
try:
while True:
print bbb.readline().strip()
except: #nasty blank except
bbb.write(chr(3)) #attempt to kill running script
bbb.close()
-=-=-=-=-
{Using <ctrl-c> to stop local script}
C:\Users\Wulfraed>BBBser.py
[A [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [C [K"
49995000
49995000
49995000
49995000
49995000
<SNIP>
49995000
49995000
49995000
49995000
49995000
C:\Users\Wulfraed>
-=-=-=-=-=-
I have no idea where the initial string of garbage comes from -- I
suspect it is the BBB BASH shell echoing something, like the command line
perhaps, but it is coming in with other crud.
Changing the command to
bbb.write('python -c "while True: print([i for i in range(10000)])"\n') and
rerunning... (Hmmm, this time the shell echo was legible)
C:\Users\Wulfraed>BBBser.py
python -c "while True: print([i for i in range(10000)])"
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,
59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,
78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96,
97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141,
142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156,
157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171,
172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186,
187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201,
202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216,
217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231,
232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246,
247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261,
262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276,
277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291,
292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306,
307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,
322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336,
337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351,
352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366,
367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381,
382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396,
397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411,
412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426,
427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441,
442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456,
457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471,
472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486,
487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501,
502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516,
517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531,
532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546,
547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561,
562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576,
577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591,
592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606,
607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621,
622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636,
637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651,
652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666,
667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681,
682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692, 693, 694, 695, 696,
697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711,
712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726,
727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741,
742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756,
757, 758, 759, 760, 761, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771,
772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786,
787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801,
802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816,
817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831,
832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846,
847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861,
862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876,
877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891,
892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906,
907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921,
922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936,
937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951,
952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966,
967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, 981,
982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996,
997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009,
1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021,
1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033,
1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045,
1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1055, 1056, 1057,
1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069,
1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081,
1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093,
1094, 1095, 1096, 1097, 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105,
1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113, 1114, 1115, 1116, 1117,
1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129,
1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141,
1142, 1143, 1144, 1145, 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153,
1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161, 1162, 1163, 1164, 1165,
1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177,
1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189,
1190, 1191, 1192, 1193, 1194, 1195, 1196, 1197, 1198, 1199, 1200, 1201,
1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210, 1211, 1212, 1213,
1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1222, 1223, 1224, 1225,
1226, 1227, 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237,
1238, 1239, 1240, 1241, 1242, 1243, 1244, 1245, 1246, 1247, 1248, 1249,
1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260, 1261,
1262, 1263, 1264, 1265, 1266, 1267, 1268, 1269, 1270, 1271, 1272, 1273,
1274, 1275, 1276, 1277, 1278, 1279, 1280, 1281, 1282, 1283, 1284, 1285,
1286, 1287, 1288, 1289, 1290, 1291, 1292, 1293, 1294, 1295, 1296, 1297,
1298, 1299, 1300, 1301, 1302, 1303, 1304, 1305, 1306, 1307, 1308, 1309,
1310, 1311, 1312, 1313, 1314, 1315, 1316, 1317, 1318, 1319, 1320, 1321,
1322, 1323, 1324, 1325, 1326, 1327, 1328, 1329, 1330, 1331, 1332, 1333,
1334, 1335, 1336, 1337, 1338, 1339, 1340, 1341, 1342, 1343, 1344, 1345,
1346, 1347, 1348, 1349, 1350, 1351, 1352, 1353, 1354, 1355, 1356, 1357,
1358, 1359, 1360, 1361, 1362, 1363, 1364, 1365, 1366, 1367, 1368, 1369,
1370, 1371, 1372, 1373, 1374, 1375, 1376, 1377, 1378, 1379, 1380, 1381,
1382, 1383, 1384, 1385, 1386, 1387, 1388, 1389, 1390, 1391, 1392, 1393,
1394, 1395, 1396, 1397, 1398, 1399, 1400, 1401, 1402, 1403, 1404, 1405,
1406, 1407, 1408, 1409, 1410, 1411, 1412, 1413, 1414, 1415, 1416, 1417,
1418, 1419, 1420, 1421, 1422, 1423, 1424, 1425, 1426, 1427, 1428, 1429,
1430, 1431, 1432, 1433, 1434, 1435, 1436, 1437, 1438, 1439, 1440, 1441,
1442, 1443, 1444, 1445, 1446, 1447, 1448, 1449, 1450, 1451, 1452, 1453,
1454, 1455, 1456, 1457, 1458, 1459, 1460, 1461, 1462, 1463, 1464, 1465,
1466, 1467, 1468, 1469, 1470, 1471, 1472, 1473, 1474, 1475, 1476, 1477,
1478, 1479, 1480, 1481, 1482, 1483, 1484, 1485, 1486, 1487, 1488, 1489,
1490, 1491, 1492, 1493, 1494, 1495, 1496, 1497, 1498, 1499, 1500, 1501,
1502, 1503, 1504, 1505, 1506, 1507, 1508, 1509, 1510, 1511, 1512, 1513,
1514, 1515, 1516, 1517, 1518, 1519, 1520, 1521, 1522, 1523, 1524, 1525,
1526, 1527, 1528, 1529, 1530, 1531, 1532, 1533, 1534, 1535, 1536, 1537,
1538, 1539, 1540, 1541, 1542, 1543, 1544, 1545, 1546, 1547, 1548, 1549,
1550, 1551, 1552, 1553, 1554, 1555, 1556, 1557, 1558, 1559, 1560, 1561,
1562, 1563, 1564, 1565, 1566, 1567, 1568, 1569, 1570, 1571, 1572, 1573,
1574, 1575, 1576, 1577, 1578, 1579, 1580, 1581, 1582, 1583, 1584, 1585,
1586, 1587, 1588, 1589, 1590, 1591, 1592, 1593, 1594, 1595, 1596, 1597,
1598, 1599, 1600, 1601, 1602, 1603, 1604, 1605, 1606, 1607, 1608, 1609,
1610, 1611, 1612, 1613, 1614, 1615, 1616, 1617, 1618, 1619, 1620, 1621,
1622, 1623, 1624, 1625, 1626, 1627, 1628, 1629, 1630, 1631, 1632, 1633,
1634, 1635, 1636, 1637, 1638, 1639, 1640, 1641, 1642, 1643, 1644, 1645,
1646, 1647, 1648, 1649, 1650, 1651, 1652, 1653, 1654, 1655, 1656, 1657,
1658, 1659, 1660, 1661, 1662, 1663, 1664, 1665, 1666, 1667, 1668, 1669,
1670, 1671, 1672, 1673, 1674, 1675, 1676, 1677, 1678, 1679, 1680, 1681,
1682, 1683, 1684, 1685, 1686, 1687, 1688, 1689, 1690, 1691, 1692, 1693,
1694, 1695, 1696, 1697, 1698, 1699, 1700, 1701, 1702, 1703, 1704, 1705,
1706, 1707, 1708, 1709, 1710, 1711, 1712, 1713, 1714, 1715, 1716, 1717,
1718, 1719, 1720, 1721, 1722, 1723, 1724, 1725, 1726, 1727, 1728, 1729,
1730, 1731, 1732, 1733, 1734, 1735, 1736, 1737, 1738, 1739, 1740, 1741,
1742, 1743, 1744, 1745, 1746, 1747, 1748, 1749, 1750, 1751, 1752, 1753,
1754, 1755, 1756, 1757, 1758, 1759, 1760, 1761, 1762, 1763, 1764, 1765,
1766, 1767, 1768, 1769, 1770, 1771, 1772, 1773, 1774, 1775, 1776, 1777,
1778, 1779, 1780, 1781, 1782, 1783, 1784, 1785, 1786, 1787, 1788, 1789,
1790, 1791, 1792, 1793, 1794, 1795, 1796, 1797, 1798, 1799, 1800, 1801,
1802, 1803, 1804, 1805, 1806, 1807, 1808, 1809, 1810, 1811, 1812, 1813,
1814, 1815, 1816, 1817, 1818, 1819, 1820, 1821, 1822, 1823, 1824, 1825,
1826, 1827, 1828, 1829, 1830, 1831, 1832, 1833, 1834, 1835, 1836, 1837,
1838, 1839, 1840, 1841, 1842, 1843, 1844, 1845, 1846, 1847, 1848, 1849,
1850, 1851, 1852, 1853, 1854, 1855, 1856, 1857, 1858, 1859, 1860, 1861,
1862, 1863, 1864, 1865, 1866, 1867, 1868, 1869, 1870, 1871, 1872, 1873,
1874, 1875, 1876, 1877, 1878, 1879, 1880, 1881, 1882, 1883, 1884, 1885,
1886, 1887, 1888, 1889, 1890, 1891, 1892, 1893, 1894, 1895, 1896, 1897,
1898, 1899, 1900, 1901, 1902, 1903, 1904, 1905, 1906, 1907, 1908, 1909,
1910, 1911, 1912, 1913, 1914, 1915, 1916, 1917, 1918, 1919, 1920, 1921,
1922, 1923, 1924, 1925, 1926, 1927, 1928, 1929, 1930, 1931, 1932, 1933,
1934, 1935, 1936, 1937, 1938, 1939, 1940, 1941, 1942, 1943, 1944, 1945,
1946, 1947, 1948, 1949, 1950, 1951, 1952, 1953, 1954, 1955, 1956, 1957,
1958, 1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, 1969,
1970, 1971, 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981,
1982, 1983, 1984, 1985, 1986, 1987, 1988, 1989, 1990, 1991, 1992, 1993,
1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025, 2026, 2027, 2028, 2029,
2030, 2031, 2032, 2033, 2034, 2035, 2036, 2037, 2038, 2039, 2040, 2041,
2042, 2043, 2044, 2045, 2046, 2047, 2048, 2049, 2050, 2051, 2052, 2053,
2054, 2055, 2056, 2057, 2058, 2059, 2060, 2061, 2062, 2063, 2064, 2065,
2066, 2067, 2068, 2069, 2070, 2071, 2072, 2073, 2074, 2075, 2076, 2077,
2078, 2079, 2080, 2081, 2082, 2083, 2084, 2085, 2086, 2087, 2088, 2089,
2090, 2091, 2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100, 2101,
2102, 2103, 2104, 2105, 2106, 2107, 2108, 2109, 2110, 2111, 2112, 2113,
2114, 2115, 2116, 2117, 2118, 2119, 2120, 2121, 2122, 2123, 2124, 2125,
2126, 2127, 2128, 2129, 2130, 2131, 2132, 2133, 2134, 2135, 2136, 2137,
2138, 2139, 2140, 2141, 2142, 2143, 2144, 2145, 2146, 2147, 2148, 2149,
2150, 2151, 2152, 2153, 2154, 2155, 2156, 2157, 2158, 2159, 2160, 2161,
2162, 2163, 2164, 2165, 2166, 2167, 2168, 2169, 2170, 2171, 2172, 2173,
2174, 2175, 2176, 2177, 2178, 2179, 2180, 2181, 2182, 2183, 2184, 2185,
2186, 2187, 2188, 2189, 2190, 2191, 2192, 2193, 2194, 2195, 2196, 2197,
2198, 2199, 2200, 2201, 2202, 2203, 2204, 2205, 2206, 2207, 2208, 2209,
2210, 2211, 2212, 2213, 2214, 2215, 2216, 2217, 2218, 2219, 2220, 2221,
2222, 2223, 2224, 2225, 2226, 2227, 2228, 2229, 2230, 2231, 2232, 2233,
2234, 2235, 2236, 2237, 2238, 2239, 2240, 2241, 2242, 2243, 2244, 2245,
2246, 2247, 2248, 2249, 2250, 2251, 2252, 2253, 2254, 2255, 2256, 2257,
2258, 2259, 2260, 2261, 2262, 2263, 2264, 2265, 2266, 2267, 2268, 2269,
2270, 2271, 2272, 2273, 2274, 2275, 2276, 2277, 2278, 2279, 2280, 2281,
2282, 2283, 2284, 2285, 2286, 2287, 2288, 2289, 2290, 2291, 2292, 2293,
2294, 2295, 2296, 2297, 2298, 2299, 2300, 2301, 2302, 2303, 2304, 2305,
2306, 2307, 2308, 2309, 2310, 2311, 2312, 2313, 2314, 2315, 2316, 2317,
2318, 2319, 2320, 2321, 2322, 2323, 2324, 2325, 2326, 2327, 2328, 2329,
2330, 2331, 2332, 2333, 2334, 2335, 2336, 2337, 2338, 2339, 2340, 2341,
2342, 2343, 2344, 2345, 2346, 2347, 2348, 2349, 2350, 2351, 2352, 2353,
2354, 2355, 2356, 2357, 2358, 2359, 2360, 2361, 2362, 2363, 2364, 2365,
2366, 2367, 2368, 2369, 2370, 2371, 2372, 2373, 2374, 2375, 2376, 2377,
2378, 2379, 2380, 2381, 2382, 2383, 2384, 2385, 2386, 2387, 2388, 2389,
2390, 2391, 2392, 2393, 2394, 2395, 2396, 2397, 2398, 2399, 2400, 2401,
2402, 2403, 2404, 2405, 2406, 2407, 2408, 2409, 2410, 2411, 2412, 2413,
2414, 2415, 2416, 2417, 2418, 2419, 2420, 2421, 2422, 2423, 2424, 2425,
2426, 2427, 2428, 2429, 2430, 2431, 2432, 2433, 2434, 2435, 2436, 2437,
2438, 2439, 2440, 2441, 2442, 2443, 2444, 2445, 2446, 2447, 2448, 2449,
2450, 2451, 2452, 2453, 2454, 2455, 2456, 2457, 2458, 2459, 2460, 2461,
2462, 2463, 2464, 2465, 2466, 2467, 2468, 2469, 2470, 2471, 2472, 2473,
2474, 2475, 2476, 2477, 2478, 2479, 2480, 2481, 2482, 2483, 2484, 2485,
2486, 2487, 2488, 2489, 2490, 2491, 2492, 2493, 2494, 2495, 2496, 2497,
2498, 2499, 2500, 2501, 2502, 2503, 2504, 2505, 2506, 2507, 2508, 2509,
2510, 2511, 2512, 2513, 2514, 2515, 2516, 2517, 2518, 2519, 2520, 2521,
2522, 2523, 2524, 2525, 2526, 2527, 2528, 2529, 2530, 2531, 2532, 2533,
2534, 2535, 2536, 2537, 2538, 2539, 2540, 2541, 2542, 2543, 2544, 2545,
2546, 2547, 2548, 2549, 2550, 2551, 2552, 2553, 2554, 2555, 2556, 2557,
2558, 2559, 2560, 2561, 2562, 2563, 2564, 2565, 2566, 2567, 2568, 2569,
2570, 2571, 2572, 2573, 29, 2660, 2661, 2662, 2663, 2664, 2665, 2666, 2667,
OKAY -- THIS PRODUCED a drop-out 2574..2659... Remember that the
.readline() is expecting everything up to the closing ] as a single output
string.
2668, 2669, 2670, 2671, 2672, 2673, 2674, 2675, 2676, 2677, 2678, 2679,
2680, 2681, 2682, 2683, 2684, 2685, 2686, 2687, 2688, 2689, 2690, 2691,
2692, 2693, 2694, 2695, 2696, 2697, 2698, 2699, 2700, 2701, 2702, 2703,
2704, 2705, 2706, 2707, 2708, 2709, 2710, 2711, 2712, 2713, 2714, 2715,
2716, 2717, 2718, 2719, 2720, 2721, 2722, 2723, 2724, 2725, 2726, 2727,
2728, 2729, 2730, 2731, 2732, 2733, 2734, 2735, 2736, 2737, 2738, 2739,
2740, 2741, 2742, 2743, 2744, 2745, 2746, 2747, 2748, 2749, 2750, 2751,
2752, 2753, 2754, 2755, 2756, 2757, 2758, 2759, 2760, 2761, 2762, 2763,
2764, 2765, 2766, 2767, 2768, 2769, 2770, 2771, 2772, 2773, 2774, 2775,
2776, 2777, 2778, 2779, 2780, 2781, 2782, 2783, 2784, 2785, 2786, 2787,
2788, 2789, 2790, 2791, 2792, 2793, 2794, 2795, 2796, 2797, 2798, 2799,
2800, 2801, 2802, 2803, 2804, 2805, 2806, 2807, 2808, 2809, 2810, 2811,
2812, 2813, 2814, 2815, 2816, 2817, 2818, 2819, 2820, 2821, 2822, 2823,
2824, 2825, 2826, 2827, 2828, 2829, 2830, 2831, 2832, 2833, 2834, 2835,
2836, 2837, 2838, 2839, 2840, 2841, 2842, 2843, 2844, 2845, 2846, 2847,
2848, 2849, 2850, 2851, 2852, 2853, 2854, 2855, 2856, 2857, 2858, 2859,
2860, 2861, 2862, 2863, 2864, 2865, 2866, 2867, 2868, 2869, 2870, 2871,
2872, 2873, 2874, 2875, 2876, 2877, 2878, 2879, 2880, 2881, 2882, 2883,
2884, 2885, 2886, 2887, 2888, 2889, 2890, 2891, 2892, 2893, 2894, 2895,
2896, 2897, 2898, 2899, 2900, 2901, 2902, 2903, 2904, 2905, 2906, 2907,
2908, 2909, 2910, 2911, 2912, 2913, 2914, 2915, 2916, 2917, 2918, 2919,
2920, 2921, 2922, 2923, 2924, 2925, 2926, 2927, 2928, 2929, 2930, 2931,
2932, 2933, 2934, 2935, 2936, 2937, 2938, 2939, 2940, 2941, 2942, 2943,
2944, 2945, 2946, 2947, 2948, 2949, 2950, 2951, 2952, 2953, 2954, 2955,
2956, 2957, 2958, 2959, 2960, 2961, 2962, 2963, 2964, 2965, 2966, 2967,
2968, 2969, 2970, 2971, 2972, 2973, 2974, 2975, 2976, 2977, 2978, 2979,
2980, 2981, 2982, 2983, 2984, 2985, 2986, 2987, 2988, 2989, 2990, 2991,
2992, 2993, 2994, 2995, 2996, 2997, 2998, 2999, 3000, 3001, 3002, 3003,
3004, 3005, 3006, 3007, 3008, 3009, 3010, 3011, 3012, 3013, 3014, 3015,
3016, 3017, 3018, 3019, 3020, 3021, 3022, 3023, 3024, 3025, 3026, 3027,
3028, 3029, 3030, 3031, 3032, 3033, 3034, 3035, 3036, 3037, 3038, 3039,
3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3050, 3051,
3052, 3053, 3054, 3055, 3056, 3057, 3058, 3059, 3060, 3061, 3062, 3063,
3064, 3065, 3066, 3067, 3068, 3069, 3070, 3071, 3072, 3073, 3074, 3075,
3076, 3077, 3078, 3079, 3080, 3081, 3082, 3083, 3084, 3085, 3086, 3087,
3088, 3089, 3090, 3091, 3092, 3093, 3094, 3095, 3096, 3097, 3098, 3099,
3100, 3101, 3102, 3103, 3104, 3105, 3106, 3107, 3108, 3109, 3110, 3111,
3112, 3113, 3114, 3115, 3116, 3117, 3118, 3119, 3120, 3121, 3122, 3123,
3124, 3125, 3126, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135,
3136, 3137, 3138, 3139, 3140, 3141, 3142, 3143, 3144, 3145, 3146, 3147,
3148, 3149, 3150, 3151, 3152, 3153, 3154, 3155, 3156, 3157, 3158, 3159,
3160, 3161, 3162, 3163, 3164, 3165, 3166, 3167, 3168, 3169, 3170, 3171,
3172, 3173, 3174, 3175, 3176, 3177, 3178, 3179, 3180, 3181, 3182, 3183,
3184, 3185, 3186, 3187, 3188, 3189, 3190, 3191, 3192, 3193, 3194, 3195,
3196, 3197, 3198, 3199, 3200, 3201, 3202, 3203, 3204, 3205, 3206, 3207,
3208, 3209, 3210, 3211, 3212, 3213, 3214, 3215, 3216, 3217, 3218, 3219,
3220, 3221, 3222, 3223, 3224, 3225, 3226, 3227, 3228, 3229, 3230, 3231,
3232, 3233, 3234, 3235, 3236, 3237, 3238, 3239, 3240, 3241, 3242, 3243,
3244, 3245, 3246, 3247, 3248, 3249, 3250, 3251, 3252, 3253, 3254, 3255,
3256, 3257, 3258, 3259, 3260, 3261, 3262, 3263, 3264, 3265, 3266, 3267,
3268, 3269, 3270, 3271, 3272, 3273, 3274, 3275, 3276, 3277, 3278, 3279,
3280, 3281, 3282, 3283, 3284, 3285, 3286, 3287, 3288, 3289, 3290, 3291,
3292, 3293, 3294, 3295, 3296, 3297, 3298, 3299, 3300, 3301, 3302, 3303,
3304, 3305, 3306, 3307, 3308, 3309, 3310, 3311, 3312, 3313, 3314, 3315,
3316, 3317, 3318, 3319, 3320, 3321, 3322, 3323, 3324, 3325, 3326, 3327,
3328, 3329, 3330, 3331, 3332, 3333, 3334, 3335, 3336, 3337, 3338, 3339,
3340, 3341, 3342, 3343, 3344, 3345, 3346, 3347, 3348, 3349, 3350, 3351,
3352, 3353, 3354, 3355, 3356, 3357, 3358, 3359, 3360, 3361, 3362, 3363,
3364, 3365, 3366, 3367, 3368, 3369, 3370, 3371, 3372, 3373, 3374, 3375,
3376, 3377, 3378, 3379, 3380, 3381, 3382, 3383, 3384, 3385, 3386, 3387,
3388, 3389, 3390, 3391, 3392, 3393, 3394, 3395, 3396, 3397, 3398, 3399,
3400, 3401, 3402, 3403, 3404, 3405, 3406, 3407, 3408, 3409, 3410, 3411,
3412, 3413, 3414, 3415, 3416, 3417, 3418, 3419, 3420, 3421, 3422, 3423,
3424, 3425, 3426, 3427, 3428, 3429, 3430, 3431, 3432, 3433, 3434, 3435,
3436, 3437, 3438, 3439, 3440, 3441, 3442, 3443, 3444, 3445, 3446, 3447,
3448, 3449, 3450, 3451, 3452, 3453, 3454, 3455, 3456, 3457, 3458, 3459,
3460, 3461, 3462, 3463, 3464, 3465, 3466, 3467, 3468, 3469, 3470, 3471,
3472, 3473, 3474, 3475, 3476, 3477, 3478, 3479, 3480, 3481, 3482, 3483,
3484, 3485, 3486, 3487, 3488, 3489, 3490, 3491, 3492, 3493, 3494, 3495,
3496, 3497, 3498, 3499, 3500, 3501, 3502, 3503, 3504, 3505, 3506, 3507,
3508, 3509, 3510, 3511, 3512, 3513, 3514, 3515, 3516, 3517, 3518, 3519,
3520, 3521, 3522, 3523, 3524, 3525, 3526, 3527, 3528, 3529, 3530, 3531,
3532, 3533, 3534, 3535, 3536, 3537, 3538, 3539, 3540, 3541, 3542, 3543,
3544, 3545, 3546, 3547, 3548, 3549, 3550, 3551, 3552, 3553, 3554, 3555,
3556, 3557, 3558, 3559, 3560, 3561, 3562, 3563, 3564, 3565, 3566, 3567,
3568, 3569, 3570, 3571, 3572, 3573, 3574, 3575, 3576, 3577, 3578, 3579,
3580, 3581, 3582, 3583, 3584, 3585, 3586, 3587, 3588, 3589, 3590, 3591,
3592, 3593, 3594, 3595, 3596, 3597, 33, 3684, 3685, 3686, 3687, 3688, 3689,
Second dropout 3598..3683
3690, 3691, 3692, 3693, 3694, 3695, 3696, 3697, 3698, 3699, 3700, 3701,
3702, 3703, 3704, 3705, 3706, 3707, 3708, 3709, 3710, 3711, 3712, 3713,
3714, 3715, 3716, 3717, 3718, 3719, 3720, 3721, 3722, 3723, 3724, 3725,
3726, 3727, 3728, 3729, 3730, 3731, 3732, 3733, 3734, 3735, 3736, 3737,
3738, 3739, 3740, 3741, 3742, 3743, 3744, 3745, 3746, 3747, 3748, 3749,
3750, 3751, 3752, 3753, 3754, 3755, 3756, 3757, 3758, 3759, 3760, 3761,
3762, 3763, 3764, 3765, 3766, 3767, 3768, 3769, 3770, 3771, 3772, 3773,
3774, 3775, 3776, 3777, 3778, 3779, 3780, 3781, 3782, 3783, 3784, 3785,
3786, 3787, 3788, 3789, 3790, 3791, 3792, 3793, 3794, 3795, 3796, 3797,
3798, 3799, 3800, 3801, 3802, 3803, 3804, 3805, 3806, 3807, 3808, 3809,
3810, 3811, 3812, 3813, 3814, 3815, 3816, 3817, 3818, 3819, 3820, 3821,
3822, 3823, 3824, 3825, 3826, 3827, 3828, 3829, 3830, 3831, 3832, 3833,
3834, 3835, 3836, 3837, 3838, 3839, 3840, 3841, 3842, 3843, 3844, 3845,
3846, 3847, 3848, 3849, 3850, 3851, 3852, 3853, 3854, 3855, 3856, 3857,
3858, 3859, 3860, 3861, 3862, 3863, 3864, 3865, 3866, 3867, 3868, 3869,
3870, 3871, 3872, 3873, 3874, 3875, 3876, 3877, 3878, 3879, 3880, 3881,
3882, 3883, 3884, 3885, 3886, 3887, 3888, 3889, 3890, 3891, 3892, 3893,
3894, 3895, 3896, 3897, 3898, 3899, 3900, 3901, 3902, 3903, 3904, 3905,
3906, 3907, 3908, 3909, 3910, 3911, 3912, 3913, 3914, 3915, 3916, 3917,
3918, 3919, 3920, 3921, 3922, 3923, 3924, 3925, 3926, 3927, 3928, 3929,
3930, 3931, 3932, 3933, 3934, 3935, 3936, 3937, 3938, 3939, 3940, 3941,
3942, 3943, 3944, 3945, 3946, 3947, 3948, 3949, 3950, 3951, 3952, 3953,
3954, 3955, 3956, 3957, 3958, 3959, 3960, 3961, 3962, 3963, 3964, 3965,
3966, 3967, 3968, 3969, 3970, 3971, 3972, 3973, 3974, 3975, 3976, 3977,
3978, 3979, 3980, 3981, 3982, 3983, 3984, 3985, 3986, 3987, 3988, 3989,
3990, 3991, 3992, 3993, 3994, 3995, 3996, 3997, 3998, 3999, 4000, 4001,
4002, 4003, 4004, 4005, 4006, 4007, 4008, 4009, 4010, 4011, 4012, 4013,
4014, 4015, 4016, 4017, 4018, 4019, 4020, 4021, 4022, 4023, 4024, 4025,
4026, 4027, 4028, 4029, 4030, 4031, 4032, 4033, 4034, 4035, 4036, 4037,
4038, 4039, 4040, 4041, 4042, 4043, 4044, 4045, 4046, 4047, 4048, 4049,
4050, 4051, 4052, 4053, 4054, 4055, 4056, 4057, 4058, 4059, 4060, 4061,
4062, 4063, 4064, 4065, 4066, 4067, 4068, 4069, 4070, 4071, 4072, 4073,
4074, 4075, 4076, 4077, 4078, 4079, 4080, 4081, 4082, 4083, 4084, 4085,
4086, 4087, 4088, 4089, 4090, 4091, 4092, 4093, 4094, 4095, 4096, 4097,
4098, 4099, 4100, 4101, 4102, 4103, 4104, 4105, 4106, 4107, 4108, 4109,
4110, 4111, 4112, 4113, 4114, 4115, 4116, 4117, 4118, 4119, 4120, 4121,
4122, 4123, 4124, 4125, 4126, 4127, 4128, 4129, 4130, 4131, 4132, 4133,
4134, 4135, 4136, 4137, 4138, 4139, 4140, 4141, 4142, 4143, 4144, 4145,
4146, 4147, 4148, 4149, 4150, 4151, 4152, 4153, 4154, 4155, 4156, 4157,
4158, 4159, 4160, 4161, 4162, 4163, 4164, 4165, 4166, 4167, 4168, 4169,
4170, 4171, 4172, 4173, 4174, 4175, 4176, 4177, 4178, 4179, 4180, 4181,
4182, 4183, 4184, 4185, 4186, 4187, 4188, 4189, 4190, 4191, 4192, 4193,
4194, 4195, 4196, 4197, 4198, 4199, 4200, 4201, 4202, 4203, 4204, 4205,
4206, 4207, 4208, 4209, 4210, 4211, 4212, 4213, 4214, 4215, 4216, 4217,
4218, 4219, 4220, 4221, 4222, 4223, 4224, 4225, 4226, 4227, 4228, 4229,
4230, 4231, 4232, 4233, 4234, 4235, 4236, 4237, 4238, 4239, 4240, 4241,
4242, 4243, 4244, 4245, 4246, 4247, 4248, 4249, 4250, 4251, 4252, 4253,
4254, 4255, 4256, 4257, 4258, 4259, 4260, 4261, 4262, 4263, 4264, 4265,
4266, 4267, 4268, 4269, 4270, 4271, 4272, 4273, 4274, 4275, 4276, 4277,
4278, 4279, 4280, 4281, 4282, 4283, 4284, 4285, 4286, 4287, 4288, 4289,
4290, 4291, 4292, 4293, 4294, 4295, 4296, 4297, 4298, 4299, 4300, 4301,
4302, 4303, 4304, 4305, 4306, 4307, 4308, 4309, 4310, 4311, 4312, 4313,
4314, 4315, 4316, 4317, 4318, 4319, 4320, 4321, 4322, 4323, 4324, 4325,
4326, 4327, 4328, 4329, 4330, 4331, 4332, 4333, 4334, 4335, 4336, 4337,
4338, 4339, 4340, 4341, 4342, 4343, 4344, 4345, 4346, 4347, 4348, 4349,
4350, 4351, 4352, 4353, 4354, 4355, 4356, 4357, 4358, 4359, 4360, 4361,
4362, 4363, 4364, 4365, 4366, 4367, 4368, 4369, 4370, 4371, 4372, 4373,
4374, 4375, 4376, 4377, 4378, 4379, 4380, 4381, 4382, 4383, 4384, 4385,
4386, 4387, 4388, 4389, 4390, 4391, 4392, 4393, 4394, 4395, 4396, 4397,
4398, 4399, 4400, 4401, 4402, 4403, 4404, 4405, 4406, 4407, 4408, 4409,
4410, 4411, 4412, 4413, 4414, 4415, 4416, 4417, 4418, 4419, 4420, 4421,
4422, 4423, 4424, 4425, 4426, 4427, 4428, 4429, 4430, 4431, 4432, 4433,
4434, 4435, 4436, 4437, 4438, 4439, 4440, 4441, 4442, 4443, 4444, 4445,
4446, 4447, 4448, 4449, 4450, 4451, 4452, 4453, 4454, 4455, 4456, 4457,
4458, 4459, 4460, 4461, 4462, 4463, 4464, 4465, 4466, 4467, 4468, 4469,
4470, 4471, 4472, 4473, 4474, 4475, 4476, 4477, 4478, 4479, 4480, 4481,
4482, 4483, 4484, 4485, 4486, 4487, 4488, 4489, 4490, 4491, 4492, 4493,
4494, 4495, 4496, 4497, 4498, 4499, 4500, 4501, 4502, 4503, 4504, 4505,
4506, 4507, 4508, 4509, 4510, 4511, 4512, 4513, 4514, 4515, 4516, 4517,
4518, 4519, 4520, 4521, 4522, 4523, 4524, 4525, 4526, 4527, 4528, 4529,
4530, 4531, 4532, 4533, 4534, 4535, 4536, 4537, 4538, 4539, 4540, 4541,
4542, 4543, 4544, 4545, 4546, 4547, 4548, 4549, 4550, 4551, 4552, 4553,
4554, 4555, 4556, 4557, 4558, 4559, 4560, 4561, 4562, 4563, 4564, 4565,
4566, 4567, 4568, 4569, 4570, 4571, 4572, 4573, 4574, 4575, 4576, 4577,
4578, 4579, 4580, 4581, 4582, 4583, 4584, 4585, 4586, 4587, 4588, 4589,
4590, 4591, 4592, 4593, 4594, 4595, 4596, 4597, 4598, 4599, 4600, 4601,
4602, 4603, 4604, 4605, 4606, 4607, 4608, 4609, 4610, 4611, 4612, 4613,
4614, 4615, 4616, 4617, 4618, 4619, 4620, 4621, 4622, 4623, 4624, 4625,
4626, 4627, 4628, 4629, 4630, 4631, 4632, 4633, 4634, 4635, 4636, 4637,
4638, 4639, 4640, 4641, 4642, 4643, 4644, 4645, 4646, 4647, 4648, 4649,
4650, 4651, 4652, 4653, 4654, 4655, 4656, 4657, 4658, 4659, 4660, 4661,
4662, 4663, 4664, 4665, 4666, 4667, 4668, 4669, 4670, 4671, 4672, 4673,
4674, 4675, 4676, 4677, 4678, 4679, 4680, 4681, 4682, 4683, 4684, 4685,
4686, 4687, 4688, 4689, 4690, 4691, 4692, 4693, 4694, 4695, 4696, 4697,
4698, 4699, 4700, 4701, 4702, 4703, 4704, 4705, 4706, 4707, 4708, 4709,
4710, 4711, 4712, 4713, 4714, 4715, 4716, 4717, 4718, 4719, 4720, 4721,
4722, 4723, 4724, 4725, 4726, 4727, 4728, 4729, 4730, 4731, 4732, 4733,
4734, 4735, 4736, 4737, 4738, 4739, 4740, 4741, 4742, 4743, 4744, 4745,
4746, 4747, 4748, 4749, 4750, 4751, 4752, 4753, 4754, 4755, 4756, 4757,
4758, 4759, 4760, 4761, 4762, 4763, 4764, 4765, 4766, 4767, 4768, 4769,
4770, 4771, 4772, 4773, 4774, 4775, 4776, 4777, 4778, 4779, 4780, 4781,
4782, 4783, 4784, 4785, 4786, 4787, 4788, 4789, 4790, 4791, 4792, 4793,
4794, 4795, 4796, 4797, 4798, 4799, 4800, 4801, 4802, 4803, 4804, 4805,
4806, 4807, 4808, 4809, 4810, 4811, 4812, 4813, 4814, 4815, 4816, 4817,
4818, 4819, 4820, 4821, 4822, 4823, 4824, 4825, 4826, 4827, 4828, 4829,
4830, 4831, 4832, 4833, 4834, 4835, 4836, 4837, 4838, 4839, 4840, 4841,
4842, 4843, 4844, 4845, 4846, 4847, 4848, 4849, 4850, 4851, 4852, 4853,
4854, 4855, 4856, 4857, 4858, 4859, 4860, 4861, 4862, 4863, 4864, 4865,
4866, 4867, 4868, 4869, 4870, 4871, 4872, 4873, 4874, 4875, 4876, 4877,
4878, 4879, 4880, 4881, 4882, 4883, 4884, 4885, 4886, 4887, 4888, 4889,
4890, 4891, 4892, 4893, 4894, 4895, 4896, 4897, 4898, 4899, 4900, 4901,
4902, 4903, 4904, 4905, 4906, 4907, 4908, 4909, 4910, 4911, 4912, 4913,
4914, 4915, 4916, 4917, 4918, 4919, 4920, 4921, 4922, 4923, 4924, 4925,
4926, 4927, 4928, 4929, 4930, 4931, 4932, 4933, 4934, 4935, 4936, 4937,
4938, 4939, 4940, 4941, 4942, 4943, 4944, 4945, 4946, 4947, 4948, 4949,
4950, 4951, 4952, 4953, 4954, 4955, 4956, 4957, 4958, 4959, 4960, 4961,
4962, 4963, 4964, 4965, 4966, 4967, 4968, 4969, 4970, 4971, 4972, 4973,
4974, 4975, 4976, 4977, 4978, 4979, 4980, 4981, 4982, 4983, 4984, 4985,
4986, 4987, 4988, 4989, 4990, 4991, 4992, 4993, 4994, 4995, 4996, 4997,
4998, 4999, 5000, 5001, 5002, 5003, 5004, 5005, 5006, 5007, 5008, 5009,
5010, 5011, 5012, 5013, 5014, 5015, 5016, 5017, 5018, 5019, 5020, 5021,
5022, 5023, 5024, 5025, 5026, 5027, 5028, 5029, 5030, 5031, 5032, 5033,
5034, 5035, 5036, 5037, 5038, 5039, 5040, 5041, 5042, 5043, 5044, 5045,
5046, 5047, 5048, 5049, 5050, 5051, 5052, 5053, 5054, 5055, 5056, 5057,
5058, 5059, 5060, 5061, 5062, 5063, 5064, 5065, 5066, 5067, 5068, 5069,
5070, 5071, 5072, 5073, 5074, 5075, 5076, 5077, 5078, 5079, 5080, 5081,
5082, 5083, 5084, 5085, 5086, 5087, 5088, 5089, 5090, 5091, 5092, 5093,
5094, 5095, 5096, 5097, 5098, 5099, 5100, 5101, 5102, 5103, 5104, 5105,
5106, 5107, 5108, 5109, 5110, 5111, 5112, 5113, 5114, 5115, 5116, 5117,
5118, 5119, 5120, 5121, 5122, 5123, 5124, 5125, 5126, 5127, 5128, 5129,
5130, 5131, 5132, 5133, 5134, 5135, 5136, 5137, 5138, 5139, 5140, 5141,
5142, 5143, 5144, 5145, 5146, 5147, 5148, 5149, 5150, 5151, 5152, 5153,
5154, 5155, 5156, 5157, 5158, 5159, 5160, 5161, 5162, 5163, 5164, 5165,
5166, 5167, 5168, 5169, 5170, 5171, 5172, 5173, 5174, 5175, 5176, 5177,
5178, 5179, 5180, 5181, 5182, 5183, 5184, 5185, 5186, 5187, 5188, 5189,
5190, 5191, 5192, 5193, 5194, 5195, 5196, 5197, 5198, 5199, 5200, 5201,
5202, 5203, 5204, 5205, 5206, 5207, 5208, 5209, 5210, 5211, 5212, 5213,
5214, 5215, 5216, 5217, 5218, 5219, 5220, 5221, 5222, 5223, 5224, 5225,
5226, 5227, 5228, 5229, 5230, 5231, 5232, 5233, 5234, 5235, 5236, 5237,
5238, 5239, 5240, 5241, 5242, 5243, 5244, 5245, 5246, 5247, 5248, 5249,
5250, 5251, 5252, 5253, 5254, 5255, 5256, 5257, 5258, 5259, 5260, 5261,
5262, 5263, 5264, 5265, 5266, 5267, 5268, 5269, 5270, 5271, 5272, 5273,
5274, 5275, 5276, 5277, 5278, 5279, 5280, 5281, 5282, 5283, 5284, 5285,
5286, 5287, 5288, 5289, 5290, 5291, 5292, 5293, 5294, 5295, 5296, 5297,
5298, 5299, 5300, 5301, 5302, 5303, 5304, 5305, 5306, 5307, 5308, 5309,
5310, 5311, 5312, 5313, 5314, 5315, 5316, 5317, 5318, 5319, 5320, 5321,
5322, 5323, 5324, 5325, 5326, 5327, 5328, 5329, 5330, 5331, 5332, 5333,
5334, 5335, 5336, 5337, 5338, 5339, 5340, 5341, 5342, 5343, 5344, 5345,
5346, 5347, 5348, 5349, 5350, 5351, 5352, 5353, 5354, 5355, 5356, 5357,
5358, 5359, 5360, 5361, 5362, 5363, 5364, 5365, 5366, 5367, 5368, 5369,
5370, 5371, 5372, 5373, 5374, 5375, 5376, 5377, 5378, 5379, 5380, 5381,
5382, 5383, 5384, 5385, 5386, 5387, 5388, 5389, 5390, 5391, 5392, 5393,
5394, 5395, 5396, 5397, 5398, 5399, 5400, 5401, 5402, 5403, 5404, 5405,
5406, 5407, 5408, 5409, 5410, 5411, 5412, 5413, 5414, 5415, 5416, 5417,
5418, 5419, 5420, 5421, 5422, 5423, 5424, 5425, 5426, 5427, 5428, 5429,
5430, 5431, 5432, 5433, 5434, 5435, 5436, 5437, 5438, 5439, 5440, 5441,
5442, 5443, 5444, 5445, 5446, 5447, 5448, 5449, 5450, 5451, 5452, 5453,
5454, 5455, 5456, 5457, 5458, 5459, 5460, 5461, 5462, 5463, 5464, 5465,
5466, 5467, 5468, 5469, 5470, 5471, 5472, 5473, 5474, 5475, 5476, 5477,
5478, 5479, 5480, 5481, 5482, 5483, 5484, 5485, 5486, 5487, 5488, 5489,
5490, 5491, 5492, 5493, 5494, 5495, 5496, 5497, 5498, 5499, 5500, 5501,
5502, 5503, 5504, 5505, 5506, 5507, 5508, 5509, 5510, 5511, 5512, 5513,
5514, 5515, 5516, 5517, 5518, 5519, 5520, 5521, 5522, 5523, 5524, 5525,
5526, 5527, 5528, 5529, 5530, 5531, 5532, 5533, 5534, 5535, 5536, 5537,
5538, 5539, 5540, 5541, 5542, 5543, 5544, 5545, 5546, 5547, 5548, 5549,
5550, 5551, 5552, 5553, 5554, 5555, 5556, 5557, 5558, 5559, 5560, 5561,
5562, 5563, 5564, 5565, 5566, 5567, 5568, 5569, 5570, 5571, 5572, 5573,
5574, 5575, 5576, 5577, 5578, 5579, 5580, 5581, 5582, 5583, 5584, 5585,
5586, 5587, 5588, 5589, 5590, 5591, 5592, 5593, 5594, 5595, 5596, 5597,
5598, 5599, 5600, 5601, 5602, 5603, 5604, 5605, 5606, 5607, 5608, 5609,
5610, 5611, 5612, 5613, 5614, 5615, 5616, 5617, 5618, 5619, 5620, 5621,
5622, 5623, 5624, 5625, 5626, 5627, 5628, 5629, 5630, 5631, 5632, 5633,
5634, 5635, 5636, 5637, 5638, 5639, 5640, 5641, 5642, 5643, 5644, 5645,
5646, 5647, 5648, 5649, 5650, 5651, 5652, 5653, 5654, 5655, 5656, 5657,
5658, 5659, 5660, 5661, 5662, 5663, 5664, 5665, 5666, 5667, 5668, 5669,
5670, 5671, 5672, 5673, 5674, 5675, 5676, 5677, 5678, 5679, 5680, 5681,
5682, 5683, 5684, 5685, 5686, 5687, 5688, 5689, 5690, 5691, 5692, 5693,
5694, 5695, 5696, 5697, 5698, 5699, 5700, 5701, 5702, 5703, 5704, 5705,
5706, 5707, 5708, 5709, 5710, 5711, 5712, 5713, 5714, 5715, 5716, 5717,
5718, 5719, 5720, 5721, 5722, 5723, 5724, 5725, 5726, 5727, 5728, 5729,
5730, 5731, 5732, 5733, 5734, 5735, 5736, 5737, 5738, 5739, 5740, 5741,
5742, 5743, 5744, 5745, 5746, 5747, 5748, 5749, 5750, 5751, 5752, 5753,
5754, 5755, 5756, 5757, 5758, 5759, 5760, 5761, 5762, 5763, 5764, 5765,
5766, 5767, 5768, 5769, 5770, 5771, 5772, 5773, 5774, 5775, 5776, 5777,
5778, 5779, 5780, 5781, 5782, 5783, 5784, 5785, 5786, 5787, 5788, 5789,
5790, 5791, 5792, 5793, 5794, 5795, 5796, 5797, 5798, 5799, 5800, 5801,
5802, 5803, 5804, 5805, 5806, 5807, 5808, 5809, 5810, 5811, 5812, 5813,
5814, 5815, 5816, 5817, 5818, 5819, 5820, 5821, 5822, 5823, 5824, 5825,
5826, 5827, 5828, 5829, 5830, 5831, 5832, 5833, 5834, 5835, 5836, 5837,
5838, 5839, 5840, 5841, 5842, 5843, 5844, 5845, 5846, 5847, 5848, 5849,
5850, 5851, 5852, 5853, 5854, 5855, 5856, 5857, 5858, 5859, 5860, 5861,
5862, 5863, 5864, 5865, 5866, 5867, 5868, 5869, 5870, 5871, 5872, 5873,
5874, 5875, 5876, 5877, 5878, 5879, 5880, 5881, 5882, 5883, 5884, 5885,
5886, 5887, 5888, 5889, 5890, 5891, 5892, 5893, 5894, 5895, 5896, 5897,
5898, 5899, 5900, 5901, 5902, 5903, 5904, 5905, 5906, 5907, 5908, 5909,
5910, 5911, 5912, 5913, 5914, 5915, 5916, 5917, 5918, 5919, 5920, 5921,
5922, 5923, 5924, 5925, 5926, 5927, 5928, 5929, 5930, 5931, 5932, 5933,
5934, 5935, 5936, 5937, 5938, 5939, 5940, 5941, 5942, 5943, 5944, 5945,
5946, 5947, 5948, 5949, 5950, 5951, 5952, 5953, 5954, 5955, 5956, 5957,
5958, 5959, 5960, 5961, 5962, 5963, 5964, 5965, 5966, 5967, 5968, 5969,
5970, 5971, 5972, 5973, 5974, 5975, 5976, 5977, 5978, 5979, 5980, 5981,
5982, 5983, 5984, 5985, 5986, 598 6073, 6074, 6075, 6076, 6077, 6078, 6079,
Third dropout 5987..6072
I'm not going to at the rest -- EACH print statement is sending
>>> len(repr([i for i in range(10000)]))
58890
58kB of data!
The above three dropouts occurred at (approximately)
>>> len(repr([i for i in range(2574)]))
14334
>>> len(repr([i for i in range(3598)]))
20478
>>> len(repr([i for i in range(5987)]))
34812
>>> 20478-143341
6144
>>> 34812-20478
14334
14kB, 6kB, and 14kB; EACH dropout lasted for 510 bytes (5100 bits, or
0.53 seconds)
I'd also like to reiterate that Xon/Xoff is considered a /software/
flow-control, and pyserial includes a method (though it is not in /my/
version of pyserial)
set_input_flow_control(enable)
Platform: Posix
Parameters: enable (bool) – Set flow control state.
Manually control flow - when software flow control is enabled.
This will send XON (true) and XOFF (false) to the other device.
I have not tried running with the Windows COM port set to interrupt at
7 (or fewer) bytes, rather than the default of 14 bytes. I suspect the
dropouts are occurring when either pyserial or python itself is doing some
memory reallocation (doubling the size of the internal receive buffer?)
debian@beaglebone:~$ python --version
Python 2.7.9
debian@beaglebone:~$ uname -a
Linux beaglebone 4.4.54-ti-r93 #1 SMP Fri Mar 17 13:08:22 UTC 2017 armv7l
GNU/Linux
debian@beaglebone:~$
Changing the program to read in smaller chunks, with timeout, AND to
MANUALLY SEND Xoff/Xon to the BBB produced:
-=-=-=-=-
import serial
import sys
# the BBB USB-client port comes up as COM4 on my system
bbb = serial.Serial("COM4", 9600, xonxoff=True, timeout=0.5)
#flush input buffer (where did .read_all() come from?)
bbb.reset_input_buffer()
bbb.write('python -c "while True: print([i for i in range(10000)])"\n')
bbb.write(chr(19)) #pause
try:
while True:
bbb.write(chr(17)) #resume
data = bbb.read(500) #get 500 bytes or timeout
bbb.write(chr(19)) #pause
sys.stdout.write(data)
except: #nasty blank except
bbb.write(chr(17))
bbb.write(chr(3)) #attempt to kill running script
bbb.close()
-=-=-=-=-
Well, I was going to include the output -- but I think I'm at the limit
for a single message in Agent...
I will state that I don't see any dropouts in the first 7 blocks
(~350kB), then had a dropouts occur in the subsequent 40 odd blocks. My
suspicion -- reading 500bytes at a time wasn't emptying the received
buffer, and eventually the buffer filled. Alternative would be to swap the
pause/resume points... Pause before issuing the read, and loop (if needed)
to read everything available and process any complete data (if \n
delimited), THEN resume and loop back to top (where almost immediately a
pause will be sent).
arsi
May 4, 2017, 6:03:30 PM5/4/17
to beagl...@googlegroups.com
Hi,
call flush after write command!!!
call flush after write command!!!
-
flush() -
Flush of file like objects. In this case, wait until all data is written.
William Hermans
May 4, 2017, 6:10:16 PM5/4/17
to beagl...@googlegroups.com
I think it's important to note that the OP is not sending data from the beaglebone, but receiving it. At least looking at the last code he pasted. Usually you wont see "COM9" on a Linux system ;)
At any rate if receiving, you don't need to use flush at all. If it's anything like fflush() in C.
William Hermans
May 4, 2017, 6:12:27 PM5/4/17
to beagl...@googlegroups.com
Additionally, the output had newline "\n" every time output was sent.
David Howlett
May 4, 2017, 6:29:19 PM5/4/17
to beagl...@googlegroups.com
>> I am confused as to what your actual hardware configuration is, with respect to the "serial communications."
The actual hardware used in the final project with the i2c connectors soldered on to a prototyping cape looks like this:
I have repeated the exact same fault on this:
>> Are you using a COM port in Windows to talk to a USB to serial cable, which is talking to a hardware serial port in the BBB?
I plugged the micro USB port on the BBB into my USB hub. The USB hub is currently attached to a windows PC but the same fault happens when the USB cable is plugged into my mac book running OSX.
Within a minute of the BBB being plugged in an extra COM port is added in windows. I connect to the com port with Termite (terminal program), ZOC7 (terminal program) or pyserial (the python serial library).
When I connect to the COM port I can login and get a shell. I ran some relevant commands in the shell so you can see the results.
root@beaglebone:/dev# w
20:57:21 up 6:52, 1 user, load average: 0.02, 0.06, 0.05
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root ttyGS0 14Nov15 1.00s 0.97s 0.02s w
root@beaglebone:/dev# who
root ttyGS0 2015-11-14 00:15
root@beaglebone:/dev# tty
/dev/ttyGS0
root@beaglebone:/dev# whoami
root
root@beaglebone:/dev# ls
alarm log_system ram11 tty16 tty44 ubi_ctrl
ashmem loop0 ram12 tty17 tty45 uinput
audio loop1 ram13 tty18 tty46 urandom
autofs loop2 ram14 tty19 tty47 usbmon0
binder loop3 ram15 tty2 tty48 usbmon1
block loop4 ram2 tty20 tty49 usbmon2
btrfs-control loop5 ram3 tty21 tty5 vcs
bus loop6 ram4 tty22 tty50 vcs1
char loop7 ram5 tty23 tty51 vcs2
console loop-control ram6 tty24 tty52 vcs3
core mapper ram7 tty25 tty53 vcs4
cpu_dma_latency mem ram8 tty26 tty54 vcs5
disk mixer ram9 tty27 tty55 vcs6
dri mmcblk0 random tty28 tty56 vcs7
dsp mmcblk0boot0 root tty29 tty57 vcsa
fb0 mmcblk0boot1 rtc0 tty3 tty58 vcsa1
fd mmcblk0p1 shm tty30 tty59 vcsa2
full mmcblk0p2 snd tty31 tty6 vcsa3
fuse mqueue sndstat tty32 tty60 vcsa4
hwrng net stderr tty33 tty61 vcsa5
i2c-0 network_latency stdin tty34 tty62 vcsa6
i2c-1 network_throughput stdout tty35 tty63 vcsa7
initctl null tty tty36 tty7 watchdog
input ppp tty0 tty37 tty8 watchdog0
kmem psaux tty1 tty38 tty9 xconsole
kmsg ptmx tty10 tty39 ttyGS0 zero
log ptp0 tty11 tty4 ttyO0
log_events pts tty12 tty40 ttyS0
logibone_mem ram0 tty13 tty41 ttyS1
log_main ram1 tty14 tty42 ttyS2
log_radio ram10 tty15 tty43 ttyS3
>> So, you've actually delved further into using the UARTs on this platform than I ever have.
I have also come to suspect that there are no hardware UARTS involved. I believe that the serial line is being emulated.
>> Anyway, my first instinct wants to say baud rate is involved somehow.
On the PC side I can set the baudrate to any number. The data rate and the error rate appear to be unaffected. For example:
x = serial.Serial('COM10', baudrate=10)
x = serial.Serial('COM10', baudrate=9600)
x = serial.Serial('COM10', baudrate=100_000_000_000)
I have also set the baud rate with Termite 3.1 to demonstrate that it is not a bug in the python serial library.
On the BBB end the speed is always reported as 9600. Some commands to change the speed succeed and others fail but the reported speed is unchanged.
root@beaglebone:/dev# stty ispeed 7200
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;
root@beaglebone:/dev# stty ispeed 14400
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;
root@beaglebone:/dev# stty ispeed 2400
stty: standard input: unable to perform all requested operations
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;
I will finish replying to everyone tomorrow. It is getting late where I am.
Dennis Lee Bieber
May 4, 2017, 6:35:15 PM5/4/17
to beagl...@googlegroups.com
On Thu, 4 May 2017 15:10:07 -0700, William Hermans
<yyr...@gmail.com> declaimed the following:
>I think it's important to note that the OP is not sending data from the
>beaglebone, but receiving it. At least looking at the last code he pasted.
>Usually you wont see "COM9" on a Linux system ;)
>
The OP is running a program on Windows that is sending a command string
TO the BBB, and that command at the shell generates output (on stdout,
connected to the Windows COM port).
<yyr...@gmail.com> declaimed the following:
>I think it's important to note that the OP is not sending data from the
>beaglebone, but receiving it. At least looking at the last code he pasted.
>Usually you wont see "COM9" on a Linux system ;)
>
TO the BBB, and that command at the shell generates output (on stdout,
connected to the Windows COM port).
David Howlett
May 4, 2017, 6:39:43 PM5/4/17
to BeagleBoard
>> I am confused as to what your actual hardware configuration is, with respect to the "serial communications."
The actual hardware used in the final project with the i2c connectors soldered on to a prototyping cape looks like this:
I have repeated the exact same fault on this hardware:
>> Are you using a COM port in Windows to talk to a USB to serial cable, which is talking to a hardware serial port in the BBB?
x = serial.Serial('COM10', baudrate=10)
x = serial.Serial('COM10', baudrate=9600)
x = serial.Serial('COM10', baudrate=100_000_000_000)
I have also set the baud rate with Termite 3.1 to demonstrate that it is not a bug in the python serial library.
On the BBB end the speed is always reported as 9600. Some commands to change the speed succeed and others fail but the reported speed is unchanged.
root@beaglebone:/dev# stty ispeed 7200
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;
root@beaglebone:/dev# stty ispeed 14400
root@beaglebone:/dev# stty
speed 9600 baud; line = 0;
root@beaglebone:/dev# stty ispeed 2400
stty: standard input: unable to perform all requested operations
William Hermans
May 4, 2017, 7:12:25 PM5/4/17
to beagl...@googlegroups.com
On Thu, May 4, 2017 at 3:32 PM, Dennis Lee Bieber <wlf...@ix.netcom.com> wrote:
On Thu, 4 May 2017 15:10:07 -0700, William Hermans
<yyr...@gmail.com> declaimed the following:
>I think it's important to note that the OP is not sending data from the
>beaglebone, but receiving it. At least looking at the last code he pasted.
>Usually you wont see "COM9" on a Linux system ;)
>
The OP is running a program on Windows that is sending a command string
TO the BBB, and that command at the shell generates output (on stdout,
connected to the Windows COM port).
So, other than repeating exactly what I said, what's your point ? Also newline eliminates the need for flush.
Dennis Lee Bieber
May 4, 2017, 7:41:24 PM5/4/17
to beagl...@googlegroups.com
On Thu, 4 May 2017 16:12:09 -0700, William Hermans
majority of the data IS being sent by the BBB. What was posted was a
program that just sent a one line command to the BBB to start the BBB
generating the data. Granted, the OP was ignoring the return data in that
test code... (Which is not the case in my overly long post -- I do ensure I
read the return data).
<yyr...@gmail.com> declaimed the following:
>On Thu, May 4, 2017 at 3:32 PM, Dennis Lee Bieber <wlf...@ix.netcom.com>
>wrote:
>
>> On Thu, 4 May 2017 15:10:07 -0700, William Hermans
>> <yyr...@gmail.com> declaimed the following:
>>
>>
>> >I think it's important to note that the OP is not sending data from the
>> >beaglebone, but receiving it. At least looking at the last code he pasted.
>> >Usually you wont see "COM9" on a Linux system ;)
>> >
>> The OP is running a program on Windows that is sending a command
>> string
>> TO the BBB, and that command at the shell generates output (on stdout,
>> connected to the Windows COM port).
>>
>
>So, other than repeating exactly what I said, what's your point ? Also
>newline eliminates the need for flush.
Really? You state "not sending data from the beaglebone" -- but the
>On Thu, May 4, 2017 at 3:32 PM, Dennis Lee Bieber <wlf...@ix.netcom.com>
>wrote:
>
>> On Thu, 4 May 2017 15:10:07 -0700, William Hermans
>> <yyr...@gmail.com> declaimed the following:
>>
>>
>> >I think it's important to note that the OP is not sending data from the
>> >beaglebone, but receiving it. At least looking at the last code he pasted.
>> >Usually you wont see "COM9" on a Linux system ;)
>> >
>> The OP is running a program on Windows that is sending a command
>> string
>> TO the BBB, and that command at the shell generates output (on stdout,
>> connected to the Windows COM port).
>>
>
>So, other than repeating exactly what I said, what's your point ? Also
>newline eliminates the need for flush.
majority of the data IS being sent by the BBB. What was posted was a
program that just sent a one line command to the BBB to start the BBB
generating the data. Granted, the OP was ignoring the return data in that
test code... (Which is not the case in my overly long post -- I do ensure I
read the return data).
Dennis Lee Bieber
May 4, 2017, 7:53:54 PM5/4/17
to beagl...@googlegroups.com
On Thu, 4 May 2017 15:39:43 -0700 (PDT), David Howlett
<david.h...@gmail.com> declaimed the
following:
>I plugged the micro USB port on the BBB into my USB hub. The USB hub is
>currently attached to a windows PC but the same fault happens when the USB
>cable is plugged into my mac book running OSX.
>
Which, since there is no FTDI chip on the BBB, is effectively an
emulated serial port... The USB-serial driver responds to I/O operations as
if it were a serial port, but things like baud rate are mostly meaningless
-- the driver moves bytes from the client buffer to the USB buffer, and USB
transfers at (if USB2.0 full speed -- 480kbps; ignoring the breaks caused
by USB packet size AND that the host computer has to poll the USB device to
ask for each packet of data; effective throughput being closer to 250kbps).
And, the Windows (or Mac) side is also an emulated serial, but as the
host side, the USB driver controls the polling of the other end asking for
data or to send packets.
>I have also come to suspect that there are no hardware UARTS involved. I
>believe that the serial line is being emulated.
>
Unless you use wires on the UART pins of the BBB, likely true. Those
may be hardware UARTs, but as soon as something is connected that converts
to USB, the actual data transfer is USB and the other end emulated.
>
>On the PC side I can set the baudrate to any number. The data rate and the
>error rate appear to be unaffected. For example:
>
>x = serial.Serial('COM10', baudrate=10)
>x = serial.Serial('COM10', baudrate=9600)
>x = serial.Serial('COM10', baudrate=100_000_000_000)
>
As I suspect -- the baud rate is irrelevant when you get into the USB
protocol. At most, it may control a "filter" as to how rapidly the drivers
relay stuff to the USB protocol. That is -- at slow rates there may only be
a few bytes of data per USB packet, whereas at really high rates each USB
packet will contain more data (and hence be more efficient when taking the
polling overhead into account).
<david.h...@gmail.com> declaimed the
following:
>I plugged the micro USB port on the BBB into my USB hub. The USB hub is
>currently attached to a windows PC but the same fault happens when the USB
>cable is plugged into my mac book running OSX.
>
emulated serial port... The USB-serial driver responds to I/O operations as
if it were a serial port, but things like baud rate are mostly meaningless
-- the driver moves bytes from the client buffer to the USB buffer, and USB
transfers at (if USB2.0 full speed -- 480kbps; ignoring the breaks caused
by USB packet size AND that the host computer has to poll the USB device to
ask for each packet of data; effective throughput being closer to 250kbps).
And, the Windows (or Mac) side is also an emulated serial, but as the
host side, the USB driver controls the polling of the other end asking for
data or to send packets.
>I have also come to suspect that there are no hardware UARTS involved. I
>believe that the serial line is being emulated.
>
may be hardware UARTs, but as soon as something is connected that converts
to USB, the actual data transfer is USB and the other end emulated.
>
>On the PC side I can set the baudrate to any number. The data rate and the
>error rate appear to be unaffected. For example:
>
>x = serial.Serial('COM10', baudrate=10)
>x = serial.Serial('COM10', baudrate=9600)
>x = serial.Serial('COM10', baudrate=100_000_000_000)
>
protocol. At most, it may control a "filter" as to how rapidly the drivers
relay stuff to the USB protocol. That is -- at slow rates there may only be
a few bytes of data per USB packet, whereas at really high rates each USB
packet will contain more data (and hence be more efficient when taking the
polling overhead into account).
Graham Haddock
May 4, 2017, 8:42:31 PM5/4/17
to BeagleBoard
So, the COM port on Windows is talking to a virtual serial port in the BBB USB widget.
There is no hardware UART involved, and no hardware clocks anywhere to throttle or pace anything.
Then, you push this for transfer rate, while running it under a non realtime OS.
I guess I am not surprised that you are having problems.
The USB widget is nice for doing simple things, with fast, simple set up for beginners.
I suspect you are trying to run it in an application where it has not been well tested.
For a stable system, I would recommend considering doing the data transfer over a real hardware serial port, or if that is not fast enough, do the transfer using Ethernet. (And I don't mean Ethernet over USB using the Widget.)
I find the true hardware serial ports, and the genuine FTDI serial to USB converters to be stable well above 100,000 bits per second. Some report success in the megabits per second range, I have just not had to personally go there, yet.
You never have been specific about how many bits per second of data that you have to move.
I think that is an important place to start.
--- Graham
==
--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/1n0IkcmhEq8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/n0fngc1j3g8kfrh49uqngndtsj8b0ohjqi%404ax.com.
William Hermans
May 4, 2017, 9:12:03 PM5/4/17
to beagl...@googlegroups.com
Dennis, keep your comments directed at helping the OP, and stop
worrying about what I'm saying. We're obviously not talking about the same thing, and your comments to me are not helping the OP solve his problem.
David, As an example if I run the following program there is no data loss:import serial.tools.list_ports# connect to the BBB serial port on Windowsx = serial.Serial('COM9')x.write(b'yes "123456"\n')while True:received = x.readline()if received != b'123456\r\n':print(received)But there is data loss if I run this:import serial.tools.list_ports# connect to the BBB serial port on Windowsx = serial.Serial('COM9')x.write(b'yes "1234567"\n')while True:received = x.readline()if received != b'1234567\r\n':print(received)
This seems to me that you are indeed overflowing your buffer. How or why I'm not sure, but if sending 6 characters all is fine, and 7 characters does not transmit properly then obviously something is wrong. I'd look into googling the driver name this device uses against any known reliability issues in Linux. One odd thing I did notice about your output you've shown us, is that everything in your received = x.readline() loop seems to be working fine.But all the failures are related to this, or a similar line you've not shown us yet: x.write(b'yes "1234567"\n')
I'm suspecting you're not showing us all your code however. Because the output you've shown us does not necessarily "jibe" with the code I'm seeing here.
I'm suspecting you're not showing us all your code however. Because the output you've shown us does not necessarily "jibe" with the code I'm seeing here.
William Hermans
May 4, 2017, 9:19:49 PM5/4/17
to beagl...@googlegroups.com
On Thu, May 4, 2017 at 5:42 PM, Graham Haddock <gra...@flexradio.com> wrote:
So, the COM port on Windows is talking to a virtual serial port in the BBB USB widget.There is no hardware UART involved, and no hardware clocks anywhere to throttle or pace anything.Then, you push this for transfer rate, while running it under a non realtime OS.I guess I am not surprised that you are having problems.
This is kind of what I'm thinking, but just sayin' Graham, "gadget" not "widgit" :P But yeah I do not know this hardware at all, but the conversion from USB to RS485, there can not be any flow control there. So the gadget side of the hardware has no flow control, where the RS485 side *may*. I'd actually speed up the transmission rate, and perhaps put in a usleep() between each send to fine tune things, and call it quits.
The USB widget is nice for doing simple things, with fast, simple set up for beginners.I suspect you are trying to run it in an application where it has not been well tested.For a stable system, I would recommend considering doing the data transfer over a real hardware serial port, or if that is not fast enough, do the transfer using Ethernet. (And I don't mean Ethernet over USB using the Widget.)I find the true hardware serial ports, and the genuine FTDI serial to USB converters to be stable well above 100,000 bits per second. Some report success in the megabits per second range, I have just not had to personally go there, yet.You never have been specific about how many bits per second of data that you have to move.I think that is an important place to start.
So *maybe* if UART4 on the beaglebone was used in full RS485 mode, talking to the USB to RS485 converter connected to the Windows machine through USB. *Maybe* that could, or would help solve these issues ? Just a guess but hey probably worth looking into.
arsi
May 5, 2017, 4:17:58 AM5/5/17
to beagl...@googlegroups.com
Hi,
I plugged the micro USB port on the BBB into my USB hub. The USB hub is currently attached to a windows PC but the same fault happens when the USB cable is plugged into my mac book running OSX.Which, since there is no FTDI chip on the BBB, is effectively an emulated serial port... The USB-serial driver responds to I/O operations as if it were a serial port, but things like baud rate are mostly meaningless -- the driver moves bytes from the client buffer to the USB buffer, and USB transfers at (if USB2.0 full speed -- 480kbps; ignoring the breaks caused by USB packet size AND that the host computer has to poll the USB device to ask for each packet of data; effective throughput being closer to 250kbps).
I haven't noticed that you are using USB serial gadged
driver.What Dennis wrote is true.
Here you can see the source code of USB serial gadged driver: http://elixir.free-electrons.com/linux/v4.3.5/source/drivers/usb/gadget/function/u_serial.c
Presented baudrate for tty is fixed to 9600:
Change
of baud rate is not handled anywhere, data are transferred at full USB speed.
The easiest solution is to do a simple handshake:
On PC:
Here you can see the source code of USB serial gadged driver: http://elixir.free-electrons.com/linux/v4.3.5/source/drivers/usb/gadget/function/u_serial.c
Presented baudrate for tty is fixed to 9600:
/* 9600-8-N-1 ... matches defaults expected by "usbser.sys" on
* MS-Windows. Otherwise, most of these flags shouldn't affect
* anything unless we were to actually hook up to a serial line.
*/
gs_tty_driver->init_termios.c_cflag = B9600 | CS8 | CREAD | HUPCL | CLOCAL;
gs_tty_driver->init_termios.c_ispeed = 9600;
gs_tty_driver->init_termios.c_ospeed = 9600;
tty_set_operations(gs_tty_driver, &gs_tty_ops);
The easiest solution is to do a simple handshake:
On PC:
import serial.tools.list_ports
# connect to the BBB serial port on Windows
x = serial.Serial('COM9')
while True:
x.write(b"\r\n")
x.flush()
received = x.readline()
print(received)
import serial.tools.list_ports
x = serial.Serial('/dev/ttyGS0')
while True:
received = x.readline()
x.write(b"DATA\r\n")
x.flush()The BBB app must be run as first... I do not know much about python, maybe it's ok ;)
The Second solution with baudrate emulation without RS232/USB converter:
On BBB enable ttyO1 and ttyO2. Connect Tx pins with Rx..
Install ser2net and configure it on port ttyO2.
Run your app on ttyO1
On windows install http://www.eterlogic.com/Products.VSPE.html and connect it to ser2net port.
And you have "simple" baud rate emulator ;)
BTW:
Why you want to use RS232, i would choose TCP ,UDP or WebServer..
if you want, can I send you, ready to use modular web server in java.. jetty+vaadin+netbeans lookup
ArSi
David Howlett
May 6, 2017, 11:25:31 AM5/6/17
to BeagleBoard, ar...@chello.sk
Sorry for double posting last time, this is my first time using google groups.
>> I wonder if you run the application that failing for you there is you pipe that data to a file,
>> instead of to stdout, if you would be experiencing the same problem ?
The data loss problem also happens when the yes command is just repeating a string over and over to stdout so I don't think the problem is python related.
>> Why don't you humor me, and give that a try ?
I tried:
$ python3.6 ~/reusable/BBB_pressure_streamer.py > some_log_file.txt
When I view the log file I can't see any errors
>> Have you tried reducing the receive FIFO level?
I had not thought of that. Today I tried setting the Tx and Rx buffers to 1, 8 and 14 bytes. Data continued to be lost on all settings.
>> Suspect I'd start with
>> stty ixon ixoff -ixany
>> and ensure the other end is set for equivalent.
I tried:
root@beaglebone:~# stty --file=/dev/ttyGS0 ixon ixoff -ixany
root@beaglebone:~# stty --all
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon ixoff
-iuclc -ixany imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke
and I also turned on flow control on the python side at the same time but I still got data loss.
>> Recommend you specify /which/ serial port
I was originally running stty while connected to the virtual serial port but I forgot to mention that detail.
>> I can't duplicate your results as
>> debian@beaglebone:~$ python3.6 -c "while True: print(sum(i for i in range(10000)))"
I previously installed python3.6 on all my boards. I should have given an example that only depended on base Debian.
>> #flush input buffer (where did .read_all() come from?)
I was using it to clear the serial buffer of all data. Calling .reset_input_buffer() is better
>> OKAY -- THIS PRODUCED a drop-out 2574..2659
Thanks, it is nice to know that someone else can replicate the issue, it is not just me doing something stupid.
>> For a stable system, I would recommend considering doing the data transfer over a real hardware serial port
Today I purchased two connectors for the hardware serial pins from:
I will see if that helps.
>> You never have been specific about how many bits per second of data that you have to move.
Today I measured the average data rate of my application to be 69,218 bytes/second.
>> This seems to me that you are indeed overflowing your buffer.
>> How or why I'm not sure, but if sending 6 characters all is fine,
>> and 7 characters does not transmit properly then obviously something is wrong.
When I send:
x.write(b'yes "123456"\n')
I receive 8 bytes per line not 6 due to the addition of /r/n. I don't think this is terribly important though
>> I'm suspecting you're not showing us all your code
My working repository is here:
A permanent link to the relevant commit is here:
There is not much to see in the repository but you are welcome to have a look.
The only thing worth noting is I use python 3.6 on the PC side too.
>> Here you can see the source code of USB serial gadged driver
Thankyou
>> The easiest solution is to do a simple handshake
That is a good idea for avoiding data loss, the troublesome buffer would never fill so there would be no data loss.
>> Why you want to use RS232, i would choose TCP ,UDP or WebServer
Personal preexisting skill set. I have been using serial connections for 4 years on multiple projects with multiple pieces of hardware. I wrongly believed that it would be easy to just use serial again.
In the future I plan to use the hardware serial connection.
William Hermans
May 6, 2017, 12:44:35 PM5/6/17
to beagl...@googlegroups.com, david.h...@gmail.com
$ python3.6 ~/reusable/BBB_pressure_streamer.py > some_log_file.txtWhen I view the log file I can't see any errors
So, actually, scripting languages in general can be far slower, that their native code counterparts. Firstly, because there is an interpreter, and secondly because of code bloat, through using libraries. Almost as a rule, larger executables will lead to poorer performing code. Python in particular is one of if not the slowest languages also when bench marked in some tests. And I do not mean by a second or two. I mean by a huge margin in seconds, for specific tests. Just as an example though, in a few tests I remember reading, Compared to Nodejs, Python was behind by a minute or more.
My point here is not to say Nodejs is better or not though. Nodejs is pretty terrible it's self. This is meant demonstrate in general that scripting languages can be terrible when your app is I/O intensive. The reason why I suggested piping the output of your script to a file was to see if there was potentially any "normal" I/O bottlenecks involved. All languages can suffer from this however, because printing to screen( stdout ) is, or can be very I/O intensive. In C, you're maybe going to max out around a couple hundred "samples" to stdout a second, before your application will noticeably slow down. With Nodejs, it's actually far less. I do not know if Python is slower than Nodejs in this case. It's been a while since I've read the benchmarks, but actually the benchmarks were not testing this capability anyhow. Maybe they should have been ?
All in all, I'm still not convinced this is the problem in whole. But I do feel that it could be a contributing factor. It still would not explain why you seem to be getting your full amount of transmissions. Just that some of those transmissions are "empty". Especially when those empty transmissions are not consistent in timing / frequency. I would suggest that you do keep this in mind, and perhaps at least consider testing output to a file further. Instead of testing while output is to stdout. At minimum, this will allow you to rule out, output to stdout as a contributing factor in your code. With very little wasted time on your behalf.
--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/d14843cc-8eb0-4fbb-bec1-3e9052c272ae%40googlegroups.com.
arsi
May 6, 2017, 3:09:24 PM5/6/17
to beagl...@googlegroups.com
>> Why you want to use RS232, i would choose TCP ,UDP or WebServer
Personal preexisting skill set. I have been using serial connections for 4 years on multiple projects with multiple pieces of hardware. I wrongly believed that it would be easy to just use serial again.
And I will finally learn a python ;)
Test this: (on linux telnet ip_of_python_host 5000 on
windows with putty telnet to port 5000 )
import select
import socket
from time import sleep
#telnet escape sequences
CLEAR_SCREEN = chr(27) + "[2J"
MOVE_TO_LINE_1 = chr(27) + "[1;1H"
MOVE_TO_LINE_2 = chr(27) + "[2;1H"
MOVE_TO_LINE_3 = chr(27) + "[3;1H"
MOVE_TO_LINE_4 = chr(27) + "[4;1H"
MOVE_TO_LINE_5 = chr(27) + "[5;1H"
MOVE_TO_LINE_6_ST = chr(27) + "[6;1H"
MOVE_TO_LINE_6 = chr(27) + "[6;15H"
CLEAR_LINE = chr(27) + "[2K"
CLEAR_LINE_TO_END = chr(27) + "[K"
SAVE_CURSOR = chr(27) + "7"
RESTORE_CURSOR = chr(27) + "8"
TXT_CMD = "Enter command:"
#
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.bind(('0.0.0.0', 5000))
server.listen(5)
inputs = [server]
counter1 = 0
counter2 = 0
counter3 = 0
counter4 = 0
counter5 = 0
addrs = {}
terminalInitOk = []
data = None
while inputs:
readable, writable, exceptional = select.select(inputs, inputs, inputs)
for s in readable:
if s is server:
connection, client_address = s.accept()
connection.setblocking(0)
inputs.append(connection)
print('Client connected...' + str(client_address))
addrs[connection] = client_address
else:
try:
if s in inputs:
data = s.recv(1024)
if data:
print('Client:' + str(addrs[s]) + ' Received command:' + data + '\n')
if s in terminalInitOk:
s.send(MOVE_TO_LINE_6_ST+ CLEAR_LINE+ TXT_CMD + MOVE_TO_LINE_6)
except:
inputs.remove(s)
terminalInitOk.remove(s)
c.close();
print('Client ' + str(addrs[s]) + ' disconnected..')
if len(inputs):
counter1 = counter1 + 1 #At least one client is attached make the calculation
counter2 = counter2 + 10
counter3 = counter3 + 100
counter4 = counter4 + 1000
counter5 = counter5 + 10000
for s in writable:
try:
if s in inputs: #send new value to client
if s in terminalInitOk:
s.send(SAVE_CURSOR + MOVE_TO_LINE_1 + CLEAR_LINE + str(counter1) \
+ MOVE_TO_LINE_2 + CLEAR_LINE + str(counter2) \
+ MOVE_TO_LINE_3 + CLEAR_LINE + str(counter3) \
+ MOVE_TO_LINE_4 + CLEAR_LINE + str(counter4)
+ MOVE_TO_LINE_5 + CLEAR_LINE + str(counter5) + RESTORE_CURSOR)
else:
#init telnet terminal
s.send(CLEAR_SCREEN +MOVE_TO_LINE_1+ str(counter1) \
+ MOVE_TO_LINE_2 + str(counter2) \
+ MOVE_TO_LINE_3 + str(counter3) \
+ MOVE_TO_LINE_4 + str(counter4) \
+ MOVE_TO_LINE_5 + str(counter5) \
+ MOVE_TO_LINE_6_ST + TXT_CMD + MOVE_TO_LINE_6)
terminalInitOk.append(s)
except:
inputs.remove(s)
terminalInitOk.remove(s)
print('Client ' + str(addrs[s]) + ' disconnected..')
s.close();
if len(writable):
sleep(0.1) #Slow down writing
for s in exceptional:
if s in inputs:
inputs.remove(s)
terminalInitOk.remove(s)
print('Client ' + str(addrs[s]) + ' disconnected..')
s.close();
arsi
Dennis Lee Bieber
May 6, 2017, 4:20:32 PM5/6/17
to beagl...@googlegroups.com
On Sat, 6 May 2017 08:25:31 -0700 (PDT), David Howlett
<david.h...@gmail.com> declaimed the
following:
>Thanks, it is nice to know that someone else can replicate the issue, it is
>not just me doing something stupid.
>
Note though that I managed to get 14kB before seeing a drop-out
(caveat: I was scrolling the captured output data looking for mis-aligned
line lengths, so a drop out exactly matching a sequence of whole numbers
and the following space would not have been seen in the wrapped window)
>
>Today I measured the average data rate of my application to be 69,218
>bytes/second.
>
If that's a sustained rate, and not burst, it is likely extreme for
serial protocols.
On 8n1 (10-bits per byte) set up, that give an effective rate of 692180
bps, or 692kbps. That's probably pushing the capabilities of the BBB to
buffer and send via an emulation of a serial port, and even most hardware
UARTs may not manage that fast a rate -- it's 6 times the 115.2kbps of most
serial ports.
>In the future I plan to use the hardware serial connection.
>
Recall that the debug serial header does NOT have hardware flow
control. The others, once pinmux is configured for UART, do support
RTS/CTS.
For contrast, Ethernet (using RJ54, not the USB gadget) starts at
10Mbps, so even with packet overhead should be capable. Though if you are
outputting just (say) 10 bytes at a time, the overhead could be significant
fraction <G> (~38 bytes worth [12 being a required gap between frames] at
the Ethernet level, and 16 bytes worth for IP packet header [no options
provided] and 16 for a TCP header [again, no options used] -- ~70 bytes of
overhead per sent frame).
Do you NEED every sent string, or are you okay with losses, as long as
it is not a drop-out within a data value?
TCP protocol ensures that data is received completely; if a packet is
lost/corrupted, the protocol should request a resend. TCP is a stream
protocol, and it is permissible for packets to be split and/or combined in
transit, so you do need to handle partial "lines" (so if the receiving end
is using <cr><lf> as a delimiter, if the end of the packet does not have
the delimiter, you need to prepend it to the next packet before searching
for delimiters).
In contrast, UDP will deliver the entire packet as sent -- no
fragmentation -- but it will just drop corrupted/lost packets. Unless the
data in the packets contains sequencing information, the application has no
way of knowing that a packet was lost. It's a "fire&forget" protocol.
Something else to consider: does your data need to be (ASCII) strings?
Using the struct module would let you build up binary data structures which
take less space in transit (though may be more difficult to verify). That
is, instead of sending 4294967295 (10 characters/bytes) you send the 4-byte
equivalent (internal 32-bit integer).
<david.h...@gmail.com> declaimed the
following:
>Thanks, it is nice to know that someone else can replicate the issue, it is
>not just me doing something stupid.
>
(caveat: I was scrolling the captured output data looking for mis-aligned
line lengths, so a drop out exactly matching a sequence of whole numbers
and the following space would not have been seen in the wrapped window)
>
>Today I measured the average data rate of my application to be 69,218
>bytes/second.
>
serial protocols.
On 8n1 (10-bits per byte) set up, that give an effective rate of 692180
bps, or 692kbps. That's probably pushing the capabilities of the BBB to
buffer and send via an emulation of a serial port, and even most hardware
UARTs may not manage that fast a rate -- it's 6 times the 115.2kbps of most
serial ports.
>In the future I plan to use the hardware serial connection.
>
control. The others, once pinmux is configured for UART, do support
RTS/CTS.
For contrast, Ethernet (using RJ54, not the USB gadget) starts at
10Mbps, so even with packet overhead should be capable. Though if you are
outputting just (say) 10 bytes at a time, the overhead could be significant
fraction <G> (~38 bytes worth [12 being a required gap between frames] at
the Ethernet level, and 16 bytes worth for IP packet header [no options
provided] and 16 for a TCP header [again, no options used] -- ~70 bytes of
overhead per sent frame).
Do you NEED every sent string, or are you okay with losses, as long as
it is not a drop-out within a data value?
TCP protocol ensures that data is received completely; if a packet is
lost/corrupted, the protocol should request a resend. TCP is a stream
protocol, and it is permissible for packets to be split and/or combined in
transit, so you do need to handle partial "lines" (so if the receiving end
is using <cr><lf> as a delimiter, if the end of the packet does not have
the delimiter, you need to prepend it to the next packet before searching
for delimiters).
In contrast, UDP will deliver the entire packet as sent -- no
fragmentation -- but it will just drop corrupted/lost packets. Unless the
data in the packets contains sequencing information, the application has no
way of knowing that a packet was lost. It's a "fire&forget" protocol.
Something else to consider: does your data need to be (ASCII) strings?
Using the struct module would let you build up binary data structures which
take less space in transit (though may be more difficult to verify). That
is, instead of sending 4294967295 (10 characters/bytes) you send the 4-byte
equivalent (internal 32-bit integer).
David Howlett
May 16, 2017, 5:18:05 AM5/16/17
to BeagleBoard
>>And I will finally learn a python ;)
Thanks for using my language. I read and thought about your code but in the end I chose to use SSH rather then telnet. SSH support is already present in the debian distribution I am using so I could get it working without writing any more code on the beagle bone side.
>> If that's a sustained rate, and not burst, it is likely extreme for serial protocols.
After rechecking my numbers, I agree it was a bad idea for me to be using serial for this data rate. I have switched to using SSH. I attach a minimal example below that I wrote to prove that the communication is faster and error free. To run this the paramiko library is required.
import paramiko
import time
# setup logging
paramiko.util.log_to_file('BBB_paramiko.log')
port = 22
# hostname = '192.168.1.34' # highest measured bytes per second over ethernet: 7,868,481
hostname = '192.168.7.2' # highest measured bytes per second over usb: 5,941,004
username = 'root'
password = ''
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(hostname, port, username, password)
stdin, stdout, stderr = client.exec_command('yes 1234567890', bufsize=2**24)
print('streaming started')
startTime = time.perf_counter()
total_data = 0
for i in range(10_000):
line = stdout.readline()
if line == '1234567890\n':
total_data += len(line)
else:
print(line)
# x = stdout.read(10_000) # this is faster then readline
# total_data += len(x)
print(f'bytes per second: {int(total_data/(time.perf_counter()-startTime))}')
client.close()
The above program transfers data with no losses at an average rate of 2,580,544 bytes per second. The speed changes according to how the data is processed on the PC side so that is probably the speed limitation now. Now that I have proven the communication works fine I will make my pressure data stream over SSH. At some point in the future I will probably remove the checksums on the pressure data once I can show there are no checksum failures in production.
I don't need any more help, thanks everyone.
David Howlett
May 16, 2017, 5:21:51 AM5/16/17
to BeagleBoard
One last thing I forgot to reply to:
>> Something else to consider: does your data need to be (ASCII) strings?
Not really, they are just easy to read. If I hit further performance issues I will switch format.
Dennis Lee Bieber
May 16, 2017, 9:27:42 AM5/16/17
to beagl...@googlegroups.com
On Tue, 16 May 2017 02:18:05 -0700 (PDT), David Howlett
<david.h...@gmail.com> declaimed the
following:
>After rechecking my numbers, I agree it was a bad idea for me to be using
>serial for this data rate. I have switched to using SSH. I attach a minimal
>example below that I wrote to prove that the communication is faster and
>error free. To run this the paramiko library is required.
>
You could probably have gone to just direct TCP/IP sockets, without
adding the overhead of SSH.
>The above program transfers data with no losses at an average rate of
>2,580,544 bytes per second. The speed changes according to how the data is
>processed on the PC side so that is probably the speed limitation now. Now
>that I have proven the communication works fine I will make my pressure
>data stream over SSH. At some point in the future I will probably remove
>the checksums on the pressure data once I can show there are no checksum
>failures in production.
>
Since SSH nominally runs over TCP/IP, and TCP data transfer is
guaranteed (individual IP packets may be lost, but the TCP protocol is
designed to detect such losses and request resends from the other end --
note that this is at the protocol level, the application never sees a
resend request), you shouldn't really need your own checksum. Every packet
already has a checksum in the TCP header, besides having packet length
information as another check.
<david.h...@gmail.com> declaimed the
following:
>After rechecking my numbers, I agree it was a bad idea for me to be using
>serial for this data rate. I have switched to using SSH. I attach a minimal
>example below that I wrote to prove that the communication is faster and
>error free. To run this the paramiko library is required.
>
adding the overhead of SSH.
>The above program transfers data with no losses at an average rate of
>2,580,544 bytes per second. The speed changes according to how the data is
>processed on the PC side so that is probably the speed limitation now. Now
>that I have proven the communication works fine I will make my pressure
>data stream over SSH. At some point in the future I will probably remove
>the checksums on the pressure data once I can show there are no checksum
>failures in production.
>
guaranteed (individual IP packets may be lost, but the TCP protocol is
designed to detect such losses and request resends from the other end --
note that this is at the protocol level, the application never sees a
resend request), you shouldn't really need your own checksum. Every packet
already has a checksum in the TCP header, besides having packet length
information as another check.
Reply all
Reply to author
Forward
0 new messages