UART Data Dropping

[Brandon Wuest]

I have been working with the IOIO board to expose a UART interface to an FPGA in which I exchange ASCII and non-ASCII messages/data between the host and client.  I have this interface working just fine on the PC using RXTX and a UART-to-USB converter, but when I move to Android and swap out the UART-to-USB for the IOIO, I run in to some problems.

So far I have been able to successfully read and write, with high reliability, all the "ASCII only" messages via the IOIO (where both the data I am reading and writing are all printable characters), but when I move to reading non-ASCII data, issues seem to crop up.  Originally the interface was setup such that I send (from the Android / Java side) a few printable characters and I received back 4096 bytes.  I was seeing incomplete data with the 4096 message size, so I started to decrease the size of the return messages thinking I was trying to read back too much data for the IOIO to handle at once.  As I decreased the number of bytes sent from the FPGA host to the Android client, I always saw less bytes than were being sent.  Also, if I had the host drop down to sending only 16 bytes, I would never read back a single byte on the Android side.  I also tried decreasing my baud rate from 115200 down to 9600 and still saw the same issues.  It seems that when these dropouts occur, at some point directly or soon after, I stop being able to receive any more data (so recovery from the drop out is next to impossible).  I did at one point increase ioio.lib.impl.Constants.BUFFER_SIZE from 1024 to 8192 which seemed increase the amount of data I could read back, but still I would always read back less than I expected.

Has anyone come across a similar issue to this before?  

Since this does not seem to be related to the amount of data I am sending from the host to the client, my only thought is that somehow the data I am transmitting from the host is getting interpreted by the IOIO board as a control character used by the connection implementation.  Does this sound possible at all?

Also not sure if this is relevant or not, but the host I am using, since it is connected to an FPGA, can be expected to be working very very near actual line speed.  Not sure if somehow if the host works right at line speed, if this could result in bytes getting missed if the clock on the IOIO is not spot on as well.

For reference, I tried this on a Nexus One, Motorola XT720, and a Samsung Galaxy Tab with all the same outcome.  Much of the debug I collected for this was via logcat running on the Android device that I monitored via SSH over wifi (so I was able to see the sequence of everything as it was going on).

Thanks for any help,

Brandon Wuest

epiqsolutions.com

[Ytai Ben-Tsvi]

On Mon, Jul 25, 2011 at 12:04 AM, Brandon Wuest <bwu...@gmail.com> wrote:

I have been working with the IOIO board to expose a UART interface to an FPGA in which I exchange ASCII and non-ASCII messages/data between the host and client.  I have this interface working just fine on the PC using RXTX and a UART-to-USB converter, but when I move to Android and swap out the UART-to-USB for the IOIO, I run in to some problems.

So far I have been able to successfully read and write, with high reliability, all the "ASCII only" messages via the IOIO (where both the data I am reading and writing are all printable characters), but when I move to reading non-ASCII data, issues seem to crop up.

 

I really can't imagine what can be different between ASCII and non-ASCII. There's nowhere in the code where some characters are treated differently than others. Can this be related to your code in some way? How do you print the data? Are you aware that the byte type in Java is signed?

 

Originally the interface was setup such that I send (from the Android / Java side) a few printable characters and I received back 4096 bytes.  I was seeing incomplete data with the 4096 message size, so I started to decrease the size of the return messages thinking I was trying to read back too much data for the IOIO to handle at once.  As I decreased the number of bytes sent from the FPGA host to the Android client, I always saw less bytes than were being sent.

With a loopback I sent megabytes on data as fast as possible without loss. There should not be any problem, and particularly in low baud rates. Does your code that does the reading read fast enough (i.e. have a dedicated thread that does a tight read() loop)?

 

Also, if I had the host drop down to sending only 16 bytes, I would never read back a single byte on the Android side.  I also tried decreasing my baud rate from 115200 down to 9600 and still saw the same issues.  It seems that when these dropouts occur, at some point directly or soon after, I stop being able to receive any more data (so recovery from the drop out is next to impossible).  I did at one point increase ioio.lib.impl.Constants.BUFFER_SIZE from 1024 to 8192 which seemed increase the amount of data I could read back, but still I would always read back less than I expected.

Has anyone come across a similar issue to this before?  

Since this does not seem to be related to the amount of data I am sending from the host to the client, my only thought is that somehow the data I am transmitting from the host is getting interpreted by the IOIO board as a control character used by the connection implementation.  Does this sound possible at all?

 

Not intentionally at least. Maybe there's a nasty bug hidden somewhere. Did you verify the stop bit and parity settings? Is it possible that the FPGA outputs open-drain and you haven't used a pull-up on your RX? (sorry, just shooting in the dark here with some possible causes for corruption - the IOIO currently silently discards frame-error bytes).

Also not sure if this is relevant or not, but the host I am using, since it is connected to an FPGA, can be expected to be working very very near actual line speed.  Not sure if somehow if the host works right at line speed, if this could result in bytes getting missed if the clock on the IOIO is not spot on as well.

 

There should be some permissible slack there. 

For reference, I tried this on a Nexus One, Motorola XT720, and a Samsung Galaxy Tab with all the same outcome.  Much of the debug I collected for this was via logcat running on the Android device that I monitored via SSH over wifi (so I was able to see the sequence of everything as it was going on).

That sounds really interesting! Do you need to root the device for running sshd? Do you need to install it yourself, or does it come with the stock OS?

Can you try to reproduce your problem with a loopback, and then sends a simple-as-possible program that reproduces it? I can then have a look.

 

Thanks for any help,

Brandon Wuest

epiqsolutions.com

--
You received this message because you are subscribed to the Google Groups "ioio-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/ioio-users/-/vsSAZWRTvzQJ.
To post to this group, send email to ioio-...@googlegroups.com.
To unsubscribe from this group, send email to ioio-users+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ioio-users?hl=en.

[Brandon Wuest]

I have been working with the IOIO board to expose a UART interface to an FPGA in which I exchange ASCII and non-ASCII messages/data between the host and client.  I have this interface working just fine on the PC using RXTX and a UART-to-USB converter, but when I move to Android and swap out the UART-to-USB for the IOIO, I run in to some problems.

So far I have been able to successfully read and write, with high reliability, all the "ASCII only" messages via the IOIO (where both the data I am reading and writing are all printable characters), but when I move to reading non-ASCII data, issues seem to crop up.

 

I really can't imagine what can be different between ASCII and non-ASCII. There's nowhere in the code where some characters are treated differently than others. Can this be related to your code in some way? How do you print the data? Are you aware that the byte type in Java is signed?

I realize there would most likely be no difference between printable and non-printable bytes.  My only thought here was that if I used non-printable characters AND caused data to drop, could I some how cause a state machine to get messed up in the IOIO controller or Java lib that would not occur when I stuck with printable characters only.  

 

Originally the interface was setup such that I send (from the Android / Java side) a few printable characters and I received back 4096 bytes.  I was seeing incomplete data with the 4096 message size, so I started to decrease the size of the return messages thinking I was trying to read back too much data for the IOIO to handle at once.  As I decreased the number of bytes sent from the FPGA host to the Android client, I always saw less bytes than were being sent.

With a loopback I sent megabytes on data as fast as possible without loss. There should not be any problem, and particularly in low baud rates. Does your code that does the reading read fast enough (i.e. have a dedicated thread that does a tight read() loop)?

I do have a dedicated loop that is using a large buffer to read as much data as it can per read call.  Let's say I didn't though (for my own education), but I did have ioio.lib.impl.Constants.BUFFER_SIZE set to 8192 with the most data ever transmitted by the host at a time was 4096, I shouldn't ever see dropouts right?  It looks like if I have a large enough buffer size internal to ioio.lib.impl.QueueInputStream, then it does not mater how "efficient" I am reading from the UART input stream that uses that class (again assuming the buffer size is larger than the max amount of data transmitted by the host)?  

 

Also, if I had the host drop down to sending only 16 bytes, I would never read back a single byte on the Android side.  I also tried decreasing my baud rate from 115200 down to 9600 and still saw the same issues.  It seems that when these dropouts occur, at some point directly or soon after, I stop being able to receive any more data (so recovery from the drop out is next to impossible).  I did at one point increase ioio.lib.impl.Constants.BUFFER_SIZE from 1024 to 8192 which seemed increase the amount of data I could read back, but still I would always read back less than I expected.

Has anyone come across a similar issue to this before?  

Since this does not seem to be related to the amount of data I am sending from the host to the client, my only thought is that somehow the data I am transmitting from the host is getting interpreted by the IOIO board as a control character used by the connection implementation.  Does this sound possible at all?

 

Not intentionally at least. Maybe there's a nasty bug hidden somewhere. Did you verify the stop bit and parity settings? Is it possible that the FPGA outputs open-drain and you haven't used a pull-up on your RX? (sorry, just shooting in the dark here with some possible causes for corruption - the IOIO currently silently discards frame-error bytes).

Stop bit and parity settings are correct.  I'll double check the UART TX pin of the FPGA, but I am fairly certain that's not the issue (no problem at the shot in the dark though, any thoughts are always helpful).  The thing that puzzles me really is that I am able to read and write data up until I start trying to move in to my non-printable/larger messages (though the non-printable characteristic and size of the messages is still in question as the cause I suppose).

 

Also not sure if this is relevant or not, but the host I am using, since it is connected to an FPGA, can be expected to be working very very near actual line speed.  Not sure if somehow if the host works right at line speed, if this could result in bytes getting missed if the clock on the IOIO is not spot on as well.

 

There should be some permissible slack there. 

For reference, I tried this on a Nexus One, Motorola XT720, and a Samsung Galaxy Tab with all the same outcome.  Much of the debug I collected for this was via logcat running on the Android device that I monitored via SSH over wifi (so I was able to see the sequence of everything as it was going on).

That sounds really interesting! Do you need to root the device for running sshd? Do you need to install it yourself, or does it come with the stock OS?

Rooting does seem to be a requirement to get a SSH server running on the device and I used a server implementation called dropbear (I followed something similar to http://www.droidforums.net/forum/droid-hacks/9038-ssh-daemon-dropbear-android-2-0-a.html).  I certainly have found this to be somewhat of a requirement for debugging since you tie up your USB connection while talking with the IOIO. :) 

 

Can you try to reproduce your problem with a loopback, and then sends a simple-as-possible program that reproduces it? I can then have a look.

I have had some trouble getting this to reproduce with the loopback, but I am going to do a little more digging to see how I can produce this failure in some more controlled way.

Thanks for the quick reply to my issue by the way, much appreciated!

[Eric42]

Hi,

Sorry shoot a dead target, but would it be possible that somehow the
ioio uses some kind of xon/xoff protocol?
I guess that on the FPGA side you just "send and forget". Could you
check if the ioio is sending something back while receiving your 4096
bytes data?

Eric

> something similar tohttp://www.droidforums.net/forum/droid-hacks/9038-ssh-daemon-dropbear...).

[Ytai Ben-Tsvi]

Nope. No such thing.
Can you check your IOIO against the USB-uart to eliminate the possibility of FPGA weirdness?

> --
> You received this message because you are subscribed to the Google Groups "ioio-users" group.

[Brandon Wuest]

So it turns out I was incorrect when I said I was having trouble replicating my issue using the loop back as I can reproduce the issue this way.  Also using the loop back I verified that the data contents have no bearing on when the failure occurs.

Below is some code that writes a block of data and reads it right back.  After some random number of iterations, I stop receiving the data I sent to myself.

This is based off the HelloIOIO sample project:

public static final int RX_PIN = 28;

public static final int TX_PIN = 29;

public static final int BAUD_RATE = 115200;

public static final Parity PARITY = Parity.NONE;

public static final StopBits STOP_BITS = StopBits.ONE;

public static final int BUFFER_SIZE = 512;

class IOIOThread extends AbstractIOIOActivity.IOIOThread {

/** The on-board LED. */

private DigitalOutput led_;

protected boolean ledOn = false;

protected Uart uart = null;

protected InputStream in = null;

protected OutputStream out = null;

protected int sampleCount = 0;

@Override

protected void setup() throws ConnectionLostException {

led_ = ioio_.openDigitalOutput(0, true);

uart = ioio_.openUart(RX_PIN, TX_PIN, BAUD_RATE, PARITY, STOP_BITS);

in = uart.getInputStream();

out = uart.getOutputStream();

}

@Override

protected void loop() throws ConnectionLostException {

byte [] outBuffer = new byte[BUFFER_SIZE];

byte [] inBuffer = new byte[BUFFER_SIZE];

// fill the outgoing buffer with some data

Arrays.fill(outBuffer, (byte)'A');

try {

System.out.println("Writing Data...");

// write out the data block

out.write(outBuffer);

int totalRead = 0;

int justRead = 0;

System.out.println("Reading Data...");

// read from the input stream until the 

do {

justRead = in.read(inBuffer, totalRead, inBuffer.length - totalRead);

if(justRead > 0) {

totalRead += justRead;

}

System.out.println("read data - justRead: " + justRead + ", totalRead: " + totalRead);

} while(totalRead < inBuffer.length && justRead > 0);

if(Arrays.equals(outBuffer, inBuffer)) {

System.out.println("[" + sampleCount + "] GOOD");

} else {

System.out.println("[" + sampleCount + "] BAD");

}

} catch(IOException ioe) {

System.out.println("[" + sampleCount + "] BAD");

ioe.printStackTrace();

}

++sampleCount;

// toggle the LED so we know things are going on

led_.write(!ledOn);

ledOn = !ledOn;

}

}

For this example, I am filling a buffer up with data, writing it, and then reading it back while verifying the contents are not corrupt.  I always see this code "hang" after the "Reading Data..." print statement.

Any thoughts?  

Is it possible the clock driving the UART is drifting wildly (that is my only shot in the dark at this point and I do not have another IOIO at the moment to try it out on)?

  

[Ytai Ben-Tsvi]

OK, we're getting there!

I've ran your program for 880 iterations with all "GOOD". So the program is good and the IOIO seems to behave well.

Possible problems:

1. Poor solder joints / connections on your pins.
2. Phone stutters for a few hundred milisec, resulting in lost data. That may have to do with garbage collection. Are you running pre-2.3 Android? What might help in this case (and in any case) is avoiding memory allocations inside loop(), for example, your buffers.

Let me know how it goes.

[Brandon Wuest]

OK so a little more information... my loop back test fails on my Motorola XT720 running a custom 2.2.1 rom, fails on my Samsung Galaxy Tab running factory 2.2.1, but does not seem to be failing on my Nexus One running factory 2.3.4.  For the two devices that failed, I ran the test with the declaration and allocation of the two buffers outside of the loop.  Also the solder joints look pretty solid on the board.

Is there something specific about 2.3 and above that works better with the IOIO board, even though all the communication is actually done over a socket?

Also I noticed that when I dug in to the IOIOlib, it looked like UART_DATA messages being handled inside of IOIOProtocol only ever contained 1 or 2 bytes of data at a time.  If pauses on the android app side seem to cause the connection / data flow problems, would it be possible/recommended to modify the IOIO firmware to have an increased TX buffer to help alleviate this?

[Vivat]

I can report a similar problem: I connected an Arduino board to the IOIO uart. This is just sending short text messages (50 characters) every 20 ms with 115200 baud. After receiving a few messages, the IOIO stops receiving data. The Log message "Client initiated disconnect" appears.
The strange thing is: It works when I add a DigitalOutput (e.g. for the onboard LED) and do a (dummy) write to this DigitalOutput after each message received. It seems that the connection between the Android Device and IOIO get somehow lost after a while when only data are received but no data are sent. However, if I try to increase the throughput (message with 100 char or a message every 10 ms), it stucks even with this dummy writes. I have an Archos 32 running Android 2.2 but it is the same with a SmartQ V3 running Android 2.1.

[Brandon Wuest]

I am still trying to work through this and have not had much luck with finding a good answer.  

One thing that I have found is that it seems if I increase to 2 stop bits, for at least the loop back test, my success rate is higher, but still not 100%.  I am starting to think if I want to overcome this issue, I really need to start looking at the firmware on the PIC to see where I might be able to tweak how data is getting sent out.

[trandi]

Just a quick remark regarding SSHD, which is indeed unbelievably handy when it comes to debugging or generally hacking your Android.
0.99£ can save you the trouble of manually installing the dropbear sshd, if you by the QuickSSHd app from the Android Market.
I've just done that and it works perfectly well.

dan
http://trandi.wordpress.com