New-to-LUFA questions

[Alan Burlison]

I'm looking at LUFA for use on some Minimus boards (ATmega32U2) and
have some questions that I haven't yet been able to find answers to,
hopefully someone here can help. To set the context of what areas I'm
looking at, I've ported my interrupt-driven avr-libc stdio USART
driver to the 32U2 so I can get basic printf-debugging working, I'd
like to do the same but over USB, which I assume means writing a
VirtualSerial driver to sit under avr-libc's stdio, I've found a
couple of threads related to that. The specific questions I have are:

1. What are the practical differences between the ClassDriver and
LowLevel examples? When would you choose the LowLevel interface over
the ClassDriver one?

2. Most of the example LUFA code seems to be built around a poll loop
that drives the LUFA library. Ideally I'd like to do all the USB
processing in ISRs as much as possible. It seems that at one point
there was an ISR-driven API for LUFA, but it was removed. Why was
that? There's also mention of it being reinstated, when will that
happen and what requirements will it have (use of timers etc). I see
that the Atmel documentation says "CONTROL endpoints should not be
managed by interrupts, but only by polling the status bits" (section
21.12)", is that the reason why it was removed? Is interrupt-driven
USB problematic on the 8-bit parts?

3. I found a thread says that LUFA allocates buffers for 16 endpoints
even on devices that only support 4, is that still the case? As I'll
need as much SRAM as possible for my eventual application, I need to
minimise LUFA's memory requirements.

If there are FAQs or web pages that answer these I'd grateful for
pointers. Thanks,

--
Alan Burlison
--

[Jim Paris]

Hey Alan,

I have an implementation of FTDI's USB-to-serial protocol for LUFA
that is pretty close to what you're asking. It emulates a FT232R.
I wrote it for a few reasons:

- I wanted something that was entirely interrupt-driven and did not
rely on any polling.

- I wanted software TX and RX queues that I could control easily, with
reliable blocking/nonblocking options for when stuff gets buffered
versus dropped in each direction.

- Existing CDC-ACM based virtual serial port implementations on
various microcontrollers are always unreliable or slow for me with
the Linux drivers. The ftdi_sio driver, on the other hand, is rock
solid.

Maybe you can use it as-is, or as an example. I just split it off
from another project, and I don't have hardware handy to test at the
moment, but it should work barring any stupid mistakes.

https://git.jim.sh/jim/lufa-ftdi.git

The atmega32u2 branch is set up for your Minimus board, with buffer
and endpoint sizes reduced accordingly to leave more free RAM.

-jim

[Alan Burlison]

On 01/10/2012 07:08, Jim Paris wrote:

> I have an implementation of FTDI's USB-to-serial protocol for LUFA
> that is pretty close to what you're asking.

I won't have time to look at this fully until this evening, but I'd just
like to say a *huge* thank-you - that, and you are freaking awesome :-)

--
Alan Burlison
--

[Alan Burlison]

On 01/10/2012 07:08, Jim Paris wrote:

> I have an implementation of FTDI's USB-to-serial protocol for LUFA
> that is pretty close to what you're asking. It emulates a FT232R.
> I wrote it for a few reasons:
>
> - I wanted something that was entirely interrupt-driven and did not
> rely on any polling.

Ditto.

> - I wanted software TX and RX queues that I could control easily, with
> reliable blocking/nonblocking options for when stuff gets buffered
> versus dropped in each direction.

Yep, that's more or less what my current USART implementation does. It
line-buffers on both input and output and does basic line discipline
processing - echoes input characters, does backspace handling and maps
\r & \n appropriately etc.

> - Existing CDC-ACM based virtual serial port implementations on
> various microcontrollers are always unreliable or slow for me with
> the Linux drivers. The ftdi_sio driver, on the other hand, is rock
> solid.

I haven't tried a simple CDC version yet and I'm not using Windows or
Linux, this is on Solaris ;-)

> Maybe you can use it as-is, or as an example. I just split it off
> from another project, and I don't have hardware handy to test at the
> moment, but it should work barring any stupid mistakes.

It enumerates OK and shows up as a device so the basics are working:

Oct 1 18:05:41 chill usba: [ID 912658 kern.info] USB 2.0 device
(usb403,6001) operating at full speed (USB 1.x) on USB 1.10 root hub:
device@1, usbftdi0 at bus address 2
Oct 1 18:05:41 chill usba: [ID 349649 kern.info] LUFA FT232
64033343837351113190
Oct 1 18:05:41 chill genunix: [ID 936769 kern.info] usbftdi0 is
/pci@0,0/pci108e,5351@1d,2/device@1
Oct 1 18:05:41 chill genunix: [ID 408114 kern.info]
/pci@0,0/pci108e,5351@1d,2/device@1 (usbftdi0) online

But when I open the device I get a timeout in the driver, so clearly
something is not quite right:

Oct 1 18:06:22 chill DESC: A USB device is not responding to host
tokens and the corresponding driver may not be able to recover from the
errors automatically.
Oct 1 18:06:23 chill DESC: A packet timeout was detected in the USB
device and the corresponding driver may not be able to recover from the
errors automatically.

But at least I have a working USART so I can stick in some debugging to
see what's going on -)

[Jim Paris]

Alan Burlison wrote:
> It enumerates OK and shows up as a device so the basics are working:
>
> Oct 1 18:05:41 chill usba: [ID 912658 kern.info] USB 2.0 device
> (usb403,6001) operating at full speed (USB 1.x) on USB 1.10 root
> hub: device@1, usbftdi0 at bus address 2
> Oct 1 18:05:41 chill usba: [ID 349649 kern.info] LUFA FT232
> 64033343837351113190
> Oct 1 18:05:41 chill genunix: [ID 936769 kern.info] usbftdi0 is
> /pci@0,0/pci108e,5351@1d,2/device@1
> Oct 1 18:05:41 chill genunix: [ID 408114 kern.info]
> /pci@0,0/pci108e,5351@1d,2/device@1 (usbftdi0) online
>
> But when I open the device I get a timeout in the driver, so clearly
> something is not quite right:
>
> Oct 1 18:06:22 chill DESC: A USB device is not responding to host
> tokens and the corresponding driver may not be able to recover from
> the errors automatically.
> Oct 1 18:06:23 chill DESC: A packet timeout was detected in the USB
> device and the corresponding driver may not be able to recover from
> the errors automatically.
>
> But at least I have a working USART so I can stick in some debugging
> to see what's going on -)

Hmm. I did get a chance to test my sample code on one of my boards,
and it works here. My guess is that there's some request that
solaris's FTDI driver is sending that my code just ignores or doesn't
respond to properly. The first thing to check might be to add a
"default:" to the switch in ftdi_control_request() and see what
bRequest is when you get there.

-jim

[Alan Burlison]

One other question: I see the USB_COM_vect ISR in your code, but
according to the LUFA documentation there's a
INTERRUPT_CONTROL_ENDPOINT #define that can be set to enable
interrupt-driven processing bu LUFA itself. Is there a reason you
rolled your own rather than using that?

[Alan Burlison]

Partly answering my own questions:

> 2. Most of the example LUFA code seems to be built around a poll loop
> that drives the LUFA library. Ideally I'd like to do all the USB
> processing in ISRs as much as possible.

There's a #define, INTERRUPT_CONTROL_ENDPOINT, that can be set to make
LUFA be interrupt driven.

--
Alan Burlison
--

[Jim Paris]

Alan Burlison wrote:
> One other question: I see the USB_COM_vect ISR in your code, but
> according to the LUFA documentation there's a
> INTERRUPT_CONTROL_ENDPOINT #define that can be set to enable
> interrupt-driven processing bu LUFA itself. Is there a reason you
> rolled your own rather than using that?

LUFA's USB_COM_vect implementation assumes that the interrupt was only
enabled for the control endpoint. Mine handles the control, TX, and
RX endpoints. LUFA's also reenables interrupts during the
USB_Device_ProcessControlRequest() call, which was an unnecessary
complication that would have required extra locking in my
ftdi_control_request function.

-jim

[Alan Burlison]

On 02/10/2012 01:51, Jim Paris wrote:

> LUFA's USB_COM_vect implementation assumes that the interrupt was only
> enabled for the control endpoint. Mine handles the control, TX, and
> RX endpoints. LUFA's also reenables interrupts during the
> USB_Device_ProcessControlRequest() call, which was an unnecessary
> complication that would have required extra locking in my
> ftdi_control_request function.

Ah, makes perfect sense. I'm finding your FTDI driver a really useful
example - thanks again :-)

--
Alan Burlison
--

[Alan Burlison]

On 1 October 2012 19:55, Jim Paris <j...@jtan.com> wrote:

> Hmm. I did get a chance to test my sample code on one of my boards,
> and it works here. My guess is that there's some request that
> solaris's FTDI driver is sending that my code just ignores or doesn't
> respond to properly. The first thing to check might be to add a
> "default:" to the switch in ftdi_control_request() and see what
> bRequest is when you get there.

I loaded this onto a Minimus board, connected it to a Linux Mint box
and connected to it with minicom. Although I got *something* coming
back it was all control chars - I tried all the available baud rates
but no joy, so something is clearly not quite right. I'll try
sticking in some debugging going to the USART and see if I can figure
out what the problem is.

--
Alan Burlison
--

[Jim Paris]

Alan Burlison wrote:
> I loaded this onto a Minimus board, connected it to a Linux Mint box
> and connected to it with minicom. Although I got *something* coming
> back it was all control chars - I tried all the available baud rates
> but no joy, so something is clearly not quite right. I'll try
> sticking in some debugging going to the USART and see if I can figure
> out what the problem is.

That's strange. The baudrate and other settings don't matter, since
it's USB. I don't have a 32u2 handy to test, but it's OK on a 32u4.
I'd be interested to hear what you find with the USART.

-jim

[László Monda]

I strongly suggest you to use screen instead of minicom. It worked so
much better for me. Probably it won't solve this problem of yours but
I think it's a useful advice in general.

http://superuser.com/questions/235588/new-serial-window-in-gnu-screen

> --
> You received this message because you are subscribed to the Google Groups "LUFA Library Support List" group.
> To post to this group, send email to lufa-s...@googlegroups.com.
> To unsubscribe from this group, send email to lufa-support...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/lufa-support?hl=en.
>



--
László Monda <http://monda.hu>

[Alan Burlison]

On 04/10/2012 14:17, L�szl� Monda wrote:

> I strongly suggest you to use screen instead of minicom. It worked so
> much better for me. Probably it won't solve this problem of yours but
> I think it's a useful advice in general.
>
> http://superuser.com/questions/235588/new-serial-window-in-gnu-screen

I'll give that a try, thanks.

--
Alan Burlison
--

[Alan Burlison]

On 03/10/2012 20:15, Jim Paris wrote:

> That's strange. The baudrate and other settings don't matter, since
> it's USB. I don't have a 32u2 handy to test, but it's OK on a 32u4.
> I'd be interested to hear what you find with the USART.

That's what I thought, that the baud rate shouldn't matter as it's not a
'real' serial interface, but I thought I'd check to be sure.

I've done some preliminary debugging with the USART and I see a stream
of control requests coming in when the device is enumerated, but no in
or out data requests at all. I also put a 'default: printf' in the
ftdi_control_request() switch statement as you suggested, but it wasn't
ever taken. I'm wondering if it might be some sort of memory issue, the
32U4 has 2.5k SRAM, the 32U2 only has 1k.

--
Alan Burlison
--

[Jim Paris]

Alan Burlison wrote:
> On 03/10/2012 20:15, Jim Paris wrote:
>
> >That's strange. The baudrate and other settings don't matter, since
> >it's USB. I don't have a 32u2 handy to test, but it's OK on a 32u4.
> >I'd be interested to hear what you find with the USART.
>
> That's what I thought, that the baud rate shouldn't matter as it's
> not a 'real' serial interface, but I thought I'd check to be sure.
>
> I've done some preliminary debugging with the USART and I see a
> stream of control requests coming in when the device is enumerated,
> but no in or out data requests at all.

Maybe there's some difference in the 32u2 USB interrupt structure that
I'm not taking into account, or some issue with which endpoints are
available and their sizes/types?

> I also put a 'default:
> printf' in the ftdi_control_request() switch statement as you
> suggested, but it wasn't ever taken. I'm wondering if it might be
> some sort of memory issue, the 32U4 has 2.5k SRAM, the 32U2 only has
> 1k.

I don't think so -- the maximum stack usage at runtime is only 90
bytes, so there should be at least 0.6k free on the 32u2. See the
stack-debug branch for how I measure stack usage.

-jim

[Jim Paris]

Jim Paris wrote:
> Maybe there's some difference in the 32u2 USB interrupt structure that
> I'm not taking into account, or some issue with which endpoints are
> available and their sizes/types?

I looked into it a bit and I still don't see any problems there. Maybe
it's a compiler issue? Here's a hex file (built from bb82d58 for the
minimus) you could try:
http://jim.sh/~jim/tmp/lufa-ftdi-minimus.hex

-jim

[Alan Burlison]

> Maybe there's some difference in the 32u2 USB interrupt structure that
> I'm not taking into account, or some issue with which endpoints are
> available and their sizes/types?

I've had a look at the datasheets for both both the U2 and U4 and I
can't spot anything that looks pertinent...

>> I also put a 'default:
>> printf' in the ftdi_control_request() switch statement as you
>> suggested, but it wasn't ever taken. I'm wondering if it might be
>> some sort of memory issue, the 32U4 has 2.5k SRAM, the 32U2 only has
>> 1k.
>
> I don't think so -- the maximum stack usage at runtime is only 90
> bytes, so there should be at least 0.6k free on the 32u2. See the
> stack-debug branch for how I measure stack usage.

OK, was just a thought - thanks.

--
Alan Burlison
--

[Alan Burlison]

> I looked into it a bit and I still don't see any problems there. Maybe
> it's a compiler issue? Here's a hex file (built from bb82d58 for the
> minimus) you could try:
> http://jim.sh/~jim/tmp/lufa-ftdi-minimus.hex

I've tried that image and the behaviour is just the same, so I don't
think it is a compiler issue. I'll keep digging and try some more
USART debugging but it's delicate work, as if I add in too much it
affects the USB timings - if I fill the USART buffers with output it
starts blocking and that throws the USB timings out enough for the
host to fault the connection.

I'm also wondering if switching back to polled handling might help
narrow things down?

--
Alan Burlison
--

[Alan Burlison]

On 04/10/2012 19:26, Jim Paris wrote:

>> I've done some preliminary debugging with the USART and I see a
>> stream of control requests coming in when the device is enumerated,
>> but no in or out data requests at all.
>
> Maybe there's some difference in the 32u2 USB interrupt structure that
> I'm not taking into account, or some issue with which endpoints are
> available and their sizes/types?

I've done some more debugging and I'm definitely only ever seeing
control endpoint interrupts, both during enumeration and when the device
is opened. I've tried one of the demo VirtualSerial examples and that
works fine, so I'm pretty sure my toolchain & libraries are OK.

One question: the virtual serial demos have 4 endpoints, control, in,
out and notification, yours only has control, in and out. Does FTDI not
require a notification endpoint?

--
Alan Burlison
--

[Jim Paris]

Alan Burlison wrote:
> > http://jim.sh/~jim/tmp/lufa-ftdi-minimus.hex
>
> I've tried that image and the behaviour is just the same, so I don't
> think it is a compiler issue. I'll keep digging and try some more
> USART debugging but it's delicate work, as if I add in too much it
> affects the USB timings - if I fill the USART buffers with output it
> starts blocking and that throws the USB timings out enough for the
> host to fault the connection.

It makes no sense that you'd see garbage data. Where would it be
coming from, if the chip's endpoint interrupts are never running?
You're not running Linux in a VM that could be messing with things,
are you?

You might also get some useful debugging by capturing the USB traffic
on the PC side:
http://wiki.wireshark.org/CaptureSetup/USB#Linux

Similarly the full dmesg output, `lsusb -vv`, etc can help. Does
a real FT232R work?

> I'm also wondering if switching back to polled handling might help
> narrow things down?

I don't have a version of that code that does polling, but you could
try running the USB_COM_vect code in a loop (make sure you wrap it in
cli() and sei(), since it expects to run with interrupts disabled).

> One question: the virtual serial demos have 4 endpoints, control,
> in, out and notification, yours only has control, in and out. Does
> FTDI not require a notification endpoint?

It's just the 3, FTDI doesn't use an extra endpoint. The port status
and other data are returned in the first 2 bytes of every data packet.

-jim

[Alan Burlison]

On 05/10/2012 21:06, Jim Paris wrote:

> It makes no sense that you'd see garbage data. Where would it be
> coming from, if the chip's endpoint interrupts are never running?
> You're not running Linux in a VM that could be messing with things,
> are you?

Nope, it was a on a bare-metal setup, running the Debian version of
Linux Mint.

> You might also get some useful debugging by capturing the USB traffic
> on the PC side:
> http://wiki.wireshark.org/CaptureSetup/USB#Linux
>
> Similarly the full dmesg output, `lsusb -vv`, etc can help. Does
> a real FT232R work?

Don't forget I'm mostly doing this on Solaris. I've had a look a the
Solaris FTDI driver and it looks like it can be compiled with debugging
support, I'll try getting hold of one of our USB folks and ask them
about how to set that up.

I can also try on another Linux machine, I'll try the suggestion of
using 'screen' rather than 'minicom'.

And yes, a real FT232R works fine, that's what I'm using to connect to
the USART for debugging. I've also tried the USB-to-USART demo that
comes with the LUFA source, that works fine as well.

>> I'm also wondering if switching back to polled handling might help
>> narrow things down?
>
> I don't have a version of that code that does polling, but you could
> try running the USB_COM_vect code in a loop (make sure you wrap it in
> cli() and sei(), since it expects to run with interrupts disabled).

OK, I'll try that as well.

>> One question: the virtual serial demos have 4 endpoints, control,
>> in, out and notification, yours only has control, in and out. Does
>> FTDI not require a notification endpoint?
>
> It's just the 3, FTDI doesn't use an extra endpoint. The port status
> and other data are returned in the first 2 bytes of every data packet.

Ah right, thanks. Was just wondering :-)

--
Alan Burlison
--

[Alan Burlison]

On 05/10/2012 23:15, Alan Burlison wrote:

> And yes, a real FT232R works fine, that's what I'm using to connect to
> the USART for debugging. I've also tried the USB-to-USART demo that
> comes with the LUFA source, that works fine as well.

I've dumped out the Solaris USB trace log from inside the kernel
debugger when the device is opened, and it's just timing out, which
exactly matches what I see at the 32U2 end. The control endpoint
packets are successfully triggering the ISR in the atmega code, but the
in & out endpoint ones aren't, so it looks like it's almost definitely
the interrupt setup that is the problem.

I've also dumped out the in-kernel USB device descriptors of the atmega
version and of a real FTDI device, and all the parts that matter seem
the same - the only difference is the wmaxPacketSize in the in/out
endpoint descriptors - 16 for the atmega, 64 for the real FTDI, but I
can't see how that should make a difference.

I'll try the polled alternative tomorrow, if that works then it should
confirm that it's the interrupt handling that is the problem.

--
Alan Burlison
--

[Jim Paris]

Hey Alan,

I finally found my atmega8u2 board to test this, and I've fixed the bug.

The problem is that the *u2 doesn't support dual-bank (ping-pong) mode
on endpoints 1 and 2, so they were never getting configured. The
solution is to either switch the TX and RX endpoints to 3 and 4, or
use the change I just pushed to the repo, which tries using dual banks
first, and falls back to a single bank if that fails.

-jim

[Alan Burlison]

Hi Jim,

> I finally found my atmega8u2 board to test this, and I've fixed the bug.
>
> The problem is that the *u2 doesn't support dual-bank (ping-pong) mode
> on endpoints 1 and 2, so they were never getting configured. The
> solution is to either switch the TX and RX endpoints to 3 and 4, or
> use the change I just pushed to the repo, which tries using dual banks
> first, and falls back to a single bank if that fails.

Thanks for this, I've tested the fix and it works for me also, although
I'm getting some control characters echoed back:

----------
You sent 97 (a)
�You sent 98 (b)
�You sent 99 (c)�
----------

I haven't had a chance to dig in any further but it looks like it might
be a buffer length issue. I'd seen mention of the dual-bank stuff in
the Atmel docs but hadn't paid it much heed, I'll go read up on it :-)

Thank-you for all your help on this, it's much appreciated!

--
Alan Burlison
--

[Jim Paris]

Alan Burlison wrote:
> Hi Jim,
>
> >I finally found my atmega8u2 board to test this, and I've fixed the bug.
> >
> >The problem is that the *u2 doesn't support dual-bank (ping-pong) mode
> >on endpoints 1 and 2, so they were never getting configured. The
> >solution is to either switch the TX and RX endpoints to 3 and 4, or
> >use the change I just pushed to the repo, which tries using dual banks
> >first, and falls back to a single bank if that fails.
>
> Thanks for this, I've tested the fix and it works for me also,
> although I'm getting some control characters echoed back:
>
> ----------
> You sent 97 (a)
> �You sent 98 (b)
> �You sent 99 (c)�
> ----------

I'm not seeing that on the at90usb1287, atmega32u4, or atmega8u2 and
Linux. I tried kermit, minicom, and modified terminal.py to show
unprintable characters and nothing's there.

Is this on the Solaris box? Maybe something's wrong with the format
of the status bytes or something, and Solaris is less forgiving.

-jim

> I haven't had a chance to dig in any further but it looks like it
> might be a buffer length issue. I'd seen mention of the dual-bank
> stuff in the Atmel docs but hadn't paid it much heed, I'll go read
> up on it :-)
>
> Thank-you for all your help on this, it's much appreciated!
>
> --
> Alan Burlison
> --
>

[Alan Burlison]

On 12/10/2012 19:32, Jim Paris wrote:

> I'm not seeing that on the at90usb1287, atmega32u4, or atmega8u2 and
> Linux. I tried kermit, minicom, and modified terminal.py to show
> unprintable characters and nothing's there.
>
> Is this on the Solaris box? Maybe something's wrong with the format
> of the status bytes or something, and Solaris is less forgiving.

I see it on Solaris using both screen and tip. The Solaris ftdi driver
is layered over the virtual serial driver, so it may well be a bit more
picky. I'll see if I can figure it out. A 'real' FTDI works fine on
Solaris, so there must be something different about the LUFA one.

--
Alan Burlison
--