Hardware: Connect 16u2 to 328 via SPI/ICSP rather than Serial in further releases. Possible?

[何苗]

Hi, I'm just thinking of this. Many arduinos come with two processors, one 16u2/8u2 and one main processor, be it 1280, 2560 or 328. It has been said that 16u2 is there as a USB serial adaptor to assist downloading of sketch into the 328, but I discovered that it is still using the bootloader-stk500 protocol. Why??

I do think connecting 16u2 to 328 upon SPI line is a better choice. Some reasons for this.
1) SPI lines on the 16u2 are not used anyway.
2) We can definitely emulate any of the programmers out there using the 16u2. If MKii/debugWire is possible, we can plug in an Arduino for AVR Studio to program and debug. Fuse setting will also be possible using this.
3) We save space by scraping the redundant boot loader on main processor. And there's no boot loading timeout at startup, since the 16u2 handles all the ISP programming that does not require a boot loader setting.
4) Use only one ICSP port as they are interconnected. And we just need to toggle the RESET line on either processor to determine whether this port is used for ISPing the 16u2, 328 or just pure SPI communication. Another jumper maybe, to toggle the RESET pin to any GPIO on 16u2.
5) We can get rid of the ArduinoISP thing. Uploading program to another Arduino or any board made easy be simply connecting one ICSP header for another. (while the programmer Arduino is set with ICSP RST line attached to 16u2's SS or whatever.) This can be done much easier than the past, which we are required to remove the micro controller to do that.
6) At the same time, still acts as a serial/whatever adaptor, as the firmware can serve concurrent functions.
7) Bootloading main processor via USB. There's no need to external programmers. However, this is only possible if we want to revert to legacy scheme (due to some reasons), or intend to prepare a removable processor for external use.
 While flashing, other devices can be attached as well, as SPI is a bus (unlike Serial, that Serial 0 needs to have no devices attached in order to upload program).

Looks quite feasible as well. Anyone can give more suggestions on how to implement this?
Thanks!:)

[Rich Obermeyer]

It is a good idea.IMHO  I already implemented it a year ago and it works great.  Programming the 328 with SPI is way faster, more reliable and you never have the sync errors on the serial port since you are pulling the reset of the application CPU on demand to program your code.  The extra bootloader code space is great.  No longer have to preprogram the bootloader into every chip.  Customers can't accidentally break the bootloader or program fuses wrong.

The down side is if you are using the 16u2 in SPI mode you can not use it as a serial link for debug messages without some programming magic.  Any debug text you type in could be interpreted as a programmer command and hang the serial port.  I have not needed it but I wanted you to know its there.  Maybe somebody knows how to block that.

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Jimmy He]

Hi Rich,

I'm not familiar with the USB protocol nor the way of implementing it on the 16u2. However, is it possible to implement it in any other way than UARTing to the 16u2 for programming? I've seen this Arduino_Board_Programmer on GitHub, basically an independent Avrdude implementation on an Arduino, that autodetects the target board, then sets preloaded fuse and commits firmware pages from the corresponding one it preloaded. Which gave me an idea then. Can we simply setup the 16u2 as an on-chip avrdude, and all IDE does is streaming hex to it (through other USB 2.0 means, not confined by the stupid Baud rate thing). Then maybe we can free up the serial on 16u2 for debugging purposes.

By the way, did you manage to emulate DebugWire on the same set chip? If one can ever simulate everything possibly available on an official AVR programmer on the little 16u2, it comes quite amazing that we can run debug even with AVR studio just with single arduino plugged in.

And do you think Arduino Team will consider it (or would already have considered it when they brought in the 8u2) if we propose this design to them? I found it hard to change unless we personally have the ability to make a PCB, surface mount the 16u2 and wiring up the SPI lines. But really, it seems like a useful design.

Thank you very much for your suggestion! :)

[Rich Obermeyer]

The USB protocol is not a real concern.  It does what you tell it and it's way faster than any serial port.  I doubt you want to get involved in the Avrdude issues.  Let the IDE match the chip you want to program and generate the hex file using the board.txt files.  Then the 16U2 is only a programming.

What is the benefit of over designing the programmer when the IDE already works?

To use the IDE for debug you would be using the same serial port as the programmer.  If there is a way of decoupling the two then debugging can be done with another virtual serial port.

The 16u2 can be used as one programmer serial and one debug port then.  I could not figure out how to get there myself but I had limited time.

Maybe you can be the one!

No debug wire for me but that sounds like a good idea too if you can get the IDE to work with it.

I hear that complaint all the time.  Debug messages is not a substitute for hardware debug.

[Nevada Smith]

Hi!
I am working on a new 16u2 Bootloader that supports HID usage. I am so close to get it work 100%. But the Proof of concept works. I can hook up a button to the spi headers and use them to press an 'a' key on my pc. And i am still able to upload sketches. Further more i will try to solder the 4 Pins for even more Hardware Ports on the 16u2. I will try to get SPI working and use this for additional Leds (because most shields use spi and this doesnt work with leds).

To your question: If i understood it right, you want to have a special Debug Serial Port and a Normal Programming Port? I am not sure if you can have 2 serials and i am also not sure why one would do that. So you want to use the spi headers (both headers connected together) to programm the uno/mega and still have the serial port??

I dont know if this work is really worth it, or if i understood it right. But thinking about it seems that it could be a nice idea to have a cheap icsp programmer and still a free serial on the uno/mega. I will try to work on it when the HID thing is finished.

Uhm do you know how much space the 8u2 (8!! not 16) has? On Darrans blog i read that his firmware is close to 4096kb but i think the 8u2 has 8kb? And the 16u2 even more, so we should be fine with this on new arduino.

[Matthijs Kooijman]

Hey folks,

> It is a good idea.IMHO

Yeah, I like this idea as well. I wonder if the Arduino dev team
considered this approach for the original Arduino designs?

I guess the bootloader approach might be chosen since it is simpler, but
now that we're using a 16u2 on most Arduino boards and the Arduino
platform has matured, this might be a great way to make things more
powerful, at little extra cost.

> I already implemented it a year ago and it works great.

Nice! Did you share any of your code somewhere?

> The down side is if you are using the 16u2 in SPI mode you can not use
> it as a serial link for debug messages without some programming magic.
> Any debug text you type in could be interpreted as a programmer
> command and hang the serial port. I have not needed it but I wanted
> you to know its there. Maybe somebody knows how to block that.

This is indeed something that would need to be solved. I can think of a
few solutions:
- Only accept programming commands when a specific baud rate is
configred, like the 1200 baud reset trick for the Leonardo. Since
the configured baud rate only affects the serial link between the
16u2 and the 328, which is not used in programmer mode, this "magic
baud rate" can be an arbitrary low baud rate, since its value is not
actually used.
- In another reply, Nico suggests that you could perhaps have two
ports: a serial communication port and a programmer (and perhaps also
debugwire?) port. This is probably the cleanest solution, though it
requires changes to the IDE (which now assumes it can use the same
serial port for both serial monitor and programming.)
- A timeout could be used, so that programming commands are only
accepted if one is received within a few seconds. This still creates
problems if the serial port is used for transmitting (binary) data
that looks like a programmer command. It also again introduces
possible timing problems, so this is probably not a very good
approach.
- Use a native USB protocol for programming. i.e. instead of using a
virtual serial port, send data through custom USB messages. On the PC
side, this would need support from avrdude, but perhaps we can mimic
the protocol of another similar device (the stk600 seems to use
native USB, for example) so no changes to avrdude are needed.

The downside of this approach is that there might be permission
issues. On linux, permission to all (virtual) serial ports is
typically managed using a single "dialout" group, but permission to
a native USB device typically needs custom udev rules, which quickly
becomes more complicated for users. Not sure how OSX and Windows
handle this.


On a related note: If this is implemented, the 16u2 should probably
double-check the fuse settings to prevent disabling the ISP programming
fuse (which needs a high voltage programmer to re-enable it).

Gr.

Matthijs

[Nevada Smith]

For my HID Keyboard thing i need to filter out HID commands from the Serial stream. I am using my own Protocol to filter special commands out for the Serial stream. This made me very mad the last days but i got it working yesterday and improved this morning. So its kinda a "man in the middle".

So what you could do is to create a special Arduino library that makes use of the Protocol and writes every Serial debug Command Packed into the Protocol and the 16u2 then sends it to another 2nd debug serial (I will do that for the keyboard/joystick things too). The other way around you can also write the debug infos from pc as Protocol to the uno but i think this isnt necessary because if you programm the Uno/Mega it will reset anyways.

Changing the baud is very complicated, i want to mention here. There is no real start/end mark for Uploading sketches (at least ive no idea how i could detect an end/beginning). So what i do is to start the 16u2 always with baud 115200. This is the programming baud and also the fastest. Thats needed for joystick recognition because its quite alot of data with the Protocol around. Not sure if a second debug Seria would solve it.  But it would be confusing for other people too.

[Jimmy He]

Hey Matthijs and all,

Really nice ideas on the native usb port! *Clap clap* I've been a super-duper newbie here and I've never dipped my finger into these low-level things. Once my crazy idea about this was even "Simulate a mass-storage device or so, and simply streaming the hex file in for programming" (if ever we can map the target programming memory for a m-s stack though)

I just noticed that the serial programming method is not even a "in-circuit programming protocol". It affects integrity of circuit as we must remove anything connected on UART0. This is annoying especially when you got a bee or something attached. ICSP, I would say, is faster and more capable, but we do need Arduino core team's attention so maybe a different wiring between the two MCUs can be applied in any further Arduino releases which makes this possible? (ICSP needs extra SPI lines attached between the 8u2 and 328)

Dual serial mode hmm, once thought of it also, but is it worth doing? I mean, if we keep ourselves confined within the baud limit then the capability of USB may be limited.

One concern though was about firmware size. If we need any more than a stack to achieve native USB, plus buffered communication for UART, RAM management is crucial. We also need to watch the use of the 16-kb flash. Serial emulator + DebugWire handler + SPI ISP handler + DFU is an almost insane combination.

Thank you very much for your attention, Geniuses! I appreciate very much that I can learn something from you. :)

[Matthijs Kooijman]

Hey Jimmy,

> Dual serial mode hmm, once thought of it also, but is it worth doing?
> I mean, if we keep ourselves confined within the baud limit then the
> capability of USB may be limited.

Uh, I think you're misunderstanding here. A virtual serial port isn't
limited to any particular baud rate. When you set a baud rate on
ttyACM0, that baud rate is only used for the actual, physical serial
connection between the 16u2 and the 328. The communication between the
PC and the 16u2 just happens through USB packets and can thus be faster.

If there is no actual serial port involved (like with a second virtual
serial port, whose incoming messages are interprted on the 16u2 itself),
the baud rate configured is insignificant and does not limit the speed.
At least this is how I understand things.

> One concern though was about firmware size. If we need any more than a
> stack to achieve native USB, plus buffered communication for UART, RAM
> management is crucial. We also need to watch the use of the 16-kb
> flash. Serial emulator + DebugWire handler + SPI ISP handler + DFU is
> an almost insane combination.

That's definately something to keep into account, though someone should
probably just try to be sure what's possible (I probably won't have time
for this, though).

Gr.

Matthijs

[Tom Igoe]

You could also use a Leonardo instead of an Uno, and use the mouse/keyboard libraries. That way you get the serial on two endpoints and the mouse/keyboard on the other two. Seems like a better tool for the job you’re trying to accomplish.

t.

[Nevada Smith]

Ive used 80% of the 8u2's space it sais. But the 16u2 has double space. The ISP sketch uses 7kb. should be fine for the 16u2. I think everyone now has a Uno R3, shouldnt be the problem. Sure the developers need to wire this thing, or another Seller needs to produce new Unos. But first someone has to develop the software. And i also think its not that bad to get a few jumpers and connect them from one header to the other. People who would use this solution should know what they are doing, so the cable thing is not that problem but an idea for further develop. (if anyone can get this running). I will try to work on this if i finished the hid thing. I also wanted to add Led support so i have to use spi anyways :)

For Serial speed: This only is important if you want to use the man in the middle. Because programming the uno leaves the baud at a special rate (115200). And if i cant detect the ending of Protgramming it will stay like this. So (sorry that i only talk about the hid thing, but thats what i expirienced) if the man in the middle wants to detect any debug message or hid info it needs to be send with this speed. And the 16u2 also needs to boot with that speed. (normally it doesnt). Or you could use hardware to set a special message like "hey its debug mode" and connect wires to the spi headers or solder the 4 optional pins. OR you can use a softserial. But due to the fact that we are in bare chip programming its not that easy to use just a library for that. There is no libraries available and also no Arduino stuff like delay() or any other basic. For me thats too much, so i'd need help then.

But do we want a working complex solution or a more easy to use solution for everybody to understand? I think we have to deal with these problems and one thing that would be acceptable for me is to connect 6 spi wires but nothing more. Maybe some leds for a little debug output while programming but thats it.

For the memory device: I think this wont work becaus the hex might be too big, and also very complicated.

And no i wont use a Leonardo. I got it working there and i love the pro micro but:
The cdc serial of the leonardo is slow, too slow for ambilight infos via cdc serial
the leonardo has less program space (okay but a bit more ram). This matters for the usb host shield. I want to connect the wiiu pro controller and emulate a joystick. This wont work with the leonardo.
You could use uno + leonardo as well but why not use the 16u2 if its given? Its cheaper and not that fat.
On top of that most people has the uno and wish to use it as keyboard. There is a solution out there but without programming support than. Thats a struggle to debug programm then.
This would make the uno and especially the mega even more powerfull then ever before.

So i see no reason why i should stop developing this :P Let me do the hard part ;D

[Rich Obermeyer]

The code I used was from Randall Bohn.

https://github.com/rsbohn/arduinoisp

I did not put it in a 16U2 but included it in another processor I already had running with USB stack.  (Microchip)

It was easy to do with the exception that the byte/word definitions of Arduino are not the same as Microchip.

I did not include any LEDs (which are included in the original source) because of pin limits.  I think that would be a good idea on a 16U2 UNO board spin.

I think the chances of getting the IDE changed to support 2 serial ports on 16U2 while I am still programming is small. Therefore I assume we are stuck with it for now. 

Matthijs had some good thought on it.  Thanks for sharing.

I think its a good idea with using the baud rate change to turn on and off the programming function since its virtual anyway.  115200 is programmer, anything else is cheap serial port access for debug.

Timeout is very bad idea for dozens of reasons.  The most important is it never works......:-) 

Custom USB is also a bad idea.  Too much work and platform dependence.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlN5xcMACgkQz0nQ5oovr7zDVwCgl5ycBEhJfRhadkHh8cdVTj2q
dPsAoMW8/m5hSbNxu7hYHHQmIHycNXTG
=aY7Q
-----END PGP SIGNATURE-----

--
Rich

[Cristian Maglie]

In data lunedì 19 maggio 2014 18:48:42, Rich Obermeyer ha scritto:
> I think the chances of getting the IDE changed to support 2 serial ports on
> 16U2 while I am still programming is small. Therefore I assume we are stuck
> with it for now.

Besides the IDE, I don't think is possible to have 2 virtual serial ports, if
I remember well the 16u2 has not enough usb endpoints to do that.

> I think its a good idea with using the baud rate change to turn on and off
> the programming function since its virtual anyway. 115200 is programmer,
> anything else is cheap serial port access for debug.

This looks the most promising option, but I suggest to use 1200 bps to select
the "programmer mode" since is a rarely used speed, while 115200 is actually
used on a lot of projects.

About the chances of being adopted by Arduino IDE: hard to say only from the
premises. From my experience is very hard to get the upload/booloader right,
you have to face all the OS problems of the world, and when USB comes into
play its even harder.

About the chances of changing the hardware of the Uno is the Arduino Team that
have the final word on that. What I can say is that modifying a well
established hardware board like the Uno R3 is something to not take lightly,
even changing a single resistor may introduce issues and have subtle side
effects on a lot of existing projects.

C

Cristian

[Matthijs Kooijman]

Hey Christian,

> Besides the IDE, I don't think is possible to have 2 virtual serial ports, if
> I remember well the 16u2 has not enough usb endpoints to do that.

Ah, that seems plausible.

> > I think its a good idea with using the baud rate change to turn on and off
> > the programming function since its virtual anyway. 115200 is programmer,
> > anything else is cheap serial port access for debug.
>
> This looks the most promising option, but I suggest to use 1200 bps to select
> the "programmer mode" since is a rarely used speed, while 115200 is actually
> used on a lot of projects.

I wonder if using the same speed as the leonardo reset toggle uses is
a good idea. It _should not_ matter I guess, but it somehow it feels
like it might come back to bite later. Using a low, rarely used speed is
definately the way to go, though.

> About the chances of being adopted by Arduino IDE: hard to say only from the
> premises. From my experience is very hard to get the upload/booloader right,
> you have to face all the OS problems of the world, and when USB comes into
> play its even harder.

I think that, if the magic-baud-rate approach was implemented, the IDE
doesn't actually need any changes at all currently? It does need a new
definition in boards.txt, I don't think we could somehow transparently
make the same definition work with both the bootloader-style upload and
the ISP-programmer-style upload.

> About the chances of changing the hardware of the Uno is the Arduino Team that
> have the final word on that. What I can say is that modifying a well
> established hardware board like the Uno R3 is something to not take lightly,
> even changing a single resistor may introduce issues and have subtle side
> effects on a lot of existing projects.

Due to the above, this is likely a good option for new boards, but not
so much as a change to existing boards.

I do like this idea a lot, I'd be thrilled if it gets implemented.

Gr.

Matthijs

[Rich Obermeyer]

I am not understanding the programming mode at 1200bps.

I always see the programmer gets reprogrammed to 115200 before a program (upload)..

The debug is programmable in the sketch to pretty much anything you want and since its virtual who really cares.

Help me understand 

"while 115200 is actually used on a lot of projects."

Rich

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlN6TIUACgkQz0nQ5oovr7x8egCfWE+OckLYwvyKnQWTqKupu5r2
5A8AoIn9ev5+sc2G4cf3CXeAVHqSQ67/
=KsDm
-----END PGP SIGNATURE-----

--
Rich

[Matthijs Kooijman]

Hey Rich,

On Mon, May 19, 2014 at 11:39:46AM -0700, Rich Obermeyer wrote:
> I am not understanding the programming mode at 1200bps.
> I always see the programmer gets reprogrammed to 115200 before a program
> (upload)..

That's true, but the 115200 is also configurable in boards.txt in the
IDE.

> The debug is programmable in the sketch to pretty much anything you want
> and since its virtual who really cares.

The actual baudrate does affect the transfer speed between the 16u2 and
the 328 (or whatever), so it does matter what you set it to.

> Help me understand "while 115200 is actually used on a lot of
> projects."

A lot of sketches and libraries default to using 115200 baud when
doing debug prints and other communication.

The baudrate we pick for enabling "programmer mode" is not available for
communication with the 328, since at that baudrate, the 16u2 will
interpret any messages itself instead of forwarding them to the 328.
Hence, it is not a good idea to use a commonly used baudrate like
115200, since that will cause sketches to fail, while people do not
realize what's wrong.

Gr.

Matthijs

[Rich Obermeyer]

>Hey Rich,

>On Mon, May 19, 2014 at 11:39:46AM -0700, Rich Obermeyer wrote:
>> I am not understanding the programming mode at 1200bps.
>> I always see the programmer gets reprogrammed to 115200 before a program
>> (upload)..

>That's true, but the 115200 is also configurable in boards.txt in the
>IDE.

Since this is default now I would not want to change it for no good reason.  I see no good reason.

To be successful I would only use the defaults now.

>> The debug is programmable in the sketch to pretty much anything you want
>> and since its virtual who really cares.

>The actual baudrate does affect the transfer speed between the 16u2 and
>the 328 (or whatever), so it does matter what you set it to.

You are thinking of the DEBUG right?  Programming is not effected by baud rate.  I always set debug to 230k.

>> Help me understand "while 115200 is actually used on a lot of
>> projects."

>A lot of sketches and libraries default to using 115200 baud when
>doing debug prints and other communication.

See below.

>The baudrate we pick for enabling "programmer mode" is not available for
>communication with the 328, since at that baudrate, the 16u2 will
>interpret any messages itself instead of forwarding them to the 328.
>Hence, it is not a good idea to use a commonly used baudrate like
>115200, since that will cause sketches to fail, while people do not
>realize what's wrong.

I disagree with that thinking.  Since the default programming baud rate is 115200 now it couldn't be available for using with any sketch.  Do you have a sample where this is the case?

Make the debug run at 230k if it conflicts..

Some how the programmer would need to know what the speed is for programming.  If it gets changed a lot then I don't know how to make it work right now since the 16U2 code does not have access to that data.  Or does it?

Maybe all this mix and match configurability is not a good thing.

>Gr.

>Matthijs

This is still a good idea.  Just need to work out the kinks.  The code should be upgraded to stk600 to support the larger flash sizes in some cases.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlN6VBkACgkQz0nQ5oovr7xNvQCgjpzQX+FGUTZTPaL26TKL5VHn
i0QAnitzFhAimjBqr51WlKvmeWSIJYK+
=WTdx
-----END PGP SIGNATURE-----

--
Rich

[Matthijs Kooijman]

Hey Rich,

> >That's true, but the 115200 is also configurable in boards.txt in the
> >IDE.
> Since this is default now I would not want to change it for no good
> reason. I see no good reason. To be successful I would only use the
> defaults now.

If the programmer mode is triggered by 115200 baud, then we could
probably make it work without changing board.txt. However, since
hardware changes are required for this, this will probably result in a
new boards.txt entry anyway, so changing the baud rate is not a problem.
But mostly, using 115200 for the programmer mode prevents debug printing
from using it, which I consider a show-stopper.

> >> The debug is programmable in the sketch to pretty much anything you want
> >> and since its virtual who really cares.
> >The actual baudrate does affect the transfer speed between the 16u2 and
> >the 328 (or whatever), so it does matter what you set it to.
> You are thinking of the DEBUG right? Programming is not effected by baud
> rate. I always set debug to 230k.

I'm not sure what you mean here. I'm talking about both the current
bootloader-style programming as well as debug printing and other
communication with the sketch. Programming must now happen at 115200,
since the bootloader configures the UART at 115200. Debug printing can
happen at any speed as long as the sketch and serial monitor agree on
the speed to use. This includes 115200, since the baud rate is not
currently used to switch between programming (communication with the
bootloader) and debug (communication with the sketch).

> >The baudrate we pick for enabling "programmer mode" is not available for
> >communication with the 328, since at that baudrate, the 16u2 will
> >interpret any messages itself instead of forwarding them to the 328.
> >Hence, it is not a good idea to use a commonly used baudrate like
> >115200, since that will cause sketches to fail, while people do not
> >realize what's wrong.
> I disagree with that thinking. Since the default programming baud rate is
> 115200 now it couldn't be available for using with any sketch. Do you have
> a sample where this is the case?

All baud rates are available for a sketch. To enter programming mode,
the PC opens the serial port, which resets the 328 and enters the
bootloader. If a programming command is sent within the first few
seconds, programming mode is entered. Otherwise, the skech is started
and programming mode is not entered.

It seems the examples at arduino.cc are not using 115200, but I'm using
115200 every day, on all my different Arduinos, without problems.

> Some how the programmer would need to know what the speed is for
> programming. If it gets changed a lot then I don't know how to make it
> work right now since the 16U2 code does not have access to that data. Or
> does it?
> Maybe all this mix and match configurability is not a good thing.

I'm not sure what you're saying here.

Gr.

Matthijs

[Rich Obermeyer]

I guess there is no hope then.

You all will have to continue doing it the slow way.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlN6gpkACgkQz0nQ5oovr7w1cQCgn4EXngcQsp7TesYQGiYg7gKI
QHkAmgJHQuQYGV/UaJv/85r4zRxtTy15
=fppc
-----END PGP SIGNATURE-----

--
Rich

[Jimmy He]

Hi,

Folks, I see no point of discussing about tricking the serial line into debug mode. As Rich suggested at the beginning, we can definitely use the stk600 native USB protocol to do all the stuff, so we have a freed up serial port that doesn't even need to reset the target upon DTR. 

As IcSP goes on miso, mosi and sck, while reset line pulled active, there is no need for a boot loader on the 328 side as well.

And I heard debugWire is simply twitting the rst line, so it seems wiring is not quite confusing.

Anyone could explain what is the discussion going on with the 1200 baud rate? @_@

Thank you very much!!:)

Regards Ty,

Jimmy

Sent from my iPad

--

[William "Chops" Westfield]

On May 19, 2014, at 2:35 AM, Jimmy He wrote:

> I just noticed that the serial programming method is not even a "in-circuit programming protocol". It affects integrity of circuit as we must remove anything connected on UART0. This is annoying especially when you got a bee or something attached

Do you think this will be better with MISO, MOSI, and SCK also connected between the two chips, instead of just RxD/TxD? (And no specific chip-select for the arduino; THAT will cause interesting behavior among other SPI peripherals you might have connected, during reset/upload! I guess with the AVR held in reset, the pins used for SPI chip selects would be floating, perhaps holding enough charge to be in their previous states.)

WestfW

[Rich Obermeyer]

It's rock solid since it holds the chip and any potential peripherals connected in reset.
The rx/tx of the current design is unreliable and a kludge at best. The UARTs glitch on reset and constantly cause interesting behavior.

There will be no interesting behavior wit the SPI since it will just work.
Its been shipping in a product for a while and works properly.

[Matthijs Kooijman]

Hey Jimmy,

> Folks, I see no point of discussing about tricking the serial line into debug
> mode. As Rich suggested at the beginning, we can definitely use the stk600
> native USB protocol to do all the stuff, so we have a freed up serial port that
> doesn't even need to reset the target upon DTR.

That's definately a possibility (and I like it from a technical pov),
but I'm afraid that that will create more problems with cross-platform
support, permissions and drivers - piggybacking onto the existing serial
port support is probably more reliable.

Also, Christian suggested that there are not enough USB endpoints on the
16u2 for a second serial port, so I suspect that also applies to a
second stk600 native USB interface.

About not requiring reset on DTR - that's not required either if we use
the 1200-baud-programmer-mode trick.

> As IcSP goes on miso, mosi and sck, while reset line pulled active, there is no
> need for a boot loader on the 328 side as well.

Yup, but this also applies to all four approaches I proposed a while
back.

> Anyone could explain what is the discussion going on with the 1200 baud rate?

The idea is that when you open up the virtual serial port at 1200 baud
(or any other specific "magic" baud rate we choose), then the 16u2 will
not forward the data to the real serial port on the 328, but instead
will interpret the data as programming commands, using the stk500
protocol. With all other baud rates, the 16u2 acts as a pass-through to
the 328 as it does now.

Gr.

Matthijs

[Nevada Smith]

How many Endpoints does the 16u2 has and how can i find out if i use too many? There is a lufa example for two Serials that i could try to adpat.

I looked into the SPI library... nothing special it seems. We just need to port the ICSP sketch.
I am wondering: The ISP sketch is made for an uno and mega? but not for a leonardo. If it doesnt work on the 32u4, why will it work on the 16u2?

[Jimmy He]

Dear Matthijs,
I found some official atmel documentation on these things
- Building composite USB device using 16u2
- Explanation of stk600 protocol
Maybe they can offer great help, since Leonardo also made similar stuff by building HID and serial on the same USB interface (that's a u4 though)
Sorry for not having these docs with me currently but they are definitely googleable:)
Thanks!

Jimmy

发自我的 iPhone

[William "Chops" Westfield]

> How many Endpoints does the 16u2 has

It looks like it has 4 endpoints, and a CDC (COM port) needs at least 3. (bulk in/out, int in for ctl)

I'd be careful about assuming that a native USB interface would be faster than the comm port. According to CDC documentation online, CDC requires the bulk endpoints, which are theoretically only available in full-speed mode (and higher.) Whereas a lot of the Native USB programmers enumerate as HID devices, which are (can be?) low-speed devices. So a com port may actually transfer data faster than a USB device, if it's backed up by fast hardware like the 1Mbps serial link some people have been using.

WestfW

[Jimmy He]

Hey Neveda,

Porting the ISP sketch? Hmm... 

I'd suppose the ISP sketch uses serial stk500 protocol? :0

For native USB stk600 I think we truly need to port a serious amount of info directly from Lufa and the stk600 documentation... 

And Leonardo emulates an HID and serial on the same interface as well, which means we have definitely more than one serial on the interface(it is unsure though, whether two things do work concurrently, or we should build something at the endpoint to sort the information)

Just random thought, if we get real problem getting two working at a time, we can maybe switch between programming mode and serial/debug mode whereby we set 1200 baud to determine whether it's debug wire or serial. Note that we have jp2 or jp5 header beside the 16u2 ICSP (unpopulated) for four extra GPIO pins on the 16u2, where we can place a jumper maybe. :) [A more clever alt, for programs built specifically for debugging purposes, begin serial at 1200bps from the uC side]

They're just some superficial ideas, but will try it later (sorry school ends late)

Thanks a lot!

Cheers,

Jimmy

发自我的 iPhone

--

[Matthijs Kooijman]

Hey folks,

> It's rock solid since it holds the chip and any potential peripherals connected in reset.

I'm not sure if the 16u2 could actually hold peripherals in reset,
unless those peripherals actually have their reset lines connected to
the Arduino reset line.

To keep the peripherals off the SPI bus, their CS lines should be kept
high. It's good practice to add pullups to all CS pins for exactly this
problem.

However, looking at the schematics for the Arduino ethernet and wifi
shields, it seems that these shields don't actually have a pullup
connected to CS, so they might cause problems during programming.

Atmel docs recommend putting in a series resistor to prevent issues with
multiple drivers, but that will mostly prevent things from blowing and
probably make programming work, but might still cause slave devices to
act on the programming data they receive.

I was thinking to fix this, by adding a tristate buffer in the SCK line (e.g.
between the 328 and 16u2 on one hand, and the SCK pin on the header on
the other, which is toggled by the reset pin. However, since the SCK pin
on the ISP header is used both as an input (when using an external
programmer) and an output (when using a slave device), that won't work.
Connecting the buffer to a GPIO on the 16u2 _should_ work, though.


Note that I agree that the current TX/RX approach is also prone to
problems, if something is connected to those pins. However, it's less
common to connect something to TX/RX than to the SPI pins, so it doesn't
often _actually_ pose a problem, I guess.

Gr.

Matthijs

[Nevada Smith]

USB HID can only update every 1ms. This makes me mad in my USB Keyboard bootloader project.

The 16u2 CAN act as HID and CDC Serial at the same time. But HID is slow :/ (i need 5ms delay between every keypress, asked for support on some forums)

I just worked my way through. USB is very complicated and i just start to understand these things. It seems that i am using 4 endpoints with the HID + CDC thing. So thats all for the 16u2?

[Nevada Smith]

its been quite here for a while...
in the meantime i finished my project. With some effort everything is possible! :D
https://github.com/NicoHood/HID

I tried to comple this for the 16u2 with no success. It was recognized as the right device, with drivers but i couldnt upload. it said sth like usb not found.
https://code.google.com/p/micropendous/wiki/AVRISP
and i really dont want to find the bug, because the arduinp isp example will propably be more interessing because we can built it into the firmware.

I also tried to modify the arduino isp sketch. I got it working on an uno, mega and leonardo. Leonardo only serial1, mega all serials. I just tried to port it to the 16u2 without success. Okay it was just a very messy copy paste, i need to check whats going on there.
But i think weve got a few problems. the ram limitation will make life very hard. every serial buffer takes 125 bytes. so half of the ram (512kb) is taken. i tested it with 50 bytes each and i could stil upload sketches. not sure if the buffer is oversized but 50 is not very much you should know. And you need to solder the j2 header to get another SS pin. you could use the unos reset but thats no so cool if you want to run the uno too.  not sure about that. for testing i will just use PORTB4 as SS
if they just could put a 32u4 there that would be totally overpowered :D

maybe someone can help me now if you are interrested. first i dont care about the baud, just forward every serial stuff to the iscp without any other serial exchange. i removed some stuff from the original code to get rid of leds and add the right port (because we dont have pinmode).
Code here: http://pastebin.com/QF4rVSBh

[Nevada Smith]

Hey guys! You still alive?

Good news! I got it working. The 16u2 now works as ISP. Read the readme for all information:
https://github.com/NicoHood/Hoodloader

(HID Project Github isnt updated yet until i tested this more)

There are a few things left we need to to:
Beta testers
Which SS pin discussion (Reset or pinheader, or both depending on baud setting?)
Which baud discussion (not sure if baud 1 is compatible on all systems)
Someone who can fix the Mega Fuse upload bug

Any comments on that? Someone can say "it works for me"?

Something i want to add:
We cannot add another Serial, the 16u2 only has 5 Endpoints of which 4 are used by default and the HID uses the last. Midi is also not possible then.
This is the only way to use the 16u2 as ISP: CDC.

Oh and for the next Arduino design: Please also connect I2C, please!

[Jimmy He]

Hey Nevada,

Wow thank you!:) I will beta-test it later. Gonna build a prototype Arduino board on that basis as well. We'd suppose the SS pin was just taken from the P5 header? We got four there.

Now, there might be one thing to consider:

If the serial is left with only debug purposes, do we really need a UART for debugging?

Maybe just use that HID endpoint to debug through I2C, or even a designed protocol through the rest of the pins on P5 header? By that means we can relieve the use of Serial as well! :) Further releases might just be able to use Debug.println() haha

I really think the hardware team will consider this. If we still have AVRs in further releases. In essence Arduino Zero already features a similar design, but that even supports in-curcuit debugging which is a even greater beast.

[Nevada Smith]

If i could change the Arduino board i'd do a few basic things:
Connect SPI together.
Connect I2C together
Still connect Serial, but with bridge to uncouple
On/Off switch
Autoreset jumper -> for ISP, there we have the SS pin
More ram than the 16u2. 500 bytes is evil if you see the implementation!!!

Some problems with that: Serial0 didnt work for me with extern devices. I think the 16u2 interfered the signal thatswhy the bridge to unconnect the 328.
The Hardware SS is not needed. It just needs to be output. If we had an autoreset jumper we could deactivate the reset of the main chip and add a jumper cable to use it as SS for other chips.
I2C would be for the HID probably. It gives us a 100% free Serial and a twodircetional function (for keyboard leds for example). HID can be used for Keyboard and so on or as raw HID as well. At the moment it is a HID->Serial converter but on the PC side i can only receive Data. Sending only works on a single report. A workaround would be to use a bigger AVR with more endpoints or just someone who knows RAW HID better.

In general i just ported the Arduino as ISP and improved a few syntax things. Someone whith more knowledge about the protocol maybe should have a closer look and implement all stk500v1 functions. Its working but maybe not the perfect way.

And for sure i'd keep the Uno/Mega Series. I'd help develop the Bootloader for sure. Lets create R4 ;)

And if someone could fix the Mega fuses bug, that'd be great!

Now for the utopic changes:
Break out all pins and use a 32u4. Then we could use it as programmer, HID, Serial output and even more because it has more endpoints, flash and ram. If we breakout all pins we could mod more and use it as 2nd chip as well for advanced users. Or we could make this work as full Arduino leonardo. But this is just an idea, i think i am the only one who would like a 32u4 with full breakout.

[Nevada Smith]

Searched around a bit. We need at least a 16u4 or 32u4 because of I2C. Thinking about this... its great! We really could add a debug.print() without serial0. then We can free the Serial0 for Shields. No softserial is needed then.
The 16u4/32u4 also has Jtag debugging! But i think this is only to debug the chip itself, not other chips. More endpoints, more DPRAM. This meens we can maybe add a midi interface as well!

So many more things would be possible. Just looking through the datasheet. It really makes me want to use a 16u4/32u4 (better 32u4).

I currently have no experience with creating costum PCBs. But I'd definitely work on one if someone could help me. I'd do the Software part for sure.

That sounds like a really nice upgrade to the Uno. ISP, HID, Debug Monitor and even more we can add to the Bootloader functions!

[Jimmy He]

Hey Smith that's very good idea! For long I've been desiring an all-in-one USB controller where you can simply ask it to switch around the function - and now it seems we can definitely get there! :) Maybe more than debugger functions, since the USB controller is no longer simply forwarding serial messages, we can simply rewire all the keyboard, mouse and midi(to be created) objects to the same exact line. Things might start to get easier for things like Midi.enable();

I'm personally trying the Hoodloader by customly building a prototype board using the two processors. Prototyping might be even easier on 32u4 as we just need a Leonardo or Pro Micro plus a 328 controller. Gonna try to create it and also share a prototype schematics.

I'd prefer i2c as no pins are wasted. (However, it might be fixed in master mode. )

We do need some official support here. To bring it to the reality requires a huge amount of effort not only on the framework but also on hardware design. And the board is definitely going R4(for Uno and mega) or Quattro (if they wanna preserve the previous boards). Backward compatibility needs to be considered on the software level. 

Update you later on my prototype. 

Inviato da iPhone

--

[Nevada Smith]

Tell me what you think in this new, separate post:
https://groups.google.com/a/arduino.cc/forum/#!topic/developers/Nc9SmWtC190