nanopb on Arduino

[stu....@gmail.com]

I'm new to both Arduino and Nanopb and can't get my head around the way
the following code behaves. The basic idea is to modify and delimit protocol
buffer messages using the SLIProtocol - hence pre and post pending the message with 192(END) characters. I'd like a sendMessage function to do this for me. I've happily compiled the simple examples etc. This works as it is, sending messages happily. Comment out the 2 lines of code highlighted below and it outputs some random rubbish again and again. I just can't get my head round why - maybe this is Arduino, maybe it's my environment but maybe this is what you'd expect and I just need to learn .... I've tried and tried and just can't get to grips with this - thanks for any help.

Stu

extern "C" {
#include <pb_encode.h>
#include <pb_decode.h>
#include <message4.pb.h>
}

void sendMessage(const void * mOutgoing){
uint8_t buffer[32];
pb_ostream_t ostream = pb_ostream_from_buffer(buffer, sizeof(buffer));
pb_encode(&ostream, block_fields, mOutgoing);
size_t num_written = ostream.bytes_written;
Serial.write(192);
for(int i=0; i < num_written; i++){
Serial.write(buffer[i]); // extra work required for the real protocol
}
Serial.write(192);
}

void setup(){
Serial.begin(9600);
}

void loop(){
static int counter = 0;
block mMessage;
mMessage.mytestint = counter++;
if (counter == 100) counter = 0;
const void * pTest = &mMessage;

// comment out the next 2 lines and it doesn't work
uint8_t sendbuffer[32];
pb_ostream_t sendostream = pb_ostream_from_buffer(sendbuffer,
sizeof(sendbuffer));
// but what have they got to do with the following call?

sendMessage(pTest);
}

[Petteri Aimonen]

Hi,

> I'd like a sendMessage function to do this for me. I've happily
> compiled the simple examples etc. This works as it is, sending
> messages happily. Comment out the 2 lines of code highlighted below
> and it outputs some random rubbish again and again.

Your code looks fine to me. The symptoms point to some stack trouble,
like stack overflow, even though it doesn't appear like your code should
use much stack memory.

Just as a test, could you try making these two variables static?

> void sendMessage(const void * mOutgoing){

> static uint8_t buffer[32];

> void loop(){
> static block mMessage;

This should further reduce stack usage, so maybe then it will work
without the dummy code in loop().

Let me know how it goes. Maybe Google would find some way to check stack
usage on Arduino.

--
Petteri

[Michael Haberler]

Am 28.04.2014 um 23:52 schrieb stu....@gmail.com:

> I'm new to both Arduino and Nanopb and can't get my head around the way
> the following code behaves. The basic idea is to modify and delimit protocol
> buffer messages using the SLIProtocol - hence pre and post pending the message with 192(END) characters.

Hi Stu,

I think using a serial IP framing method is a great idea, and maybe contributing a code fragment would be useful to other folks

If I may add a suggestion (being an ISP from the very early days and having suffered through the SLIP era ;)

I think it'd be only marginally more effort to use PPP encapsulation: http://en.wikipedia.org/wiki/Point-to-Point_Protocol#Encapsulation - the upside of this is: you get a checksum at the frame level which you dont have with SLIP; depending on choice either 16 or 32bit for extra strength

I would think there's significant reusable code for PPP, for instance the lwip stack: http://savannah.nongnu.org/projects/lwip/, see http://git.savannah.gnu.org/cgit/lwip.git/tree/src/netif/ppp for fragments

- Michael

[stu....@gmail.com]

Michael,
I'll happily contribute my SLIP implementation (once it works of course :-)). I've alse looked at both PPP and Varint delimited messages. Google's protocol buffer c++ api, which I'm using at the other end of the wire, doesn't make using varint delimited messages easy. I've implemented/wrapped this myself but the code was starting to bloat. Something to come back to later.

I did look at lwip but if it needs 10k + RAM and 40K ROM it's not going to fit on my Arduino UNO at 2k RAM and 32k ROM without some reworking. I also looked at an Arduino library (SerialIP) which seems to work nicely but is still large and probably overkill.

So nothing is dismissed, I just thought that SLIP would make an easy starting point! (arrggh).

[stu....@gmail.com]

Petteri,

Thank you.

Making the uint_t buffer static had no effect on my program - other than to make it better.

Making the message static broke the code as well (the function is called but it doesn't write any protocol between my 2 enclosing END markers).

I agree with your thought about overflowing stacks causing wierd issues but this seems like a tiny fragment and it breaks when I remove stuff from the stack not when I add it.

Given your sanity check, I suspect that this is something Arduino. This is my first real go with Arduino so I could well be missing something really simple.

[Michael Haberler]

Am 29.04.2014 um 08:52 schrieb stu....@gmail.com:

Stu,

>
> Michael,
> I'll happily contribute my SLIP implementation (once it works of course :-)). I've alse looked at both PPP and Varint delimited messages. Google's protocol buffer c++ api, which I'm using at the other end of the wire, doesn't make using varint delimited messages easy. I've implemented/wrapped this myself but the code was starting to bloat. Something to come back to later.
>
> I did look at lwip but if it needs 10k + RAM and 40K ROM it's not going to fit on my Arduino UNO at 2k RAM and 32k ROM without some reworking.

maybe a misunderstanding - I didnt suggest to use the lwip stack in toto, but rather the PPP LCP framing, which is just a small fraction of this codebase.
the link http://git.savannah.gnu.org/cgit/lwip.git/tree/src/netif/ppp shows the directory where the PPP framing stuff of lwip lives. The reason is - PPP framing is much more robust than SLIP, and has optional checksums. SLIP was a hack which lived way too long.

on varint decoding:

here are some examples for varint decoding besides nanopb (you could use nanopb easily just as well!):

http://cxwangyi.wordpress.com/2010/07/20/encoding-and-decoding-of-the-varint32-type-defined-by-google-protocol-buffers/
http://web.archive.org/web/20111219152850/http://blog.reverberate.org/wp-content/uploads/2008/07/pb.c
https://github.com/haberman/upb/wiki

> I also looked at an Arduino library (SerialIP) which seems to work nicely but is still large and probably overkill.
>
> So nothing is dismissed, I just thought that SLIP would make an easy starting point! (arrggh).
>

> --
> You received this message because you are subscribed to the Google Groups "nanopb" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nanopb+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[stu....@gmail.com]

Michael,

>
> on varint decoding:
>
>
>
> here are some examples for varint decoding besides nanopb (you could use nanopb easily just as well!):
>
>
>
> http://cxwangyi.wordpress.com/2010/07/20/encoding-and-decoding-of-the-varint32-type-defined-by-google-protocol-buffers/
>
> http://web.archive.org/web/20111219152850/http://blog.reverberate.org/wp-content/uploads/2008/07/pb.c
>
> https://github.com/haberman/upb/wiki
>

Thanks for the links. Varints are quite useful for parsing / knowing when one has been received fully, but I was finding that they can be a bit of a pain if the sender and receiver get of of sync with each other. Your PPP suggestion is good - it would save me hand crafting PPP-like behaviour.

As it happens, my next line of development in my project was probably to migrate away from the serial port I'm using at the moment - to ethernet. If I do this on arduino, I'll probably use a hardware shield which fully implements the full tcp/ip stack. I thought this would be a gentle introduction for myself - how wrong could I be...

Stu

[Petteri Aimonen]

Hi,

> Given your sanity check, I suspect that this is something Arduino.
> This is my first real go with Arduino so I could well be missing
> something really simple.

On the other hand, people have been successfully using nanopb on Arduino
and there shouldn't be anything that special.

Can you send the .proto file you are using, the correct binary data when
it works, and the incorrect binary data when it does not work? Maybe
these will give some clues to me.

--
Petteri

[Stuart Reeks]

Petteri,

Here are three files: the proto, uncommented.txt and commented.txt. The
uncommented is the programming running correctly (I believe) I suspect a
'hexdump' shows a few erroneous bytes as a buffer is flushed right at
the start, but it then gets on with sending delimited messages. The
uncommented file is the dump from when I comment out the 2 lines of code
- I could have left minicom running longer but it keeps outputting
exactly the same bytes.

Thanks for looking at this.

Stu