Issues De-Serialize Embedded nanopb Messages Using C# Serial Port Application

[Matt MacLeod]

Hello, 

I am new to nanopb and I am trying to create an embedded application that transmits encoded nanopb messages over UART to my C# desktop application. 

The C# side, reads the serial port and collects a byte array that contains the serialized nanopb messages. 

I have tested the serial packet collection API and I am confident that I am collecting the correct bytes, however. I am getting the following errors (for versions 1 and 2 of my c# code respectively )  when attempting to de-serialize the nanopb messages using C#  protobuf-net implementation: 

'System.IO.EndOfStreamException' occurred in protobuf-net.dll: 'Attempted to read past the end of the stream.'

'Google.Protobuf.InvalidProtocolBufferException' occurred in Google.Protobuf.dll: 'While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length.'

Ive tested the nanopb implementation on the embedded side and there is no issues, it is only when I transmit over serial that I am getting these issues. 

See below proto file: 

syntax = "proto3";

// import "nanopb.proto";

// special enum types for each ID

enum MessageID {

   FORCE = 0;

   ACCEL = 1;

   TEMP = 2;

   BAT = 3;

   TIME = 4;

   MIC = 5;

}

// Accel data is int16 (not supported by protobuf messaging)

// Array of KX134_FIFO_THRESHOLD int16 samples (x,y,z) interleaved

// For value of KX134_FIFO_THRESHOLD see kx134.h file

// As of June 20, 2022 had been set to 37

// Revision June 22, 2022 WMI int has been set to 39 corresponding to 234 bytes

message Acceleration {

   bytes data =1;

   

}

//reads 24 bit temperatre samples from adc

//driver code automatically alternates between measuring the CJC (internal temp onboard the IC)

// and the ADC measured temp val (both are raw values representing voltage (mV) and need to be corolated with type K tables )

message Temperature {

 

   int32 voltageADC =1;

   int32 voltageInternal=2;

}

// reads 16 bit adc value from force sensor

//values will be linearized according to the NRC specification (to be added to this documentation)

message Force {

 

   int32 mvReading =1;

}

message OdrFreqData {

 

   uint32  odr_frequency_tenth_hz=1;

   uint32  odr_delta_tenth_hz=2;

}

// At present we will use this message to report the current battery voltage level in mV (integer)

message Battery {

   int32 mvReading =1;

   // bytes data = 1 [(nanopb).max_size = 16,(nanopb).fixed_length= true];

}

See below nano pb Implementation: 

/*

  Function: Funciton to encode as nanoPB messages based off message ID

  Output: Lenght of bytes that were encoded

*/

int pb_encode_data(MessageID id,  uint8_t *rawData,

                  uint8_t data_len, pb_ostream_t stream ){

  // use case switch since number of message types will increase drastically as we evolve messaging api

  switch(id){

    case MessageID_ACCEL:

      {

        Acceleration message = Acceleration_init_zero;

        memcpy(&message.data[0], &rawData[0], data_len);

        if (!pb_encode(&stream, Acceleration_fields, &message)) {

            return 0;

        }else{

            return stream.bytes_written;

        }

      }

    case MessageID_TIME:

      {

        OdrFreqData message = OdrFreqData_init_zero;

        msg_sample_rate_monitor_t   *pmsg_sample_rate_monitor;

        pmsg_sample_rate_monitor = (msg_sample_rate_monitor_t *)rawData;

        message.odr_frequency_tenth_hz = pmsg_sample_rate_monitor->odr_frequency_tenth_hz;

        message.odr_delta_tenth_hz = pmsg_sample_rate_monitor->odr_delta_tenth_hz;

        if (!pb_encode(&stream, OdrFreqData_fields, &message)) {

            return 0;

        }else{

            LOG_INF("Bytes Written: %d", stream.bytes_written);

            return stream.bytes_written;

        }

        // pbMessage.message = &message;

      }

      break;

    case MessageID_FORCE:

      break;

    case MessageID_TEMP:

      break;

    case MessageID_BAT:

      break;

    default:

      return NULL;

  }

}

/*

  Function: Generic Handler to encode data into nanoPB messages

  Output: Pointer to Encoded Data Buffer

*/

static uint8_t * message_encode(MessageID id, uint8_t *rawData,

                                 uint8_t len, size_t * message_length){

  static uint8_t buffer[Acceleration_size] = {0}; // allocate large buffer the size of Acceleration size (the largest dataset)

 

  // static size_t message_length;

  pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));

  *message_length = pb_encode_data(id,rawData,len,stream);

  if ( message_length > 0){

    return buffer;      

  }else{

    return NULL;

  }

}

See below C# Protobuf decode implementation: 

Version that produced error 1: 

            OdrFreqData odrMessage = new OdrFreqData();

            using (var ms = new MemoryStream(message)){

      

                odrMessage = OdrFreqData.Parser.ParseFrom(ms);

                Console.WriteLine(odrMessage.OdrDeltaTenthHz);

                Console.WriteLine(odrMessage.OdrFrequencyTenthHz);

                ms.Close();

            }

Version that produced error 2: 

           OdrFreqData odrMessage;

            using (var ms = new MemoryStream(message)){

       

                odrMessage = ProtoBuf.Serializer.Deserialize<OdrFreqData>(ms);

                Console.WriteLine(odrMessage.OdrDeltaTenthHz);

                Console.WriteLine(odrMessage.OdrFrequencyTenthHz);

                ms.Close();

            }

Any help would be appreciated. 

Thanks, 

Matt 

[Petteri Aimonen]

Hi,

I suggest you make a hex dump of the raw data on both embedded and C#
side to see if they match.

You can also use this to check what is in the message or if it is
corrupted:
https://protobuf-decoder.netlify.app/

One quite typical reason is for some specific characters to get mangled.
For example 0x0D (CR), 0x0A (LF), 0x11 (XON), 0x13 (XOFF) are prone to
get corrupted if the serial port is not set to binary mode.

--
Petteri