Send binary data without conversion to string?

[DrBwts]

Hi,

RPi 4, mosquitto, C

I have a high through put of data that is currently being stored locally on an SD card. I have had a look at sending the data via the Pi's eth0 port using MQTT. Basically the conversion of each chunk of 20k doubles to string for the payload was choking my program out.

Is the conversion to string necessary? As the payload is send as bytes is there a way of sending my raw data with no conversion step?

Thanks

Nic

[Simon Walters]

What library are you using to send using MQTT in your C program?

What/why/how is your data size being doubled?  Strings are normally just a sequence of bytes with possibly one termination character

regards

Simon

--
To learn more about MQTT see https://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mqtt/864feb6d-aa67-4168-9c71-550690f2e340n%40googlegroups.com.

[DrBwts]

Hi,

I'm using Eclipse Paho 

I am wanting to send double floats which are 4 bytes long but when using MQTT I have found that I have to convert each double float into an array of chars to send in the payload. The floats are 19 figures long which means instead of sending 4 bytes per float I'm now sending 20 bytes, this coupled with the time it takes to convert a float  to a string makes the process too slow for my application (I'm sampling 14 channels at 25kHz, yes I do need that level of resolution). So I was wondering if I need to make that conversion at all & can just send the raw doubles?

[Dougie Lawson]

MQTT is agnostic to the data format. You send a stream of bytes, no interpretation is made of whether those bytes are characters, integers, floats, JPEG, JSON, HTML or whatever. They are just bytes. 

So you don't need to "encode" or "decode" any data. Send raw bytes, interpret the result in your receiver program.

Regards, Dougie

To view this discussion on the web visit https://groups.google.com/d/msgid/mqtt/ea7f120a-2ab7-48c3-87ca-e3459844264dn%40googlegroups.com.

--

http://twitter.com/DougieLawson

[Greg Troxel]

Dougie Lawson <dl1...@gmail.com> writes:

1> MQTT is agnostic to the data format. You send a stream of bytes, no

> interpretation is made of whether those bytes are characters, integers,
> floats, JPEG, JSON, HTML or whatever. They are just bytes.
>
> So you don't need to "encode" or "decode" any data. Send raw bytes,
> interpret the result in your receiver program.

Yes, but one likely needs to transform to/from network byte order.

I would suggest the OP consider protobufs; that might be a good blend of
efficiency and dealing with a bunch of issues.

[Andy Stanford-Clark]

It’s a bit messy, but you want to get the bytes from the float and put them into a char/byte array… 

Casting to/from a struct seems to be the way to do this… 

e.g.
https://stackoverflow.com/questions/24420246/c-function-to-convert-float-to-byte-array

https://os.mbed.com/forum/helloworld/topic/2053/

Hope some of this gives you the pointer you need…

Andy

To view this discussion on the web visit https://groups.google.com/d/msgid/mqtt/ea7f120a-2ab7-48c3-87ca-e3459844264dn%40googlegroups.com.

[DrBwts]

Aaahhh union is what I'm after,

Thanks all

[Andy Stanford-Clark]

Yeah, they always feel a bit of a hack, but seems to be the way, short of explicitly shifting the bits via a pointer to the float and popping them into byte array elements by hand.

To view this discussion on the web visit https://groups.google.com/d/msgid/mqtt/d423e16a-843e-4d26-a0ff-3c4b81045851n%40googlegroups.com.

[Ivan Mari]

You could also take a look at: https://cbor.io/

To view this discussion on the web visit https://groups.google.com/d/msgid/mqtt/D51D6D46-0BCE-4BB2-B9DE-E6BF8F9D4C5A%40stanford-Clark.com.