[rabbitmq-discuss] Message payload encoding

[Helena Edelson]

Is there a preference, as in performance or other reason (please specify if so) for a message payload to be encoded for Rabbit as UTF-8 or base64?
I have been doing UTF-8; just curious.

Thanks,
Helena
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq...@lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[Peter Ohler]

If you don't mind I will take a shot at that one.

There are a few consideration that should be looked at. It is often the case that message size is not the most significant factor in performance, especially if the message size is small. Consider what it is you are sending. If it is a UTF-8 string then sending the string directly takes no preparation and is smaller than base64 encoding which also incurs and overhead in serialization and deserialization.

If what you are sending is an Java, Python, or Ruby object then there will be significant overhead in the serialization itself. For Ruby I wrote a fast object to XML serializer (Ox gem) that is much faster than the native Marshal encoding and that alone improved performance of our systems significantly.

Something else worth considering besides is how difficult it will be to debug and trace message exchanges. A binary format or base64 requires extra decoding to see what is in the body of a message while UTF-8 strings or XML can be viewed and understood directly. That is a real time saver during development and even in production when trying to fix a problem quickly.

-Pete

[Simon MacMullen]

On 29/07/11 19:00, Helena Edelson wrote:
> UTF-8 or base64

I'm curious - those are the two options supported by the management
plugin - are you sending messages through that?

If so, the idea is that UTF-8 is more convenient if your message payload
is a string, but if your message needs to contain arbitrary binary
content then we need to encode it somehow (as the message will end up as
a string in a JSON document).

If you're using AMQP then this doesn't apply, the message contents can
be any binary, and using base64 encoding would be an odd choice!

It you are sending messages via the management API, be aware that it's
designed for testing / diagnostics / experimentation - performance is
not great, and there's no way to get greater reliability than consuming
with noack.

Cheers, Simon

--
Simon MacMullen
RabbitMQ, VMware

[Helena Edelson]

Thanks Pete. Some payloads are Scala objects (serialized/deser by scalaxb/base64) and some plain xsl schemas (which some are doing base64 but I'd prefer UTF-8).

cheers,
Helena
________________________________________
From: Peter Ohler [pe...@ohler.com]
Sent: Friday, July 29, 2011 8:12 PM
To: Helena Edelson
Cc: rabbitmq...@lists.rabbitmq.com
Subject: Re: [rabbitmq-discuss] Message payload encoding