Topics in High Performance Messaging

[Francis Stephens]

https://www.informatica.com/downloads/1568_high_perf_messaging_wp/Topics-in-High-Performance-Messaging.htm

I just stumbled over this. It really is an excellent read.

As an aside, I came upon this article while trying to find resources on securing UDP messages, mostly focussed on spoofing prevention. The best advice I could find recommended sha1 hashing the message with a shared secret and adding that as a signature to each message. Does anyone have experience dealing with the performance/security trade off this would involve. Thanks.

[Martin Thompson]

On 1 December 2015 at 15:15, Francis Stephens <francis...@gmail.com> wrote:

https://www.informatica.com/downloads/1568_high_perf_messaging_wp/Topics-in-High-Performance-Messaging.htm

I just stumbled over this. It really is an excellent read.

I had the fortune of discovering Todd and 29West in the early days of LMAX. LBM quickly replaced our home grown network stack. This document was of great help to me in the early days for understanding what is going on.

Working with Todd on Aeron has been a great learning experience :-)

 

As an aside, I came upon this article while trying to find resources on securing UDP messages, mostly focussed on spoofing prevention. The best advice I could find recommended sha1 hashing the message with a shared secret and adding that as a signature to each message. Does anyone have experience dealing with the performance/security trade off this would involve. Thanks.

I've had a number of discussions with people needing secure messaging but also high-performance messaging. In order to run most messaging systems securely they have to sit on top of a VPN or SSL. This equalises them all to relatively poor performance. The immutable log buffer in Aeron lends its self to very fast cryptography integration that can be done mostly in parallel. Next year one of our clients could be sponsoring the development of a secure from of the Aeron protocol. We have a few proper security folk interested in joining us in this effort.

If doing native cryptography it is worth looking at libsodium.

https://github.com/jedisct1/libsodium

 

[Francis Stephens]

Thanks Martin. I was very impressed by that document. It is not common to get such a thorough review of a technical topic that is so approachable.

I suppose in practice most people running low latency UDP messaging systems are doing so behind their own firewalls and rely on external security to protect them?

[Martin Thompson]

On 2 December 2015 at 14:11, Francis Stephens <francis...@gmail.com> wrote:

I suppose in practice most people running low latency UDP messaging systems are doing so behind their own firewalls and rely on external security to protect them?

Yes most low-latency UDP traffic is on private networks behind firewalls, or it is over trusted extranets such a BT Radianz. "Security" is sometimes employed but this is really just to stop traffic being consumed by "accident", i.e. configuration errors.

Many are now considering cloud services and would like that traffic secured but in a way that does not compromise performance too much. 

 

[Rajiv Kurian]

Though meant to be a more efficient generic web transport https://en.wikipedia.org/wiki/QUIC has some interesting opinions on how to do security when using UDP.

More details on the security parts of it - https://docs.google.com/document/d/1g5nIXAIkN_Y-7XJW5K45IblHd_L2f5LTaDUDwvZ5L6g/edit and https://eprint.iacr.org/2015/582.pdf

[Francis Stephens]

Yes, I am looking at cloud services as an option and trying to get a feel for the trade-offs. Also, no UPD multicast in the cloud (AFAIK), but if 'reasonable' performance can be achieved the convenience can be a huge win. We'll see :)

[Francis Stephens]

That looks great. Thanks Rajiv.

[Todd Montgomery]

Thanks for the kind words, Martin! I credit the document to Bob Van. He took all the stuff in my head, added stuff in his, and made it accessible.

Multicast brings in a number of security concerns that unicast might not normally address. Specifically, different forms of non-repudiation, efficient rekeying on group membership changes, etc. However, all of the issues are quite well known and have decent solutions. QUIC addresses a few, but really does not address them all (it doesn't need to).

BTW, we do plan to provide a multi-send form of multicast for cloud usage of Aeron eventually. Timeline not set yet.

From an implementation standpoint, the mechanics of the Aeron logbuffer allow for the most efficient form of encryption to be applied in a pipeline form without slowing a publisher or receiver. With todays processors, the use of a logbuffer and the right architecture could, I am totally convinced, have the least impact on throughput and latency out of any approach taken by any transport in use today. Making confidentiality much more accessible to use cases that traditionally have avoided it due to performance impact.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.