Intra-process queue between Java and C++

[Roman Leventov]

I think about the possibility of building an asynchronous application with back pressure where some upstream operators are in Java and some downstream ones are in C++. For this purpose, some queues would be needed to pass the data between Java and C++ layers. It seems that porting JCTools's bounded array queues to off-heap should be doable, but I couldn't find existing prototypes or discussions of such thing so maybe I overlook some inherent complications with this idea.

Did anybody think about something like this or has implemented in proprietary systems?

[Martin Thompson]

There are implementations of FIFO ring buffers for Java and C++ used in Aeron for doing exactly this.

https://github.com/real-logic/aeron/tree/master/aeron-client/src/main/cpp/concurrent

https://github.com/real-logic/agrona/tree/master/agrona/src/main/java/org/agrona/concurrent

You could also use Aeron IPC.

[Roman Leventov]

Martin, thanks a lot!

I thought about Aeron IPC, but as far as I understand it maps to the queue model only when there is a single producer and a single consumer. Also it felt a little too heavyweight for small fixed-sized messages. Generally Aeron's Data frames have 32-byte headers. RingBuffers have only 16-byte headers, and it looks like it could be harmlessly reduced down to 8 or even 0 for e. g. fixed format 32-byte messages.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Martin Thompson]

Aeron IPC is more functional than a plain queue but I get your point that it is not a drop in replacement.

The ring buffers are typical queue based semantics and use an 8 byte header in Agrona and Aeron. 4 bytes for message length and 4 bytes for message type to give some flexibility. Fixed format messages can optimise well but have limited applicability.

On Friday, 30 March 2018 16:36:09 UTC+1, Roman Leventov wrote:

Martin, thanks a lot!

I thought about Aeron IPC, but as far as I understand it maps to the queue model only when there is a single producer and a single consumer. Also it felt a little too heavyweight for small fixed-sized messages. Generally Aeron's Data frames have 32-byte headers. RingBuffers have only 16-byte headers, and it looks like it could be harmlessly reduced down to 8 or even 0 for e. g. fixed format 32-byte messages.

[Greg Young]

Will the 16 bytes save you time?

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Studying for the Turing test

[Roman Leventov]

Right, 8 bytes, not 16, I've misread https://github.com/real-logic/agrona/blob/master/agrona/src/main/java/org/agrona/concurrent/ringbuffer/RecordDescriptor.java. Thanks for note.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

[Roman Leventov]

Not much, I think. Anyway, as Martin pointed to OneToOneRingBuffer it's irrelevant now.

On Fri, 30 Mar 2018, 19:17 Greg Young, <gregor...@gmail.com> wrote:

Will the 16 bytes save you time?

On Fri, Mar 30, 2018 at 10:36 PM, Roman Leventov <leven...@gmail.com> wrote:

Martin, thanks a lot!

I thought about Aeron IPC, but as far as I understand it maps to the queue model only when there is a single producer and a single consumer. Also it felt a little too heavyweight for small fixed-sized messages. Generally Aeron's Data frames have 32-byte headers. RingBuffers have only 16-byte headers, and it looks like it could be harmlessly reduced down to 8 or even 0 for e. g. fixed format 32-byte messages.

On 30 March 2018 at 13:00, Martin Thompson <mjp...@gmail.com> wrote:

There are implementations of FIFO ring buffers for Java and C++ used in Aeron for doing exactly this.

https://github.com/real-logic/aeron/tree/master/aeron-client/src/main/cpp/concurrent

https://github.com/real-logic/agrona/tree/master/agrona/src/main/java/org/agrona/concurrent

You could also use Aeron IPC.

On Friday, 30 March 2018 09:55:23 UTC+1, Roman Leventov wrote:

I think about the possibility of building an asynchronous application with back pressure where some upstream operators are in Java and some downstream ones are in C++. For this purpose, some queues would be needed to pass the data between Java and C++ layers. It seems that porting JCTools's bounded array queues to off-heap should be doable, but I couldn't find existing prototypes or discussions of such thing so maybe I overlook some inherent complications with this idea.

Did anybody think about something like this or has implemented in proprietary systems?

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Studying for the Turing test

--

You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

[Kristoffer Sjögren]

What would a multi-client/server protocol look like for two-way session communication using shared memory ring buffers? Especially during setup where both processes needs to agree on where the in/out ring buffers reside in memory. I'm guessing the initial mmap could be expensive?

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

[Todd Montgomery]

Aeron itself uses a Broadcast scheme for request/response with clients. All clients listen to the Broadcast for responses as well as async events. A lot of times a pure request/response isn't the right semantic. There is often the need for async events and multi-client information.

But if you need to do client/server mechanism with single client response buffers, the easiest way is to have the client send the location for the response on its request (Reply-To style). Easily done with files, BTW.

On Sun, Apr 1, 2018 at 1:44 PM, Kristoffer Sjögren <sto...@gmail.com> wrote:

What would a multi-client/server protocol look like for two-way session communication using shared memory ring buffers? Especially during setup where both processes needs to agree on where the in/out ring buffers reside in memory. I'm guessing the initial mmap could be expensive?

On Fri, Mar 30, 2018 at 6:40 PM, Roman Leventov <leven...@gmail.com> wrote:

Right, 8 bytes, not 16, I've misread https://github.com/real-logic/agrona/blob/master/agrona/src/main/java/org/agrona/concurrent/ringbuffer/RecordDescriptor.java. Thanks for note.

On Fri, 30 Mar 2018, 19:16 Martin Thompson, <mjp...@gmail.com> wrote:

Aeron IPC is more functional than a plain queue but I get your point that it is not a drop in replacement.

The ring buffers are typical queue based semantics and use an 8 byte header in Agrona and Aeron. 4 bytes for message length and 4 bytes for message type to give some flexibility. Fixed format messages can optimise well but have limited applicability.

On Friday, 30 March 2018 16:36:09 UTC+1, Roman Leventov wrote:

Martin, thanks a lot!

I thought about Aeron IPC, but as far as I understand it maps to the queue model only when there is a single producer and a single consumer. Also it felt a little too heavyweight for small fixed-sized messages. Generally Aeron's Data frames have 32-byte headers. RingBuffers have only 16-byte headers, and it looks like it could be harmlessly reduced down to 8 or even 0 for e. g. fixed format 32-byte messages.

There are implementations of FIFO ring buffers for Java and C++ used in Aeron for doing exactly this.

https://github.com/real-logic/aeron/tree/master/aeron-client/src/main/cpp/concurrent

https://github.com/real-logic/agrona/tree/master/agrona/src/main/java/org/agrona/concurrent

You could also use Aeron IPC.

On Friday, 30 March 2018 09:55:23 UTC+1, Roman Leventov wrote:

I think about the possibility of building an asynchronous application with back pressure where some upstream operators are in Java and some downstream ones are in C++. For this purpose, some queues would be needed to pass the data between Java and C++ layers. It seems that porting JCTools's bounded array queues to off-heap should be doable, but I couldn't find existing prototypes or discussions of such thing so maybe I overlook some inherent complications with this idea.

Did anybody think about something like this or has implemented in proprietary systems?

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Rahil Bohra]

If you can tolerate the overhead of JNI it sounds like zeromq (zeromq.org) might fit the bill: though as far as i know there is no built in back pressure other than a high watermark on queued messages for a socket. 

[Kristoffer Sjögren]

Todd: Sending the location for the response in the request was the first idea that came to mind. But as clients in this case would be very short lived processes i'm trying to preallocate/setup as much as possible during server startup.

Is it feasible to use a shared aeron_mpsc_rb for incoming client requests and use the aeron_broadcast_transmitter for (async) responses? Where server indicate the recipient by means of a clientId message header (other already connected client would ignore this message). Or is the aeron_broadcast_transmitter best used for protocol/control events without much payload? I guess the server could also broadcast the location of a preallocated single client aeron_spsc_rb for larger payloads in the reply?

[Francesco Nigro]

There are a couple of things to be considered:

1. size request and response buffers to avoid deadlocks (there is an interesting topic on this list re it too) -> could be used a broadcast buffer too to avoid it, but it change the semantic by adding a new "failure" state
   
2. consider liveness of processes and blocking operations while submitting/reading from the buffers -> Agrona ring buffers has the proper API to perform such tests/ops but need to implement the logic

[Todd Montgomery]

Good points. The Agrona (and Aeron C versions) do have the proper basic logic to avoid blocked queues from failure during appends/offers. It's not simple, but the hard part is done for you. Just need to hook the logic into a duty cycle.

The broadcast buffer is a very unique type of structure. It allows for the case of a slow receiver to fall behind and loose messages rather than block the transmitter. And it lets the receiver know that this occurred. That means that some of the interesting deadlocks can be avoided. Recovery of lost messages (or just resync of state) can be done a number of ways. Including just a system graceful reset.

--

[Francesco Nigro]

The broadcast buffer is a very unique type of structure.

So true!

It relies on a nice mechanics similar to StampedLock and is quite interesting indeed: I have implemented for Agrona a version of it based on the FastFlow-like logic instead to allow "precise" loss counters, using predefined max message length.

Probably with actor-like services that provide a finite set of responses with reasonable max lengths, it could be a good match if compared to the original :P

Obviously the same usage patterns highlighted by Todd will apply independently by the chosen implementation: the semantic behind a message loss enables lot of interesting logics to be implemented on top, while simiplifying most of the deadlock issues.

[Todd Montgomery]

On Tue, Apr 3, 2018 at 6:47 AM, Kristoffer Sjögren <sto...@gmail.com> wrote:

Todd: Sending the location for the response in the request was the first idea that came to mind. But as clients in this case would be very short lived processes i'm trying to preallocate/setup as much as possible during server startup.

With a reply-to that contains a file name, the file would be created by the client. So, it should keep latency low for allocation. And would be no allocation on the server.... if that is the side you want to make sure is not held up with allocation. However, it does have both sides mapping the region.

An alternative to a reply-to via filename is a single response buffer area. As clients "connect", they atomically increment a counter. That counter (mod the size of the area) dictates their response slot.

So, in that case, everything but the client mapping is pre-allocated. It just means you need a section of memory and you have to handle collisions of 2 clients grabbing the same mapping (same mod

value).

 

Is it feasible to use a shared aeron_mpsc_rb for incoming client requests and use the aeron_broadcast_transmitter for (async) responses? Where server indicate the recipient by means of a clientId message header (other already connected client would ignore this message). Or is the aeron_broadcast_transmitter best used for protocol/control events without much payload? I guess the server could also broadcast the location of a preallocated single client aeron_spsc_rb for larger payloads in the reply?

Exactly!

aeron_mpsc_rb + aeron_broadcast is exactly what the Aeron driver does. Everything you said is exactly correct. The responses are small. And some are async in nature. Such as image available/unavailable. A clientId (as well as command correlation Ids) are an atomic value via the aeron_mpsc_rb interface. aeron_mpsc_rb_next_correlation_id to be precise. It's an atomic counter that can be used to generate a unique id for the driver and all clients (as long as they all use it). Aeron doesn't have any large responses. But if it did, it would do exactly as you mention, send the filename of the ring buffer. This is what it effectively does with the logbuffers for publications and subscriptions.

[Kristoffer Sjögren]

Exactly!

aeron_mpsc_rb + aeron_broadcast is exactly what the Aeron driver does. Everything you said is exactly correct. The responses are small. And some are async in nature. Such as image available/unavailable. A clientId (as well as command correlation Ids) are an atomic value via the aeron_mpsc_rb interface. aeron_mpsc_rb_next_correlation_id to be precise. It's an atomic counter that can be used to generate a unique id for the driver and all clients (as long as they all use it). Aeron doesn't have any large responses. But if it did, it would do exactly as you mention, send the filename of the ring buffer. This is what it effectively does with the logbuffers for publications and subscriptions.

Thanks Todd, appreciate the detailed response! Its a lot clearer to me now. I'll experiment a bit and see where it ends up. The documentation on Aeron is great by the way.