Ordering messages in ACH

[climbfo...@gmail.com]

Hello,

I would like to have more information about how incoming messages are processed by ACH. Does ACH has a mechanism which manage message priorities ?

1 - Firstly, I would like to know how incoming messages are processed by ACH.

I would like to confirm whether ACH handles messages in the same order as it receives or if ACH handles these messages in out of order way.

Take an example, we have 3 processes A, B, and C. A sends messages A1, A2, A3 (in this order), B sends B1, B2 (in this order) and C sends C1.

We know there are several possibilities concerning incomming messages order. Consider that ACH receives messages in this order : A1,A2,B1,A3,C1,B2.

In your opinion, what is the order that ACH handles messages ? Is it A1,A2,B1,A3,C1,B2 ?

2 - Then, I'm interested in message priority (if this kind of mechnism exists in ACH).

If we take the previous example but with highest priority for C1 message, what happens to the order in which ACH handles messages ? Is it now : C1, A1, A2, B1, A3, B2 ?

Best regards,

ZRA

[Neil T. Dantam]

On 03/31/2014 10:16 AM, climbfo...@gmail.com wrote:
> I would like to have more information about how incoming messages
> are processed by ACH. Does ACH has a mechanism which manage message
> priorities ?

On 04/01/2014 04:51 AM, climbfo...@gmail.com wrote:
> Does the realtime term used in the paper just means low-latency ? Is
> there a mechanism inside ACH (except process priorities) which allow
> realtime IPC ?

"Real-time" means bounded latency, and Ach provides this, subject to
the scheduling capabilities of the kernel (so you should probably use
something like Linux PREEMPT_RT if you need <100us response).

Priority inheritance is used to prevent priority inversion with
multiple receivers. For example, a low-priority logger will not stall
a high-priority control process if both access the same channel.

(Actually on Haswell-or-later CPUs with recent GLIBC, both processes
can poll the channel concurrently thanks to Intel's hardware
transactional memory)

The particular use case Ach is designed for is when we are most
interested in the latest data, e.g., the current joystick axis or
robot velocity. In this case, we can access the latest data in O(1).
With sockets or messages queues, one would need to skip over older
data and full buffers can result in newer data being discarded (very
bad!).

> 1 - Firstly, I would like to know how incoming messages are
> processed by ACH.

Basically, messages are passed through a circular buffer with
variable-sized entries and per-receiver "next" pointers.

For the full details, please see our paper from 2012:
http://www.golems.org/papers/dantam2012robust.pdf

> I would like to confirm whether ACH handles messages in the same
> order as it receives or if ACH handles these messages in out of
> order way.

Yes, Ach messages are received in-order. For single-host use, this is
always the case.

The only way to receive messages out-of-order would be to relay
messages over the network using an unordered transport (e.g., UDP).

> Take an example, we have 3 processes A, B, and C. A sends messages
> A1, A2, A3 (in this order), B sends B1, B2 (in this order) and C
> sends C1.

> ...

> Consider that ACH receives messages in this order :
> A1,A2,B1,A3,C1,B2.

> ...

> what is the order that ACH handles messages ? Is it
> A1,A2,B1,A3,C1,B2 ?

Yes, exactly that.

> 2 - Then, I'm interested in message priority (if this kind of mechnism exists in ACH).

No fine-grained message priority, only the ability to access only the
most recent message.

> If we take the previous example but with highest priority for C1
> message, what happens to the order in which ACH handles messages ?
> Is it now : C1, A1, A2, B1, A3, B2 ?

If these are all written to the same channel, receivers could see
either:
- [C1, A1, A2, B1, A3, B2]
- or [B2]

If each process writes to a different channels, receivers can see:
- [A1 A2 A3] or [A3]
- [B1 B2] or [B2]
- [C1]

Each receiver independently selects whether it wants the next message
in sequence or the latest message (see ACH_O_LAST option to
ach_get()).


Out of curiosity, what is your potential use case?

Cheers,
-ntd

[climbfo...@gmail.com]

I'm actually working on a PLC software for high voltage industry. Many task are enclosed on this software and each tasks have to communicate with each other.

We use the process priority mechanism in order to unstack messages in a process that have a highest priority before unstacking messages in other process.

In addition of that, one process have a list of received messages to unstack/handle. In this list some messages may be urgent (surge, overcurrent, etc.). These messages have to be handled before all other messages. For example process A received A1,A2,A3,A4,A5,A6,A7,A8[urgent],A9, A10[urgent],A11, A12. In this list, A8 and A10 are ugent and have to be processed before all other messages in the process A.

That's why I'm looking for an IPC system that I can use in realtime context, with process priority and message priority features.

Best regards,
ZRA

[chaotic....@gmail.com]

Sorry to interject, but I thought I'd offer a suggestion.

If the classes of different message priorities are finite, countable, and predictable, then you could create a separate ach channel for each message priority, and then spawn an individual process to handle each channel (since ach does not currently support multiplexed waiting on ach channels). Using sched_setscheduler(), you can set the priority level of each process, giving the highest scheduling priority to the process which handles the highest priority channel.

As long as the way messages get handled lends itself to a multi-process framework, this seems like the best approach.

--Grey