On Sun, Apr 22, 2012 at 11:16 AM, Jonathan Lettvin
<jlet...@gmail.com> wrote:
Yes. Almost. I asked for help too early, and realized my error too late.
Basically, if I produced new arrays for every message or every memmap,
memory and disk would be consumed too rapidly.
So I am trying to figure out how to re-use arrays and memmaps.
I think I got what I needed from pubsub already, and I thank you for the simplicity of design.
Now I should get my own act together and use it properly.
No problem.
If you are interested in what I am doing, read on.
My project is a time-sequence image processing filter stack using unpublished algorithms.
The data I work on are largish (1e7-1e9 bytes).
Each filter has two state data planes (old and new).
Each time step causes two orthogonal actions:
- Each filter generates a new state by operations involving input data and old state.
- Each filter generates output by operations involving comparing old and new state.
During stage 1, the message data must not change but the states do.
During stage 2, the states must not change but the message data do.
I want to use messages to couple filters together into a stack
where a filter changes state only after all its server inputs are ready and
a filter generates its output only after all its client filters finish changing state.
I'm curious about this. Pubsub is designed to be used where there is data that has to be communicated without knowledge of the consumers (their type/class, how many, etc), and where consumers don't need to know the source of the data. Ie, fire and forget.
Based on that, it seems that you want the filters to be "black boxes" and each one will listen to a certain type of message to receive data it should process, and each one will send a message with the new data, is that right? Can you clarify what you mean by "message data" vs "state"?
Oliver