Hi everyone,
I couldn't find an answer anywhere and decided to ask here: is there a way to force flume optimizer to fuse together functions in C++?
Example:
PCollection.ParDo("Read Data", new ReadDataFn).ParDo("Push To Sevice", new PushFn).ParDo("Mark BTable with pusher-returned IDs for respective Pushed packages", new MarkPushedFn);
I would like to write result ID coming from something pushed to a service in one function as close together to marking it in my own records so as to avoid duplicating pushes in the future.
Is there a way to mark a chain for force-fusing somehow?
Thanks!