Any way to force fuse stages?

7 views
Skip to first unread message

Aleksey Boyko

unread,
Feb 5, 2016, 8:37:11 PM2/5/16
to flume-user
Hi everyone,

I couldn't find an answer anywhere and decided to ask here: is there a way to force flume optimizer to fuse together functions in C++?

Example:

PCollection.ParDo("Read Data", new ReadDataFn).ParDo("Push To Sevice", new PushFn).ParDo("Mark BTable with pusher-returned IDs for respective Pushed packages", new MarkPushedFn);

I would like to write result ID coming from something pushed to a service in one function as close together to marking it in my own records so as to avoid duplicating pushes in the future.

Is there a way to mark a chain for force-fusing somehow?

Thanks!
Reply all
Reply to author
Forward
0 new messages