help knowing which components to use...

28 views
Skip to first unread message

Ryan Gahl

unread,
Feb 23, 2015, 6:36:24 PM2/23/15
to flow-based-...@googlegroups.com

(sorry if the image I'm trying to attach here is too big or doesn't come through)

So, I have a graph (as shown below) and I'm having trouble figuring out which components are the right ones to use to complete the puzzle.

The fetchObjects component in the upper left emits a packet for each item found in an S3 bucket. I then am extracting the 'Key' property.

So here's the problem. I want to send this value to the 3 boxes in the lower right, but I need to do it in such a way that I create a unique read/write pair of streams for each emitted key. In other words I am not sure how to synchronize these components such that I guarantee that the read/write streams that get fed into the last box (which simply pipes a read stream to a write stream) are for the same key...

Am I overthinking this? Is it as simple as just connecting the output from ExtractProperty to both inputs (assuming I also insert a 'build filename' component that creates a path for the writeStream component)? Or do I need core/Split or something from the noflo-groups library? 


Reply all
Reply to author
Forward
0 new messages