Running Multiple Processing Piplines

12 views
Skip to first unread message

Christopher Miles

unread,
Sep 28, 2011, 7:43:11 AM9/28/11
to py...@googlegroups.com
Hello,

If I want to have multiple processing pipelines (perhaps one for each type of document collection), it looks like the most straightforward solution is to simply run multiple instances of Pypes, each with the one pipeline of operations. Is this the recommended way of getting multiple processing pipelines? I want to make sure I'm not missing something. 

Thank you
--
Miles

Eric Gaumer

unread,
Oct 1, 2011, 6:44:31 AM10/1/11
to py...@googlegroups.com
Yes that's correct. Being graph oriented, you can also create different branches of logic within the same instance. You can then write a "routing" stage that sends documents down specific paths based on some routing key (e.g., a field identifying the collection type).

Both approaches should work fine.

-Eric
Reply all
Reply to author
Forward
0 new messages