Possible to return property/value/state from a flow?

10 views
Skip to first unread message

PaulON

unread,
Jun 27, 2016, 9:17:07 AM6/27/16
to cascading-user
Hey,

is is possible to set a value/property (not a tuple) with a flow and then access it outside of the flow?

We are trying to log when an even happens within a flow so we can take specific action outside of cascading.

We could do it by having specific counters for each event type, but was hoping for an nicer approach?

I see flow.getProperty() but since there is no corresponding setProperty() I guess these are immutable job properties?

Cheers!
P.

Ken Krugler

unread,
Jun 27, 2016, 9:46:39 AM6/27/16
to cascadi...@googlegroups.com
Those are one-way (into the job).

And yes, as far as I know using counters is the easiest approach that works.

For specific situations in the past we’ve added a field to an operation’s output tuple that records (rare) events, and is null most of the time.

Then you can partitioned (or split) on that, but in general counters have been Good Enough.

— Ken

--------------------------
Ken Krugler
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr



Andre Kelpe

unread,
Jun 27, 2016, 12:35:05 PM6/27/16
to cascading-user
Flow.getProperty resolves settings that have been determined at plan
time. There is no mechanism to set a property cluster side and have it
show up on the client side. The only exception are counters, since
those are provided by the underlying platform as a way to communicate
things back.

- Andre
> --
> You received this message because you are subscribed to the Google Groups
> "cascading-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cascading-use...@googlegroups.com.
> To post to this group, send email to cascadi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/cascading-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/cascading-user/ea415e1b-dc05-41b0-8716-4750550cf61d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
André Kelpe
an...@concurrentinc.com
http://concurrentinc.com
Reply all
Reply to author
Forward
0 new messages