Custom Variables in class Function

[Ching-Lun Lin]

Hi,

I have a function that has a static variable which is defined in the constructor. But when the function is operating, the defined variable is not passed to slave nodes.

I would really appreciate if anyone can advice me on if I am setting the variable correctly, and if this is the expected behavior. Thanks in advance.

Context: my cascading job is executed on AWS EMR, using Cascading 2.6.

The output is:

flag at MyFunction(): true
flag at prepare(): false
flag at operate(): false // repeat n times, n is number of tuples
flag at cleanup(): false

The function is triggered by:

Pipe pipe = new Each(sourcePipe, Fields.ALL, new MyFunction(true), Fields.ALL);

And the function looks like this:

public class MyFunction extends BaseOperation<Tuple> implements Function<Tuple> {
  private static boolean flag = false;
  
  public MyFunction(boolean flag) { 
    MyFunction.flag = flag;
    System.out.println("flag at MyFunction(): " + flag);
  } 
  @Override
  public void prepare(FlowProcess flowProcess, OperationCall<Tuple> operationCall) {
    System.out.println("flag at prepare(): " + flag);
  }
  @Override 
  public void operate(FlowProcess flowProcess, FunctionCall<Tuple> functionCall) {
    System.out.println("flag at operate(): " + flag);
  } 
  @Override 
  public void cleanup(FlowProcess flowProcess, OperationCall<Tuple> operationCall) { 
      System.out.println("flag at cleanup(): " + flag)
  } 
}

[Andre Kelpe]

You have to put that in the properties object, which you pass to the FlowConnector. You can then read the values back in the #prepare method of your custom function. The FlowProcess instance will give you access to all properties of your current flow.

- André

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/c9d603ba-8f98-40ef-bf69-679758cade44%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

André Kelpe
an...@concurrentinc.com
http://concurrentinc.com

[Ching-Lun Lin]

Thank you André, it works perfectly for my need!