Problem Logging to STDOUT with the Debug function

[Ted Naleid]

I'm new to cascading, and I'm having trouble getting the `Debug`
output working. I'm using the WordCount example from the "Diving In"
section in the cascading user guide. The example works great, but
when I put a Debug statement in the flow, I'm not getting any output
in the stdout section in the jobtracker UI.

assembly = new Each( assembly, new Fields( "line" ),
function );

Debug debug = new Debug(Debug.Output.STDOUT, true);
debug.setPrintTupleEvery(1);
debug.setPrintFieldsEvery(1);
assembly = new Each( assembly, DebugLevel.VERBOSE, debug );

// group the Tuple stream by the "word" value
assembly = new GroupBy( assembly, new Fields( "word" ) );
...
FlowConnector flowConnector = new FlowConnector();
FlowConnector.setDebugLevel( properties, DebugLevel.VERBOSE );
Flow flow = flowConnector.connect( "word-count", source, sink,
assembly );

I'm following the example in the user guide:
http://www.cascading.org/1.2/userguide/html/ch07s02.html

When I run a regular map reduce based WordCount example, and put in a
`System.out.println`, I do see output, so it appears to be a cascading
related thing I'm doing wrong.

I put a bunch of detail in this stackoverflow question:

http://stackoverflow.com/questions/9691005/how-can-i-get-hadoop-with-cascading-to-show-me-debug-log-output

Any help is appreciated.

Thanks,
Ted

P.S. I've looked at some of the other Debug problem related threads on
the usergroup, but didn't get any further with those:
http://groups.google.com/group/cascading-user/browse_thread/thread/4f2327d64a14d7b1/b3cc99849685f047
and http://groups.google.com/group/cascading-user/browse_thread/thread/58ec83577b8eb5be/2c27c92ca8cc6eae

[Gary Snider]

Can you try something simpler?

(get rid of the first 3 debug lines and change the assembly as follows)
assembly = new Each( assembly, new Debug("hello",true)); 

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To post to this group, send email to cascadi...@googlegroups.com.
To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.

[Ted Naleid]

On Mar 13, 2012, at 4:16 PM, Gary Snider wrote:

Can you try something simpler?

(get rid of the first 3 debug lines and change the assembly as follows)
assembly = new Each( assembly, new Debug("hello",true)); 

Sweet, that works! Thanks!

Changing it to this:

        assembly = new Each( assembly, new Fields( "line" ), function );

        // simpler debug statement

        assembly = new Each( assembly, new Debug("hello", true) );

        assembly = new GroupBy( assembly, new Fields( "word" ) );

Outputs this in the jobdetails UI under stderr:

Task Logs: 'attempt_201203131143_0028_m_000000_0'

stdout logs

---

stderr logs

2012-03-13 16:21:41.304 java[78617:1903] Unable to load realm info from SCDynamicStore
hello: ['word']
hello: ['CYMBELINE']
<SNIP>

I had tried this directly from the docs, and that doesn't work for me (even though I've also set the FlowConnector debugLevel to VERBOSE):

assembly = new Each( assembly, DebugLevel.VERBOSE, new Debug() );

It seems that it's something related to the DebugLevel.VERBOSE from the documentation as when I try this, I still get no output:

assembly = new Each( assembly, DebugLevel.VERBOSE, new Debug("hello", true) );

Changing it to remove the DebugLevel also gives me output

assembly = new Each( assembly, new Debug() );

I can also get it to switch to stdout by doing this:

assembly = new Each( assembly, new Debug(Debug.Output.STDOUT) );

Thanks again for your help,  I appreciate it.

-Ted

[Chris K Wensel]

this very well could be a bug in the 2.0 planner. will try and repro and resolve it before 2.0 final.

chris

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To post to this group, send email to cascadi...@googlegroups.com.
To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.

--

Chris K Wensel

ch...@concurrentinc.com

http://concurrentinc.com

[Chris K Wensel]

looked at your code more closely, you aren't actually giving the properties to the FlowConnector

       FlowConnector flowConnector = new FlowConnector();
       FlowConnector.setDebugLevel( properties, DebugLevel.VERBOSE );

should read

       FlowConnector.setDebugLevel( properties, DebugLevel.VERBOSE );

       FlowConnector flowConnector = new FlowConnector( properties ); // <<<<<<

ckw