Minor bug w/cleanup code in Flow.run

3 views
Skip to first unread message

Ken Krugler

unread,
Oct 26, 2009, 8:44:52 AM10/26/09
to cascadi...@googlegroups.com
Hi Chris,

I'm using Cascading 1.0.16, and ran into a bug of my own where I had a non-serialized class.

This triggered the expected java.io.NotSerializableException when flow.run() was executing.

But in the run method's finally block, there's a call to cleanTemporaryFiles().

And this in turn fails because some of the temp files haven't been created yet, so a call to TempHfs.getPath() fails with an IllegalStateException:

TempHfs(Hfs).getPath() line: 268
TempHfs(Hfs).getQualifiedPath(JobConf) line: 278
TempHfs(Hfs).deletePath(JobConf) line: 342
TempHfs.deletePath(JobConf) line: 132
FlowStep.clean(JobConf) line: 350
Flow.cleanTemporaryFiles() line: 1046
Flow.run() line: 878

This gets dumped out to the console, which is confusing. Especially since the exception that gets thrown by Flow is the generic FlowException, which a generic message of "unhandled exception". It wraps the NotSerializableException, but you need to dump the stack trace to see what's really going on there.

Thanks,

-- Ken

--------------------------
Ken Krugler
TransPac Software, Inc.

Chris K Wensel

unread,
Oct 26, 2009, 11:51:29 AM10/26/09
to cascadi...@googlegroups.com
Thanks Ken.

ckw
Reply all
Reply to author
Forward
0 new messages