java.lang.IllegalArgumentException: no such vertex in graph

[Chris Curtin]

Not sure where to start with this one.

This application was written against 1.0.4. 0.19.?? and then put on hold about a year ago. It worked back then.

Upgraded it today to 0.20.2, Cascading 1.2.1. One change: started using GlobHFS with a directory filter instead of manually reading the source files and creating a Tap for each file, then adding to a MultiSourceTap.

I get the following error as each task fails. It doesn't seem to be reaching any of my code. The GlobHFS is matching around 300 files.

java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
	... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
	... 10 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
	... 13 more
Caused by: cascading.flow.FlowException: internal error during mapper configuration
	at cascading.flow.FlowMapper.configure(FlowMapper.java:65)
	... 18 more
Caused by: java.lang.IllegalArgumentException: no such vertex in graph
	at org.jgrapht.graph.AbstractGraph.assertVertexExist(Unknown Source)
	at org.jgrapht.graph.AbstractBaseGraph$DirectedSpecifics.getEdgeContainer(Unknown Source)
	at org.jgrapht.graph.AbstractBaseGraph$DirectedSpecifics.outgoingEdgesOf(Unknown Source)
	at org.jgrapht.graph.AbstractBaseGraph.outgoingEdgesOf(Unknown Source)
	at cascading.flow.FlowStep.getNextScopes(FlowStep.java:465)
	at cascading.flow.stack.FlowMapperStack.buildStack(FlowMapperStack.java:93)
	at cascading.flow.stack.FlowMapperStack.<init>(FlowMapperStack.java:85)
	at cascading.flow.FlowMapper.configure(FlowMapper.java:58)

And idea what is going wrong?

(the cluster works for all the other Cascading jobs we are running, so I don't think it is a cluster issue)

Thanks,

Chris

[Chris Curtin]

Definitely related to the GlobHfs. Changed back to a list of the HDFS directory with a tap per file and it is working.

[Chris K Wensel]

I'll see if I can reproduce this sometime this weekend.

ckw

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To post to this group, send email to cascadi...@googlegroups.com.
To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.

--

Chris K Wensel

ch...@concurrentinc.com

http://www.concurrentinc.com

-- Concurrent, Inc. offers mentoring, support, and licensing for Cascading

[GregG]

Any luck in reproducing this error? I get the exact same error when
adding a PathFilter to a GlobHfs in cascading 1.2.1 and hadoop 0.20.2.
Thanks.

-Greg

On Mar 10, 8:25 pm, Chris K Wensel <ch...@wensel.net> wrote:
> I'll see if I can reproduce this sometime this weekend.
>
> ckw
>
> On Mar 10, 2011, at 10:23 AM, Chris Curtin wrote:
>
>
>
>
>
>
>
>
>
> > Definitely related to the GlobHfs. Changed back to a list of the HDFS directory with a tap per file and it is working.
>
> > On Thursday, March 10, 2011 1:08:19 PM UTC-5, Chris Curtin wrote:
> > Not sure where to start with this one.
>
> > This application was written against 1.0.4. 0.19.?? and then put on hold about a year ago. It worked back then.
>
> > Upgraded it today to 0.20.2, Cascading 1.2.1. One change: started using GlobHFS with a directory filter instead of manually reading the source files and creating a Tap for each file, then adding to a MultiSourceTap.
>
> > I get the following error as each task fails. It doesn't seem to be reaching any of my code. The GlobHFS is matching around 300 files.
>
> > --
> > You received this message because you are subscribed to the Google Groups "cascading-user" group.
> > To post to this group, send email to cascadi...@googlegroups.com.
> > To unsubscribe from this group, send email to cascading-use...@googlegroups.com.

> > For more options, visit this group athttp://groups.google.com/group/cascading-user?hl=en.
>
> --
> Chris K Wensel
> ch...@concurrentinc.comhttp://www.concurrentinc.com

[Chris K Wensel]

I've made an attempt at fixing this.

please test this revision of 1.2.3-wip and let me know if things are resolved.

http://www.concurrentinc.com/downloads/

ckw

> For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.

[GregG]

Sorry, I still get the same error with 1.2.3-wip-73 (using conjars).

java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:
354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)

at org.apache.hadoop.mapred.LocalJobRunner
$Job.run(LocalJobRunner.java:177)

Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:

39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)

at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:
34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:

39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)

at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
88)
... 13 more
Caused by: cascading.flow.FlowException: internal error during mapper
configuration
at cascading.flow.FlowMapper.configure(FlowMapper.java:65)
... 18 more
Caused by: java.lang.IllegalArgumentException: no such vertex in graph
at org.jgrapht.graph.AbstractGraph.assertVertexExist(Unknown
Source)
at org.jgrapht.graph.AbstractBaseGraph
$DirectedSpecifics.getEdgeContainer(Unknown Source)
at org.jgrapht.graph.AbstractBaseGraph
$DirectedSpecifics.outgoingEdgesOf(Unknown Source)
at org.jgrapht.graph.AbstractBaseGraph.outgoingEdgesOf(Unknown
Source)
at cascading.flow.FlowStep.getNextScopes(FlowStep.java:465)
at
cascading.flow.stack.FlowMapperStack.buildStack(FlowMapperStack.java:
93)
at
cascading.flow.stack.FlowMapperStack.<init>(FlowMapperStack.java:85)
at cascading.flow.FlowMapper.configure(FlowMapper.java:58)

... 18 more

-Greg

[GregG]

If it helps, here is a simple test class I created that can be used to
reproduce this error:


import java.io.IOException;
import java.io.Serializable;
import java.util.Properties;

import cascading.flow.Flow;
import cascading.flow.FlowConnector;
import cascading.pipe.Pipe;
import cascading.scheme.Scheme;
import cascading.scheme.TextLine;
import cascading.tap.GlobHfs;
import cascading.tap.Hfs;
import cascading.tap.SinkMode;
import cascading.tap.Tap;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.PathFilter;


public class TestPathFilter
{
public static class PathFilterTest implements PathFilter,
Serializable
{
public boolean accept(Path path)
{
return true;
}
}

public static final void main(String[] args)
throws IOException
{
Properties properties = new Properties();
FlowConnector.setApplicationJarClass(properties,
TestPathFilter.class);
FlowConnector flowConnector = new FlowConnector(properties);

Scheme input = new TextLine();
Tap in = new GlobHfs(input, "in", new PathFilterTest());

Scheme output = new TextLine();
Tap out = new Hfs(output, "out", SinkMode.REPLACE);

Pipe assembly = new Pipe("test");

Flow flow = flowConnector.connect("test-path", in, out, assembly);
flow.complete();
}
}

Just create a dir called "in" and put a file in there and run this and
you should hit the error, even with cascading-1.2.3-wip-73.

Thanks.

-Greg

[Chris K Wensel]

awesome, I'll give it a try asap.

thanks!
chris

[Chris K Wensel]

turns out this isn't a Cascading bug, but a bug with the PathFilter implementation not implementing hashCode() and equals().

when the PathFilter is deserialized in the cluster, it has a new identity and thus a new hash, so it won't be found in the previous hash bucket in any internal hashmap.

I'll attempt at improving the error messages here in the next release so its more obvious.

ckw

On Mar 22, 2011, at 3:52 PM, GregG wrote:

[GregG]

Great, thanks. I added hashCode() and equals() to my (real)
implementation and it seems to work fine now, even in cascading 1.2.2.
Thanks.

-Greg

> > For more options, visit this group athttp://groups.google.com/group/cascading-user?hl=en.
>
> --
> Chris K Wensel
> ch...@concurrentinc.comhttp://www.concurrentinc.com

[Chris K Wensel]

made an attempt at offering an improved error message. will be in the next build.

> For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.