org.jgrapht loading errors

[Brian Karlak]

Hello --

I am trying to run CoGroup join operation with cascading-0.8.2 on an hadoop-0.18.1 cluster.  The pipe compiles and the job launches. However, all tasks fail with the following error

cascading.flow.FlowException: internal error during mapper configuration
	at cascading.flow.FlowMapper.configure(FlowMapper.java:60)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:33)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
	at
 org.apache.hadoop.mapred.MapTask.run(MapTask.java:223)
	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
Caused by: java.lang.NoClassDefFoundError: org/jgrapht/ext/EdgeNameProvider
	at cascading.flow.stack.FlowMapperStack.(FlowMapperStack.java:72)
	at cascading.flow.FlowMapper.configure(FlowMapper.java:53)
	... 7 more
Caused by: java.lang.ClassNotFoundException: org.jgrapht.ext.EdgeNameProvider
	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
	... 9 more

I have checked, and jgrapht-jdk1.5.jar is in my build.  Any thoughts as to what may be going on, and how I can fix it?

Jobs without joins appear to work normally.

Thanks in advance for whatever help you can provide.

Brian

[Chris K Wensel]

Make sure that your application jar has a 'lib' directory with cascading and it's dependent libraries. Then make sure you set

// set the current job jar

Properties properties = new Properties();

FlowConnector.setJarClass( properties, Main.class );

FlowConnector flowConnector = new FlowConnector( properties );

Where the Main.class is your class with the 'main' function.

There should also be a log message in the console displaying the jar being used as the application jar. 

hope this helps,

chris

--

Chris K Wensel

ch...@wensel.net

http://chris.wensel.net/

http://www.cascading.org/

[Brian Karlak]

Thanks, I see my error now.  I had the proper preamble in place, but a silly cut/paste error had me using a new, unparameterized FlowConnector when setting up my connect().

Many thanks for the help!

Brian

	at cascading.flow.stack.FlowMapperStack. (FlowMapperStack.java:72)

	at cascading.flow.FlowMapper.configure(FlowMapper.java:53)
	... 7 more
Caused by: java.lang.ClassNotFoundException: org.jgrapht.ext.EdgeNameProvider
	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
	... 9 more

I have checked, and jgrapht-jdk1.5.jar is in my build.  Any thoughts as to what may be going on, and how I can fix it?

Jobs without joins appear to work normally.

Thanks in advance for whatever help you can provide.

Brian

[Chris K Wensel]

Glad its working for you. Have fun!

chris