cascading impatient part1 - compile error

413 views
Skip to first unread message

Tom Hood

unread,
Dec 29, 2012, 8:09:30 PM12/29/12
to cascadi...@googlegroups.com
Hi,

I'm new to cascading, gradle, and git.  I git-cloned the Impatient.git and attempted to build part 1.

Output of "gradle clean jar" is:

:clean UP-TO-DATE
:compileJava
/home/hood/work/cascading/Impatient/part1/src/main/java/impatient/Main.java:50: error: cannot access JobConf
    Tap inTap = new Hfs( new TextDelimited( true, "\t" ), inPath );
                ^
  class file for org.apache.hadoop.mapred.JobConf not found
1 error
:compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 22.052 secs


If I simply copy the part1 Main.java into a different directory and type "javac Main.java" from that directory it compiles.

It almost seems like my CLASSPATH is being overridden by gradle such that by the time it attempts to compile Main.java the hadoop-core-1.0.4.jar is missing from the CLASSPATH ??

I'm running on OpenIndiana (through VirtualBox on a Windows 7 laptop).

Here's some environment settings:
> java -version
java version "1.7.0_10"
Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
Java HotSpot(TM) Server VM (build 23.6-b04, mixed mode)
 
> echo $CLASSPATH
.:/home/hood/Downloads/avro-java/avro-1.7.3.jar:/home/hood/Downloads/avro-java/jackson/jackson-core-asl-1.9.11.jar:/home/hood/Downloads/avro-java/jackson/jackson-mapper-asl-1.9.11.jar:/home/hood/Downloads/cascading-2.1.1/cascading-core-2.1.1.jar:/home/hood/Downloads/cascading-2.1.1/cascading-hadoop-2.1.1.jar:/home/hood/Downloads/cascading-2.1.1/cascading-local-2.1.1.jar:/home/hood/cots/hadoop-1.0.4/hadoop-core-1.0.4.jar
 
> echo $HADOOP_HOME/
/home/hood/cots/hadoop-1.0.4/

Anyway, I'll keep looking at it.  I'm sure there's some very basic thing I'm missing, possibly java related (I'm a C++ programmer learning java at the same time).

Any help would be appreciated.

Thanks,
-- Tom

Paco Nathan

unread,
Dec 29, 2012, 8:36:10 PM12/29/12
to cascadi...@googlegroups.com
Hi Tom,

Which version of Gradle to you have? There was an issue with 1.1; however, both 1.0 and 1.2 work fine.

There are several variables (OpenIndiana, Windows 7, Java 1.7, etc.) which are quite different from what I've run before -- so I'm quite glad to hear about this (and get it running for you)

The following would be good to run, to help troubleshoot:

    gradle --debug clean jar

That should help identify where/what is happening. Part 1 is about as vanilla as Cascading apps get, so it's very likely an issue with Gradle or the Java set up.

Thanks,
Paco



--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To view this discussion on the web visit https://groups.google.com/d/msg/cascading-user/-/gCG4pR4Ke6cJ.
To post to this group, send email to cascadi...@googlegroups.com.
To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.

Tom Hood

unread,
Dec 29, 2012, 8:41:29 PM12/29/12
to cascadi...@googlegroups.com

> gradle -version

------------------------------------------------------------
Gradle 1.3
------------------------------------------------------------

Gradle build time: Tuesday, November 20, 2012 11:37:38 AM UTC
Groovy: 1.8.6
Ant: Apache Ant(TM) version 1.8.4 compiled on May 22 2012
Ivy: 2.2.0
JVM: 1.7.0_10 (Oracle Corporation 23.6-b04)
OS: SunOS 5.11 x86

-- Tom

Tom Hood

unread,
Dec 29, 2012, 8:59:21 PM12/29/12
to cascadi...@googlegroups.com
Thanks for the response.

Attached is the output of "gradle --debug clean jar"

I'll try installing gradle 1.0 and java 1.6 since we know that worked for you.

-- Tom


On Saturday, December 29, 2012 5:09:30 PM UTC-8, Tom Hood wrote:
gradle-debug-clean-jar.txt

Tom Hood

unread,
Dec 29, 2012, 10:03:29 PM12/29/12
to cascadi...@googlegroups.com

The problem goes away if I use java 1.6.0_38

Both gradle 1.0 and 1.3 can build part1 with java 1.6.0_38.  Neither can build it with java 1.7.0_10

Thanks for the help.  Now I can actually start being impatient :-) 

-- Tom

Paco Nathan

unread,
Dec 29, 2012, 10:15:12 PM12/29/12
to cascadi...@googlegroups.com
Java 1.7, no less...

Also, which distribution of Hadoop are you using? 
Since it was having troubles at:

  class file for org.apache.hadoop.mapred.JobConf not found

I'm re-trying these examples with Gradle 1.3 to see if there are any changes. FWIW, I've heard of at least one issue (with Scala examples) which is supposed to be fixed in 1.4

Meanwhile, welcome to Impatient!


--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To view this discussion on the web visit https://groups.google.com/d/msg/cascading-user/-/6ZJN-LE7-zcJ.

Tom Hood

unread,
Dec 29, 2012, 10:32:07 PM12/29/12
to cascadi...@googlegroups.com

Here's the hadoop version:

Hadoop 1.0.4
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290
Compiled by hortonfo on Wed Oct  3 05:13:58 UTC 2012
From source with checksum fe2baea87c4c81a2c505767f3f9b71f4

-- Tom
Reply all
Reply to author
Forward
0 new messages