What are the currect library dependencies for Cloudera Distribution CDH4.1

46 views
Skip to first unread message

Venkat Reddy

unread,
Sep 15, 2014, 6:46:05 AM9/15/14
to scoobi...@googlegroups.com
Hi,

I have the following Cloudera Hadoop Distribution:
Hadoop 2.0.0-cdh4.1.1

I am using sbt 0.13 for scoobi.

I was able to successfully compile and run WordCount app given at the link:
https://github.com/NICTA/scoobi/blob/master/examples/wordCount/src/main/scala/WordCount.scala

It was run in LocalJobRunner mode due to some default library dependencies 
specified in build.sbt as given in the following link:
http://nicta.github.io/scoobi/guide/com.nicta.scoobi.guide.QuickStart.html#Quick+Start


I changed these library dependencies so that the job runs on the hadoop cluster (CDH4.1).
I made the changes as per the following link:
http://nicta.github.io/scoobi/guide/com.nicta.scoobi.guide.Deployment.html#Other+Hadoop+versions

Here is build.sbt:
name := "MyApplication"

version := "1.0"

scalaVersion := "2.10.3"


libraryDependencies ++= Seq(
    "com.nicta"                %% "scoobi"            % "0.8.5-cdh4" intransitive(),
    "org.apache.hadoop"        %  "hadoop-client"     % "2.0.0-mr1-cdh4.0.1" exclude("asm", "asm") % "provided",
    "org.apache.hadoop"        %  "hadoop-core"       % "2.0.0-mr1-cdh4.0.1",                    
    "org.apache.avro"          %  "avro-mapred"       % "1.7.4"  classifier "hadoop2",
    "org.scala-lang"           %  "scala-compiler"    % "2.10.3",
    "com.thoughtworks.xstream" %  "xstream"           % "1.4.4" intransitive(),
    "javassist"                %  "javassist"         % "3.12.1.GA",
    "com.googlecode.kiama"     %% "kiama"             % "1.6.0",
    "com.chuusai"              %  "shapeless_2.10.2"  % "2.0.0-M1"  )


resolvers ++= Seq("sonatype releases" at "http://oss.sonatype.org/content/repositories/releases",
                  "sonatype snapshots" at "http://oss.sonatype.org/content/repositories/snapshots”)


——————
I get the following error:
Unspecified value parameter name.
    "org.apache.hadoop"        %  "hadoop-client"     % "2.0.0-mr1-cdh4.0.1" exclude("asm", "asm") % "provided",
                                                                             ^
[error] Type error in expression
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? 

What are the library dependancies that I need to use for CDH4.1? 

With the best regards,
Venkat
Reply all
Reply to author
Forward
0 new messages