cascading.flow.planner.PlannerException: could not build flow from assembly: [unable to read fields from tap: Hfs on hortonworks's sandbox

[Vinit Patel]

Hi,

I am trying to run demo described at http://hortonworks.com/hadoop-tutorial/cascading-hortonworks-data-platform-2-1/.  At then end when I run " hadoop jar ./build/libs/impatient.jar data/rain.txt output/wc", it throws the following exception:

[cascade@sandbox part2]$ pwd

/home/cascade/Impatient/part2

[cascade@sandbox part2]$

[cascade@sandbox part2]$ hadoop jar ./build/libs/impatient.jar data/rain.txt output/wc

14/04/28 11:07:17 INFO util.HadoopUtil: resolving application jar from found main method on: impatient.Main

14/04/28 11:07:17 INFO planner.HadoopPlanner: using application jar: /home/cascade/Impatient/part2/./build/libs/impatient.jar

14/04/28 11:07:17 INFO property.AppProps: using app.id: 1F0424167BE042C0A053A2504730EE17

Exception in thread "main" cascading.flow.planner.PlannerException: could not build flow from assembly: [unable to read fields from tap: Hfs["TextLine[['line']->[ALL]]"]["hdfs://sandbox.hortonworks.com:8020/user/cascade/data/rain.txt"], does not exist]

        at cascading.flow.planner.FlowPlanner.handleExceptionDuringPlanning(FlowPlanner.java:576)

        at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:265)

        at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:80)

        at cascading.flow.FlowConnector.connect(FlowConnector.java:459)

        at impatient.Main.main(Main.java:77)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Caused by: cascading.tap.TapException: unable to read fields from tap: Hfs["TextLine[['line']->[ALL]]"]["hdfs://sandbox.hortonworks.com:8020/user/cascade/data/rain.txt"], does not exist

        at cascading.scheme.util.DelimitedParser.parseFirstLine(DelimitedParser.java:295)

        at cascading.scheme.hadoop.TextDelimited.retrieveSourceFields(TextDelimited.java:968)

        at cascading.tap.Tap.retrieveSourceFields(Tap.java:361)

        at cascading.flow.BaseFlow.retrieveSourceFields(BaseFlow.java:215)

        at cascading.flow.BaseFlow.<init>(BaseFlow.java:181)

        at cascading.flow.hadoop.HadoopFlow.<init>(HadoopFlow.java:98)

        at cascading.flow.hadoop.planner.HadoopPlanner.createFlow(HadoopPlanner.java:211)

        at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:224)

        ... 8 more

I noticed that it is looking for "/user/cascade/data/rain.txt" folder. However, this folder does not exist. Do I need to setup any anything to reflect the proper input file path?

[Vinit Patel]

I found the solution. The step which I was missing was creating/moving the data file to HDFS. 

So, I run the following commands to create folder in HDFS

$  hadoop fs -mkdir /user/cascade/data

$  hadoop fs -copyFromLocal ./data/rain.txt /user/cascade/data/rain.txt

Then, I run " hadoop jar ./build/libs/impatient.jar data/rain.txt output/wc". It worked!!

[Andre Kelpe]

Glad you figured it out and thanks for posting the solution for future users.

- André

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/6527b0c9-9edc-4388-88b9-5c91a5e17c5d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
André Kelpe
an...@concurrentinc.com
http://concurrentinc.com