starfish on hadoop 1.1.2

[liwu...@gmail.com]

Hello, everyone!

Can Starfish run on Hadoop 1.1.2?

When I test it using wordcount programme there are some foult as below:

No enum const class edu.duke.starfish.profile.profileinfo.execution.profile.enums.MRCounter.PHYSICAL_MEMORY_BYTES

No enum const class edu.duke.starfish.profile.profileinfo.execution.profile.enums.MRCounter.COMMITTED_HEAP_BYTES

No enum const class edu.duke.starfish.profile.profileinfo.execution.profile.enums.MRCounter.CPU_MILLISECONDS

No enum const class edu.duke.starfish.profile.profileinfo.execution.profile.enums.MRCounter.VIRTUAL_MEMORY_BYTES

java.lang.IllegalArgumentException: No enum const class edu.duke.starfish.profile.profileinfo.execution.profile.enums.MRCounter.VIRTUAL_MEMORY_BYTES

at java.lang.Enum.valueOf(Enum.java:196)

at edu.duke.starfish.profile.profileinfo.execution.profile.enums.MRCounter.valueOf(MRCounter.java:8)

at edu.duke.starfish.profile.profiler.loaders.MRJobHistoryLoader.parseMRTaskAttemptCounters(MRJobHistoryLoader.java:748)

at edu.duke.starfish.profile.profiler.loaders.MRJobHistoryLoader.populateMRTaskAttemptInfo(MRJobHistoryLoader.java:493)

at edu.duke.starfish.profile.profiler.loaders.MRJobHistoryLoader.populateMRReduceInfo(MRJobHistoryLoader.java:599)

at edu.duke.starfish.profile.profiler.loaders.MRJobHistoryLoader.populateClusterAndMRJobInfo(MRJobHistoryLoader.java:377)

at edu.duke.starfish.profile.profiler.loaders.MRJobHistoryLoader.loadJobAndClusterData(MRJobHistoryLoader.java:338)

at edu.duke.starfish.profile.profiler.loaders.MRJobHistoryLoader.getMRJobInfoWithDetails(MRJobHistoryLoader.java:143)

at edu.duke.starfish.profile.profiler.Profiler.exportProfileXMLFile(Profiler.java:498)

at edu.duke.starfish.profile.profiler.Profiler.gatherJobExecutionFiles(Profiler.java:258)

at org.apache.hadoop.mapreduce.Job.$btrace$BTraceProfiler$onJob_waitForCompletion_return(Job.java:112)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:537)

at org.apache.hadoop.examples.WordCount.run(WordCount.java:78)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)

at org.apache.hadoop.examples.WordCount.main(WordCount.java:85)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)

at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)

at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:55)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Expected 16 records for the REDUCE phase for attempt_201410310915_0006_r_000000_0

edu.duke.starfish.profile.profiler.loaders.tasks.MRTaskProfileLoader$ProfileFormatException: Expected 16 records for the REDUCE phase for attempt_201410310915_0006_r_000000_0

at edu.duke.starfish.profile.profiler.loaders.tasks.MRReduceProfileLoader.validateReduceRecords(MRReduceProfileLoader.java:582)

at edu.duke.starfish.profile.profiler.loaders.tasks.MRReduceProfileLoader.getAndValidateProfileRecords(MRReduceProfileLoader.java:411)

at edu.duke.starfish.profile.profiler.loaders.tasks.MRReduceProfileLoader.loadExecutionProfile(MRReduceProfileLoader.java:108)

at edu.duke.starfish.profile.profiler.loaders.tasks.MRTaskProfileLoader.loadExecutionProfile(MRTaskProfileLoader.java:152)

at edu.duke.starfish.profile.profiler.loaders.MRTaskProfilesLoader.loadTaskExecutionProfile(MRTaskProfilesLoader.java:210)

at edu.duke.starfish.profile.profiler.loaders.MRTaskProfilesLoader.loadExecutionProfile(MRTaskProfilesLoader.java:134)

at edu.duke.starfish.profile.profiler.Profiler.exportProfileXMLFile(Profiler.java:506)

at edu.duke.starfish.profile.profiler.Profiler.gatherJobExecutionFiles(Profiler.java:258)

at org.apache.hadoop.mapreduce.Job.$btrace$BTraceProfiler$onJob_waitForCompletion_return(Job.java:112)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:537)

at org.apache.hadoop.examples.WordCount.run(WordCount.java:78)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)

at org.apache.hadoop.examples.WordCount.main(WordCount.java:85)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)

at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)

at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:55)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

14/11/02 00:12:25 ERROR profiler.Profiler: Unable to create the job profile for job_201410310915_0006

14/11/02 00:12:25 INFO profiler.Profiler: Job profiling completed! Output directory: results

[liwu...@gmail.com]

anybody knows? 

在 2014年11月3日星期一UTC+8上午11时02分03秒，liwu...@gmail.com写道：

[liwu...@gmail.com]

anybody knows? 

在 2014年11月3日星期一UTC+8上午11时02分03秒，liwu...@gmail.com写道：

Hello, everyone!

[YoungKun Min]

I think Starfish cannot parse Hadoop 1.1.2 log format. What is version of yout Starfish? And try Starfish hadoop log analyzer, not the 0.3.0 binary.

[yu wang]

You'd better use starfish on the specific versions.

It throws exception on ProfileLoader or HistoryLoader, maybe it's because that starfish cannot parse the log format of the version you use.

在 2014年11月3日星期一UTC+8上午11时02分03秒，liwu...@gmail.com写道：

[揣立武]

Thanks to your answer.

I can't find the hadoop0.20.2 on the Internet and can't install the javafx1.2.1. 

So do you have the software packages?

--
You received this message because you are subscribed to a topic in the Google Groups "Starfish" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hadoop-starfish/gIXm6Yb-7fo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hadoop-starfi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[yu wang]

you can find all the historical archives here
https://archive.apache.org/dist/hadoop/core/

I'm glad if it helps.