first run benchmark without data

[林柏年]

Hi, I am new to Cloudera & big bench

I was tried to run the first command: "$INSTALL_DIR/bin/bigBench" runBenchmark -m 4 -f 100 -s 2

but it printed the Error message: line 49: java command not found(I have installed java with Cloudera manager.)

After searching for solutions, I Manually setting JAVAHOME PATH "export JAVA_HOME=/usr/java/jdk1.7.0.nn.." by editing 

/etc/default/bigtop-util in server & agent host, but it still not working.

Is there any suggestion?  thanks

[Dave Jaffe]

Are you running the runBenchmark command from a client system for Hadoop. For example, can you run the command "hdfs fs -ls" from that system?

Dave Jaffe

dja...@vmware.com

--
You received this message because you are subscribed to the Google Groups "Big Data Benchmark for BigBench" group.
To unsubscribe from this group and stop receiving emails from it, send an email to big-bench+unsubscribe@googlegroups.com.
To post to this group, send email to big-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/big-bench/b3700f02-f42d-4070-89c4-4e76144a01ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Dave Jaffe

davej...@gmail.com

[林柏年]

Here is the my output message when running  hdfs fs -ls & hadoop fs -ls

[root@viper0 ~]# hdfs fs ls

Error: Could not find or load main class fs

[root@viper0 ~]# hadoop fs -ls

Warning: fs.defaultFs is not set when running "ls" command.

Found 16 items

-rw-------   1 root root       1821 2017-02-24 21:38 .bash_history

-rw-r--r--   1 root root         18 2009-05-20 06:45 .bash_logout

-rw-r--r--   1 root root        176 2009-05-20 06:45 .bash_profile

-rw-r--r--   1 root root        176 2004-09-22 23:59 .bashrc

-rw-r--r--   1 root root        100 2004-09-22 23:59 .cshrc

drwxr-----   - root root       4096 2017-02-24 17:16 .pki

-rw-------   1 root root       1024 2017-02-24 17:18 .rnd

drwx------   - root root       4096 2017-02-24 17:01 .ssh

-rw-r--r--   1 root root        129 2004-12-03 16:42 .tcshrc

-rw-------   1 root root       4957 2017-02-24 21:31 .viminfo

-rw-------   1 root root       1099 2008-06-05 20:01 anaconda-ks.cfg

drwxr-xr-x   - root root       4096 2017-02-24 21:00 big-bench

-rwxr--r--   1 root root     516309 2017-03-08 15:49 cloudera-manager-installer. bin

-rw-r--r--   1 root root         63 2017-03-19 23:13 index.html

-rw-r--r--   1 root root       8835 2008-06-05 20:01 install.log

-rw-r--r--   1 root root       3314 2008-06-05 19:59 install.log.syslog

Dave Jaffe於 2017年4月11日星期二 UTC+8上午7時01分35秒寫道：

Are you running the runBenchmark command from a client system for Hadoop. For example, can you run the command "hdfs fs -ls" from that system?

Dave Jaffe

dja...@vmware.com

On Mon, Apr 10, 2017 at 2:55 PM, 林柏年 <s101...@gmail.com> wrote:

Hi, I am new to Cloudera & big bench

I was tried to run the first command: "$INSTALL_DIR/bin/bigBench" runBenchmark -m 4 -f 100 -s 2

but it printed the Error message: line 49: java command not found(I have installed java with Cloudera manager.)

After searching for solutions, I Manually setting JAVAHOME PATH "export JAVA_HOME=/usr/java/jdk1.7.0.nn.." by editing 

/etc/default/bigtop-util in server & agent host, but it still not working.

Is there any suggestion?  thanks

--
You received this message because you are subscribed to the Google Groups "Big Data Benchmark for BigBench" group.

To unsubscribe from this group and stop receiving emails from it, send an email to big-bench+...@googlegroups.com.

To post to this group, send email to big-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/big-bench/b3700f02-f42d-4070-89c4-4e76144a01ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Dave Jaffe

davej...@gmail.com

[林柏年]

Here is the my output message when running  hdfs fs -ls & hadoop fs -ls

[root@viper0 ~]# hdfs fs ls

Error: Could not find or load main class fs

[root@viper0 ~]# hadoop fs -ls

Warning: fs.defaultFs is not set when running "ls" command.

Found 16 items

-rw-------   1 root root       1821 2017-02-24 21:38 .bash_history

-rw-r--r--   1 root root         18 2009-05-20 06:45 .bash_logout

-rw-r--r--   1 root root        176 2009-05-20 06:45 .bash_profile

-rw-r--r--   1 root root        176 2004-09-22 23:59 .bashrc

-rw-r--r--   1 root root        100 2004-09-22 23:59 .cshrc

drwxr-----   - root root       4096 2017-02-24 17:16 .pki

-rw-------   1 root root       1024 2017-02-24 17:18 .rnd

drwx------   - root root       4096 2017-02-24 17:01 .ssh

-rw-r--r--   1 root root        129 2004-12-03 16:42 .tcshrc

-rw-------   1 root root       4957 2017-02-24 21:31 .viminfo

-rw-------   1 root root       1099 2008-06-05 20:01 anaconda-ks.cfg

drwxr-xr-x   - root root       4096 2017-02-24 21:00 big-bench

-rwxr--r--   1 root root     516309 2017-03-08 15:49 cloudera-manager-installer. bin

-rw-r--r--   1 root root         63 2017-03-19 23:13 index.html

-rw-r--r--   1 root root       8835 2008-06-05 20:01 install.log

-rw-r--r--   1 root root       3314 2008-06-05 19:59 install.log.syslog

Dave Jaffe於 2017年4月11日星期二 UTC+8上午7時01分35秒寫道：

Are you running the runBenchmark command from a client system for Hadoop. For example, can you run the command "hdfs fs -ls" from that system?

Dave Jaffe

dja...@vmware.com

On Mon, Apr 10, 2017 at 2:55 PM, 林柏年 <s101...@gmail.com> wrote:

Hi, I am new to Cloudera & big bench

I was tried to run the first command: "$INSTALL_DIR/bin/bigBench" runBenchmark -m 4 -f 100 -s 2

but it printed the Error message: line 49: java command not found(I have installed java with Cloudera manager.)

After searching for solutions, I Manually setting JAVAHOME PATH "export JAVA_HOME=/usr/java/jdk1.7.0.nn.." by editing 

/etc/default/bigtop-util in server & agent host, but it still not working.

Is there any suggestion?  thanks

--

--

Dave Jaffe

davej...@gmail.com

[林柏年]

Here is my usersetting.conf : 

## ==========================
## JAVA environment
## ==========================
export BIG_BENCH_JAVA="java"


## ==========================
## default settings for benchmark
## ==========================
export BIG_BENCH_DEFAULT_DATABASE="bigbench"
export BIG_BENCH_DEFAULT_ENGINE="hive"
export BIG_BENCH_DEFAULT_MAP_TASKS="80"
export BIG_BENCH_DEFAULT_SCALE_FACTOR="1000"
export BIG_BENCH_DEFAULT_NUMBER_OF_PARALLEL_STREAMS="2"
export BIG_BENCH_DEFAULT_BENCHMARK_PHASE="run_query"


## ==========================
## HADOOP environment
## ==========================


##folder containing the cluster setup *-site.xml files like core-site.xml
export BIG_BENCH_HADOOP_CONF="/etc/hadoop/conf.cloudera.hdfs"
export BIG_BENCH_HADOOP_LIBS_NATIVE="/opt/cloudera/parcels/CDH/lib/hadoop/lib/native"




## ==========================
## HDFS config and paths
## ==========================
export BIG_BENCH_USER="$USER"
export BIG_BENCH_HDFS_ABSOLUTE_PATH="/user/$BIG_BENCH_USER" ##working dir of benchmark.
export BIG_BENCH_HDFS_RELATIVE_HOME="benchmarks/bigbench"
export BIG_BENCH_HDFS_RELATIVE_INIT_DATA_DIR="$BIG_BENCH_HDFS_RELATIVE_HOME/data"
export BIG_BENCH_HDFS_RELATIVE_REFRESH_DATA_DIR="$BIG_BENCH_HDFS_RELATIVE_HOME/data_refresh"
export BIG_BENCH_HDFS_RELATIVE_QUERY_RESULT_DIR="$BIG_BENCH_HDFS_RELATIVE_HOME/queryResults"
export BIG_BENCH_HDFS_RELATIVE_TEMP_DIR="$BIG_BENCH_HDFS_RELATIVE_HOME/temp"
export BIG_BENCH_HDFS_ABSOLUTE_HOME="$BIG_BENCH_HDFS_ABSOLUTE_PATH/$BIG_BENCH_HDFS_RELATIVE_HOME"
export BIG_BENCH_HDFS_ABSOLUTE_INIT_DATA_DIR="$BIG_BENCH_HDFS_ABSOLUTE_PATH/$BIG_BENCH_HDFS_RELATIVE_INIT_DATA_DIR"

How do I configure these two statement?

export BIG_BENCH_HADOOP_CONF="/etc/hadoop/conf.cloudera.hdfs"
export BIG_BENCH_HADOOP_LIBS_NATIVE="/opt/cloudera/parcels/CDH/lib/hadoop/lib/native"

These are files in under my /etc/hadoop/  :

[root@viper0 conf]# cd /etc/hadoop/
[root@viper0 hadoop]# ls
conf  conf.dist  conf.empty  conf.impala
[root@viper0 hadoop]# cd conf
[root@viper0 conf]# ls
capacity-scheduler.xml  hadoop-metrics2.properties  mapred-queues.xml.template  ssl-server.xml.example
configuration.xsl       hadoop-metrics.properties   mapred-site.xml             yarn-env.sh
container-executor.cfg  hadoop-policy.xml           mapred-site.xml.template    yarn-site.xml
core-site.xml           hdfs-site.xml               slaves
fair-scheduler.xml      log4j.properties            ssl-client.xml.example

should I export BIG_BENCH_HADOOP_CONF="/etc/hadoop/conf/" ? Also, there is nothing under my /opt/cloudera/parcels/

Is there anything wrong with my installation of CM ? Thanks

林柏年於 2017年4月11日星期二 UTC+8上午3時55分44秒寫道：

[Michael Frank]

Hi,

could you post a little bit more context around the error message? - everything on stdout form start, including used command line, till the error.

What do you get if you execute the following command line from the same shell/host you tried to run bigbench:

java -version

If that fails or it is not an open/oracle jdk >=1.7 or 1.8 , that might be your issue.

Cheers,
Michael

[林柏年]

Hi Michael,

the problem seems to be solved by setting the correct path of java. 

I got these messages by running "hadoop version"

[root@viper0 conf]# hadoop version
Hadoop 2.6.0-cdh5.4.10
Subversion http://github.com/cloudera/hadoop -r 968adb268337548ebd7123d2ab6fcd481001fa78
Compiled by jenkins on 2016-04-12T18:08Z
Compiled with protoc 2.5.0
From source with checksum 94f877675fa68ac3d0cfb48eb692adbb
This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-cdh5.4.10.jar

I changed the path of JAVA_HOME under ~/.bashrc  to " /usr/lib/hadoop/hadoop-common-2.6.0-cdh5.4.10.jar" and it did works.

first run of big-bench:

[root@viper0 Big-Data-Benchmark-for-Big-Bench]# "./bin/bigBench" runBenchmark -m 4 -f 100 -s 2
/root/big-bench/Big-Data-Benchmark-for-Big-Bench/bin/runBenchmark: line 49: java: command not found

and $ java -version will get the same result message.

thanks for your help : )