Issues getting BigBench to run

[robe...@wandisco.com]

Hey everyone,

I'm trying to do a test run of BigBench before we deploy it full-time and I'm having some issues configuring it right now. We've nailed down our issues with the benchmark to the configuration. In particular, my coworker and I are confused as how to exactly the userSettings.conf file, especially with regards to the HADOOP environment and HDFS config and paths section. Our cluster is running CDH5.11.0 with CENTOS 6. Any help would be greatly appreciated.

[Bhaskar Gowda]

Hi Robert - can you plz post issue you are facing with the configuration files.

[robe...@wandisco.com]

I'll attach the logs and a screenshot of the terminal output. Let me know if there's more I can do.

On Tuesday, July 18, 2017 at 5:44:58 PM UTC-7, robe...@wandisco.com wrote:

[Bhaskar Gowda]

plz check the following:

1. Java version 1.7 or 1.8
2. YARN, HDFS, HIVE, Spark gateway roles on the node where you are running the benchmark

On Tuesday, July 18, 2017 at 5:44:58 PM UTC-7, robe...@wandisco.com wrote:

[robe...@wandisco.com]

I've checked all of these and they're all correct.

[Dave Jaffe]

Robert-

The only questionable thing I can see in your userSettings.conf is the line

export BIG_BENCH_USER="$HDFS"

Do you have $HDFS set in your environment? Is it set to hdfs? What user do you run BigBench as?

I run BigBench as user root, and set that line to

export BIG_BENCH_USER="$USER"

You might have to make the Hadoop top directory writeable by all before you run BigBench:

sudo -u hdfs hadoop fs -chmod 777 /

Please let me know if this helps.

Dave Jaffe

VMware

World Leader in Active Data Replication™

Find out more wandisco.com

THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY AND MAY BE PRIVILEGED

If this message was misdirected, WANdisco, Inc. and its subsidiaries, ("WANdisco") does not waive any confidentiality or privilege. If you are not the intended recipient, please notify us immediately and destroy the message without disclosing its contents to anyone. Any distribution, use or copying of this email or the information it contains by other than an intended recipient is unauthorized. The views and opinions expressed in this email message are the author's own and may not reflect the views and opinions of WANdisco, unless the author is authorized by WANdisco to express such views or opinions on its behalf. All email sent to or from this address is subject to electronic storage and review by WANdisco. Although WANdisco operates anti-virus programs, it does not accept responsibility for any damage whatsoever caused by viruses being passed.

--
You received this message because you are subscribed to the Google Groups "Big Data Benchmark for BigBench" group.
To unsubscribe from this group and stop receiving emails from it, send an email to big-bench+unsubscribe@googlegroups.com.
To post to this group, send email to big-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/big-bench/cc1653e0-06de-4e4a-af53-db7f9cfff2f5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dave Jaffe

davej...@gmail.com

[robe...@wandisco.com]

Hi Dave,

I initially ran BigBench is root user, and had the config for that part exactly the same as you. I'll change it back and try the chmod thing though!

To unsubscribe from this group and stop receiving emails from it, send an email to big-bench+...@googlegroups.com.

To post to this group, send email to big-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/big-bench/cc1653e0-06de-4e4a-af53-db7f9cfff2f5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dave Jaffe

davej...@gmail.com

[robe...@wandisco.com]

Hey Dave,

Just tried changing the permissions and went back to running as root. It's still having the exact same issues.

On Wednesday, July 19, 2017 at 8:35:26 AM UTC-7, Dave Jaffe wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to big-bench+...@googlegroups.com.

To post to this group, send email to big-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/big-bench/cc1653e0-06de-4e4a-af53-db7f9cfff2f5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dave Jaffe

davej...@gmail.com

[robe...@wandisco.com]

Here's my latest log.

[robe...@wandisco.com]

Hi Dave, 

Sorry for the spam. My co-worker just pointed me to the failed logs for MapReduce, and I'm having NoClassDef error. I'm attaching the log.

On Wednesday, July 19, 2017 at 8:35:26 AM UTC-7, Dave Jaffe wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to big-bench+...@googlegroups.com.

To post to this group, send email to big-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/big-bench/cc1653e0-06de-4e4a-af53-db7f9cfff2f5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dave Jaffe

davej...@gmail.com

[Dave Jaffe]

I'm afraid I'm out of ideas. You can run a simple Hadoop job, say Teragen, from the same directory, correct?

hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen10000000000 terasort1TB_input

[robe...@wandisco.com]

Hey Dave,

We've done so more digging and we think we've pinpointed the error; we're just not sure how to address it. So we looked into the NoClassDef error and it seems like BigBench is having issues pulling the hadoop-common jar.

[Bhaskar Gowda]

can you plz re-check what version of JAVA you are running, change it to 1.8/1.7  oracle JDK is preferred but OpenJDK will work too. I also suggest you to revert the user from HDFS to $USER and disable HDFS permissions in CDH manager to see if it helps.

On Tuesday, July 18, 2017 at 5:44:58 PM UTC-7, robe...@wandisco.com wrote:

[robe...@wandisco.com]

I checked and we are using 1.7JDK. User is back to USER as well.

[robe...@wandisco.com]

Hey everyone. I fixed this issue (we still have a few issues right now, but we fixed the one that this post is about.) We had neglected to install zip on all nodes. In regards to this; why would we get the errors we did with a missing hadoop-common jar if we lacked zip?

On Tuesday, July 18, 2017 at 5:44:58 PM UTC-7, robe...@wandisco.com wrote: