error when run q20,25,q26,q28

[auge pang]

Hi,

I'm currently running BigBench and the queries seem to work just fine, except for the query number 20,25,26. 

The error message of query 20 I've got is the following: 

===============================================

Validating query : q20

-----------------------------------------------

benchmark phase: engine_validation_power_test

stream number  : 0

user parameter file: 

user settings file : 

log: /home/server501/tpcx-bb/logs/q20_hive_engine_validation_power_test_0.log

===============================================

Golden result set file /home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result not found

Auge Pang's information:Validation of /home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result and hadoop fs -cat /user/server501/benchmarks/bigbench/queryResults/q20_hive_engine_validation_power_test_0_result/*

17/03/24 20:45:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Files /home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result and /dev/fd/62 differ

Validation of /home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result failed: Query returned incorrect results

Validation failed: Query results are not OK

An error occured while running command:

==========

query_run_validate_method

==========

Please check the log files for details

cat: Unable to write to output stream.

Command: [bash, /home/server501/tpcx-bb/bin/bigBench, validateQuery, -q, 20, -p, engine_validation_power_test, -t, 0, -f, 1, -e, hive, -d, bigbench, -m, 8, -U]

got non-zero return value: 1

BigBench engine validation: Check all query results     stream          0, query 20 ( 1/ 1) failed. Time:          0h:00m:02s:012ms

BigBench engine validation: Check all query results     failed. Time:          0h:00m:02s:016ms

==============

Benchmark run terminated

Reason: An error occured while running a command in phase ENGINE_VALIDATION_RESULT_VALIDATION

==============

java.io.IOException: Error while running query 20. More information in logfile: /home/server501/tpcx-bb/logs/q20_hive_engine_validation_power_test_0.log

at io.bigdatabenchmark.v1.driver.BigBench$QueryThread.call(BigBench.java:1492)

at io.bigdatabenchmark.v1.driver.BigBench$QueryThread.call(BigBench.java:1)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

************

TPCx-BB

Result

v1.1

************

NOK: Some required BigBench phases were skipped. No final result available.

17/03/24 20:45:30 INFO client.RMProxy: Connecting to ResourceManager at n1/192.168.237.136:8032

17/03/24 20:45:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

[1] 20:45:31 [FAILURE] n2 Exited with error code 1

[2] 20:45:31 [FAILURE] n3 Exited with error code 1

[1] 20:45:31 [FAILURE] n2 Exited with error code 127

[2] 20:45:31 [FAILURE] n3 Exited with error code 127

[1] 20:45:32 [FAILURE] n2 Exited with error code 1

[2] 20:45:32 [FAILURE] n3 Exited with error code 1

[Michael Frank]

Hi Auge Pang,

General descrption:
Your error occured during benchmark phase "engine_validation". The engine_validation phase is a precautious step, introduced to verifiy your cluster and hive/spark are at least capeable to run a Scale factor 1 benchmark (1 GB of data) without errors and produce the correct results, before proceeding to larger scale factors. Failing in this stage usually means, something is wrong with your cluster setup.
The results from this phase are compared against a "golden result set". Each query containes an corresponding "golden result set" stored in:

tpcx-bb/engines/hive/queries/q??/results/q??-result  where ?? in: [01,30]

From your q20_hive_engine_validation_power_test_0.log file is see that following file is missing or not accessible:

/home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result

 

see 11. line from the bottom in your log file:

Golden result set file /home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result

 

Missing file is that one: https://github.com/intel-hadoop/Big-Data-Benchmark-for-Big-Bench/blob/master/engines/hive/queries/q20/results/q20-result 

I also see some comments you apparently added yourself. Please make sure you did not manipulate or delete any important files.

Once that is resolved and validation still gives you trouble:
 for example we know that due to implementation changes in the machine learning algorithms themselfs, different spark version may produce different sesults
you could skip the engine validation phases by changeing the following  line in https://github.com/intel-hadoop/Big-Data-Benchmark-for-Big-Bench/blob/master/conf/bigBench.properties

workload=CLEAN_ALL,ENGINE_VALIDATION_DATA_GENERATION,ENGINE_VALIDATION_LOAD_TEST,ENGINE_VALIDATION_POWER_TEST,ENGINE_VALIDATION_RESULT_VALIDATION,CLEAN_DATA,DATA_GENERATION,BENCHMARK_START,LOAD_TEST,POWER_TEST,THROUGHPUT_TEST_1,BENCHMARK_STOP,VALIDATE_POWER_TEST,VALIDATE_THROUGHPUT_TEST_1

 To:

workload=CLEAN_ALL,CLEAN_DATA,DATA_GENERATION,BENCHMARK_START,LOAD_TEST,POWER_TEST,THROUGHPUT_TEST_1,BENCHMARK_STOP,VALIDATE_POWER_TEST,VALIDATE_THROUGHPUT_TEST_1

Or without manipulating the conf file using the -i option:

 ./bigBench runBenchmark -i "CLEAN_ALL,CLEAN_DATA,DATA_GENERATION,BENCHMARK_START,LOAD_TEST,POWER_TEST,BENCHMARK_STOP,VALIDATE_POWER_TEST" <..other options>

Cheers ,
Michael

[auge pang]

Hi Michael,

      Thanks you for your help.  But the file /home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result already exists in the system and has access rights.

      I changed the conf file like following:

         workload=CLEAN_ALL,CLEAN_DATA,DATA_GENERATION,BENCHMARK_START,LOAD_TEST,,THROUGHPUT_TEST_1,BENCHMARK_STOP,VALIDATE_POWER_TEST,VALIDATE_THROUGHPUT_TEST_1

     query 20 also get the following error:

INFO: BigBench validation: Power test results                 stream          0, query 20 (20/30) log file: /home/server501/tpcx-bb/logs/q20_hive_power_test_0.log

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench runCmd

INFO: Command: [bash, /home/server501/tpcx-bb/bin/bigBench, validateQuery, -q, 20, -p, power_test, -t, 0, -f, 1, -m, 8, -s, 2, -U]

got non-zero return value: 1

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench$QueryThread call

INFO: BigBench validation: Power test results                 stream          0, query 20 (20/30) failed. Time:          0h:00m:04s:543ms

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench runQueries

INFO: BigBench validation: Power test results                 failed. Time:          0h:01m:59s:566ms

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench run

INFO: ==============

Benchmark run terminated

Reason: An error occured while running a command in phase VALIDATE_POWER_TEST

==============

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: 

************

TPCx-BB

Result

v1.1

************

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: T_LOAD = 718.722

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: T_LD = 0.1 * T_LOAD: 71.8722

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: T_PT = 7274.436825570289

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: T_T_PUT = 12474.375

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: T_TT = 6237.1875

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: === Checking validity of the final result ===

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: OK: All required BigBench phases were performed.

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: OK: All 30 queries were running in the power test.

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: OK: All 30 queries were running in the first throughput test.

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: OK: Pretend mode was inactive. All commands were executed.

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: === Final result ===

Mar 28, 2017 6:20:29 AM io.bigdatabenchmark.v1.driver.BigBench computeResult

INFO: VALID BBQpm@1 = 0.2644046061137735

    

在 2017年3月25日星期六 UTC+8下午8:31:22，Michael Frank写道：

Hi Auge Pang,

         

General descrption:
Your error occured during benchmark phase "engine_validation". The engine_validation phase is a precautious step, introduced to verifiy your cluster and hive/spark are at least capeable to run a Scale factor 1 benchmark (1 GB of data) without errors and produce the correct results, before proceeding to larger scale factors. Failing in this stage usually means, something is wrong with your cluster setup.
The results from this phase are compared against a "golden result set". Each query containes an corresponding "golden result set" stored in:

tpcx-bb/engines/hive/queries/q??/results/q??-result  where ?? in: [01,30]

From your q20_hive_engine_validation_power_test_0.log file is see that following file is missing or not accessible:

/home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result

 

see 11. line from the bottom in your log file:

Golden result set file /home/server501/tpcx-bb/engines/hive/queries/q20/results/q20-result

 

Missing file is that one: https://github.com/intel-hadoop/Big-Data-Benchmark-for-Big-Bench/blob/master/engines/hive/queries/q20/results/q20-result 

I also see some comments you apparently added yourself. Please make sure you did not manipulate or delete any important files.

Once that is resolved and validation still gives you trouble:
 for example we know that due to implementation changes in the machine learning algorithms themselfs, different spark version may produce different sesults
you could skip the engine validation phases by changeing the following  line in https://github.com/intel-hadoop/Big-Data-Benchmark-for-Big-Bench/blob/master/conf/bigBench.properties

workload=CLEAN_ALL,ENGINE_VALIDATION_DATA_GENERATION,ENGINE_VALIDATION_LOAD_TEST,ENGINE_VALIDATION_POWER_TEST,ENGINE_VALIDATION_RESULT_VALIDATION,CLEAN_DATA,DATA_GENERATION,BENCHMARK_START,LOAD_TEST,POWER_TEST,THROUGHPUT_TEST_1,BENCHMARK_STOP,VALIDATE_POWER_TEST,VALIDATE_THROUGHPUT_TEST_1

 To:

workload=CLEAN_ALL,CLEAN_DATA,DATA_GENERATION,BENCHMARK_START,LOAD_TEST,,THROUGHPUT_TEST_1,BENCHMARK_STOP,VALIDATE_POWER_TEST,VALIDATE_THROUGHPUT_TEST_1