Benchmark run terminated Reason: An error occured while running a command in phase ENGINE_VALIDATION

[张醒之]

error info :

==============

Benchmark run terminated

Reason: An error occured while running a command in phase ENGINE_VALIDATION_DATA_GENERATION

==============

java.io.IOException: Error while generating dataset. More information in logfile: /usr/local/bigbench/logs/dataGeneration-run_query.log

        at io.bigdatabenchmark.v1.driver.BigBench.generateData(BigBench.java:819)

        at io.bigdatabenchmark.v1.driver.BigBench.run(BigBench.java:418)

        at io.bigdatabenchmark.v1.driver.RunBigBench.main(RunBigBench.java:52)

and log info :

Error: java.io.IOException: ################# ERROR  Task 7/80 ###########################

Message:Return code of subprocess was !=0: 143StackTrace:

io.bigdatabenchmark.hadoop.HadoopClusterExec$NonZeroReturnCodeException: Return code of subprocess was !=0: 143

at io.bigdatabenchmark.hadoop.HadoopClusterExec$DistributedExecMapper.map(HadoopClusterExec.java:740)

at io.bigdatabenchmark.hadoop.HadoopClusterExec$DistributedExecMapper.map(HadoopClusterExec.java:1)

at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Is there something wrong with my configuration?  which configuration file and where?

[Michael Frank]

Hi,
the data generation stage is the first job running during the benchmark. So it usually gets all the blame if something is missconfigured.
Error code 143 indicates that your container is terminated forcefully by YARN because of memory restrictions.
Please verify your cluster settings in regards of memory settings per container and the jvm processes. Each container and jvm should at least be assigned 1-2GB of memory.

See: https://github.com/intel-hadoop/Big-Data-Benchmark-for-Big-Bench#cluster-setup

Cheers
Michael

[张醒之]

Hi,

Thank you. I will try your proposal.