Not able to run CPU2006 benchmarks

[abdulla...@brown.edu]

I installed the cpu2006 benchmark suite, but it does not run with Sniper4.1. Did anyone try running the spec cpu2006 benchmarks with sniper? If so, where did you get those benchmarks, what steps did you do before you ran with sniper?

Thanks, 

Nowroz

[Trevor E. Carlson]

Nowroz,

We are able to run Spec CPU2006 in Sniper without issues. What errors are you seeing? If you could add some of the error output that you are seeing, we'll try to help you fix the problem.

Trevor 

--
--
--
You received this message because you are subscribed to the Google
Groups "Sniper simulator" group.
To post to this group, send email to snip...@googlegroups.com
To unsubscribe from this group, send email to
snipersim+...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/snipersim?hl=en
 
---
You received this message because you are subscribed to the Google Groups "Sniper simulator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snipersim+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[abdulla...@brown.edu]

Dear Trevor, 

Does the latest version of Sniper come with CPU2006 benchmarks? 

Thanks, 

Nowroz

[Wim Heirman]

Nowroz,

You'll need your own SPEC CPU license, we are also not able to
redistribute the SPEC sources or inputs ourselves.

A future version of Sniper will include the CPU2006 integration
scripts (you'll just need to point them to your own copy of the
CPU2006 ISO image). We have not released this yet though.

Regards,
Wim

[abdulla...@brown.edu]

Hi Wim,

Thanks a lot for your response. I did get the CPU2006 benchmarks, put it under the benchmark folder in Sniper. I am getting the following error when I run following command.

./run-sniper -p cpu2006-400.perlbench

:~/workspace/x86simulator/sniper/benchmarks$ ./run-sniper -p cpu2006-400.perlbench
No module named cpu2006
Traceback (most recent call last):
  File "./run-sniper", line 257, in <module>
    program = __import__(package).Program(program, nthreads, inputsize, benchmark_options)
ImportError: No module named cpu2006


Thanks,

Nowroz

[abdulla...@brown.edu]

Hi Wim, 

Do you have to compile your CPU2006 outside sniper to run it with Sniper? I ran my CPU2006 outside sniper, and it is able to run. But I am not sure how to point it correctly to the correct binary files to run the CPU2006 benchmarks with Sniper. Is there a manual where it describes how to run a specific benchmark not provided already with Sniper? 

Thanks, 

Nowroz

[Trevor E. Carlson]

Nowroz,

If your SPEC CPU2006 benchmarks are running correctly outside of Sniper, then there is a good chance that they will run inside of Sniper. To run a single application in Sniper, you can use the normal command-line options. See Chapter 3 in the manual, or the example below:

$SNIPER_ROOT/run-sniper -- $CPU2006_PERLBENCH_DIR/perlbench <options>

Alternatively, you can wait for when we release the integrated benchmarks infrastructure with CPU2006 integration, or you can create your own. Look at the $BENCHMARKS_ROOT/local/__init__.py for an example on how to integrate your own benchmarks into the benchmarks directory.

Trevor

[abdulla...@brown.edu]

Hi Trevor,

Thanks a lot for you reply. I have 3 questions.

1. As per you suggestion I tried the following using splash2-
../sniper-4.1/run-sniper -- ../benchmarks/splash2/splash2/codes/apps/barnes/BARNES 

But the program seems to get stuck, I am getting the followiing -

[SNIPER] Start
Running ['bash', '-c', '/home/sreda/workspace/x86simulator/sniper/sniper-4.1/pin_kit/ia32/bin/pinbin -mt -injection child -xyzzy -enable_vsm 0 -t /home/sreda/workspace/x86simulator/sniper/sniper-4.1/lib/pin_sim -c /home/sreda/workspace/x86simulator/sniper/sniper-4.1/config/base.cfg --general/total_cores=1 --general/output_dir=/home/sreda/workspace/x86simulator/sniper/kapil --config=/home/sreda/workspace/x86simulator/sniper/sniper-4.1/config/nehalem.cfg --config=/home/sreda/workspace/x86simulator/sniper/sniper-4.1/config/gainestown.cfg -- ../benchmarks/splash2/splash2/codes/apps/barnes/BARNES']
[SNIPER] Enabling performance models
[SNIPER] Setting instrumentation mode to DETAILED


2. One more question, using '--' while specifying benchmark, is it optional? When I ran - ../sniper-4.1/run-sniper -h, it gives me following output, which does not show any option for specifying a single benchmark. Does the default run-sniper file need to be changed to run benchmarks?

Run program under the Sniper simulator
Usage:
  ../sniper-4.1/run-sniper  [-n <ncores (1)>]  [-d <outputdir (.)>]  [-c <sniper-config>]  [-c [objname:]<name[.cfg]>,<name2[.cfg]>,...]  [-g <sniper-options>]  [--roi]  [--roi-script]  [--viz]  [--perf]  [--gdb]  [--gdb-wait]  [--gdb-quit]  [--appdebug]  [--appdebug-manual]  [--appdebug-enable]  [--power]  [--cache-only]  [--fast-forward]  [--no-cache-warming]  [--save-patch]  [--pin-stats]  [--mpi [--mpi-ranks=<ranks>] ]  {--traces=<trace0>,<trace1>,... [--response-traces=<resptrace0>,<resptrace1>,...] | -- <cmdline>}

3. If a benchmark need input data, how do we specify that in the command line?

Thanks,

Nowroz

[Trevor E. Carlson]

Nowroz,

For 1. regarding BARNES not working, it looks like you are running the application manually, instead of through the integrated benchmarks directory.  Have you see the integrated benchmarks chapter in the Sniper manual [1] (Chapter 3.2)? Running Splash-2 benchmarks in Sniper does not require manual execution of the application. The example below runs barnes with 4 cores with the large input set, and supplies the correct parameters to the program for it to work properly.

$ cd $BENCHMARKS_ROOT

$ make

$ ./run-sniper -p splash2-barnes -n 4 -i large

For 2., yes, the '--' option is optional, but it helps to reduce mistaken arguments for parameters to run-sniper.

For 3., you can specify the command line as you normally would with the regular application. For example, if your command is the following:

$ ./my-app -i my-input.data

Then you run Sniper like this:

$ cd $SNIPER_ROOT

$ ./run-sniper -c gainestown -- ./my-app -i my-input.data

Take a look at Chapter 3 in the manual for more details [1].

Trevor

[1] http://snipersim.org/documents/sniper-manual.pdf

[abdulla...@brown.edu]

Hi Trevor,

Thanks a lot for your reply. I was able to run the integrated benchmarks using following.

$ ./run-sniper -p splash2-barnes -n 4 -i large

But I am trying to run single benchmarks from CPU2006, so I was just test running the splash2 to see if that one works. I did the same thing with perlbench and got the same output as following. What I did is, I compiled perlbench outside sniper, using runspec,

runspec --config=mytest.cfg --action=build --tune=base perlbench

It built successfully and gave me executable files under the folder perlbench. Now I am trying to run it with Sniper, as a single benchmark. I am using the run-sniper file that came with Sniper-4.1.

/sniper-4.1/run-sniper -- ../../benchmarks/cpu2006/benchspec/CPU2006/400.perlbench/exe/perlbench_base.amd64-m64-gcc43-nn

It gets stuck with the following -

[SNIPER] Start
Running ['bash', '-c', '/home/sreda/workspace/x86simulator/sniper/sniper-4.1/pin_kit/ia32/bin/pinbin -mt -injection child -xyzzy -enable_vsm 0 -t /home/sreda/workspace/x86simulator/sniper/sniper-4.1/lib/pin_sim -c /home/sreda/workspace/x86simulator/sniper/sniper-4.1/config/base.cfg --general/total_cores=1 --general/output_dir=/home/sreda/workspace/x86simulator/sniper/kapil/perlB --config=/home/sreda/workspace/x86simulator/sniper/sniper-4.1/config/nehalem.cfg --config=/home/sreda/workspace/x86simulator/sniper/sniper-4.1/config/gainestown.cfg -- ../../benchmarks/cpu2006/benchspec/CPU2006/400.perlbench/exe/perlbench_base.amd64-m64-gcc43-nn']

[SNIPER] Enabling performance models
[SNIPER] Setting instrumentation mode to DETAILED

Thanks,

Nowroz

[abdulla...@brown.edu]

Hi Trevor, 

I also want to mention that, I tried running the test benchmarks inside sniper, as follows and verified that sniper compiled properly.

$ cd sniper / test / fft

$ make run

...

[ SNIPER ] Start

I was able to run fft properly. To run cpu2006 benchmarks, did you have to change the makefiles inside cpu2006? Is there an extra step to connect the cpu2006 benchmarks into the Sniper?

Thanks, 

Nowroz

[abdulla...@brown.edu]

Hello all,

I am still struggling with the CPU2006 benchmarks. I was wondering if you could provide me with specific steps you did to run the cpu2006 benchmarks and the config file you used, if you have run the benchmarks with Sniper.

Some of the questions I have :
1. Did you use runspec provided with the cpu2006 benchmarks to build and run the benchmarks?
2. Most of the cpu2006 benchmarks need input, how did you provide that into your sniper?
3. What did you change in ./run-sniper to run the benchmarks?

Thanks,

Nowroz

[Trevor E. Carlson]

Nowroz,

Have you seen the recent patch that Wim recently posted to the mailing list [1] ? In that message, he details the way to integrate SPEC CPU2006 with Sniper using the benchmarks directory. Could you try following the directions there and tell us if you run into any problems?

Thanks,

Trevor

[1] https://groups.google.com/d/msg/snipersim/3-q-xGbXWhs/xqLHUHEM-pEJ

[abdulla...@brown.edu]

Hi Trevor,

Thanks a lot for all your help. I followed the instruction to run with the benchmark_cpu2006.patch, it works fine, but got stuck during compilation of 'gamess', I get the same error:

PATH_TO_BENCHMARKS/benchmarks/

cpu2006/bin/specperl: 1: PATH_TO_BENCHMARKS/benchmarks/cpu2006/bin/specperl: Syntax error: "(" unexpected
make[2]: *** [aldeci.fppized.f] Error 2


Thanks,

Nowroz

[Trevor E. Carlson]

Nowroz,

What version of the OS and compiler are you using? Are you running on a 32-bit or 64-bit machine? Also, did you install gfortran? We've tested the SPEC CPU2006 distribution for 64-bit machines, but have not tested it for 32-bit. According to another Sniper user [1], it is possible to get some, but not all of the benchmarks working on a 32-bit system by removing the ones that don't compile.

Trevor

[1] https://groups.google.com/d/msg/snipersim/3-q-xGbXWhs/Ls3pewZT7LIJ

[abdulla...@brown.edu]

Hi Trevor, 

I removed the benchmarks which do not run with my 32-bit OS, but unfortunately I need to run the benchmark 'gamess' particularly, which is giving me error. I was wondering if you would have any idea where this error is coming from, it seems like something is wrong with the cpu2006/bin/specperl, it might just be a syntax error in one of the files generated by benchmark_cpu2006.patch, which can be removed. If you have any idea of possible direction to solve this problem, please let me know.

PATH_TO_BENCHMARKS/benchmarks/

cpu2006/bin/specperl: 1: PATH_TO_BENCHMARKS/benchmarks/cpu2006/bin/specperl: Syntax error: "(" unexpected
make[2]: *** [aldeci.fppized.f] Error 2

Thanks, 

Nowroz

[Naveed Kargathra]

Hello Wim, I have a licensed ISO of spec2006 benchmarks. But I still can't install those benchmarks to run with sniper. what are the steps involved to Integrate spec2006 benchmarks with sniper?