Using ROI with benchmarks

[abdulla...@brown.edu]

Hello, 

I am trying to use roi options with cpu2006 benchmarks. I need to look at only 1.5min - 2 min of each application runtime. This time is not the computer time, but the application runtime. What I mean is, one application can finish running 5 minutes in total, but Sniper might take a few hours to simulate the 5 minutes of the application. I am interested in the application runtime.  I need to run Sniper with each application for a specific amount of time for my analysis. I am wondering if this is do-able? I have been looking at the Sniper manual, it talks about SimRoiBegin() and SimRoiEnd(), I do not think the cpu2006 benchmarks come with those marker. Please let me know if it is at possible to tell Sniper to run each application for a specific amount of time. 

Also, in the manual, it is mentioned, roi options only work with single application - "Note that all magic instructions, including ROI markers, are currently only supported in single-application mode, not when using trace les or multiple multi-threaded applications." - I am wondering if the current version of roi works with multiple applications for example, using --benchmark-splash2-barnes-test-1, splash2-barnes-test-1. 

Thanks, 

Nowroz

[Trevor E. Carlson]

Nowroz,

It sounds to me like you want to run the SPEC CPU2006 applications for a specific amount of simulated runtime. You can do this with a script in Sniper pretty easily. What you will want to do is combine two scripts that already exist in Sniper. One (sniper/scripts/periodic-stats.py), periodically records statistics, and the other (sniper/scripts/stop-by-icount.py) stops program execution after a specific amount of time. I would suggest that you create a new script that after a certain amount of time, turns off ROI (and therefore saves the application statistics and completes the applications run in fast-forwarding mode instead of the slower detailed mode).

If you would like to learn more about about scripting in Sniper, take a look at a recent tutorial [1,2], as there are step-by-step instructions and examples there. To get started, take a look at our HPCA tutorial [3], starting with slide 85.

Trevor

[1] http://snipersim.org/w/Tutorial:HPCA_2013_Sniper

[2] http://snipersim.org/w/Sniper_Tutorials

[3] http://snipersim.org/documents/2013-02-23%20Sniper%20HPCA%20Tutorial.pdf

--
--
--
You received this message because you are subscribed to the Google
Groups "Sniper simulator" group.
To post to this group, send email to snip...@googlegroups.com
To unsubscribe from this group, send email to
snipersim+...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/snipersim?hl=en
 
---
You received this message because you are subscribed to the Google Groups "Sniper simulator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snipersim+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[abdulla...@brown.edu]

Hi Trevor, 

Thanks a lot for your answer. I am looking at the slides to understand more about how the scripts work. I have tried using the stop-by-count.py script, to stop the simulation after a certain number of instructions. I also want to fast-forward at the beginning, I do not want to look at the first 1.5 mins of the application, also it is taking way to long to simulate the cpu2006 benchmarks for that long. I want to fast-forward to 1.5 min, then collect data for 30sec. Is there any script to fast-forward the simulations at the beginning? Or to change the roi-begin?

Thanks, 

Nowroz

[Trevor E. Carlson]

Nowroz,

I understand that you might want to fast-forward, but choosing arbitrary points to fast-forward to is not a good way to choose the regions of interest. Is there a reason that your application needs to run for a specific time? If not, you might want to look into SinPoints or even our new PinPoints [1] methodology. With these methodologies you can create selected regions that will be representative.

It is possible to fast-forward by a number of instructions, but fast-forwarding by time doesn't really make sense, as you will need to keep the detailed core model active while fast-forwarding. You can set the different modes for init and roi processing (take a look at --general/inst_mode_init and --general/inst_mode_roi or the --fast-forward and --no-cache-warming options to run-sniper for more information). You'll want to combine the fast-forward init mode with the detailed roi mode, and control the entering and exiting of ROI all from the script (also look at the --roi-script option to run-sniper).

Trevor

[1] http://snipersim.org/w/Tutorial:HPCA_2013_PinPoints

[abdulla...@brown.edu]

Hi Trevor, 

Thanks a lot for our answer. You are right, fast forwarding by time does not makes sense. I am actually trying to fast forward by number of instructions now. I am trying to use the fast forward mode and detailed roi mode on the fly. I am thinking of using the sim.util.Every function to periodically change between the states of fast forward and detailed roi. I am not able to locate the variable names, which folder or file should I look at for variables such as --general/inst_mode_init?
I see base.cfg defines most of the variable, and self.config.ncores, sim.config.outputdir can be used in the python scripts. I tried similar, sim.config.inst_mode_init='fast_forward', it did not work. 

To be specific we want to periodically change between fast forward and detailed mode through some script, how do we change the required variables, such as inst_mode_init on the fly. Please give us one example, or point us to a script which doing similar task. 

Thanks, 

Nowroz

[Wim Heirman]

Hi Nowroz,

general/inst_mode_init refers to configuration values, either in
base.cfg, or passed to run-sniper on the command-line using -g
--general/inst_mode_init. You can read these values from Python
through sim.config.get("general/inst_mode_init"). But updating these
values on the fly won't help, they are only read during initialization
of the simulator.

What you'll want to do is use the sim.control.set_instrumentation_mode
interface, passing one of sim.control.FASTFORWARD / sim.control.WARMUP
/ sim.control.DETAILED as the argument allows you to dynamically
switch between the instrumentation modes. You'll also want to write a
statistics snapshot before each switch using
sim.stats.write("<unique-name>") so you can later compute IPC for each
interval and compute the appropriate average.

It sounds to me as if you'll be doing periodic sampling. Note that we
will soon release some infrastructure for this which we've used for
the experiments in our ISPASS paper [1]. This infrastructure is
modular so you'll be able to specify your own sample selection
algorithm (from C++ code, not a script), in addition to our own
time-based periodic sampling.

Regards,
Wim

[1] http://snipersim.org/w/Paper:Ispass2013Carlson

[TG]

Hi Wim and Trevor,

Since there's some discussion about stop-by-icount.py script on this thread, I thought I'll ask a question - 

I saw that the script was changed in last few months to include start param for cache warmup. Unfortunately I'm yet to merge the changes from sniper-4.2 and have been using the older stop-by-icount.py.

My use case if as follows. My code has SimRoiStart() and End(). I would like to enter the detailed mode and simulate 2M instructions, then go to Fast Fwd. I run as - 

./run-sniper -n 1 -s stop-by-icount:2000000 --no-cache-warming --roi -- <app>

With this I saw that '[SCRIPT] Ending ROI after %d instructions' was printed many times. Looking at sim/util.py , I think the callback which is executed from Every/hook_periodic would call Every/hook_roi_end which again calls Every/hook_periodic and so on..

If I move the line : "self.time_next = time + self.interval" before the "if doCall" , then this is avoided.

Am I looking at things correctly? Again, this may be different with the new changes to script in sniper-4.2, but I just wanted to check.

Thanks

[Wim Heirman]

Hi Tanmay,

The new version of the script uses a different periodic callback that
happens per instruction, not by time, and the corresponding
util.EveryIns rather than util.Every -- so none of this will be
applicable to the new version.

Still, it looks as if the recursion bug you're describing is still
possible when using util.Every in other use cases. Updating
self.time_next before doing the callback sounds like a safe thing to
do.

Also, the newest version of the stop-by-icount script has an explicit
done flag which is set on roi-end; once this flag is set the script
ignores all future callbacks (this is necessary as by-instruction
periodic callbacks do happen after ROI, whereas time-based ones
usually don't since simulated time no longer advances).

Regards,
Wim

[TG]

Thanks for the information Wim. I had a look at the new script and I believe that I might need to tweak it for the use case I described.

Basically, I want to simulate X instruction in detailed mode after encountering the SimRoiStart(). With the previous script I could just say "-s stop-by-icount:2000000" and this would happen since time doesn't advance before SimRoiStart().

Now with the new script, I see that the ins_count is incremented in the non-detailed mode as well, and I don't know the number of instructions before SimRoiStart() to be able to simulate X instructions in the detailed mode. 

Tanmay

[Zilmarij Iqbal]

I wanted to clarify if all the cores/apps would be running throughout the simulation in stop-by-icount mode.

Any insights on this, please share.

Thanks,

Zilmarij Iqbal