How to Enable Performance Counters

[Zechen Ma]

Dear Firesim Folks,

I am trying to set up and enable hardware performance counters so that I can read microarchitectural events (e.g. cache hits/misses, branch mispredictions)

What I have done is to put the following code sequence in /home/zma/firesim/target-design/chipyard/generators/testchipip/src/main/resources/testchipip/bootrom/bootrom.S. I tried putting the code at the end of the file (right before mret) and at the very beginning (right after .start):

  li t3, -1
  csrw mcounteren, t3
  csrw scounteren, t3

  li t3, 0x100
  csrw mhpmevent3, t3   // exception
  li t3, 0x201
  csrw mhpmevent4, t3   // branch mispredition
  li t3, 0x401
  csrw mhpmevent5, t3   // control-flow target mispredition
  li t3, 0x801
  csrw mhpmevent6, t3   // flush
  li t3, 0x102
  csrw mhpmevent7, t3   // I$ miss
  li t3, 0x202
  csrw mhpmevent8, t3   // D$ miss
  li t3, 0x402
  csrw mhpmevent9, t3   // D$ release
  li t3, 0x802
  csrw mhpmevent10, t3  // ITLB miss
  li t3, 0x1002
  csrw mhpmevent11, t3 // DTLB miss
  li t3, 0x2002
  csrw mhpmevent12, t3 // L2 TLB miss

I did `make` in the bootrom folder and rebuilt the bitstream to make sure the code I added to the bootrom is effective.

To test if it is indeed effective, I added `jr zero` at the end of the code gadget above, for the purpose of crashing the booting process. But it never worked (i.e. I was still able to boost linux). My firesim version is 1.17.1

I was wondering if this is the right way of enabling hardware performance counters? If not, where should I put these code and how do I turn on the performance counter tracking?

Thanks so much in advance!

Sincerely,

Zechen Ma

[Abraham Gonzalez]

I wouldn't recommend changing the bootrom (since as you mentioned, you would have to recompile the bitstream anytime you want to change which sampled signals to look at). A simpler way to test if the bootrom changes have been properly applied is to run a metasim with a baremetal test to check if the counters are sampling properly. It seems suspect that changes to the bootrom to kill the simulation aren't applying (maybe the wrong bitstream is being used).

This old thread might be helpful: https://groups.google.com/g/firesim/c/hWoqInJK4ME/m/bJJNKX5wBwAJ

I would verify that perf. counters are enabled on the DUT. See mixins like: https://github.com/ucb-bar/chipyard/blob/8d110405070d1d4d25117e359783382f52014213/generators/chipyard/src/main/scala/config/fragments/TileFragments.scala#L55. That would seem to align with the behavior you are seeing where there is no effect of writing the CSRs? I would try a Chipyard baremetal simulation (m-mode simulation) to verify that things are enabled correctly before using counters in OpenSBI/BBL/Linux (but once it works in m-mode for baremetal tests, you should be able to modify OpenSBI to add the counter setup code).

Also as the other thread states, AutoCounter is a nice tool that allows you to sample signals without perturbing the DUT. I would take a look at that.

--
You received this message because you are subscribed to the Google Groups "FireSim" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/firesim/a6ebab1d-5c08-4fce-8548-cae953b000ecn%40googlegroups.com.

--

Abraham J. Gonzalez

Electrical Engineering and Computer Science Ph.D. Student

The University of California at Berkeley