MCYCLE CSR

[vithurson subasharan]

"The mcycle CSR holds a count of the number of cycles the hart has executed since some time in the arbitary past" ?
I am using blocking caches in my implementation.
Should mcycle be incremented while pipeline is being blocked by the caches?

[Guy Lemieux]

and what about when the CPU clock is stopped, i.e. when
in a sleep mode?

guy

> --
> You received this message because you are subscribed to the Google Groups
> "RISC-V HW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to hw-dev+un...@groups.riscv.org.
> To post to this group, send email to hw-...@groups.riscv.org.
> Visit this group at
> https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
> To view this discussion on the web visit
> https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/9324bc51-a1f3-4cfb-b69b-75e8e2d82a38%40groups.riscv.org.

[kr...@berkeley.edu]

Yes.

Krste

[vithurson subasharan]

thanks... :)

On 25 Oct 2017 11:42 p.m., <kr...@berkeley.edu> wrote:

>>>>> On Wed, 25 Oct 2017 08:14:50 -0700 (PDT), vithurson subasharan <vith...@gmail.com> said:

| "The mcycle CSR holds a count of the number of cycles the hart has executed
| since some time in the arbitary past" ?
| I am using blocking caches in my implementation.
| Should mcycle be incremented while pipeline is being blocked by the
| caches?

Yes.

Krste


| --
| You received this message because you are subscribed to the Google Groups
| "RISC-V HW Dev" group.
| To unsubscribe from this group and stop receiving emails from it, send an email

| to hw-dev+unsubscribe@groups.riscv.org.

[kr...@berkeley.edu]

>>>>> On Wed, 25 Oct 2017 08:19:45 -0700, Guy Lemieux <glem...@vectorblox.com> said:
| and what about when the CPU clock is stopped, i.e. when
| in a sleep mode?

Good question.

Given that sleep modes are not part of the ISA specification, and the
type and degrees of sleep is also going to vary by implementation, I
think the answer must be implementation defined.

At least the mcycle value should be only monotonically increasing
(i.e., cannot forget mcycle value on sleep).

Krste

| To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/CALo5CZwfG793CQjq59JBn8sprV%2Bibkxai90_btp_p%2BJ_t2wR6w%40mail.gmail.com.

[Frohic Food]

mcycle should count clocks.

If the clock is slowed down or stopped, this is still the case.  I.e. mcycle increments slower at a lower frequency or stops completely if the clock is cut.

The arbitrary time in the past is arbitrary.  I.e. implementation specific.  I'd expect most implementations to zero mcycle upon reset.  However possibly not.  mcycle could be zeroed upon power up, and then keep incrementing each clock during reset.  [Note, it's not uncommon for microcontrollers to record the source of a reset (watchdog, external reset pin, powerup), this means this logic isn't reset upon reset!  mcycle might also not be cleared upon reset and could provide useful information upon powerup.]

On the other end of the 'arbitrary' scale, an implementation may choose to zero mcycle after leaving a sleep mode.  The choice is arbitrary.

On Wed, Oct 25, 2017 at 4:19 PM, Guy Lemieux <glem...@vectorblox.com> wrote:

and what about when the CPU clock is stopped, i.e. when
in a sleep mode?

guy

On Wed, Oct 25, 2017 at 8:14 AM, vithurson subasharan
<vith...@gmail.com> wrote:
> "The mcycle CSR holds a count of the number of cycles the hart has executed
> since some time in the arbitary past" ?
> I am using blocking caches in my implementation.
> Should mcycle be incremented while pipeline is being blocked by the caches?
>
> --
> You received this message because you are subscribed to the Google Groups
> "RISC-V HW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to hw-dev+unsubscribe@groups.riscv.org.

> To post to this group, send email to hw-...@groups.riscv.org.
> Visit this group at
> https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
> To view this discussion on the web visit
> https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/9324bc51-a1f3-4cfb-b69b-75e8e2d82a38%40groups.riscv.org.

--
You received this message because you are subscribed to the Google Groups "RISC-V HW Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to hw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/CALo5CZwfG793CQjq59JBn8sprV%2Bibkxai90_btp_p%2BJ_t2wR6w%40mail.gmail.com.

[Christopher Celio]

mcycle, in my opinion, counts cycles, so if the clock stops so does mcycle. We have mtime, which is outside the domain of the core, if we need some more general ground truth that is not affected by frequency/power shenanigans. 

Should mcycle be incremented while pipeline is being blocked by the caches? 

The purpose of mcycle is to help the programmer have a fine-grain sense of the performance/behavior of his code ("how long does this basic block take?"). If you disable mcycle on cache misses, the programmer will lose significant insight.

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+un...@groups.riscv.org.

To post to this group, send email to hw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/CAPQXEto3vkAh0RK9-qpWEec2JxHDwJr9GQBX5uT6_MBuDG7fxQ%40mail.gmail.com.

[kr...@berkeley.edu]

I agree mcycle should count the number of clocks delivered to the
core.

How different kinds of reset treat the mcycle counter is probably a
platform issue. If the reset is part of a short power-down/power-up
sequence in which the application thread on the hart continues
execution, then the mcycle value should probably be saved/restored.
If the reset kills ongoing application thread and starts a new thread,
then old mcycle can probably be lost.

Krste

|| email to hw-dev+un...@groups.riscv.org.
|| To post to this group, send email to hw-...@groups.riscv.org.
|| Visit this group at
|| https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
|| To view this discussion on the web visit
|| https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/

| 9324bc51-a1f3-4cfb-b69b-75e8e2d82a38%40groups.riscv.org.

| --
| You received this message because you are subscribed to the Google
| Groups "RISC-V HW Dev" group.
| To unsubscribe from this group and stop receiving emails from it, send
| an email to hw-dev+un...@groups.riscv.org.
| To post to this group, send email to hw-...@groups.riscv.org.
| Visit this group at https://groups.google.com/a/groups.riscv.org/group/
| hw-dev/.

| To view this discussion on the web visit https://groups.google.com/a/

| groups.riscv.org/d/msgid/hw-dev/CALo5CZwfG793CQjq59JBn8sprV%
| 2Bibkxai90_btp_p%2BJ_t2wR6w%40mail.gmail.com.

| --
| You received this message because you are subscribed to the Google Groups
| "RISC-V HW Dev" group.
| To unsubscribe from this group and stop receiving emails from it, send an
| email to hw-dev+un...@groups.riscv.org.
| To post to this group, send email to hw-...@groups.riscv.org.
| Visit this group at https://groups.google.com/a/groups.riscv.org/group/
| hw-dev/.

| To view this discussion on the web visit https://groups.google.com/a/
| groups.riscv.org/d/msgid/hw-dev/

| CAPQXEto3vkAh0RK9-qpWEec2JxHDwJr9GQBX5uT6_MBuDG7fxQ%40mail.gmail.com.

| --
| You received this message because you are subscribed to the Google Groups
| "RISC-V HW Dev" group.
| To unsubscribe from this group and stop receiving emails from it, send an email
| to hw-dev+un...@groups.riscv.org.
| To post to this group, send email to hw-...@groups.riscv.org.
| Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
| To view this discussion on the web visit https://groups.google.com/a/
| groups.riscv.org/d/msgid/hw-dev/

| DCCB6468-F685-413B-A46C-D191C5627E9A%40eecs.berkeley.edu.

[Frohic Food]

Upon reflection, I'm slightly concerned that arbitrary time in the past is loose enough that zeroing mcycle any time wouldn't violate the spec.  I agree that what is required is a monotonically increasing counter.  Maybe we should tighten the spec here and require that mcycle cannot be cleared if any of a hart's state is maintained.  This would cover the sleep case where state would be preserved. 

|| email to hw-dev+unsubscribe@groups.riscv.org.

|| To post to this group, send email to hw-...@groups.riscv.org.
|| Visit this group at
|| https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
|| To view this discussion on the web visit
|| https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/
|         9324bc51-a1f3-4cfb-b69b-75e8e2d82a38%40groups.riscv.org.

|         --
|         You received this message because you are subscribed to the Google
|         Groups "RISC-V HW Dev" group.
|         To unsubscribe from this group and stop receiving emails from it, send

|         an email to hw-dev+unsubscribe@groups.riscv.org.

|         To post to this group, send email to hw-...@groups.riscv.org.
|         Visit this group at https://groups.google.com/a/groups.riscv.org/group/
|         hw-dev/.
|         To view this discussion on the web visit https://groups.google.com/a/

|         groups.riscv.org/d/msgid/hw-dev/CALo5CZwfG793CQjq59JBn8sprV%
|         2Bibkxai90_btp_p%2BJ_t2wR6w%40mail.gmail.com.

|     --
|     You received this message because you are subscribed to the Google Groups
|     "RISC-V HW Dev" group.
|     To unsubscribe from this group and stop receiving emails from it, send an

|     email to hw-dev+unsubscribe@groups.riscv.org.

|     To post to this group, send email to hw-...@groups.riscv.org.
|     Visit this group at https://groups.google.com/a/groups.riscv.org/group/
|     hw-dev/.
|     To view this discussion on the web visit https://groups.google.com/a/
|     groups.riscv.org/d/msgid/hw-dev/

|     CAPQXEto3vkAh0RK9-qpWEec2JxHDwJr9GQBX5uT6_MBuDG7fxQ%40mail.gmail.com.

| --
| You received this message because you are subscribed to the Google Groups
| "RISC-V HW Dev" group.
| To unsubscribe from this group and stop receiving emails from it, send an email

| to hw-dev+unsubscribe@groups.riscv.org.

| To post to this group, send email to hw-...@groups.riscv.org.
| Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
| To view this discussion on the web visit https://groups.google.com/a/
| groups.riscv.org/d/msgid/hw-dev/

| DCCB6468-F685-413B-A46C-D191C5627E9A%40eecs.berkeley.edu.

[Andrew Waterman]

I agree with the sentiment but think this is a platform constraint
that doesn't belong in the ISA spec itself.

>> || email to hw-dev+un...@groups.riscv.org.

>> || To post to this group, send email to hw-...@groups.riscv.org.
>> || Visit this group at
>> || https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
>> || To view this discussion on the web visit
>> || https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/
>> | 9324bc51-a1f3-4cfb-b69b-75e8e2d82a38%40groups.riscv.org.
>>
>> | --
>> | You received this message because you are subscribed to the
>> Google
>> | Groups "RISC-V HW Dev" group.
>> | To unsubscribe from this group and stop receiving emails from
>> it, send

>> | an email to hw-dev+un...@groups.riscv.org.

>> | To post to this group, send email to hw-...@groups.riscv.org.
>> | Visit this group at
>> https://groups.google.com/a/groups.riscv.org/group/
>> | hw-dev/.
>> | To view this discussion on the web visit
>> https://groups.google.com/a/
>> | groups.riscv.org/d/msgid/hw-dev/CALo5CZwfG793CQjq59JBn8sprV%
>> | 2Bibkxai90_btp_p%2BJ_t2wR6w%40mail.gmail.com.
>>
>>
>>
>> | --
>> | You received this message because you are subscribed to the Google
>> Groups
>> | "RISC-V HW Dev" group.
>> | To unsubscribe from this group and stop receiving emails from it,
>> send an

>> | email to hw-dev+un...@groups.riscv.org.

>> | To post to this group, send email to hw-...@groups.riscv.org.
>> | Visit this group at
>> https://groups.google.com/a/groups.riscv.org/group/
>> | hw-dev/.
>> | To view this discussion on the web visit
>> https://groups.google.com/a/
>> | groups.riscv.org/d/msgid/hw-dev/
>> |
>> CAPQXEto3vkAh0RK9-qpWEec2JxHDwJr9GQBX5uT6_MBuDG7fxQ%40mail.gmail.com.
>>
>>
>> | --
>> | You received this message because you are subscribed to the Google
>> Groups
>> | "RISC-V HW Dev" group.
>> | To unsubscribe from this group and stop receiving emails from it, send
>> an email

>> | to hw-dev+un...@groups.riscv.org.

>> | To post to this group, send email to hw-...@groups.riscv.org.
>> | Visit this group at
>> https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
>> | To view this discussion on the web visit https://groups.google.com/a/
>> | groups.riscv.org/d/msgid/hw-dev/
>> | DCCB6468-F685-413B-A46C-D191C5627E9A%40eecs.berkeley.edu.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "RISC-V HW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to hw-dev+un...@groups.riscv.org.

> To post to this group, send email to hw-...@groups.riscv.org.
> Visit this group at
> https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
> To view this discussion on the web visit

> https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/CAPQXEtrpAQcrtwY1A3hT7dztfLia1KCv2fE7U-cW6AyrLzpYaQ%40mail.gmail.com.

[Frohic Food]

Actually upon further reflection I think the spec is fine.

"The mcycle CSR holds a count of the number of cycles the hart has executed since some arbitrary time in the past."

So even if the sleep mode involved saving the hart's whole state externally and powering the core down, whenever the hart is restored its mcycle must still provide a monotonically increasing count of machine cycles from before the sleep.

Reset is a different matter because although resets can occur in the future, the reset must always be in the hart's past.

[Rishiyur Nikhil]

>     Krste:

>     At least the mcycle value should be only monotonically increasing

This was what I was trying to get to, in an earlier thread on this topic.

There was a lot of discussion on implementation nuances (clocks, clock scaling, asynch logic, simulators, ...)

but I'd like to zero in on a formal property of the RDCYCLE instruction independent of these widely varying implementation questions.

I had proposed either "monotonically non-decreasing" (weaker) or "monotonically increasing" (stronger).

(with caveats about 64-bit wraparound, of course).

Is that it, or is there anything more?

Nikhil

On Wed, Oct 25, 2017 at 2:14 PM, <kr...@berkeley.edu> wrote:

>>>>> On Wed, 25 Oct 2017 08:19:45 -0700, Guy Lemieux <glem...@vectorblox.com> said:
| and what about when the CPU clock is stopped, i.e. when
| in a sleep mode?

Good question.

Given that sleep modes are not part of the ISA specification, and the
type and degrees of sleep is also going to vary by implementation, I
think the answer must be implementation defined.

At least the mcycle value should be only monotonically increasing
(i.e., cannot forget mcycle value on sleep).

Krste

| guy

| On Wed, Oct 25, 2017 at 8:14 AM, vithurson subasharan
| <vith...@gmail.com> wrote:
|| "The mcycle CSR holds a count of the number of cycles the hart has executed
|| since some time in the arbitary past" ?
|| I am using blocking caches in my implementation.
|| Should mcycle be incremented while pipeline is being blocked by the caches?
||
|| --
|| You received this message because you are subscribed to the Google Groups
|| "RISC-V HW Dev" group.
|| To unsubscribe from this group and stop receiving emails from it, send an

|| email to hw-dev+unsubscribe@groups.riscv.org.

|| To post to this group, send email to hw-...@groups.riscv.org.
|| Visit this group at
|| https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
|| To view this discussion on the web visit
|| https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/9324bc51-a1f3-4cfb-b69b-75e8e2d82a38%40groups.riscv.org.

| --
| You received this message because you are subscribed to the Google Groups "RISC-V HW Dev" group.

| To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+unsubscribe@groups.riscv.org.

| To post to this group, send email to hw-...@groups.riscv.org.
| Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
| To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/CALo5CZwfG793CQjq59JBn8sprV%2Bibkxai90_btp_p%2BJ_t2wR6w%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "RISC-V HW Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to hw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/23024.54404.546182.773158%40KAMacBookAir2012.local.

[Mauro Olivieri]

Hello everyone,
apart from sleep modes and other special conditions, I have a general doubt about defining "Number of cycles the **hart** has executed". This is quite clear in a multi-core implementation with one hardware thread running on each core. But what if we consider implementations that exploit techniques like simultaneous multi-threading on a single core supporting ILP? what is exactly the "number of cycles a hart has executed" in such an implementation?

thanks in advance for clarifying

M

--

You received this message because you are subscribed to the Google Groups "RISC-V HW Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to hw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/40d2c313-0375-405c-b566-14371d8caacd%40groups.riscv.org.

[Samuel Falvo II]

On Thu, Oct 26, 2017 at 5:08 AM, Mauro Olivieri
<olivieri...@gmail.com> wrote:
> each core. But what if we consider implementations that exploit techniques
> like simultaneous multi-threading on a single core supporting ILP? what is
> exactly the "number of cycles a hart has executed" in such an
> implementation?
> thanks in advance for clarifying

I think the problem that pops up all the time is the phrase, "has
executed". MCYCLE does not count things executed. Use MINSTRET for
that. MCYCLE counts cycles elapsed since some arbitrary epoch.

For SMT, HT, or related technologies, it would be a significant loss
of performance data if each thread had its own MCYCLE which
incremented only when that thread took possession of the execution
units. You want to measure the impact of SMT, HT, et. al., positive
or negative, you have to have MCYCLE monotonically increasing. That
said, each thread CAN have its own private MCYCLE counter, just as
long as it increments monotonically. Precisely how these threads
time-slice (if they do at all) should not be relevant to the behavior
of MCYCLE.

Two independent hardware threads sharing a common bus interface
on-chip would look no different to software than two separate cores,
each with a single thread running on them, sharing a common bus
interconnect. SMT, HT, et. al. would not alter this relationship.

--
Samuel A. Falvo II

[Christopher Celio]

> For SMT, HT, or related technologies, it would be a significant loss
> of performance data if each thread had its own MCYCLE which
> incremented only when that thread took possession of the execution
> units.

I believe that each hart MUST have its own mcycle (it is architectural state), but that each hart on a physical core has their mcycles incremented together. Let me show my reasoning:

The intent behind mcycle suggests that each physical core has its "own" mcycle counter, and if 4 harts are scheduled simultaneously onto one core --- I want to know if one hart gets descheduled for 100 cycles.

But what happens if one of the 4 harts gets migrated to another physical core, which has its own separate mcycle? Time may go backwards for the migrated hart if the new core has been running at a lower frequency. Therefore, each hart must have its own mcycle (the architectural state must be migrated), which continues to increment in parallel with all other harts running on the same physical core.


-Chris

> --
> You received this message because you are subscribed to the Google Groups "RISC-V HW Dev" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+un...@groups.riscv.org.

> To post to this group, send email to hw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/CAEz%3Dso%3DTs8ymDS5p-7V_tQUtXdSL_ax%3Doy6S9-0svP7QUOjX0A%40mail.gmail.com.

[Jacob Bachmeyer]

Christopher Celio wrote:
>> For SMT, HT, or related technologies, it would be a significant loss
>> of performance data if each thread had its own MCYCLE which
>> incremented only when that thread took possession of the execution
>> units.
>>
>
> I believe that each hart MUST have its own mcycle (it is architectural state), but that each hart on a physical core has their mcycles incremented together. Let me show my reasoning:
>
> The intent behind mcycle suggests that each physical core has its "own" mcycle counter, and if 4 harts are scheduled simultaneously onto one core --- I want to know if one hart gets descheduled for 100 cycles.
>

This seems to be the purpose of minstret, which counts instructions
executed, while mcycle counts elasped time in "flexible" units of cycles
and mtime counts elapsed time in some fraction of seconds.

> But what happens if one of the 4 harts gets migrated to another physical core, which has its own separate mcycle? Time may go backwards for the migrated hart if the new core has been running at a lower frequency. Therefore, each hart must have its own mcycle (the architectural state must be migrated), which continues to increment in parallel with all other harts running on the same physical core.
>

This is the same as migrating a user thread from a faster core to a
slower core -- in such a system, the performance counters (except mtime)
can appear to move backwards when a thread is migrated.

On a related note, does the ISA currently require that all harts in a
system have synchronized mtime counters? Or at least that all mtime
counters in the system advance at the same rate?


-- Jacob

[Sean Halle]

Thanks Jacob, that actually brings up a point for us, as well.  In the past, our automated performance tuning has encountered issues with gathering performance info when software threads migrate often, between different cores.  That's the case for one of our high performance software platforms.  Upon each migration the performance counters, of course, change.  It would be nice to come up with a solution to that problem, if we can put our heads together, on this list, and brainstorm something.  Ideas?

--
You received this message because you are subscribed to the Google Groups "RISC-V HW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+un...@groups.riscv.org.
To post to this group, send email to hw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/59F264F2.80403%40gmail.com.

[Christopher Celio]

Shouldn't ALL architectural state migrate? That includes HPM counters (of which mcycle is one).

-Chris

[Jacob Bachmeyer]

Sean Halle wrote:
> Thanks Jacob, that actually brings up a point for us, as well. In the
> past, our automated performance tuning has encountered issues with
> gathering performance info when software threads migrate often,
> between different cores. That's the case for one of our high
> performance software platforms. Upon each migration the performance
> counters, of course, change. It would be nice to come up with a
> solution to that problem, if we can put our heads together, on this
> list, and brainstorm something. Ideas?

Context switching and migration generally tend to make a mess of
performance measurements anyway. The approach I use with RDTSC is to
use three bins: one accumulates all samples, one accumulates samples
less than or comparable to the current mean, one accumulates "outliers"
much greater than the current mean. The mean is computed over all
samples. This allows to get reasonable measurements of time needed to
execute a code fragment on a multitasking system. On an SMP system, it
is necessary to bind the process that collects the samples to a single
logical CPU.

The important takeaway from this that I see for you (and RISC-V) is that
mtime should be required to be global and synchronized across an entire
system, to provide a stable reference independent of migrations and
multiprocessing. System software that performs migrations can then also
record values of minstret and mcycle before and after a thread is
migrated. The counter delta registers from prior revisions of the
privileged ISA solved this problem exactly. Perhaps their removal
should be reconsidered? An implementation that does not want that
complexity could always hardwire all deltas to zero.

The presence of a supervisor-controlled time delta applied to the time
counter used by RDTIME could even allow systems to assign a defined
epoch to time, making it an easily-accessed real-time clock.


-- Jacob

[Christopher Celio]

Context switching and migration generally tend to make a mess of performance measurements anyway.

How much architectural state must be migrated? 32 integer registers? 32 FP registers? 32 vector registers?  CSR state?  Do you also need to move cache state (ARM's big.LITTLE)? The cost to read/write mcycle/minstret seems relatively small in comparison. And since we can have up to 29 HPMs, perhaps the OS can use similar tricks to accelerator_enabled to decide if it needs to migrate them or not.