>>>>> On Sun, 19 Nov 2017 01:11:00 -0800 (PST), Tommy Murphy <tommy_...@hotmail.com
| Thanks Kristen.
| But some of this raises more questions than it provides answers for me.
| Can the term/concept be defined in a clear, concise and jargon free manner as I asked earlier?
|| In simple language
| Hmmm, that's debatable... :-)
In simple language, a hart is a "hardware thread" and means the same
as any other architecture's hardware thread. If your use case does
not involve anything more than unprotected machine-mode code running
directly on a real machine, then a hart is literally a hardware
thread. The difference between RISC-V cores and harts is the same as
that between x86/POWER/ARM/MIPS/SPARC cores and hardware threads -
there is nothing new here. (If you consider "hardware thread" to be
jargon, I'd suggest a more general computer architecture tutorial
before diving into RISC-V land.)
The reason for my more complicated definitions are that since the
1960s, architects and system designers have been adding layers of
abstraction and virtualization to provide more protection and greater
efficiency. So, what seems to be a hardware construct at one level is
actually virtualized at another level, yet it is usually still most
simply described as if it were hardware, as that's the mental model
software authors have when programming at each level and, except for
performance, the behavior is close, if not identical, to that of a
direct hardware implementation.
For example, the RISC-V instruction set definitions in the user manual
describe being executed within an execution environment. The execution
environment can be provided by a bare machine (giving full access to
the physical address space), a hypervisor (implementing a recursive
hypervisor instance or implementing the SBI for each guest OS), or an
operating system (providing a user process). Code is written assuming
some execution-environment binary interface (e.g., ABI for user-level
application code or what we call SBI for operating system code) that
describes how the environment is initialized, what instructions are
allowed and what memory, architectural state, and CSRs are visible,
and also what environment calls are provided to request service from
the execution environment (e.g., I/O, or exit). Hypervisors or OSs
provide execution environments as virtual machines, each supplying a
particular abstract machine interface to the code running inside the
virtual machine, and all multiplexed by the hypervisor/OS onto the
resources the hypervisor/OS obtains from its own execution
environment. (RISC-V privileged architecture was designed to support
recursive virtualization so a hypervisor could actually be running
under another hypervisor, for example.)
Back to hart definitions - from the perspective of the software
running in a given virtual machine, a hart is something that has the
defined hart architectural state and that independently fetches and
executes RISC-V instructions within that virtual machine. Now, the
virtual machine implementation might time-multiplex a set of virtual
machine harts onto fewer harts in the underlying physical hardware (or
underlying virtual machine) but has to do so in a way such that they
operate like independent hardware threads. In particular, it must be
able to preempt threads and cannot wait indefinitely for guest
software on a hart to "yield" control of the hart. (As an aside, the
execution environment could be emulating the RISC-V harts on an x86
server a la QEMU or rv8.)
The important distinction between hardware thread and software thread
here is really that the software inside a virtual machine is not
responsible for causing progress of each of hardware threads exposed
inside the virtual machine, that is the responsibility of the outer
execution environment - so it seems like "hardware" to the software
inside the virtual machine.
Given the above confusions, you might ask why we picked hardware
thread as a name instead of "virtual processor" or "environment
thread". These and other suggestions had their own potential causes
of confusion. For example, CPU has lost any useful meaning now that
it's often used to refer to a whole SoC. Virtual processor is
probably the best alternative, except "processor" is usually
associated with "core" rather than thread, so we'd have questions
whether a virtual processor is multithreaded and we'd need another
name for those threads. Also, it seems strange to describe the sole
thread on a real core as a virtual processor. Virtual thread can be
confused with a regular software thread context.
|| a full set of RISC-V architectural registers
| Does this mean x and fp regs only?
It means the set of architectural registers associated with a RISC-V
hart depending on the supported set of instruction set extensions and
privilege level at which it is running. So, no fp registers if there
is no floating-point, and the set of CSRs visible depends on the
extensions and privilege level. The execution-environment binary
interface defines this for each hart in the support virtual machine.
| After all some csrs are memory mapped so presumably cannot be unique to a hart.
CSRs are used for hart-specific state only. CSRs are not memory
mapped - they are accessed by CSR instructions. Some state that is
writable via memory-mapped IO can be read via CSRs (e.g., mstatus.MSIP
and other interrupt pending inputs). We avoid state that is writable
both via a CSR address and through memory-mapped IO.
|| it means the user-visible harts and memory (i.e., a multithreaded Unix user process).
| Did you mean e.g. rather than i.e.?
Yes, e.g.. Apologies for confusion.
| Otherwise this seems to severely constrain what a hart can do.
| If harts are independent then surely one could run Linux and another
| could run FreeRTOS for example?
Yes. In the simplest case, M-mode software provides a simple virtual
machine to each S-mode OS using PMPs to partition the physical address
space of the machine, and by providing each OS instance with a given
subset of the total machine harts. The M-mode interrupt delegation
registers can be used to direct certain device interrupts directly to
particular harts within each OS.
|| one hart per core without hardware multithreading, or multiple harts per core with hardware multithreading
| Whatever about hart becoming clearer I still don't get how a hart
| differs from a core either at an architectural hardware level or an
| abstract software level?
As above, RISC-V is no different than any other architecture wrt cores
versus hardware threads. A core is usually considered a purely
physical thing. A core implements one or more harts, where if there
are multiple harts, they are time-multiplexing some common hardware
components (e.g., instruction fetch, physical registers, ALUs,
predictor state, etc.). The concept of a core is usually not directly
represented at other abstraction levels (except as a performance hint
to preferentially schedule work on harts on different cores instead of
harts on the same core). You could imagine exposing the hart-core
mapping more abstractly so higher virtualization levels could avoid
scheduling work on the same core if other cores were idle, but I think
experience has been this gets too complicated/restrictive for
software. Another reason you might expose cores to higher-level code
is for security reasons, where you might not allow code from two
different security domains to run simultaneously on different harts on
the same core to prevent high-bandwidth timing attacks.
|| Some systems can migrate harts dynamically between cores
| How can the do this if hart is a hardware resource?
Hopefully, this should be clearer now.
|| each hart is a real hardware thread
|| we retain the concept of a hart as a resource abstraction
| How can it be both real hardware AND an abstraction?
The abstraction can just be implemented directly in hardware as one
valid implementation, and even when virtualized it behaves like
hardware. In all cases, it's a resource within an execution
environment that has state and advances along executing a RISC-V
instruction stream independently of other software inside the same
While I can imagine some desire to have a term that could only be used
to describe a real piece of hardware that executes code, that would
have very limited usefulness. Even when just developing the silicon,
and running the system under simulation, you would have to use a
different name to describe your simulated hardware engine - and
engineers don't think that way. A simulation of an "X" is still
called an "X".
| To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/9331facc-232b-4ccc-8143-8cc3ae200d6c%40groups.riscv.org