On Mon, 2020-06-15 at 10:59 -0400, Gabriel L. Somlo wrote:
> My $0.02:
> Re. Linux drivers: I think the idea of a rebasing, clean,
> "for-upstream" repo containing a current representation of what
> we think is the "Right Thing (TM)" to work against at any given time
> is a great one. Also, bonus points if @benh is offering to curate,
> given his inside knowledge into what's likely to be most palatable
> to (and encounter least resistance from) upstream.
Let's see, I don't want to walk on the toes of others. The Antmicro
folks seem to be maintaining the patch series for upstream at the
moment, so if they want to continue owning that process I won't get in
the way :-)
> Re. CSR "architecture", my utopian end result would be
> registers I can directly access via load/store operations
> (straightforward MMIO), in a predictable (LE) endianness.
I would love for CSRs to just be ordinary registers that a driver
directly accesses with readl/writel indeed :-)
That said, I don't want to be the one to dismiss the use cases
described by Tim that require a more complex mechanism to cope with
8-bit "transport" or even non-MMIO transports such as USB etc..
> Also, let's not lose track of large (> 64 bit) registers, registers
> that aren't a multiple of 4 or 8 bytes in size.
Those will never be accessed atomically, I think we agree on this. So
in this regard, they are effectively just multiple CRSs containing
"related" values. We have an open github issue on this, I think you and
I agree that the current layout of these things is sub-optimal as they
don't end up looking like a "byte stream" to the OS and they should.
> Maybe having CSRs
> that are known to represent numbers (and therefore are laid out in
> LE order) vs. CSRs that represent byte strings (which can be
> memcpy'd) would be worth a bit of additional gateware hassle?
It's not a significant hassle at all HW side I believe (am I wrong here
?) But yes, such CSRs are like FIFOs. IE, They convey byte streams
rather than numbers, thus they need to be layout in what is
traditionally called "byte address invariant" mode, where the address
of indivudal bytes is what matters.
The typical problem with endianness is that people (esp CPU designers
:-) tend to confuse the transport with the data that is transported.
Ideally a bus shouldn't have an endianness. It should have a "byte
order" which defines which bytes represent the first address and which
byte represents the last address in ascending order (ie, which bit
lanes since bits don't have to be numbered in a descending way... see
old school IBM stuff who numbers bits "backwards").
Endianness per-se is a property of a "value" or more specifically a
larger-than-a-byte "values" (this in itself is a simplification of the
general concept that assumes that a byte is an octet, ie 8 bits, but
let's not go down that specific rabbit hole :-)
A byte stream (FIFO or sequence of registers holding a complex piece of
data) do not have an endian per se. Fields inside that byte stream
might but it's completely unrelated to how the stream is transported.
So for things like CSRs, in essence, what matters is whether they
contain a "vlaue" in which case they do have endianness and need to
present the parts of that value onto the appropriate byte lanes for a
given endian, or they represent a higher level structure (or byte
stream which may have no structure at all) in which case they should
represent the bytes in ascending address order, which ensures that
You may notice that both a are fundamentally equivalent: One should be
able to memcpy's a whole bunch of CSRs and contain an identical image
in memory that can be "interpreted" with the appropriate endian
> I'm ok with 32-bits being the "atomic" transfer width. However, I'd
> also also like to see 64-bit-and-larger CSRs aligned on a 64-bit
> address boundary, so a 64-bit access can be made by the CPU and
> transparently handled by the 64-to-32 down converter.
Fair enough, that's just a matter layout. We probably want to "fix" CSR
addresses anyways (another discussion and another github issue), so
that could be part of the "rules" and CSRStorage could even enforce it.
As far as I'm concerned, a detail :-)
> Any additional features (and complexity) is optional as far as I'm
> concerned -- I'm down with whatever as long as there's a well defined
> interface drivers can use that covers all the cases: up-to-64-bit LE
> numeric registers, arbitrary-length byte-string (memcpy) registers,
> and whatever I've missed because *I* personally haven't hit my head
> against it yet... :)
Right, so in Linux terms: readl/writel/readq/writeq for the former,
memcpy_to/from_io for the latter. There's also the FIFO case which is a
variant of the "byte stream" case, for which there is the "s" accessors
(readsl etc..). This is the standard driver toolbox.
> (in the current accessor set -- common.h in the litex bios, litex.h
> the litex-rocket-rebase kernel branch -- I also had arrays of uint16
> and uint32 embedded inside larger -- 64 or 128 bit -- registers, but
> I suppose if each driver rolls its own for *that* corner case it
> should be acceptable, and rare).
This can be handled locally in drivers.