RfD: Memory Access - v2 (Long)
Peter Knaggs
===================
Change History
==============
20100301 Updated after discussion on groups and lists,
Changed word names to those use by Mitch Bradly as they
have more precedence than those originally proposed.
20100225 Corrections to section numbering,
Corrections to reference implementation,
Added Josh Grams unit tests,
Moved L words to EXT.
20100224 Revised reference implementation
20090923 Restored B@ and B!.
20090921 Forth200x review release.
20090902 Forth200x review update.
20090829 First writing according to the Proposals Process as
described in the Draft 200x Standard.
Problem
=======
ANS Standard Forth lacks a way of accessing memory elements of a
fixed width in a portable way. This is useful for sharing data
between applications in the same or different machines.
We assume that the data is byte-oriented.
Solution
========
A new set of words is proposed, the MEMORY-ACCESS wordset, so that
the desired memory size can be selected.
Typical use
===========
CREATE DATA 1 L,
2 W,
3 B,
DATA DUP L@ . 4 BYTES + DUP W@ . 2 BYTES + B@ .
Remarks
=======
1. These words form the new Memory-Access wordset.
2. The names follow the notation:
<endian> - <size> <action> <space>
where:
endian: BE for big-endian
LE for little-endian
size: B for 8 bit byte
W for 16 bit word
L for 32 bit long-word
X for 64 bit extended-word
action: !, @ or , with the usual meaning
Systems with multiple address spaces, e.g. Harvard architectures
and cross-compilers often require an address space indicator:
space: C code space
D data space
R register space
T target address space in cross-compiler
J JTAG or debug link
Note that all operations are unsigned.
Should we allow for signed operations?
3. The term 'native order' as been chosen as 'local order' might be
confused with some order related to local variables. The word
'host' is also used in the Berkeley Sockets API.
4. When operating on data larger than an address unit, memory
operations shall be capable of unaligned operation, e.g. when
fetching a 32 bit item from a non 32-bit aligned address, the
operation will succeed.
5. Systems shall not implement words requiring or returning items
larger than the cell size. It only causes portability issues
rather than solving them.
The rationale for these remarks is that data transfer standards
exist that are big-endian, e.g. TCP/IP, and little-endian, e.g.
USB. This forces us to make a clear distinction between big-
endian, little-endian and native data.
For cell addressed machines which use an address unit larger than
8 bits, it is assumed that the upper part of a cell is simply
ignored. This proposal makes no attempt to deal with packing of
bytes for memory efficiency. Providing that other operations such
as TYPE produce the expected result, the implementation may deal
with packing of data as it sees fit. However, the model of one
byte per cell will always work with least implementation
complexity.
Proposal
========
18. The optional Memory-Access word set
18.1 Introduction
-----------------
All memory access operations in this wordset are defined on one or
more successive address-aligned bytes. In MEMORY-ACCESS, wherever
it say "store" and "fetch", the following applies:
(1) It is assumed that bytes do not require address alignment.
(2) For address units larger than 8 bits, each address unit
contains one byte stored in the eight least significant bits.
Additional bits shall be ignored by the fetch operations, and
should be set to zero by the store operations.
(3) For address units smaller than 8 bits, end orientation of a
byte stored in successive address units is implementation
dependent.
The words in this wordlist generally take for form:
<endian> - <size> <action> <space>
where:
endian: BE for big-endian
LE for little-endian
size: B for 8 bit byte
W for 16 bit word
L for 32 bit long-word
X for 64 bit extended-word
action: !, @ or , with the usual meaning
Systems with multiple address spaces, e.g. Harvard architectures
and cross-compilers often require an address space indicator:
space: C code space
D data space
R register space
T target address space in cross-compiler
J JTAG or debug link
18.2 Additional terms
---------------------
big-endian:
The most significant byte of a multi-byte value is stored at the
lowest memory address. Also known as network order.
little-endian:
The lest significant byte of a multi-byte value is stored at the
lowest memory address.
native-order:
The byte ordering for multi-byte values which suites the system
architecture. (See big-endian and little-endian.)
18.3 Additional usage requirements
----------------------------------
18.3.1 Environmental queries
----------------------------
Append table 18.1 to table 3.5.
See: 3.2.6 Environmental queries.
Table 18.1: Environmental Query Strings
String Value data type Constant? Meaning
MEMORY-ACCESS flag no Memory-access word set present
MEMORY-ACCESS-EXT flag no Memory-access extensions word
set present
18.4 Additional documentation requirements
------------------------------------------
None
18.5 Compliance and labeling
----------------------------
18.5.1 ANS Forth systems
------------------------
The phrase "Providing the Memory Access word set" shall be appended
to the label of any Standard System that provides all of the
Memory Access word set.
The phrase "Providing name(s) from the Memory Access Extensions word
set" shall be appended to the label of any Standard System that
provides portions of the Memory Access Extensions word set.
The phrase "Providing the Memory Access Extensions word set" shall
be appended to the label of any Standard System that provides all
of the Memory-Access and Memory Access Extensions word sets.
18.5.2 ANS Forth programs
-------------------------
The phrase "Requiring the Memory Access word set" shall be appended
to the label of Standard Programs that require the system to provide
the Memory Access word set.
The phrase "Requiring name(s) from the Memory Access Extensions word
set" shall be appended to the label of Standard Programs that require
the system to provide portions of the Memory Access Extensions word
set.
The phrase "Requiring the Facility Extensions word set" shall be
appended to the label of Standard Programs that require the system
to provide all of the Memory Access and Memory Access Extensions
word sets.
18.6 Glossary
-------------
18.6.1 Memory-Access words
--------------------------
18.6.1.aaaa B! "b-store" MEMORY-ACCESS
( x addr -- )
Store the 8 LSBs of x at addr. In system where the address unit is
larger than 8 bits, the upper bits are set to zero.
18.6.1.aaab B@ "b-fetch" MEMORY-ACCESS
( addr -- x )
Fetch the 8 LSBs of x from addr. If the cell size is greater than
8 bits, the result is zero-extended.
18.6.1.aaac BE-W! "b-e-w-store" MEMORY-ACCESS
( x addr -- )
Store the 16 LSBs of x at addr in big-endian format irrespective of
alignment. In systems where the address unit is larger than 16 bits,
the upper bits are set to zero.
18.6.1.aaad BE-W, "b-e-w-comma" MEMORY-ACCESS
( x -- )
Reserve 16 bits of data space and store the 16 LSBs of x in them in
big-endian format irrespective of alignment.
18.6.1.aaae BE-W@ "b-e-w-fetch" MEMORY-ACCESS
( addr -- x )
Fetch the 16 LSBs of x from addr in big-endian format irrespective
of alignment. If the cell size is greater than 16 bits, the result
is zero-extended.
18.6.1.aaaf BYTES "bytes" MEMORY-ACCESS
( n1 -- n2 )
N2 is the size in address units of 8 bit units.
18.6.1.aaag LE-W! "l-e-w-store" MEMORY-ACCESS
( x addr -- )
Store the 16 LSBs of x at addr in litle-endian format irrespective
of alignment. In systems where the address unit is larger than 16
bits, the upper bits are set to zero.
18.6.1.aaah LE-W, "l-e-w-comma" MEMORY-ACCESS
( x -- )
Reserve 16 bits of data space and store the 16 LSBs of x in them in
little-endian format irrespective of alignment.
18.6.1.aaai LE-W@ "l-e-w-fetch" MEMORY-ACCESS
( x addr -- )
Fetch the 16 LSBs of x from addr in little-endian format irrespective
of alignment. If the cell size is greater than 16 bits, the result
is zero-extended.
18.6.1.aaaj W! "w-store" MEMORY-ACCESS
( x addr -- )
Store the 16 LSBs of x at addr in native order irrespective of
alignment. In systems where the address unit is larger than 16 bits,
the upper bits are set to zero.
18.6.1.aaak W, "w-comma" MEMORY-ACCESS
( x -- )
Reserve 16 bits of data space and store the 16 LSBs of x in them in
native order irrespective of alignment.
18.6.1.aaal W@ "w-fetch" MEMORY-ACCESS
( addr -- x )
Fetch the 16 LSBs of x from addr in native order irrespective of
alignment. If the cell size is greater than 16 bits, the result is
zero-extended.
18.6.1.aaam WALIGN "w-align" MEMORY-ACCESS
( -- )
If the data-space pointer is not 16-bit aligned, reserve enough
space to align it.
See: 3.3.3 Data space, 3.3.3.1 Address alignment.
18.6.1.aaan WALIGNED "w-aligned" MEMORY-ACCESS
( addr1 -- addr2 )
Addr2 is the first 16-bit aligned address greater than or equal to
addr1.
See: 3.3.3.1 Address alignment, 18.6.1.aaam WALIGN.
18.6.2 Memory-Access extension words
------------------------------------
18.6.2.aaaa BE-L! "b-e-l-store" MEMORY-ACCESS EXT
( x addr -- )
Store the 32 LSBs of x at addr in big-endian format irrespective of
alignment. In systems where the address unit is larger than 32 bits,
the upper bits are set to zero.
18.6.2.aaab BE-L, "b-e-l-comma" MEMORY-ACCESS EXT
( x -- )
Reserve 32 bits of data space and store the 32 LSBs of x in them in
big-endian format irrespective of alignment.
18.6.2.aaac BE-L@ "b-e-l-fetch" MEMORY-ACCESS EXT
( addr -- x )
Fetch the 32 LSBs of x from addr in big-endian format irrespective
of alignment. If the cell size is greater than 32 bits, the result
is zero-extended.
18.6.2.aaad L! "l-store" MEMORY-ACCESS EXT
( x addr -- )
Store the 32 LSBs of x at addr in native order irrespective of
alignment. In systems where the address unit is larger than 32 bits,
the upper bits are set to zero.
18.6.2.aaae L, "l-comma" MEMORY-ACCESS EXT
( x -- )
Reserve 32 bits of data space and store the 32 LSBs of x in them in
native order irrespective of alignment.
18.6.2.aaaf L@ "l-fetch" MEMORY-ACCESS EXT
( addr -- x )
Fetch the 32 LSBs of x from addr in native order irrespective of
alignment. If the cell size is greater than 32 bits, the result is
zero-extended.
18.6.2.aaag LALIGN "l-align" MEMORY-ACCESS EXT
( -- )
If the data-space pointer is not 32-bit aligned, reserve enough
space to align it.
See: 3.3.3 Data space, 3.3.3.1 Address alignment.
18.6.2.aaah LALIGNED "l-aligned" MEMORY-ACCESS EXT
( addr1 -- addr2 )
Addr2 is the first 32-bit aligned address greater than or equal to
addr1.
See: 3.3.3.1 Address alignment, 18.6.2.aaag LALIGN.
18.6.2.aaai LE-L! "l-e-l-store" MEMORY-ACCESS EXT
( x addr -- )
Store the 32 LSBs of x at addr in little-endian format irrespective
of alignment. In systems where the address unit is larger than 32
bits, the upper bits are set to zero.
18.6.2.aaaj LE-L, "l-e-l-comma" MEMORY-ACCESS EXT
( x -- )
Reserve 32 bits of data space and store the 32 LSBs of x in them in
little-endian format irrespective of alignment.
18.6.2.aaak LE-L@ "l-e-l-fetch" MEMORY-ACCESS EXT
( addr -- x )
Fetch the 32 LSBs of x at addr in little-endian format irrespective
of alignment. If the cell size is greater than 32 bits, the result
is zero-extended.
A.18 The optional Memory-Access word set
Forth program frequently has to transfer data over protocols that
define the byte order of the data being transferred. Data transfer
standards exist that are big-endian, e.g. TCP/IP, and little-endian,
e.g. USB. This forces us to make a clear distinction between big-
endian, little-endian and native data.
The term 'native order' as been chosen as 'local order' might be
confused with some order related to local variables. The word
'host' is also used in the Berkeley Sockets API.
When operating on data larger than an address unit, memory operations
shall be capable of unaligned operation, e.g. when fetching a 32 bit
item from a non 32-bit aligned address, the operation will succeed.
Systems shall not implement words requiring or returning items
larger than the cell size. For example, L@ would be required to
return two 16-bit values on 16-bit systems, while on a 32-bit system
it would be easier it returned a single 32-bit value. In order to
avoid confusion the 32- and 64- bit words are defined in the extended
words section so that only those systems 32- or 64- bit cell need
provide them.
For cell addressed machines, which use an address unit larger than
8 bits, it is assumed that the upper part of a cell is simply
ignored. There is no attempt to deal with packing of bytes for
memory efficiency. Providing that other operations such as TYPE
produce the expected result, the implementation may deal with packing
of data as it sees fit. However, the model of one byte per cell will
always work with least implementation complexity.
Should a program ever need to detect the native order of the system,
it can do so by using the following code:
$1234 PAD ! PAD B@ $34 =
This is true when the programming is running on a little-endian system
and false otherwise.
Reference Implementation
========================
This implementation makes three assumptions:
(1) It is working on a byte addressed system.
(2) A character is stored as a byte.
(3) A stack cell is a minimum of 32-bits wide.
\ Assuming a Char is 8-bits - Not valid on some systems
: B! ( x addr -- ) SWAP $FF AND SWAP C! ;
: B@ ( addr -- x ) C@ $FF AND ;
SYNONYM BYTES CHARS ( n1 -- n2 )
\ Internal helper words (not part of the proposal)
: b@+ ( x1 addr1 -- x2 addr2 ) SWAP 8 LSHIFT OVER B@ + SWAP 1 BYTES + ;
: b@- ( x1 addr1 -- x2 addr2 ) 1 BYTES - DUP B@ ROT 8 LSHIFT + SWAP ;
: b!+ ( x1 addr1 -- x2 addr2 ) 2DUP B! 1 BYTES + SWAP 8 RSHIFT SWAP ;
: b!- ( x1 addr1 -- x2 addr2 ) 1 BYTES - 2DUP B! SWAP 8 RSHIFT SWAP ;
\ Big-endian Memory Access Words
: BE-W@ ( addr -- x ) 0 SWAP b@+ b@+ DROP ;
: BE-L@ ( addr -- x ) 0 SWAP b@+ b@+ b@+ b@+ DROP ;
: BE-W! ( x addr -- ) 2 BYTES + b!- b!- 2DROP ;
: BE-L! ( x addr -- ) 4 BYTES + b!- b!- b!- b!- 2DROP ;
: BE-W, ( x -- ) HERE 2 BYTES ALLOT BE-W! ;
: BE-L, ( x -- ) HERE 4 BYTES ALLOT BE-L! ;
\ Little-endian Memory Access Words
: LE-W@ ( addr -- x ) 0 SWAP 2 BYTES + b@- b@- DROP ;
: LE-L@ ( addr -- x ) 0 SWAP 4 BYTES + b@- b@- b@- b@- DROP ;
: LE-W! ( x addr -- ) b!+ b!+ 2DROP ;
: LE-L! ( x addr -- ) b!+ b!+ b!+ b!+ 2DROP ;
: LE-W, ( x -- ) HERE 2 BYTES ALLOT LE-W! ;
: LE-L, ( x -- ) HERE 4 BYTES ALLOT LE-L! ;
\ Native Memory Access Words
$1234 PAD !
PAD B@ $34 = [IF]
\ Little Endian System
SYNONYM W@ LE-W@
SYNONYM L@ LE-L@
SYNONYM W! LE-W!
SYNONYM L! LE-W!
SYNONYM W, LE-W,
SYNONYM L, LE-L,
[ELSE]
\ Big Endian System
SYNONYM W@ BE-W@
SYNONYM L@ BE-L@
SYNONYM W! BE-W!
SYNONYM L! BE-W!
SYNONYM W, BE-W,
SYNONYM L, BE-L,
[THEN]
\ Alignment Words
: WALIGN ( -- ) HERE 1 AND ALLOT ;
: LALIGN ( -- ) 4 HERE 3 AND - 3 AND ALLOT ;
: WALIGNED ( addr1 -- addr2 ) 1 + [ 1 INVERT ] LITERAL AND ;
: LALIGNED ( addr1 -- addr2 ) 3 + [ 3 INVERT ] LITERAL AND ;
Testing
=======
HEX
T{ 12345678 PAD BE-L! -> }T
T{ 0 PAD 4 BYTES + L! -> }T
\ BE-L@
T{ PAD BE-L@ -> 12345678 }T
T{ PAD 1 BYTES + BE-L@ -> 34567800 }T
\ BE-W@
T{ PAD BE-W@ -> 1234 }T
T{ PAD 1 BYTES + BE-W@ -> 3456 }T
T{ PAD 2 BYTES + BE-W@ -> 5678 }T
T{ PAD 3 BYTES + BE-W@ -> 7800 }T
\ B@
T{ PAD B@ -> 12 }T
T{ PAD 1 BYTES + B@ -> 34 }T
T{ PAD 2 BYTES + B@ -> 56 }T
T{ PAD 3 BYTES + B@ -> 78 }T
\ LE-L@
T{ PAD LE-L@ -> 78563412 }T
T{ PAD 1 BYTES + LE-L@ -> 00785634 }T
\ LE-W@
T{ PAD LE-W@ -> 3412 }T
T{ PAD 1 BYTES + LE-W@ -> 5634 }T
T{ PAD 2 BYTES + LE-W@ -> 7856 }T
T{ PAD 3 BYTES + LE-W@ -> 0078 }T
\ B!
T{ 0 PAD BE-L! FFFFFFFF PAD 1 BYTES + B! -> }T
T{ PAD BE-L@ -> 00FF0000 }T
\ BE-W!
T{ 0 PAD BE-L! FFFFFFFF PAD 1 BYTES + BE-W! -> }T
T{ PAD BE-L@ -> 00FFFF00 }T
T{ 12345678 PAD 1 BYTES + BE-W! -> }T
T{ PAD BE-L@ -> 00567800 }T
\ BE-L!
T{ 0 PAD BE-L!BE FFFFFFFF PAD 1 BYTES + BE-L! -> }T
T{ PAD BE-L@ PAD 4 BYTES + BE-L@ -> 00FFFFFF FF000000 }T
T{ 12345678 PAD 1 BYTES + BE-L! -> }T
T{ PAD BE-L@ PAD 4 BYTES + BE-L@ -> 00123456 78000000 }T
\ B!
T{ 0 PAD LE-L! FFFFFFFF PAD 1 BYTES + B! -> }T
T{ PAD LE-L@ -> 0000FF00 }T
\ LE-W!
T{ 0 PAD LE-L! FFFFFFFF PAD 1 BYTES + LE-W! -> }T
T{ PAD LE-L@ -> 00FFFF00 }T
T{ 12345678 PAD 1 BYTES + LE-W! -> }T
T{ PAD LE-L@ -> 00567800 }T
\ LE-L!
T{ 0 PAD LE-L! FFFFFFFF PAD 1 BYTES + LE-L! -> }T
T{ PAD LE-L@ PAD 4 BYTES + LE-L@ -> FFFFFF00 000000FF }T
T{ 12345678 PAD 1 BYTES + LE-L! -> }T
T{ PAD LE-L@ PAD 4 BYTES + LE-L@ -> 34567800 00000012 }T
\ WALIGNED
T{ 0 BYTES WALIGNED -> 0 BYTES }T
T{ 1 BYTES WALIGNED -> 2 BYTES }T
T{ 2 BYTES WALIGNED -> 2 BYTES }T
T{ 3 BYTES WALIGNED -> 4 BYTES }T
\ LALIGNED
T{ 0 BYTES LALIGNED -> 0 BYTES }T
T{ 1 BYTES LALIGNED -> 4 BYTES }T
T{ 2 BYTES LALIGNED -> 4 BYTES }T
T{ 3 BYTES LALIGNED -> 4 BYTES }T
T{ 4 BYTES LALIGNED -> 4 BYTES }T
T{ 5 BYTES LALIGNED -> 8 BYTES }T
\ ToDo: BYTES
\ ToDo: BE-W, LE-W, W! W, W@ WALIGN
\ ToDo: BE-L, LE-L, L! L, L@ LALIGN
Authors
=======
Federico de Ceballos
Universidad de Cantabria
federico...@unican.es
Stephen Pelc
MicroProcessor Engineering Ltd
steph...@mpeforth.com
Peter Knaggs
University of Exeter
p...@bcs.org.uk
Peter Knaggs
>
> 2. The names follow the notation:
>
> <endian> - <size> <action> <space>
>
> where:
>
> endian: BE for big-endian
> LE for little-endian
> size: B for 8 bit byte
> W for 16 bit word
> L for 32 bit long-word
> X for 64 bit extended-word
> action: !, @ or , with the usual meaning
>
> Systems with multiple address spaces, e.g. Harvard architectures
> and cross-compilers often require an address space indicator:
>
> space: C code space
> D data space
> R register space
> T target address space in cross-compiler
> J JTAG or debug link
>
> Note that all operations are unsigned.
> Should we allow for signed operations?
Although there was reference to signed numbers in the first version,
this has been removed from revised version as there are no signed words
in the proposal.
As Bruce McFarling has identified this would not be so easy. We could
define a set of sign extension words (B>S, W>S and L>S) but they would
only extend the hosts native sign processing. As most of the time people
will be wanting to move to/from the native sign and two's complement then
maybe B>S and friends should be defined to convert from 2c to native.
I don't like this idea, and would rather two simple words to convert
a native format cell to/from a 2c value. On the other hand, how often
are negative values represented in this type of data.
--
Peter Knaggs
Peter Knaggs
>
> 18.6.2.aaad L! "l-store" MEMORY-ACCESS EXT
> ( x addr -- )
>
> Store the 32 LSBs of x at addr in native order irrespective of
> alignment. In systems where the address unit is larger than 32 bits,
> the upper bits are set to zero.
>
> 18.6.2.aaae L, "l-comma" MEMORY-ACCESS EXT
> ( x -- )
>
> Reserve 32 bits of data space and store the 32 LSBs of x in them in
> native order irrespective of alignment.
>
> 18.6.2.aaaf L@ "l-fetch" MEMORY-ACCESS EXT
> ( addr -- x )
>
> Fetch the 32 LSBs of x from addr in native order irrespective of
> alignment. If the cell size is greater than 32 bits, the result is
> zero-extended.
>
> 18.6.2.aaag LALIGN "l-align" MEMORY-ACCESS EXT
> ( -- )
>
> If the data-space pointer is not 32-bit aligned, reserve enough
> space to align it.
>
> See: 3.3.3 Data space, 3.3.3.1 Address alignment.
>
> 18.6.2.aaah LALIGNED "l-aligned" MEMORY-ACCESS EXT
> ( addr1 -- addr2 )
>
> Addr2 is the first 32-bit aligned address greater than or equal to
> addr1.
>
> See: 3.3.3.1 Address alignment, 18.6.2.aaag LALIGN.
WALIGN, WALIGNED, LALIGN and LALIGNED are defined yet the are not
actually required as the only words that would use them (W! W, W@ L!
L, and L@) include the phrase "irrespective of alignment" in their
definition.
Should we thus:
(a) remove the xALIGN xALIGNED words or
(b) remove the "irrespective of alignment" in the native order access
words.
I prefer option (b) as that would allow a system to use the optimum
memory access method.
--
Peter Knaggs
Peter Knaggs
> ( x -- )
>
> Reserve 16 bits of data space and store the 16 LSBs of x in them in
> big-endian format irrespective of alignment.
>
> ( x -- )
>
> Reserve 16 bits of data space and store the 16 LSBs of x in them in
> little-endian format irrespective of alignment.
> 18.6.2.aaab BE-L, "b-e-l-comma" MEMORY-ACCESS EXT
> ( x -- )
>
> Reserve 32 bits of data space and store the 32 LSBs of x in them in
> big-endian format irrespective of alignment.
>
> 18.6.2.aaaj LE-L, "l-e-l-comma" MEMORY-ACCESS EXT
> ( x -- )
>
> Reserve 32 bits of data space and store the 32 LSBs of x in them in
> little-endian format irrespective of alignment.
Are these words that commonly used, or are they really only included
just to complete the set? I can not see any occasion when I would
actually use them.
--
Peter Knaggs
Bruce McFarling
> Are these words that commonly used, or are they really only included
> just to complete the set? I can not see any occasion when I would
> actually use them.
Even if creating, eg, a pre-defined data packet ... BYTES ALLOT and
MOVE suffice *provided* there are no alignment restrictions.
Indeed:
( int16 ) HERE 2 BYTES ALLOT W!
... also works in a pinch in the rare occasion that it might come up.
Bruce McFarling
> Should we thus:
> (a) remove the xALIGN xALIGNED words or
> (b) remove the "irrespective of alignment" in the native order access
> words.
> I prefer option (b) as that would allow a system to use the optimum
> memory access method.
But this wordset is most importantly for externally defined binary
data, and a sequence of externally defined *might not be defined in
such a way that it will be aligned for maximum local efficiency*.
Indeed, some data layouts will have been arrived in contexts and at a
time where hardware that benefited from alignment above the 16bit
level was not a concern.
Under (a), any sequence of 8bit bytes can be interpreted as any
sequence of defined-width integers, as originally specified,
irrespective of the native alignment restrictions.
If this is inefficient for tight inner loops, then given the memory
access words proposed here, and knowledge of the local implementation
cell size, its straightforward to copy the data into locally-aligned
data structures for processing and then use the memory access words to
copy them back. And, of course, that structure can be designed with
the specific data specification at hand, rather than trying to solve
the problem in general, which sometimes ends up swatting a fly with a
sledgehammer.
So I'd prefer to keep the cans of worms closed, and opt for (a). This
also has the advantage of further reducing the namespace hit of the
proposal.
Stephen Pelc
wrote:
>On Fri, 12 Mar 2010 00:57:36 -0000, Peter Knaggs <p...@bcs.org.uk> wrote:
>>
>> 18.6.1.aaad BE-W, "b-e-w-comma" MEMORY-ACCESS
>> 18.6.1.aaah LE-W, "l-e-w-comma" MEMORY-ACCESS
>> 18.6.2.aaab BE-L, "b-e-l-comma" MEMORY-ACCESS EXT
>> 18.6.2.aaaj LE-L, "l-e-l-comma" MEMORY-ACCESS EXT
>Are these words that commonly used, or are they really only included
>just to complete the set? I can not see any occasion when I would
>actually use them.
Try writing a USB descriptor or TCP/IP template without them.
Stephen
--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads
Anton Ertl
>As Bruce McFarling has identified this would not be so easy. We could
>define a set of sign extension words (B>S, W>S and L>S) but they would
>only extend the hosts native sign processing. As most of the time people
>will be wanting to move to/from the native sign and two's complement then
>maybe B>S and friends should be defined to convert from 2c to native.
I think that it is a bad idea to fetch some funny format into a stack
cell, if the only thing one can do with it is to convert it to a
proper format later. It's better to do the conversion on fetching (or,
in the other direction, on storing). Yes, if we want big-endian,
little-endian, and native byte order, then this means we have to
introduce more words. I think that this is a small price to pay for
avoiding the confusion that comes from introducing several new
on-stack number formats.
As an additional benefit, it is also easier to implement such words
efficiently, because most architectures have sign-extending fetching
words (e.g., movsx on IA-32/AMD64).
>I don't like this idea, and would rather two simple words to convert
>a native format cell to/from a 2c value.
It's unclear to me what you propose here. Do you want sign-extending
2s-complement loading words, plus on-stack conversion words from
converting from 2s-complement to the native format. That would
combine the disadvantages of the on-stack conversion with those of
conversion on fetch/store; the only saving grace is, that because all
implementations will have 2s-complement as native format, the
additional words will be noops, so, like CHARS, they will not be used.
>On the other hand, how often
>are negative values represented in this type of data.
What is "this type of data"? Stuff coming from files or over the net?
That probably depends on the application. When writing a
disassembler, I need sign-extension pretty often.
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2009: http://www.euroforth.org/ef09/
Josh Grams
>
> RfD - Memory access
>===================
>
> Change History
>==============
>
> 20100301 Updated after discussion on groups and lists,
> Changed word names to those use by Mitch Bradly as they
> have more precedence than those originally proposed.
> 2. The names follow the notation:
>
> <endian> - <size> <action> <space>
So is OpenFirmware going to change, then? Last I knew they used the
<size><action><endian> form (e.g. W@BE).
--Josh
Josh Grams
>
> WALIGN, WALIGNED, LALIGN and LALIGNED are defined yet the are not
> actually required as the only words that would use them (W! W, W@ L!
> L, and L@) include the phrase "irrespective of alignment" in their
> definition.
>
> Should we thus:
> (a) remove the xALIGN xALIGNED words or
> (b) remove the "irrespective of alignment" in the native order access
> words.
>
> I prefer option (b) as that would allow a system to use the optimum
> memory access method.
Ditto. There are certainly protocols where you need misaligned access,
but the obvious definitions using BYTES, B@, and B! should (I think)
work everywhere this proposal is implemented, so I don't see that as a
big problem.
--Josh
Stephen Pelc
>Ditto. There are certainly protocols where you need misaligned access,
>but the obvious definitions using BYTES, B@, and B! should (I think)
>work everywhere this proposal is implemented, so I don't see that as a
>big problem.
Yes it will work but application programmers will hate it. Having
unaligned operation solves all sorts of problems when dealing with
some of the really horrid hardware and buffering schemes in existence.
There are some parts of TCP that are hard to do if BE-L@/! do not
work at least on 16 bit aligned data rather than 32 bit aligned.
For little-endian there's always USB.
Bruce McFarling
> Try writing a USB descriptor or TCP/IP template without them.
Aha. Without the commas present, I'd likely write it with WFIELD: etc.
for a structure and use the structure items and LE-W@ BE-W@ etc. to
write the template.
But I can see how one could usefully clone a pre-defined template and
only fill in the fields to be modified with the structure items.
Useful info.
OK, keep the comma words.
Bruce McFarling
> actually required as the only words that would use them (W! W, W@ L!
> L, and L@) include the phrase "irrespective of alignment" in their
> definition.
Mitch Bradley in the 200x email list has pointed out information that
means this premise is false.
That is, the premise is that the sole producer/consumer of the data is
implementation memory access. But for, eg, network data packets,
another producer/consumer of the data will be DMA hardware, which may
require alignment of their base on appropriate boundaries.
> Should we thus:
> (a) remove the xALIGN xALIGNED words or
No, since they are required to set data up for some operations
involving a primary target of this wordset.
> (b) remove the "irrespective of alignment" in the native order access
> words.
No, since working with data organized according to some arbitrary
externally defined specification does not permit assuming that
individual items in a block of data will be aligned for greatest
access speed on the hardware of the system.
> I prefer option (b) as that would allow a system to use the optimum
> memory access method.
A two lane each way wide open expressway has an overpass that runs
over a dirt and gravel road. Which right of way is optimal? Answer:
the one that connects to your destination.
The optimum memory access method is the one that allows *the task to
be performed*. @ and ! and kindred are the "implementation can aligned
for best speed if desired" words.
W@ and W! and kindred are the "greatest flexibility in coping with
arbitrary data layouts" words. Imposing alignment restrictions *on*
them breaks their functionality.
Bruce McFarling
OpenFirmware doesn't have them, though an OpenFirmware implementation
may of course provide them. From the email list, Mitch Bradley's
implementation provides the above:
"My OFW implementation includes the following words in the global
Forth
dictionary: le-l@, le-l!, le-w@, le-w!, be-l@, be-l!, be-w@, and be-
w!."
Peter Knaggs
<an...@mips.complang.tuwien.ac.at> wrote:
>
> I think that it is a bad idea to fetch some funny format into a stack
> cell, if the only thing one can do with it is to convert it to a
> proper format later. It's better to do the conversion on fetching (or,
> in the other direction, on storing). Yes, if we want big-endian,
> little-endian, and native byte order, then this means we have to
> introduce more words. I think that this is a small price to pay for
> avoiding the confusion that comes from introducing several new
> on-stack number formats.
If it is unsigned it will be in the correct format, the proposed words
would convert from the unsigned format to the native signed format.
> As an additional benefit, it is also easier to implement such words
> efficiently, because most architectures have sign-extending fetching
> words (e.g., movsx on IA-32/AMD64).
True.
>> I don't like this idea, and would rather two simple words to convert
>> a native format cell to/from a 2c value.
>
> It's unclear to me what you propose here.
I am not actually making a proposal at the moment. I wanted to provoke
a discussion on this to see how the group felt on the matter.
--
Peter Knaggs
Peter Knaggs
>
>> 2. The names follow the notation:
>>
>> <endian> - <size> <action> <space>
>
> So is OpenFirmware going to change, then? Last I knew they used the
> <size><action><endian> form (e.g. W@BE).
<size><action><endian> was the original proposal from the Exeter 200x
workshop.
OpenFirmware does not actually provide equivalences, but Mitch Bradley
does use <endian>-<size><action> in his systems.
--
Peter Knaggs
Bruce McFarling
Yes, except since "S" is overloaded as an abbreviation for both "sign"
and "string" I prefer:
B>SIGN W>SIGN extension: L>SIGN
SIGN>B SIGN>W extension: SIGN>L
What they would do is to sign extend from the external data size to
the native cell (and conversely). The fact that the specification
needs not be concerned with native cell size or numeric representation
is just a side effect of providing the main functionality as compactly
as possible.
The fact that a exotic numeric representation of non-exotic cell size
would get standard names to use for what would likely be already
existing two's>native and native>two's conversions is also a side-
effect.
The extension of that proposal might also be used to reserve the
meaning of the signed equivalents of the Memory Access words.
> As most of the time people will be wanting to move to/from the native sign and two's complement then maybe B>S and friends should be defined to convert from 2c to native.
Yes, as externally defined binary data formats relying on numeric
representations other than two's complement are nearly as rare as
Forth systems that do, standardizing conversions from external sign-
complement, one's complement etc. of defined size to the native cell
seems to not be worthwhile.
> I don't like this idea, and would rather two simple words to convert
> a native format cell to/from a 2c value. On the other hand, how often
> are negative values represented in this type of data.
That's what these do. Unlike unsigned, there is no such thing as "a"
2c signed value ... you need to know width to know when you wrap
around from Max-Positive to Max-Negative.
Anton Ertl
>On Fri, 12 Mar 2010 10:49:03 -0000, Anton Ertl
><an...@mips.complang.tuwien.ac.at> wrote:
>>
>> I think that it is a bad idea to fetch some funny format into a stack
>> cell, if the only thing one can do with it is to convert it to a
>> proper format later. It's better to do the conversion on fetching (or,
>> in the other direction, on storing). Yes, if we want big-endian,
>> little-endian, and native byte order, then this means we have to
>> introduce more words. I think that this is a small price to pay for
>> avoiding the confusion that comes from introducing several new
>> on-stack number formats.
>
>If it is unsigned it will be in the correct format, the proposed words
>would convert from the unsigned format to the native signed format.
These are signed non-native numbers that have been zero-extended when
fetching them onto the stack. They have no meaning (at least as far
as the application is concerned) as unsigned numbers, not more than
e.g., their meaning as bit patterns, floating-point numbers or
addresses. Given that, I think your sentence makes no sense,
especially not the bit about the "correct format".
>I am not actually making a proposal at the moment. I wanted to provoke
>a discussion on this to see how the group felt on the matter.
My feeling is that we should have signed and unsigned variants of
these words, and that the syntax should be as usual: no prefix for
signed, U for unsigned.
Gforth does get it not quite right: It has SW@ UW@ W! SL@ UL@ L!, and
an undocumented W@ and L@ (for backwards compatibility) that are
aliases for UW@ and UL@.
Note that if we assume 2s-complement arithmetic (which is sensible
IMO) and if a cell is at least as wide as the target memory location,
we don't need to differentiate between signed and unsigned stores,
that's why there is only W! and L!.
Josh Grams
Oh, huh. My G4 has the <size><action><endian> forms. I assumed it was
standard.
--Josh
Anton Ertl
>2. The names follow the notation:
>
> <endian> - <size> <action> <space>
<space> is often used for the BL character in syntax descriptions.
Better call it <address space>.
>
> where:
>
> endian: BE for big-endian
> LE for little-endian
The noun for "endian" is "byte order".
Are you missing a line for "native"?
> Systems with multiple address spaces, e.g. Harvard architectures
> and cross-compilers often require an address space indicator:
>
> space: C code space
> D data space
> R register space
> T target address space in cross-compiler
> J JTAG or debug link
I don't see any address spaces in the proposed words. I assume that
they all refer to data space (nothing else is standardised). Maybe
you should just leave this complication away. Given that they are not
defined anywhere I would not know how to implement such spaces.
> Note that all operations are unsigned.
> Should we allow for signed operations?
Yes!
I.e., have
<byte order>-<unsigned?><size><action>
where
unsigned?: for signed or alternatively
S for signed (if there is a conflict with "")
U for unsigned
>5. Systems shall not implement words requiring or returning items
> larger than the cell size. It only causes portability issues
> rather than solving them.
Then you have to divide it into several extensions (for voting and
extension queries).
Also, I really don't see Gforth implementing X@ only on 64-bit
platforms. Either everywhere or nowhere.
You could also have variations of the L and X words that deal with
doubles on the stack.
>For cell addressed machines which use an address unit larger than
>8 bits, it is assumed that the upper part of a cell is simply
>ignored. This proposal makes no attempt to deal with packing of
>bytes for memory efficiency.
Good.
However, I see a difference between native accesses and the others: I
expect "native" data to come from other subsystems on the same machine
instead of from files or over the network, and that data will not be
unpacked. I am not sure if the native non-cell access words are
useful on cell-addressed machines at all. But if they are, it's most
likely quite different from the BE and LE variants. Maybe the native
ones should also have a separate extension from the other stuff.
>Providing that other operations such
>as TYPE produce the expected result, the implementation may deal
>with packing of data as it sees fit. However, the model of one
>byte per cell will always work with least implementation
>complexity.
I don't see how TYPE can work as expected with, e.g., /STRING unless
each character has its own address. So I think the first sentence
might be misleading, and the second sentence could be formulated more
strongly.
Another reason why this might be misleading is that TYPE works with
chars, and this proposal deals with bytes, so on the surface they have
nothing to do with each other. And on a cell-addressed machines there
may actually be a real difference between bytes and characters.
>Proposal
>========
>
>18. The optional Memory-Access word set
>
>18.1 Introduction
>-----------------
>
>All memory access operations in this wordset are defined on one or
>more successive address-aligned bytes. In MEMORY-ACCESS, wherever
>it say "store" and "fetch", the following applies:
>
>(1) It is assumed that bytes do not require address alignment.
>
>(2) For address units larger than 8 bits, each address unit
> contains one byte stored in the eight least significant bits.
> Additional bits shall be ignored by the fetch operations, and
> should be set to zero by the store operations.
Good.
>(3) For address units smaller than 8 bits, end orientation of a
> byte stored in successive address units is implementation
> dependent.
Yes. Does this play a role? Do we have wording for the memory order
of cell parts or dfloat parts in the present standard?
>18.2 Additional terms
>---------------------
>
>big-endian:
> The most significant byte of a multi-byte value is stored at the
> lowest memory address. Also known as network order.
>
>little-endian:
> The lest significant byte of a multi-byte value is stored at the
> lowest memory address.
s/lest/least/
>native-order:
> The byte ordering for multi-byte values which suites the system
> architecture. (See big-endian and little-endian.)
That will be interesting for cell-addressed machines:-).
>18.3.1 Environmental queries
>----------------------------
>
>Append table 18.1 to table 3.5.
>
>See: 3.2.6 Environmental queries.
>
> Table 18.1: Environmental Query Strings
>
>String Value data type Constant? Meaning
>MEMORY-ACCESS flag no Memory-access word set present
>MEMORY-ACCESS-EXT flag no Memory-access extensions word
> set present
Given the reactions to the "wordset queries" RfD, which show that
wordset queries are a feature that is hardly used, and also not
universally implemented, my present plan is to leave wordset queries
as a Forth-94 feature that will be supported in Forth200x only as
Forth-94 compatibility feature (i.e., you can query for the Forth-94
versions of the wordsets, not for the Forth200x versions), and to make
it a deprecated feature in the next standard and remove it in some
later standard.
So I don't think we should introduce wordset queries for any new
wordsets.
>18.6 Glossary
>-------------
>
>18.6.1 Memory-Access words
>--------------------------
>
>18.6.1.aaaa B! "b-store" MEMORY-ACCESS
> ( x addr -- )
>
> Store the 8 LSBs of x at addr. In system where the address unit is
> larger than 8 bits, the upper bits are set to zero.
IMO repeating the second sentence everywhere is just bloat. You
spelled it out earlier.
>18.6.1.aaab B@ "b-fetch" MEMORY-ACCESS
> ( addr -- x )
>
> Fetch the 8 LSBs of x from addr. If the cell size is greater than
> 8 bits, the result is zero-extended.
The cell size is guaranteed to be greater than 8 bits.
>18.6.1.aaac BE-W! "b-e-w-store" MEMORY-ACCESS
> ( x addr -- )
>
> Store the 16 LSBs of x at addr in big-endian format irrespective of
> alignment. In systems where the address unit is larger than 16 bits,
> the upper bits are set to zero.
This should be: Store the 8 LSBs zero-extended at addr+1 (or, if you
want to cater for 1 BYTES>1, at addr BYTE+), and the next 8 bits
zero-extended at addr.
Likewise for all the other words dealing with multiple bytes, except
maybe the native order words.
>18.6.1.aaad BE-W, "b-e-w-comma" MEMORY-ACCESS
> ( x -- )
>
> Reserve 16 bits of data space and store the 16 LSBs of x in them in
> big-endian format irrespective of alignment.
No. Reserve 2 BYTES (as in the proposed word BYTES) of data space ...
>18.6.1.aaaf BYTES "bytes" MEMORY-ACCESS
> ( n1 -- n2 )
>
> N2 is the size in address units of 8 bit units.
s/of 8 bit/of N1 8-bit/
>18.6.1.aaam WALIGN "w-align" MEMORY-ACCESS
> ( -- )
>
> If the data-space pointer is not 16-bit aligned, reserve enough
> space to align it.
I assume that natural alignment to 2 bytes is meant here, irrespective
of whether the hardware benefits from this or not. That should be
spelled out (and in the other alignment words).
I miss the X words. Maybe this is just as well. Or maybe we should
have the X words in the extension wordset, with a double as on-stack
representation.
>Should a program ever need to detect the native order of the system,
>it can do so by using the following code:
>
> $1234 PAD ! PAD B@ $34 =
>
>This is true when the programming is running on a little-endian system
>and false otherwise.
It will also return true on a cell-addressed machine. Of course there
is no native byte order on a cell-addressed machine.
>Testing
>=======
Yes! Tests! Very good!
Overall I think this proposal is turning out pretty well.
Anton Ertl
>WALIGN, WALIGNED, LALIGN and LALIGNED are defined yet the are not
>actually required as the only words that would use them (W! W, W@ L!
>L, and L@) include the phrase "irrespective of alignment" in their
>definition.
They may be useful for defining data structures that are aligned in
that way. OTOH, such external data structures are typically pretty
rigidly defined (or you have other problems), so you might just as
well insert "1 BYTES +" or somesuch in the appropriate place, or
define the fields with explicit offsets from the base, e.g.,
14 bytes 0 +FIELD foo-bar drop
16 bytes 0 +FIELD foo-flip drop
>Should we thus:
>(a) remove the xALIGN xALIGNED words or
>(b) remove the "irrespective of alignment" in the native order access
> words.
I don't have a clear picture yet of how the native order access words
will be used. And if we require alignment for them (their Gforth
implementations certainly do), for which situations will it not be
possible to use the LE or BE variants to perform the unaligned access.
For the purposes I have in mind, either the data is aligned (coming
from another system component), the hardware does not care about
alignment (IA-32, IA-64, PPC), or I know the byte order (for an
assembler/disassembler, or for access to some hardware).
But OTOH I expect that these words will be used rarely enough that the
unaligned access overhead will rarely be incurred (and on a lot
hardware that overhead depends only on whether the data is really
unaligned).
Peter Knaggs
<an...@mips.complang.tuwien.ac.at> wrote:
>
> "Peter Knaggs" <p...@bcs.org.uk> writes:
>> 2. The names follow the notation:
>>
>> <endian> - <size> <action> <space>
>
> <space> is often used for the BL character in syntax descriptions.
> Better call it <address space>.
Agreed.
>>
>> where:
>>
>> endian: BE for big-endian
>> LE for little-endian
>
> The noun for "endian" is "byte order".
> Are you missing a line for "native"?
True, how does <order> sound, where a blank is used to indicate the
native order.
>> Systems with multiple address spaces, e.g. Harvard architectures
>> and cross-compilers often require an address space indicator:
>>
>> space: C code space
>> D data space
>> R register space
>> T target address space in cross-compiler
>> J JTAG or debug link
>
> I don't see any address spaces in the proposed words. I assume that
> they all refer to data space (nothing else is standardised). Maybe
> you should just leave this complication away. Given that they are not
> defined anywhere I would not know how to implement such spaces.
So we make D (data space) the default.
>> Note that all operations are unsigned.
>> Should we allow for signed operations?
>
> Yes!
>
> I.e., have
>
> <byte order>-<unsigned?><size><action>
>
> where
> unsigned?: for signed or alternatively
> S for signed (if there is a conflict with "")
> U for unsigned
I would prefer to make the unsigned version the default.
>> 5. Systems shall not implement words requiring or returning items
>> larger than the cell size. It only causes portability issues
>> rather than solving them.
>
> Then you have to divide it into several extensions (for voting and
> extension queries).
The L (32-bit) words are now in the EXT.
> Also, I really don't see Gforth implementing X@ only on 64-bit
> platforms. Either everywhere or nowhere.
The X (64-bit) words are not actually in the proposal. The text is
simply their to reserve the names.
> You could also have variations of the L and X words that deal with
> doubles on the stack.
True, but I see two problems: (a) the order of the items on the stack,
L is not so bad as the words simply take/return a double, X is not so
simple; (b) the name of such words is unclear to me, unless we go for
an <action> of 2! 2@ and 2,
>> (3) For address units smaller than 8 bits, end orientation of a
>> byte stored in successive address units is implementation
>> dependent.
>
> Yes. Does this play a role? Do we have wording for the memory order
> of cell parts or dfloat parts in the present standard?
ANS:
> 3.1.4.1 Double-cell integers
>
> On the stack, the cell containing the most significant part of a
> double-cell integer shall be above the cell containing the least
> significant part.
...
> Placing the single-cell integer zero on the stack above a single-
> cell unsigned integer produces a double-cell unsigned integer with
> the same value. See 3.2.1.1 Internal number representation.
So I guess we probably can state the order of the bits, although I
am not sure to do so.
>> 18.3.1 Environmental queries
>> ----------------------------
>>
>> Append table 18.1 to table 3.5.
>>
>> See: 3.2.6 Environmental queries.
>>
>> Table 18.1: Environmental Query Strings
>>
>> String Value data type Constant? Meaning
>> MEMORY-ACCESS flag no Memory-access word set present
>> MEMORY-ACCESS-EXT flag no Memory-access extensions word
>> set present
>
> Given the reactions to the "wordset queries" RfD, which show that
> wordset queries are a feature that is hardly used, and also not
> universally implemented, my present plan is to leave wordset queries
> as a Forth-94 feature that will be supported in Forth200x only as
> Forth-94 compatibility feature (i.e., you can query for the Forth-94
> versions of the wordsets, not for the Forth200x versions), and to make
> it a deprecated feature in the next standard and remove it in some
> later standard.
>
> So I don't think we should introduce wordset queries for any new
> wordsets.
It is still the only standard way of discovering if the wordset is
available. [DEFINED] and [UNDEFINED] are part of the optional tools
ext word set.
>> 18.6.1.aaaa B! "b-store" MEMORY-ACCESS
>> ( x addr -- )
>>
>> Store the 8 LSBs of x at addr. In system where the address unit is
>> larger than 8 bits, the upper bits are set to zero.
>
> IMO repeating the second sentence everywhere is just bloat. You
> spelled it out earlier.
While I agree that repeating it is bloat, people don't read the
introduction. I prefer the belt and braces approach.
>> 18.6.1.aaab B@ "b-fetch" MEMORY-ACCESS
>> ( addr -- x )
>>
>> Fetch the 8 LSBs of x from addr. If the cell size is greater than
>> 8 bits, the result is zero-extended.
>
> The cell size is guaranteed to be greater than 8 bits.
True, how about "Fetch the 8 LSBs of x from addr, with the upper bits of
x set to zero."
>> 18.6.1.aaac BE-W! "b-e-w-store" MEMORY-ACCESS
>> ( x addr -- )
>>
>> Store the 16 LSBs of x at addr in big-endian format irrespective of
>> alignment. In systems where the address unit is larger than 16 bits,
>> the upper bits are set to zero.
>
> This should be: Store the 8 LSBs zero-extended at addr+1 (or, if you
> want to cater for 1 BYTES>1, at addr BYTE+), and the next 8 bits
> zero-extended at addr.
As we have defined big- and little-endian we may as well refer to it.
Alternatively, I would prefer something like "Store the upper 8 bits
at the lowest memory address, and the lower 8 bits at the next
available memory address".
> Likewise for all the other words dealing with multiple bytes, except
> maybe the native order words.
Surely this is more bloat, simply referring back to big- and little-
endian resolves this.
>> 18.6.1.aaad BE-W, "b-e-w-comma" MEMORY-ACCESS
>> ( x -- )
>>
>> Reserve 16 bits of data space and store the 16 LSBs of x in them in
>> big-endian format irrespective of alignment.
>
> No. Reserve 2 BYTES (as in the proposed word BYTES) of data space ...
How about: Reserve sufficient data space to store 16 bits, storing the
16 LSBs ...
>> 18.6.1.aaam WALIGN "w-align" MEMORY-ACCESS
>> ( -- )
>>
>> If the data-space pointer is not 16-bit aligned, reserve enough
>> space to align it.
>
> I assume that natural alignment to 2 bytes is meant here, irrespective
> of whether the hardware benefits from this or not. That should be
> spelled out (and in the other alignment words).
>
> I miss the X words. Maybe this is just as well. Or maybe we should
> have the X words in the extension wordset, with a double as on-stack
> representation.
To bring it into accordance with the existing ALIGN words it should
read:
If the data-space pointer is not 16-bit aligned, reserve enough data
space to make it so.
--
Peter Knaggs
Peter Knaggs
wordlist as suggested by A.10.6.2. BEGIN-STRUCTURE.
--
Peter Knaggs
Peter Knaggs
> Should we consider adding BFIELD: WFIELD: LFIELD: and XFIELD: to this
> wordlist as suggested by A.10.6.2. BEGIN-STRUCTURE.
If so, then should they be aligned or not?
--
Peter Knaggs
Bruce McFarling
wrote:
> These are signed non-native numbers that have been zero-extended when
> fetching them onto the stack. They have no meaning (at least as far
> as the application is concerned) as unsigned numbers, not more than
> e.g., their meaning as bit patterns, floating-point numbers or
> addresses. Given that, I think your sentence makes no sense,
> especially not the bit about the "correct format".
Provided it fits into the cell, under the proposal, there is one
unique bit pattern on stack that means a given unsigned integer value,
with for address units of eight bits or larger two distinct patterns
of the lower eight bits at each memory address that correspond to that
unique bit pattern on the stack.
So you only need to know that it represents an unsigned integer value
within the implementation range of unsigned integer values, and that
the correct end orientation was used to fetch the value to the stack,
and there is no ambiguity as to what unsigned integer value that it
represents.
By contrast, it represents a signed two's complement integer, you
cannot determine which one it is without the original width. Hence,
either:
(1) a corresponding signed version for each and every defined width
memory access word
(2) conversion of a defined width two's complement signed value to and
from local
(3) a signed version of the local defined width fetch and store alone,
and defined width big/little endian converters.
When this RfD is again brought to a CfV and passes, having fixed the
glitches in the language from the first introduction, the horse will
have left the barn on (3), leaving (1) or (2).
A proposal could well combine (1) and (2), provided that (1) is in the
EXT section of the proposal. Either (1) is a fetch first than a sign
extension, in which case (2) is a factor, or (1) is supported by sign
extending hardware, which can also be used to provide (2) efficiently.
Bruce McFarling
wrote:
> I don't have a clear picture yet of how the native order access words
> will be used.
Code common to a 32-bit implementation and 64-bit implementation with
the same implementation model could usefully identify when the cell is
generic and when its specifically 32bits wide. If the bias is to only
use the defined width when required and leave it as the local cell
width wherever practicable, the efficiency of the defined width access
is not as critical.
Bruce McFarling
Aligned, +FIELD gives unaligned directly.
Anton Ertl
>>> LE for little-endian
>>
>> The noun for "endian" is "byte order".
>> Are you missing a line for "native"?
>
>True, how does <order> sound, where a blank is used to indicate the
>native order.
Good. Except that you should include the "-" with <order>, i.e., "BE-"
is big-endian, and "" is native.
>> I.e., have
>>
>> <byte order>-<unsigned?><size><action>
>>
>> where
>> unsigned?: for signed or alternatively
>> S for signed (if there is a conflict with "")
>> U for unsigned
>
>I would prefer to make the unsigned version the default.
That's counter to the usual convention that the default is signed and
"U" is unsigned (e.g., "<" vs. "U<"). I think that breaking this
convention would be a very bad idea. I would affect not only these
words, but all the others; programmers would then have to learn a more
complex set of rules.
So if we cannot make signed the default because of conflicts with
existing practice, we should make both explicit, i.e. S for signed (as
in S>D) and U for unsigned. We definitely should not make unsigned
the default.
>>> 5. Systems shall not implement words requiring or returning items
>>> larger than the cell size. It only causes portability issues
>>> rather than solving them.
>>
>> Then you have to divide it into several extensions (for voting and
>> extension queries).
>
>The L (32-bit) words are now in the EXT.
Yes, but in the CfV we ask whether systems support the complete
proposal. I think that we should treat it as two proposals (X:memory
and X:memory-ext or somesuch) there, and wrt extension queries.
>> You could also have variations of the L and X words that deal with
>> doubles on the stack.
>
>True, but I see two problems: (a) the order of the items on the stack,
>L is not so bad as the words simply take/return a double, X is not so
>simple;
Why not? L2@ and L2! would be in X:memory and X2@ and X2! would be in
X:memory-ext (and would exist only on systems with cell size >=32
bits).
>(b) the name of such words is unclear to me, unless we go for
>an <action> of 2! 2@ and 2,
That seems the most intuitive for me, too.
>>> (3) For address units smaller than 8 bits, end orientation of a
>>> byte stored in successive address units is implementation
>>> dependent.
>>
>> Yes. Does this play a role? Do we have wording for the memory order
>> of cell parts or dfloat parts in the present standard?
>
>ANS:
>> 3.1.4.1 Double-cell integers
Yes, we have defined it for doubles, but AFAIK we don't have it
explicitly undefined for anything else. It's just implicitly
undefined. I don't see a reason to explicitly undefine it for bytes.
>So I guess we probably can state the order of the bits, although I
>am not sure to do so.
I don't think we should do so.
>> Given the reactions to the "wordset queries" RfD, which show that
>> wordset queries are a feature that is hardly used, and also not
>> universally implemented, my present plan is to leave wordset queries
>> as a Forth-94 feature that will be supported in Forth200x only as
>> Forth-94 compatibility feature (i.e., you can query for the Forth-94
>> versions of the wordsets, not for the Forth200x versions), and to make
>> it a deprecated feature in the next standard and remove it in some
>> later standard.
>>
>> So I don't think we should introduce wordset queries for any new
>> wordsets.
>
>It is still the only standard way of discovering if the wordset is
>available.
First, for these extensions, one can query for them with extension
queries.
More importantly, it seems that most programmers don't see a need to
query for wordsets (and probably no need to query for extensions,
either). They just use the words, and if a system does not have them,
the system will complain much more reliably than any answer to wordset
or extension queries will be.
>[DEFINED] and [UNDEFINED] are part of the optional tools
>ext word set.
But even that is probably still more reliable than wordset queries.
Because while a standard system must have ENVIRONMENT?, there is no
requirement to give a useful answer to wordset queries, and my
impression is that a number of systems don't bother to give useful
answers.
>>> 18.6.1.aaab B@ "b-fetch" MEMORY-ACCESS
>>> ( addr -- x )
>>>
>>> Fetch the 8 LSBs of x from addr. If the cell size is greater than
>>> 8 bits, the result is zero-extended.
>>
>> The cell size is guaranteed to be greater than 8 bits.
>
>True, how about "Fetch the 8 LSBs of x from addr, with the upper bits of
>x set to zero."
Fine.
>>> 18.6.1.aaac BE-W! "b-e-w-store" MEMORY-ACCESS
>>> ( x addr -- )
>>>
>>> Store the 16 LSBs of x at addr in big-endian format irrespective of
>>> alignment. In systems where the address unit is larger than 16 bits,
>>> the upper bits are set to zero.
>>
>> This should be: Store the 8 LSBs zero-extended at addr+1 (or, if you
>> want to cater for 1 BYTES>1, at addr BYTE+), and the next 8 bits
>> zero-extended at addr.
>
>As we have defined big- and little-endian we may as well refer to it.
>Alternatively, I would prefer something like "Store the upper 8 bits
>at the lowest memory address, and the lower 8 bits at the next
>available memory address".
Fine. But if you want to support "1 BYTES > 1", you may want to
replace "next available memory address" with "address of the next
byte" or somesuch.
>> Likewise for all the other words dealing with multiple bytes, except
>> maybe the native order words.
>
>Surely this is more bloat, simply referring back to big- and little-
>endian resolves this.
Yes. The advantage would be that it is clear that not more than 8
bits are used per au. Maybe that effect can also be achieved with a
less bloated formulation, such as:
|Store the 16 LSBs of x in the two bytes starting at addr in
|big-endian format.
The 8 bits per byte, the zero-extension and the non-alignment would
come from the definition of big-endian.
>>> 18.6.1.aaad BE-W, "b-e-w-comma" MEMORY-ACCESS
>>> ( x -- )
>>>
>>> Reserve 16 bits of data space and store the 16 LSBs of x in them in
>>> big-endian format irrespective of alignment.
>>
>> No. Reserve 2 BYTES (as in the proposed word BYTES) of data space ...
>
>How about: Reserve sufficient data space to store 16 bits, storing the
>16 LSBs ...
No, that would be only on au on cell-addressed machines, whereas we
need to have one au per byte here, and distribute each 8-bit chunk to
it's own au.
>>> 18.6.1.aaam WALIGN "w-align" MEMORY-ACCESS
>>> ( -- )
>>>
>>> If the data-space pointer is not 16-bit aligned, reserve enough
>>> space to align it.
>>
>> I assume that natural alignment to 2 bytes is meant here, irrespective
>> of whether the hardware benefits from this or not. That should be
>> spelled out (and in the other alignment words).
>>
>> I miss the X words. Maybe this is just as well. Or maybe we should
>> have the X words in the extension wordset, with a double as on-stack
>> representation.
>
>To bring it into accordance with the existing ALIGN words it should
>read:
>
> If the data-space pointer is not 16-bit aligned, reserve enough data
> space to make it so.
No. Again that's wrong on cell-addressed systems (where everything is
16-bit aligned). If we want these words at all, we want them to
result in an address divisible by 2 BYTES (4 BYTES for LALIGNED).
Anton Ertl
>Should we consider adding BFIELD: WFIELD: LFIELD: and XFIELD: to this
>wordlist as suggested by A.10.6.2. BEGIN-STRUCTURE.
Probably.
Anton Ertl
>On Mar 13, 1:06=A0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:
>> I don't have a clear picture yet of how the native order access words
>> will be used.
>
>Code common to a 32-bit implementation and 64-bit implementation with
>the same implementation model could usefully identify when the cell is
>generic and when its specifically 32bits wide.
I have experience with 32-bit and 64-bit implementations since 1995.
There are very few cases in my experience when I want to access
specifically 32-bit memory (and that memory is certainly not cells);
the two that come to mind is
1) writing a disassembler (especially for machines with 32-bit
instructions), and even there I could easily use the BE or LE words.
Indeed, IIRC on one architecture (PowerPC?) the instructions are
explicitly in one byte order, even though the machine could be used in
the other byte order in theory; so at least on that machine one should
actually use the explicitly ordered access words.
2) Writing a cross-compiler that can compile to a different cell width
and byte order. Oops, scratch that, because we target explicit byte
orders, native order words don't help us for this purpose, either.
One other use I can think of is in a foreign language interface. I
think the requirements from that use are quite different from the
requirements for which we need BE and LE words, so some other choices
may be quite different from the BE and LE choices (in particular, how
to distribute bits to aus on systems where 1 au has more than 8 bits).
Bernd Paysan
> Mitch Bradley in the 200x email list has pointed out information that
> means this premise is false.
>
> That is, the premise is that the sole producer/consumer of the data is
> implementation memory access. But for, eg, network data packets,
> another producer/consumer of the data will be DMA hardware, which may
> require alignment of their base on appropriate boundaries.
Hm, that makes some sort of aligning quite difficult. Imagine creating
a IPv4 over Ethernet frame in place, i.e. you'll reserve the 14 (or 18
for VLAN) bytes in front of the IPv4 packet, fill in the IPv4 and TCP
header information (all misaligned now), and as a last step, you use the
routing table and the ARP cache to fill in the Ethernet addresses. The
DMA engine may require 32 bit alignment (or even worse), so it's
completely impossible to fulfill that request without violating
alignment rules for the IP headers - and of course also the data inside
the IP protocol.
--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/
Bruce McFarling
> Bruce McFarling wrote:
> > Mitch Bradley in the 200x email list has pointed out information that
> > means this premise is false.
> > That is, the premise is that the sole producer/consumer of the data is
> > implementation memory access. But for, eg, network data packets,
> > another producer/consumer of the data will be DMA hardware, which may
> > require alignment of their base on appropriate boundaries.
> Hm, that makes some sort of aligning quite difficult.
I was just thinking about that. I'll take it for granted that as Mitch
Bradley uses it, it solves the specific problem he faces, but don't
know enough about or have enough experience in the task to see the
gotchas.
> Imagine creating a IPv4 over Ethernet frame in place, i.e. you'll reserve the 14 (or 18 for VLAN) bytes in front of the IPv4 packet, fill in the IPv4 and TCP header information (all misaligned now), and as a last step, you use the routing table and the ARP cache to fill in the Ethernet addresses. The DMA engine may require 32 bit alignment (or even worse), so it's completely impossible to fulfill that request without violating alignment rules for the IP headers - and of course also the data inside the IP protocol.
The required alignment could well be on 64bit boundary, the classic
"byte paragraph".
If someone knows *which*, they will know precisely which layer of the
nested packets and frames needs to be aligned. They know how much
prefix info there is to that point. They size the whole prefix, do the
xALIGN or xALIGNED as the case may be, define the start of the
required-to-be-aligned frame/packet, and define prefix fields as
negative offsets from that.
OK, but portably, we don't know WHICH alignment that needs to be *even
if we know which data is handled by the application with alignment-not-
required words and which data is handed to a primitive for handling as
a chunk that may for some implementation require alignment*. So
putting in LALIGNED or XALIGN or wevs in the source is subject to
change, the first system configuration is encountered with more
binding alignment constraint.
So that is all speculation, but it seems to me that Anton is most
likely right, that what is required here is MAXALIGNED:
* The memory access operators allow a word to consume whatever data
has been produced for it to work on,
* but generalizing the kind of alignment restriction that Mitch
Bradley describe on the data that the routine produces, its more like
a MAXALIGN function than a WALIGN function.
It would certainly be more *regular and repeatable* behavior for
WALIGN and friends to actually be explicitly defined as the intrinsic
alignment operators for those address sizes ... explicitly on 2*byte-
access, 4*byte-access, and 8*byte-access boundaries.
Anton Ertl
One other word that we might consider including is:
BYTEWISE ( fam1 -- fam2 )
Modify the implementation-defined file access method fam1 to
additionally select a bytewise, i.e., not line oriented, file access
method, giving access method fam2.
Rationale: The fam should be modified with BYTEWISE when reading data
from a file that is then to be processed with the memory-access words,
or when writing data to a file that was created with the memory-access
words.
On byte-addressed machines this will be an alias for BIN, but on
cell-addressed machines BIN may read files to fill cells, and we need
a separate way to specify that we want each byte to go into a separate
cell.
Peter Knaggs
<an...@mips.complang.tuwien.ac.at> wrote:
> "Peter Knaggs" <p...@bcs.org.uk> writes:
>> RfD - Memory access
>> ===================
>
> One other word that we might consider including is:
>
> BYTEWISE ( fam1 -- fam2 )
>
> Modify the implementation-defined file access method fam1 to
> additionally select a bytewise, i.e., not line oriented, file access
> method, giving access method fam2.
How does this differ from the existing
11.6.1.0765 BIN FILE
( fam1 -- fam2 )
Modify the implementation-defined file access method fam1 to
additionally select a "binary", i.e., not line oriented, file
access method, giving access method fam2.
Rationale:
Some operating systems require that files be opened in a different
mode to access their contents as an unstructured stream of binary
data rather than as a sequence of lines.
The arguments to READ-FILE and WRITE-FILE are arrays of character
storage elements, each element consisting of at least 8 bits. The
Technical Committee intends that, in BIN mode, the contents of
these storage elements can be written to a file and later read
back without alteration. The Technical Committee has declined to
address issues regarding the impact of “wide” characters on the
File and Block word sets.
--
Peter Knaggs
Bruce McFarling
> How does this differ from the existing
> 11.6.1.0765 BIN FILE
> ( fam1 -- fam2 )
> Modify the implementation-defined file access method fam1 to
> additionally select a "binary", i.e., not line oriented, file
> access method, giving access method fam2.
> The arguments to READ-FILE and WRITE-FILE are arrays of character
> storage elements, each element consisting of at least 8 bits.
This would change the arguments to READ-FILE and WRITE-FILE to be
arrays of byte storage elements.
For coherence, all counts that are character counts in FILE and FILE
EXT would become byte counts for a file in the new file access mode.
All changes would be restricted to byte<>char systems. In
byte<>au=char systems, data may be contained in bit8-bit15 or
higher ... OCTET mode data would be returned in line with the W@ etc.
behaviors. In byte=au<>char systems, char is a multiple of bytes, and
if BIN returns packed data, the data will be accessible with W@, etc.,
but there is no way to specify an odd number of bytes.
Peter Knaggs
wrote:
> On Mar 21, 4:31 pm, "Peter Knaggs" <p...@bcs.org.uk> wrote:
>> How does this differ from the existing
>> 11.6.1.0765 BIN
>
> arrays of byte storage elements.
Yes, I see that I have fallen foul of a common error I see in many
posting in this group and replied before considering the whole of the
message. Something I have tried hard not to do over the years.
--
Peter Knaggs