64 bit advantage?
Peter L. Montgomery
Chris Morgan <mih...@ix.netcom.com> writes:
>Aaron Spink <sp...@kraftwerk.pa.dec.com> writes:
>
>> Chris Morgan <mih...@ix.netcom.com> writes:
>>
>>
>> > This lead me thinking - are any of the current benchmarks capable of
>> > showing the difference between "True" 64 bit architectures and 32 bit
>> > ones with bells and whistles?
>>
>> DB, A lot of cryptographic areas, extended precision math(ie, >32 bit
>> numbers), Anything that needs a lot of memory.
>
>And does the SPEC benchmark contain any of these things?
>
>I want to know whether the perceived processor rankings take into
>account the possibilities with true 64 bit cpus. Do Alpha fans feel
>that the advantage the chip has in those situations is correctly
>represented in SPEC benchmarks? I often get the feeling that comments
>along the lines of "SPEC is not as representative as running your apps
>yourself" don't help as much as "SPEC is out of date and should
>include the following types of operations". If SPEC is somewhat
>democratic it is posible there is a 32 bit bias which would naturally
>mask the 64 bit machines' true stature. I don't know if this is true
>but I'd like to hear what others think.
I believe SPEC requires standard Fortran or C.
There is no 64-bit integer type built into either language.
Even it we somehow survive the declarations, such as
integer, parameter :: MY64 = SELECTED_INT_KIND(18) ! 18 digits
integer (KIND = MY64) x1, x2, x3
(where SELECTED_INT_KIND may return -1, causing a compilation error), or
typedef unsigned long long my64_t;
my64_t x1, x2, x3;
(long long is proposed for C9X), we lack the primitive operations we need.
One very important operation is the 128-bit product of
two 64-bit signed or unsigned integers. The 64-bit MIPS and PowerPC
architectures support both of these products, and the Alpha
supports the unsigned products (from which a compiler can
generate code for a signed product), but I know no portable way
to access these instructions from Fortran or C.
C does provide access to the lower product even when it overflows
(via unsigned arithmetic, one of C's strengths), but Fortran does not.
Some Alpha compilers provide a name like __UMULH or _int_mult_upper for the
upper product, but I find no such support in SGI's MIPS compilers.
There is no incentive for Sun to provide it when the UltraSPARC
lacks hardware support for the upper product.
These operations belong in the language standard,
but the committee finds no existing practice to standardize.
C9X is trying to promote integer types such as int32_t and int64_t
but the constants 32 and 64 are hardwired into the type names.
I want to start with one type (say long) and create a second
type with exactly twice as many bits. I also want the number
of bits in a type available as a compile time constant, for use
when computing shift counts. My needs are not being met.
The languages lack support for the population count function.
This was in the CDC 6400 architecture even before FORTRAN 66,
has been in Cray architectures for 25 years, and has been found
sufficiently useful to include in SPARC v9 and Alpha EV6 instruction sets.
It has a simple mathematical definition if we require its argument
to be a nonnegative integer, and is readily implemented as a library
function on machines lacking direct hardware support.
Why not make it a standard function?
We need language support for functions which may not have direct
hardware support. Consider the (truncated) square root of an unsigned
64-bit integer x. The languages support floating point square roots --
why not integer? You may suggest something like
iroot = (unsigned int32_t)sqrt((double)x);
but this is flawed if arithmetic is 53-bit IEEE. For example
x = 2^64 - 2^33 and x = 2^64 - 2^33 + 1 = (2^32 - 1)^2
will yield identical values for (double)x after rounding,
although their truncated square roots are different.
It is very hard to get this right if we use floating
point arithmetic, and anything we try will need
big adjustments if we someday change x to a 128-bit integer but
retain 53-bit floating point arithmetic. The integer square
root should be part of a standard math library, written, optimized,
and tested once for each architecture and integer length.
The Alpha ev6 has leading and trailing zero counts,
just as the x86 has had for years. The leading zero count of a
positive integer should be available indirectly, through its truncated
base-2 logarithm: for example log2(18) = 4. This too has a simple
mathematical definition, and is used throughout my codes.
The binary GCD procedure below illustrates the usefulness of
a trailing zero count instruction (defined on a nonzero operand) --
there should be very good branch prediction throughout the procedure
if the (x1 > y1) test uses a conditional move instruction.
But languages don't provide access to this instruction
for potential SPEC benchmarks to illustrate its usefulness.
typedef unsigned int64_t u64_t;
u64_t GCD(u64_t x, u64_t y) /* Greatest common divisor */
{
u64_t x1, y1;
int ipow2;
if (x == 0 || y == 0) return x | y;
ipow2 = trailing_zero_count(x | y);
/* Power of 2 dividing both x and y */
x1 = x >> trailing_zero_count(x);
y1 = y >> trailing_zero_count(y);
while (x1 != y1) { /* Both x1 and y1 are odd */
u64_t z1 = x1 ^ y1;
if (x1 > y1) x1 = y1; /* MIN(x1, y1) */
y1 = (x1 ^ z1) - x1; /* MAX(x1, y1) - MIN(x1, y1) */
y1 >> = trailing_zero_count(z1);
}
return x1 << ipow2;
}
Come to think of it, why isn't the integer GCD, one of the
first-taught algorithms, part of standard languages?
Its mathematical definition is well standardized,
but the best implementation varies heavily with the hardware.
--
Peter-Lawren...@cwi.nl Home: San Rafael, California
Microsoft Research and CWI
Nick Maclaren
In article <F95D1...@cwi.nl>, pmon...@cwi.nl (Peter L. Montgomery) writes:
|>
|> I believe SPEC requires standard Fortran or C.
|> There is no 64-bit integer type built into either language.
Yes. Sigh.
|> One very important operation is the 128-bit product of
|> two 64-bit signed or unsigned integers. The 64-bit MIPS and PowerPC
|> architectures support both of these products, and the Alpha
|> supports the unsigned products (from which a compiler can
|> generate code for a signed product), but I know no portable way
|> to access these instructions from Fortran or C.
There isn't one - that's why! Come back BCPL, all is forgiven :-)
My proposal to include a slightly enhanced BCPL muldiv in C9X was
rejected, unfortunately.
|> C9X is trying to promote integer types such as int32_t and int64_t
|> but the constants 32 and 64 are hardwired into the type names.
|> I want to start with one type (say long) and create a second
|> type with exactly twice as many bits. I also want the number
|> of bits in a type available as a compile time constant, for use
|> when computing shift counts. My needs are not being met.
Yes. The former is a bit hairy to specify in a language like C, but
the latter is not, and has been requested for both C89 and C9X.
Many compilers have provided it, but have withdrawn it when it was
pointed out that sizeof(int) is required NOT to be a preprocessor
constant :-(
|> The languages lack support for the population count function.
|> It has a simple mathematical definition if we require its argument
|> to be a nonnegative integer, and is readily implemented as a library
|> function on machines lacking direct hardware support.
|> Why not make it a standard function?
I believe that the argument is that it has insufficient utility.
Actually, I agree with this, but it is certainly a lot more useful
than a lot of the crud that has gone into C9X.
|> We need language support for functions which may not have direct
|> hardware support. Consider the (truncated) square root of an unsigned
|> 64-bit integer x. The languages support floating point square roots --
|> why not integer? ... The integer square
|> root should be part of a standard math library, written, optimized,
|> and tested once for each architecture and integer length.
My experiments indicate that assuming that the standard C libraries
have been either optimised or tested is definitely unwise. They are
almost all ghastly, almost beyond belief. But this doesn't actually
affect your argument - there are functions for truncated division
etc., after all.
|> The Alpha ev6 has leading and trailing zero counts, ...
|>
|> Come to think of it, why isn't the integer GCD, one of the
|> first-taught algorithms, part of standard languages?
|> Its mathematical definition is well standardized,
|> but the best implementation varies heavily with the hardware.
Because Fortran was designed for traditional numerical work, where
these are esoteric functions :-)
Seriously. C added a lot of stuff for writing system utilities,
but this sort of thing was not done by the original authors, and
few of the new standardisation committees have had people on them
that have had an interest in this sort of thing.
Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QG, England.
Email: nm...@cam.ac.uk
Tel.: +44 1223 334761 Fax: +44 1223 334679
Larry Kilgallen
> Chris Morgan <mih...@ix.netcom.com> writes:
>>And does the SPEC benchmark contain any of these things?
> I believe SPEC requires standard Fortran or C.
> There is no 64-bit integer type built into either language.
> C9X is trying to promote integer types such as int32_t and int64_t
> but the constants 32 and 64 are hardwired into the type names.
> I want to start with one type (say long) and create a second
> type with exactly twice as many bits. I also want the number
> of bits in a type available as a compile time constant, for use
> when computing shift counts. My needs are not being met.
Your needs can be met, but you have overconstrained the problem by
requiring the answer be standard Fortran or standard C. If you
look at standard Ada, you will see that a compiler _must_ support
the hardware-natural data sizes for the target machine. You can
define your own type names, actually constructing the types from
constants of your choice.
Ada 95 is the current standard, and stardardizes a lot more numerics
than the original Ada 83.
Larry Kilgallen
John R. Mashey
|> Chris Morgan <mih...@ix.netcom.com> writes:
|> >Aaron Spink <sp...@kraftwerk.pa.dec.com> writes:
|> >
|> >> Chris Morgan <mih...@ix.netcom.com> writes:
|> >>
|> >>
|> >> > This lead me thinking - are any of the current benchmarks capable of
|> >> > showing the difference between "True" 64 bit architectures and 32 bit
|> >> > ones with bells and whistles?
|> >I want to know whether the perceived processor rankings take into
|> >account the possibilities with true 64 bit cpus. Do Alpha fans feel
|> >that the advantage the chip has in those situations is correctly
|> >represented in SPEC benchmarks? I often get the feeling that comments
|> >along the lines of "SPEC is not as representative as running your apps
|> >yourself" don't help as much as "SPEC is out of date and should
|> >include the following types of operations". If SPEC is somewhat
|> >democratic it is posible there is a 32 bit bias which would naturally
|> >mask the 64 bit machines' true stature. I don't know if this is true
1) SPEC has no pretense of trying to provide benchmarks for everything.
2) CPU designers often put features in CPUs that some customers think are
important, even if no "standard" benchmark ever cares about it.
3) It's not that there is a 32-bit bias, it's that there is a legitimate
bias towards portability across many machines. [As explained ad nauseum
before, mechanisms like "long" (circa 1978) and "long long" let you write
portable code that at least works as well as it can on smaller machines,
and also works efficiently on larger machines, unlike code in which
int[2]s are all over the place, and are difficult to optimize on the largers.
|> One very important operation is the 128-bit product of
|
|> generate code for a signed product), but I know no portable way
|> to access these instructions from Fortran or C.
Correct, no portable way. Maybe, after the C9X stuff is done, and if
there is some more general sizing mechanism done later, there might be hope
for doing upper-products in some generic way at each size level.
For some of Peter's wishes, it would certainly be nice to standardize on
the function names, even if the compiler can't generate the inline code.
--
-john mashey DISCLAIMER: <generic disclaimer: I speak for me only...>
EMAIL: ma...@sgi.com DDD: 650-933-3090 FAX: 650-933-4392
USPS: Silicon Graphics/Cray Research 40U-005,
2011 N. Shoreline Blvd, Mountain View, CA 94043-1389
Noone Really
> Many compilers have provided it, but have withdrawn it when it was
> pointed out that sizeof(int) is required NOT to be a preprocessor
> constant :-(
That is too strong! Unless sizeof is defined as amcro, sizeof on a
typename in parentheses in a preprocessing directive always violates a
constraint: so, after providing one mandatory diagnostic, the compiler
can do anything it likes. Furthermore, an implementation can provide
__sizeof__ if it likes.
The standard can rarely constrain an implementation to the extent that
it cannot provide an abstract feature.
---
Peter da Silva
Nick Maclaren <nm...@cus.cam.ac.uk> wrote:
>In article <F95D1...@cwi.nl>, pmon...@cwi.nl (Peter L. Montgomery) writes:
>|> There is no 64-bit integer type built into either language.
>Yes. Sigh.
Technically, there is. It's called "long".
IMO "long" should have been a 64 bit type in 32/V and its successors.
Then... now we could worry about "long long" or "int128"...
--
In hoc signo hack, Peter da Silva <pe...@baileynm.com>
`-_-' "Heb jij vandaag je wolf al geaaid?"
'U`
"Tell init(8) to lock-n-load, we're goin' zombie slaying!"
Nick Maclaren
Peter da Silva <pe...@baileynm.com> wrote:
>In article <7dd8om$f77$1...@pegasus.csx.cam.ac.uk>,
>Nick Maclaren <nm...@cus.cam.ac.uk> wrote:
>>In article <F95D1...@cwi.nl>, pmon...@cwi.nl (Peter L. Montgomery) writes:
>>|> There is no 64-bit integer type built into either language.
>
>
>Technically, there is. It's called "long".
>
>IMO "long" should have been a 64 bit type in 32/V and its successors.
>
>Then... now we could worry about "long long" or "int128"...
I can follow your saecond two points, but not your first. Yes, it
probably is what SHOULD have been done, but is NOT what HAS been
put into C9X. 'long' is guaranteed only to be a 32-bit type in
C89 and C9X.
Nick Maclaren
Noone Really <nooneat...@earthlink.net> wrote:
>Nick Maclaren wrote:
>> Many compilers have provided it, but have withdrawn it when it was
>> pointed out that sizeof(int) is required NOT to be a preprocessor
>> constant :-(
>
>That is too strong! Unless sizeof is defined as amcro, sizeof on a
>typename in parentheses in a preprocessing directive always violates a
>constraint: so, after providing one mandatory diagnostic, the compiler
>can do anything it likes. Furthermore, an implementation can provide
>__sizeof__ if it likes.
I should have said "preprocessor constant expression". Yes, doing
this means defining sizeof as a built-in macro which is acceptable
in function context in a preprocessor expression - AND THAT IS ALL.
The macro cannot be IMPLEMENTED in the language (or else I would
write the damn thing myself), which makes it no different from
several others.
Your comment about __sizeof__ is irrelevant, as it would be unusable
in portable programs, and Peter Montgomery and I needed it precisely
to write portable, efficient code.
>The standard can rarely constrain an implementation to the extent that
>it cannot provide an abstract feature.
Eh? This is irrelevant.
Peter da Silva
Nick Maclaren <nm...@cus.cam.ac.uk> wrote:
>I can follow your saecond two points, but not your first. Yes, it
>probably is what SHOULD have been done, but is NOT what HAS been
>put into C9X. 'long' is guaranteed only to be a 32-bit type in
>C89 and C9X.
You're right, you can't depend on the size of a long, or an int... but that
has always been true for people writing portable code. I've written code
that's run on systems with 16, 32, and 36 bit ints, and little endian and
big-endian 32-bit longs, as well as 36 and 64 bit longs. It's not tough.
And code that depends on any type being any size isn't portable.
If you *need* a 64 bit type, then you have to do what Gnu "configure" does and
run a test program that writes a .h file that contains the defined types you
use. This is an old and known problem and ANSI didn't create it.
Nick Maclaren
In article <7dfvqa$g...@web.nmti.com>, pe...@baileynm.com (Peter da Silva) writes:
|> In article <7dfkds$lbf$1...@pegasus.csx.cam.ac.uk>,
|> Nick Maclaren <nm...@cus.cam.ac.uk> wrote:
|> >I can follow your saecond two points, but not your first. Yes, it
|> >probably is what SHOULD have been done, but is NOT what HAS been
|> >put into C9X. 'long' is guaranteed only to be a 32-bit type in
|> >C89 and C9X.
|>
|> You're right, you can't depend on the size of a long, or an int... but that
|> has always been true for people writing portable code. I've written code
|> that's run on systems with 16, 32, and 36 bit ints, and little endian and
|> big-endian 32-bit longs, as well as 36 and 64 bit longs. It's not tough.
|> And code that depends on any type being any size isn't portable.
But you can assume that 'long' is at least 32 bits, though not the
order nor how the sign bit is handled.
|> If you *need* a 64 bit type, then you have to do what Gnu "configure" does and
|> run a test program that writes a .h file that contains the defined types you
|> use. This is an old and known problem and ANSI didn't create it.
I disagree! I find it much more reliable to comment this clearly
in a README and the Makefile and get the user to specify the type
or the required compiler options - and using options is a far more
common way of getting 64-bit types than by using a different type.
There are also other solutions.
Whatever you do, the problem and the solutions predate C by a very
long way.
Peter da Silva
Nick Maclaren <nm...@cus.cam.ac.uk> wrote:
>I disagree! I find it much more reliable to comment this clearly
>in a README and the Makefile and get the user to specify the type
>or the required compiler options - and using options is a far more
>common way of getting 64-bit types than by using a different type.
That's another alternative. If you don't mind the user specifying that
sort of information to get the software to build then there's even less
of an issue.
David R Tribble
>> but the constants 32 and 64 are hardwired into the type names.
>> I want to start with one type (say long) and create a second
>> type with exactly twice as many bits. I also want the number
>> of bits in a type available as a compile time constant, for use
>> when computing shift counts. My needs are not being met.
Nick Maclaren wrote:
> Yes. The former is a bit hairy to specify in a language like C, but
> the latter is not, and has been requested for both C89 and C9X.
> Many compilers have provided it, but have withdrawn it when it was
> pointed out that sizeof(int) is required NOT to be a preprocessor
> constant :-(
I made a proposal in May 1995 to the ISO committee to add
preprocessor macros to <limits.h> to describe the machine
characteristics, but it was rejected. My idea was to have macros
that specified things like: bit sizes of the built-in types; byte
endianness of bytes within words, words within fullwords, and
fullwords within longwords; datatype alignments; the signedness
of plain char, etc. For example, I proposed a macro named
'_BITS_LONG' that evaluated to the number of bits in a 'long int',
and another named '_ALIGN_LONG' that evaluated to the alignment
(in bytes) for the same type.
This sort of thing is exactly what you are asking for. It's a
pity the committee didn't adopt at least some of my proposal; this
topic comes up every few months or so on <news:comp.std.c>, so
there is obviously some interest out there for this kind of
capability. (I believe the committee thought my proposal tried to
cover too much ground and thus wasn't simple enough.)
-- David R. Tribble, dtri...@technologist.com --
For those with historical curiosity, my original proposal can be
found at: <http://www.flash.net/~dtribble/text/c9xmach.txt>.
(It used to be available on the DMK/SC22WG14 FTP site as a file
called "machine-characteristics.txt.gz", but it appears to have
disappeared.)
David R Tribble
> C9X is trying to promote integer types such as int32_t and int64_t
> but the constants 32 and 64 are hardwired into the type names.
> I want to start with one type (say long) and create a second
> type with exactly twice as many bits. I also want the number
> of bits in a type available as a compile time constant, for use
> when computing shift counts. My needs are not being met.
Other than the obvious solution of adding a few macros to <limits.h>
that describe the sizes of the built-in datatypes (which I've
written about in another post on this thread), we could extend the
language in such a way that the actual bit sizes (32, 64, whatever)
were not "hardwired" into the datatype names. Something along the
lines of:
int<2> i2; // sizeof(i2) == 2
int<4> i4; // sizeof(i4) == 4
int<8> i8; // sizeof(i8) == 8
Since sizeof() is defined such that sizeof(char)==1, and the
number of bits in a char is unspecified, we have a portable
method for declaring integers of distinct sizes. In the example
above, i4 is an integer that is the same size as char[4], but
this does not necessarily mean it has 32 bits; it could just as
well have 36 bits on a 32-bit machine (with 9-bit chars), 48
bits on a 24-bit machine (with 12-bit chars), etc. (We may
have to restrict N to be a member of {1,2,4,8} for all int<N>.)
If you think that extending the language is too big of a step
to make, we can try another approach, where the 'int32_t' type
names are replaced with more portable names like 'int4_t',
where sizeof(int4_t)==4. Now we aren't specifying the bit sizes
of these types explicitly, but rather the sizes are expressed in
terms of sizeof().
sizeof(int1_t) == 1 // e.g., signed char
sizeof(int2_t) == 2 // e.g., signed short
sizeof(int4_t) == 4 // e.g., signed int or signed long
sizeof(int8_t) == 8 // e.g., signed long long
(If I'm not mistaken, this resembles the way integer types are
declared in FORTRAN, e.g. INTEGER*4.)
James Kuyper
...
> were not "hardwired" into the datatype names. Something along the
> lines of:
>
> int<2> i2; // sizeof(i2) == 2
> int<4> i4; // sizeof(i4) == 4
> int<8> i8; // sizeof(i8) == 8
>
> Since sizeof() is defined such that sizeof(char)==1, and the
> number of bits in a char is unspecified, we have a portable
> method for declaring integers of distinct sizes. In the example
> above, i4 is an integer that is the same size as char[4], but
> this does not necessarily mean it has 32 bits; it could just as
> well have 36 bits on a 32-bit machine (with 9-bit chars), 48
> bits on a 24-bit machine (with 12-bit chars), etc. (We may
> have to restrict N to be a member of {1,2,4,8} for all int<N>.)
When porting code between machines with different byte sizes, I suspect
that it's far more common to want to retain the same guaranteed minimum
number of bits, than the same number of bytes.
...
> (If I'm not mistaken, this resembles the way integer types are
> declared in FORTRAN, e.g. INTEGER*4.)
Does FORTRAN support the idea of implementation-defined byte sizes, like
C does?
Tim Shoppa
>
> David R Tribble wrote:
> > (If I'm not mistaken, this resembles the way integer types are
> > declared in FORTRAN, e.g. INTEGER*4.)
But that's not standard FORTRAN. (Though it *is* a common
extension to FORTRAN IV and FORTRAN 77).
> Does FORTRAN support the idea of implementation-defined byte sizes, like
> C does?
Again, you have to define what Fortran you mean. The Fortran
standards go to great lengths to allow a standard Fortran program
to be valid on machines of any byte size, any character set, and
any data size. And any particular implementation is allowed to
pack "characters" into other data types with great freedom.
(Historically, this freedom is a necessity, for example
to accomodate machines that may pack 6 6-bit characters or
5 7-bit characters into a 36-bit word.)
Tim.
Peter L. Montgomery
David R Tribble <dtri...@technologist.com> writes:
>Peter L. Montgomery wrote:
>> C9X is trying to promote integer types such as int32_t and int64_t
>> but the constants 32 and 64 are hardwired into the type names.
>> I want to start with one type (say long) and create a second
>> type with exactly twice as many bits. I also want the number
>> of bits in a type available as a compile time constant, for use
>> when computing shift counts. My needs are not being met.
>
>that describe the sizes of the built-in datatypes (which I've
>written about in another post on this thread), we could extend the
>language in such a way that the actual bit sizes (32, 64, whatever)
>lines of:
>
> int<2> i2; // sizeof(i2) == 2
> int<4> i4; // sizeof(i4) == 4
> int<8> i8; // sizeof(i8) == 8
>
>Since sizeof() is defined such that sizeof(char)==1, and the
>number of bits in a char is unspecified, we have a portable
>method for declaring integers of distinct sizes. In the example
>above, i4 is an integer that is the same size as char[4], but
>this does not necessarily mean it has 32 bits; it could just as
>well have 36 bits on a 32-bit machine (with 9-bit chars), 48
>bits on a 24-bit machine (with 12-bit chars), etc. (We may
>have to restrict N to be a member of {1,2,4,8} for all int<N>.)
>
in int<2> can be a compile-time constant expression.
Like James Kuyper (elsewhere in this discussion) I prefer
a count in bits to one in bytes, although a byte count is acceptable
if we can easily convert it to a bit count,
perhaps through multiplication by CHAR_BIT.
The allowed values in the <> will be implementation-dependent.
Implementation-defined macros will convert a required minimum number
of bits (or bytes) into a supported value. For example,
/*
Definitions for an implementation supporting
8-, 16-, 32-, 64-, 128-, 192-, 256-bit integers,
with 64 bits being fastest.
*/
#define INTSIZE_SHORTEST 8 /* Shortest size supported (usually char) */
#define INTSIZE_FASTEST 64 /* Hardware natural size */
#define INTSIZE_LONGEST 256 /* Longest size supported (intmax_t) */
#define INTSIZE_NEXT(n) ((n) <= 8 ? 8 : (n) <= 16 ? 16 : (n) <= 32 ? 32 \
: (n) <= 256 ? ((n) + 63)/64*64 : -1)
/* Round up to 8, 16, 32, 64, 128, 192, or 256 */
#define INTSIZE_NEXT_FAST(n) INTSIZE_NEXT(n)
/* Vendor considers all supported types to be `fast' */
/*
INTSIZE_FASTEST is intended to return the size of the
hardware registers.
This is the size recommended for subscripts, loop counters,
shift counts, and other scalars. This is also a suggested radix
for multiple-precision arithmetic.
[Should these sizes have different names, for architectures
such as UltraSPARC and CRAY with 64-bit registers
but no full 64 x 64 multiply?
Possible names are INTSIZE_REGISTER, INTSIZE_NATURAL, INTSIZE_RADIX.
INTSIZE_MAXIMUM and INTSIZE_MINIMUM are possible substitutes
for INTSIZE_LONGEST and INTSIZE_SHORTEST.]
A conforming implementation must support an integer type with
at least 2*INTSIZE_FASTEST bits. Equivalently,
INTSIZE_LONGEST >= 2*INTSIZE_FASTEST.
INTSIZE_FASTEST, INTSIZE_LONGEST, INTSIZE_SHORTEST
should not be used for binary data which may be
read by another program or by a later version of this program.
[Then, too, neither should short, int, long.]
INTSIZE_NEXT_FAST(b), where b <= INTSIZE_LONGEST, is supposed
to return the size of the smallest `fast' type with at least b bits.
The implementor decides what is `fast'. INTSIZE_NEXT(b) returns
the size of the smallest supported type with at least b bits,
fast or otherwise.
For example, if an implementation supports all sizes
1 bit, 2 bits, 3 bits, ..., up to 32 bits, then INTSIZE_NEXT(13) = 13.
If in addition the hardware has only 8-, 16-, 32-bit memory fetches,
with 32 being fastest (e.g., Pentium), the implementor can choose
whether to define INTSIZE_NEXT_FAST(13) = 16 or 32.
He should probably choose 16 -- then an application can use
INTSIZE_NEXT_FAST(13) = 16 to optimize for space or
MAX(INTSIZE_FASTEST, INTSIZE_NEXT_FAST(13)) = MAX(32, 16) = 32
to optimize for speed.
*/
------------------------
Given these macros and an int<> construct (needed only in typedefs,
number of bits must come from an INTSIZE_* macro), the header of a
SPEC benchmark for multiple-precision arithmetic could resemble
#include <some .h file>
#define BITS_PER_DIGIT INTSIZE_FASTEST
#define BITS_PER_DIGIT2 INTSIZE_NEXT_FAST(2*BITS_PER_DIGIT)
typedef unsigned int<BITS_PER_DIGIT> digit1_t; /* Digit in MP number */
/* Exactly BITS_PER_DIGIT bits */
typedef unsigned int<BITS_PER_DIGIT2> digit2_t; /* Two digits in MP number */
/* At least 2*BITS_PER_DIGIT bits */
Later the program might have
digit1_t x, y, zhi, zlo;
digit2_t z;
z = (digit2_t)x * (digit2_t)y; /* Full product x*y */
zhi = (digit1_t)(z >> BITS_PER_DIGIT); /* High half of product */
zlo = (digit1_t)z; /* Low half of product */
Any implementation which wants to do well on SPEC will optimize
these double-length operations (e.g., shifts by exactly INTSIZE_FASTEST bits,
as in zhi above), once some benchmarks use the operations.
SPEC lacks the operations in 1999 because there is no portable way
to express them, not because no important applications needs them.
The way out of this chicken-egg dilemma, which has existed since
K&R C, starts by defining the portable constructs:
1) Language standards must include the necessary constructs.
2) Conforming implementations supply the feature, perhaps
unoptimized.
3) Benchmarks using the features are written and incorporated
into SPEC. Vendors who support the feature well
rise among the ratings.
4) Marketing departments of vendors who have not
optimized the constructs put pressure on their compiler
groups to improve their implementation.
Architectural groups may pipeline the integer multiply
in new processors if it improves the SPEC ratings.
5) Users realize that they can use the features efficiently.
The use of the features grows.
Some algorithms once coded using floating point
arithmetic are recoded to use integer arithmetic.
The full sequence 1) - 5) may take 10-15 years. By then some processors
will have 128-bit ALUs (Arithmetic-Logic units).
The Subject: theme `64 bit advantages' arose when
Chris Morgan asked whether any SPEC benchmarks demonstrate
the advantages of Alpha and other 64-bit architectures.
Benchmarks which could so benefit need to adapt the lengths of
their data types to the machine architecture in order to demonstrate this.
Future 128-bit architectures can shine immediately once benchmarks
can be written to utilize each machine's potential.
Clive D.W. Feather
writes
>>Something along the
>>lines of:
>> int<2> i2; // sizeof(i2) == 2
> It is important to have a syntax such as this where the `2'
>in int<2> can be a compile-time constant expression.
There have been various attempts to define a complete proposal along
these lines. Without real-world experience of it, WG14 felt too
uncomfortable with all the nooks and crannies of the wording to include
it in C9X.
--
Clive D.W. Feather | Director of | Work: <cl...@demon.net>
Tel: +44 181 371 1138 | Software Development | Home: <cl...@davros.org>
Fax: +44 181 371 1037 | Demon Internet Ltd. | Web: <http://www.davros.org>
Written on my laptop; please observe the Reply-To address
Andy Glew
>these lines. Without real-world experience of it, WG14 felt too
>uncomfortable with all the nooks and crannies of the wording to include
>it in C9X.
Does anyone want to step up and define something like a config script
that will generate Peter's type declarations as macros?
One can imagine that such a script could take
// config has figured out that int is 32 bits, and is fastest
#define Int_Fastest int
#define Int_32 int
#define Int_31 int
...
// compiler has figured out that short is 16 bits, etc.
#define Int_16 short
#define Int_Largest long long
#define Int_64 long long
...
// some sort of microbenchmark along the lines of lmbench
// has figured out that (long)int * (long)int = long is fast,
// but that (long long)log * (long long) long = long long is disproportionately slow
#define Int_Largest_MxMto2M Int_32
#define Int_Largest_MxMto2M_mult(a,b) ((Int64)(a)*(Int64)(b))
#define Int_Largest_MxMto2M_mult_lo(a,b) (((Int64)(a)*(Int64)(b))&0x0FFFFFFFF)
#define Int_Largest_MxMto2M_mult_hi(a,b) (((Int64)(a)*(Int64)(b))>>32)
#defines or typedefs for "Sized" types smaller than long long could probably all be created this
way. It would be hard to automatically detect via a C program if Int_Largest * Int_Largest
gave a 2* Int_Largest operation, but *that* could probably be filled in by hand coders
with inline assembly.
If some proposed standard header file like this is automatically creatable, then
programs written using it will be portable *AND* adapatable to whatever the best
hardware support there is for them on a given machine. If we can promote
this as a standard benchmark, then there will finally be incentive for manufacturers
to provide better support for extended precision. Although I would hope that some
widely used application program would use this header first.
Greg Lindahl
> SPEC lacks the operations in 1999 because there is no portable way
> to express them, not because no important applications needs them.
Huh? "int" is INTSIZE_FASTEST. The others can be gotten via a
configure script. I don't believe that SPEC outlaws them, especially
since several existing spec benchmarks have them.
-- g
Terje Mathisen
[snipped a nice argument about the need for portable constructs allowing
wide operations]
> SPEC lacks the operations in 1999 because there is no portable way
> to express them, not because no important applications needs them.
> The way out of this chicken-egg dilemma, which has existed since
> K&R C, starts by defining the portable constructs:
>
> 1) Language standards must include the necessary constructs.
>
> 2) Conforming implementations supply the feature, perhaps
> unoptimized.
>
> 3) Benchmarks using the features are written and incorporated
> into SPEC. Vendors who support the feature well
> rise among the ratings.
>
> 4) Marketing departments of vendors who have not
> optimized the constructs put pressure on their compiler
> groups to improve their implementation.
> Architectural groups may pipeline the integer multiply
> in new processors if it improves the SPEC ratings.
>
> 5) Users realize that they can use the features efficiently.
> The use of the features grows.
> Some algorithms once coded using floating point
> arithmetic are recoded to use integer arithmetic.
>
> The full sequence 1) - 5) may take 10-15 years. By then some processors
> will have 128-bit ALUs (Arithmetic-Logic units).
Unless we do get something like this, the only other feasible way to get
fast and portable multi-precision arithmetic would be to bless a
particular mp library (preferably the Gnu MP), and include some new Spec
benchmarks that exercise the local implementation of this library.
Actually, I would like to see both of these paths taken, but it seems
like it might be too late to get there via standard C. :-(
Terje
--
- <Terje.M...@hda.hydro.com>
Using self-discipline, see http://www.eiffel.com/discipline
"almost all programming can be viewed as an exercise in caching"
Steven Correll
David R Tribble <dtri...@technologist.com> wrote:
> sizeof(int1_t) == 1 // e.g., signed char
> sizeof(int2_t) == 2 // e.g., signed short
> sizeof(int4_t) == 4 // e.g., signed int or signed long
> sizeof(int8_t) == 8 // e.g., signed long long
>
>declared in FORTRAN, e.g. INTEGER*4.)
The "INTEGER*4" syntax is a common extension, but despite (or because of?)
longstanding experience with it, the Fortran standards committee explicitly
chose not to adopt it. Instead, Fortran 90 provides a "kind=n" specification,
where "n" need not be related to the number of bytes; plus intrinsic functions
to aid a portable program in figuring out what number to use for "n" in order
to obtain the desired precision:
parameter (my_kind_of_int=selected_int_kind(6))
integer(kind=my_kind_of_int) i, j, k
parameter (my_kind_of_real=selected_real_kind(8,9))
real(kind=my_kind_of_real) a, b, c
These declarations use "the smallest available integer type capable of holding
numbers of magnitude 10**6", and "the smallest available real type capable of 8
decimal digits of precision and 10**9 range."
As I recall, some people had been burned by the need to change REAL*4 to REAL*8
in thousands of lines of code when porting from one machine to another
(because of differences in non-IEEE floating point formats, certain problems
ran fine in single precision on one machine but required double precision on
another) and these folks were determined to make it as easy as possible
to write a program that would port with no changes, and to discourage programs
that would require global changes. An F90 compiler must be capable of
diagnosing "INTEGER*4" as non-conforming, although any commercially viable
compiler has to support it.
--
Steven Correll == 1931 Palm Ave, San Mateo, CA 94403 == s...@netcom.com
Peter L. Montgomery
In article <7do3du$10...@news3.newsguy.com>
>> to express them, not because no important applications needs them.
>
>configure script. I don't believe that SPEC outlaws them, especially
>since several existing spec benchmarks have them.
According to p. 196 of K&R 2
`Plain int objects have the natural size suggested by
the host machine architecture; the other types sizes
are provided to meet special needs.'
This intention is not happening in practice.
This discussion began with the DEC Alpha.
Section A.1 (Hardware-Software Compact) of the 1992
Alpha Architecture Reference manual has a paragraph beginning
`Each computer architecture has a "natural word size."
For the PDP-11, it is 16 bits; for VAX, 32 bits;
and for Alpha, 64 bits.'
Section A.3.1 (Data Alignment -- Factor of 10) of this manual advises
`Frequently used scalars should reside in registers.
Each scalar datum allocated in memory should normally
be allocated an aligned quadword to itself,
even if the datum is only a byte wide. This allows
aligned quadword loads and stores, and avoids
partial-quadword writes (which may be half as fast
as full-quadword writes, due to such factors as
read-modify-write a quadword to do quadword ECC calculation).
Two authorities use the term `natural size' or `natural word size'.
If we believe both, then Plain int should be 64 bits on the Alpha.
Alas DEC's cc compiler uses 32 bits for int and 64 for long.
Microsoft's uses 32 bits for both int and long.
The intentions in K&R are not being met.
The Alpha architecture manual advises that 32-bit stores
may be slower than 64-bit stores, even though the architecture has
both stl (store longword) and stq (store quadword).
Sometimes we need more Alpha instructions to do a computation
in 32-bit mode than in 64-bit mode. Even if we know our data is
between 0 and 32767, we can benefit by using long rather than int.
Here are some examples, with the generated Alpha instructions shown:
*/
signed int shift_by1( signed int x, signed int y) { return x >> y;}
unsigned int shift_by2(unsigned int x, unsigned int y) { return x >> y;}
signed long shift_by3( signed long x, signed long y) { return x >> y;}
unsigned long shift_by4(unsigned long x, unsigned long y) { return x >> y;}
/*
Shift example: Conversion example:
Signed int sextl, sextl, sra lds, cvtlq, cvtqt
Unsigned int sextl, sextl, zapnot, srl, sextl
Signed long sra ldt, cvtqt
Unsigned long srl
*/
double tod1( signed int *x) { return *x;}
double tod3( signed long *x) { return *x;}
Robert Harley
pmon...@cwi.nl (Peter L. Montgomery) writes:
> Sometimes we need more Alpha instructions to do a computation
> in 32-bit mode than in 64-bit mode. Even if we know our data is
> between 0 and 32767, we can benefit by using long rather than int.
> Here are some examples, with the generated Alpha instructions shown:
> signed int shift_by1( signed int x, signed int y) { return x >> y;}
> unsigned int shift_by2(unsigned int x, unsigned int y) { return x >> y;}
> signed long shift_by3( signed long x, signed long y) { return x >> y;}
> unsigned long shift_by4(unsigned long x, unsigned long y) { return x >> y;}
> Signed int sextl, sextl, sra
> Unsigned int sextl, sextl, zapnot, srl, sextl
> Signed long sra
> Unsigned long srl
EGCS on Alpha Linux produces:
Signed int sra
Unsigned int zapnot, srl, sextl
Signed long sra
Unsigned long srl
for these.
The extra sign extensions are unnecessary since 32-bit integers are
known to be in a normalised format, namely sign-extended to 64 bits.
Bye,
Rob.
Larry Kilgallen
> According to p. 196 of K&R 2
>
> `Plain int objects have the natural size suggested by
> the host machine architecture; the other types sizes
> are provided to meet special needs.'
>
> This intention is not happening in practice.
> The intentions in K&R are not being met.
I am not a C programmer (read this post in comp.arch), but even I know
that K&R has been superceded by a formal standard.
Larry Kilgallen
Eric Hildum
bits at all. Since a character is 16 bits, the short is then 32, with the rest
handling the integer sizes up to 128 bits quite nicely.
Or are you still in the parochial US char is one byte mode? ;-)
Steven Correll wrote:
> In article <36FC31F8...@technologist.com>,
> David R Tribble <dtri...@technologist.com> wrote:
> > sizeof(int1_t) == 1 // e.g., signed char
> > sizeof(int2_t) == 2 // e.g., signed short
> > sizeof(int4_t) == 4 // e.g., signed int or signed long
> > sizeof(int8_t) == 8 // e.g., signed long long
> >
> >(If I'm not mistaken, this resembles the way integer types are
> >declared in FORTRAN, e.g. INTEGER*4.)
--
---------------------------
Eric Hildum
Eric....@Japan.NCR.COM
Ian Kemmish
>>> to express them, not because no important applications needs them.
>>
>>configure script. I don't believe that SPEC outlaws them, especially
>>since several existing spec benchmarks have them.
>
>
> `Plain int objects have the natural size suggested by
> the host machine architecture; the other types sizes
> are provided to meet special needs.'
>
Ckearly, speed of execution is regarded as a special need:-)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Ian Kemmish 18 Durham Close, Biggleswade, Beds SG18 8HZ, UK
i...@five-d.com Tel: +44 1767 601 361
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Behind every successful organisation stands one person who knows the secret
of how to keep the managers away from anything truly important.
Zalman Stern
: Or are you still in the parochial US char is one byte mode? ;-)
- 8-bit character sets cover a lot more ground than the US.
- Until very recently, the Japanese computer companies were putting up
rather serious resistance to Unicode. Probably for the same legacy support
reasons people elsewhere like 8-bit character sets.
- In C or C++, the type "char" is not the right type to use to represent
text characters in many applications, but it damn well better stay 8-bits
on an 8-bit byte machine. This is because the standard guarantees
sizeof(char) == 1 (and this has always been spec'ed as part of C). Changing
this would break most C programs and is hardly worth doing to clean up a
minor naming issue.
-Z-
Zalman Stern
: I am not a C programmer (read this post in comp.arch), but even I know
: that K&R has been superceded by a formal standard.
In the post, Peter is appealing to K&R as a statement of a top-level design
goal. This is legit as a standard will not tell you the purpose or deep
meaning of something, only its required syntax and semantics. But it is
useful to note that the "fastest int" goal of K&R is generally realizable
in fully conforming compilers.
Vendors have chosen to make backwards compatibility and convenient access
to all useful integer sizes more important goals than the "fastest int"
design goal. (If int were 64-bits on Alpha, what would the 32-bit type be
called?) The apparent solution is to introduce a more detailed realm of
type abstraction. (E.g. "intmax_t" explicitly states a purpose which "long"
does not.) Many people are legitimately worried that any type abstraction
will fall prey to the same backwards compatibility arguments.
(The worry is legitimate because many compiler vendors place an incredibly
large value on backward compatability in the compilation environment. Which
is perhaps a good thing except that its almost become a religious principle
in certain places and decisions about compatibility are not made by looking
at quantitative data. So one pretty much has to consider the evolutionary
path when defining a new feature into the standard. Although it is
basically impossible for the standard to mandate a given type is "fastest."
There is simply no good way to define fastest across the entire set of
programs. Compare this to "intmax_t" where the standard can actually
require sizeof(intmax_t) to be > than sizeof(t) for any t in a given set.)
-Z-
John R. Mashey
It has long been the case, but is especially true now, that the
natural "fastest" integer size may be (using current 64-bit CPUs as examples):
1) The "natural" integer register width [i.e., 64 bits]
2) Equally, any of the sizes [64, 32, 16, 8]
3) Equally, any of the smaller sizes [32, 16, 8]
4) unequally, one of the smaller sizes [likely 32]
And which is true depends on:
a) The specific Instruction Set Architecture.
b) The specific implementation of the ISA, includign datapath widths,
andgory details of cache hierarchy (like write-thru versus write-back caches,
and whether or not the nearest level of write-back cache has ECC or not,
and if so, is it ECC on 32, 64, or larger number of bits.
c) Whether the integer is signed or unsigned, interacting with a).
d) What kinds of operations are being done.
e) Cache effects relating from size of the data object.
f) The extent of global register allocation and tracking done by the compiler.
f) How the code is written...
For example, of 64-bit ISAs (but partially applicable to some 32-bit ISAs):
A1) Some ISAs (like MIPS) have a 64-bit integer LOAD instructions,
plus two each of 32-bit, 16-bit, and 8-bit loads, where you can choose
zero-extension or sign-extension. Sometimes all loads are the same speed,
sometimes the full-width load is fastest, and the rest are slower,
sometimes the full-width load and the zero-extends are fast, and the otehrs
cost and extra cycle.
A2) Some ISAs have 64- and 32-bit LOADs, and do 8s and 16s with extraction ops.
A3) Some ISAs have either LOAD-zero-extend, or LOAD-sign-extend for
8, 16, 32, but not both (so it matters whether or not it is signed or
unsigned), and the other one needs explicit sign-extend.
A4) Some ISAs may offer LOAD-8 or LOAD-16 that only effect the low-order
bits of the register, so they act more like inserts.
Hence, it can either cost the same to get data into a register, or have
widely-varying costs.
B1) When storing data from a register into memory, most ISAs have
STORE 98, 16, 32, 64).
B2) Some ISAs may offer STORE-64 and STORE-32 (earliest Alphas), but
omit STORE-8 and STORE-16, using LOAD-64, insert, STORE-64 sequences.
B3) Some CPUs use L1 caches that are write-back with byte-parity,
and all stores take the same time. If L1 is write-back with ECC,
ECC will be computed on some size, and any store less than that will
incur extra cycles for read-modify-write. CPUs with store-through
L1 and ECC'd L2 ca neitehr incur extra cycles, or not many if they used
buffered writebacks in the hope of aggregating writes.
B4) Within the same ISA, different cache designs will appear over time,
and cache line-sizes, bus-widths, etc, sometimes change.
C1) An ISA may offer operations on any of the sizes, and then maybe
sign/zero extend the result to the full register width, or they might not
do this extension.
C2) An ISA might offer direct support for 2 sizes only: in some 64-bit CPUs
(as in MIPS, for example), there are 32-bit adds (with and without overflow
detection), which, on the 64-bit implementations, produce the same low-order
32-bits as did the 32-bit implementations, and then sign-extend the result
to 64-bits. The 64-bit implementations also provide full 64-bit adds.
C3) An ISA might really have only one size of add, and everything else has to be
extended first.
C4) The time to do a 32-bit add or 64-bit add are usually the same on
current CPUs, but not necesarily so, and this especially didn';t use to be
true (where some CPU family members had narrower datapaths, narrower ALUs,
and had to do even 32-bit adds via multiple passes of 16-bit or even 8-bit
ALUs ... whereas other family members had full-width ALUs).
C5) I used ADD as an example, it being common. Usually, ADD and SUBTRACThave the same characteristics, but different operations may or may not be
provided in an orthogonal fashion, i.e., 64-bit MIPS have both 64- and 32-bit
ADDS, but only have 64-bit ORs and ANDs.
D1) For most implementations of most ISAs, integer multiplication is slower
than integer addition, and integer division, if provided at all, is slower yet.
64-bit integer multiplies are usually slower than 32-bit ones (but might not
be), and 64-bit integer divides are really slower than 32-bit ones,
even on CPUs that do have the divide hardware.
[This was a contributing reason to many people's wish not to do ILP64,
i.e., performance hit from inherent extra cycles from doubling the wdith.]
Of course, if you are doing multi-precision arithmetic, you are happy to
have hardware to help you and access to it.
E1) Compiler optimization matters. If there is extra overhead in fetching
smaller data, then that would argue for fastest being the full-register
width, but if a good optimizer fetches it to a register, expands it once,
and then re-uses it there, there's little difference, in whic hcase cache
effects may dominate.
E2) Cache effects matter ... a lot. We already have CPUs that can peak at 4
instructions per clock, and were headed higher. Within 2 years, we'll starting
seeing GHz chips (1ns cycle, 4-6 IPC peak), with for example, ~150ns
real access times to memory, meaning that one could execute ~1000 instructions
in the time to do one cache miss. Hence, on such machines, if you have big
arrays of integers, and using smallersizes lessens the cache miss rate,
at the costs of a few more instruction cycles, it may not be a bad idea
to go smaller. Of course, if the total data is small, and likely to be
cache-resident anyway, then the tradeoff is different.
E3) Finally, complex out-of-order CPUs (like MIPS R10K, PA-RISC, XEONS, etc)
make it very hard to make strong conclusions by looking at micro-level
cycle counts anyway.
SUMMARY: in the old UNIX/C days [i.e., when K&R was first written],
it was pretty easy to know that 16-bit ints were fast on PDP-11s,
and 32-bit ints fast on VAXen. On 68000/68010, even though the registers
were 32-bits wide, 16-bits was often faster, and some people stuck with
16-bit ints as a result. With most curretn 64-bit CPUs, soemtiems
64-bit is faster, but *usually* 32-bit gets enough (extra) support
compared to 16 that it is likely to be as fast as 64-bit, and better
than 16-bit ... but the bottom line of all of this, is that
"fastest" is a very slippery metric, and no one should ever expect
that any one size is uniformly faster, because it hardly ever is,
because it depends on all these factors listed above.
Dave Hansen
<Eric....@Japan.NCR.com> wrote:
>I was looking at the table below, and wondering why there is the question about 64
>bits at all. Since a character is 16 bits, the short is then 32, with the rest
>handling the integer sizes up to 128 bits quite nicely.
>
>Or are you still in the parochial US char is one byte mode? ;-)
Well actually, it's the C standard which requires "char is one byte,"
as in sizeof(char)==1. It does not, however, require that "byte"
means "eight bits." However, I don't know of any system offhand that
defines CHAR_BITS to be anything but 8. (Although I do know of a
couple where CHAR_BITS *couldn't* be 8, but I've never programmed
those with C).
I think you might be mixing up character data with the char type. The
char type is singularly unsuited for representing character data.
Regards,
-=Dave
Just my (10-010) cents
I can barely speak for myself, so I certainly can't speak for B-Tree.
Change is inevitable. Progress is not.
Dennis Ritchie
n2620 says "A 'plain' int has the natural size suggested
by the execution environment." I didn't look too hard
in the ungreppable current standard.
Dennis
James Kuyper
...
> on an 8-bit byte machine. This is because the standard guarantees
> sizeof(char) == 1 (and this has always been spec'ed as part of C). Changing
> this would break most C programs and is hardly worth doing to clean up a
True. However, CHAR_BIT is explicitly not required to be 8, and an
implementation which set it to 16 would not, IMO, break most code
(though it would break a lot of code).
Geoffrey KEATING
> Unfortunately, the whole idea of "fastest int" may well be out-of-date.
I might add to this that for many of any large program's data
structures, it is the size of the structure that has the most impact
on speed, simply because the structure is present, but not used, while
the real compute-intensive part of the program is being executed, and
the program uses significant memory/cache/disk/whatever capacity.
--
Geoff Keating <Geoff....@anu.edu.au>
Clive D.W. Feather
<zal...@netcom.com> writes
>Compare this to "intmax_t" where the standard can actually
>require sizeof(intmax_t) to be > than sizeof(t) for any t in a given set.)
Not quite. The standard makes no statement about sizes, merely about the
relative ranges of values.
Zalman Stern
: True. However, CHAR_BIT is explicitly not required to be 8, and an
: implementation which set it to 16 would not, IMO, break most code
: (though it would break a lot of code).
"Most code" is incredibly dependent on where you sit in the world. But
consider that if you set CHAR_BIT to 16 on a byte addressed machine, memcpy
no longer has 8-bit byte granularity. In fact, there would be no reasonable
way to talk about bytes anymore. All software in digital imaging and
digital video I worked on in the last six years would break badly. I expect
perl, X11 servers, and many device drivers would break. It would be quite
like the problems experienced with C on word addressed machines
(e.g. PDP-10), where char * pointers have a different representation than
other pointers, but it would be worse than that because you wouldn't be
able to address 8-bit bytes at all.
-Z-
Robert Harley
zal...@netcom.com (Zalman Stern) writes:
> James Kuyper (kuy...@wizard.net) wrote:
> : True. However, CHAR_BIT is explicitly not required to be 8, and an
> : implementation which set it to 16 would not, IMO, break most code
> : (though it would break a lot of code).
>
> digital video I worked on in the last six years would break badly. I expect
Zalman, I think that there's a language problem here.
One group of people use the word "C" to mean an abstract version of
the language in which every program should work on every imaginable
implementation as defined by the 7000-page draft-standard-du-jour.
Other people, including you and I, use "C" to mean Real World C, which
compiles with implementations that have the distinct advantage of
actually existing, and which runs correctly on our target hardware.
The other crowd will probably drop into this thread for a bout of
flaming, as they usually do when anyone gores their sacred cows, but
in fact their statements about C and ours are not contradictory since
we're talking about two different languages altogether.
In Real World C, chars have 8 bits. They just do. All integral
arithmetic is done in twos-complement binary. It just is. Several
simple "get real" assumptions like these make our work possible.
The advantage of our point of view is that while we're sending out
finished applications written in Real World C, the other crowd is
still on line three, poring over some expression and trying to figure
out if it would give the right answer on an imaginary implementation
that switched between nines-complement BCD and signed-magnitude
ternary every Tuesday, except when the moon is full.
Bye,
Rob.
Nick Maclaren
Robert Harley <har...@corton.inria.fr> wrote:
>
>Other people, including you and I, use "C" to mean Real World C, which
>compiles with implementations that have the distinct advantage of
>actually existing, and which runs correctly on our target hardware.
Since you are writing for a single machine, why are you interested
in a standard language in the first place? That strikes me as a
bizarre restriction.
>In Real World C, chars have 8 bits. They just do. All integral
>arithmetic is done in twos-complement binary. It just is. Several
>simple "get real" assumptions like these make our work possible.
Ah. A newcomer to this game. Yes, that is more-or-less true, but
it wasn't true even 20 years ago, and may not be in 10 years. My
programs have rather longer lifetimes than that.
>The advantage of our point of view is that while we're sending out
>finished applications written in Real World C, the other crowd is
>still on line three, poring over some expression and trying to figure
>out if it would give the right answer on an imaginary implementation
>that switched between nines-complement BCD and signed-magnitude
>ternary every Tuesday, except when the moon is full.
A slight exaggeration. However, when our 'customers' change their
system, my code is compiled, retested and working while you are still
arranging meetings to discuss the manpower requirements for the
migration.
Yes, you are quite right. We live in different worlds :-)
Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QG, England.
Email: nm...@cam.ac.uk
Tel.: +44 1223 334761 Fax: +44 1223 334679
Dean F. Sutherland
>
> Zalman, I think that there's a language problem here.
>
> One group of people use the word "C" to mean an abstract version of
> the language in which every program should work on every imaginable
> implementation as defined by the 7000-page draft-standard-du-jour.
>
> Other people, including you and I, use "C" to mean Real World C, which
> compiles with implementations that have the distinct advantage of
> actually existing, and which runs correctly on our target hardware.
>
<snip>
> In Real World C, chars have 8 bits. They just do. All integral
> arithmetic is done in twos-complement binary. It just is. Several
> simple "get real" assumptions like these make our work possible.
I've worked in the recent past on Real World compilers for Real World
machines with various combinations of the following features that break
your "get real" assumptions:
16-bit wide word addressed memory
32-bit wide word addressed memory
24-bit wide word addressed memory
48-bit wide word addressed memory
(chars on these have number-of-bits matching word size)
sizeof(char) == sizeof(short) == sizeof(int) == sizeof(long)
<= sizeof void*
distinct address spaces for code and data
(eg. it is possible to have &function == &variable, where
the function and variable are NOT in the same location)
more than one data address space
Do we spread struct elements across both address spaces? If so, we can
end up with two fields at the same bit offset from the beginning of the
struct. If not, we lose 1/2 our memory bandwidth. And what about
arrays? Put successive elements in alternate data spaces??? This wins
for fetch bandwidth, but not for iteration. And since this is a
word-addressed machine, some variants of these choices could lead to
anything from 1 char per address (wasting 56bits/address) to 8 chars per
address (dense use of memory, but indexing will be a real mess).
> The advantage of our point of view is that while we're sending out
> finished applications written in Real World C, the other crowd is
> still on line three, poring over some expression and trying to figure
> out if it would give the right answer on an imaginary implementation
> that switched between nines-complement BCD and signed-magnitude
> ternary every Tuesday, except when the moon is full.
>
> Bye,
> Rob.
Right. Sure. The quite real-world architectures that I'm talking about
above are manufactured in large volumes. They get designed into LOTS of
products. You probably own several of them without even knowing it.
Have a cell-phone? Hard disk? Anti-lock brakes?
These "language lawyer" issues really do matter in the "Real World!"
Dean F. Sutherland
James Kuyper
>
> zal...@netcom.com (Zalman Stern) writes:
> > James Kuyper (kuy...@wizard.net) wrote:
> > : True. However, CHAR_BIT is explicitly not required to be 8, and an
> > : implementation which set it to 16 would not, IMO, break most code
> > : (though it would break a lot of code).
> >
> > [...] All software in digital imaging and
> > digital video I worked on in the last six years would break badly. I expect
> > perl, X11 servers, and many device drivers would break. [...]
I don't doubt it. Hardware level code is necessarily sensitive to such
things. However, you're also unlikely to find CHAR_BIT set to 16 on
machines where there are significant 8-bit operations, and nobody should
expect hardware-oriented code to be portable across such boundaries. If
all that an implementor wants to do is support 16 bit characters, as
opposed to 16 bit char's, that's what wchar_t is for. Most higher-level
code can be made pretty much insensitive to changes in CHAR_BIT by the
suitable use of sizeof() and <limits.h>.
> Zalman, I think that there's a language problem here.
>
> One group of people use the word "C" to mean an abstract version of
> the language in which every program should work on every imaginable
> implementation as defined by the 7000-page draft-standard-du-jour.
It was a little over 500 pages, the last time I checked.
> Other people, including you and I, use "C" to mean Real World C, which
> compiles with implementations that have the distinct advantage of
> actually existing, and which runs correctly on our target hardware.
>
> The other crowd will probably drop into this thread for a bout of
> flaming, as they usually do when anyone gores their sacred cows, but
> in fact their statements about C and ours are not contradictory since
> we're talking about two different languages altogether.
But if you don't care about standard C, why are you monitoring
comp.std.c? Standard C is an abstraction; if you don't care about such
abstractions, you should stick to newsgroups devoted to the particular
platforms that match the particular assumptions you like to make. Learn
all you can about the special extensions and limitations to C that apply
only on those platforms, and write your code accordingly - since you
don't care about other platforms, you don't need to worry about the
portability of your code.
> In Real World C, chars have 8 bits. They just do. All integral
> arithmetic is done in twos-complement binary. It just is. Several
> simple "get real" assumptions like these make our work possible.
In Real World C, chars also have 16 bits, and probably have a
non-power-of-2 size on 36 bit machines (I don't know the details). In
Real World C, arithmetic is also done in ones-complement and
sign-magnitude binary. Not very often, but it does happen. The fact that
this isn't true on your machine doesn't mean that either the standard or
these discussions should ignore those platforms, though it's certainly
relevant to debate whether standard C should continue allowing such
implmentations.
> The advantage of our point of view is that while we're sending out
> finished applications written in Real World C, the other crowd is
> still on line three, poring over some expression and trying to figure
> out if it would give the right answer on an imaginary implementation
> that switched between nines-complement BCD and signed-magnitude
> ternary every Tuesday, except when the moon is full.
It takes a little longer to write portable code the first time, but not
more than 10-20% longer once you get used to it, and you save in the
long run on maintenance time.
Dave Hansen
wrote:
FWIW, the January 18th draft says exactly the same thing.
Regards,
-=Dave
Just my (10-010) cents
I can barely speak for myself, so I certainly can't speak for B-Tree.
Change is inevitable. Except from a vending machine.
Robert Herndon
> zal...@netcom.com (Zalman Stern) writes:
>> James Kuyper (kuy...@wizard.net) wrote:
>> : True. However, CHAR_BIT is explicitly not required to be 8, and an
>> : implementation which set it to 16 would not, IMO, break most code
>> : (though it would break a lot of code).
>>
>> [...] All software in digital imaging and
>> digital video I worked on in the last six years would break badly. I expect
>> perl, X11 servers, and many device drivers would break. [...]
> Zalman, I think that there's a language problem here.
> One group of people use the word "C" to mean an abstract version of
> the language in which every program should work on every imaginable
> implementation as defined by the 7000-page draft-standard-du-jour.
....
> In Real World C, chars have 8 bits. They just do. All integral
> arithmetic is done in twos-complement binary. It just is. Several
> simple "get real" assumptions like these make our work possible.
....
> The advantage of our point of view is that while we're sending out
> finished applications written in Real World C, the other crowd is
> still on line three, poring over some expression and trying to figure
> out if it would give the right answer on an imaginary implementation
> that switched between nines-complement BCD and signed-magnitude
> ternary every Tuesday, except when the moon is full.
Some things don't change.
Young whippersnappers still think they know everything. When you
were yet in liquid form, machines with 9- (PDP-10) and 10-bit bytes
(C machine) were not rare. Machines with different pointer formats
for bytes and words existed (DGs). One's complement machines weren't
rare either (CDC 6600), tho' I don't recall ever writing or porting
a C program on one.
We called your attitude "VAXocentrism", and there were few things
more painful than having to port a piece of code written by a
dedicated VAXocentrist.
A little care in coding (i.e., writing what you really mean), along
with a little suffering (e.g., having to port one's own code to a new
machine, two years after it was written) was occasionally sufficient
to repair this attitude. Once one has been through the process a
few times, and bent ones thinking appropriately to deal with it,
coding is not particularly slower. It can save an awful lot of
time, pain, and agony later. And one's programs, most often, become
neater and easier to read.
A pundit once quipped, "many of the people who now suffer from
VAXocentrism have never even seen a VAX".
Kids still can't write. Some of them just do it in C now.
The unfortunate thing is that I don't know if there's a smiley to
go along with this.
----
To turn this back to architecture, I'll suggest that it's perhaps
this kind of thinking that is responsible for Sun's performance
problems with the SPARC architecture.
Zalman Stern
: I've worked in the recent past on Real World compilers for Real World
: machines with various combinations of the following features that break
: your "get real" assumptions:
: 16-bit wide word addressed memory
: 32-bit wide word addressed memory
: 24-bit wide word addressed memory
: 48-bit wide word addressed memory
: (chars on these have number-of-bits matching word size)
Setting CHAR_BIT to the minimum size of the hardware is an ok thing to
do. Setting it to twice the minimum size generally sucks. (Also, I expect
noone in their right mind tries to port large quantities of software
written for desktop/server environments to the machines Dean is talking
about, but it has to do with lots of things, not just the CHAR_BIT issue.)
I didn't mean to say you couldn't legally build a C implementation where
CHAR_BIT != 8, just that changing CHAR_BIT from 8 to 16 in an existing
implementation because you felt "char" should hold tect characters and they
need to be 16-bits would be an incredibly expensive thing to do in with
very little benefit.
Or put another way: Learn to love wchar_t and bail on the "CHAR_BIT == 8 is
a small-alphabet conspiracy" BS.
-Z-
Will Rose
: Zalman Stern wrote:
: ...
: > on an 8-bit byte machine. This is because the standard guarantees
: > sizeof(char) == 1 (and this has always been spec'ed as part of C). Changing
: > this would break most C programs and is hardly worth doing to clean up a
: True. However, CHAR_BIT is explicitly not required to be 8, and an
: implementation which set it to 16 would not, IMO, break most code
: (though it would break a lot of code).
With increasing use of DSPs programmed in higher level languages, I'd
expect to see more of #define CHAR_BIT 32. That's the only place I've
seen it hitherto, and the code isn't really general purpose.
Andy Glew
> Learn to love wchar_t
While we're on the subject: wchar_t is ugly.
I suspect that it would have been picked up more
quickly if human friendly names such as
unicode_char
were standard.
Sure, you can roll your own: you say
typedef wchar_t unicode_char
and I say
#define char_unicode_t wchar_t
Oh! Let's call the whole thing off!
Zalman Stern
: Some things don't change.
The software and hardware landscape has changed significantly in the last
20+ years.
: Young whippersnappers still think they know everything. When you
: were yet in liquid form, machines with 9- (PDP-10) and 10-bit bytes
: (C machine) were not rare.
I have written C code on a PDP-10. As much as I loved Twenex, I have no
desire to go back.
: Machines with different pointer formats
: for bytes and words existed (DGs). One's complement machines weren't
: rare either (CDC 6600), tho' I don't recall ever writing or porting
: a C program on one.
: We called your attitude "VAXocentrism", and there were few things
: more painful than having to port a piece of code written by a
: dedicated VAXocentrist.
Basically all that hardware diversity started its demise with the 360, a
byte-addressed 32-bit twos complement machine ~15 years before the first
VAX. Its been a slow process, but we are achieving consensus that this is a
decent way to design computer architectures. We even see evidence that
deviations are unmarketable. E.g. Alpha adding full support for 8 and 16
bit data items after initially omitting load/store instructions for these
sizes. (Also done for the AMD 29k, etc.)
None of which has much to do with my original point which is that the
number of bits needed to represent text characters is not necessarily the
same number of bits you want in the smallest C type (which must be called
"char").
-Z-
David Adrien Tanguay
> In Real World C, chars also have 16 bits, and probably have a
> non-power-of-2 size on 36 bit machines (I don't know the details).
CHAR_BIT == 9 on Bull mainframes, for which we provide the C compiler.
I'm always amazed at how big a deal people make of the CHAR_BIT != 8
thing, when in practice you almost never notice it. Even code written
by people who just know that CHAR_BIT == 8 rarely has any porting
problems. The assumption that pointers are really just integers causes
more trouble than CHAR_BIT.
--
David Tanguay d...@Thinkage.on.ca http://www.thinkage.on.ca/~dat/
Thinkage, Ltd. Kitchener, Ontario, Canada [43.24N 80.29W]
Zalman Stern
: While we're on the subject: wchar_t is ugly.
Yeah, I woke up with one on the pillow next to me last week and boy you
should have heard me screaming. Served as a good reminder that I should
always use typedefs for protection.
I expect the whole C language is ugly to people who's native language uses
a different alphabet.
-Z-
Bill Godfrey
> Vendors have chosen to make backwards compatibility and convenient access
> to all useful integer sizes more important goals than the "fastest int"
> design goal. (If int were 64-bits on Alpha, what would the 32-bit type be
> called?)
short?
Bill, feeling that he's missed the point.
David R Tribble
>> Vendors have chosen to make backwards compatibility and convenient
>> access to all useful integer sizes more important goals than the
>> "fastest int" design goal. (If int were 64-bits on Alpha, what would
>> the 32-bit type be called?)
Bill Godfrey wrote:
> short?
>
> Bill, feeling that he's missed the point.
When we ported our 500,000-line C project to DEC/Alpha Unix (circa
1993), we had surprisingly few problems. DEC decided to keep
ints 32 bits wide and only changed longs and pointers to 64 bits.
This meant that certain types like time_t didn't change in size.
Which meant that most of our data structures (written to and read
from files, or sent over sockets) did not change much, either.
The architecture was also done quite well, in that all integer
values were converted to 64 bits when pushed onto the stack (i.e.,
when passed as args to functions). So code such as the following,
which worked on 32-bit systems, continued to work on the 64-bit
system (even though such code is incorrect):
int x;
long y;
printf("x=%ld, y=%d", x, y);
printf("&x=%lx, %y=%x", &x, &y);
The net result was that the only code we had to fix were
questionable conversions between integers and pointers (which are
so easy to get wrong anyway), and 'int' alignment being assumed in
places where 'long' alignment was actually needed.
We ended up with a few useful assumptions about datatypes:
1. You can't assume sizeof(int)==sizeof(void*), but you are
pretty safe assuming sizeof(void*)==sizeof(ulong). Which means
that if you really must convert pointers to integers and back,
convert them to 'ulong' only.
2. It's pretty safe to assume that 'int' is 32 bits. This is
almost universal on Unix machines. (We weren't interested in
other platforms at the time.)
3. It's pretty safe to assume that 'short' is 16 bits. This is
even more universal.
(I'm not saying everyone can make these assumptions, just that we
felt justified in doing so for our porting needs.)
In retrospect, I would judge that DEC did a good job with both
the Alpha CPU and with Alpha/Unix.
FWIW, I consider it the "best" choice that 'short' is 16 bits wide,
'int' is 32 bits, and 'long' is 64 bits. But I realize this
choice doesn't suit everyone.
-- David R. Tribble, dtri...@technologist.com --
Steven Correll
Andy Glew <gl...@cs.wisc.edu> wrote:
>
>> Learn to love wchar_t
>
>
>While we're on the subject: wchar_t is ugly.
>
>quickly if human friendly names such as
>
> unicode_char
>
>were standard.
This leads to a real comp.arch question (bear with me for a bit).
The Unicode and ISO10646 standards actually encompass various
encodings including 2-octet-wide UTF-16 (which most people think of when they
hear the word "Unicode") and variable-octets-per-character UTF-8
(invented by the Plan 9 folks and used by Java). IMHO UTF-8 is far more
sensible than UTF-16: the average storage for non-Asian data is close to
1 octet per character, the average storage for Asian data not too far
above 2 octets per character (taking into account that even Asian data
has lots of blanks and digits 0-9, which require only one octet apiece),
and UTF-8 requires only that programs and filesystems be 8-bit clean, not
that they be rewritten carefully to avoid treating '\0' as a terminator or
'\n' as a newline. The main downside: an operation which needs
to know the real number of characters can't just divide the number of
octets by 2; and converting between UTF-16 and UTF-8 is expensive.
Here's the comp.arch puzzler: is there any hardware support for UTF-8
that would be reasonable in a RISC context?
(The tables below use 8-character-wide tabs.)
The rule for delineating characters without decoding them is (my own code for
this shifts and masks the nibble, then indexes into a table):
High-order nibble Number of
of first octet octets in character
0000..0111 1
1000..1011 (this is an internal octet, not the first)
1100..1101 2
1110..1111 3
The rules for mapping UTF-16 to UTF-8 are:
UTF-16 value "c"
0..127 128..2^11-1 2^11-1..2^16-1
UTF-8
octet#1 c ((c>>6)&0x1f)|0xc0 ((c>>12)&0xf)|0xe0
octet#2 (none) (c&0x3f)|0x80 ((c>>6)&0x3f)|0x80
octet#3 (none) (none) (c&0x3f)|0x80
Or maybe it's clearer (as in plan9.bell-labs.com/magic/man2html/6/utf):
01. x in [00000000.0bbbbbbb] -> 0bbbbbbb
10. x in [00000bbb.bbbbbbbb] -> 110bbbbb, 10bbbbbb
11. x in [bbbbbbbb.bbbbbbbb] -> 1110bbbb, 10bbbbbb, 10bbbbbb
My own code for mapping UTF-16 to UTF-8 does a "find first bit" using a
shift and mask and lookup table, and then depending on the bitnumber,
executes the code in the appropriate column above.
My own code for mapping UTF-8 to UTF-16 first finds the number of octets, then
executes one of three cases:
c = s[0];
c = ((s[0]&0x1f)<<6)|(s[1]&0x3f)
c = ((s[0]&0xf)<<12)|((s[1]&0x3f)<<6)|(s[2]&0x3f)
--
Steven Correll == 1931 Palm Ave, San Mateo, CA 94403 == s...@netcom.com
Éamonn McManus
> - In C or C++, the type "char" is not the right type to use to represent
> text characters in many applications, but it damn well better stay 8-bits
> sizeof(char) == 1 (and this has always been spec'ed as part of C).
As a minor historical note, this has not always been spec'd as part of
C. K&R 1 says:
The sizeof operator yields the size, in bytes, of its operand. (A
"byte" is undefined by the language except in terms of the value
of sizeof. However, in all existing implementations a byte is the
space required to hold a char.)
ANSI C turned the observation about "all existing implementations"
into a requirement.
,
Eamonn http://www.gr.opengroup.org/~emcmanus
"Make a remark," said the Red Queen: "it's ridiculous to leave all the
conversation to the pudding!"
Andy Glew
> for non-Asian data is close to
>1 octet per character, the average storage for Asian data not too far
>above 2 octets per character (taking into account that even Asian data
>has lots of blanks and digits 0-9, which require only one octet apiece),
>and UTF-8 requires only that programs and filesystems be 8-bit clean, not
>that they be rewritten carefully to avoid treating '\0' as a terminator or
>'\n' as a newline. The main downside: an operation which needs
>to know the real number of characters can't just divide the number of
>octets by 2; and converting between UTF-16 and UTF-8 is expensive.
My take: use UTF-16.
And use a separate compression layer to save space.
Which will probably save more space than the manual compression
of UTF-8.
Norman Diamond
>In Real World C, chars have 8 bits. They just do.
On the second Real World Machine on which Real World C was implemented,
chars had 9 bits. But it wasn't a matter of they just did. They could
have had 6 bits. Take a look at your K&R-1 again. That was Real World C,
when no one even thought of hijacking it yet.
>All integral arithmetic is done in twos-complement binary. It just is.
This tends to be more accurate, though not completely accurate. A few
months ago someone from Unisys posted here. Unisys hasn't disappeared
yet. C89 makes life very very difficult for them, but they coped.
As you said,
>The other crowd will probably drop into this thread for a bout of flaming,
Sorry that truth is a flame in your world.
--
<< If this were the company's opinion, I would not be allowed to post it. >>
"I paid money for this car, I pay taxes for vehicle registration and a driver's
license, so I can drive in any lane I want, and no innocent victim gets to call
the cops just 'cause the lane's not goin' the same direction as me" - J Spammer
Norman Diamond
>>Learn to love wchar_t
>
>While we're on the subject: wchar_t is ugly.
True. Unfortunately there were still lots of operations around that
had to be done on 8 or 9 bits, and EBCDIC files and even ASCII files
that were imported and had to be dealt with, etc., so char couldn't
be made 16 bits.
>I suspect that it would have been picked up more
>quickly if human friendly names such as
> unicode_char
>were standard.
Wouldn't have made any difference. wchar_t was invented because it
was needed. If Unicode were around at the time, I'm sure everyone
would have been glad to adopt Unicode immediately instead of finding
national solutions.
I'm not an expert but guess that wchar_t was invented and used around
20 years ago. When did Unicode get under way?
By the way, if Unicode were around 40 years ago, then everyone would
be happy to use Unicode instead of EBCDIC and ASCII too, right?
>Sure, you can roll your own: you say
> typedef wchar_t unicode_char
Doesn't work. The sizes are the same but the encoding is still
native wchar_t if that's what you read from the file and is still
unicode_char if that's what you read from the file.
Dik T. Winter
> My take: use UTF-16.
UTF-16 is no more sensible than UTF-8. It encodes an 10646 symbol in
either 2 or 4 bytes and is not able to encode all 10646 symbols. Use
UCS-2 which is the 2 byte Basic Multilingual Plane of 10646, aka Unicode.
--
dik t. winter, cwi, kruislaan 413, 1098 sj amsterdam, nederland, +31205924131
home: bovenover 215, 1025 jn amsterdam, nederland; http://www.cwi.nl/~dik/
Jeff Kenton
>In article <7e2ht6$ddo$1...@news.doit.wisc.edu> "Andy Glew" <gl...@cs.wisc.edu> writes:
> > My take: use UTF-16.
>UTF-16 is no more sensible than UTF-8. It encodes an 10646 symbol in
>either 2 or 4 bytes and is not able to encode all 10646 symbols. Use
>UCS-2 which is the 2 byte Basic Multilingual Plane of 10646, aka Unicode.
>--
These are both answers that ignore specifics of what a particular
application is trying to do.
The answer may turn out to be the usual, unsatisfying, "it depends".
--
-------------------------------------------------------------------------
= Jeff Kenton jke...@world.std.com =
-------------------------------------------------------------------------
Dennis Yelle
[...]
> When we ported our 500,000-line C project to DEC/Alpha Unix (circa
> 1993), we had surprisingly few problems. DEC decided to keep
> ints 32 bits wide and only changed longs and pointers to 64 bits.
> This meant that certain types like time_t didn't change in size.
> Which meant that most of our data structures (written to and read
> from files, or sent over sockets) did not change much, either.
>
> The architecture was also done quite well, in that all integer
> values were converted to 64 bits when pushed onto the stack (i.e.,
> when passed as args to functions). So code such as the following,
> which worked on 32-bit systems, continued to work on the 64-bit
> system (even though such code is incorrect):
> int x;
> long y;
> printf("x=%ld, y=%d", x, y);
> printf("&x=%lx, %y=%x", &x, &y);
Is it legal to promote int to long when
it is passed through ... ?
Is it legal to promote (unsigned) int to (unsigned) long long when
it is passed through ... ?
If so, does this solve the problem of
not being able to print a size_t etc. ?
Dennis Yelle
David Harmon
<d...@thinkage.on.ca> wrote:
>CHAR_BIT == 9 on Bull mainframes, for which we provide the C compiler.
Just curious, what is the bit pattern of NULL ?
Robert Herndon
> Robert Herndon (rm...@rmi.net) wrote:
>> Some things don't change.
> The software and hardware landscape has changed significantly in the last
> 20+ years.
What hasn't changed is that most people still can still find
ways to program badly. "There is not now, and will never be,
a language in which it is the least bit difficult to program
badly" was first uttered more than 20 years ago. I think it was
Robert Harley who was advocating 'quick and dirty' use of various
assumptions about wordsize and data formats to save programming
time. I think that in many cases, it's a very false economy.
>> Young whippersnappers still think they know everything. When you
>> were yet in liquid form, machines with 9- (PDP-10) and 10-bit bytes
>> (C machine) were not rare.
> I have written C code on a PDP-10. As much as I loved Twenex, I have no
> desire to go back.
Ditto. 18-bit address spaces (or even worse, the PDP-11's 16-bit
address space) is just too painful to live with any more for many
things.
>> Machines with different pointer formats
>> for bytes and words existed (DGs). One's complement machines weren't
>> rare either (CDC 6600), tho' I don't recall ever writing or porting
>> a C program on one.
This one (pointer formats) still causes me the most grief. Crays
were not byte addressible, although C synthesized 8-bit byte addressibility
for C. Int pointers fit in 32 bits, char pointers took 64. This broke
more code...
>> We called your attitude "VAXocentrism", and there were few things
>> more painful than having to port a piece of code written by a
>> dedicated VAXocentrist.
> Basically all that hardware diversity started its demise with the 360, a
> byte-addressed 32-bit twos complement machine ~15 years before the first
> VAX. Its been a slow process, but we are achieving consensus that this is a
> decent way to design computer architectures. We even see evidence that
> deviations are unmarketable. E.g. Alpha adding full support for 8 and 16
> bit data items after initially omitting load/store instructions for these
> sizes. (Also done for the AMD 29k, etc.)
I'm not sure I'd state it so baldly. One could make a much better
case that RISCification of instruction sets started with the CDC 6600.
The PDP-11 certainly had a large effect in here as well, but I'm
inclined to think it was a lot of things. Certainly the 360 had a
lot of exceptions to "modern" architectures. Then again, so does a
Pentium. One could certainly also argue that the rise of C and Unix
have had a great effect on the demise of hardware diversity, and
probably a larger one than the 360.
> None of which has much to do with my original point which is that the
> number of bits needed to represent text characters is not necessarily the
> same number of bits you want in the smallest C type (which must be called
> "char").
This is true enough.
Zalman Stern
: short?
Then what do you call the 16-bit type? To forestall the obvious iteration,
the idea is that on Alpha, a 64-bit byte addressed machine targeted at
general purpose UNIX/NT style computing, it is very useful to have first
class access to 8, 16, 32, and 64-bit integer values using pointers. The
8-bit type must be called "char" because it is the smallest and
sizeof(char) must equal 1. (And to forestall yet another possible iteration
back to square one, yes you could set CHAR_BIT == 16, but then you wouldn't
have first class access to an 8-bit type.)
-Z-
Zalman Stern
: Then what do you call the 16-bit type?
James Kuyper sent email saying one can use wchar_t for this. Which seems
correct to me. So I'll rescind the comment. Though it is a little troubling
to force wchar_t into that role as one might want to make wchar_t be
32-bits to support the full UCS-4 encoding. I think it is a good idea to
provide all reasonable access sizes distinctly from wchar_t and then make
wchar_t be the "right" size for text characters in the environment in
question.
-Z-
Douglas A. Gwyn
> True. Unfortunately there were still lots of operations around that
> had to be done on 8 or 9 bits, and EBCDIC files and even ASCII files
> that were imported and had to be dealt with, etc., so char couldn't
> be made 16 bits.
Well, yes, it could have. Or more precisely, the implementation
could have been allowed to make "char" big enough to hold any
character code it needed to in that environment (8, 16, and 32
bits all come to mind). But one would have to provide a separate
"byte" type to cater to the possibly smaller granularities of
such things as memory addressing boundaries/allocation units/
punched paper tape/whatever. There was in fact a proposal to do
just that (my name for "byte" was "short char" to avoid using a
new keyword). One important feature was that text-oriented I/O
streams automatically converted back and forth to "char" while
binary-oriented streams worked in bytes. I18n would have been
a lot simpler with such a scheme.
Douglas A. Gwyn
> > sizeof(char) == 1
> ANSI C turned the observation about "all existing implementations"
> into a requirement.
Dennis insisted that it was an intended property.
That pretty much doomed my attempt to prevent such a requirement.
Piercarlo Grandi
[ ... ]
nmm1> In article <rz77lrw...@corton.inria.fr>,
nmm1> Robert Harley <har...@corton.inria.fr> wrote:
[ ... ]
>> In Real World C, chars have 8 bits. They just do. All integral
>> arithmetic is done in twos-complement binary. It just is. Several
>> simple "get real" assumptions like these make our work possible.
nmm1> Ah. A newcomer to this game. Yes, that is more-or-less true, but
nmm1> it wasn't true even 20 years ago, and may not be in 10 years. My
nmm1> programs have rather longer lifetimes than that.
[ ... ]
Ahhh, and here we are again. Nick, our good Robert is not quite a
newcomer to this game, but a vicar of the ancient "All the world's a
VAX" heresy.
I'll mention here the immortal words of Henry Spencer to support yours,
from the:
<URL:http://www.compsoc.man.ac.uk/~isoma/spare/humour/C_ten_commandments.html>
Ten Commandments for C programming:
=======================================================================
10.Thou shalt foreswear, renounce, and abjure the vile heresy which
claimeth that All the world's a VAX, and have no commerce with the
benighted heathens who cling to this barbarous belief, that the
days of thy program may be long even though the days of thy
current machine be short.
by Henry Spencer
=======================================================================
Andy Glew
of how to define machine-specific types perfectly matched to
the hardware.
As I remember it, Algol-68 had rather generic "int", and "bool",
and I think it also had "char", defined similar to C's (although there were
probaly size and range specifications).
But the machine specific example started out by defining a "bit"
- not a "bool" - then an array of bit as "word", and eventually defined
equivalences between these machine specific data types and
the language defined data types, something like
type [0:31]bit = word;
type word = unsigned int<0,2**32-1>
etc.
(I am sure that I do not have Algol-68 syntax here.)
Always struck me as an elegant linkage. Especially if used
with something like Java's reflection API.
James Kuyper
>
> Zalman Stern (zal...@netcom.com) wrote:
> : Then what do you call the 16-bit type?
>
> James Kuyper sent email saying one can use wchar_t for this. Which seems
Sorry, I must have selected the wrong button; that had been meant as a
newsgroup reply.
Zalman Stern
: But the machine specific example started out by defining a "bit"
: - not a "bool" - then an array of bit as "word", and eventually defined
: equivalences between these machine specific data types and
: the language defined data types, something like
: type [0:31]bit = word;
: type word = unsigned int<0,2**32-1>
More recently, Modula-3 has a construct like "BITS 16 FOR TYPE INTEGER"
where "BITS ... FOR" is a type constructor based on another type ("INTEGER"
above). I probably have the the syntax incorrect as I can't seem to find
any of the numerous M3 language references I used to have, but the Modula 3
Report is available on the Web. (Yeah, I'm too lazy to dig up the URL.)
-Z-
Douglas A. Gwyn
> type [0:31]bit = word;
> type word = unsigned int<0,2**32-1>
There is more to a type, or its representation, than how many
subcomponents it has. A complete solution has to support
specification of such things as endianness, signedness model,
etc. WG14 has debated proposals along these lines, but decided
it would be too much new invention with possibility of unforeseen
disruptions of the existing C type system. We encourage
experimentation with such extensions, so that there will be a
better chance of adding something along these lines in C0x.
Zalman Stern
[Regarding sizeof(char) == 1]
: Dennis insisted that it was an intended property.
: That pretty much doomed my attempt to prevent such a requirement.
If sizeof(char) != 1, how do you deal with code like:
buffer = malloc(strlen(input) + 1);
It is very rare to see a multiply by sizeof(char) in such a construct and
in fact I expect most programmers would frown on code that used such a
thing.
-Z-
Douglas A. Gwyn
The conversion is only needed on rare occasions when byte meets
character, as in your storage allocation example. The rest of
the time, character-using code looks just like the K&R1 style.
It's rather a moot point now, but remember that you already have
the "sizeof(char)==1" mindset. I actually had a *lot* of code
back when the rule was uncertain, that *did* carefully convert to
bytes via sizeof:
char *input; ...
char *buffer = malloc((strlen(input)+1) * sizeof(char));
Or more likely,
char *buffer = strdup(input);
which shows that this idea doesn't have to be obtrusive.
Of course, with C89's approach one has to do much the same thing:
wchar_t *input; ...
wchar_t *buffer = malloc((wstrlen(input)+1) * sizeof(wchar_t));
and C89 has made the original str* functions, string literals, and
char used for character codes all "second class", while forcing a
duplicate "wide version" of every library function that deals with
characters, and exposing the mapping between external and internal
representations (more like rubbing one's nose in it!).
Plan 9 takes an approach that is intermediate between what I proposed
and the C89 wchar_t facility. It is still annoying to have to
explicitly program conversions between external (UTF-8) and
internal (Unicode) representations. Rob entitled one paper "Window
Systems Should Be Transparent", the message being that the system
should not distract the user from what his actual purpose is; well,
character I/O should also be transparent, for the same reason.
It's apparently too late for C, but designers of new languages can
learn from our experience. (Unfortunately, they seem to think that
nailing down the coding to 16-bit Unicode is sufficient; it's not.)
James Kuyper
>
> Steven Correll wrote:
> > Here's the comp.arch puzzler: is there any hardware support for UTF-8
> > that would be reasonable in a RISC context?
This subthread has lost all special relevance to comp.std.c. Could you
redirect further followups to comp.arch exclusively?
Dik T. Winter
> > (The tables below use 8-character-wide tabs.)
> > The rule for delineating characters without decoding them is (my own code for
> > this shifts and masks the nibble, then indexes into a table):
> > High-order nibble Number of
> > of first octet octets in character
> > 0000..0111 1
> > 1000..1011 (this is an internal octet, not the first)
> > 1100..1101 2
> > 1110..1111 3
The last line is wrong. It should be 1110..1110. Bytes 11110xxx indicate
4 octets in character, 111110xx 5 octets and 1111110x 6 octets for a 31-bit
code. (UTF-8 encodes UCS-4, the 4 byte code of 10646.)
>
> I assume the following bytes can use all bits, do you have to convert
> between different endianisms as well, or can you just assume a 'natural'
> endian storage format?
Nope, bytes after the initial byte use only the lower 6 bits, the high
2 bits *must* be 10. (This makes it possible to detect looking at a
byte only whether you are in the middle of a character or not.) Endianism
does not come into play. You should regard UTF-8 coded data as a byte
stream. For UCS-4 and UCS-2 (the 16-bit coding of Unicode) endianism
comes into play, but there is a special symbol: BOM (Byte Order Mark)
that should be used to indicate endiannes. It is 0xFFFE, but can come
out as 0xFEFF (which indicates endian reversal), and with leading or
trailing zeros in UCS-4. Both are not valid Unicode values.
Note again the wrong use of UTF-16 for UCS-2. UTF-16 is a way to
encode the first 17 planes of ISO 10646 in 2 or 4 bytes. For this
reason codes 0xD800 to 0xDFFF are not valid in Unicode for a symbol.
Terje Mathisen
> Here's the comp.arch puzzler: is there any hardware support for UTF-8
> that would be reasonable in a RISC context?
Yes, bit scan instructions, as well as a fast way to handle misaligned
read/write:
>
> (The tables below use 8-character-wide tabs.)
>
> The rule for delineating characters without decoding them is (my own code for
> this shifts and masks the nibble, then indexes into a table):
>
> High-order nibble Number of
> of first octet octets in character
> 0000..0111 1
> 1000..1011 (this is an internal octet, not the first)
> 1100..1101 2
> 1110..1111 3
This could be done with a couple of tables and a (possibly misaligned)
32-bit load.
I'd guess the Altivec register-internal nybble lookup operation could
give you a fast way to perform this conversion.
I assume the following bytes can use all bits, do you have to convert
between different endianisms as well, or can you just assume a 'natural'
endian storage format?
>
> The rules for mapping UTF-16 to UTF-8 are:
>
> UTF-16 value "c"
> 0..127 128..2^11-1 2^11-1..2^16-1
> UTF-8
> octet#1 c ((c>>6)&0x1f)|0xc0 ((c>>12)&0xf)|0xe0
> octet#2 (none) (c&0x3f)|0x80 ((c>>6)&0x3f)|0x80
> octet#3 (none) (none) (c&0x3f)|0x80
>
> Or maybe it's clearer (as in plan9.bell-labs.com/magic/man2html/6/utf):
>
> 01. x in [00000000.0bbbbbbb] -> 0bbbbbbb
> 10. x in [00000bbb.bbbbbbbb] -> 110bbbbb, 10bbbbbb
> 11. x in [bbbbbbbb.bbbbbbbb] -> 1110bbbb, 10bbbbbb, 10bbbbbb
>
> My own code for mapping UTF-16 to UTF-8 does a "find first bit" using a
> shift and mask and lookup table, and then depending on the bitnumber,
So, you're using a 512-entry table, or do you do some compression?
> executes the code in the appropriate column above.
It would be neat to have branchless code for this, but this would more
or less force me into always doing 3 stores, even when only one is
needed. :-(
>
> My own code for mapping UTF-8 to UTF-16 first finds the number of octets, then
> executes one of three cases:
>
> c = s[0];
> c = ((s[0]&0x1f)<<6)|(s[1]&0x3f)
> c = ((s[0]&0xf)<<12)|((s[1]&0x3f)<<6)|(s[2]&0x3f)
Oops, I guess the succeeding bytes still only encodes 6 more bits. :-(
Having branchfull code to do this should still be OK as long as most
input data (i.e. text) tends to stay in the same group for a long time,
i.e. well-predicted switch statements.
Using a 32 or 64-bit register as a write buffer would probably pay off
in the form of better throughput, having write-combining L1 cache lines
would be give the same effect more easily.
Terje
--
- <Terje.M...@hda.hydro.com>
Using self-discipline, see http://www.eiffel.com/discipline
"almost all programming can be viewed as an exercise in caching"
Eric Hildum
Since most basic data types are at least 16 bits (and yes, I am including
characters in this - note where I live), is there a valid reason to continue to
support an archaic data format? C is a badly designed language, and it seems a
little unreasonable to continue supporting its worst elements in hardware -
software emulation is fine for that sort of thing.
If we really need an eight bit numeric datatype, then it should be explicitly
defined as such with appropriate symantics, not mixed in with a completely
unrelated character data type.
Zalman Stern wrote:
> James Kuyper (kuy...@wizard.net) wrote:
> : True. However, CHAR_BIT is explicitly not required to be 8, and an
> : implementation which set it to 16 would not, IMO, break most code
> : (though it would break a lot of code).
>
> "Most code" is incredibly dependent on where you sit in the world. But
> consider that if you set CHAR_BIT to 16 on a byte addressed machine, memcpy
> no longer has 8-bit byte granularity. In fact, there would be no reasonable
> way to talk about bytes anymore. All software in digital imaging and
> digital video I worked on in the last six years would break badly. I expect
> perl, X11 servers, and many device drivers would break. It would be quite
> like the problems experienced with C on word addressed machines
> (e.g. PDP-10), where char * pointers have a different representation than
> other pointers, but it would be worse than that because you wouldn't be
> able to address 8-bit bytes at all.
>
> -Z-
--
---------------------------
Eric Hildum
Eric....@Japan.NCR.COM
Eric Hildum
not yet live in the real world, and thus do stupid things like assume that
characters are 7 bits, and that a zero byte is the end of a string. The real
world is NOT limited to North America... and most of those "finished"
applications are not; they usually need to be completely rewritten before they
are.
Robert Harley wrote:
> zal...@netcom.com (Zalman Stern) writes:
> > James Kuyper (kuy...@wizard.net) wrote:
> > : True. However, CHAR_BIT is explicitly not required to be 8, and an
> > : implementation which set it to 16 would not, IMO, break most code
> > : (though it would break a lot of code).
> >
> > [...] All software in digital imaging and
> > digital video I worked on in the last six years would break badly. I expect
> > perl, X11 servers, and many device drivers would break. [...]
>
> Zalman, I think that there's a language problem here.
>
> One group of people use the word "C" to mean an abstract version of
> the language in which every program should work on every imaginable
> implementation as defined by the 7000-page draft-standard-du-jour.
>
> Other people, including you and I, use "C" to mean Real World C, which
> compiles with implementations that have the distinct advantage of
> actually existing, and which runs correctly on our target hardware.
>
> The other crowd will probably drop into this thread for a bout of
> flaming, as they usually do when anyone gores their sacred cows, but
> in fact their statements about C and ours are not contradictory since
> we're talking about two different languages altogether.
>
> In Real World C, chars have 8 bits. They just do. All integral
> arithmetic is done in twos-complement binary. It just is. Several
> simple "get real" assumptions like these make our work possible.
>
> The advantage of our point of view is that while we're sending out
> finished applications written in Real World C, the other crowd is
> still on line three, poring over some expression and trying to figure
> out if it would give the right answer on an imaginary implementation
> that switched between nines-complement BCD and signed-magnitude
> ternary every Tuesday, except when the moon is full.
>
> Bye,
> Rob.
Douglas A. Gwyn
> It is not clear to me that byte level addressing is all that useful anymore.
> Since most basic data types are at least 16 bits (and yes, I am including
> characters in this - note where I live), is there a valid reason to continue to
> support an archaic data format?
Even the C89 spec allows you to implement char as 16 bits if you so
choose. Nobody seems to so choose, because there is an awful lot
of octet-based stuff around.
Peter L. Montgomery
>It is not clear to me that byte level addressing is all that useful anymore.
>Since most basic data types are at least 16 bits (and yes, I am including
>characters in this - note where I live), is there a valid reason to continue to
>little unreasonable to continue supporting its worst elements in hardware -
>software emulation is fine for that sort of thing.
>
>If we really need an eight bit numeric datatype, then it should be explicitly
>defined as such with appropriate symantics, not mixed in with a completely
>unrelated character data type.
Byte-level addressing of numeric data is important in one
application I am running on many machines (MIPS, SPARC, Alpha, Pentium).
Let f be a polynomial with (possibly large) integer coefficients.
If a prime p divides a value f(x0), then the same p divides f(x0+p).
The program allocates a big byte array with one entry for each x0,
and adds round(log(p)) (wrt some appropriate base) to entry x0 whenever
p divides f(x0). Later we scan the entire array to find entries
with large values, meaning several primes divide that f(x0).
We could change it to use 16-bit or 32-bit array entries,
rather than 8-bit entries, but couldn't fit as much in cache.
The addressing is non-sequential, since we leap from
entry x0 to entry x0+p where p may be near 1 million.
Byte logarithms provide enough accuracy for our application.
--
Peter-Lawren...@cwi.nl Home: San Rafael, California
Microsoft Research and CWI
Zalman Stern
[...]
Simply arrange to have the following code inserted before anything else:
typedef char byte;
#define char wchar_t
All libraries you use will have to be compiled this way. (And probably the
operating system too, etc.) Of course if anything in your code or those
libraries is actually using octets (or something octet like) it will
probably break. Eric's stipulation is that this code is of less concern
than the character handling code. If you believe otherwise, then perhaps
you'll be happier with unadulterated char and wchar_t in ANSI C.
The thing we can learn from this is that mixing two distinct type intents,
namely "this type represents characters" and "this type represents octets"
is best avoided. This just convinces me more that letting wchar_t be the
only standard defined type of a certain size in an implementation is a bad
idea. (Because then, anyone wanting to get at that size type has to
overload two intents onto the wchar_t type.)
-Z-
Peter da Silva
Eric Hildum <Eric....@Japan.NCR.com> wrote:
>It is not clear to me that byte level addressing is all that useful anymore.
It is if you want to interact with international computer networks. I don't
know if there's a non-octet-based network in Japan, but virtually all protocols
I'm aware of are designed around 8 bit bytes.
--
In hoc signo hack, Peter da Silva <pe...@baileynm.com>
`-_-' "Heb jij vandaag je wolf al geaaid?"
'U`
"Tell init(8) to lock-n-load, we're goin' zombie slaying!"
st...@blighty.com
>
>It is not clear to me that byte level addressing is all that useful anymore.
>characters in this - note where I live), is there a valid reason to continue to
>support an archaic data format? C is a badly designed language, and it seems a
>little unreasonable to continue supporting its worst elements in hardware -
>software emulation is fine for that sort of thing.
Unless it impacts your performance on legacy code - IIRC the 8 bit
support instructions were added to the Alpha instruction set mostly because
NT needed them.
There are some really 8bit-centric structures embedded deep in the kernels
of many operating systems, NT is just one of the offenders.
>If we really need an eight bit numeric datatype, then it should be explicitly
>defined as such with appropriate symantics, not mixed in with a completely
>unrelated character data type.
We have an eight bit datatype. It's called 'char', for purely hysterical
reasons[1].
What's needed is a standard character datatype that's as painless to use
as ASCII/char, and is supported happily by traditional library functions.
That pretty much means that each character has to be the same width -
I don't want to have to parse an entire string to get the thousandth
character, or parse an entire file to get the last line[2].
Mixing formats on the same system will really upset the unix 'every file
is a stream of bytes' idiom, though, and file interchange will be painful
(look at the gross hacks that are scattered through a lot of code
to handle LF vs CRLF now - and that's easy compared to dynamically
translating character sets).
Cheers,
Steve
[1] Yeah, yeah, I know. But in practice, on most platforms char = byte,
and on many platforms char != character.
[2] If the OS wants to block compress files on disk that's OK as long
as it is transparent to most clients.
--
-- Steve Atkins -- st...@blighty.com
Maynard Handley
>In comp.arch Zalman Stern <zal...@netcom.com> wrote:
>> Robert Herndon (rm...@rmi.net) wrote:
>>> Some things don't change.
>> The software and hardware landscape has changed significantly in the last
>> 20+ years.
>
> What hasn't changed is that most people still can still find
>ways to program badly. "There is not now, and will never be,
>a language in which it is the least bit difficult to program
>badly" was first uttered more than 20 years ago. I think it was
>Robert Harley who was advocating 'quick and dirty' use of various
>assumptions about wordsize and data formats to save programming
>time. I think that in many cases, it's a very false economy.
A brief comment here.
What all these posts basically some down to is "people should be able to
read the future". This is not a reasonable thing to require.
Yes there are obviously stupid practices that can be avoided in coding.
However I'd like to think what we are discussing here are issues beyond
Programming 101.
In the third or fourth or fifth rewriting from scratch of a solved problem
it's fairly easy to know what the appropriate generalizations are. This is
hardly the case the first time through. Sometimes one omits what prove to
be important generalizations, but just as frequently one tries too hard
for generalizations that are hard to code, add unnecessary complexity, and
land up delaying the product or killing it. (As an Apple insider I've seen
a fair amount of this and I suspect MS and other companies have had the
same thing happen.) Sometimes the generalized problem simply is a MUCH
HARDER problem.
People claiming that "you should program for generality" are assuming that
"programming for generality" is free. It's fairly cheap to be general if
you know the scope of the generality is little-endian vs big-endian. But
that's also not a very hard problem.
A more realistic (in the sense of an interesting problem) example might
be, in the case of QuickTime vs Video For Windows and friends, does one
allow frame-accurate random access? Obviously frame-accurate random access
gives a nicer user experience and is more general than the "somewhere in
the last second or two" random access of Video For Windows. However it
leads to a whole model of video that is complicated (though managable) for
MPEG, leading to something that simply doesn't seem to mesh well at all
(for DVD).
(Ob legalism for those who care about this, this is not a claim that QT
cannot playback DVD, neither is it a claim that QT can and will playback
DVD. It's simply pointing out that DVD playback with its menus, seamless
branching etc, does not comfortably fit into the way QT views time, and in
fact might fit in more comfortably with the less general model of Video
for Windows.)
I support the extra generality of QT over VfW, but that generality has not
been free, and out of luck rather than good architecture, some future
developments have meshed rather better with VfW than QT.
People claiming they have a panacea, that they can view what the future
holds ten years from now and tell you how to structure your code are
speaking nonsense. Especially if that panacea involves worrying about a
bunch of issues (like will chars hold 7 bits or be 1's complements) that
are quite simply irrelevant to the bulk of programmers. As Zalman said, in
the world most programmers inhabit, there just simply ARE certain ways
these things work.
Yes, this changed over the past twenty years. But claiming it's going to
change in the same way over the next twenty seems pretty unlikely.
A more realistic nostrum might be to accept that you will have to rewrite
everything in five years anyway, in the light of experience, and
concentrate on ways to make that rewrite work without falling apart.
(Which means both the marketing/political issues of how to get management
buy-in, and the engineering/technical issues of how to keep the project
from second-system-syndrome.)
Maynard
Norman Diamond
>Norman Diamond wrote:
>>True. Unfortunately there were still lots of operations around that
>>had to be done on 8 or 9 bits, and EBCDIC files and even ASCII files
>>that were imported and had to be dealt with, etc., so char couldn't
>>be made 16 bits.
>
>Well, yes, it could have.
Not in C.
>Or more precisely, the implementation
>could have been allowed to make "char" big enough to hold any
>character code it needed to in that environment (8, 16, and 32
>bits all come to mind).
Unfortunately there were still lots of operations around that
had to be done on 8 or 9 bits, and EBCDIC files and even ASCII files
that were imported and had to be dealt with, etc., so it was
necessary to have a C type that could handle 8 or 9 bit entities
as first class objects.
>But one would have to provide a separate
>"byte" type to cater to the possibly smaller granularities of
>such things as memory addressing boundaries/allocation units/
>punched paper tape/whatever. There was in fact a proposal to do
>just that (my name for "byte" was "short char" to avoid using a
>new keyword).
Right. In other words, "Well, yes, it could have, by defining
the C language differently."
Meanwhile, for people who had to use the C language, even for
implementors who had to implement the C language, char couldn't
be made 16 bits.
Maybe you're confused because the subject line talks about Fortran
and this thread is crossposted to comp.arch. I'm reading this
in comp.std.c. I think this thread started with an idea of defining
the C language differently in a way that can emulate some Fortran
declaration syntax. Someone seemed to think it was possible to
provide the same functionality by defining CHAR_BIT as 16 (or 18).
It isn't.
Maybe you can get the C language defined differently so it will be
more useful. Meanwhile, for people who have to use the C language,
even for implementors who have to implement the C language, char
can't be made 16 bits.
>One important feature was that text-oriented I/O
>streams automatically converted back and forth to "char" while
>binary-oriented streams worked in bytes. I18n would have been
>a lot simpler with such a scheme.
Yeah it would have been. You know, even those of us who have to
read garbage like "Well, yes, it could have" might remember that
you're not always such a prick, if you'll get some useful stuff
standardized instead of telling us to fantasize.
--
<< If this were the company's opinion, I would not be allowed to post it. >>
"I paid money for this car, I pay taxes for vehicle registration and a driver's
license, so I can drive in any lane I want, and no innocent victim gets to call
the cops just 'cause the lane's not goin' the same direction as me" - J Spammer
Dave Hansen
<Eric....@Japan.NCR.com> wrote:
>It is not clear to me that byte level addressing is all that useful anymore.
Two letters for you. I/O.
Regards,
-=Dave
Just my (10-010) cents
I can barely speak for myself, so I certainly can't speak for B-Tree.
Change is inevitable. Progress is not.
Norman Diamond
>It is not clear to me that byte level addressing is all that useful anymore.
>is there a valid reason to continue to support an archaic data format?
Yes. On nearly every computer you use, some files are encoded in SJIS
or EUC. Some characters occupy 8 bits (in which case usually the top
bit is a 0 but there are a few exceptions). Most characters occupy
16 bits. In order to convert these formats to and from wide characters,
the implementation's stdio library needs to be coded in assembly language.
Once upon a time someone thought he invented a high level syntax which
could replace assembly language. He thought that implementations of
other languages could be coded using his language. But as he observed
recently, there was no demand for this kind of language. His invention
was hijacked by other people and is no longer usable where assembly
language is needed.
But anyway yes, your computers have lots of files encoded in an hacked
combination of archaic data formats, you're not going to run a conversion
program on all your files every time you update your OS or your C
implementation, and your employer isn't going to run a conversion
program on all its databases whenever they update their OS or C
implementation.
Besides, your posting was written using a very limited subset of SJIS or
EUC. Your communication software converts between one (or both) of
those and JIS Romaji. When your posting was transmitted to other
machines, it was likely done in JIS, which is yet another hacked
combination of archaic data formats with shift sequences. But your
posting didn't result in any shift sequences because you only used
characters which fit in 8 bits (with the top bit a 0).
So, yes, we still need to support it. Since C has sizeof(char) == 1,
it is impractical to implement CHAR_BIT == 16.
Peter Seebach
Eric Hildum <Eric....@Japan.NCR.com> wrote:
>Part of the problem is that the vast majority of "real world" C programmers do
>not yet live in the real world, and thus do stupid things like assume that
>characters are 7 bits, and that a zero byte is the end of a string.
Well, a zero byte *is* the end of a string in C. That's what the spec says.
(Not the end of a wide string, necessarily, but it is the end of any other
string.)
-s
--
Copyright 1999, All rights reserved. Peter Seebach / se...@plethora.net
C/Unix wizard, Pro-commerce radical, Spam fighter. Boycott Spamazon!
Will work for interesting hardware. http://www.plethora.net/~seebs/
Visit my new ISP <URL:http://www.plethora.net/> --- More Net, Less Spam!
Will Rose
: > It is not clear to me that byte level addressing is all that useful anymore.
: > Since most basic data types are at least 16 bits (and yes, I am including
: > characters in this - note where I live), is there a valid reason to continue to
: > support an archaic data format?
: Even the C89 spec allows you to implement char as 16 bits if you so
: choose. Nobody seems to so choose, because there is an awful lot
: of octet-based stuff around.
You could, I suppose, argue that byte level addressing isn't much use
any more in applications; I don't think it's a sound argument, but that's
just me. However, C is a systems programming language by design, and the
basic element has to be the smallest accessible element of memory, which
generally in these days is 8 bits.
The few C implementations I've seen which had characters the same size
as ints were all running on hardware where access to smaller elements
of memory wasn't possible.
Steven Correll
> > > High-order nibble Number of
> > > of first octet octets in character
> > > 0000..0111 1
> > > 1000..1011 (this is an internal octet, not the first)
> > > 1100..1101 2
> > > 1110..1111 3
>
>4 octets in character, 111110xx 5 octets and 1111110x 6 octets for a 31-bit
>
>Note again the wrong use of UTF-16 for UCS-2. UTF-16 is a way to
>encode the first 17 planes of ISO 10646 in 2 or 4 bytes.
Thanks for the corrections. The last line in the table should indeed be
1110 only, and all the references to UTF-16 should be UCS-2.
--
Steven Correll == 1931 Palm Ave, San Mateo, CA 94403 == s...@netcom.com
Zalman Stern
: A brief comment here.
: read the future".
I think the "gray bearded curmudgeons" are saying "if you did character
processing the way we did it when different character sizes and
representations were incredibly common and we had to deal with them as a
matter of course, then C programs would be more portable." And I'm not sure
I agree with this, but I don't have enough first hand experience to say.
Before C, languages either had significant runtime support for text
processing, or one was forced to write code that was probably less portable
than it would be in C. As has been pointed out here, one can without undue
effort write a C program that works on 8 and 9 bit-per-byte systems. If one
had to do this on a system without something like "character pointers" one
would have to use shift and mask with constants that are likely platform
dependent. The languages with significant support for text processing
might have worked well for many things. As usual, you have the problem of
how well implemented the runtime is (especially for long pieces of text)
and whether its processing model matches the one your application needs,
etc.
But even so, a text processing API is unlikely to be helpful for pixel
processing, or network protocol work. We can use "byte pointers" in C to do
graphics quite handily. The alternative would be to wait until graphics
gets standardized into the library, or to expend more effort on the
graphics portability library. As its stands now, a I know many people who
hack out basic graphics and imaging code in C and it runs on a variety of
platforms without much effort. They're relying on the fact that the
language has "byte pointers" which more or less behave the same way on all
platforms, but who cares? Its productive programming. A similar argument
can be made for network protocols.
None of this means I believe C is an optimal design. It isn't. But its
sheer folie to write of the ease with which many types are software are
written in C and the reasonably good portabilty this software enjoys.
-Z-
James Kuyper
...
> The few C implementations I've seen which had characters the same size
> as ints were all running on hardware where access to smaller elements
> of memory wasn't possible.
That's exactly the kind of situation we're talking about. There are
also, I believe, implementations where char is 9, 12(?) or 16 bits, for
the same reason.
Joe Keane
If some jag-offs make their own version of C that breaks some basic
rules, it's their own version, it's not standard C.
For example, a while back, companies made compilers for some bastard
version of C with `near' and `far' pointers. This doesn't mean that C
now has near and far pointers. It didn't then, and it doesn't now.
Yes they were popular compilers. Yes a lot of programs were written in
that screwed-up language. So what. Most of the people that worked on
those systems went insane and just babble incoherently anyway.
Since this is a standards group, we should, for the most part, concern
ourselves with the standard, that means, compilers that follow the
standard, and programs that follow the standard.
Now to my point: `int' is a word.
This is given in K&R 1, and to my knowledge it has not been revoked by
Dennis Ritchie.
You may say, in some cases, it's not clear what a `word' is. You can
come up with examples of machines, real or contrived, where people could
disagree on the word size. That's fine. In these cases, there may be
multiple reasonable choices where none of them are clearly wrong.
But what's also true is that there are many machines where the word size
is clear. The PDP-11 is a 16-bit machine. The VAX is a 32-bit machine.
Someone can explain their theory about why either of these have some
other word size, but they would be way in left field.
Now we look at the C compilers for those machines. On the PDP-11 `int'
is 16 bits. On the VAX `int' is 32 bits. Again someone can claim it
should be something else, but we can say that they're just wrong.
Did it sometimes cause problems that `int' is not always the same size?
Of course it did. Sometimes people made assumptions that they shouldn't
have, so they had to change their code. But we can't be too sorry for
them; this property of `int' was set out from the beginning, so they
knew pretty well what they were doing.
Perhaps at the time, some people said that `int' should remain 16 bits
to allow this broken code to continue to work. Perhaps some other
people said that this is a retarded idea, because it is.
Now, somewhat later, but still in many people's lifetimes, we have some
compilers that actually do this. The reasons given are exactly the
same. Is it any better than it was then? No it's not.
I guess there's some weak suggestion that the Alpha is a 32-bit machine,
or the definition of C has changed somehow. Neither of these are true.
I've looked at the Alpha. Saying that it's a 32-bit machine is no more
credible than saying the PDP-11 is a 64-bit machine.
Thank you for your time.
--
Joe Keane, amateur mathematician
David R Tribble
> : Then what do you call the 16-bit type?
Zalman Stern wrote:
> James Kuyper sent email saying one can use wchar_t for this. Which
> seems correct to me. So I'll rescind the comment. Though it is a
> little troubling to force wchar_t into that role as one might want to
> make wchar_t be 32-bits to support the full UCS-4 encoding. I think it
> is a good idea to provide all reasonable access sizes distinctly from
> wchar_t and then make wchar_t be the "right" size for text characters
> in the environment in question.
But wchar_t could be unsigned. Now how do you say 'signed 16-bit
int'?
Better to leave well enough alone and follow DEC's lead of using
(as you originally stated) 8-bit char, 16-bit short, 32-bit int,
64-bit long, and 64-bit pointer (and 64-bit long long). Anyone
for Java?
-- David R. Tribble, dtri...@technologist.com --
David R Tribble
> sizeof(int1_t) == 1 // e.g., signed char
> sizeof(int2_t) == 2 // e.g., signed short
> sizeof(int4_t) == 4 // e.g., signed int or signed long
> sizeof(int8_t) == 8 // e.g., signed long long
>
> (If I'm not mistaken, this resembles the way integer types are
> declared in FORTRAN, e.g. INTEGER*4.)
Eric Hildum wrote:
> I was looking at the table below, and wondering why there is the
> question about 64 bits at all. Since a character is 16 bits, the short
> is then 32, with the rest handling the integer sizes up to 128 bits
> quite nicely.
>
> Or are you still in the parochial US char is one byte mode? ;-)
I see the smiley. If I had things my way, 'char' would indeed be
unsigned 16-bit Unicode. That's one reason 'wchar_t' is such a
disappointment; since it is guaranteed to have only 8 bits, it's
useless for Unicode. (This fact is mentioned in the Unicode
reference spec; the suggest using your own user-defined type instead
of 'wchar_t').
Still, there is the tiny problem of how to read and write 8-bit
octets to your network interface card. Not to mention the problems
that arise when 'char' is allowed to be signed. Perhaps we need
a 'byte' type for 8-bit "byte" units. Anyone for Java? ;-)
James Kuyper
>
> James Kuyper (kuy...@wizard.net) wrote:
> : That's exactly the kind of situation we're talking about.
>
> Its one of the situations we're talking about. Some people in this thread
> are seriously proposing changing "char" to be bigger(*) as a matter of
> course, including in existing environments. Perhaps also adding a new
I hadn't gotten that impression. Is there anyone who's explicitly
suggested that char be 16 bits on a platform that is, at the hardware
level, 8-bit addressable?
James Kuyper
...
> unsigned 16-bit Unicode. That's one reason 'wchar_t' is such a
> disappointment; since it is guaranteed to have only 8 bits, it's
It's not "guaranteed to have only 8 bits". It's "only guaranteed to have
8 bits". There's a big difference between those two statements.
> useless for Unicode. (This fact is mentioned in the Unicode
It's quite useable for Unicode, if an implementor chooses to use it that
way. It's just not guaranteed to be Unicode, which is the more important
disadvantage.
Zalman Stern
: That's exactly the kind of situation we're talking about.
Its one of the situations we're talking about. Some people in this thread
are seriously proposing changing "char" to be bigger(*) as a matter of
course, including in existing environments. Perhaps also adding a new
smaller type to C or perhaps not. I insist that the "char is the smallest
type, having sizeof(char) == 1" is a much more relied upon contractual
obligation of C than the recently screamed about "long is the longest type"
one.
As usual, the argument is not being helped by having fifteen different
subissues mixed in together. (Not that I really care. Most of this is not
going to change no matter what so productive people find other solutions to
the problems.)
(*) I imagine these folks are thinking 16-bits for standard 8-bit byte
platforms. There certainly used to be platforms where even though CHAR_BIT
== 8, sizeof(wchar_t) == 4. I wonder how these folks feel about making char
32-bits on an 8-bit byte platform.
-Z-
Eric Hildum
suited to storage and transmission of information than the manipulation. I think
UTF-16 is probably the better form to use when processing character data.
Terje Mathisen wrote:
> Steven Correll wrote:
> > Here's the comp.arch puzzler: is there any hardware support for UTF-8
> > that would be reasonable in a RISC context?
>
> Yes, bit scan instructions, as well as a fast way to handle misaligned
> read/write:
>
> >
> > (The tables below use 8-character-wide tabs.)
> >
> > The rule for delineating characters without decoding them is (my own code for
> > this shifts and masks the nibble, then indexes into a table):
> >
> > High-order nibble Number of
> > of first octet octets in character
> > 0000..0111 1
> > 1000..1011 (this is an internal octet, not the first)
> > 1100..1101 2
> > 1110..1111 3
>
--
---------------------------
Eric Hildum
Eric....@Japan.NCR.COM
Eric Hildum
it should be a separate data type from char, with syntax and semantics appropriate
for that data type.
The problem is that char is defined as both a character and a numeric type in C. This
leads to no end of problems - such as bugs when char is set to 16 bits.
I am not sure I would consider an Octet a data type as it is usually used to describe
the amount of storage in a message used by some other data type (character, numeric).
Have you ever seen any usage of octet where semantics such as addition or collating
are defined?
"Douglas A. Gwyn" wrote:
> Eric Hildum wrote:
> > It is not clear to me that byte level addressing is all that useful anymore.
> > Since most basic data types are at least 16 bits (and yes, I am including
> > characters in this - note where I live), is there a valid reason to continue to
> > support an archaic data format?
>
> Even the C89 spec allows you to implement char as 16 bits if you so
> choose. Nobody seems to so choose, because there is an awful lot
> of octet-based stuff around.
--
---------------------------
Eric Hildum
Eric....@Japan.NCR.COM
Douglas A. Gwyn
> Maybe you're confused because the subject line talks about Fortran
> and this thread is crossposted to comp.arch. I'm reading this
> in comp.std.c. I think this thread started with an idea of defining
> the C language differently in a way that can emulate some Fortran
> declaration syntax. Someone seemed to think it was possible to
> provide the same functionality by defining CHAR_BIT as 16 (or 18).
> It isn't.
> Maybe you can get the C language defined differently so it will be
> more useful. Meanwhile, for people who have to use the C language,
> even for implementors who have to implement the C language, char
> can't be made 16 bits.
The current and forthcoming C standards do permit CHAR_BIT to be
defined as 16. If the customers are happy, the implementor can
in fact take that approach. Obviously, on many platforms (which
have a lot of 8- or 9-bit oriented data files) the customers
wouldn't be happy.
> >One important feature was that text-oriented I/O
> >streams automatically converted back and forth to "char" while
> >binary-oriented streams worked in bytes. I18n would have been
> >a lot simpler with such a scheme.
> Yeah it would have been. You know, even those of us who have to
> read garbage like "Well, yes, it could have" might remember that
> you're not always such a prick, if you'll get some useful stuff
> standardized instead of telling us to fantasize.
If you don't learn from past mistakes, you're doomed to repeat
them.
TTK Ciar
Eric Hildum <Eric....@Japan.NCR.com> wrote:
>I think you missed my second point, which is that if an byte numeric
>type is needed, it should be a separate data type from char, with
>syntax and semantics appropriate for that data type.
I'm starting to think this is close to the best solution. Question:
Should our almost-C compiler interpret string literals as arrays of
characters or arrays of bytes?
If we want compatability with existing C code, we should probably
keep the "char" type as the de facto "smallest word supported by the
hardware" type, and invent a new "lang" type which stores lingual
information, imo. But I'm still pondering that, so I'm not sure.
>The problem is that char is defined as both a character and a numeric
>type in C. This leads to no end of problems - such as bugs when char
>is set to 16 bits.
And confusion when a programmer looks at an unfamiliar algorithm and
can't tell if it's manipulating numbers or language..
-- TTK
Douglas A. Gwyn
> I think you missed my second point, which is that if an byte numeric type is needed,
> it should be a separate data type from char, with syntax and semantics appropriate
> for that data type.
No, I hardly missed that point, since it was the essential idea in
my original "short char" proposal for the forthcoming C standard
back in the 1980s. (Which lost out to the wchar_t approach.)
> The problem is that char is defined as both a character and a numeric type in C. This
> leads to no end of problems - such as bugs when char is set to 16 bits.
Yes, which is why I *agree* that there should be separate types.
In C89 (and C9x), the type for "byte" is spelled "unsigned char"
and the type for "character" is spelled "wchar_t". It is
confused by the appearance of character support (old-style
character constants, strings, str* functions, etc.) that
work only when the character codes all fit within a byte.
In my "short char" proposal, that confusion was not present.
> I am not sure I would consider an Octet a data type as it is usually used to describe
> the amount of storage in a message used by some other data type (character, numeric).
> Have you ever seen any usage of octet where semantics such as addition or collating
> are defined?
I don't see why that matters. It is sufficient that many programs
exist that need to *handle* octet data, such as packing and unpacking
encodings like UTF-8. Indeed, there have been many people who have
in the past expressed an opinion that "byte" means "octet", since
they are so used to that being the case on their platforms.
Flexible specification of integer representations is not a new
topic in C standardization. I refer you to the SBEIR proposal
that was considered in conjunction with <inttypes.h>. I think
I can speak (informally) for the C standards committee when I
say that we encourage experimentation with such ideas, with an
eye to potential inclusion in a future edition of the standard.
I personally want to encourage development of bit-addressable
architectures. This is hard to achieve due to a "chicken and
egg" problem: Programming languages won't offer good support
for such a capability until it becomes common, but architects
are not going to provide such a facility until it can be
exploited by popular programming languages.