Uninitialized memory, malloc and unsigned char
Tim Woodall
http://groups.google.co.uk/group/comp.std.c/browse_frm/thread/77dcd100ce6d4e24/08f556268d6d6f5e?lnk=gst&q=uninitialized++char+group%3Acomp.std.c#
but that still leaves me unclear.
(I don't really want to kick off that thread all over again, I'm more
interested in whether they have been any DRs since 2000 that might have
clarified some of the questions below)
Does accessing uninitialized memory via unsigned char cause undefined
behaviour?
The standard doesn't actually define uninitialized. It talks about
indeterminate values (3.17.1). An indeterminate value is either
unspecified or a trap representation.
But unsigned char is not allowed to have a trap representation. Which
implies you can access automatic variables via unsigned char:
6.7.8 #10
If an object that has automatic storage duration is not initialized
explicitly, its value is indeterminate.
But then appendix J ( the behaviour shall be undefined ) says,
- The value of an object with automatic storage duration is used while
it is indeterminate.
This can be interpreted two ways (at least).
By analogy, let indeterminate be rectangle and unspecified be square
with trap being an oblong:
1st interpretation: using a rectangle invokes undefined behaviour. An
uninitialized unsigned char is a rectangle (as well as a square) and
therefore using it causes undefined behaviour.
2nd interpretation: using an oblong invokes undefined behaviour. An
uninitialized unsigned char is NOT an oblong and therefore using it
causes no undefined behaviour.
Personally, I prefer the first reading because appendix J could have
said "is used while it is a trap representation".
Appendix J also has:
- The value of any bytes in a new object allocated by the realloc
function beyond the size of the old object are used.
Here it's explicity talking about bytes but I can see how malloc might
not actually create the object until you write to it. But there clearly
is some memory that you cannot read using unsigned char until you've
written to it.
One last muse:
struct s { unsigned char a; unsigned char b; };
int main(void)
{
struct s x,y,z;
y.a=z.a; y.b=z.b; /* Is this allowed ? */
x=y; /* Is this allowed ? */
x=z; /* And is this allowed ? */
return 0;
}
Assuming uninitialized chars can be read then y.a=z.a; y.b=z.b; will
clearly be allowed which achieves the same effect as assiging z to x
other than copying any possible padding bits.
Tim.
--
God said, "div D = rho, div B = 0, curl E = - @B/@t, curl H = J + @D/@t,"
and there was light.
Jun Woong
> I've read through a long thread from 2000:http://groups.google.co.uk/group/comp.std.c/browse_frm/thread/77dcd10...
> but that still leaves me unclear.
>
> (I don't really want to kick off that thread all over again, I'm more
> interested in whether they have been any DRs since 2000 that might have
> clarified some of the questions below)
>
I've not read the thread above, so the following answers are based on
my own interpretation of the standard.
> Does accessing uninitialized memory via unsigned char cause undefined
> behaviour?
>
Nope. unsigned char is not allowed to have padding bits [*], which
means every possible bit-pattern is guaranteed to contribute to a
value.
[*] To be precise, if unsigned char (byte) has padding bits on a
machine, all other types also have to have them at the same bit-
position and a C implementation on that machine is required to make
them invisible to a user.
[...]
>
> But then appendix J ( the behaviour shall be undefined ) says,
>
> - The value of an object with automatic storage duration is used while
> it is indeterminate.
>
That appendix is just informative. The item you quote can be taken as
a sort of (inaccurate) summary. Don't depend on it when trying to get
a precise meaning of the standard.
[...]
>
> One last muse:
>
> struct s { unsigned char a; unsigned char b; };
>
> int main(void)
> {
> struct s x,y,z;
> y.a=z.a; y.b=z.b; /* Is this allowed ? */
I think so.
> x=y; /* Is this allowed ? */
> x=z; /* And is this allowed ? */
Validity of these two assignments depends on whether the struct value
can be indeterminate even when all of its members cannot. A relevant
wording is 6.2.6.1p6 (I quoted the whole paragraph to preserve the
context):
When a value is stored in an object of structure or union type,
including in a member object, the bytes of the object
representation that correspond to any padding bytes take
unspecified values. The value of a structure or union object is
never a trap representation, even though the value of a member of
the structure or union object may be a trap representation.
I think this ensures that the holes between sturct members do not
affect validity of the struct.
I'm surprised, however, to see the last sentence quoted above. That
seems to say that an implementation is prohibited from incurring
undefined behavior even when the member of a struct being assigned
has a trap representation and it does the member-to-member
assignment. Is this really intended, or does the sentence in question
have to read as
The value of a structure or union object is not allowed to have a
trap representation unless one of its members has a trap
representation.
?
--
Jun, Woong (woong at icu.ac.kr)
Samsung Electronics Co., Ltd.
``All opinions expressed are mine, and do not represent
the official opinions of any organization.''
Tim Woodall
Jun Woong <wo...@icu.ac.kr> wrote:
> Tim Woodall <devn...@woodall.me.uk> wrote:
>> I've read through a long thread from 2000:http://groups.google.co.uk/group/comp.std.c/browse_frm/thread/77dcd10...
>> but that still leaves me unclear.
>>
>> (I don't really want to kick off that thread all over again, I'm more
>> interested in whether they have been any DRs since 2000 that might have
>> clarified some of the questions below)
>>
>
> I've not read the thread above, so the following answers are based on
> my own interpretation of the standard.
>
>> Does accessing uninitialized memory via unsigned char cause undefined
>> behaviour?
>>
>
> Nope. unsigned char is not allowed to have padding bits [*], which
> means every possible bit-pattern is guaranteed to contribute to a
> value.
>
> [*] To be precise, if unsigned char (byte) has padding bits on a
> machine, all other types also have to have them at the same bit-
> position and a C implementation on that machine is required to make
> them invisible to a user.
>
with the standard?
1. All memory at startup is in an indeterminate state.
2. The first access to each byte of memory defines whether it is read
only memory (a read) or read/write memory (a write).
There are no trap representations here in unsigned char. A read of
uninitialized memory will succeed and return an unspecified value but
any further write to that memory will then fail.
I missed that. However, I think the standard makes sense here.
This wording means that you can copy a structure and then access the
members that are known to be defined without invoking undefined
behaviour. Only if you attempt to access the individual undefined
members will you invoke undefined behaviour.
void eg(void)
{
struct st x;
x.val1 = 7;
function_that_only_uses_val1_in_struct(x); /* Pass by value */
Richard Tobin
Jun Woong <wo...@icu.ac.kr> wrote:
> [...] The value of a structure or union object is
> never a trap representation, even though the value of a member of
> the structure or union object may be a trap representation.
>
>I think this ensures that the holes between sturct members do not
>affect validity of the struct.
>
>I'm surprised, however, to see the last sentence quoted above. That
>seems to say that an implementation is prohibited from incurring
>undefined behavior even when the member of a struct being assigned
>has a trap representation and it does the member-to-member
>assignment.
Presumably the idea is that given
struct s {int a; int b;} x, y;
x.a=1;
then x.b may have a trap representation so that y.b = x.b may trap,
but y = x is allowed.
It would be very inconvenient if you could not do a structure
assigment unless all the members were initialised.
-- Richard
--
In the selection of the two characters immediately succeeding the numeral 9,
consideration shall be given to their replacement by the graphics 10 and 11 to
facilitate the adoption of the code in the sterling monetary area. (X3.4-1963)
Francis Glassborow
> In article <77890674-f33c-412f...@u6g2000prc.googlegroups.com>,
> Jun Woong <wo...@icu.ac.kr> wrote:
>> [...] The value of a structure or union object is
>> never a trap representation, even though the value of a member of
>> the structure or union object may be a trap representation.
>>
>> I think this ensures that the holes between sturct members do not
>> affect validity of the struct.
>>
>> I'm surprised, however, to see the last sentence quoted above. That
>> seems to say that an implementation is prohibited from incurring
>> undefined behavior even when the member of a struct being assigned
>> has a trap representation and it does the member-to-member
>> assignment.
>
> Presumably the idea is that given
>
> struct s {int a; int b;} x, y;
> x.a=1;
>
> then x.b may have a trap representation so that y.b = x.b may trap,
> but y = x is allowed.
>
> It would be very inconvenient if you could not do a structure
> assigment unless all the members were initialised.
I think it is worse than that. It seems that even if no member has been
initialised that reading the (indeterminate) value must not trap. If
that is true then compilers must ensure that all instances of a struct
must be initialised to some extent. Worse, it means that a member must
never acquire a trap value, that seems rubbish to me.
I think the intent is that padding must never cause trapping, but it is
OK if the value of a member traps.
Richard Bos
No, they needn't. They must ensure that no struct or union, when
accessed _as itself_, triggers a trap. That is, assigning whole struct
or union values to other struct or union objects (of the same type,
naturally) must always work, regardless of the contents of that struct
or union. IOW, the struct/union's validity _does not depend_ on its
members' validity; it is always valid, regardless of what's in it.
Essentially, this means that you can't use too many dirty tricks when
copying structs/unions. If the largest member of a union is a long int,
it is tempting to use long int assignment to copy that union; but this
is not an allowed shortcut if that long int assignment could trap. The
implementation _must_ use a safe method - probably byte-by-byte - to
copy or read the union. That's all. As long as that safe method _is_
safe even for uninitialised bytes - _not_ member values - that is
enough.
> Worse, it means that a member must never acquire a trap value,
> that seems rubbish to me.
As indeed it is. On many systems, it would be impossible. What if you
have a union of a float and an unsigned long int? You could easily set
the float to a trap value by giving the unsigned long a suitable value.
And there's no way around that. But _the union itself_ is not, even
then, a trap value.
(I don't think your reading is surprising, btw. I read it myself that
way, some time ago. But I read it wrong then, and you read it wrong now.
I _do_ think that this sentence could and should have had an example
attached to it to clarify it.)
Richard
Wojtek Lerch
news:z_6dnU829sH3LKfV...@bt.com...
> Richard Tobin wrote:
>> In article
>> <77890674-f33c-412f...@u6g2000prc.googlegroups.com>,
>> Jun Woong <wo...@icu.ac.kr> wrote:
>>> [...] The value of a structure or union object is
>>> never a trap representation, even though the value of a member of
>>> the structure or union object may be a trap representation.
>> It would be very inconvenient if you could not do a structure
>> assigment unless all the members were initialised.
>
> I think it is worse than that. It seems that even if no member has been
> initialised that reading the (indeterminate) value must not trap.
Reading the value of the whole struct must not trap. Reading an
uninitialised member alone can trap. Whether the trap can happen or not
depends on whether the type of the object whose value is read (i.e. the type
of the lvalue that is converted to a value) is a structure.
> If that is true then compilers must ensure that all instances of a struct
> must be initialised to some extent. Worse, it means that a member must
> never acquire a trap value, that seems rubbish to me.
No, the quoted text specifically says a structure representation that
contains trap representations in its members is not a trap representation
itself. All that compilers must ensure is that operations such as
assignment and function calls can copy whole structures without ever
trapping. If they're implemented as bytewise copies, they can safely copy
structures whose members contain trap representations.
> I think the intent is that padding must never cause trapping, but it is OK
> if the value of a member traps.
It's OK if the value of a member traps when you read the value of the
member, but not when you read the value of the whole structure.
Richard Tobin
Francis Glassborow <francis.g...@btinternet.com> wrote:
>> Presumably the idea is that given
>>
>> struct s {int a; int b;} x, y;
>> x.a=1;
>>
>> then x.b may have a trap representation so that y.b = x.b may trap,
>> but y = x is allowed.
>I think it is worse than that. It seems that even if no member has been
>initialised that reading the (indeterminate) value must not trap.
Yes, I only put the initialisation of x.a in to motivate the example.
>If
>that is true then compilers must ensure that all instances of a struct
>must be initialised to some extent.
Only compilers on machines that have trap representations. And they
have the alternative of doing structure assignment in some way that
doesn't trigger the trap, which must be possible as byte accesses
don't trap.
goo...@woodall.me.uk
> > Jun Woong <wo...@icu.ac.kr> wrote:
> >> [...] The value of a structure or union object is
> >> never a trap representation, even though the value of a member of
> >> the structure or union object may be a trap representation.
>
> >> I think this ensures that the holes between sturct members do not
> >> affect validity of the struct.
>
> >> I'm surprised, however, to see the last sentence quoted above. That
> >> seems to say that an implementation is prohibited from incurring
> >> undefined behavior even when the member of a struct being assigned
> >> has a trap representation and it does the member-to-member
> >> assignment.
>
> > Presumably the idea is that given
>
> > struct s {int a; int b;} x, y;
> > x.a=1;
>
> > then x.b may have a trap representation so that y.b = x.b may trap,
> > but y = x is allowed.
>
> > It would be very inconvenient if you could not do a structure
> > assigment unless all the members were initialised.
>
> I think it is worse than that. It seems that even if no member has been
> initialised that reading the (indeterminate) value must not trap.
Yes. But that comes back to my original point of is there a difference
between indeterminate but not a trap representation and unspecified,
i.e. does the following invoke undefined behaviour:
int main(void)
{
struct s { int c; } x,y;
x=y; /* This cannot trap and x will be initialized to an unspecified
value; but y is read while it is indeterminate. Can that have bad
effects? */
}
Also what about the following:
int main(void)
{
struct s { int c; } x, *y;
y=malloc(sizeof *y); /* assume this succeeds */
x=*y;
}
Appendix J (which is informative, not normative) says you cannot
access the *bytes* of memory returned from malloc until they have been
initialized (IIRC it actually only says bytes for the extra memory of
realloc but modify the above to use realloc if you prefer)
I don't know where in the normative standard this claim is derived
from.
Personally, I feel this area of the standard is in need of a DR to
clarify.
There's this bit: (from c89 but there's similar wording in C99)
All accessible objects have values as of the time longjmp was
called, except that the values of objects of automatic storage
duration that do not have volatile type and have been changed between
the setjmp invocation and longjmp call are indeterminate.
Baring a few operations that can set a value to indeterminate anyway
regardless of the setjmp/longjmp, the value of these objects should
surely be unspecified, not indeterminate.
setvbuf:
If buf is not a null pointer,
the array it points to may be used instead of a buffer allocated by
the setvbuf function./105/ The argument size specifies the size of the
array. The contents of the array at any time are indeterminate.
It's the "at any time" here - especially as buf is a char array.
fgets, ditto on read error.
etc.
(This came about discussing the Debian OpenSSL bug. I said that
accessing uninitialized memory is a bug (in my books it's a bug even
if it is allowed in certain circumstances in the standard) but it
appears there's enough ambiguity to argue the OpenSSL case either way
in a strict reading of the standard)
Tim.
Francis Glassborow
OK, but I find the requirement pointless. It just imposes a burden on
implementations without any benefit to users. Worse, it requires that a
struct can be passed by value even when the consequence of using the
result may be undefined behaviour. It prohibits an implementation from
doing anything to detect the problem before you actually access a member
that traps.
Jun Woong
> On Mon, 26 May 2008 00:34:49 -0700 (PDT), Jun Woong <wo...@icu.ac.kr> wrote:
> > Nope. unsigned char is not allowed to have padding bits [*], which
> > means every possible bit-pattern is guaranteed to contribute to a
> > value.
> >
> > [*] To be precise, if unsigned char (byte) has padding bits on a
> > machine, all other types also have to have them at the same bit-
> > position and a C implementation on that machine is required to make
> > them invisible to a user.
>
> Yes, that's true. But would the following implementation be compatible
> with the standard?
>
> 1. All memory at startup is in an indeterminate state.
> 2. The first access to each byte of memory defines whether it is read
> only memory (a read) or read/write memory (a write).
>
> There are no trap representations here in unsigned char. A read of
> uninitialized memory will succeed and return an unspecified value but
> any further write to that memory will then fail.
>
I don't see any guarantee from the standard that renders this
implementation to be conforming. If I understand your example, on it
the following (conforming) code must fail:
int a; /* in an indeterminate state */
unsigned char *uc = (void *)&a;
*uc; /* read an unspecified value */
a = 0; /* must fail! */
[...]
My belief was that, in order not to invoke UB every member of a
struct needed to be initialized properly. I've always thought that
the above wording meant just that paddings between members are not
allowed to affect validity of the whole struct.
What the current standard guarantees now is that the following code
would never fail:
void foo(void)
{
struct { ... } x, y;
x = y;
}
I don't see why such an erroneous code should be endorsed.
If it's general to initialize only a part of a struct before using
the wording in question should be restricted more than it is, I
think.
Jun Woong
[...]
> i.e. does the following invoke undefined behaviour:
>
> int main(void)
> {
> struct s { int c; } x,y;
> x=y; /* This cannot trap and x will be initialized to an unspecified
> value; but y is read while it is indeterminate. Can that have bad
> effects? */
>
> }
>
No. Strict reading of the standard guarantees it never fails. In
other words, the current wording of the standard says that there is
no trap representation allowed for struct or union types. To put it
using code:
{
T x;
x;
Reading x without initialization is allowed when T is unsigned char,
struct or union.
> Also what about the following:
>
> int main(void)
> {
> struct s { int c; } x, *y;
> y=malloc(sizeof *y); /* assume this succeeds */
> x=*y;
>
> }
>
It's okay, too. Though I am not sure whether this is indeed intended
or not.
> Appendix J (which is informative, not normative) says you cannot
> access the *bytes* of memory returned from malloc until they have been
> initialized (IIRC it actually only says bytes for the extra memory of
> realloc but modify the above to use realloc if you prefer)
>
Those conclusions above all can be driven from the *normative* text
of the standard.
> I don't know where in the normative standard this claim is derived
> from.
>
I've cited one before, which says about the representation of the
structure and union. For unsigned char, 6.2.6.2p1 says:
For unsigned integer types other than unsigned char, the bits of
the object representation shall be divided into two groups: value
bits and padding bits (there need not be any of the latter).
and footnote 44 says in part:
Some combinations of padding bits might generate trap
representations, ...
and 6.2.6.1p4 also says:
Values stored in non-bit-field objects of any other object type
consist of n * CHAR_BIT bits, where n is the size of an object of
that type, in bytes. The value may be copied into an object of type
unsigned char [n] (e.g., by memcpy); the resulting set of bytes is
called the object representation of the value.
You can also find a C90 DR which describes the representations of
integer types in a more straightfoward way.
> Personally, I feel this area of the standard is in need of a DR to
> clarify.
>
> There's this bit: (from c89 but there's similar wording in C99)
> All accessible objects have values as of the time longjmp was
> called, except that the values of objects of automatic storage
> duration that do not have volatile type and have been changed between
> the setjmp invocation and longjmp call are indeterminate.
>
> Baring a few operations that can set a value to indeterminate anyway
> regardless of the setjmp/longjmp, the value of these objects should
> surely be unspecified, not indeterminate.
>
It just means such an object (which has been changed between setjmp
and longjmp) should not be read.
> setvbuf:
> If buf is not a null pointer,
> the array it points to may be used instead of a buffer allocated by
> the setvbuf function./105/ The argument size specifies the size of the
> array. The contents of the array at any time are indeterminate.
>
> It's the "at any time" here - especially as buf is a char array.
>
The similar intent applied here. The wording cited ensures that it's
meaningless to try to read some useful data from the buffer. Note
that an implementation is not obliged to use the buffer an user
provided.
Jun Woong
[The standar allows no trap representation for struct and union]
>
> OK, but I find the requirement pointless. It just imposes a burden on
> implementations without any benefit to users. Worse, it requires that a
> struct can be passed by value even when the consequence of using the
> result may be undefined behaviour. It prohibits an implementation from
> doing anything to detect the problem before you actually access a member
> that traps.
>
Agreed.
I admit that it's useful in some cases to allow a struct to be
used initialized only in part, but the current requirement is too
loose to keep away from being abused.
Keith Thompson
> Francis Glassborow <francis.glassbo...@btinternet.com> wrote:
>
> [The standar allows no trap representation for struct and union]
>
>> OK, but I find the requirement pointless. It just imposes a burden on
>> implementations without any benefit to users. Worse, it requires that a
>> struct can be passed by value even when the consequence of using the
>> result may be undefined behaviour. It prohibits an implementation from
>> doing anything to detect the problem before you actually access a member
>> that traps.
>
> Agreed.
>
> I admit that it's useful in some cases to allow a struct to be
> used initialized only in part, but the current requirement is too
> loose to keep away from being abused.
So how would you tighten it up (without breaking existing code)?
For example, consider:
#define MAX some_number
struct vector {
size_t len;
double arr[MAX];
};
The idea here is that only elements 0 through len-1 are valid, but
there's no good way to explain that to the compiler.
The current rules make good sense for typical implementations that
implement structure copying by the equivalent of memcpy(), and aren't
going to try to check for trap representations anyway.
Tightening the rules could make things easier for more sophisticated
implementations that try to check for errors as early as possible, but
given examples like the above, such checks could break existing code.
And a *really* clever implementation that wants to do additional
checking could probably do most of it under the existing rules.
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jun Woong
> Jun Woong <wo...@icu.ac.kr> writes:
[...]
> > I admit that it's useful in some cases to allow a struct to be
> > used initialized only in part, but the current requirement is too
> > loose to keep away from being abused.
>
> So how would you tighten it up (without breaking existing code)?
>
> For example, consider:
>
> #define MAX some_number
> struct vector {
> size_t len;
> double arr[MAX];
> };
>
> The idea here is that only elements 0 through len-1 are valid,
Can the member len have the value 0? If it is, is it meaningful to
pass the struct whose every member has an indeterminate value to
other functions *as* a value?
> but
> there's no good way to explain that to the compiler.
>
I'm not claiming that the standard should have required every member
of a struct be properly initialized, but that it should have
prevented the value of a struct being used completely uninitizlied.
Is such a requirement so restrictive that it puts the blocks on any
useful and significant code?
> The current rules make good sense for typical implementations that
> implement structure copying by the equivalent of memcpy(), and aren't
> going to try to check for trap representations anyway.
>
> Tightening the rules could make things easier for more sophisticated
> implementations that try to check for errors as early as possible, but
> given examples like the above, such checks could break existing code.
>
I wonder how many significant codes use a construct like one given
above. Initializing an array of structs with a pre-initilized one or
passing a struct to other functions are only cases where a struct
value is used in a meaningful way, I think. Now, does it have any
meaning to initialize an array with a struct full of indeterminate
values? How about passing such a struct to other functions especially
when it contains a large object like an array?
> And a *really* clever implementation that wants to do additional
> checking could probably do most of it under the existing rules.
>
An warning is always permissible when a struct is used uninitizlied
whether it really results in UB or not, which seems to be enough for
error-cheking.
What I dislike to see is, to argue that the following code is
gracefully endorsed by the standard:
struct { member declarations
whose types are allowed
to have a trap representation } foo, bar;
foo = bar; /* no problem since the standard says so!!! */
Keith Thompson
> Keith Thompson <ks...@mib.org> wrote:
>> Jun Woong <wo...@icu.ac.kr> writes:
> [...]
>> > I admit that it's useful in some cases to allow a struct to be
>> > used initialized only in part, but the current requirement is too
>> > loose to keep away from being abused.
>>
>> So how would you tighten it up (without breaking existing code)?
>>
>> For example, consider:
>>
>> #define MAX some_number
>> struct vector {
>> size_t len;
>> double arr[MAX];
>> };
>>
>> The idea here is that only elements 0 through len-1 are valid,
>
> Can the member len have the value 0?
Sure, why not?
> If it is, is it meaningful to
> pass the struct whose every member has an indeterminate value to
> other functions *as* a value?
No, but you're missing the fact that len has a valid value of 0, and
that specifies the logical meaning of the entire struct. arr and its
elements don't have valid values, but any code that uses the struct
will never look at the values of the element of arr. (This is in
accordance with the program logic, which the compiler doesn't know
about and shouldn't try to second-guess.)
[...]
> What I dislike to see is, to argue that the following code is
> gracefully endorsed by the standard:
>
> struct { member declarations
> whose types are allowed
> to have a trap representation } foo, bar;
>
> foo = bar; /* no problem since the standard says so!!! */
I argue that if any one member of a struct has a meaningful value, we
need to assume that the struct as a whole could have a meaningful
value, and we need to be able to copy that value around (via
assignment, initialization, or parameter passing) without worrying
about run-time traps.
I wouldn't strongly object to a rule that says that if *all* the
members of a struct are uninitialized, then the struct as a whole
*might* have a trap representation, but that seems like a very narrow
special case that would complicate the language without a whole lot of
benefit.
Jun Woong
> >> struct vector {
> >> size_t len;
> >> double arr[MAX];
> >> };
> >>
> >> The idea here is that only elements 0 through len-1 are valid,
> >
> > Can the member len have the value 0?
>
> Sure, why not?
>
> > If it is, is it meaningful to
> > pass the struct whose every member has an indeterminate value to
> > other functions *as* a value?
>
> No, but you're missing the fact that len has a valid value of 0,
No, I didn't miss it. I asked if len can have a value of 0 since you
said "elements 0 through len-1," and I asked the second question to
say that there is no meaningful use of the struct when no member of
it has a valid value.
[...]
>
> > What I dislike to see is, to argue that the following code is
> > gracefully endorsed by the standard:
> >
> > struct { member declarations
> > whose types are allowed
> > to have a trap representation } foo, bar;
> >
> > foo = bar; /* no problem since the standard says so!!! */
>
> I argue that if any one member of a struct has a meaningful value, we
> need to assume that the struct as a whole could have a meaningful
> value, and we need to be able to copy that value around (via
> assignment, initialization, or parameter passing) without worrying
> about run-time traps.
This is exactly what I expected to see in the standard.
>
> I wouldn't strongly object to a rule that says that if *all* the
> members of a struct are uninitialized, then the struct as a whole
> *might* have a trap representation, but that seems like a very narrow
> special case that would complicate the language without a whole lot of
> benefit.
>
To someone, regulating such an erroneous construct seems to improve
the language rather than to complicate it, considering that the
standard already contribute more than one paragraph to specify the
validity of struct values.
goo...@woodall.me.uk
> Tim Woodall <devn...@woodall.me.uk> wrote:
> > On Mon, 26 May 2008 00:34:49 -0700 (PDT), Jun Woong <wo...@icu.ac.kr> wrote:
> [...]
>
> I don't see any guarantee from the standard that renders this
> implementation to be conforming.
Isn't that backwards. Doesn't an implementation that follows the
standard to the letter become conforming by definition. Therefore
there should be something in the standard that prohibits that
implementation.
> If I understand your example, on it
> the following (conforming) code must fail:
>
> int a; /* in an indeterminate state */
> unsigned char *uc = (void *)&a;
> *uc; /* read an unspecified value */
> a = 0; /* must fail! */
>
You're just begging the question here.
My underlying question is "is there any such thing as an indeterminate
unsigned char?" and the standard is ambiguous.
7.20.3.3 #2
The realloc function deallocates the old object pointed to by ptr and
returns a pointer to a new object that has the size specified by size.
The contents of the new object shall be the same as that of the old
object prior to deallocation, up to the lesser of the new and old
sizes. Any *bytes* in the new object beyond the size of the old object
have *indeterminate* values.
This is normative text.
3.2 alignment: requirement that objects of a particular type be
located on storage boundaries with addresses that are particular
multiples of a byte address.
Also in 3.6 it says Note 1 It is possible to express the address of
each individual byte of an object uniquely.
Given that unsigned char can be used to copy an object, then these two
imply unsigned char is equivalent to byte except for perverse things
like the least significant bit of a byte might not be the least
significant bit of the unsigned char (but there must be a 1-1 mapping
between the bits of an unsigned char and the bits of a byte.)
Infact I've just found 6.2.6.1 #4
Values stored in non-bit-field objects of any other object type
consist of n × CHAR_BIT bits, where n is the size of an object of that
type, in bytes. The value may be copied into an object of type
unsigned char [n] (e.g., by memcpy); the resulting set of bytes is
called the object representation of the value.
At the very least, I think 7.20.3.3 #2 needs changing to something
like.
... Any new objects beyond the size of the old objects have
indeterminate values.
or
... Any bytes in the new object beyond the size of the old object have
unspecified values.
That appendix J note appears to believe that 7.20.3.3 #2 implies that
accessing the *bytes* of memory beyond the old object causes undefined
behaviour.
As another example, consider a machine memory with a parity bit that
cannot be accessed from code but causes a hardware trap on read if
there's a parity error.
At power on all the bits are random (so there will be parity errors if
you try and read the memory)
The wording for the uninitialized struct (that the value of the struct
is not a trap representation) implies that on this machine the
compiler must ensure that all the parity bits are correct. But I
cannot see similar wording for the initialization of any other type,
including unsigned char.
I can see a potential problem here with malloced memory used to hold a
struct object. On this hypothetical machine the compiler would have to
ensure that, on casting the pointer to a struct pointer the parity
bits were correct. Which would imply that the following rather bizarre
code would be ok but could crash without the cast to (struct s*).
(assume the malloc succeeds)
void* p = malloc(sizeof(struct s));
(struct s*)p;
*(unsigned char*)p;
I'm not saying this is sane. I'm not even suggesting that this is what
the standard intended. But it appears to be something that the
DeathStation could do.
Tim.
Keith Thompson
> Keith Thompson <ks...@mib.org> wrote:
>> Jun Woong <wo...@icu.ac.kr> writes:
> [...]
>> >> #define MAX some_number
>> >> struct vector {
>> >> size_t len;
>> >> double arr[MAX];
>> >> };
>> >>
>> >> The idea here is that only elements 0 through len-1 are valid,
>> >
>> > Can the member len have the value 0?
>>
>> Sure, why not?
>>
>> > If it is, is it meaningful to
>> > pass the struct whose every member has an indeterminate value to
>> > other functions *as* a value?
>>
>> No, but you're missing the fact that len has a valid value of 0,
>
> No, I didn't miss it. I asked if len can have a value of 0 since you
> said "elements 0 through len-1," and I asked the second question to
> say that there is no meaningful use of the struct when no member of
> it has a valid value.
I though your remark about "every member has an indeterminate value"
was meant to apply to len == 0. Re-reading the paragraph, it still
looks that way, but if that's not what you meant, then I agree.
[...]
Harald van Dijk
> On May 27, 12:38 am, Jun Woong <wo...@icu.ac.kr> wrote:
>> Tim Woodall <devn...@woodall.me.uk> wrote:
>> > On Mon, 26 May 2008 00:34:49 -0700 (PDT), Jun Woong <wo...@icu.ac.kr>
>> > wrote:
>> [...]
>
>
>> I don't see any guarantee from the standard that renders this
>> implementation to be conforming.
>
> Isn't that backwards. Doesn't an implementation that follows the
> standard to the letter become conforming by definition. Therefore there
> should be something in the standard that prohibits that implementation.
A conforming implementation must accept correct programs, so if the
standard doesn't say that a program is incorrect for whatever reason, and
an implementation doesn't accept it, it's the implementation that's
nonconforming, not the program.
>> If I understand your example, on it
>> the following (conforming) code must fail:
>>
>> int a; /* in an indeterminate state */ unsigned char *uc = (void
>> *)&a;
>> *uc; /* read an unspecified value */ a = 0; /* must fail! */
>>
>>
> You're just begging the question here.
>
> My underlying question is "is there any such thing as an indeterminate
> unsigned char?" and the standard is ambiguous.
There is such a thing as an indeterminate unsigned char. It may hold any
value, it may even change between two consecutive reads, but it may not be
a trap representation. An indeterminate value is defined as "either an
unspecified value or a trap representation". The fact that unsigned char
cannot have trap representations, means that an indeterminate value of
unsigned char is nothing less or more than an unspecified value of
unsigned char. It doesn't mean that indeterminate unsigned chars cannot
exist.
> [...]
> At the very least, I think 7.20.3.3 #2 needs changing to something like.
>
> ... Any new objects beyond the size of the old objects have
> indeterminate values.
>
> or
>
> ... Any bytes in the new object beyond the size of the old object have
> unspecified values.
7.20.3.3 describes malloc. 7.20.3.4 describes ralloc. That said, given the
above, your wording means exactly the same as the current wording.
> That appendix J note appears to believe that 7.20.3.3 #2 implies that
> accessing the *bytes* of memory beyond the old object causes undefined
> behaviour.
That appendix note has probably remained since C90, where it would have
been correct. In C99, it's no longer correct, but it's not normative
anyway.
> As another example, consider a machine memory with a parity bit that
> cannot be accessed from code but causes a hardware trap on read if
> there's a parity error.
>
> At power on all the bits are random (so there will be parity errors if
> you try and read the memory)
>
> The wording for the uninitialized struct (that the value of the struct
> is not a trap representation) implies that on this machine the compiler
> must ensure that all the parity bits are correct. But I cannot see
> similar wording for the initialization of any other type, including
> unsigned char.
The only possibility for undefined behaviour on reading indeterminate
values is given in 6.2.6.1p5 (trap representations). Since unsigned char
cannot have trap representations, the standard doesn't say anywhere that a
program isn't allowed to read them, which means that a program that does
so can be a correct program, which any conforming implementation must
accept.
goo...@woodall.me.uk
> There is such a thing as an indeterminate unsigned char. It may hold any
> value, it may even change between two consecutive reads, but it may not be
> a trap representation. An indeterminate value is defined as "either an
> unspecified value or a trap representation". The fact that unsigned char
> cannot have trap representations, means that an indeterminate value of
> unsigned char is nothing less or more than an unspecified value of
> unsigned char. It doesn't mean that indeterminate unsigned chars cannot
> exist.
>
6.2.6.1 #6 has
When a value is stored in an object of structure or union type,
including in a member object, the bytes of the object representation
that correspond to any padding bytes take unspecified values.
Likewise 6.2.6.1 #7 has the same unspecified byte values.
I don't think the standard should be using unspecified unsigned char
and indeterminate unsigned char interchangeably and still include that
note in appendix J.
(I'm looking at n1124.pdf here so it might have changed)
The general concensus does seem to be that a unspecified unsigned char
and an indeterminate unsigned char are the same thing.
So the only thing that is actually wrong with the standard is the bit
in appendix J - which might be only a note but exactly reflects how I
thought the wording of the standard ought to be interpreted.
> The only possibility for undefined behaviour on reading indeterminate
> values is given in 6.2.6.1p5 (trap representations). Since unsigned char
> cannot have trap representations, the standard doesn't say anywhere that a
> program isn't allowed to read them, which means that a program that does
> so can be a correct program, which any conforming implementation must
> accept.
But can bytes have trap representations even though unsigned char
cannot?
Oh well. I'm probably just going to have to accept the fact that the
standard is (unnecessarily) sloppily worded. Rather like these bits
that I believe have been the subject of a rejected DR:
The three types char, signed char, and unsigned char are collectively
called
the character types. The implementation shall define char to have the
same range,
representation, and behavior as either signed char or unsigned char
If the stored
value of an object has such a representation and is read by an lvalue
expression that does
not have character type, the behavior is undefined.
Yes. I know this doesn't actually say that the object representation
can be read by character type but why not say "... does not have the
behavior of unsigned char type" or even better, get rid of the not.
The same reasoning that leads to this statement being correct for
signed char also leads to believing that bytes can have a trap
representation when read by unsigned char.
Tim.
Jun Woong
[...]
>
> I though your remark about "every member has an indeterminate value"
> was meant to apply to len == 0. Re-reading the paragraph, it still
> looks that way, but if that's not what you meant, then I agree.
>
Yes, after re-reading I admit that it's very natural to read it that
way. I was hectic at work in writing it. It was my fault, sorry.
What I tried to say, however, was to ask two questions which are
separate.
Jun Woong
> On May 27, 5:58 pm, Harald van Dijk <true...@gmail.com> wrote:> There is such a thing as an indeterminate unsigned char. It may hold any
> > value, it may even change between two consecutive reads, but it may not be
> > a trap representation. An indeterminate value is defined as "either an
> > unspecified value or a trap representation". The fact that unsigned char
> > cannot have trap representations, means that an indeterminate value of
> > unsigned char is nothing less or more than an unspecified value of
> > unsigned char. It doesn't mean that indeterminate unsigned chars cannot
> > exist.
>
> That still makes the standard sloppy.
>
> 6.2.6.1 #6 has
> When a value is stored in an object of structure or union type,
> including in a member object, the bytes of the object representation
> that correspond to any padding bytes take unspecified values.
>
> Likewise 6.2.6.1 #7 has the same unspecified byte values.
>
I guess the term "unspeicified" was choosen to emphasize that
paddings between members never affect the validity of the struct or
union that contains them.
> I don't think the standard should be using unspecified unsigned char
> and indeterminate unsigned char interchangeably and still include that
> note in appendix J.
>
I agree that the inaccurate wording given in appendix J should be
dropped or revised in the next revision of the standard, but I don't
think that the "sloppy" expressions, byte with an unspecified value
and byte with an indeterminate value, really cause a problem.
> (I'm looking at n1124.pdf here so it might have changed)
>
> The general concensus does seem to be that a unspecified unsigned char
> and an indeterminate unsigned char are the same thing.
>
Becuase an indeterminate value can cause a problem (i.e., UB) only
when there is a trap representation for a type used to interpret the
value, and unsigned char does not have such a representation, those
two in fact indicate the same thing. But I don't think the committee
were careful in using them and they need to do.
> So the only thing that is actually wrong with the standard is the bit
> in appendix J - which might be only a note but exactly reflects how I
> thought the wording of the standard ought to be interpreted.
>
Considering there were some DRs for correcting C90's informative
appendices, I think you can file a DR for it.
> > The only possibility for undefined behaviour on reading indeterminate
> > values is given in 6.2.6.1p5 (trap representations). Since unsigned char
> > cannot have trap representations, the standard doesn't say anywhere that a
> > program isn't allowed to read them, which means that a program that does
> > so can be a correct program, which any conforming implementation must
> > accept.
>
> But can bytes have trap representations even though unsigned char
> cannot?
>
What really matters here is the type used to interpret the bits
stored in a byte. unsigned char cannot have a trap representation
while signed char can.
> Oh well. I'm probably just going to have to accept the fact that the
> standard is (unnecessarily) sloppily worded. Rather like these bits
> that I believe have been the subject of a rejected DR:
>
> The three types char, signed char, and unsigned char are collectively
> called
> the character types. The implementation shall define char to have the
> same range,
> representation, and behavior as either signed char or unsigned char
>
> If the stored
> value of an object has such a representation and is read by an lvalue
> expression that does
> not have character type, the behavior is undefined.
>
> Yes. I know this doesn't actually say that the object representation
> can be read by character type but why not say "... does not have the
> behavior of unsigned char type" or even better, get rid of the not.
>
This has been pointed out several times, and I absolutely agree with
you that the wording in question needs to be fixed.
> The same reasoning that leads to this statement being correct for
> signed char also leads to believing that bytes can have a trap
> representation when read by unsigned char.
>
In C90, all character types are guaranteed not to have a trap
representation; you can find this in the committee's answer to a
relevant DR. I guess the wording you quoted above came from an
intermediate stage of revision of C90 to C99.
lawrenc...@siemens.com
>
> I'm surprised, however, to see the last sentence quoted above. That
> seems to say that an implementation is prohibited from incurring
> undefined behavior even when the member of a struct being assigned
> has a trap representation and it does the member-to-member
> assignment. Is this really intended, or does the sentence in question
> have to read as
>
> The value of a structure or union object is not allowed to have a
> trap representation unless one of its members has a trap
> representation.
It is really intended -- structs and unions do not have trap
representations. See DR #222, which is what motivated that sentence.
-- Larry Jones
When you're SERIOUS about having fun, it's not much fun at all! -- Calvin
Jun Woong
> Jun Woong <wo...@icu.ac.kr> wrote:
>
> > I'm surprised, however, to see the last sentence quoted above. That
> > seems to say that an implementation is prohibited from incurring
> > undefined behavior even when the member of a struct being assigned
> > has a trap representation and it does the member-to-member
> > assignment. Is this really intended, or does the sentence in question
> > have to read as
> >
> > The value of a structure or union object is not allowed to have a
> > trap representation unless one of its members has a trap
> > representation.
>
> It is really intended -- structs and unions do not have trap
> representations. See DR #222, which is what motivated that sentence.
>
Thanks for the reference. It was why I felt unfamiliar with the
wording that it was added by TC2!
I agree that such a wording has to be there given many existing
constructs using partially-initialized structures, even if I tried to
fully initialize them even when only used in part.
Now what I wonder is, if the committee really intended to allow an
erroneous code like
struct {
/* members whose types allow a trap representation */
} foo, bar;
foo = bar;
to be strictly conforming.
I think small touch could improve that new wording to preclude this
anomaly.
goo...@woodall.me.uk
I don't think this matters if we accept that unsigned char cannot trap
in any circumstances.
All the standard does is require that structure members be copied by
something analogous to memcpy rather than member assignment.
But it's fairly obvious that it has to work like this anyway.
union u {
double d;
unsigned char c[sizeof(double)];
};
void func(void) {
union u x,y;
memcpy(&y.c, &double_trap_representation, sizeof y.c);
union_assignment_in_different_compilation_unit(&x, &y);
}
union_assignment_in_different_compilation_unit(&x, &y) {
*x = *y;
}
Unless the union is going to be required to carry additional bits
around with it signaling the valid members then I don't think there's
any way the compiler can be aware which member(s) to use to do the
assignment.
Note that my earlier impression of what the standard was saying with
regard to uninitialized bytes is not incompatible with this.
Tim.
Jun Woong
> On May 30, 7:24 am, Jun Woong <wo...@icu.ac.kr> wrote:
> > Thanks for the reference. It was why I felt unfamiliar with the
> > wording that it was added by TC2!
> >
> > I agree that such a wording has to be there given many existing
> > constructs using partially-initialized structures, even if I tried to
> > fully initialize them even when only used in part.
> >
> > Now what I wonder is, if the committee really intended to allow an
> > erroneous code like
> >
> > struct {
> > /* members whose types allow a trap representation */
> > } foo, bar;
> >
> > foo = bar;
> >
> > to be strictly conforming.
> >
> > I think small touch could improve that new wording to preclude this
> > anomaly.
>
> I don't think this matters if we accept that unsigned char cannot trap
> in any circumstances.
>
I was not talking about its implementability. That wording added by
TC2 is really necessary, but it covers too much endorsing obviously
incorrect codes.
> All the standard does is require that structure members be copied by
> something analogous to memcpy rather than member assignment.
>
Only on an implementation where a trap representation exists and
disabling a trap representation from being trapped is not possible.
> But it's fairly obvious that it has to work like this anyway.
>
[example snipped]
>
> Unless the union is going to be required to carry additional bits
> around with it signaling the valid members then I don't think there's
> any way the compiler can be aware which member(s) to use to do the
> assignment.
>
Agreed.
Douglas A. Gwyn
means other than aliasing to array of type unsigned char) merely
results in undefined behavior, which excuses a conforming
implementation from having to do something specific. Although
one form of undefined behavior can be that a trap is generated,
that is not necessary even for accessing a so-called "trap
representation". More usually, unwanted sign extension or
something of the sort is likely to occur.
Jun Woong
> Please keep in mind that accessing a "trap representation" (by
> means other than aliasing to array of type unsigned char) merely
> results in undefined behavior, which excuses a conforming
> implementation from having to do something specific.
The new wording introduced by TC3 increased the number of cases where
accessing a trap representation was not allowed to result in
undefined behavior.