memset undefined behavior
Douglas A. Gwyn
> an unsigned char can have padding bits too.
No, it can't.
Irrwahn Grausewitz
> struct foo {
> int a, b;
> char c;
> long d;
> };
>
> struct foo bar;
>
> memset(&bar, 0, sizeof bar);
>
> This would invoked undefine behavior because it sets all bits of the object
> to 0. A struct can have padding bits, so it might alter those.
AFAICT setting the padding bytes(!) of a struct to zero shouldn't do
any harm:
ISO/IEC 9899:TC1 6.2.6.1p6
[...] The values of padding bytes shall not affect whether the value
of such an object is a trap representation. Those bits of a structure
or union object that are in the same byte as a bit-field member, but
are not part of that member, shall similarly not affect whether the
value of such an object is a trap representation.
> int a;
>
> memset(&a, 0, sizeof a);
>
> likewise, this sets all bits of object 'a' to zero. 'a' has a sign bit and
> padding bits. So when those are altered it invokes indefined behavior.
Right, but /extremely/ unlikely to happen with *any* integer type on
*any* architecture in actual use today (as I recently learned from a
post by Jack Klein). The problematic cases are with pointer and
floating point objects.
> now my question,
>
> unsigned char a;
>
> memset(&a, 0, sizeof a);
>
> an unsigned char can have padding bits too.
No.
ISO/IEC 9899:TC1 6.2.6.2#1: [emphasis added]
For unsigned integer types /other than unsigned char/, the bits of
the object representation shall be divided into two groups: value
bits and padding bits [...]
> so if these are altered by having
> all bits set to zero, will this invoked undefined behavior?
Mu. (unasking your question)
HTH
Regards
--
Irrwahn
(irrw...@freenet.de)
Fergus Henderson
>struct foo {
> int a, b;
> char c;
> long d;
>};
>
>struct foo bar;
>
>memset(&bar, 0, sizeof bar);
>
>This would invoked undefine behavior because it sets all bits of the object
>to 0. A struct can have padding bits, so it might alter those.
>
>int a;
>
>memset(&a, 0, sizeof a);
>
>likewise, this sets all bits of object 'a' to zero. 'a' has a sign bit and
>padding bits. So when those are altered it invokes indefined behavior.
Merely creating a trap representation with memset() doesn't invoke
undefined behaviour, does it? It's only if you create or read a trap
representation using an lvalue expression that doesn't have character type
that you get undefined behaviour. In the case of your calls to memset(),
I don't see any lvalue expression whose value is a trap representation.
--
Fergus Henderson <f...@cs.mu.oz.au> | "I have always known that the pursuit
The University of Melbourne | of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Douglas A. Gwyn
> Merely creating a trap representation with memset() doesn't invoke
> undefined behaviour, does it? It's only if you create or read a trap
> representation using an lvalue expression that doesn't have character type
> that you get undefined behaviour. In the case of your calls to memset(),
> I don't see any lvalue expression whose value is a trap representation.
Right. There is an issue of whether using memset to fill
an object declared as having type int with all 0-valued
bytes produces a valid int representation of 0. This was
discussed a few WG14 meetings ago, but I don't recall the
outcome.
Christian Bau
I think the C Standard should add something along the lines of "for all
integer types, all bits zero is a valid representation of the value 0".
Then you _might_ think about whether the same could be done for floating
point types and pointer types. If there are no C implementations in use
at the moment where all bits zero is not a valid representation of a
float or double 0.0, or of a null pointer, then this might be added and
increase the value of calloc and memset significantly.
Mantorok Redgormor
> Mantorok Redgormor wrote:
> > an unsigned char can have padding bits too.
>
> No, it can't.
sorry, I confused unsigned int with unsigned char. unsigned int is the one
that can have padding bits. My apologies.
--
nethlek
Sheldon Simms
> Fergus Henderson wrote:
>> Merely creating a trap representation with memset() doesn't invoke
>> undefined behaviour, does it? It's only if you create or read a trap
>> representation using an lvalue expression that doesn't have character type
>> that you get undefined behaviour. In the case of your calls to memset(),
>> I don't see any lvalue expression whose value is a trap representation.
>
> Right. There is an issue of whether using memset to fill
> an object declared as having type int with all 0-valued
> bytes produces a valid int representation of 0.
It seems pretty clear to me that all (including padding) bits zero in
an int could well be a trap representation. I don't see how anyone
could argue otherwise.
In fact, it's not clear to me that what would otherwise be normal
integer values couldn't in fact be trap representations. 6.2.6.2
explicitly allows what would otherwise be the twos-complement value
-(INT_MAX+1) to be a trap representation. Why not other values as
well?
-Sheldon
Jeremy Yallop
DR 263 (closed, 2002-04-18):
Suggested Technical Corrigendum
Append to 6.2.6.2#5:
For any integer type, the object representation where all the bits
are zero shall be a representation of the value zero in that type.
http://std.dkuug.dk/JTC1/SC22/WG14/www/docs/dr_263.htm
Jeremy.
Irrwahn Grausewitz
> In article <JNidnYVW-dq...@comcast.com>,
> "Douglas A. Gwyn" <DAG...@null.net> wrote:
<snip>
> > Right. There is an issue of whether using memset to fill
> > an object declared as having type int with all 0-valued
> > bytes produces a valid int representation of 0. This was
> > discussed a few WG14 meetings ago, but I don't recall the
> > outcome.
>
> I think the C Standard should add something along the lines of "for all
> integer types, all bits zero is a valid representation of the value 0".
See DR#263.
Regards
--
Irrwahn
(irrw...@freenet.de)
Irrwahn Grausewitz
> Irrwahn Grausewitz <irrw...@freenet.de> wrote:
<snip>
> > AFAICT setting the padding bytes(!) of a struct to zero shouldn't do
> > any harm:
> >
> > ISO/IEC 9899:TC1 6.2.6.1p6
> > [...] The values of padding bytes shall not affect whether the value
> > of such an object is a trap representation. Those bits of a structure
> > or union object that are in the same byte as a bit-field member, but
> > are not part of that member, shall similarly not affect whether the
> > value of such an object is a trap representation.
>
> you mention padding bytes but doesn't memset set bits not bytes?
memset writes entities of unsigned char (== bytes); /structs/ cannot
have padding bits, only padding bytes; /struct members/ may well have
padding bits.
> but other than that, I take it that memsetting structs is perfectly okay?
As long as you don't try to access a member of this 'zeroed' struct
that is of pointer type or floating point type: yes.
> however, memsetting any pointer type is not perfectly okay?
Yes, and the same with floating point types.
> so,
>
> struct foo *bar;
>
> memset(&bar, 0, sizeof *bar);
ITYM: memset( &bar, 0, sizeof bar );
> or
>
> int *a;
>
> memset(&a, 0, sizeof *a);
ITYM: memset( &a, 0, sizeof a );
> will create trap representations for pointers, regardless of pointer type.
They /may/ create trap representations on some architectures.
> though,
>
> struct foo bar;
>
> memset(&bar, 0, sizeof bar);
>
> can have all bits zero and always be a legit representation.
... with the above mentioned restrictions regarding pointers and f.p.
types, yes.
<snip>
Dan Pop
>On Thu, 13 Nov 2003 03:21:41 -0500, Douglas A. Gwyn wrote:
>
>> Fergus Henderson wrote:
>>> Merely creating a trap representation with memset() doesn't invoke
>>> undefined behaviour, does it? It's only if you create or read a trap
>>> representation using an lvalue expression that doesn't have character type
>>> that you get undefined behaviour. In the case of your calls to memset(),
>>> I don't see any lvalue expression whose value is a trap representation.
>>
>> Right. There is an issue of whether using memset to fill
>> an object declared as having type int with all 0-valued
>> bytes produces a valid int representation of 0.
>
>It seems pretty clear to me that all (including padding) bits zero in
>an int could well be a trap representation. I don't see how anyone
>could argue otherwise.
Well, DR 263 (http://anubis.dkuug.dk/JTC1/SC22/WG14/www/docs/dr_263.htm)
does argue otherwise:
Suggested Technical Corrigendum
Append to 6.2.6.2#5:
For any integer type, the object representation where all the bits
are zero shall be a representation of the value zero in that type.
Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Dan...@ifh.de
Dan Pop
>struct foo {
> int a, b;
> char c;
> long d;
>};
>
>struct foo bar;
>
>memset(&bar, 0, sizeof bar);
>
>This would invoked undefine behavior because it sets all bits of the object
>to 0. A struct can have padding bits, so it might alter those.
Altering them is harmless.
>int a;
>
>memset(&a, 0, sizeof a);
>
>likewise, this sets all bits of object 'a' to zero. 'a' has a sign bit and
>padding bits. So when those are altered it invokes indefined behavior.
Nope. There is nothing wrong with setting the sign bit to zero, and
according to a Defect Report (mentioned elsethread) all bits zero is
supposed to be a valid representation of the value zero in any integer
type.
>now my question,
>
>unsigned char a;
>
>memset(&a, 0, sizeof a);
>
>an unsigned char can have padding bits too.
Nope, it is the *only* type that cannot have padding bits.
>so if these are altered by having
>all bits set to zero, will this invoked undefined behavior?
The most important point you're missing is that *generating* a trap
representation doesn't invoke undefined behaviour. An uninitialised
local automatic object may also contain a trap representation.
Undefined behaviour happens when the value of an object containing a trap
representation is used.
So:
double *p;
memset(&p, 0, sizeof p);
Everything is fine until now. The value of p is indeterminate and it may
be a trap representation.
if (p == NULL) ...
Undefined behaviour, if p happens to contain a trap representation.
Dan Pop
>memset writes entities of unsigned char (== bytes); /structs/ cannot
>have padding bits, only padding bytes; /struct members/ may well have
>padding bits.
Are you sure? Then how would you call the unused *bits* in the following
struct:
struct foo { unsigned int bit:1; };
? They're certainly not part of the struct member and there MUST be
CHAR_BIT - 1 of them that are not included in any padding byte.
Sheldon Simms
Where is the argument? That looks more like a proposed change in
the standard to me.
-Sheldon
Hallvard B Furuseth
The argument just above the text quoted in the DR, and what it argues is
exactly the opposite of what Dan says: Without the above change,
all-bits-zero could be a trap representation - but it shouldn't be.
--
Hallvard
Douglas A. Gwyn
> See DR#263.
Douglas A. Gwyn
> Where is the argument? That looks more like a proposed change in
> the standard to me.
Actually it's an approved change, expected to be bundled up in TC#2
in the not too distant future. Essentially this requires a property
that has always been true of all implementations we know of, and
which was taken for granted before C99's introduction of the notion
of "trap representation" (which is a technical issue determining
one way in which a program can not be strictly conforming, not
necessarily anything in the actual architecture.)
Christian Bau
The C Standard must always be a compromise - the more it guarantees, the
more useful it is for the programmer and the harder to implement. If it
guarantees less, then it is less useful for the programmer and easier to
implement.
Does anyone know an of an implementation where all bits zero is a trap
representation? How do you think customers would react if a computer
manufacturer come up with a new chip where all bits zero is a trap
representation? They would tell them to shove it.
So to me it seems that the cost/benefit ratio of demanding that all bits
zero is not a trap representation seems absolutely excellent.
James Kuyper
How can one tell which DR's have been approved? I looked at the web link
that was provided, hoping to see something that said "Status: approved",
or something like that. I didn't find it.
Dan Pop
http://anubis.dkuug.dk/JTC1/SC22/WG14/www/docs/summary.htm lists the
status of each DR. "Closed" presumably means that the final committee
decision can be found by following the associated link.
Irrwahn Grausewitz
> In <3FB5057B...@saicmodis.com> James Kuyper <kuy...@saicmodis.com> writes:
<snip>
> >How can one tell which DR's have been approved? I looked at the web link
> >that was provided, hoping to see something that said "Status: approved",
> >or something like that. I didn't find it.
>
> http://anubis.dkuug.dk/JTC1/SC22/WG14/www/docs/summary.htm lists the
> status of each DR. "Closed" presumably means that the final committee
> decision can be found by following the associated link.
Which reveals what seems to be a cut & paste error in the very last
section of DR263. Shouldn't it read "Proposed T.C." instead of
"Suggested T.C.", if it is closed and approved?
Regards
--
Irrwahn
(irrw...@freenet.de)
James Kuyper
> >How can one tell which DR's have been approved? I looked at the web link
> >that was provided, hoping to see something that said "Status: approved",
> >or something like that. I didn't find it.
>
> http://anubis.dkuug.dk/JTC1/SC22/WG14/www/docs/summary.htm lists the
> status of each DR. "Closed" presumably means that the final committee
> decision can be found by following the associated link.
That presumption is easily checked, and turns out to be wrong. My
question still stands.
lawrenc...@eds.com
>
> Which reveals what seems to be a cut & paste error in the very last
> section of DR263. Shouldn't it read "Proposed T.C." instead of
> "Suggested T.C.", if it is closed and approved?
Yes, it should. I've notified the authorities. :-)
-Larry Jones
What's the matter? Don't you trust your own kid?! -- Calvin
lawrenc...@eds.com
>
> Dan Pop wrote:
>>
>> http://anubis.dkuug.dk/JTC1/SC22/WG14/www/docs/summary.htm lists the
>> status of each DR. "Closed" presumably means that the final committee
>> decision can be found by following the associated link.
>
> That presumption is easily checked, and turns out to be wrong. My
> question still stands.
The presumption is correct -- what exactly lead you to conclude
otherwise?
The various statuses are:
Open - The committee has not yet reached concensus on a response. There
may be notes summarizing committee discussions, particularly if the
issue is a tricky one. The issue is automatically on the agenda for the
next meeting.
Review - The committee appears to have reached concensus on a response
and the final wording is being reviewed. These usually change to
"Closed" after the next meeting, but may revert to "Open" if people
disagree with the substance of the response.
Closed - The committee has reached concensus and approved the final
wording, but the response has not yet been published, so any changes to
the standard are not yet officially in effect. Although no longer under
active consideration, the issue can be reopened and the response changed
if there is good reason to do so.
Closed, Published - The response is official and can no longer be
changed. Any changes to the standard are officially in effect.
-Larry Jones
Hello, I'm wondering if you sell kegs of dynamite. -- Calvin
James Kuyper
>
> James Kuyper <kuy...@saicmodis.com> wrote:
> >
> > Dan Pop wrote:
> >>
> >> http://anubis.dkuug.dk/JTC1/SC22/WG14/www/docs/summary.htm lists the
> >> status of each DR. "Closed" presumably means that the final committee
> >> decision can be found by following the associated link.
> >
> > That presumption is easily checked, and turns out to be wrong. My
> > question still stands.
>
> The presumption is correct -- what exactly lead you to conclude
> otherwise?
The fact that I saw nothing at that location which was identified as
"the final committee decision". I saw a "Problem" section, and
"Suggested Technical Corrigendum" section. I saw nothing to indicate
whether or not the committee agreed that there was in fact a problem,
nor anything indicating whether the proposed solution had been accepted,
nor whether it was accepted as-is or with modifications.
> The various statuses are:
>
> Open - The committee has not yet reached concensus on a response. There
> may be notes summarizing committee discussions, particularly if the
> issue is a tricky one. The issue is automatically on the agenda for the
> next meeting.
>
> Review - The committee appears to have reached concensus on a response
> and the final wording is being reviewed. These usually change to
> "Closed" after the next meeting, but may revert to "Open" if people
> disagree with the substance of the response.
>
> Closed - The committee has reached concensus and approved the final
> wording, but the response has not yet been published, so any changes to
> the standard are not yet officially in effect. Although no longer under
> active consideration, the issue can be reopened and the response changed
> if there is good reason to do so.
>
> Closed, Published - The response is official and can no longer be
> changed. Any changes to the standard are officially in effect.
You don't list seperate "Accepted" or "Rejected" categories, which I
have presumed means that an issue is marked as Closed whenever the
committee reaches a consensus, whether or not that consensus is
favorable. Is that not correct?
lawrenc...@eds.com
>
> The fact that I saw nothing at that location which was identified as
> "the final committee decision". I saw a "Problem" section, and
> "Suggested Technical Corrigendum" section. I saw nothing to indicate
> whether or not the committee agreed that there was in fact a problem,
> nor anything indicating whether the proposed solution had been accepted,
> nor whether it was accepted as-is or with modifications.
Any DR in a Review or Closed state should have a "Committee Response"
and/or a "Proposed Technical Corrigendum" section at the end (after a
horizontal rule) which indicates the committee's decision. If that is
not the case, then an error has occurred -- either the committee's
response is mislabeled or the status is incorrect. Which DR were you
looking at?
> You don't list seperate "Accepted" or "Rejected" categories, which I
> have presumed means that an issue is marked as Closed whenever the
> committee reaches a consensus, whether or not that consensus is
> favorable. Is that not correct?
That is correct. The Committee Response indicates whether the suggested
changes were accepted or not.
-Larry Jones
Summer vacation started! I can't be sick! -- Calvin
James Kuyper
> James Kuyper <kuy...@saicmodis.com> wrote:
> >
> > The fact that I saw nothing at that location which was identified as
> > "the final committee decision". I saw a "Problem" section, and
> > "Suggested Technical Corrigendum" section. I saw nothing to indicate
> > whether or not the committee agreed that there was in fact a problem,
> > nor anything indicating whether the proposed solution had been accepted,
> > nor whether it was accepted as-is or with modifications.
>
> Any DR in a Review or Closed state should have a "Committee Response"
> and/or a "Proposed Technical Corrigendum" section at the end (after a
> horizontal rule) which indicates the committee's decision. If that is
> not the case, then an error has occurred -- either the committee's
> response is mislabeled or the status is incorrect. Which DR were you
> looking at?
The one currently under discussion: #263. It has neither of those
sections.
Even if it did have a "Proposed Technical Corrigendum" section, that
title alone wouldn't have clued me in to the fact that this proposal
had been accepted. Perhaps different wording should be used? A note
saying something like "Approved for inclusion in TC2" would clarify
the matter.
I checked several Closed DRs adjacent to 263, most of which contained
a "Proposed Committee Response". However, two of the ones I looked at
deserve special comment.
For DR 259 the "Proposed Committee Response" says "The standard is
clear enough, and no change is needed." without bothering to indicate
which of the two possibilities is the one that the standard is "clear
enough" about.
For DR 271 "Proposed Committee Response" is followed by a "Committee
Discussion" section which says that "The Committee believes this
should be considered for a future revision of the Standard." That at
least clearly indicates the approval of the committee, but still
leaves open the question "Has it been considered yet, and if so, with
what result, for which revision?"
lawrenc...@eds.com
>
> The one currently under discussion: #263. It has neither of those
> sections.
Yes, the "Suggested Technical Corrigendum" below the horizontal line
should be "Proposed Technical Corrigendum" (an obvious case of cut and
paste rather than cut, paste, and edit).
> Even if it did have a "Proposed Technical Corrigendum" section, that
> title alone wouldn't have clued me in to the fact that this proposal
> had been accepted. Perhaps different wording should be used? A note
> saying something like "Approved for inclusion in TC2" would clarify
> the matter.
The terminology is confusing, but the scheme is that "Suggestions" are
made by the submitter, "Proposals" are made by committee members (or the
committee as a whole). Approval for inclusion in a published TC isn't
done until the decision is made to publish, so any such notice would be
premature. As I said, unpublished closed DRs can be reopened and
revised, although it doesn't happen very often.
> For DR 259 the "Proposed Committee Response" says "The standard is
> clear enough, and no change is needed." without bothering to indicate
> which of the two possibilities is the one that the standard is "clear
> enough" about.
Neither; the committee rejects the dichotomy entirely. The committee
believes that the standard is clear enough that empty parentheses in a
macro invocation can represent either no arguments or one empty
argument, depending on the definition of the macro (which is what the
Suggested Technical Corrigendum says).
> For DR 271 "Proposed Committee Response" is followed by a "Committee
> Discussion" section which says that "The Committee believes this
> should be considered for a future revision of the Standard." That at
> least clearly indicates the approval of the committee, but still
> leaves open the question "Has it been considered yet, and if so, with
> what result, for which revision?"
All of the DRs apply to C99. The Committee Response indicates that it
was considered and rejected for C99 ("not ... appropriate to add new
requirements"). The Committee Discussion is a "note to self" for the
committee to consider making changes along those lines in some future
(major) revision of the Standard.
-Larry Jones
Wow, how existential can you get? -- Hobbes
James Kuyper
...
> The terminology is confusing, but the scheme is that "Suggestions" are
> made by the submitter, "Proposals" are made by committee members (or the
> committee as a whole). Approval for inclusion in a published TC isn't
> done until the decision is made to publish, so any such notice would be
> premature. As I said, unpublished closed DRs can be reopened and
> revised, although it doesn't happen very often.
Doug indicated that the proposed TC for DR 263 had already been approved
for inclusion in TC2. If it has, then such a notation would not be
premature.
Fergus Henderson
>James Kuyper <kuy...@wizard.net> wrote:
>>
>> For DR 259 the "Proposed Committee Response" says "The standard is
>> clear enough, and no change is needed." without bothering to indicate
>> which of the two possibilities is the one that the standard is "clear
>> enough" about.
>
>Neither; the committee rejects the dichotomy entirely. The committee
>believes that the standard is clear enough that empty parentheses in a
>macro invocation can represent either no arguments or one empty
>argument, depending on the definition of the macro (which is what the
>Suggested Technical Corrigendum says).
It would IMHO help a LOT if the official record of responses in cases like
this one included something like what you just said above.
--
Fergus Henderson <f...@cs.mu.oz.au> | "I have always known that the pursuit
The University of Melbourne | of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.
Douglas A. Gwyn
> Doug indicated that the proposed TC for DR 263 had already been approved
> for inclusion in TC2. If it has, then such a notation would not be
> premature.
That is a blanket principle for all "closed" DRs that contain
"proposed Technical Corrigenda" in the Committee Response.
Perhaps the status terminology could be better explained in
the parent web page.
lawrenc...@eds.com
>
> Doug indicated that the proposed TC for DR 263 had already been approved
> for inclusion in TC2. If it has, then such a notation would not be
> premature.
Doug's statement was also premature. There is a general principle that
all closed DRs are included in the next TC, but there has to be an
official vote to do so. And, as I've said before, closed DRs can be
reopened.
-Larry Jones
Physical education is what you learn from having your face in
someone's armpit right before lunch. -- Calvin
lawrenc...@eds.com
>
> It would IMHO help a LOT if the official record of responses in cases like
> this one included something like what you just said above.
Noted.
-Larry Jones
ANY idiot can be famous. I figure I'm more the LEGENDARY type! -- Calvin