GCC bitfield packing
neal
Given the structure:
struct re_pattern_buffer
{
unsigned char *buffer;
unsigned long int allocated;
unsigned long int used;
reg_syntax_t syntax;
char *fastmap;
char *translate;
size_t re_nsub;
unsigned int can_be_null:1;
unsigned int regs_allocated:2;
unsigned int fastmap_accurate:1;
unsigned int no_sub:1;
unsigned int not_bol:1;
unsigned int not_eol:1;
unsigned int newline_anchor:1;
};
According to the C99 spec (ISO/IEC 9899:TC2) on page 102, "an
implementation may allocate any addressable storage unit large enough
to hold a bitfield. If enougth space remains, *a bit-field that
immediately follows another bit-fields in a structure shall be packed
into adjacent bits of the same unit*"
So I expect the bytes required to pack all those bitfields in the
above struct is 4, but GCC give me 8 on X86_64 and 4 on i386. How gcc
decide the storage unit for bitfields? I really appreciate your help.
Best regards,
Neal
David R Tribble
The trailing bitfield members of the struct only require 8 bits (one
byte) for storage. It looks like, then, that the compiler is padding
the structure to make it be an integral multiple of 4 (or 8) bytes in
length. I would guess that gcc chooses 4 for i386 and 8 for the
64-bit CPU in the interests of efficiency.
This can be verified by adding another small bitfield to the end
of the structure and seeing if it remains the same size.
There are command-line options to change the compiler's behavior
(but I don't know what they are off the top of my head).
-drt
Jack Klein
comp.std.c:
First, your example is not clear at all. What is the size of a
structure defined with just those bits? Are you saying that the space
occupied by just those seven bit fields, defining eight value bits, is
more than one byte? I think you are quite wrong.
Second, saying "GCC" is rather meaningless. There have been dozens,
if not hundreds, of released versions of the compiler over the years,
some of them predating C99.
Third, if you have a question about why the implementers of a
particular compiler decided to do things a certain way, the people to
ask are the implementers of the compiler. They are the ones who made
the decision.
Fourth, you are probably mistaken in what you think the compiler is
doing. Most likely sizeof (struct re_pattern_buffer) is returning a
larger value than you think is correct. For example, if you define
the structure without the bit-field elements and take its size, then
add them and take the size again, you see a difference of 4 or 8
bytes. That is almost certainly a result of padding at the end to
align the structure.
See this question and answer in the FAQ:
http://c-faq.com/struct/endpad.html
--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://c-faq.com/
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.contrib.andrew.cmu.edu/~ajo/docs/FAQ-acllc.html
Michal Necasek
> First, your example is not clear at all. What is the size of a
> structure defined with just those bits? Are you saying that the space
> occupied by just those seven bit fields, defining eight value bits, is
> more than one byte? I think you are quite wrong.
>
Since the structure contains pointers and longs, which are going to be
64 bits on AMD64, it'd be very surprising if the structure size wasn't a
multiple of eight bytes. As you say, the difference is almost certainly
padding and unrelated to the bit-field per se.
Michal
neal
size of the storage?
In this example, it uses "unsigned int a:1", what will happen if I
change it to "unsigned char a:1". I really have no ideal what the
"unsigned int" mean in bitfield declaration? Does the base type
specify the minimum storage for a bitfield? For example "unsigned int
a:1", might it require at least sizeof(unsigned int) bytes for this
field?
Best regards,
Ning
neal
> particular compiler decided to do things a certain way, the people to
> ask are the implementers of the compiler. They are the ones who made
> the decision.
allocate correct storage for bitfields.
>
> Fourth, you are probably mistaken in what you think the compiler is
> doing. Most likely sizeof (struct re_pattern_buffer) is returning a
> larger value than you think is correct. For example, if you define
> the structure without the bit-field elements and take its size, then
> add them and take the size again, you see a difference of 4 or 8
> bytes. That is almost certainly a result of padding at the end to
> align the structure.
this.
Yes, result is gotten by substracting the size of struct without
bitfields from the size of struct with bitfields. It apparently should
be end padding. By the way, can you enlight me the align of struct?
Should it just be the align of it's first field.
Best regards
neal
struct re_pattern_buffer_main_1_5
{
unsigned int can_be_null:1;
unsigned int regs_allocated:2;
unsigned int fastmap_accurate:1;
unsigned int no_sub:1;
unsigned int not_bol:1;
unsigned int not_eol:1;
unsigned int newline_anchor:1;
};
(unsigned int) is 4 bytes on X86_64. Is this due to the "unsigned int"
base type used?
When I change "unsigned int" to "unsigned char", the struct has 1 byte
in GCC. I'm wondering what's purpose of specifying a type for
bitfields aside their size.
Best regards,
SuperKoko
neal wrote:
> Thanks Michal. Given the following
> struct re_pattern_buffer_main_1_5
> {
> unsigned int can_be_null:1;
> unsigned int regs_allocated:2;
> unsigned int fastmap_accurate:1;
> unsigned int no_sub:1;
> unsigned int not_bol:1;
> unsigned int not_eol:1;
> unsigned int newline_anchor:1;
> };
> GCC will allocate 4 bytes instead of 8 bytes on X86_64, and the sizeof
> (unsigned int) is 4 bytes on X86_64. Is this due to the "unsigned int"
> base type used?
>
probably "yes".
> When I change "unsigned int" to "unsigned char", the struct has 1 byte
> in GCC. I'm wondering what's purpose of specifying a type for
> bitfields aside their size.
>
The aim is to give control to the programmer on the "underlying type"
used (note that we get outside the scope of the C standard and enter in
the world of GCC-specific things).
When you fetch or store something in a bitfield, with most CPU on earth
(including X86-64) there is no dedicated CPU instruction.
So, for fetching, the compiler has to load a whole memory unit (at
least 1 byte on X86-64) and extract the correct bits through
register-level bitwise operations.
For storing a bitfield, the compiler has:
1) To load a memory unit containing this bitfield (if it is not yet
loaded in a register)
2) To set the correct bits through bitwise operations
3) To store back this memory unit.
With GCC, this memory unit is the underlying type you specify (unsigned
int, unsigned char or whatever).
Giving the programmer the choice of the underlying type allows the
programmer to control various parameters:
1) Space efficiency vs speed efficiency (unsigned short is more space
efficient when you have 16 bits of bitfield data than unsigned int, but
is slower)
2) Dependencies of various bitfields
Two bitfields allocated in the same allocation unit are dependent.
Storing in one bitfield requires (as I said above) fetching & storing
the whole allocation unit.
In multithreaded applications that can matter much!
And even in a monothreaded application, those dependencies can be taken
into account for efficiency reasons.
For example, setting at the same time three contiguous bitfields that
live in the same allocation unit might be optimized by the compiler (he
can load & store the allocation unit only once).
That's particulary sensitive if the bitfields are set to constant
expressions.
Compare this two structure:
In fact, consider this structure:
struct X {
unsigned int x:7;
unsigned int y:15;
unsigned int yy:1;
unsigned char z;
};
To this one:
struct Y {
unsigned short y:15;
unsigned short yy:1;
unsigned char x:7;
unsigned char z;
};
Depending on your problem, struct X might be better than struct Y or
vice-versa.
If you remove the z field, the choice might be modified too.
GCC gives the choice for the low-level memory layout of bitfields.
kuy...@wizard.net
> Thanks David, Will the base type of bitfield declaration affect the
> size of the storage?
The actual amount of storage taken us depends only upon the number of
bits specified. The size of the storage unit that those bits are stored
in is up to the implementation, and could, if the implementation
chooses, depend upon the type of the bitfield declaration.
> In this example, it uses "unsigned int a:1", what will happen if I
> change it to "unsigned char a:1". I really have no ideal what the
> "unsigned int" mean in bitfield declaration?
The C standard requires an implementation to support 4 types for a bit
field: _Bool, int, signed int, and unsigned int. Use 'signed int' if
you wish the bit field to be signed, and 'unsigned int' if you with the
bit field to be unsigned. It's implementation-defined whether a plain
'int' bitfield is signed or unsigned (in conflict with the usage of
'int' in ordinary declarations). Therefore, you should use plain 'int'
in portable code only if you don't care whether it's signed or
unsigned. In my experience, its pretty rare to define a bitfield and
NOT care about whether its signed or unsigned.
An implementation is allowed to accept additional types for bitfields;
code that's intended to be portable should not take advantage of this
option.
> Does the base type
> specify the minimum storage for a bitfield?
Actually, the base type determines the maximum number of bits in the
bit-field width. It has no standard-imposed effect on size of the
storage unit the bitfield is allocated from.
> For example "unsigned int
> a:1", might it require at least sizeof(unsigned int) bytes for this
> field?
The bitfield 'a' is only allowed to take up 1 bit of memory. The size
of the storage unit that bit is allocated from is up to the
implentation. Adjacent bitfields that can also fit into that storage
unit are required to be allocated from it.
kuy...@wizard.net
...
> be end padding. By the way, can you enlight me the align of struct?
> Should it just be the align of it's first field.
The offset of every member of a struct, from the beginning of the
struct, is a contant for that member. Since you can create a correctly
aligned pointer to any non-bitfield member, then for every non-bitfield
member, the alignment requirements of the struct must be a multiple of
the alignment requirement for that member's data type. The smallest
possible alignment requirement for a struct is therefore the least
common multiple (LCM) of the alignment requirements for it's member
types. However, an implementation is allowed to impose stricter
requirements on a struct's alignment than the minimum.
neal
16, so the computation of LCM can be simplified as just computing their
maximum. Is there an excpetion? Do we need to consider the base type
of bitfields in alignment computation? for example,
struct {
char c;
short f;
unsigned int f:15;
unsigned int g:15;
}
If we ignore "unsigned int" of bitfields f, g, then the alignment is
just 2 (assume alignof(short) == 2), otherwise it should be 4 (assume
alignof(unsigned int) == 4).
Best regards,
Neal
neal
> neal wrote:
> > Thanks David, Will the base type of bitfield declaration affect the
> > size of the storage?
>
> The actual amount of storage taken us depends only upon the number of
> bits specified. The size of the storage unit that those bits are stored
> in is up to the implementation, and could, if the implementation
> chooses, depend upon the type of the bitfield declaration.
>
> > In this example, it uses "unsigned int a:1", what will happen if I
> > change it to "unsigned char a:1". I really have no ideal what the
> > "unsigned int" mean in bitfield declaration?
>
> The C standard requires an implementation to support 4 types for a bit
> field: _Bool, int, signed int, and unsigned int. Use 'signed int' if
> you wish the bit field to be signed, and 'unsigned int' if you with the
> bit field to be unsigned. It's implementation-defined whether a plain
> 'int' bitfield is signed or unsigned (in conflict with the usage of
> 'int' in ordinary declarations). Therefore, you should use plain 'int'
> in portable code only if you don't care whether it's signed or
> unsigned. In my experience, its pretty rare to define a bitfield and
> NOT care about whether its signed or unsigned.
mean when I do integer promotion, I should promote the bitfield to
"signed int", there is an example
struct subst
{
struct regex *regx;
struct replacement *replacement;
countT numb;
struct output *outf;
unsigned int global:1;
unsigned int print:2;
unsigned int eval:1;
unsigned int max_id:4;
} cmd;
... = 1 << cmd.eval;
So cmd.eval shoud be promoted to "unsigned int 1" right? If we declare
it as signed int eval:1 then the promoted integer value should be
XFFFFFFFF. Do I understand this correct?
Thanks a lot.
neal
SuperKoko wrote:
> neal wrote:
> The aim is to give control to the programmer on the "underlying type"
> used (note that we get outside the scope of the C standard and enter in
> the world of GCC-specific things).
> When you fetch or store something in a bitfield, with most CPU on earth
> (including X86-64) there is no dedicated CPU instruction.
> So, for fetching, the compiler has to load a whole memory unit (at
> least 1 byte on X86-64) and extract the correct bits through
> register-level bitwise operations.
> For storing a bitfield, the compiler has:
> 1) To load a memory unit containing this bitfield (if it is not yet
> loaded in a register)
> 2) To set the correct bits through bitwise operations
> 3) To store back this memory unit.
Very appreciate you explain the manipulation of bitfields. To my
understanding, the endian is important factor to generate correct
bitwise operations, Do we have to store a bitfield in only one storge
unit? It might cross two units I guess in some case.
>
> With GCC, this memory unit is the underlying type you specify (unsigned
> int, unsigned char or whatever).
>
> Giving the programmer the choice of the underlying type allows the
> programmer to control various parameters:
> 1) Space efficiency vs speed efficiency (unsigned short is more space
> efficient when you have 16 bits of bitfield data than unsigned int, but
> is slower)
> 2) Dependencies of various bitfields
> Two bitfields allocated in the same allocation unit are dependent.
> Storing in one bitfield requires (as I said above) fetching & storing
> the whole allocation unit.
> In multithreaded applications that can matter much!
I didn't get why it affect multithreaded programs more?
> And even in a monothreaded application, those dependencies can be taken
> into account for efficiency reasons.
> For example, setting at the same time three contiguous bitfields that
> live in the same allocation unit might be optimized by the compiler (he
> can load & store the allocation unit only once).
> That's particulary sensitive if the bitfields are set to constant
> expressions.
But I believe the performance impact is really trivial unless the
assignments of bitfields in a loop.
Best regards,
Neal
kuy...@wizard.net
> kuy...@wizard.net wrote:
...
> > The C standard requires an implementation to support 4 types for a bit
> > field: _Bool, int, signed int, and unsigned int. Use 'signed int' if
> > you wish the bit field to be signed, and 'unsigned int' if you with the
> > bit field to be unsigned. It's implementation-defined whether a plain
> > 'int' bitfield is signed or unsigned (in conflict with the usage of
> > 'int' in ordinary declarations). Therefore, you should use plain 'int'
> > in portable code only if you don't care whether it's signed or
> > unsigned. In my experience, its pretty rare to define a bitfield and
> > NOT care about whether its signed or unsigned.
> This is what I don't understand --"the bit field to be signed". Do this
> mean when I do integer promotion, I should promote the bitfield to
> "signed int", there is an example
Unless you're talking as someone who's writing a compiler, you don't do
the promotion; that happens automatically; the compiler takes care of
the details.
> struct subst
> {
> struct regex *regx;
> struct replacement *replacement;
> countT numb;
> struct output *outf;
> unsigned int global:1;
> unsigned int print:2;
> unsigned int eval:1;
> unsigned int max_id:4;
> } cmd;
>
> ... = 1 << cmd.eval;
>
> So cmd.eval shoud be promoted to "unsigned int 1" right? If we declare
> it as signed int eval:1 then the promoted integer value should be
> XFFFFFFFF. Do I understand this correct?
You're talking about:
signed int eval:1;
If so, since you've declared it as having only 1 bit, and a signed type
always has at least 1 sign bit, the only bit it contains is interpreted
as a sign bit. What value the bit represents depends upon whether the
implementation is using 2's complement, 1's complement, or
sign-magnitude for signed integer bitfields. If it's 2's complement, it
represents a value of -1; otherwise, it represents a value of -0. In
neither case does it represent 0xFFFFFFFF.
If, after extracting the value of cmd.eval, you convert it to a
int16_t, then it will have a value of either 0xFFFFFFFF, or 0x0,
depending upon whether cmd.eval has a 2's complement representation.
Note: if you change it to a signed type, don't use it in bitwise
expressions like "1 << cmd.eval"; bitwise operations should only be
performed on unsigned types.
kuy...@wizard.net
> THanks Kuyper,
> kuy...@wizard.net wrote:
...
> > The offset of every member of a struct, from the beginning of the
> > struct, is a contant for that member. Since you can create a correctly
> > aligned pointer to any non-bitfield member, then for every non-bitfield
> > member, the alignment requirements of the struct must be a multiple of
> > the alignment requirement for that member's data type. The smallest
> > possible alignment requirement for a struct is therefore the least
> > common multiple (LCM) of the alignment requirements for it's member
> > types. However, an implementation is allowed to impose stricter
> > requirements on a struct's alignment than the minimum.
> The only possible alignments of primitive data types are 1, 2, 4, 8,
> 16,
That's true of most, and possible all, implementations currently in
use. But it's not required by the C standard. There are real (if old)
architectures where other alignments would have been plausible, though
I'm not sure whether any conforming implementations of have been made
for such machines.
> so the computation of LCM can be simplified as just computing their
> maximum. Is there an excpetion?
Not on systems where alignment requirements are all powers of 2. I
wouldn't write code which depended upon that, however. On the other
hand, I also avoid writing code that relies on 2's complement
representations, which are equally ubiquitous. Some people would say
I'm being overcautious, and there's some truth in that accusation.
> Do we need to consider the base type
> of bitfields in alignment computation?
Check your implementation's documentation. It's impossible to actually
calculate alignment requirements without implementation-specific
information. It's actually the storage unit, not the base type, that
matters. The storage unit to use is up to the implementation to decide.
Whether or not that storage unit depends upon the base type is up to
the implementation. The alignment requirements of the storage unit are
up to the implementation. While the logic I gave sets a lower limit on
the alignment requirement, it's up to the implementation whether or not
the struct has exactly that lower limit, or is a higher common multiple
of the alignment requirements.
for example,
> struct {
> char c;
> short f;
> unsigned int f:15;
> unsigned int g:15;
> }
> If we ignore "unsigned int" of bitfields f, g, then the alignment is
> just 2 (assume alignof(short) == 2), otherwise it should be 4 (assume
> alignof(unsigned int) == 4).
It would be legal for an implementation to put f and g into the same
8-byte storage unit, with an alignment requirement of 8. There have
been implementations where all structures had an alignment requirement
of 16, regardless of contents, and I know this from personal experience
with such machines. That was also legal.
The standard isn't sufficient to answer that question; you have to
check your implementation's documentation. The answer may be different
on different implementations. Best advice: avoid writing code that
depends upon knowing the exact alignment requirement.
The size of any type has to be a multiple of it's alignment
requirement, and memory that's been allocated using malloc() family
members is guaranteed to have suitable alignment for any type. Those
two facts are usually sufficient to deal with any alignment issue that
you might run into. The only exception is if you have to actually
implement the malloc() family, or a function with similar alignment
requirements; you can't do that portably, though it is possible to come
close.
kuy...@wizard.net
...
> SuperKoko wrote:
...
> Very appreciate you explain the manipulation of bitfields. To my
> understanding, the endian is important factor to generate correct
> bitwise operations, Do we have to store a bitfield in only one storge
> unit? It might cross two units I guess in some case.
It's up to the implementation whether or not a bitfield can cross
storage units.
...
> > 2) Dependencies of various bitfields
> > Two bitfields allocated in the same allocation unit are dependent.
> > Storing in one bitfield requires (as I said above) fetching & storing
> > the whole allocation unit.
> > In multithreaded applications that can matter much!
> I didn't get why it affect multithreaded programs more?
Because one thread can access one bitfield stored in a given storage
unit, while another thread can access a different bitfield stored in
the same storage unit. Work out the details, and I think you'll see
what the problem is.
ena8...@yahoo.com
Technically, it could yield a value of -1, -32768, -2147483648, 493,
or any other int value, in any of the 2's/1's/s+m representations,
since it may be a trap representation.
neal
> Unless you're talking as someone who's writing a compiler, you don't do
> the promotion; that happens automatically; the compiler takes care of
> the details.
start a new topic about integer promotion since there are some
sentences in the standard I don't understand.
Best regards,
Neal
Michal Necasek
> Because one thread can access one bitfield stored in a given storage
> unit, while another thread can access a different bitfield stored in
> the same storage unit. Work out the details, and I think you'll see
> what the problem is.
>
I don't. If you're going to modify the bitfield simultaneously from
multiple threads, you'd better have some locking, regardless of whether
two threads are attempting to modify the same bitfield or two different
bitfields that happen to be allocated in the same storage unit.
I would hope no one is dumb enough to implement separate locks for
each individual bitfield.
Michal
kuy...@wizard.net
The point is that, if shared allocation weren't permitted, seperate
locks for each bitfield would merely be extremely inefficient.
Wojtek Lerch
news:1156791930.7...@i3g2000cwc.googlegroups.com...
> Michal Necasek wrote:
>> kuy...@wizard.net wrote:
>>
>> > Because one thread can access one bitfield stored in a given storage
>> > unit, while another thread can access a different bitfield stored in
>> > the same storage unit. Work out the details, and I think you'll see
>> > what the problem is.
>> I would hope no one is dumb enough to implement separate locks for
>> each individual bitfield.
>
> The point is that, if shared allocation weren't permitted, seperate
> locks for each bitfield would merely be extremely inefficient.
The C standard says nothing about what the "store unit" that bit fields are
allocated from has to do with the granularity of memory accesses for the
purpose of locking. As far as I know, there are implementations where it's
unsafe to have two threads accessing two consecutive elements of an array of
chars or shorts, or maybe even two completely unrelated static shorts that
happen to be allocated next to each other in memory.
David R Tribble
> Given the following
> struct re_pattern_buffer_main_1_5
> {
> unsigned int can_be_null:1;
> unsigned int regs_allocated:2;
> unsigned int fastmap_accurate:1;
> unsigned int no_sub:1;
> unsigned int not_bol:1;
> unsigned int not_eol:1;
> unsigned int newline_anchor:1;
> };
>
> GCC will allocate 4 bytes instead of 8 bytes on X86_64, and the sizeof
> (unsigned int) is 4 bytes on X86_64. Is this due to the "unsigned int"
> base type used?
'unsigned int' is the most "correct" choice for bitfields.
> When I change "unsigned int" to "unsigned char", the struct has 1 byte
> in GCC. I'm wondering what's purpose of specifying a type for
> bitfields aside their size.
That's a new one on me. I've never heard of a compiler adjusting
struct size based on bitfield types, but it certainly is possible for
a compiler vendor to do this (as an extension). Reading the
standard, I get the impression that the underlying type should have
nothing to do with the struct size. Most compilers I've seen take
note of command line options and #pragmas for packing bits into
structs instead.
-drt
SuperKoko
On x86 CPU, I think that most implementations do that (at least all the
implementations I use)
Wojtek Lerch wrote:
> The C standard says nothing about what the "store unit" that bit fields are
> allocated from has to do with the granularity of memory accesses for the
> purpose of locking. As far as I know, there are implementations where it's
> unsafe to have two threads accessing two consecutive elements of an array of
> chars or shorts, or maybe even two completely unrelated static shorts that
> happen to be allocated next to each other in memory.
Of course, but NOTHING is standard with threads!
In fact, you can't assume that compilers provide "thread-safe" ways to
read/fetch any value.
What I said was in the specific & strict context of GCC (and knowing
that it probably only applies to a few multi-threaded platforms that
GCC supports).
Hans-Bernhard Broeker
> neal wrote:
> > When I change "unsigned int" to "unsigned char", the struct has 1 byte
> > in GCC. I'm wondering what's purpose of specifying a type for
> > bitfields aside their size.
> That's a new one on me. I've never heard of a compiler adjusting
> struct size based on bitfield types, but it certainly is possible for
> a compiler vendor to do this (as an extension).
Bitfields of type "unsigned char" are already an extension (in the
range of implementation-defined types). The details of their
behaviour are thus implementation-defined, too. And of course, just
about every aspect of bit field memory layout is
implementation-defined to begin with.
--
Hans-Bernhard Broeker (bro...@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.
Wojtek Lerch
news:1156842413.6...@b28g2000cwb.googlegroups.com...
> What I said was in the specific & strict context of GCC (and knowing
> that it probably only applies to a few multi-threaded platforms that
> GCC supports).
Yes, but you said that four posts before the one I responded to. Two posts
later, all the traces of the GCC context were gone, and the discussion
sounded pretty generic again, talking about "the implementation" and a
hypothetical C standard that disallows bitfields to share an allocation
unit.
neal
> David R Tribble <da...@tribble.com> wrote:
> > neal wrote:
>
> > > When I change "unsigned int" to "unsigned char", the struct has 1 byte
> > > in GCC. I'm wondering what's purpose of specifying a type for
> > > bitfields aside their size.
>
> > That's a new one on me. I've never heard of a compiler adjusting
> > struct size based on bitfield types, but it certainly is possible for
> > a compiler vendor to do this (as an extension).
>
> Bitfields of type "unsigned char" are already an extension (in the
> range of implementation-defined types). The details of their
> behaviour are thus implementation-defined, too. And of course, just
> about every aspect of bit field memory layout is
> implementation-defined to begin with.
is implementation-defined. Eventhough the memory layout is
implementation-defined, any code conforming to C99 standard should
behave the same?
Best regards,
kuy...@wizard.net
neal wrote:
> Hans-Bernhard Broeker wrote:
...
> > Bitfields of type "unsigned char" are already an extension (in the
> > range of implementation-defined types). The details of their
> > behaviour are thus implementation-defined, too. And of course, just
> > about every aspect of bit field memory layout is
> > implementation-defined to begin with.
> I think not only bitfield memory layout but also C struct memory layout
> is implementation-defined.
Yes, to a very considerable extent; section 6.7.2.1p13 says:
"Within a structure object, the non-bit-field members and the units in
which bit-fields
reside have addresses that increase in the order in which they are
declared. A pointer to a
structure object, suitably converted, points to its initial member (or
if that member is a
bit-field, then to the unit in which it resides), and vice versa. There
may be unnamed
padding within a structure object, but not at its beginning."
That's pretty much the only restrictions on the layout of structures.
> ... Eventhough the memory layout is
> implementation-defined, any code conforming to C99 standard should
> behave the same?
Not true, at all.
a) To say that code "conforms" to the C99 standard means far less than
it might appear to mean. The definition of "conforming code" that
appears in the C standard was a political compromise whose only
usefulness was that it allowed the standard to be approved. Code is
conforming if it can be accepted by a conforming implementation of C.
If the behavior of the code is undefined, what it does after accepting
the code is up to the implementation; all that matters is that it
accepts it.
There is only one feature that C code can possess which a conforming
compiler is required to reject: a #error directive that survives
conditional compilation. Lincoln's Gettysburg Address doesn't contain
any #error directives; therefore a conforming implementation of C could
be written that would accept it (and I believe someone has actually
written one that will do so, just as an excersize to show how
ridiculous this definition of "conforming code" is). Therefore,
Lincoln's Gettysburg Address conforms to the C99 standard.
b) There is a much stricter conformance category, "strictly
conforming". According to section 4p5 "It shall not produce output
dependent on any unspecified, undefined, or implementation-defined
behavior, and shall not exceed any minimum implementation limit."
That is a constraint on what code qualifies as strictly conforming; it
imposes no additional constraint on the implementation. In particular,
it imposes no additional restrictions on how an implementation lays out
structures. If your code is written in such a fashion that it would
produce output which depends upon how an implementation uses it's
freedom to lay out structures, then your code is not strictly
conforming.
neal
frustrated me most is the undefined behavior in the standard. For
example, 6.5.16p4 "The order of evaluation of the operands is
unspecified. If an attempt to made to modify the result of an
assignment operator or to access it after the next sequence point, the
behavior is undefined." I have even seen the inconsistancy caused by
this in different versions of GCC. For example,
int main ()
{
static int v[10];
int i;
i = 0;
for (i = 0; i < 10; ++i)
v[i] = 0;
i = 0;
v[++i] = (++i, 100);
for (i = 0; i < 10; ++i)
printf ("v[%d] = %d\n", i, v[i]);
return 0;
}
compile it by GCC 3.3.5, you get v[1] = 100, and GCC 4.0.3 produces
code with v[2] = 100. Apparently GCC 4.0.3 evaluates the right hand
side of assignment expression first while GCC 3.3.5 evaluates the left
hand side first. Would it be possible to define a strict evaluation
order in the standard? Sometimes too much flexiability is a burden to
compiler writers as well as programmers.
kuy...@wizard.net
...
> hand side first. Would it be possible to define a strict evaluation
Yes, it's certainly possible, It's also been hotly debated. The key
trade-off is between efficiency of implementation, and ease of use. The
fundamental point of disagreement is about how big that trade-off is.
One of the simplest ways to create pressure for making such a change
would be to, first, figure out how what you think that strict
evaluation order should be. Then, create an implementation which
guarantees that order. Finally, and most importantly, have a lot of
people get experience using that implementation, and from that
experience generate anecdotal and quantitative evidence demonstrating
the costs are low, and the benefits are large. Nothing provides better
support for standardization of a proposed feature, than the existence
of implemenations that are popular because they already provide that
feature.
> ... Sometimes too much flexiability is a burden to
> compiler writers as well as programmers.
For a compiler writer, the only burden you should feel when the
behavior is undefined is the burden of choice. As a compiler writer,
you're free to handle the undefined cases any way you want to: the way
that's most convenient for you, the way that makes the most sense to
you, the way that the customers want it; whatever you want.
Keith Thompson
> neal wrote:
> ...
>> hand side first. Would it be possible to define a strict evaluation
>> order in the standard? ...
>
> Yes, it's certainly possible, It's also been hotly debated. The key
> trade-off is between efficiency of implementation, and ease of use. The
> fundamental point of disagreement is about how big that trade-off is.
>
> One of the simplest ways to create pressure for making such a change
> would be to, first, figure out how what you think that strict
> evaluation order should be. Then, create an implementation which
> guarantees that order. Finally, and most importantly, have a lot of
> people get experience using that implementation, and from that
> experience generate anecdotal and quantitative evidence demonstrating
> the costs are low, and the benefits are large. Nothing provides better
> support for standardization of a proposed feature, than the existence
> of implemenations that are popular because they already provide that
> feature.
Agreed. IMHO the best way to handle this would be for an existing
compiler to provide a command-line option to impose strict evaluation
order (which could disable some optimizations), allowing for a direct
comparison.
I suppose that the compiler could still generate code that evaluates
operands in a different order *if* it can prove that it makes no
difference (e.g., if the evaluation of the operands has no side
effects).
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Richard Bos
> neal wrote:
> ...
> > hand side first. Would it be possible to define a strict evaluation
> > order in the standard? ...
>
> Yes, it's certainly possible, It's also been hotly debated. The key
> trade-off is between efficiency of implementation, and ease of use. The
> fundamental point of disagreement is about how big that trade-off is.
>
> One of the simplest ways to create pressure for making such a change
> would be to, first, figure out how what you think that strict
> evaluation order should be. Then, create an implementation which
> guarantees that order.
Then, port that implementation to as varied a set of hardware and OSes
as you can.
> Finally, and most importantly, have a lot of
> people get experience using that implementation, and from that
> experience generate anecdotal and quantitative evidence demonstrating
> the costs are low, and the benefits are large.
And that it is efficiently implementable not only on Intel under
MS-Windows and Linux, but also on a Sparc under Solaris, on MacOS X, on
an IBM AS/400, on Symbian...
Richard
neal
> neal wrote:
> ...
> > hand side first. Would it be possible to define a strict evaluation
> > order in the standard? ...
>
> Yes, it's certainly possible, It's also been hotly debated. The key
> trade-off is between efficiency of implementation, and ease of use. The
> fundamental point of disagreement is about how big that trade-off is.
>
> One of the simplest ways to create pressure for making such a change
> would be to, first, figure out how what you think that strict
> evaluation order should be. Then, create an implementation which
> guarantees that order. Finally, and most importantly, have a lot of
> people get experience using that implementation, and from that
> experience generate anecdotal and quantitative evidence demonstrating
> the costs are low, and the benefits are large. Nothing provides better
> support for standardization of a proposed feature, than the existence
> of implemenations that are popular because they already provide that
> feature.
order without at least an internal guideline to follow up. Given the
example of GCC, it's a very popular open source C compiler, but it's
not even consistant between different versions. I haven't evaluated
other C compilers, but I would expect the commercial C compilers be
more consistant since they are presumably better on backward
compatibility. Putting a feature such as a strict evaluation into the
standard might not be feasible in near future, would it be possible
some experts in this group define alternative evaluation orders and
publish them for later implementation? We then can graduately and
formally define some alternative choices for all undefined behavior in
the standard.
>
> > ... Sometimes too much flexiability is a burden to
> > compiler writers as well as programmers.
>
> For a compiler writer, the only burden you should feel when the
> behavior is undefined is the burden of choice. As a compiler writer,
> you're free to handle the undefined cases any way you want to: the way
> that's most convenient for you, the way that makes the most sense to
> you, the way that the customers want it; whatever you want.
"right" choice is hard because the cariteria of "right" is not defined.
If a compiler writer just writes one for fun, he/she don't care about
how to handle undefined behavior, but in most case, he/she want his/her
compiler can "correctly" compile existing code. Unfortunately, the
"correctness" is established on an ill-defined feature, such as the
evaluation orders implemented by "popular" compilers.
neal
> kuy...@wizard.net writes:
> > neal wrote:
> > ...
> >> hand side first. Would it be possible to define a strict evaluation
> >> order in the standard? ...
> >
> > Yes, it's certainly possible, It's also been hotly debated. The key
> > trade-off is between efficiency of implementation, and ease of use. The
> > fundamental point of disagreement is about how big that trade-off is.
> >
> > One of the simplest ways to create pressure for making such a change
> > would be to, first, figure out how what you think that strict
> > evaluation order should be. Then, create an implementation which
> > guarantees that order. Finally, and most importantly, have a lot of
> > people get experience using that implementation, and from that
> > experience generate anecdotal and quantitative evidence demonstrating
> > the costs are low, and the benefits are large. Nothing provides better
> > support for standardization of a proposed feature, than the existence
> > of implemenations that are popular because they already provide that
> > feature.
>
> Agreed. IMHO the best way to handle this would be for an existing
> compiler to provide a command-line option to impose strict evaluation
> order (which could disable some optimizations), allowing for a direct
> comparison.
publish them, then the direct comparison will be more sound.
>
> I suppose that the compiler could still generate code that evaluates
> operands in a different order *if* it can prove that it makes no
> difference (e.g., if the evaluation of the operands has no side
> effects).
hard due to the using of pointer. It requires really sophisticated
points-to/alias analysis.
Keith Thompson
> Keith Thompson wrote:
[...]
>> Agreed. IMHO the best way to handle this would be for an existing
>> compiler to provide a command-line option to impose strict evaluation
>> order (which could disable some optimizations), allowing for a direct
>> comparison.
> This is surely doable, If we can first formalize several choices and
> publish them, then the direct comparison will be more sound.
>>
>> I suppose that the compiler could still generate code that evaluates
>> operands in a different order *if* it can prove that it makes no
>> difference (e.g., if the evaluation of the operands has no side
>> effects).
> Unfortunately, proving an expression having no side effects is really
> hard due to the using of pointer. It requires really sophisticated
> points-to/alias analysis.
That's not a real problem. As far as the standard is concerned, a
compiler is under no obligation to perform such analysis. If it can
get better code by doing so, that's great, but if the analysis is too
difficult it's perfectly free to give up and generate more naive code.
Michal Necasek
> That's not a real problem. As far as the standard is concerned, a
> compiler is under no obligation to perform such analysis. If it can
> get better code by doing so, that's great, but if the analysis is too
> difficult it's perfectly free to give up and generate more naive code.
>
IMO the best solution would be to detect the situation and emit a
warning. Typically there's simply no way this code would be portable, so
what programmers *really* want is to avoid it. Trying to define the
behaviour will only entice sloppy programmers into relying on things
they shouldn't be relying on.
As someone else mentioned, commercial compilers usually don't change
their behaviour between versions, but different compilers still behave
differently.
The other day I wrote something like 'x = *s++ + *s++ + *s++;' and was
surprised by the result. It was trivial to fix the code but a warning
would have saved me some time.
Michal
Keith Thompson
> Keith Thompson wrote:
>> That's not a real problem. As far as the standard is concerned, a
>> compiler is under no obligation to perform such analysis. If it can
>> get better code by doing so, that's great, but if the analysis is too
>> difficult it's perfectly free to give up and generate more naive code.
>>
> IMO the best solution would be to detect the situation and emit a
> warning. Typically there's simply no way this code would be portable,
> so what programmers *really* want is to avoid it. Trying to define the
> behaviour will only entice sloppy programmers into relying on things
> they shouldn't be relying on.
Sure, but you don't want a warning every time you have a subexpression
with a side effect. Certainly a warning for anything that invokes
undefined behavior would be a good thing (though that's not always
possible, which is why a lot of things are undefined behavior in the
first place). It would also be nice to have a warning if the result
or side effects of an expression can vary depending on allowed
variations in evaluation order.
There's some tendency for the compiler to assume that the programmer
knows what he's doing; sometimes the compiler should say "No, you
really *don't* know what you're doing!".
And, of course, many compilers do issue such warnings, especially if
you choose the right set of command-line options.
[...]
> The other day I wrote something like 'x = *s++ + *s++ + *s++;' and
> was surprised by the result. It was trivial to fix the code but a
> warning would have saved me some time.
Frankly, it wouldn't have occurred to me to write that in the first
place.
Harald van Dijk
At least two compilers do warn for that (with appropriate compiler
options):
gcc 4.1.1: warning: operation on ‘s’ may be undefined
tendra 4.1.2: Variable 's' modified twice in expression.
Personally, I'd have liked it if compilers were allowed to refuse to
even try to compile this, but unfortunately, as things are, that
requires the compiler to prove that the line will always be reached at
least once.
neal
reason, the another problem is compilers have to be very conservative
and issue warnings on whatever suspective expressions which might not
really have side effects. One example is function call,
int i; /* global variable */
*foo () = ++i;
foo () might/might not change the value of i. Should compilers issue a
warning or reject this?
Neal
Douglas A. Gwyn
> One of the simplest ways to create pressure for making such a change
> would be to, first, figure out how what you think that strict
> evaluation order should be. Then, create an implementation which
> guarantees that order. Finally, and most importantly, have a lot of
> people get experience using that implementation, and from that
> experience generate anecdotal and quantitative evidence demonstrating
> the costs are low, and the benefits are large. Nothing provides better
> support for standardization of a proposed feature, than the existence
> of implemenations that are popular because they already provide that
> feature.
However, for changes of the kind under discussion, you also need to
include a very wide range of architectures in the implementation
testing. What is an easy decision for a given architecture might
be a poor decision from the point of view of another.
Also, since reasonable C programmers generally avoid relying on
the details that you'd be nailing down, they wouldn't be able to
detect any such change (other than possible execution cost), so
their feedback would be of limited value. This kind of decision
is best left up to a group consisting largely of implementors with
a wide variety of experience. Hmm, that's already how it is..
Douglas A. Gwyn
> And that it is efficiently implementable not only on Intel under
> MS-Windows and Linux, but also on a Sparc under Solaris, on MacOS X, on
> an IBM AS/400, on Symbian...
And don't forget embedded systems/microcontrollers, as that is
the application realm where C is most dominant.
Wojtek Lerch
news:2IFJg.7084$q63....@newssvr13.news.prodigy.com...
> As someone else mentioned, commercial compilers usually don't change
> their behaviour between versions, but different compilers still behave
> differently.
Is that really true? I would expect a decent compiler to order the
evaluation of subexpressions based on an analysis of how it may affect
possible optimizations. I imagine that in many cases it might depend on
very subtle details of both the optimizer's algorithms and the code being
compiled. I would find it disappointing if compiler vendors decided to give
up an oportunity to improve their optimizer to avoid changing the order of
evaluation.
But, on the other hand, perhaps people who write bad code have more
influence on compiler vendors than I do. ;-)
Harald van Dijk
I think you mean foo() might return &i: if foo() only changes i, the
behaviour is unspecified but not undefined. And no, I don't think
compilers should be allowed to reject that. I think compilers should be
allowed to reject any statement that cannot ever be executed with
defined behaviour. To explain where I would draw the line in my
hypothetical not-quite-C language:
int *p = f();
*p = 0; /* this must be allowed (unless the compiler can prove f()
never returns a pointer to an int) */
int *p = f();
if(p == 0)
*p = 0; /* this may cause the program to fail to compile */
else
*p = 0;
Jun Woong
Harald van Dijk wrote:
[...]
> I think compilers should be
> allowed to reject any statement that cannot ever be executed with
> defined behaviour.
>
The committee also think that.
--
Jun, Woong (woong at icu.ac.kr)
Samsung Electronics Co., Ltd.
``All opinions are mine and do not represent any organization''
Harald van Dijk
> Harald van Dijk wrote:
> [...]
> > I think compilers should be
> > allowed to reject any statement that cannot ever be executed with
> > defined behaviour.
> >
>
> The committee also think that.
http://www.open-std.org/JTC1/SC22/WG14/www/docs/dr_109.html
No, they do not.
neal
> I think you mean foo() might return &i: if foo() only changes i, the
> behaviour is unspecified but not undefined. And no, I don't think
> compilers should be allowed to reject that. I think compilers should be
> allowed to reject any statement that cannot ever be executed with
> defined behaviour. To explain where I would draw the line in my
I cannot distinguish unspecified/undefined behavior, to my
understanding the "unspecified" behevior is due to the evaluation order
is not defined.
> hypothetical not-quite-C language:
>
> int *p = f();
> *p = 0; /* this must be allowed (unless the compiler can prove f()
> never returns a pointer to an int) */
>
> int *p = f();
> if(p == 0)
> *p = 0; /* this may cause the program to fail to compile */
> else
> *p = 0;
This is not easy, it always need points-to analysis to detect the
possible deference of null pointer in your example. You can see the
case in a complicated example.
if (p == 0)
{
bar ();
*p = 0; /* this may/may not be ok */
}
Actually, to the best of my knowledge, most type safe languages such as
Java, C#, detect this at runtime. It's a really difficult problem in
static analysis.
kuy...@wizard.net
> Harald van Dijk wrote:
> > I think you mean foo() might return &i: if foo() only changes i, the
> > behaviour is unspecified but not undefined. And no, I don't think
> > compilers should be allowed to reject that. I think compilers should be
> > allowed to reject any statement that cannot ever be executed with
> > defined behaviour. To explain where I would draw the line in my
>
> I cannot distinguish unspecified/undefined behavior, to my
> understanding the "unspecified" behevior is due to the evaluation order
> is not defined.
You need to learn the distinction. When the behavior is unspecified,
that means that there is a list, usually finite, and often not
explicitly stated, of permitted behaviors. An implementation is free to
choose one of those permitted behaviors, but if it chooses a behavior
that is not on the list, it becomes non-conforming.
When the behavior is undefined. The list is completely open ended.
There is, literally, nothing an implemenatation could chose to do that
would render it non-conforming. This is a VERY important distinction.
Learn it.
In this particular case, you can construct the list of permitted
behaviors by listing all of the different permitted orders for the
evaluation of the relevant expressions and the occurance of their side
effects. A conforming implementation is required to produce the same
result as if it had used one of the permitted orders. It doesn't have
to actually use one of those orders, it just has to produce the same
effects. However, if it produces a result that doesn't match any of
those orders, it's not a conforming implementation of C.
> > hypothetical not-quite-C language:
> >
> > int *p = f();
> > *p = 0; /* this must be allowed (unless the compiler can prove f()
> > never returns a pointer to an int) */
> >
> > int *p = f();
> > if(p == 0)
> > *p = 0; /* this may cause the program to fail to compile */
> > else
> > *p = 0;
>
> This is not easy, it always need points-to analysis to detect the
> possible deference of null pointer in your example. You can see the
> case in a complicated example.
>
> if (p == 0)
> {
> bar ();
> *p = 0; /* this may/may not be ok */
> }
> Actually, to the best of my knowledge, most type safe languages such as
> Java, C#, detect this at runtime. It's a really difficult problem in
> static analysis.
But the compiler is NOT required to detect it statically (or even
dynamically). That's the whole point of making the behavior undefined.
Depending upon the definition of bar, it might or might not be possible
to determine statically whether this code dereferences an invalid
pointer. A high quality implementation would look into the definition
of bar(), if it's in the same translation unit, to see whether it can
determine whether or not the code is safe. A much higher-quality
implementation could examine bar() even if it's defined in a different
translation unit, which means the check would have to be done at link
time. However, the C++ standard deliberately avoids mandating the
quality of implementation, which means that it's possible to produce a
low-quality but conforming implmentation of C++ for a new platform even
if it's so new that there hasn't been time to put together a high
quality implementation, or for a platform that has so few users that
there isn't enough money available to put together a high quality
implementation.
This is one of the key reasons why C++ is so widely implemented.
Jun Woong
Harald van Dijk 작성:
Oh, I misread your posting; I should have seen your examples more
carefully. I thought you meant that compilers should be allowed to
reject statements that *every execution of the program meets and*
cannot be executed with defined behavior.
I think the committee's answer is quite reasonable. If you answer to
the DR, what do you write? Just "an implementation is allowed to
reject any progem when it contains a statement that cannot be executed
without invoking undefined behavior," which makes it possible for the
following code to be rejected
if (user_input())
1 / 0;
because the expression statement "1 / 0;" cannot be executed without
undefined behavior?
It must be quite good and simple answer that an implementation can
reject a prgoram only when its *every* execution results in UB"; given
that an implementation already has a good way to say the programmer
that his code has a surely problematic construct, I don't see any
reasonable reason the standard should deal with such annoying problem.
--
Jun, Woong (woong at icu.ac.kr)
Samsung Electronics Co., Ltd.
``All opinions are mine and do not represet any organization''
Harald van Dijk
> Harald van Dijk 작성:
>
> > Jun Woong wrote:
> > > Harald van Dijk wrote:
> > > [...]
> > > > I think compilers should be
> > > > allowed to reject any statement that cannot ever be executed with
> > > > defined behaviour.
> > > >
> > >
> > > The committee also think that.
> >
> > http://www.open-std.org/JTC1/SC22/WG14/www/docs/dr_109.html
> >
> > No, they do not.
>
> Oh, I misread your posting; I should have seen your examples more
> carefully. I thought you meant that compilers should be allowed to
> reject statements that *every execution of the program meets and*
> cannot be executed with defined behavior.
>
> I think the committee's answer is quite reasonable.
I think it is the only correct answer for the C standard as it is (I
mentioned a "hypothetical not-quite-C language" in which such
statements could be rejected), and I think it is quite reasonable, but
it is not my preferred answer.
> If you answer to
> the DR, what do you write? Just "an implementation is allowed to
> reject any progem when it contains a statement that cannot be executed
> without invoking undefined behavior," which makes it possible for the
> following code to be rejected
>
> if (user_input())
> 1 / 0;
>
> because the expression statement "1 / 0;" cannot be executed without
> undefined behavior?
That is exactly the sort of thing I would like to see compilers allowed
(to clarify: not required) to reject. However, the wording (that I
myself used) is poor because it allows /all/ dead code to cause a
program to fail to compile, and there can be legitimate reasons for
dead code. I'm not sure what proper wording would be.
> It must be quite good and simple answer that an implementation can
> reject a prgoram only when its *every* execution results in UB"; given
> that an implementation already has a good way to say the programmer
> that his code has a surely problematic construct, I don't see any
> reasonable reason the standard should deal with such annoying problem.
Nor do I see a reason to force compilers to create meaningful output
for what can only be either a bug or extremely poor style. As a
real-world example: GCC treats an indexed string literal as a constant
(except for "constant expressions"); there is no difference in
generated code between "return 'c';" and "return "abc"[2];". That is
very reasonable, and I expect many other compilers behave the same.
However, there is a bug in that "abc"[2]++ fails to compile. Why must
that be a bug? What benefit is there in allowing it? What legitimate
reason is there to do this (other than to test conformance)?
Jun Woong
Harald van Dijk wrote:
> Jun Woong wrote:
[...]
>
> > If you answer to
> > the DR, what do you write? Just "an implementation is allowed to
> > reject any progem when it contains a statement that cannot be executed
> > without invoking undefined behavior," which makes it possible for the
> > following code to be rejected
> >
> > if (user_input())
> > 1 / 0;
> >
> > because the expression statement "1 / 0;" cannot be executed without
> > undefined behavior?
>
> That is exactly the sort of thing I would like to see compilers allowed
> (to clarify: not required) to reject. However, the wording (that I
> myself used) is poor because it allows /all/ dead code to cause a
> program to fail to compile, and there can be legitimate reasons for
> dead code. I'm not sure what proper wording would be.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
That is my point. I'm afraid that allowing implementations to reject
codes like yours doesn't deserve a new wording; I think contriving it
is not easy as it is for ordinary compilers to analyze the codes to
reject.
> > It must be quite good and simple answer that an implementation can
> > reject a prgoram only when its *every* execution results in UB"; given
> > that an implementation already has a good way to say the programmer
> > that his code has a surely problematic construct, I don't see any
> > reasonable reason the standard should deal with such annoying problem.
>
> Nor do I see a reason to force compilers to create meaningful output
> for what can only be either a bug or extremely poor style. As a
> real-world example: GCC treats an indexed string literal as a constant
> (except for "constant expressions"); there is no difference in
> generated code between "return 'c';" and "return "abc"[2];". That is
> very reasonable, and I expect many other compilers behave the same.
> However, there is a bug in that "abc"[2]++ fails to compile. Why must
> that be a bug? What benefit is there in allowing it? What legitimate
> reason is there to do this (other than to test conformance)?
Here I'm not sure I got your point. Are you saying that gcc results in
an internal error since it is required to compile that code
successfully even when given in a place that is never executed?
I also don't see any benefit in allowing such a code to be translated,
but neither in allowing an implementation to reject it if a new
complicated wording must be devised to discern which one should be
allowed and which one should not.
Jun Woong
Jun Woong wrote:
[...]
>
> I also don't see any benefit in allowing such a code to be translated,
Oh, I meant "in requiring" in fact.
Harald van Dijk
> Harald van Dijk wrote:
> > Jun Woong wrote:
> [...]
> > > If you answer to
> > > the DR, what do you write? Just "an implementation is allowed to
> > > reject any progem when it contains a statement that cannot be executed
> > > without invoking undefined behavior," which makes it possible for the
> > > following code to be rejected
> > >
> > > if (user_input())
> > > 1 / 0;
> > >
> > > because the expression statement "1 / 0;" cannot be executed without
> > > undefined behavior?
> >
> > That is exactly the sort of thing I would like to see compilers allowed
> > (to clarify: not required) to reject. However, the wording (that I
> > myself used) is poor because it allows /all/ dead code to cause a
> > program to fail to compile, and there can be legitimate reasons for
> > dead code. I'm not sure what proper wording would be.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> That is my point. I'm afraid that allowing implementations to reject
> codes like yours doesn't deserve a new wording; I think contriving it
> is not easy as it is for ordinary compilers to analyze the codes to
> reject.
I would like to repeat that no compiler would be required to reject
such programs, or even issue a diagnostic. /Any/ conforming C
implementation would also be a conforming not-quite-C implementation.
> > > It must be quite good and simple answer that an implementation can
> > > reject a prgoram only when its *every* execution results in UB"; given
> > > that an implementation already has a good way to say the programmer
> > > that his code has a surely problematic construct, I don't see any
> > > reasonable reason the standard should deal with such annoying problem.
> >
> > Nor do I see a reason to force compilers to create meaningful output
> > for what can only be either a bug or extremely poor style. As a
> > real-world example: GCC treats an indexed string literal as a constant
> > (except for "constant expressions"); there is no difference in
> > generated code between "return 'c';" and "return "abc"[2];". That is
> > very reasonable, and I expect many other compilers behave the same.
> > However, there is a bug in that "abc"[2]++ fails to compile. Why must
> > that be a bug? What benefit is there in allowing it? What legitimate
> > reason is there to do this (other than to test conformance)?
>
> Here I'm not sure I got your point. Are you saying that gcc results in
> an internal error since it is required to compile that code
> successfully even when given in a place that is never executed?
gcc (at least 4.1.1) cannot compile this strictly conforming program,
no matter which options you pass it.
int main(void) {
return 0;
""[0]++;
}
It will always give an error message: error: increment of read-only
location. As things are, this is a bug in gcc (and recognised as such
by its developers), and my point is that in /my/ ideal C-like language,
it would be a feature instead of a bug.
> I also don't see any benefit in allowing such a code to be translated,
> but neither in allowing an implementation to reject it if a new
> complicated wording must be devised to discern which one should be
> allowed and which one should not.
I agree that there are downsides to both, but I don't think it would be
as hard to find correct and agreeable wording as you do (or seem to),
even if I may not be able to come up with it myself, so naturally, my
conclusion is different.
ena8...@yahoo.com
The point of granting the freedom to evaluate subexpressions
in any order is that it can permit more optimization.
Compiler writers that don't want to take advantage of that
can choose to emit code that always follows a single
ordering. This particular flexibility doesn't carry any
burden, since making use of it is completely optional.
ena8...@yahoo.com
There's no value in changing the Standard to allow a compiler
to refuse to translate such a program. There's nothing
stopping a compiler from refusing to translate such a program
right now. Compilers that do refuse just aren't C compilers.
Jun Woong
Harald van Dijk wrote:
[...]
>
> no matter which options you pass it.
>
> int main(void) {
> return 0;
> ""[0]++;
> }
>
> It will always give an error message: error: increment of read-only
> location. As things are, this is a bug in gcc (and recognised as such
> by its developers), and my point is that in /my/ ideal C-like language,
> it would be a feature instead of a bug.
>
Did the developers say that it was not easy to fix it to successfully
translate those problematic codes, or they not feel importance so
decide not to waste their time?
However, gcc is quite better than MSVC in this area; MSVC stops
translating similar codes even when the statement to cause UB given in
a branch of "if" whose condition expression is a constant evaluated to
false.
>
> I agree that there are downsides to both, but I don't think it would be
> as hard to find correct and agreeable wording as you do (or seem to),
The committee would not change their position unless one proves that
requiring such codes to be translated is necessary and *really* a
burden to some implementations, and that the wording to fix it is as
concise as the original one by the committee. As I repeatedly said,
at first glance it never looks easy and useful to find the wording.
Harald van Dijk
> Harald van Dijk wrote:
> [...]
> >
> > gcc (at least 4.1.1) cannot compile this strictly conforming program,
> > no matter which options you pass it.
> >
> > int main(void) {
> > return 0;
> > ""[0]++;
> > }
> >
> > It will always give an error message: error: increment of read-only
> > location. As things are, this is a bug in gcc (and recognised as such
> > by its developers), and my point is that in /my/ ideal C-like language,
> > it would be a feature instead of a bug.
> >
>
> Did the developers say that it was not easy to fix it to successfully
> translate those problematic codes, or they not feel importance so
> decide not to waste their time?
It has been confirmed a bug and a regression, but it hasn't got much
attention otherwise, presumably because there are much more serious
bugs (internal compiler errors, wrong code generation, things like
that).
> However, gcc is quite better than MSVC in this area; MSVC stops
> translating similar codes even when the statement to cause UB given in
> a branch of "if" whose condition expression is a constant evaluated to
> false.
GCC won't compile
int main(void) {
if(0) ""[0]++;
return 0;
}
either. Move it to a separate function which is never called, and it
still won't be compiled.
> >
> > I agree that there are downsides to both, but I don't think it would be
> > as hard to find correct and agreeable wording as you do (or seem to),
>
> The committee would not change their position unless one proves that
> requiring such codes to be translated is necessary and *really* a
> burden to some implementations, and that the wording to fix it is as
> concise as the original one by the committee. As I repeatedly said,
> at first glance it never looks easy and useful to find the wording.
Oh, I don't expect the C standard to be changed even if agreeable
wording is found; it would very likely break a lot of valid code.
Jun Woong
Harald van Dijk wrote:
> Jun Woong wrote:
[...]
> > However, gcc is quite better than MSVC in this area; MSVC stops
> > translating similar codes even when the statement to cause UB given in
> > a branch of "if" whose condition expression is a constant evaluated to
> > false.
>
> GCC won't compile
>
> int main(void) {
> if(0) ""[0]++;
> return 0;
> }
>
> either. Move it to a separate function which is never called, and it
> still won't be compiled.
>
I meant other kinds of the statement to similarly result in UB, for
example, 1 / 0. I think the bug in gcc is not serious; it will be
easily fixed once they make up their mind.
> >
> > The committee would not change their position unless one proves that
> > requiring such codes to be translated is necessary and *really* a
> > burden to some implementations, and that the wording to fix it is as
> > concise as the original one by the committee. As I repeatedly said,
> > at first glance it never looks easy and useful to find the wording.
>
> Oh, I don't expect the C standard to be changed even if agreeable
> wording is found; it would very likely break a lot of valid code.
A lot of valid code? Aren't we talking about the code that is
problematic when it makes its way to the product code unless it is
meant to test an implementation's conformance?
--
Jun, Woong (woong at icu.ac.kr)
Samsung Electronics Co., Ltd.
``All opinions are mine and do not represent any organization''
Harald van Dijk
> > > The committee would not change their position unless one proves that
> > > requiring such codes to be translated is necessary and *really* a
> > > burden to some implementations, and that the wording to fix it is as
> > > concise as the original one by the committee. As I repeatedly said,
> > > at first glance it never looks easy and useful to find the wording.
> >
> > Oh, I don't expect the C standard to be changed even if agreeable
> > wording is found; it would very likely break a lot of valid code.
>
> A lot of valid code? Aren't we talking about the code that is
> problematic when it makes its way to the product code unless it is
> meant to test an implementation's conformance?
Yes. People aren't perfect, mistakes do happen. As long as such code is
never executed, it is allowed, and dead code doesn't get debugged much.
Martin Str|mberg
> GCC won't compile
> int main(void) {
> if(0) ""[0]++;
> return 0;
> }
> either. Move it to a separate function which is never called, and it
> still won't be compiled.
Somebody correct me where I'm wrong:
Isn't it UB to change string literals?
So the problem above is because the ""[0]++ part can never be executed
and thus can't cause UB, so it must be accepted?
Hence even
if(0)
{
*(int *)0 = 42; /* Dereference NULL pointer. */
fprintf(stderr, "%c", '0'); /* Vararg call without prototype. */
}
is ok?
Right,
MartinS
Harald van Dijk
> Harald van D??k <tru...@gmail.com> wrote:
> > GCC won't compile
>
> > int main(void) {
> > if(0) ""[0]++;
> > return 0;
> > }
>
> > either. Move it to a separate function which is never called, and it
> > still won't be compiled.
>
> Somebody correct me where I'm wrong:
>
> Isn't it UB to change string literals?
Right.
> So the problem above is because the ""[0]++ part can never be executed
> and thus can't cause UB, so it must be accepted?
Right.
> Hence even
> if(0)
> {
> *(int *)0 = 42; /* Dereference NULL pointer. */
> fprintf(stderr, "%c", '0'); /* Vararg call without prototype. */
> }
> is ok?
No null pointer will ever be dereferenced, so that is okay. As for
fprintf(), C99 does not support implicit function declarations. In C89,
it is equivalent to the below code.
if(0)
{
*(int *)0 = 42; /* Dereference NULL pointer. */
{
extern int fprintf(); /* Incompatible declaration */
fprintf(stderr, "%c", '0');
}
}
No, that is not okay, because the behaviour is undefined if you even
declare fprintf() (or any other function or object) inappropriately,
regardless of whether you ever use it.
Douglas A. Gwyn
> I cannot distinguish unspecified/undefined behavior, to my
> understanding the "unspecified" behevior is due to the evaluation order
> is not defined.
You need to look up the official definitions.
The idea of *unspecified* behavior is that there are a few
*valid* choices, and the standard does not tell the implementation
which one to choose - but it must choose one. Programs can be
written to do the right thing no matter which choice is made.
The idea of *undefined behavior* is that anything at all might
happen, including abnormal program termination, so programs
cannot trigger undefined behavior if they want to maintain control.
An implementation is free to treat some instances of undefined
behavior as merely unspecified, and choose a deterministic reult,
but they are not required to do so.
The order of evaluation of certain subexpressions is not specified,
but the implementation does have to evaluate them in some order.