Why are VLAs optional in C11?

[Keith Thompson]

C99 introduced VLAs (variable length arrays); a conforming C99
compiler *must* implement them.

C11 made them an optional feature; a conforming C11 compiler can
fail to implement VLAs if it predefines the macro __STDC_NO_VLA__.

BTW, a positive name would have been more
convenient ("#ifdef __STDC_VLA__" is more readable than
"#ifndef __STDC_NO_VLA__"), but I suppose it's too late to
change that.

Why did C11 make VLAs optional?

The obvious answer would be to cater to some compiler implementer
that would be willing to implement C11 *without* VLAs, but considers
implementing C11 *with* VLAs to be too burdensome. Note that full
support for C99 was notoriously slow; were VLAs part of the reason
for that?

Was this the reason for making VLAs optional? If so, what compiler
implementer supported, or would have supported, either C99 or C11
*without* VLAs?

The two non-examples I'm most familiar with are gcc (which does
support VLAs and presumably will continue to do so) and Microsoft
(which has expressed a lack of interest in supporting anything past
C90 *at all*, except perhaps for some features that are also in C++).

Or is there another rationale?

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[Jens Gustedt]

Am 18.08.2012 00:42, schrieb Keith Thompson:
> Why did C11 make VLAs optional?

I vaguely remember reading that the motif was similar as for the
complex data types. There are platforms with specialized compilers
that really don't need VLA (embedded stuff comes into mind). They
shouldn't lag behind in addopting the core features of modern C. The
same strategy has been followed for other new features such as
threads, for example.

Up to now, there was the main difference in hosted and standalone
environments. This classification was perhaps to coarse. With C11, C
becomes a toolbox with optional modules that is build arround some
sort of core language.

It is really unfortunate, that we don't have the rationale document
for C11, yet.

Jens

[Keith Thompson]

Jens Gustedt <jens.g...@loria.fr> writes:
> Am 18.08.2012 00:42, schrieb Keith Thompson:
>> Why did C11 make VLAs optional?
>
> I vaguely remember reading that the motif was similar as for the
> complex data types. There are platforms with specialized compilers
> that really don't need VLA (embedded stuff comes into mind). They
> shouldn't lag behind in addopting the core features of modern C. The
> same strategy has been followed for other new features such as
> threads, for example.

Then let me add to my previous question: Would it have been reasonable
to make VLAs mandatory for hosted implementations but optional for
freestanding implementations?

[...]

[Eric Sosman]

On 8/17/2012 6:42 PM, Keith Thompson wrote:
> C99 introduced VLAs (variable length arrays); a conforming C99
> compiler *must* implement them.
>
> C11 made them an optional feature; a conforming C11 compiler can
> fail to implement VLAs if it predefines the macro __STDC_NO_VLA__.
>
> BTW, a positive name would have been more
> convenient ("#ifdef __STDC_VLA__" is more readable than
> "#ifndef __STDC_NO_VLA__"), but I suppose it's too late to
> change that.

That choice, at least, is clear: Since a C99 compiler is not
under any obligation to define a macro with this indication, a
test

#if __STDC_VLA__

... would come up "false" on a fully-conforming C99 implementation
*with* VLA support. That is, a compiler that conformed to C99 and
had passed any number of certification tests could suddenly become
non-conforming by fiat were __STDC_VLA__ added, ex post facto, to
the conformance requirement.

> Why did C11 make VLAs optional?

More generally, why did C11 abandon the principle expressed
in the Rationale:

By defining both conforming hosted and conforming
Beyond this two-level scheme, no additional subsetting
is defined for C, since the C89 Committee felt strongly
that too many levels dilutes the effectiveness of a
standard.

? 6.10.8.3 introduces one two three four five six seven eight
"conditional feature macros," which along with the hosted vs.
freestanding choice gives

F I V E H U N D R E D T W E L V E

different language subsets, all entitled to call themselves
"fully conforming Standard-compliant dyed-in-the-wool accept-
no-substitutes JenYouWine C."

Embrace, extend, extinguish -- and it's an inside job!

--
Eric Sosman
eso...@ieee-dot-org.invalid

[John Nagle]

On 8/17/2012 3:42 PM, Keith Thompson wrote:
> C99 introduced VLAs (variable length arrays); a conforming C99
> compiler *must* implement them.
>
> C11 made them an optional feature; a conforming C11 compiler can
> fail to implement VLAs if it predefines the macro __STDC_NO_VLA__.
>
> BTW, a positive name would have been more
> convenient ("#ifdef __STDC_VLA__" is more readable than
> "#ifndef __STDC_NO_VLA__"), but I suppose it's too late to
> change that.
>
> Why did C11 make VLAs optional?
>
> The obvious answer would be to cater to some compiler implementer
> that would be willing to implement C11 *without* VLAs, but considers
> implementing C11 *with* VLAs to be too burdensome.

Microsoft has stated their position:

"We have added support for C-99 features (variadic macros, long long)
and we really do listen to our customers (both big and small) and the
overwhelming feedback we get is to focus on C++. Even one of our largest
and most focal group of C customers - developers of device drivers - are
now asking us to make C++ more friendly to the device driver space.

Having said that: 2 C-99 features (VLA's and the ability to mix
declarations and statements) are on the wish list for the next release -
but it all depends on the resources we can dedicate to the compiler -
and there are a lot of other C++-0x features that users want us to support."

Jonathan Caves - Visual C++ Compiler Team

(ref:
http://social.msdn.microsoft.com/Forums/en-US/vcprerelease/thread/6099f453-db2c-49c3-b59a-b799c379cebb)

John Nagle

[John Nagle]

On 8/17/2012 4:19 PM, Jens Gustedt wrote:
> Am 18.08.2012 00:42, schrieb Keith Thompson:
>> Why did C11 make VLAs optional?

...

> It is really unfortunate, that we don't have the rationale document
> for C11, yet.

True. Here are some of the known problems with VLAs:

- Putting arbitrarily large arrays on the stack causes trouble
in multithreaded programs in implementations where stack growth
is bounded.

- There's no way to recover from an out-of-memory condition when
allocating a VLA.

- Microsoft declines to support them.

- VLAs aren't used much. There appear to be only three in
Google Code, and no VLA parameters. The Linux kernel had one,
but it was taken out because there was no way to handle an
out of space condition. (If anyone can find an example of
a VLA parameter in publicly visible production code, please
let me know.)

- The semantics of VLA parameters is painful. They're
automatically reduced to pointers, with the length information
lost. "sizeof" returns the size of a pointer.

- Prototypes of functions with VLA parameters do not have
to exactly match the function definition. This is
incompatible with C++ style linkage and C++ function
overloading, preventing the extension of this feature
into C++.

John Nagle

[Alan Curry]

In article <k0n48g$tsc$1...@dont-email.me>, John Nagle <na...@animats.com> wrote:
>out of space condition. (If anyone can find an example of
>a VLA parameter in publicly visible production code, please
>let me know.)

MPlayer has a few.

svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer

Examples:

command.c:1304
if (mpctx->demuxer->type == DEMUXER_TYPE_TV) {
int l = strlen(prop->name);
char tv_prop[3 + l + 1];
sprintf(tv_prop, "tv_%s", prop->name);
return mp_property_do(tv_prop, action, arg, mpctx);
}

ffmpeg/libavutil/pca.c:73
int ff_pca(PCA *pca, double *eigenvector, double *eigenvalue){
int i, j, pass;
int k=0;
const int n= pca->n;
double z[n];

input/input.c (2 separate locations) (end and iter are local variables)
char name[end-iter+1];
strncpy(name,iter,end-iter);
name[end-iter] = '\0';

libmenu/menu.c:749 (w and h are function parameters)
int stride = (w+7)&(~7); // round to 8
char pic[stride*h],pic_alpha[stride*h];

libmpcodecs/ad_twin.c:507
float out[priv->framesize*sh_audio->channels];

libmpdemux/demux_mov.c:2038
int len=trak->samples[i].size;
char buf[len];

--
Alan Curry

[John Nagle]

On 8/17/2012 10:34 PM, Alan Curry wrote:
> In article <k0n48g$tsc$1...@dont-email.me>, John Nagle <na...@animats.com> wrote:
>> out of space condition. (If anyone can find an example of
>> a VLA parameter in publicly visible production code, please
>> let me know.)
>
> MPlayer has a few.

...

Those are all local VLAs. A VLA parameter looks like

void fn(size_t n, double mat[n][n])
{
}

Seen any of those, anyone?

John Nagle

[Jens Gustedt]

Am 18.08.2012 06:04, schrieb John Nagle:
> True. Here are some of the known problems with VLAs:
>

-- snip --

> - The semantics of VLA parameters is painful.

true, but you yourself with your proposal of reference show a way out
of that, that would make them bearable

> They're
> automatically reduced to pointers, with the length information
> lost. "sizeof" returns the size of a pointer.

technically not true, or only half the truth

only the size for innermost dimension is lost, the other dimensions
come accross and are perfectly accessible through sizeof. The first
dimension (as a sizeof expression) is usually not needed for the
computation inside the function.

And the reason that this dimension is lost is not the fault of VLA,
this dimension is lost for all "array" parameters.

Most disadvantages of VLA disappear if you just use pointers to VLA.

double (*A)[n][m] = malloc(sizeof(double[n][m]));

is easy to read, keeps track of both dimensions, is checkable for
allocation problems and is in no risk of smashing the stack.

If you want to introduce references �-la C++ to make the *use* of them
easier, this would be a strong argument for them.

+++

I have to observe that you reply here in this thread and not in the
other one where I gave you a detailed analysis of your proposal and
where I basically think I have proven that with your proposal you are
doing VLA (or more precisely VM types) anyhow.

Since this is already the second time, should I conclude that you are
more interested in VLA-bashing than in taking your own proposal
forward?

Jens

[Jens Gustedt]

Am 18.08.2012 01:23, schrieb Keith Thompson:
> Jens Gustedt <jens.g...@loria.fr> writes:
>> Am 18.08.2012 00:42, schrieb Keith Thompson:
>>> Why did C11 make VLAs optional?
>>
>> I vaguely remember reading that the motif was similar as for the
>> complex data types. There are platforms with specialized compilers
>> that really don't need VLA (embedded stuff comes into mind). They
>> shouldn't lag behind in addopting the core features of modern C. The
>> same strategy has been followed for other new features such as
>> threads, for example.
>
> Then let me add to my previous question: Would it have been reasonable
> to make VLAs mandatory for hosted implementations but optional for
> freestanding implementations?

Probably not. There will be more and more borderline architectures in
the future that combine aspects of hosted and freestanding
environments. Best example are platforms with GPU. E.g OpenCL is
supposed to implement most elements of C99, but not VLA. It combines
compilation on the host environment and on the GPU device.

And programming for devices will become more specialized. For a device
like GPU that does parallel geometric processing you might want
complex numbers (so you'd support that option), atomics and threads
(maybe the C11 can be made compatible with their lowlevel threads) but
you don't want VLA on the device.

The programming model would be very close of what nowadays is a hosted
environment, but without VLA.

Jens

[Alan Curry]

In article <k0nbeb$ojg$1...@dont-email.me>, John Nagle <na...@animats.com> wrote:
>On 8/17/2012 10:34 PM, Alan Curry wrote:
>> In article <k0n48g$tsc$1...@dont-email.me>, John Nagle
><na...@animats.com> wrote:
>>> out of space condition. (If anyone can find an example of
>>> a VLA parameter in publicly visible production code, please
>>> let me know.)
>>
>> MPlayer has a few.
>...
>
> Those are all local VLAs. A VLA parameter looks like

Oops. I misread the question.

>
> void fn(size_t n, double mat[n][n])
> {
> }
>
>Seen any of those, anyone?

None in any code I have nearby. If anyone wants to try my search,

find . -name '*.c' -exec egrep \
'^[^ /#].*\<([a-zA-Z_][a-zA-Z0-9_]*)\>,.*\[[^]]*\<\1\>' \
/dev/null '{}' +

(there's a space and a tab in the initial negated character class)

gave only 8 false positives out of 20 million lines searched.

--
Alan Curry

[Keith Thompson]

John Nagle <na...@animats.com> writes:
[...]

> - The semantics of VLA parameters is painful. They're
> automatically reduced to pointers, with the length information
> lost. "sizeof" returns the size of a pointer.

That's no different from non-VLA arrays.

[...]

[jacob navia]

Le 18/08/12 06:04, John Nagle a �crit :

> On 8/17/2012 4:19 PM, Jens Gustedt wrote:
>> Am 18.08.2012 00:42, schrieb Keith Thompson:
>>> Why did C11 make VLAs optional?
> ...
>> It is really unfortunate, that we don't have the rationale document
>> for C11, yet.
>
> True. Here are some of the known problems with VLAs:
>
> - Putting arbitrarily large arrays on the stack causes trouble
> in multithreaded programs in implementations where stack growth
> is bounded.
>

This is not specific to VLAs. In general VLAs REDUCE stack growth.

Without VLAs you write:

#define MAXELEMENTS 512

int fn(void)
{
int table[MAXELEMENT];
}


MOST cases will be 10-20, some cases it would go to 100, and the
programmer, careful to avoid overflows decides 512.

With VLAs in most cases you will just use 10-20 elements, not one more
as it is NEEDED. There is no need for an arbitrary maximum.

Under windows, the stack can start with 1MB, if I remember correctly.

> - There's no way to recover from an out-of-memory condition when
> allocating a VLA.
>

Neither is possible to recover when you write:

int fn(void)
{
int foo;

}


> - Microsoft declines to support them.
>

o Microsoft declines to support mixing declarations and code.

o Microsoft did not introduce "long long" into their compiler line until
2009, using before its __int64 keyword.

o Microsoft doesn't support the standard C library emitting a warning
at each call of most library functions "this function is deprecated"

o Microsoft doesn't like the C language at all. Let's close shop then.
Microsoft may have a point. There is no need for a C committee if we
have Microsoft.

WHAT?

You are again MICROSOFT, the software giant? You must be nuts.

> - VLAs aren't used much. There appear to be only three in
> Google Code, and no VLA parameters. The Linux kernel had one,
> but it was taken out because there was no way to handle an
> out of space condition. (If anyone can find an example of
> a VLA parameter in publicly visible production code, please
> let me know.)
>

Just one example:

struct foo;

This is an opaque definition of a data type, published by a library
that keeps the internals hidden.

problem: You can't make a local variable of type "foo" since the size
isn't know. If the library provides a library function

size_t SizeofFoo(void);

You can declare a local variable of type foo with

int FunctionThatUsesAFoo(void)
{
char myFooBuf[SizeofFoo()];
struct foo *myFoo = &myFooBuf;
}

This is VERY important for libraries that hide their internals but want
to allow local variables with their types saving one expensive call to
malloc().

> - The semantics of VLA parameters is painful. They're
> automatically reduced to pointers, with the length information
> lost. "sizeof" returns the size of a pointer.
>

In my implementation I had a REAL sizeof that will return the actual
size. I was told to replace it, but I think I will not.

> - Prototypes of functions with VLA parameters do not have
> to exactly match the function definition. This is
> incompatible with C++ style linkage and C++ function
> overloading, preventing the extension of this feature
> into C++.

I think that adding new features to C++ is a REALLY BAD IDEA (tm).

One of the useful things of VLAs is that they are incompatible with C++
then.

jacob

[Ike Naar]

On 2012-08-18, jacob navia <ja...@spamsink.net> wrote:
> struct foo;
>
> This is an opaque definition of a data type, published by a library
> that keeps the internals hidden.
>
> problem: You can't make a local variable of type "foo" since the size
> isn't know. If the library provides a library function
>
> size_t SizeofFoo(void);
>
> You can declare a local variable of type foo with
>
> int FunctionThatUsesAFoo(void)
> {
> char myFooBuf[SizeofFoo()];
> struct foo *myFoo = &myFooBuf;
> }

Caveat: there can be an alignment mismatch.

> This is VERY important for libraries that hide their internals but want
> to allow local variables with their types saving one expensive call to
> malloc().
>
>> - The semantics of VLA parameters is painful. They're
>> automatically reduced to pointers, with the length information
>> lost. "sizeof" returns the size of a pointer.
>
> In my implementation I had a REAL sizeof that will return the actual
> size.

The actual size of what? If, inside the function body, the parameter
has been reduced to a pointer, then applying sizeof to such a
pointer should, of course, return the size of the pointer.

If your sizeof, applied to a pointer, does not yield the size of the
pointer, it is a broken sizeof.

Or is it the case that you do not let the array decay to a pointer,
so that sizeof is applied to an array?

[John Nagle]

Thanks. Can anyone else try it on large volumes of
code they have? So far, no one has been able to find even
one VLA parameter in production code anywhere. I'd really
like to find at least one.

I've tried Google Code Search (it's still live, but only
covers code stored in Google Code), and found no VLA parameters.
VLA arrays showed up in one game (once), the Erlang library
(twice), and the GCC test suite (once). There was one VLA
in the Linux kernel, but it was removed as a risk to
kernel stack overflow.

(Krugle's and Ohloh also have code search tools on line,
but neither allows a regular expression, so they're not
useful here.)


John Nagle

[Ike Naar]

On 2012-08-18, Alan Curry <pac...@kosh.dhis.org> wrote:
> None in any code I have nearby. If anyone wants to try my search,
>
> find . -name '*.c' -exec egrep \
> '^[^ /#].*\<([a-zA-Z_][a-zA-Z0-9_]*)\>,.*\[[^]]*\<\1\>' \
> /dev/null '{}' +
>
> (there's a space and a tab in the initial negated character class)
>
> gave only 8 false positives out of 20 million lines searched.

There may be false negatives.
The find command finds

void g(size_t n, /* ... , */ double a[n])

but not

void g(size_t n,
/* ... , */
double a[n])

[Keith Thompson]

jacob navia <ja...@spamsink.net> writes:
> Le 18/08/12 06:04, John Nagle a écrit :

>> On 8/17/2012 4:19 PM, Jens Gustedt wrote:
>>> Am 18.08.2012 00:42, schrieb Keith Thompson:
>>>> Why did C11 make VLAs optional?
>> ...
>>> It is really unfortunate, that we don't have the rationale document
>>> for C11, yet.
>>
>> True. Here are some of the known problems with VLAs:
>>
>> - Putting arbitrarily large arrays on the stack causes trouble
>> in multithreaded programs in implementations where stack growth
>> is bounded.
>>
>
> This is not specific to VLAs. In general VLAs REDUCE stack growth.
>
> Without VLAs you write:
>
> #define MAXELEMENTS 512
>
> int fn(void)
> {
> int table[MAXELEMENT];
> }
>
>
> MOST cases will be 10-20, some cases it would go to 100, and the
> programmer, careful to avoid overflows decides 512.
>
> With VLAs in most cases you will just use 10-20 elements, not one more
> as it is NEEDED. There is no need for an arbitrary maximum.

Without VLAs, I'd probably use malloc().

VLAs can either increase or reduce stack growth, depending on how you
use them.

[...]

>> - Microsoft declines to support them.

[...]

> o Microsoft doesn't like the C language at all. Let's close shop then.
> Microsoft may have a point. There is no need for a C committee if we
> have Microsoft.
>
> WHAT?
>
> You are again MICROSOFT, the software giant? You must be nuts.

This isn't about anybody being against anybody. Microsoft exists and
is significant. There are other C compilers for Windows, but using
one of them may not be an option for some projects (for example,
management might insist on using only Microsoft development tools).
Conclusion: *some* C programmers (those who need to use Microsoft
C compilers) are unable to use most C99 and C11 features. I don't
like it any more than you do, but sarcasm isn't going to change it --
and it may be quite relevant to this discussion.

>> - VLAs aren't used much. There appear to be only three in
>> Google Code, and no VLA parameters. The Linux kernel had one,
>> but it was taken out because there was no way to handle an
>> out of space condition. (If anyone can find an example of
>> a VLA parameter in publicly visible production code, please
>> let me know.)
>>
>
> Just one example:
>
> struct foo;
>
> This is an opaque definition of a data type, published by a library
> that keeps the internals hidden.
>
> problem: You can't make a local variable of type "foo" since the size
> isn't know. If the library provides a library function
>
> size_t SizeofFoo(void);
>
> You can declare a local variable of type foo with
>
> int FunctionThatUsesAFoo(void)
> {
> char myFooBuf[SizeofFoo()];
> struct foo *myFoo = &myFooBuf;
> }

Using malloc instead of an automatic char array would avoid alignment
problems.
[...]

>> - The semantics of VLA parameters is painful. They're
>> automatically reduced to pointers, with the length information
>> lost. "sizeof" returns the size of a pointer.
>>
>
> In my implementation I had a REAL sizeof that will return the actual
> size. I was told to replace it, but I think I will not.

You should at least fix it. With the version of lcc-win64 I installed
on June 9, this program:

#include <stdio.h>

void func(int n, int param[n]) {
printf("sizeof param = %zu\n", sizeof param);
}

int main(void) {
enum { LEN = 10 };
int obj[LEN];
printf("sizeof obj = %zu\n", sizeof obj);
func(LEN, obj);
return 0;
}

produces this output:

sizeof obj = 40
sizeof param = 10

A conforming C99 or C11 compiler must compute `sizeof param` to
be the same as `sizeof (int*)`, since param is a pointer. If,
as you suggest, `sizeof param` should instead be the size of the
argument, then it should be 40 rather than 10. If you intend it to
be the number of elements rather than the size in bytes, I suggest
introducing a new implementation-defined operator, perhaps something
like `_Lengthof`. (It's between you and your customers to decide
whether lcc-win needs to be conforming.)

(Incidentally, `...` is the Markdown syntax for code samples; see
<http://en.wikipedia.org/wiki/Markdown>. I think I'm going to start
using it here.)

[...]

[jacob navia]

Le 18/08/12 22:48, Keith Thompson a écrit :

I wrote:
>> o Microsoft doesn't like the C language at all. Let's close shop then.
>> > Microsoft may have a point. There is no need for a C committee if we
>> > have Microsoft.
>> >
>> > WHAT?
>> >
>> > You are again MICROSOFT, the software giant? You must be nuts.

Then Mr Thompson answered:

> This isn't about anybody being against anybody.

That's your opinion. I am definitely against the policy of microsoft
of degrading the support of standard C and against their opinion
that is they that rule what windows users should use as a language.

> Microsoft exists

I am not denying that... :-)

> and
> is significant.

I am just denying that. 640K should be OK for everyone said someone
years ago. No, that opinion wasn't significant even if microsoft was
a software giant then. For me having a lot of money doesn't make you
right, sorry. You snipped all other arguments (declarations at any
place, making the whole standard C library "deprecated") how convenient
for you.

> There are other C compilers for Windows, but using
> one of them may not be an option for some projects (for example,
> management might insist on using only Microsoft development tools).

And so what?

If management insists on using "MICROSOFT ONLY" why should be
declare that ALL OTHERS even those users that do not use Microsoft
(Apple programmers, iOS programmers, Android programmers,
Sun programmers, and a VERY long list) should have a language
that is defined by Microsoft?

> Conclusion:*some* C programmers (those who need to use Microsoft
> C compilers) are unable to use most C99 and C11 features.

Life is tough when you have to use Microsoft, I know. But if life
is tough for people using Microsoft, at least do not generalize and
make it tough for everyone!

> I don't
> like it any more than you do, but sarcasm isn't going to change it --
> and it may be quite relevant to this discussion.
>

I believe that there is nothing about Microsoft that is relevant here
since WHATEVER WE DO Microsoft will NOT follow it since it has given
up supporting C as a language. That is a fact, that doesn't change if
they say somewhere in a blog that if they would have more resources
they *could* implement some standard C feature from a standard that is
12 years old.

The committee tried to please Microsoft and declared VLA's optional.

Great.

The result?

Has Microsoft changed anything about its C99 support?

Please, let's get real.

[Keith Thompson]

jacob navia <ja...@spamsink.net> writes:
> Le 18/08/12 22:48, Keith Thompson a écrit :
> I wrote:
>>> o Microsoft doesn't like the C language at all. Let's close shop then.
>>> > Microsoft may have a point. There is no need for a C committee if we
>>> > have Microsoft.
>>> >
>>> > WHAT?
>>> >
>>> > You are again MICROSOFT, the software giant? You must be nuts.
>
> Then Mr Thompson answered:
>
>> This isn't about anybody being against anybody.
>
> That's your opinion. I am definitely against the policy of microsoft
> of degrading the support of standard C and against their opinion
> that is they that rule what windows users should use as a language.

I'm not disputing your opinion. I actually agree with much of it.
I'm suggesting that *this discussion* isn't about being for or
against Microsoft policies.

>> Microsoft exists
>
> I am not denying that... :-)
>
>> and
>> is significant.
>
> I am just denying that. 640K should be OK for everyone said someone
> years ago. No, that opinion wasn't significant even if microsoft was
> a software giant then. For me having a lot of money doesn't make you
> right, sorry. You snipped all other arguments (declarations at any
> place, making the whole standard C library "deprecated") how convenient
> for you.

I snipped things that I wasn't commenting on, either because I agreed
with them or because I just didn't have anything to say about them.
Inferring some ulterior motive from that is a mistake.

Are you under the impression that I *approve* of Microsoft not
supporting mixed declarations and statements in C and deprecrating
some functions in the standard C library? If so, please indicate
when and where I said that so I can clarify it.

The fact that you and I both happen to disagree with certain
Microsoft policies does not imply that Microsoft is insignificant.

For the record, I would prefer it if Microsoft would fully support
C99 in its C compiler, and fully support C11 as soon as practical.
But since I don't run Microsoft, and am only barely a Microsoft user,
I don't get a vote. They make decisions for both technical and
financial reasons; they've made the business decision to concentrate
their resources on C++ rather than C.

Please stop mistaking my statement about what they've done for
agreement with it.

>> There are other C compilers for Windows, but using
>> one of them may not be an option for some projects (for example,
>> management might insist on using only Microsoft development tools).
>
> And so what?
>
> If management insists on using "MICROSOFT ONLY" why should be
> declare that ALL OTHERS even those users that do not use Microsoft
> (Apple programmers, iOS programmers, Android programmers,
> Sun programmers, and a VERY long list) should have a language
> that is defined by Microsoft?

Where on Earth are you getting this?

I did not say or imply that all users should be limited to what
Microsoft supports. What I said, which you actually quoted, was:

>> Conclusion:*some* C programmers (those who need to use Microsoft
>> C compilers) are unable to use most C99 and C11 features.

*Some* C programmers. Those who *need* to use Microsoft C compilers.
What is unclear about that?

(At my previous job, we used Microsoft's compiler. Using a different
compiler was not an option, at least not for me. It was C++ and C#,
not C, so it's not *entirely* relevant, but there are programmers
who have no choice but to use Microsoft development environments.
This is not a value judgement, it is a statement of fact.)

> Life is tough when you have to use Microsoft, I know. But if life
> is tough for people using Microsoft, at least do not generalize and
> make it tough for everyone!

I don't.

>> I don't
>> like it any more than you do, but sarcasm isn't going to change it --
>> and it may be quite relevant to this discussion.

[snip]

> The committee tried to please Microsoft and declared VLA's optional.

Has someone on the committee actually said that it was done to please
Microsoft? If so, I missed it -- which is exactly why I started this
discussion. My hope is that someone *on the C committee* will explain
why VLAs were made optional in C11.

[snip]

Incidentally, there was an lcc-win bug report in my previous article.
Did you see it? Would you like me to resubmit it through other
channels?

[Pierre Asselin]

John Nagle <na...@animats.com> wrote:
> On 8/18/2012 1:29 AM, Alan Curry wrote:

> > [ how to look for VLAs in prototypes ]

> Thanks. Can anyone else try it on large volumes of
> code they have? So far, no one has been able to find even
> one VLA parameter in production code anywhere. I'd really
> like to find at least one.

I don't have to search. I had to write some numerical code at work
two years ago and I used C99 features, VLAs among them, just to
see if they are any good. Work for hire, proprietary, can't show
it, sorry.

The VLAs are *extraordinarily* useful. First as a replacement
for alloca(). Second, to convert between flat and multidimensional
arrays.

float flat[3*nx*ny*nz];
float (*structured)[ny][nz][3]= (void *)flat;
float *other_flat= &structured[0][0][0][0];

That sort of thing. And, as you ask, in function prototypes.

int fou(size_t n, size_t m, float A[/* n */][m]);

(Not a VLA parameter, since there are no array parameters,
but very much a pointer to VLA parameter.)

--
pa at panix dot com

[jacob navia]

Le 19/08/12 03:14, Keith Thompson a écrit :

> Incidentally, there was an lcc-win bug report in my previous article.
> Did you see it? Would you like me to resubmit it through other
> channels?

yes, I am missing multiplying by the size of the element of the array in
your example sizeof int.

[Keith Thompson]

You're welcome, glad I could help.

(Any comment on the fact that it's non-conforming?)

[jacob navia]

Le 19/08/12 12:07, Keith Thompson a écrit :

> jacob navia <ja...@spamsink.net> writes:
>> Le 19/08/12 03:14, Keith Thompson a écrit :
>>> Incidentally, there was an lcc-win bug report in my previous article.
>>> Did you see it? Would you like me to resubmit it through other
>>> channels?
>>
>> yes, I am missing multiplying by the size of the element of the array in
>> your example sizeof int.
>
> You're welcome, glad I could help.
>
> (Any comment on the fact that it's non-conforming?)
>

1) This is a (now) optional feature. So, I think it is better when you
call the sizeof operator to have the size of the table and not the size
of some pointer because of obscure C rules that impose the lost of
information that can be given to the user.

2) This is my personal opinion, but this proposal from Mr Nagle shows
that it would be interesting to NOT throw this info away.

[John Nagle]

The problem is the semantics of fixed-size array parameters
(but not local fixed-size arrays). For those, too, the size of the
array is just the size of a pointer. VLA parameters just repeat
that mess from fixed-size arrays.

The underlying problem is that early C barely had arrays at
all, just pointers. Square brackets were originally just
syntactic sugar for pointer arithmetic. As C arrays acquired
somewhat stronger semantics, the new semantics were added in
ways that were backwards compatible with old code. The
result is that C arrays are second-class objects.

This is what led me to propose references to variable
size arrays as an addition to C. A reference to an array
has full array semantics; it's not reduced to a pointer
without size information. It looks like the design
in my paper

http://www.animats.com/papers/languages/safearraysforc36.pdf

would work; it's been criticized by about ten people and
the major holes have been plugged. It's also something
that could potentially be extended to C++.

But the existing VLA parameter mechanism is in the way.
That's why I'm interested in finding out how widespread
the use of that feature is. So far, nobody has been able
to find a single instance of a VLA parameter in publicly
visible code, although one person says they used it
in their own code at least once.

As I keep pointing out, this is the source of most
buffer overflows. The language doesn't know how big most
arrays are, and sometimes the programmer isn't quite sure.
A big fraction of program crashes and security holes come
from this fact. So it's worth fixing.

John Nagle

[Keith Thompson]

John Nagle <na...@animats.com> writes:
[...]

> The problem is the semantics of fixed-size array parameters
> (but not local fixed-size arrays). For those, too, the size of the
> array is just the size of a pointer. VLA parameters just repeat
> that mess from fixed-size arrays.

No, the problem with fixed-size array parameters is that *there is no
such thing*. It's not true that "the size of the array is just the size
of the pointer". The size of the pointer is the size of the pointer.

void func(double arr[100]);

is nothing more than syntactic sugar (or perhaps syntactic vinegar)
for this:

void func(double *arr);

I agree that it's a problem. My point is that any solution needs to
account for the existing syntax and semantics. You propose to introduce
something like array parameters, and that's fine, but I don't think
changing the semantics of any existing declarations is feasible.

I haven't read your entire proposal. Do you propose that, given this
parameter declaration:

double arr[N]

the *type* of arr depends on whether N is a constant expression? If so,
I suggest that that would cause too much confusion, aside from being
incompatible with C90, C99, and C11.

[...]

> But the existing VLA parameter mechanism is in the way.

Is it? I haven't read your entire proposal, but aren't your new array
(or array-like) parameters implemented in terms of C++ish references?
If you can define your new features without changing the semantics of
*any* existing code, that would be a tremendous advantage.

Can you provide a concrete example of a function declaration whose
semantics you would change? (Perhaps you've already done so, but this
is a big thread.)

[...]

And one more or less random thing: in the section on multidimensional
arrays, I don't see anything new. The example you give is:

size_t order = 3;
...
float tab[order][order];// square array of order N.

which is a perfectly legal VLA in C99 and later. It's restricted to
block scope; if you're proposing allowing it at file scope, you don't
mention it.

[Keith Thompson]

Keith Thompson <ks...@mib.org> writes:
> John Nagle <na...@animats.com> writes:
> [...]
>> The problem is the semantics of fixed-size array parameters
>> (but not local fixed-size arrays). For those, too, the size of the
>> array is just the size of a pointer. VLA parameters just repeat
>> that mess from fixed-size arrays.
>
> No, the problem with fixed-size array parameters is that *there is no
> such thing*. It's not true that "the size of the array is just the size
> of the pointer". The size of the pointer is the size of the pointer.
>
> void func(double arr[100]);
>
> is nothing more than syntactic sugar (or perhaps syntactic vinegar)
> for this:
>
> void func(double *arr);

[...]

John, perhaps you're approaching this from the wrong direction.

You seem to be starting with the idea that `void func(double
arr[100])` defines `arr` as a parameter of array type (but
incidentally, C treats it as a parameter of pointer type).

Instead, I suggest starting with the idea that C does not have
parameters of pointer type (but incidentally, it permits pointer
parameters to be defined with array syntax).

That might make it easier to propose a consistent mechanism for
adding array or array-like parameters to the language.

[Owen Shepherd]

John Nagle wrote:
> This is what led me to propose references to variable
> size arrays as an addition to C. A reference to an array
> has full array semantics; it's not reduced to a pointer
> without size information. It looks like the design
> in my paper
>
> http://www.animats.com/papers/languages/safearraysforc36.pdf
>
> would work; it's been criticized by about ten people and
> the major holes have been plugged. It's also something
> that could potentially be extended to C++.

Regarding the above, is there any reason for not reusing the C++11 for
each syntax

...
for(auto i : obj)

considering it is reasonably clear, concise, won't conflict with any
existing code, and would maintain orthogonality with C++?

That said, I can't see the C++ standards committee ever adopting it.
Their position - and mine as a C++ user - is that you should use
std::array<>. Generally, C++ doesn't add features which can be
implemented equally well in the standard library (hence why C++11
doesn't adopt, say, _Complex; it has std::complex<>)

[Jens Gustedt]

Am 27.08.2012 23:17, schrieb Owen Shepherd:
> Generally, C++ doesn't add features which can be
> implemented equally well in the standard library (hence why C++11
> doesn't adopt, say, _Complex; it has std::complex<>)

which in that case is really a pity, because I think that "double
_Complex" has a bit of different semantics than "std::complex<double>".
The C type it is guaranteed not to have padding, which I think is
difficult to guarantee for any (straight) library implementation in C++,
isn't it?

(And again one of the missed oportunities of increasing overlap between
the two languages.)

Jens

[John Nagle]

One problem is that many common APIs (notably Windows and Linux)
use C-type bindings. When you have to pass an array to a system call,
a raw C pointer has to be extracted from the collection and passed
to an interface that doesn't know C++. At that point, size information
has been lost, and the possibility of a buffer overflow appears.
C++ doesn't fix the problem, it just wallpapers over it with templates.
All too often, the mold seeps through the wallpaper.

Another problem is that C++ collection classes don't have efficient
subscript checking. Some implementations do offer subscript checking
of collection classes. But because the check is just a template
expansion, it has to be done where the template says to do it - on
every reference. The compiler doesn't have the authority to haul
the check out of the loop when possible and do it once at loop
entrance.

Example:

std::array<int,100> tab;
for (int i=0; i<200; i++) { tab[i] = 0; }

With checking enabled, this will produce an error. But the
subscript check will be made on every iteration. It won't be
hoisted out of the loop during optimization. The compiler
doesn't know that it would be OK to do this once at the
top of the loop.

However, what I've been proposing in the safe array
area isn't addressed to C++. It's addressed to C. I do
want to propose only changes that could be potentially
added to C++, because the languages should not diverge
unnecessarily. (And because Microsoft takes the position
that they won't support features in C that can't go into
C++. After all, today, most compilers for C are really C++
compilers with some features turned off.)

John Nagle

[James Kuyper]

On 08/27/2012 10:36 PM, John Nagle wrote:
...

> Another problem is that C++ collection classes don't have efficient
> subscript checking. Some implementations do offer subscript checking
> of collection classes. But because the check is just a template
> expansion, it has to be done where the template says to do it - on
> every reference. The compiler doesn't have the authority to haul
> the check out of the loop when possible and do it once at loop
> entrance.

Sure it does - the as-if rule always applies. If the compiler can be
certain that a check will be passed, it can always eliminate that check.

> Example:
>
> std::array<int,100> tab;
> for (int i=0; i<200; i++) { tab[i] = 0; }

Shouldn't that be tab.at(i), in order to invoke subscript checking?
Personally, I'd use std::fill_n(tab.begin(), 200, 0);
--
James Kuyper

[John Nagle]

On 8/27/2012 7:54 PM, James Kuyper wrote:
> On 08/27/2012 10:36 PM, John Nagle wrote:
> ...
>> Another problem is that C++ collection classes don't have efficient
>> subscript checking. Some implementations do offer subscript checking
>> of collection classes. But because the check is just a template
>> expansion, it has to be done where the template says to do it - on
>> every reference. The compiler doesn't have the authority to haul
>> the check out of the loop when possible and do it once at loop
>> entrance.
>
> Sure it does - the as-if rule always applies. If the compiler can be
> certain that a check will be passed, it can always eliminate that check.

Efficient optimization requires that the problem be reported early.
It's often possible to hoist subscript checks to the top of a loop.
But then, the check will fail before the first iteration, even if
many iterations could be completed before overflow. Potentially,
each iteration could call a function which might potentially do
a longjmp and escape the loop before a subscript error occurred.
To allow this optimization, the compiler must have permission
to report a constraint violation as soon as it becomes
inevitable. And "inevitable" must exclude exiting the
loop via longjmp, exit, abort, or uncaught signal, which
are events that may be triggered in another translation
unit the compiler cannot currently see. I discuss that in my
paper on safe arrays.

Thus, a compiler must know that constraint checks get
special handling to do them efficiently.

There are some papers on this subject. This one has a
good list of references:

http://www.classes.cs.uchicago.edu/archive/2004/fall/32630-1/papers/p270-kolte.pdf

Most of the existing work is for FORTRAN, Pascal, or COBOL.
I'm unaware of any C implementation with optimized subscript
checking.

>> Example:
>>
>> std::array<int,100> tab;
>> for (int i=0; i<200; i++) { tab[i] = 0; }
>
> Shouldn't that be tab.at(i), in order to invoke subscript checking?
> Personally, I'd use std::fill_n(tab.begin(), 200, 0);

Oh, the subscript form in C++ collection classes doesn't have
subscript checking. Right. This sort of thing is why the compiler
should optimize checking, so you can afford to check all the time.
This reinforces my point.

John Nagle

[pdx_s...@yahoo.com]

On Saturday, August 18, 2012 at 5:33:19 AM UTC-7, jacob navia wrote:

> In my implementation [of VLA parameters] I had a REAL sizeof that will return the actual

> size. I was told to replace it, but I think I will not.

An implementation of sizeof that returns something other than the size of the pointer when given a VLA parameter violates the fundamental rules of C pointer arithmetic. You cannot pass an array to a function in C, except by wrapping a struct or union around it. Confusingly, C allows you to write function parameters that -appear- to be arrays, e.g.

int main(int argc, char *argv[]) // misleading declaration: argv is NOT an array; rather, it's merely a vector

The compiler knows that an array cannot be passed directly to a function because any time an expression of type array of X appears in expression context (e.g. the actual parameter in a function call), the compiler promotes the type to pointer to X by taking the address of the first element. So when the compiler sees an array type in a parameter list, it silently re-writes it to what it really is:

int main(int argc, char **argv)

This is a throw-back to pre-ANSI C when the compiler would also silently re-write formal parameters of type "char" and "short" to "int" and "float" to "double". It's too bad ANSI C didn't disallow writing parameters that appear incorrectly as array types, but the fact that they didn't made VLA parameters possible.

In my example, regardless of which declaration I use for argv, argv is a pointer in every way. For example, we can say ++argv. We wouldn't be able to increment it if it were an array.

The VLA parameter is similarly silently rewritten as a pointer type, and just as above, when we have:

void f(int n, int a[n])

We can say ++a inside the function.

[Keith Thompson]

"pdx_s...@yahoo.com" <pdx_s...@yahoo.com> writes:
> On Saturday, August 18, 2012 at 5:33:19 AM UTC-7, jacob navia wrote:
>
>> In my implementation [of VLA parameters] I had a REAL sizeof that will return the actual
>> size. I was told to replace it, but I think I will not.
>
> An implementation of sizeof that returns something other than the size
> of the pointer when given a VLA parameter violates the fundamental
> rules of C pointer arithmetic.

[...]

Agreed.

Are you aware that you just replied to an article that was posted 10
years ago?

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for Philips
void Void(void) { Void(); } /* The recursive call of the void */

[Tomasz Stanislawski]

> void f(int n, int a[n])
>
> We can say ++a inside the function.

There is some debate if it is an actual VLA type. The standard requires to transform the declaration of `int a[n]` to `int* a`.
To my understanding, the original VLA type is erased during at function's *definition* and its top-level size expression should not never be evaluated.
The situation is different while passing a pointer to an array:

```
void f(int n, int (*a)[n]) { ... }
```

Now, `n` must be evaluated at every invocation of the function.