Macro replacement interpretation

[Edward Diener]

Section 16.3 paragraph 4 of the C++11 standard reads:

"If the identifier-list in the macro definition does not end with an
ellipsis, the number of arguments (including those arguments consisting of
no preprocessing tokens) in an invocation of a function-like macro shall
equal the number of parameters in the macro definition. Otherwise, there
shall be more arguments in the invocation than there are parameters in the
macro definition (excluding the ...). There shall exist a ) preprocessing
token that terminates the invocation."

Does this mean that invoking a macro with more arguments than there are
parameters in the identifier-list in the macro definition is not a
preprocessing error ?

Does this mean that invoking a macro with less arguments than there are
parameters in the identifier-list in the macro definition is a
preprocessing error ?

My testing with the major C++ compilers on Windows and Linux, including
gcc, clang, Intel, and Oracle but leaving out VC++ completely because its
preprocessor is so C++ standard non-conformant, shows that error messages
are usually produced if the number of arguments does not equal the number
of parameters in the identifier-list, but this is not always the case (
most notably with Oracle depending on the version ). This suggest that the
answer to my first question is NO and the answer to my second question is
YES. But I want to know the official answer on whether either situation
should be considered a compiler error or whether just a warning is
justified.



--
[ comp.std.c++ is moderated. To submit articles, try posting with your ]
[ newsreader. If that fails, use mailto:std-cpp...@vandevoorde.com ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html ]

[James Kuyper]

You can't get an official answer here - anyone can post their personal
opinion on the matter here, even me. An official answer requires the
filing of DR (Defect Report) with the C++ committee. Note that DRs can
be used even if you don't believe there's a defect - when used in this
fashion, they should properly be called RFIs (Request For
Interpretation), but ISO didn't bother creating a separate RFI process.

At one time, posting a message containing "Defect Report" in the
Subject: header to this newsgroup would result in someone from the
committee creating an actual defect report for you. I'm not sure whether
they're still doing that (hopefully the relevant person will respond to
this message by confirming it), but it's worth a try.

Note: I used to be an expert in C++98; I've not kept up with all of the
changes that were made in C++2003 and C++2011, but this is a rather
elementary issue by comparison. You're not likely to get a well-informed
response from this newsgroup that significantly disagrees with the
responses I gave you on this same topic in comp.lang.c++.moderated. I
won't rule out nit-picking, however.

For the benefit of those not monitoring c.l.c.m, my response was to
point out that
1. The standard does not distinguish between error messages and warning
messages, it talks only about diagnostics. The standard requires that
the method for distinguishing diagnostic messages from other messages
generated by the translation process be documented, but otherwise
imposes no constraints on the contents of those messages.

2. When a program contains at least one violation of a diagnosable rule,
at least one diagnostic message is required, but in general, the
behavior in such cases is otherwise undefined. Therefore, in particular,
the implementation is free to continue translating the program,
interpreting the rule violation in any manner it chooses.

3. I cited 1.4p8: "A conforming implementation may have extensions
(including additional library functions), provided they do not alter the
behavior of any well-formed program. Implementations are required to
diagnose programs that use such extensions that are ill-formed according
to this International Standard. Having done so, however,
they can compile and execute such programs."

[Jakob Bohm]

On 26/03/2015 19:09, Edward Diener wrote:

>
> Section 16.3 paragraph 4 of the C++11 standard reads:
>
> "If the identifier-list in the macro definition does not end with an

> ellipsis, the number of arguments (including those arguments consisting o=

f
> no preprocessing tokens) in an invocation of a function-like macro shall
> equal the number of parameters in the macro definition. Otherwise, there

> shall be more arguments in the invocation than there are parameters in th=

e
> macro definition (excluding the ...). There shall exist a ) preprocessing
> token that terminates the invocation."
>
> Does this mean that invoking a macro with more arguments than there are
> parameters in the identifier-list in the macro definition is not a
> preprocessing error ?
>

No, the first sentence explicitly says that for a regular macro with
a regular argument list, it is an error to pass less arguments and an
error to pass more arguments.

The second sentence explicitly says that for a varargs macro (one whose
#define specifies an argument list ending in "...)") it is an error not
to provide *at least* one argument for the "..." part.

If that quotation is the complete specification, it seems that there is
no syntax to define a macro accepting 0 or more parameters, and no
syntax to name the last mandatory parameter. Thus it might be
cumbersome to use the standard syntax to implement
int printf(const char *fmt, ...) as a macro that invokes
int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...).

Now before the C and C++ committees published a (final) standard for
how to use "..." in a macro argument list definition, many compilers
implemented their own syntax for doing the same. Therefore those
compilers now have a need (at least when not running in special
"standard C/C++ only" modes) to continue to accept their historic
syntax for an extended period of time (e.g. 10 years after correctly
implementing the standard syntax in a released compiler).




Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 S=C3=B8borg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[Francis Glassborow]

What is with the ten years? Any implementation can, AFAIK, an extension for
as long as its makers wish as long as they issue a diagnostic message when
such an extension is used.

Note that a compiler is, IIUC, only (an optional) part of an
implementation. An implementation can run source code via an interpreter.
That can even be just for some part of the code.

The complete tool set (including Standard libraries) is an implementation
so could we use correct terminology here (where we are dealing with the C++
Standard a degree of pedantry seems appropriate)

Now the pedants can correct my ignorance :(


--

[Edward Diener]

Agreed. No function-like macro can accept 0 parameters.

, and no

> syntax to name the last mandatory parameter.
>

I do not know what is meant by this. Why would one need a "name" for the
last mandatory parameter ? For any given macro the name of the last
mandatory parameter is evident.

Thus it might be
> cumbersome to use the standard syntax to implement
> int printf(const char *fmt, ...) as a macro that invokes
> int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...).
>

If the ... might have 0 parameters then the macro is:

int printf(...)

and the first __VAR_ARGS__ parameter is the const char *fmt. The Boost PP
library has functionality to get the size of variadic data or get any
element of the variadic data.

> Now before the C and C++ committees published a (final) standard for
> how to use "..." in a macro argument list definition, many compilers
> implemented their own syntax for doing the same. Therefore those
> compilers now have a need (at least when not running in special
> "standard C/C++ only" modes) to continue to accept their historic
> syntax for an extended period of time (e.g. 10 years after correctly
> implementing the standard syntax in a released compiler).
>

Yes, reasonable.

My problem with the diagnostic message is that the message rarely says what
is the C++ standard syntax as opposed to the alternate syntax that is being
allowed as a compiler extension. Therefore the alternate syntax becomes the
C++ standard syntax for many programmers. But I understand that this is a
problem that is outside the realm of the C++ standard itself. Once the C++
standard "allowed" non-standard syntax with diagnostics to still be
standard conforming then it is up to the compilers what the diagnostic
message entails.


--

[Francis Glassborow]

On 3/30/2015 7:21 PM, Edward Diener wrote:

> My problem with the diagnostic message is that the message rarely says what
> is the C++ standard syntax as opposed to the alternate syntax that is being
> allowed as a compiler extension. Therefore the alternate syntax becomes the
> C++ standard syntax for many programmers. But I understand that this is a
> problem that is outside the realm of the C++ standard itself. Once the C++
> standard "allowed" non-standard syntax with diagnostics to still be
> standard conforming then it is up to the compilers what the diagnostic
> message entails.
>
>
>

But the C++ Standard says no such thing. What it requires is that the
implementation tell the programmer if it finds something that is contrary
to the Standard (except in the cases where it explicitly says 'no
diagnostic required'). After an implementation has done that the source
code is no longer conforming C++ and so the Standard can say nothing about
it and make no requirements. How could it say anything else? It has
determined that the code is not C++, that is it. If an implementer then
decides that they are willing to continue that is their decision but it has
nothing to do with Standard C++.

All the Standard can say is that a conforming implementation may not
silently accept non-conforming source code.

You may not like this and think it may lead to poor understanding amongst
progranners but it is not the job of either the Standard or an
implementation to educate programmers.

Now what maybe a problem is that the actual requirement is that an
implementation shall emit at least one diagnostic message for any TU that
contains any non-conforming source code. It does not require that it
identify the problematic code, nor does it require that it be the first
instance of a problem. It could, IIUC, accept the whole TU, emit object
code and finally emit a diagnostic such as 'This TU contains at least one
instance of non-conforming code which this implementation has chosen a to
accept.' It is only QoI that makes such a diagnostic purely academic.

Finally consider:

extern "myC++" {
// whatever I want
}

I think that as long as the implementer has documented the meaning of
'extern "myC++" ' they do not have to even issue a diagnostic for any
extensions they elect to support. Now that is nasty and fortunately no
implementer has gone down that route.

Francis

[James Kuyper]

On 03/30/2015 02:21 PM, Edward Diener wrote:
>
> On 3/29/2015 5:01 AM, Jakob Bohm wrote:
>
>>
>> On 26/03/2015 19:09, Edward Diener wrote:
>>
>>
>>> Section 16.3 paragraph 4 of the C++11 standard reads:

...

>>> equal the number of parameters in the macro definition. Otherwise, there
>>> shall be more arguments in the invocation than there are parameters in
>>> th=

...

>> If that quotation is the complete specification, it seems that there is
>> no syntax to define a macro accepting 0 or more parameters

...

>> , and no
>> syntax to name the last mandatory parameter.
>>
>
> I do not know what is meant by this. Why would one need a "name" for the
> last mandatory parameter ? For any given macro the name of the last
> mandatory parameter is evident.

"Otherwise, there shall be more arguments in the invocation than there

are parameters in the macro definition." In this context, "otherwise
refers to function-like macros with "..." in the parameter list. Note
the "shall" - the behavior is undefined unless there is at least one
argument that has no corresponding named parameter. That first extra
argument is therefore mandatory (at least, if you wish to avoid
undefined behavior); it is also the last mandatory argument (at least as
far as C is concerned - the expansion of the function-like macro itself
may impose additional requirements).

>> Thus it might be
>> cumbersome to use the standard syntax to implement
>> int printf(const char *fmt, ...) as a macro that invokes
>> int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...).
>>
>
> If the ... might have 0 parameters then the macro is:
>
> int printf(...)

I'm not sure what the above is supposed to be. You call it a macro, yet
there's an 'int' rather than a #define at the front, which makes it
appear to be a declaration. The declaration for printf is

int printf(const char * restrict format, ...);

In order for printf() to be defined as macro that calls
fprintf_private() while allowing it be called with only a format string
(as is legal), it would have to look something like this:

// Remove any pre-existing #define of printf() by the implementation.
#undef printf
#define printf(...) fprintf_private(stdout, \\
count_va_args(#__VA_ARGS__), __VA_ARGS__)

If it weren't done that way, printf("Hello World!") would violate
16.3p4, so fmt cannot be passed as the second argument of
fprintf_private(), as originally specified by Jakob Bohm. It would have
to be passed along with the variable part of the argument list.

[Jakob Bohm]

Agreed

> Thus it might be
>>> cumbersome to use the standard syntax to implement
>>> int printf(const char *fmt, ...) as a macro that invokes
>>> int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...).
>>>
>>>
>> If the ... might have 0 parameters then the macro is:
>>
>> int printf(...)
>>
>
> I'm not sure what the above is supposed to be. You call it a macro, yet
> there's an 'int' rather than a #define at the front, which makes it
> appear to be a declaration. The declaration for printf is
>
> int printf(const char * restrict format, ...);
>

Interesting. So the following is no longer standard conformant (due to
restrict):

{
const char *x = "Hello %s\n";
printf(x, x); /* "Hello Hello %s\n\n" */

}


> In order for printf() to be defined as macro that calls
> fprintf_private() while allowing it be called with only a format string
> (as is legal), it would have to look something like this:
>
> // Remove any pre-existing #define of printf() by the implementation.
> #undef printf
> #define printf(...) fprintf_private(stdout, \\
> count_va_args(#__VA_ARGS__), __VA_ARGS__)
>
>

Naah, I wanted to pass the length of the format string (in chars), not
the argument count, to illustrate use of the actual last mandatory
argument.

If it weren't done that way, printf("Hello World!") would violate
> 16.3p4, so fmt cannot be passed as the second argument of
> fprintf_private(), as originally specified by Jakob Bohm. It would have
> to be passed along with the variable part of the argument list.
>

Which was exactly my point. In contrast, the standard mechanism
for handling varargs in functions (as opposed to macros), does use
a name for the last mandatory argument as illustrated by the printf()
prototype above.

For example (imagine this is part of the standard library in a
hosted implementation):

int printf(const char* restrict fmt, ...)
{
va_list args;
int res;

va_start(args, fmt);
res = _vprintf_private(stdout, fmt, strlen(fmt), args);
va_end(args);

return res;
}


And yes, the example is contrived to illustrate the limitation.

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 S=C3=B8borg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[James Kuyper]

On 03/31/2015 05:39 PM, Francis Glassborow wrote:
...

> Finally consider:
>
> extern "myC++" {
> // whatever I want
> }
>
> I think that as long as the implementer has documented the meaning of
> 'extern "myC++" ' they do not have to even issue a diagnostic for any
> extensions they elect to support. Now that is nasty and fortunately no
> implementer has gone down that route.

I don't think so - that already matches the current syntax for language
linkage.
7.5p1 says "Some of the properties associated with an entity with
language linkage are specific to each implementation and are not
described here." but other properties of language linkage are discussed
in the standard, so an implementation's freedom to implement extensions
by use of that syntax is extremely limited.
7.5p2 says "Use of a string-literal other than "C" or "C++" is
conditionally supported, with implementation-defined semantics. [ Note:
Therefore, a linkage-specification with a string literal that is unknown
to the implementation requires a diagnostic. =E2=80=94end note ]"
Note that it is only the semantics that are implementation-defined; that
clause does not permit a supported language linkage to redefine the
syntax rules.

--
James Kuyper

[James Kuyper]

On 04/01/2015 06:38 PM, Jakob Bohm wrote:
>
> On 31/03/2015 23:38, James Kuyper wrote:
>
>>
>> On 03/30/2015 02:21 PM, Edward Diener wrote:
>>
>>>
>>> On 3/29/2015 5:01 AM, Jakob Bohm wrote:

...

>>>> Thus it might be
>>>> cumbersome to use the standard syntax to implement
>>>> int printf(const char *fmt, ...) as a macro that invokes
>>>> int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...).

...

>> In order for printf() to be defined as macro that calls
>> fprintf_private() while allowing it be called with only a format string
>> (as is legal), it would have to look something like this:
>>
>> // Remove any pre-existing #define of printf() by the implementation.
>> #undef printf
>> #define printf(...) fprintf_private(stdout, \\
>> count_va_args(#__VA_ARGS__), __VA_ARGS__)
>>
>>
> Naah, I wanted to pass the length of the format string (in chars), not
> the argument count, to illustrate use of the actual last mandatory
> argument.

In that case, then it's much simpler: replace count_va_args() with
sizeof or strlen(). The only clue in your message was the fact that the
parameter's name was "fmtlen".

> Which was exactly my point.

I was responding to Edward Diener, explaining your point. I often do
that if the person who made the point hasn't gotten around to responding
yet. I apologize if you would have preferred to explain it yourself - I
sometimes get impatient - but sometimes such arguments carry more weight
if they are made by multiple people. Sometimes it helps to explain the
same thing multiple different ways.


--

[Edward Diener]

That is fine but the meaning of your terminology "and no syntax to name the
last mandatory parameter" still eludes me. Are you saying that the
paragraph in the standard should have been phrased differently ?

> Thus it might be
>>> cumbersome to use the standard syntax to implement
>>> int printf(const char *fmt, ...) as a macro that invokes
>>> int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...).
>>>
>>>
>> If the ... might have 0 parameters then the macro is:
>>
>> int printf(...)
>>
>
> I'm not sure what the above is supposed to be. You call it a macro, yet
> there's an 'int' rather than a #define at the front, which makes it
> appear to be a declaration.
>

I was responding to the "standard syntax to implement
int printf(const char *fmt, ...) as a macro" etc. I assume this meant
something like:

#define printf(fmt,...)

and the suggestion that it might be cumbersome comes from the fact that a
varying argument function does not need any arguments to be passed to the
function for the ellipsis but that a variadic macro must have at least one
argument passed when it specifies the variadic parameter syntax of '...'.

My solution to that would be to have the variadic function be:

#define printf(...) etc.

where the first variadic argument is the 'fmt' and the optional variadic
arguments would correspond to the ellipsis arguments. The variadic macro
support in Boost PP makes it possible to extract any variadic parameter
from the variadic as well as determine the number of variadic parameters of
the variadic data. So while it might be cumbersome to use the standard
syntax it is not impossible to do so.

The declaration for printf is
>
> int printf(const char * restrict format, ...);
>
> In order for printf() to be defined as macro that calls
> fprintf_private() while allowing it be called with only a format string
> (as is legal), it would have to look something like this:
>
> // Remove any pre-existing #define of printf() by the implementation.
> #undef printf
> #define printf(...) fprintf_private(stdout, \\
> count_va_args(#__VA_ARGS__), __VA_ARGS__)
>
> If it weren't done that way, printf("Hello World!") would violate
> 16.3p4, so fmt cannot be passed as the second argument of
> fprintf_private(), as originally specified by Jakob Bohm. It would have
> to be passed along with the variable part of the argument list.
>

I do not believe your syntax above does work, but then I am not sure what
the syntax of fprintf_private is supposed to be. From your previous post
you specified it as:

int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...)

but that suggests that 'fmtlen' is a required argument also. In your
implementation above I assume 'stdout' is the argument for 'FILE * f' but I
don't see an argument for 'fmt'. I am guessing that
'count_va_args(#__VA_ARGS__)' is the argument for 'fmtlen' and that
'fmtlen' is the number of varying arguments from 0 to whatever it is.

Assuming that you are trying to create a 'printf' macro to invoke 'int
fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...)', where
'stdout' is hardcoded as 'FILE *f' and 'fmtlen' is the number of varying
arguments, the Boost PP syntax would be:

#include <boost/preprocessor/comparison/equal.hpp>
#include <boost/preprocessor/control/iif.hpp>
#include <boost/preprocessor/tuple/enum.hpp>
#include <boost/preprocessor/tuple/po_front.hpp>
#include <boost/preprocessor/variadic/elem.hpp>
#include <boost/preprocessor/variadic/size.hpp>
#include <boost/preprocessor/variadic/to_tuple.hpp>


#define printf(...) \
BOOST_PP_IIF \
( \
BOOST_PP_EQUAL(BOOST_PP_VARIADIC_SIZE,1), \
printf_SINGLE, \
printf_MULTIPLE \
) \
(__VA_ARGS__)

#define printf_SINGLE(...) \
fprintf_private \
( \
stdout, \
BOOST_PP_VARIADIC_ELEM(0,__VA_ARGS__), \
0 \
)

#define printf_MULTIPLE \
fprintf_private \
( \
stdout, \
BOOST_PP_VARIADIC_ELEM(0,__VA_ARGS__), \
BOOST_PP_VARIADIC_SIZE(__VA_ARGS__), \
BOOST_PP_TUPLE_ENUM \
( \
BOOST_PP_TUPLE_POP_FRONT \
( \
BOOST_PP_VARIADIC_TO_TUPLE(__VA_ARGS__) \
) \
) \
)

I agree this is not as simple as allowing an end-user to not pass an
argument for variadic data as gcc does, but it is certainly doable via
Boost PP. Maybe "cumbersome" is an apt word unless one knows Boost PP macro
programming <g>.

[Francis Glassborow]

On 4/2/2015 8:24 PM, James Kuyper wrote:

>
> I was responding to Edward Diener, explaining your point. I often do
> that if the person who made the point hasn't gotten around to responding
> yet. I apologize if you would have preferred to explain it yourself - I
> sometimes get impatient - but sometimes such arguments carry more weight
> if they are made by multiple people. Sometimes it helps to explain the
> same thing multiple different ways.
>
>

And I, for one, appreciate that so please continue. Several minds trying to
express the same or similar points are often more effective than just one.

Francis

[James Kuyper]

I'm only talking about how the standard is actually phrased; I was not
discussing any possible changes to it.
The phrase "no syntax to name the last mandatory parameter" was Jakob
Bohm's, not mine. A better way to say it is that "there is no syntax to
provide a parameter name corresponding to the last mandatory argument".
Simplistic example:

#define macro(first_parameter, second_parameter, ...) \
first_parameter + second_parameter + (__VA_ARGS__)

The smallest permitted number of arguments for this macro is three, so
the last mandatory argument is the third one, for which there is no
corresponding parameter name.

>> Thus it might be
>>>> cumbersome to use the standard syntax to implement
>>>> int printf(const char *fmt, ...) as a macro that invokes
>>>> int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...).
>>>>
>>>>
>>> If the ... might have 0 parameters then the macro is:
>>>
>>> int printf(...)
>>>
>>
>> I'm not sure what the above is supposed to be. You call it a macro, yet
>> there's an 'int' rather than a #define at the front, which makes it
>> appear to be a declaration.
>>
>
> I was responding to the "standard syntax to implement
> int printf(const char *fmt, ...) as a macro" etc. I assume this meant
> something like:
>
> #define printf(fmt,...)

printf("Hello world!") is perfectly legal, but would violate 16.3p4 if
the macro were implemented that way. Since the macro is supposed to be a
replacement for the actual function, that makes this method of
implementing it unacceptable.

> and the suggestion that it might be cumbersome comes from the fact that a

It's not just cumbersome - it has undefined behavior when printf() is
called with a single argument.

> varying argument function does not need any arguments to be passed to the
> function for the ellipsis but that a variadic macro must have at least one
> argument passed when it specifies the variadic parameter syntax of '...'.
>
> My solution to that would be to have the variadic function be:
>
> #define printf(...) etc.

And, as indicated above, dealing with that issue produces the result
that the last mandatory argument (the format string, in this case) has
no corresponding parameter name.

>> int printf(const char * restrict format, ...);
>>
>> In order for printf() to be defined as macro that calls
>> fprintf_private() while allowing it be called with only a format string
>> (as is legal), it would have to look something like this:
>>
>> // Remove any pre-existing #define of printf() by the implementation.
>> #undef printf
>> #define printf(...) fprintf_private(stdout, \\
>> count_va_args(#__VA_ARGS__), __VA_ARGS__)
>>
>> If it weren't done that way, printf("Hello World!") would violate
>> 16.3p4, so fmt cannot be passed as the second argument of
>> fprintf_private(), as originally specified by Jakob Bohm. It would have
>> to be passed along with the variable part of the argument list.
>>
>
> I do not believe your syntax above does work, but then I am not sure what

You're correct, which was the point I was making. As I said above: "fmt
cannot be passed as the second argument of fprintf_private()". It has to
be moved to the third argument to make this work.

> the syntax of fprintf_private is supposed to be. From your previous post
> you specified it as:
>
> int fprintf_private(FILE *f, const char *fmt, size_t fmtlen, ...)

Since the format string doesn't have it's own name, it can't be
separated out from the rest of the variable arguments of the macro,
(I'm ignoring you comments about using Boost, since I'm talking
exclusively about C++, not about any additional libraries that might be
available) which means that it must immediately precede the variable
arguments in the function call. The declaration would have to be
rearranged accordingly:

int fprintf_private(FILE *f, size_t fmtlen, const char *fmt, ...);