escaped commas in macros

[dgutson .]

There are situations in which the cpp wrongly takes commas separating
template arguments as macro arguments delimiters.
For example:

template <class X, class Y> struct S;
#define Macro(arg) S arg;

The extra ( ) workaround used in C cannot be used here:
Macro((<int, int>));

This is important for X-macros for example.

I propose to enhance the cpp to be able to ignore escaped commas (with
\), so this can be done:
Macro(<int\, int>);

We already implemented this extension in gcc (patch to be sent on
Monday) which is working.
What we did is: comma escaping occurs at the top level, but not inside ( ).
For example
SomeMacro(f(a\,b))
does no escape the comma and thus has current behavior.

Feedback?

Thanks,

Daniel.

--
Who’s got the sweetest disposition?
One guess, that’s who?
Who’d never, ever start an argument?
Who never shows a bit of temperament?
Who's never wrong but always right?
Who'd never dream of starting a fight?
Who get stuck with all the bad luck?

[Greg Marr]

On Friday, October 9, 2015 at 12:20:22 AM UTC-4, dgutson wrote:

There are situations in which the cpp wrongly takes commas separating
template arguments as macro arguments delimiters.

This issue was discussed at the beginning of last year.  One of the big sticking points mentioned in that thread is that the preprocessor behavior is inherited from C, and so would have to be changed there first.

https://groups.google.com/a/isocpp.org/forum/?fromgroups&pli=1#!topic/std-proposals/_ayMs1oDeMs 

[dgutson .]

But the issue only arises in C++, and the preprocessor is already different.

>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[David Krauss]

On 2015–10–09, at 12:20 PM, dgutson . <daniel...@gmail.com> wrote:

There are situations in which the cpp wrongly takes commas separating
template arguments as macro arguments delimiters.
For example:

template <class X, class Y> struct S;
#define Macro(arg) S arg;

The extra ( ) workaround used in C cannot be used here:
   Macro((<int, int>));

Solution A:

#define EXPAND( ARG ) ARG

#define Macro( ARG ) S EXPAND ARG

Macro((<int, int>))

Solution B:

#define Macro( ... ) S __VA_ARGS__

Macro(<int, int>)

Solution C:

#define COMMA ,

Macro(<int COMMA int>)

This is important for X-macros for example.

What is an X-macro?

I propose to enhance the cpp to be able to ignore escaped commas (with
\), so this can be done:
   Macro(<int\, int>);

We already implemented this extension in gcc (patch to be sent on
Monday) which is working.
What we did is: comma escaping occurs at the top level, but not inside ( ).
For example
   SomeMacro(f(a\,b))
does no escape the comma and thus has current behavior.

I think I still prefer solutions A and B.

[Lingxi Li]

On Friday, October 9, 2015 at 2:22:02 PM UTC+8, David Krauss wrote:

I think I still prefer solutions A and B.

How about defining a general grouping macro like

#define ARG(...) __VA_ARGS__

So, the usage would be

Macro(ARG(<int, int>))

[Roman Perepelitsa]

On Fri, Oct 9, 2015 at 8:21 AM, David Krauss <pot...@gmail.com> wrote:

On 2015–10–09, at 12:20 PM, dgutson . <daniel...@gmail.com> wrote:

There are situations in which the cpp wrongly takes commas separating
template arguments as macro arguments delimiters.
For example:

template <class X, class Y> struct S;
#define Macro(arg) S arg;

The extra ( ) workaround used in C cannot be used here:
   Macro((<int, int>));

Solution A:

#define EXPAND( ARG ) ARG

#define Macro( ARG ) S EXPAND ARG

Macro((<int, int>))

Solution B:

#define Macro( ... ) S __VA_ARGS__

Macro(<int, int>)

Solution C:

#define COMMA ,

Macro(<int COMMA int>)

Solution D:

  // If x is (expr), expands to expr.

  // Otherwise expands to x.

  #define UNPARENTHESIZE_IF_PARENTHESIZED(x) /* complicated */

  #define MACRO(type) UNPARENTHESIZE_IF_PARENTHESIZED(type)

  MACRO(int) => int

  MACRO((int)) => int

  MACRO((pair<int, int>)) => pair<int, int>

The implementation of UNPARENTHESIZE_IF_PARENTHESIZED is non-trivial. I believe it also requires a compiler extension (the last one in https://gcc.gnu.org/onlinedocs/gcc/Variadic-Macros.html#Variadic-Macros).

It's possible to writing macros that behave well in the presence of arguments with commas. It's just hard.

Roman Perepelitsa.

[David Krauss]

On 2015–10–09, at 5:20 PM, Lingxi Li <lilin...@gmail.com> wrote:

How about defining a general grouping macro like

#define ARG(...) __VA_ARGS__

So, the usage would be

Macro(ARG(<int, int>))

This macro is what I meant to say for solution A. The approach is very similar, call it A’. Any macro expecting a template argument list might take a parenthesized argument as a courtesy. For surprising cases you can DIY.

On 2015–10–09, at 5:28 PM, Roman Perepelitsa <roman.pe...@gmail.com> wrote:

The implementation of UNPARENTHESIZE_IF_PARENTHESIZED is non-trivial.

Please, do share.

I believe it also requires a compiler extension (the last one in https://gcc.gnu.org/onlinedocs/gcc/Variadic-Macros.html#Variadic-Macros).

I’ve proposed this for standardization in N4220. Unfortunately, it’s not been reviewed.

It's possible to writing macros that behave well in the presence of arguments with commas. It's just hard.

It’s hard to optionally strip parens automatically. It’s easy to expect one level of parens and strip them. It’s also easy to use __VA_ARGS__, albeit ugly.

Bah, my cursed brain went and solved the riddle. I don’t see any need for an extension.

#define UNPAREN(...) UNPAREN __VA_ARGS__
#define DONE_UNPAREN

#define CAT_LIT(A, ...) A ## __VA_ARGS__
#define CAT(A, ...) CAT_LIT( A, __VA_ARGS__ )
#define UNPARENTHESIZE_IF_PARENTHESIZED(...) CAT( DONE_, UNPAREN __VA_ARGS__  )

http://coliru.stacked-crooked.com/a/3578969a94bb175a

I’d almost call this trivial. CAT/CAT_LIT is a common utility needed for any preprocessor metaprogramming.

[Lingxi Li]

I fail to see the point in implementing this UNPARENTHESIZE_IF_PARENTHESIZED thing. The original goal is to pass argument containing commas as a single argument. So, suppose there is a function macro that takes three arguments

#define TERNARY(a, b, c) ...

So how could UNPARENTHESIZE_IF_PARENTHESIZED help with this goal?

TERNARY(UNPARENTHESIZE_IF_PARENTHESIZED(...), UNPARENTHESIZE_IF_PARENTHESIZED(...), UNPARENTHESIZE_IF_PARENTHESIZED(...))

Like this? If so, I don't think it is any better than my ARG macro, except that it is much more complex. Or you mean to implement the trick for every function macro that wants this functionality? I don't think there would be many people thinking that it's worth the effort.

[Roman Perepelitsa]

On Fri, Oct 9, 2015 at 1:59 PM, David Krauss <pot...@gmail.com> wrote:

On 2015–10–09, at 5:20 PM, Lingxi Li <lilin...@gmail.com> wrote:

How about defining a general grouping macro like

#define ARG(...) __VA_ARGS__

So, the usage would be

Macro(ARG(<int, int>))

This macro is what I meant to say for solution A. The approach is very similar, call it A’. Any macro expecting a template argument list might take a parenthesized argument as a courtesy. For surprising cases you can DIY.

On 2015–10–09, at 5:28 PM, Roman Perepelitsa <roman.pe...@gmail.com> wrote:

The implementation of UNPARENTHESIZE_IF_PARENTHESIZED is non-trivial.

Please, do share.

  // Requires Boost.Preprocessor.

  #include "boost/boost/preprocessor.hpp"

  // Expands to 1 if called without args. Otherwise expands to 0.

  // I believe this can't be implemented in pure C or C++ without extensions.

  #define IS_EMPTY(...) IS_EMPTY_I(__VA_ARGS__)

  #define IS_EMPTY_I(...) \

    BOOST_PP_EQUAL(1, BOOST_PP_VARIADIC_SIZE(_, ##__VA_ARGS__))

  // Expands to 1 if the input is parenthesized. Otherwise expands to 0.

  #define IS_PARENTHESIZED(...) IS_EMPTY(BOOST_PP_EAT __VA_ARGS__)

  // If the input is parenthesized, removes the parentheses. Otherwise expands to

  // the input unchanged.

  #define UNPARENTHESIZE_IF_PARENTHESIZED(...)               \

    BOOST_PP_IF(IS_PARENTHESIZED(__VA_ARGS__), BOOST_PP_REM, \

                BOOST_PP_EMPTY())                            \

    __VA_ARGS__

Note that I'm not an expert in preprocessor. I came up with this macro when I needed it, and it works fine for my use cases. It might have flaws that I don't see, and there might be a simpler implementation.

I believe it also requires a compiler extension (the last one in https://gcc.gnu.org/onlinedocs/gcc/Variadic-Macros.html#Variadic-Macros).

I’ve proposed this for standardization in N4220. Unfortunately, it’s not been reviewed.

It's possible to writing macros that behave well in the presence of arguments with commas. It's just hard.

It’s hard to optionally strip parens automatically. It’s easy to expect one level of parens and strip them. It’s also easy to use __VA_ARGS__, albeit ugly.

Bah, my cursed brain went and solved the riddle. I don’t see any need for an extension.

#define UNPAREN(...) UNPAREN __VA_ARGS__
#define DONE_UNPAREN

#define CAT_LIT(A, ...) A ## __VA_ARGS__
#define CAT(A, ...) CAT_LIT( A, __VA_ARGS__ )
#define UNPARENTHESIZE_IF_PARENTHESIZED(...) CAT( DONE_, UNPAREN __VA_ARGS__  )

http://coliru.stacked-crooked.com/a/3578969a94bb175a

I’d almost call this trivial. CAT/CAT_LIT is a common utility needed for any preprocessor metaprogramming.

This implementation removes leading parentheses: UNPARENTHESIZE_IF_PARENTHESIZED((int)42) expands to `int 42`, while we need `(int)42`.

On Fri, Oct 9, 2015 at 2:55 PM, Lingxi Li <lilin...@gmail.com> wrote:

I fail to see the point in implementing this UNPARENTHESIZE_IF_PARENTHESIZED thing. The original goal is to pass argument containing commas as a single argument. So, suppose there is a function macro that takes three arguments

#define TERNARY(a, b, c) ...

So how could UNPARENTHESIZE_IF_PARENTHESIZED help with this goal?

TERNARY(UNPARENTHESIZE_IF_PARENTHESIZED(...), UNPARENTHESIZE_IF_PARENTHESIZED(...), UNPARENTHESIZE_IF_PARENTHESIZED(...))

Like this? If so, I don't think it is any better than my ARG macro, except that it is much more complex. Or you mean to implement the trick for every function macro that wants this functionality? I don't think there would be many people thinking that it's worth the effort.

UNPARENTHESIZE_IF_PARENTHESIZED (or UIP for short) should be used inside the implementation of TERNARY. Here's an example of a macro that accepts a C++ type as one of the arguments. In order to supports types with commas, such as pair<int, int>, it passes the type argument through UIP.

  // Instead of writing `int x;` you can write `VAR(int, x)`.

  #define VAR(type, name) UIP(type) name;

  VAR(int, x) => int x;

  VAR((pair<int, int>), x) => pair<int, int> x;

Roman Perepelitsa.

[David Krauss]

On 2015–10–09, at 9:50 PM, Roman Perepelitsa <roman.pe...@gmail.com> wrote:

  // Requires Boost.Preprocessor.

  #include "boost/boost/preprocessor.hpp"

  // Expands to 1 if called without args. Otherwise expands to 0.

  // I believe this can't be implemented in pure C or C++ without extensions.

  #define IS_EMPTY(...) IS_EMPTY_I(__VA_ARGS__)

  #define IS_EMPTY_I(...) \

    BOOST_PP_EQUAL(1, BOOST_PP_VARIADIC_SIZE(_, ##__VA_ARGS__))

  // Expands to 1 if the input is parenthesized. Otherwise expands to 0.

  #define IS_PARENTHESIZED(...) IS_EMPTY(BOOST_PP_EAT __VA_ARGS__)

  // If the input is parenthesized, removes the parentheses. Otherwise expands to

  // the input unchanged.

  #define UNPARENTHESIZE_IF_PARENTHESIZED(...)               \

    BOOST_PP_IF(IS_PARENTHESIZED(__VA_ARGS__), BOOST_PP_REM, \

                BOOST_PP_EMPTY())                            \

    __VA_ARGS__

Note that I'm not an expert in preprocessor. I came up with this macro when I needed it, and it works fine for my use cases. It might have flaws that I don't see, and there might be a simpler implementation.

http://coliru.stacked-crooked.com/a/3578969a94bb175a

This implementation removes leading parentheses: UNPARENTHESIZE_IF_PARENTHESIZED((int)42) expands to `int 42`, while we need `(int)42`.

Yes, this shortcoming renders it fairly useless for expressions. It’s only suitable for typenames and things that can’t start with “(”.

Unfortunately, your implementation doesn’t work on GCC. The newest, most-conforming version of the ,##__VA_ARGS__ extension erases the comma only when the final comma was absent from the macro argument list. For a macro parameter list (...), there is no comma to go missing, and the extension is never applied. This is what’s proposed in N4220.

I’m not aware of any way to determine that an argument is not empty. A macro can accept only empty arguments and error-out otherwise, but that’s no better than always requiring user-supplied parens.

Anyhoo… when your macro does work, it’s better than solution A, and I still like solution A better than the proposal.

[dgutson .]

On Fri, Oct 9, 2015 at 3:21 AM, David Krauss <pot...@gmail.com> wrote:
>
> On 2015–10–09, at 12:20 PM, dgutson . <daniel...@gmail.com> wrote:
>
> There are situations in which the cpp wrongly takes commas separating
> template arguments as macro arguments delimiters.
> For example:
>
> template <class X, class Y> struct S;
> #define Macro(arg) S arg;
>
> The extra ( ) workaround used in C cannot be used here:
> Macro((<int, int>));
>
>
> Solution A:
>
> #define EXPAND( ARG ) ARG
> #define Macro( ARG ) S EXPAND ARG
>
> Macro((<int, int>))
>
> Solution B:
>
> #define Macro( ... ) S __VA_ARGS__
>
> Macro(<int, int>)

Sorry, I didn't mention that one of the drivers/goals of this proposal
is to enable the interaction of existing macros (or an existing
preprocessor library) without having to modify it. Therefore only the
call site can be controlled.

>
> Solution C:
>
> #define COMMA ,
> Macro(<int COMMA int>)

This can work but is not reader-friendly.

>
> This is important for X-macros for example.
>
>
> What is an X-macro?

An old technique:
https://en.wikipedia.org/wiki/X_Macro

>
> I propose to enhance the cpp to be able to ignore escaped commas (with
> \), so this can be done:
> Macro(<int\, int>);
>
> We already implemented this extension in gcc (patch to be sent on
> Monday) which is working.
> What we did is: comma escaping occurs at the top level, but not inside ( ).
> For example
> SomeMacro(f(a\,b))
> does no escape the comma and thus has current behavior.
>
>
> I think I still prefer solutions A and B.

I prefer the escaped comma :)

>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> Visit this group at
> http://groups.google.com/a/isocpp.org/group/std-proposals/.

[dgutson .]

Elegant, but I still prefer the escaped comma for readability. Consider:

#define mymacro(T, name) T name

template<class X, class Y>
struct S{};

#define ARG(...) __VA_ARGS__

mymacro(ARG(S<int, int>), s);


versus

mymacro(S<int\, int>, s);

(without extra macros such as ARG() or COMMA).

FWIW, it was a 5 lines patch.

>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> Visit this group at
> http://groups.google.com/a/isocpp.org/group/std-proposals/.

[David Krauss]

On 2015–10–10, at 12:41 AM, dgutson . <daniel...@gmail.com> wrote:

FWIW, it was a 5 lines patch.

I wrote a patch to do macro pre-expansion normally on ,##__VA_ARGS__. On GCC, it was ~10 lines. On Clang, ~100. Preprocessor implementations are wildly different. (On my personal one, it was 1 line. Just sayin’.)

[Moritz Klammler]

> I propose to enhance the cpp to be able to ignore escaped commas
> (with \), so this can be done:
>
> Macro(<int\, int>);
>
> We already implemented this extension in gcc (patch to be sent on
> Monday) which is working.
>
> What we did is: comma escaping occurs at the top level, but not inside
> ( ). For example
>
> SomeMacro(f(a\,b))
>
> does no escape the comma and thus has current behavior.

I'm very much in favor of this!

Some people have posted smart ways to work around the problem with
today's features but many of them are non-trivial and complicated to
understand and apply correctly.

The cleanest solution so far, in my opinion, has been proposed by
Lingxi Li.

#define PROTECT( ... ) __VA_ARGS__

MACRO ( PROTECT( < int, int > ) )

David Krauss' solution I find also okay but somewhat less readable.

#define COMMA ,

MACRO ( < int COMMA int > )

Some other solutions I honestly don't fully understand.

The biggest problem I see with this is that the temptation to do silly
things that pollute the scarce macro name-space like

#define A( ... ) __VA_ARGS__

or

#define C ,

is much too high. In a public header file, I wouldn't even want to
`#define` names like `PROTECT` or `COMMA` and using the above solutions
with names like `MY_PROJECT_INTERNAL_COMMA` quickly becomes unwieldy.

While both solutions are elegant by today's standards, I do think that
they require more understanding of the pre-processor's parsing than a
casual user should need to have in order to do a seemingly simple thing
like passing a template instantiation as a macro argument.

The proposed solution of using a backslash escape is easy to understand
and straight-forward to apply. In fact, I imagine that it might be what
many people would intuitively expect and try when they first encounter
the problem.

As far as I can tell, the proposed addition is at no danger of breaking
anything because a backslash outside of string or character literals has
no defined meaning today. It also does not seem to unreasonably waste
empty spots in the already thin grammar space. If, in the future, the
backslash would be given any meaning for the C++ language, the obvious
consequence would be to allow using `\\` in macro arguments to escape
a backslash per se.

C compatibility is very important for the CPP. I cannot see any reason
this addition would endanger it. If the C standards decides to adopt it
in a future revision, good. If not, and for the time being, every valid
CPP construct by the C standard ought to remain to also be one by the
C++ standard.

Should the feature prove useful, further extensions could be to also
allow escaping unbalanced parenthesis or quotation marks. I'm
hesitating to suggest this, however, because I cannot really think of a
valid use-case but only of many horrible abuses of passing these tokens
around as macro arguments.

Implementing the suggested feature in the pre-processor might not be as
trivial as a five line patch for all CPP implementations out there but I
do believe that it should be possible to modify any reasonably designed
CPP easily enough for this feature to be supported.

As it seems to me, this feature could be so useful that if GCC started
supporting it, people would demand it so much that Clang would follow
shortly after. Then we would have a de facto standard. We could
equally well make it a de jure one right away, thereby avoiding
portability headache.

The CPP is not receiving much love in the C++ community; and right so.
However, one reason it is so little loved is because it is so awkward to
use at times. Making it a little less awkward -- as long as it does not
come at the cost of harming the C++ language itself -- would be a Good
Thing in my opinion.


Moritz

[Bengt Gustafsson]

The problem with all of these approaches is that it gets more or less noisy at the call site.

If the proposal is to change the CPP anyway, why not put the annotation where it belongs: At the #define of the macro. When you write the define you know that a certain macro

parameter is the name of a class and the annotation to avoid ending the actual at commas between <> should be in the define. For instance like this:

#define MY_DECLARATION(typename B, C)    B C;

MY_DECLARATION(map<string, int>, name)

now works as expected.

There may be use cases where you want to send the template parameter list only, but I think those are quite rare by comparison, and then you can design the macro to take <string, int> as the actual instead of string, int.

What are the remaining use cases where this would not work?

[David Krauss]

On 2015–10–13, at 4:01 PM, Bengt Gustafsson <bengt.gu...@beamways.com> wrote:

The problem with all of these approaches is that it gets more or less noisy at the call site.

Yes.

Also, they all don’t encapsulate very well. Commas revert to being ordinary once the unpacked parameter is passed as the argument to another macro. Scalability is sacrificed unless you defer unpacking as long as possible, as in my Solution A. (Not in Lingxi’s caller-side version, though, which unpacks before passing.)

What are the remaining use cases where this would not work?

You’ve attacked the problem of type-ids, but left out expressions and braced lists.

The preprocessor recognizes () as parens, but C++ additionally has <>, {}, and [], any of which can enclose a comma within an expression. (The square brackets express lambda captures.)

[Matthew Woehlke]

On 2015-10-13 04:01, Bengt Gustafsson wrote:
> The problem with all of these approaches is that it gets more or less noisy
> at the call site.
>
> If the proposal is to change the CPP anyway, why not put the annotation
> where it belongs: At the #define of the macro. When you write the define
> you know that a certain macro
> parameter is the name of a class and the annotation to avoid ending the
> actual at commas between <> should be in the define. For instance like this:
>
> #define MY_DECLARATION(typename B, C) B C;
>
> MY_DECLARATION(map<string, int>, name)
>
> now works as expected.

Why not:

#deffun MY_DECLARATION(A, B)

...and perform argument splitting as if MY_DECLARATION were a regular
C++ function and not a macro? This would preclude feeding it arguments
with unbalanced tokens, but how often does that happen in practice? (We
still would have #define and the previously mentioned work-arounds in
case of that need.)

Then you could do something like:

MY_FUNCTION(map<int, pair<int, float>>, []{
int x = 0, y[] = { 1, 2, 3, 4 };
// ...
})

...with all sorts of commas in the arguments, but the compiler knows how
to parse it via token balancing.

--
Matthew

[dgutson .]

Guys, thanks for all your feedback.
Please note:
a) that I already explicitly stated that I do want to focus on the
calling side, since the macro may belong to a 3rd party library and/or
a C library
b) that I don't want to teach C++ to the preprocessor (which also
will involve a very large change), so keywords such as 'typename' or
complicated grammar is out of scope of my (arguably trivial) proposal.
c) that token balancing (which can be also categorized as teaching
C++ to the CPP) is not useful, since c.1) it may brake existing code,
and c.2) does not work in the following examples:

#define IF(condition1, condition2) if(lower condition1 && condition2 upper)
IF(<b, c>) return 1;

#define BRACKET(x) x] = 1
BRACKET(a[1);

d) I try to keep things simple. I state that my proposal is a
minimal as it can be.

If anybody prefers the "teach C++ to the preprocessor" approach, I
invite you to write your own (separate) proposal so there will be two
solutions to vote for. My proposal stays simple.
At this point, please let me know if there would be a champion to
co-author (or be a reviewer) and present this in a WG21 meeting.

Thanks!

Daniel.

>
> --
> Matthew

[Sean Middleditch]

On Tuesday, October 13, 2015 at 7:00:07 AM UTC-7, Matthew Woehlke wrote:

Why not:

  #deffun MY_DECLARATION(A, B)

...and perform argument splitting as if MY_DECLARATION were a regular
C++ function and not a macro? This would preclude feeding it arguments

Because the preprocessor is a completely separate translation stage that happens way before we can disambiguate < between the start of template parameters and a less-than operator. C++ has a nasty grammar and cannot be disambiguated without at least a little semantic insight (e.g. name tables to know if an identifier refers to a type or template, among other things).

Also, preprocessor changes in C++ are pretty iffy. I don't see fundamental changes or new features to the preprocessor originating in WG21 as a likely eventuality. Which applies to this whole thread, of course, not just your suggestion. :)

 

--
Matthew

[dgutson .]

Why is this so hard? I can write this as a C proposal as a last resort. But the C++ preprocessor is already different than the C one, I still don't understand the 'go to C first' specially considering that this problem does not affect to C, why would they even care?
So what was the history regarding the // comments? That's the first historic difference that comes to my mind now.

[David Krauss]

On 2015–10–14, at 12:01 AM, dgutson . <daniel...@gmail.com> wrote:

Why is this so hard? I can write this as a C proposal as a last resort. But the C++ preprocessor is already different than the C one

It depends what you mean by “preprocessor.” C++ lexes tokens differently but macro and directive processing are the same (predefined values aside). When it comes to other languages, all have lexers and tokens but most don’t have “preprocessors.” So you could say that C and C++ are different languages with the same preprocessor.

So what was the history regarding the // comments? That's the first historic difference that comes to my mind now.

C99 added // comments. But, comments are part of lexing.

[dgutson .]

On Tue, Oct 13, 2015 at 1:59 PM, David Krauss <pot...@gmail.com> wrote:
>
> On 2015–10–14, at 12:01 AM, dgutson . <daniel...@gmail.com> wrote:
>
> Why is this so hard? I can write this as a C proposal as a last resort. But
> the C++ preprocessor is already different than the C one
>
>
> It depends what you mean by “preprocessor.” C++ lexes tokens differently but
> macro and directive processing are the same (predefined values aside). When
> it comes to other languages, all have lexers and tokens but most don’t have
> “preprocessors.” So you could say that C and C++ are different languages
> with the same preprocessor.

So, in short: considering that this is not an issue for C, where
should this proposal go?

>
> So what was the history regarding the // comments? That's the first historic
> difference that comes to my mind now.
>
>
> C99 added // comments. But, comments are part of lexing.

I know about C99, but I'm asking for the history behind the // : how
was that added at that time?

>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> Visit this group at
> http://groups.google.com/a/isocpp.org/group/std-proposals/.

[Thiago Macieira]

On Tuesday 13 October 2015 14:38:11 dgutson . wrote:
> > C99 added // comments. But, comments are part of lexing.
>
> I know about C99, but I'm asking for the history behind the // : how
> was that added at that time?

By changing the C preprocessor to accept it too. You may remember that even in
C89 mode, most C compilers accept // comments, unless they've been placed in
strict C89 mode.

So, yes, you can turn some features on and off in the preprocessor.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[Nicol Bolas]

On Tuesday, October 13, 2015 at 2:47:40 PM UTC-4, Thiago Macieira wrote:

On Tuesday 13 October 2015 14:38:11 dgutson . wrote:
> > C99 added // comments. But, comments are part of lexing.
>
> I know about C99, but I'm asking for the history behind the // : how
> was that added at that time?

By changing the C preprocessor to accept it too. You may remember that even in
C89 mode, most C compilers accept // comments, unless they've been placed in
strict C89 mode.

So, yes, you can turn some features on and off in the preprocessor.

The point he's making is some are arguing that his suggestion isn't acceptable because it would have to be proposed to the C standards committee first, then propagate up to C++. And he's showing that the two preprocessors already can have significant differences between them, so there is no reason why there can't be one more.

[dgutson .]

El 13/10/2015 15:47, "Thiago Macieira" <thi...@macieira.org> escribió:
>
> On Tuesday 13 October 2015 14:38:11 dgutson . wrote:
> > > C99 added // comments. But, comments are part of lexing.
> >
> > I know about C99, but I'm asking for the history behind the // : how
> > was that added at that time?
>
> By changing the C preprocessor to accept it too. You may remember that even in
> C89 mode, most C compilers accept // comments, unless they've been placed in
> strict C89 mode.

Ok, I see, the problem with this is that this is a C++-only thing (wereas the // was a useful addition for C), so there's no reason the C Committee could be interested in accepting it, so I foresee a dead end in that way. Is your advise to still try it anyway?

>
> So, yes, you can turn some features on and off in the preprocessor.
> --
> Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
>    Software Architect - Intel Open Source Technology Center
>       PGP/GPG: 0x6EF45358; fingerprint:
>       E067 918B B660 DBD1 105C  966C 33F5 F005 6EF4 5358
>

[Thiago Macieira]

On Tuesday 13 October 2015 16:20:27 dgutson . wrote:
> Ok, I see, the problem with this is that this is a C++-only thing (wereas
> the // was a useful addition for C), so there's no reason the C Committee
> could be interested in accepting it, so I foresee a dead end in that way.

Right.

> Is your advise to still try it anyway?

It's up to you. I don't think you'll be able to come up with a technically
valid proposal for C++ alone, much less one that the C committee will be
interested in hearing.

[dgutson .]

On Tue, Oct 13, 2015 at 6:51 PM, Thiago Macieira <thi...@macieira.org> wrote:
> On Tuesday 13 October 2015 16:20:27 dgutson . wrote:
>> Ok, I see, the problem with this is that this is a C++-only thing (wereas
>> the // was a useful addition for C), so there's no reason the C Committee
>> could be interested in accepting it, so I foresee a dead end in that way.
>
> Right.
>
>> Is your advise to still try it anyway?
>
> It's up to you. I don't think you'll be able to come up with a technically
> valid proposal for C++ alone, much less one that the C committee will be
> interested in hearing.

Interested in hearing but uninterested in implementing?

Does anybody know the C committee mailing list?

>
> --
> Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
> Software Architect - Intel Open Source Technology Center
> PGP/GPG: 0x6EF45358; fingerprint:
> E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[Thiago Macieira]

On Tuesday 13 October 2015 19:51:04 dgutson . wrote:
> On Tue, Oct 13, 2015 at 6:51 PM, Thiago Macieira <thi...@macieira.org>
wrote:
> > On Tuesday 13 October 2015 16:20:27 dgutson . wrote:
> >> Ok, I see, the problem with this is that this is a C++-only thing (wereas
> >> the // was a useful addition for C), so there's no reason the C Committee
> >> could be interested in accepting it, so I foresee a dead end in that way.
> >
> > Right.
> >
> >> Is your advise to still try it anyway?
> >
> > It's up to you. I don't think you'll be able to come up with a technically
> > valid proposal for C++ alone, much less one that the C committee will be
> > interested in hearing.
>
> Interested in hearing but uninterested in implementing?
>
> Does anybody know the C committee mailing list?

I don't think they do things over email. They appear to only follow the ISO
requirements.

There must be a way to publish papers for their consideration, but I don't
know how.

See http://www.open-std.org/jtc1/sc22/wg14/www/contacts, maybe those folks can
help you.

[Bengt Gustafsson]

I don't think that the C preprocessor compatiblity issue is a real one. This is just a way to avoid having to think about preprocessor improvements.

I like the deffun idea but as someone pointed out we would have to sacrifice the possibility of sending an expression containing a < in the "less than" meaning to such a macro. This is of course sad but I see no way that we could parse both meanings of < correctly at the time the preprocessor runs as this requires knowledge of what identifiers stand for (templates or variables).

This brings us back to my suggestion with "typename" annotation on macro formals, which lacks the ability to handle braced lists and lambda capture lists. I think this still covers most interesting sue cases (combined if needed with a __VA_ARGS__ handling to make sure everything in a tail argument gets expanded properly). I very seldom see macros which take TWO arguments containing "arbitrary" code, which would be the remaining problem.

But maybe this is too much to ask, and if so I still think that the original \ proposal to quote a comma is reasonable even if it requires more teaching of how, where and why this escaping has to be done. 

One sticky point could be combination effect with the #ARG possibility inside the macro body, which makes a call site with a \, combination at least mean something:

#define MACRO(STR,VAL) MyFun(#STR, VAL)

MACRO(\, 3)

Well I don't really know what this expands to today, but probably:

MyFun("\", 3)

which subsequently produces a mismatched " error or

MyFun("\\", 3)

which is potentially useful I guess.

Even if there is some piece of code out there that uses this it must be only a handful of places in total.

I tried this on VS and there is no compiler added escaping (first case). I don't know if it is compilant but it at least makes any potentially useful cases incompatible with the Microsoft compiler, which talks for this proposal.

Den fredag 9 oktober 2015 kl. 06:20:22 UTC+2 skrev dgutson:

There are situations in which the cpp wrongly takes commas separating
template arguments as macro arguments delimiters.
For example:

template <class X, class Y> struct S;

#define Macro(arg) S arg;

The extra ( ) workaround used in C cannot be used here:
    Macro((<int, int>));

This is important for X-macros for example.

I propose to enhance the cpp to be able to ignore escaped commas (with
\), so this can be done:
    Macro(<int\, int>);

We already implemented this extension in gcc (patch to be sent on
Monday) which is working.
What we did is: comma escaping occurs at the top level, but not inside ( ).
For example
    SomeMacro(f(a\,b))
does no escape the comma and thus has current behavior.

Feedback?

Thanks,

   Daniel.

[Matthew Woehlke]

On 2015-10-14 04:31, Bengt Gustafsson wrote:
> I like the deffun idea but as someone pointed out we would have to
> sacrifice the possibility of sending an expression containing a < in the
> "less than" meaning to such a macro. This is of course sad but I see no way
> that we could parse both meanings of < correctly at the time the
> preprocessor runs as this requires knowledge of what identifiers stand for
> (templates or variables).

You'd have to put such an expression in ()'s. If it's really an
expression, presumably you could do so.

This also assumes that #deffun wouldn't bite the bullet and defer
expansion until a later stage in processing, i.e. one where the compiler
knows about types and could treat it as something more like an inline
function call.

As others have noted, however, such a proposal being accepted seems
unfortunately unlikely. Code generation (PP and otherwise) is an area I
think could very much be improved, but there is a lot of inertia against
doing so.

Maybe someone feeling sufficiently motivated should just write a new,
stand-alone preprocessor that doesn't restrict itself to the existing
CPP and publish it. Then try to get it integrated into gcc and/or clang.
(Bonus points for making it a BSD library so that the implementation can
be shared.) There's nothing in the standard that says one cannot feed
generated code into the compiler, after all.

--
Matthew

[dgutson .]

(intentionally top posted)

FWIW, status is:
- patch: https://gcc.gnu.org/ml/gcc-patches/2015-10/msg02957.html
- WG14 contacted (David Keaton, the convener)

Daniel.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[dgutson .]

On Tue, Oct 27, 2015 at 5:32 PM, dgutson . <daniel...@gmail.com> wrote:
> (intentionally top posted)
>
> FWIW, status is:
> - patch: https://gcc.gnu.org/ml/gcc-patches/2015-10/msg02957.html
> - WG14 contacted (David Keaton, the convener)

I'm bringing another situation where this is important: lambda capture list.