Yikes, va_start doesn't work with pack expansions!

[Daryle Walker]

This is based on a recent query I made on Stack Overflow.  I tried test code on a compiling website running GCC 4.8.

    template < typename ...Args >
    int  my_func2( short x, std::va_list &aa, Args &&...a );

    template < typename ...Args >
    int  my_func( short x, Args &&...a, ... )
    {
        std::va_list  args2;
        va_start( args2, a... );
        //...
    }

The code only worked when sizeof...(a) exactly one, neither more nor less.  Worse, when a was empty, I could use x as the last-normal parameter instead!

What I expect to happen:
I can use "a" as the last-normal parameter in va_start, no matter if a represents one, more, or zero expansion arguments.  I shouldn't be able to use x when a is empty.  (And what happens if there isn't an extra prior argument like x?)

I'm flexible on whether we can use "a", or we must still use "a...".  For the latter, maybe va_start needs to be a variadic macro?  Hmm, how should it work if a is empty and there's no prior arguments?

Possible solution:
Change Section 18.10 [support.runtime], paragraph 3, footnote 227 to:

Note that va_start is required to work as specified even if unary operator& is overloaded for the type of parmN, or parmN is a pack expansion, or both.

Thanks.

[Daniel Krügler]

2013/5/16 Daryle Walker <dar...@gmail.com>:

> This is based on a recent query I made on Stack Overflow. I tried test code
> on a compiling website running GCC 4.8.
>
> template < typename ...Args >
> int my_func2( short x, std::va_list &aa, Args &&...a );
>
> template < typename ...Args >
> int my_func( short x, Args &&...a, ... )
> {
> std::va_list args2;
> va_start( args2, a... );
> //...
> }
>
> The code only worked when sizeof...(a) exactly one, neither more nor less.
> Worse, when a was empty, I could use x as the last-normal parameter instead!

There are several problems with that code:

1) va_start expects exactly two arguments.

2) The second argument shall not be a reference type, and if, the
behaviour is undefined.

3) I don't understand how you expect the parameter pack and the
ellipses parameter to cooperate.

4) Your example is so incomplete that I don't understand in which call
context you believe it should work.

The first two arguments are requirements imposed by a macro that stems
from the C Standard Library.

> What I expect to happen:
> I can use "a" as the last-normal parameter in va_start, no matter if a
> represents one, more, or zero expansion arguments.

Why do you expect that? It might be technical feasible by changing the
macro specification of C++ to expect a variadic macro, but I'm a
little bit surprised, why this support, that this use-case is so
important (At the moment I don't understand this use-case), that the
effort to change the specification of that macro should be changed. It
is really not so hard to provide a function template definition, that
returns always the last element of a parameter pack, so given that
definition you could write

va_start( args2, last(a...) );

instead. Even if you solve the problem, when do you expect that the
ellipses parameter could be non-empty? Do you always want to provide
the pack types of my_func as explicit template arguments?

> I shouldn't be able to
> use x when a is empty.

Why not? This can also me reasonable (but I don't see a way how to
provide an explicit empty pack, how would you do that?). Assuming you
solved the technical problems: If you don't like that, you can static
assert on the pack length, so it won't be able.

> (And what happens if there isn't an extra prior
> argument like x?)

A reasonable assumption for me would be that parameter x is the last
one. What other choices do you envision?

> I'm flexible on whether we can use "a", or we must still use "a...". For
> the latter, maybe va_start needs to be a variadic macro? Hmm, how should it
> work if a is empty and there's no prior arguments?

I recommend to construct your own varidic function template that
solves this parameter choice for you. For proper integration of x (or
not), you could call it as so:

va_start( args2, last(x, a...) );

so it will never be empty. Implement last() to the most preferred way
you like (I don't see any reasons that it couldn't).

> Possible solution:
> Change Section 18.10 [support.runtime], paragraph 3, footnote 227 to:
> Note that va_start is required to work as specified even if unary operator&
> is overloaded for the type of parmN, or parmN is a pack expansion, or both.

Why is now unary operator& important for you? Again, you could solve
this via your own last function template that internally calls
std::addressof for the returned argument.

From what I'm seeing sofar I really cannot deduce a real need for a
standardized solution of the use-case you mention. Especially because
I think that the combination of variadic parameters with the ellipses
parameter is also not natural to use, plus the IMO missing support for
explicit empty pack expansions.

- Daniel

[Daniel Krügler]

2013/5/16 Daniel Krügler <daniel....@gmail.com>:

> Why not? This can also me reasonable (but I don't see a way how to
> provide an explicit empty pack, how would you do that?).

Well, there exist ways to solve this, for example one could be required to write

my_func<Empty>(0, 1, 2, 3);

where Empty is a token type (or value), but still I'm unconvinced of
the general benefit of providing support by the standard library,
given the fact that there are fundamental limitations of the ellipses
parameter.

As far as I see everything that you are describing here can be
realized by a simple variadic function template last() that can be
implemented without any magic.

- Daniel

[Daniel Krügler]

2013/5/16 Daniel Krügler <daniel....@gmail.com>:

I was a bit too hasty here, this approach won't work, because the
va_start macro requires:

"The parameter parmN is the identifier of the rightmost parameter in
the variable
parameter list in the function definition"

This is not realized by my function call expression.

- Daniel

[Daniel Krügler]

Nonetheless I think it can be realized as follows:

template < typename ...Args, typename Last >
int my_func(short x, Args&&...a, Last last, ...)
{
std::va_list args2;
va_start( args2, last);
//...
}

Now, last is always the last parameter and you don't need special
tricks to extract the last parameter. It still requires (as in your
original example) to provide the parameter pack types explicitly, but
I assume that this is something you don't worry about.

> - Daniel

[Daryle Walker]

On Thu, May 16, 2013 at 7:14 AM, Daniel Krügler <daniel....@gmail.com> wrote:

2013/5/16 Daryle Walker <dar...@gmail.com>:

> This is based on a recent query I made on Stack Overflow.  I tried test code
> on a compiling website running GCC 4.8.
>
>     template < typename ...Args >
>     int  my_func2( short x, std::va_list &aa, Args &&...a );
>
>     template < typename ...Args >
>     int  my_func( short x, Args &&...a, ... )
>     {
>         std::va_list  args2;
>         va_start( args2, a... );
>         //...
>     }
>
> The code only worked when sizeof...(a) exactly one, neither more nor less.
> Worse, when a was empty, I could use x as the last-normal parameter instead!

[SNIP]

> What I expect to happen:
> I can use "a" as the last-normal parameter in va_start, no matter if a
> represents one, more, or zero expansion arguments.

[SNIP]

> Possible solution:
> Change Section 18.10 [support.runtime], paragraph 3, footnote 227 to:
> Note that va_start is required to work as specified even if unary operator&
> is overloaded for the type of parmN, or parmN is a pack expansion, or both.

Why is now unary operator& important for you? Again, you could solve
this via your own last function template that internally calls
std::addressof for the returned argument.

 

The operator isn't important here. The footnote is one that already exists, so I'm just extending that sentence.

 

From what I'm seeing sofar I really cannot deduce a real need for a
standardized solution of the use-case you mention. Especially because
I think that the combination of variadic parameters with the ellipses
parameter is also not natural to use, plus the IMO missing support for
explicit empty pack expansions.

 

I'll turn this around. The combination should be supported because there is no reason that it shouldn't. 

 

...

 

In the months since I started this thread, I had a possible solution. But this depends on how argument passing in current C++ and Ancient C work.

 

Let's start with two terms:

    * static arguments: the regular arguments to a function

    * dynamic arguments: the C-level variable arguments to a function

Note that C++ variable arguments are fixed when the compiler & linker finalize them 100% in order to create object code, so they count as static arguments!

 

I think that Ancient C was down-right sloppy with function arguments, and there was little distinction between static and dynamic arguments in implementation. Back then, you could write function declarations with nothing in between the parentheses, and that meant that the parameter list was to be specified later. That's why "(void)" was explicitly needed to indicate a function took zero arguments.

 

But I think C++ has a more formal argument-passing mechanism. It should know where all of its static arguments are, and therefore the border between the static and (any) dynamic arguments. That means that the final static parameter needed in va_start is just a formality in C++. A modern version of the macro would just needs the destination va_list object. So the solution would be the C++ version of va_start ignores the final parameter and supplies the correct starting point with compiler magic. That means it will work no matter if the last argument is normal or a C++ parameter pack. Are my assumptions on compiler technology wrong here?

 

Anyway, this is now a core issue: <http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#1790>.

 

Daryle W.

 

[David Krauss]

On 10/26/13 1:07 AM, Daryle Walker wrote:
> I think that Ancient C was down-right sloppy with function arguments,
> and there was little distinction between static and dynamic arguments
> in implementation. Back then, you could write function declarations
> with nothing in between the parentheses, and that meant that the
> parameter list was to be specified later. That's why "(void)" was
> explicitly needed to indicate a function took zero arguments.

There's a distinction between the ABI and the syntax. ABIs for early platforms might have assumed that every argument is a single machine word on the stack. Formal parameter declarations were unnecessary because compiling a function call required only pushing arguments in a generic way and jumping to a linker-provided address. Later ABIs have become more sophisticated and demanding, so not everything can be passed by ellipsis and signatures must be forward-declared (or deduced by the rule that everything is an int).

> But I think C++ has a more formal argument-passing mechanism. It
> should know where all of its static arguments are, and therefore the
> border between the static and (any) dynamic arguments. That means
> that the final static parameter needed in va_start is just a
> formality in C++.

C++ doesn't have a mechanism, it just requires that control flow must enter a function with certain names bound to objects. ABIs define the memory layout and protocols, and they are free to depend on whatever the language guarantees for them. Then the compiler implements a mechanism conforming to the ABI.

As the previous reply to this thread mentions, va_start needs the name of the last parameter because it's defined that way. It is what it is.

> A modern version of the macro would just needs the
> destination va_list object. So the solution would be the C++ version
> of va_start ignores the final parameter and supplies the correct
> starting point with compiler magic. That means it will work no matter
> if the last argument is normal or a C++ parameter pack. Are my
> assumptions on compiler technology wrong here?

Yes, va_start could have been specified as an intrinsic in the first place, because compilers have always known the parameter list of the current enclosing function. The mechanism is a half-baked flexible array used for argument passing, just like initializer_list.

In practice, it usually adds 1 to its argument, then casts it to void*. The compiler needs to be sure that a parameter preceding an ellipsis is allocated on the stack.

> Anyway, this is now a core issue: <
> http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#1790>.

Does the published wording reflect your concern? It looks likely to be closed at the first discussion, because the double ellipsis is already used in libraries, and you already linked to my practical example on the SO page.

I think what you mean is that a function template using deduction on a parameter pack doesn't have a handle to pass to va_start. But that's not right either, because there cannot be any actual arguments following the ones forming the pack. If you try to specify an explicit parameter list, deduction will still add more (implicit) parameters to the end of the pack. In fact there's no way for the compiler to tell what you mean unless you disable deduction.

To disable deduction, just add a parameter after the pack. Now the pack must be explicitly specified, and you also have a handle to pass to va_start.

So it seems that the features mesh together perfectly: a non-deduced function template parameter pack needs termination by a subsequent declared parameter at the end, and a va_args list needs termination by a declared parameter at the beginning.

[Daryle Walker]

On Tuesday, October 29, 2013 12:23:51 AM UTC-4, David Krauss wrote:

On 10/26/13 1:07 AM, Daryle Walker wrote:
> Anyway, this is now a core issue: <http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#1790>.

Does the published wording reflect your concern? It looks likely to be closed at the first discussion, because the double ellipsis is already used in libraries, and you already linked to my practical example on the SO page.

I think what you mean is that a function template using deduction on a parameter pack doesn't have a handle to pass to va_start. But that's not right either, because there cannot be any actual arguments following the ones forming the pack. If you try to specify an explicit parameter list, deduction will still add more (implicit) parameters to the end of the pack. In fact there's no way for the compiler to tell what you mean unless you disable deduction.

No, I mean to definitively specify that a parameter pack can be used as the handle for va_start (i.e. as the last static argument), regardless of any deduction. In other words, "va_start( my_list, a... )" works. I wasn't saying that this case was deliberately banned, but that the case is so rare that implementors never considered it, let alone tested it. (So my tests with my compiler went into wacky near-UB land due to no one ever considering it.) Yes, I do admit that triggering the case is kind-of convoluted, but rarity shouldn't be a anti-justification for fixing something.

Did you see my response to my own SO query there? Your claim that C++ varargs will grab extra arguments anyway after the ones matching the explicit parameter list isn't true there. The following arguments went as C varargs. Maybe you're using a different compiler.

(As you said, C++ could have defined something like va_start as a primitive, since it knows what all the static arguments are. An implementation of va_start that could solve this is one that secretly ignores the second parameter. That second parameter is a dangerous false choice, since there is only one correct answer to what to put there and C++ compilers already have to know said answer. Maybe in C++17, a single-argument variant of va_start can be added to supplant the current one, and also solve the current unsolvable case of getting a va_list when there are no static arguments.)

Daryle W.