Variadic template and arguments by value

[Juha Nieminen]

Why doesn't this compile?

//------------------------------------------------------------
template<typename... Types>
class MyClass
{
public:
MyClass(Types&&...) {}
};

template<typename... Types>
MyClass<Types...> instantiate(Types... objects)
{
return { objects... };
}

int main()
{
auto instance = instantiate(1, 2, 3);
}
//------------------------------------------------------------

clang says:

test.cc:12:12: error: no matching constructor for initialization
of 'MyClass<int, int, int>'
return { objects... };
^~~~~~~~~~~~~~

(Note that I'm not asking "how do I make this compile?" I'm asking *why*
doesn't this compile. And yes, I'm taking the parameters to the
instantiate() function by value on purpose.)

[Öö Tiib]

That is interesting question as it is "why lvalue can't be bound to rvalue
reference?" Such are language rules from C++11 and these were discussed
for almost decade. Perhaps it was to guard us from accidentally moving
from lvalues.

I understood that you did not want to see example how to make it to
compile but may be someone else is interested so I post it anyway. Just
don't look. ;) That will compile:

#include <utility>

template<typename... Types>
class MyClass
{
public:
MyClass(Types&&...) {}
};

template<typename... Types>
MyClass<Types...> instantiate(Types... objects)
{

return { std::forward<Types>(objects)... };

[Juha Nieminen]

Stefan Ram <r...@zedat.fu-berlin.de> wrote:

> Juha Nieminen <nos...@thanks.invalid> writes:
>>Why doesn't this compile?

>>template<typename... Types>
>>class MyClass
>>{
>> public:
>> MyClass(Types&&...) {}
>>};
>

> The parameters are rvalue references, so they wont bind to lvalues.

That doesn't make much sense to me, because this compiles just fine:

//--------------------------------------------------------

template<typename... Types>
class MyClass
{
public:
MyClass(Types&&...) {}
};

template<typename... Types>

MyClass<Types...> instantiate(Types&&... objects)
{
return { objects... };
}

int main()
{
int a = 1, b = 2, c = 3;
auto instance = instantiate(a, b, c);
}
//--------------------------------------------------------

[Juha Nieminen]

Stefan Ram <r...@zedat.fu-berlin.de> wrote:
> Juha Nieminen <nos...@thanks.invalid> writes:
>>That doesn't make much sense to me, because this compiles just fine:

>>template<typename... Types>
>>MyClass<Types...> instantiate(Types&&... objects)
>

> The template arguments are deduced through template argument
> deduction.
>
> template< class T > void instantiate( T && ){}
>
> int main()
> { int i = 0;
> instantiate( i ); /* instantiate< int& >( int& ) */
> instantiate( 0 ); /* instantiate< int >( int&& ) */ }

I'm still very puzzled.

The original problem is that the class constructor takes things by
rvalue-reference (rather than "universal reference", even though
it looks like it does), and the original code attempted to give it
an lvalue, which cannot be given as an rvalue-reference.

However, in my second example instantiate(i) causes the parameter
to be given to the function as an lvalue-reference, which is then
given as-is to the class constructor, and it compiles. Which would
mean that an lvalue-reference is ok to be given to a function taking
an rvalue-reference, even though an lvalue is not.

Why?

And why is

template<typename... T>
void foo(T&&...);

different from

template<typename... T>
class C
{
public:
C(T&&...);
};

Both look like an "universal reference" situation, but apparently
the second isn't?

[Öö Tiib]

On Monday, 24 September 2018 23:35:41 UTC+3, Juha Nieminen wrote:
> And why is
>
> template<typename... T>
> void foo(T&&...);
>
> different from
>
> template<typename... T>
> class C
> {
> public:
> C(T&&...);
> };
>
> Both look like an "universal reference" situation, but apparently
> the second isn't?

One is template of function other is non-template constructor of
template of class.

[Chris Vine]

'foo' is a template function whereby each 'T' is a deduced type and so
comprises a "universal" (in standard-speak "forwarding") reference.
The constructor 'C' is not a template function and it does not perform
type deduction (at least in C++11/14, and I don't think that C++17
class template argument deduction magically converts your rvalue
references to forwarding references).

[Chris Vine]

It compiles because a, b and c are lvalues. This means that the
'instantiate' function deduces 'Types...' as of type 'int&...' . This
means that (i) the 'instance' object is instantiated as of type
MyClass<int&, int&, int&>, and (ii) the MyClass constructor for that
object is, by reference collapsing, instantiated as constructor
MyClass(int&, int&, int&), which thereby takes lvalues as its arguments.

[Juha Nieminen]

Chris Vine <chris@cvine--nospam--.freeserve.co.uk> wrote:
>> And why is
>>
>> template<typename... T>
>> void foo(T&&...);
>>
>> different from
>>
>> template<typename... T>
>> class C
>> {
>> public:
>> C(T&&...);
>> };
>>
>> Both look like an "universal reference" situation, but apparently
>> the second isn't?
>
> 'foo' is a template function whereby each 'T' is a deduced type and so
> comprises a "universal" (in standard-speak "forwarding") reference.
> The constructor 'C' is not a template function and it does not perform
> type deduction (at least in C++11/14, and I don't think that C++17
> class template argument deduction magically converts your rvalue
> references to forwarding references).

If I understand this correctly, what you say there may be technically
correct, but I think it's a bit confusing and misleading. You make it
sound like there's some special meaning to the foo() function being a
template function vs. the C() constructor not being itself a template
function (but still depending on template parameters).

I don't think there's such a special distinction. As if some special
magic is being applied in the former case but not the latter.

What's happening is that the C() constructor parameters are whatever
was specified by the calling code as 'T'. If the calling code instantiated
the class as, for example, C<int, int, int>, then the constructor simply
ends up being C(int&&, int&&, int&&), and thus only rvalues can be given
to it.

If the class had been, somehow, instantiated as C<int&, int&, int&>,
then the constructor would end up as being C(int& &&, int& &&, int& &&),
which collapses to C(int&, int&, int&), and thus only lvalues can be
given to it.

For it to work as some sort of "universal constructor", the class would
need to be instantiated with template types that are either value types
or references, depending on the kind of parameter you want to give the
constructor.

Although, in a sense, there may be a special meaning to a templated
function in particular, when it uses an "rvalue-reference" syntax in
its parameters. In this situation the templated type is automatically
made to be of reference or non-reference type depending on whether the
parameter you are giving to it is an lvalue or an rvalue. This happens
automatically when you don't explicitly specify the template parameter
types. With the class you have to explicitly specify the template parameter
types, so it becomes more complicated. (Is it even possible?)

[Chris Vine]

I was answering here your non-rhetorical question "Both look like an
"universal reference" situation, but apparently the second isn't?". I
was explaining that you are correct that "the second isn't" in as
non-confusing and direct a way as I could. I answered the other point
you make below in my accompanying posting of 23:58:09Z: I was intending
you to read them together.

The overarching issue is that for there to be a forwarding reference,
the function doing the forwarding must be a template function so that
reference-collapsing can take place. That is basically it: so there is
some "special magic". One consequence is that virtual functions cannot
use forwarding. I think it was a mistake to implement forwarding
through template function type deduction and reference-collapsing: that
seems to me to be an unnecessary mixing of concerns. But it is what C++
does.

For what it is worth I also think it is highly confusing to the beginner
that the function:

template<typename T>
void foo(T&&);

deduces T to be 'int&' when the function is passed an int lvalue
argument and to be 'int' when the function is passed an int rvalue
argument. It is not how this works when passed a lvalue

template<typename T>
void foo(T&);

When that is passed an int lvalue it deduces T to be 'int'.

> What's happening is that the C() constructor parameters are whatever
> was specified by the calling code as 'T'. If the calling code instantiated
> the class as, for example, C<int, int, int>, then the constructor simply
> ends up being C(int&&, int&&, int&&), and thus only rvalues can be given
> to it.

Yes.

> If the class had been, somehow, instantiated as C<int&, int&, int&>,
> then the constructor would end up as being C(int& &&, int& &&, int& &&),
> which collapses to C(int&, int&, int&), and thus only lvalues can be
> given to it.

Also yes (see my other posting).

> For it to work as some sort of "universal constructor", the class would
> need to be instantiated with template types that are either value types
> or references, depending on the kind of parameter you want to give the
> constructor.

But in practice you probably don't want to instantiate an object of type
C<int&, int&, int&> just because the object is intialized by lvalues.
Assuming a case where the C class keeps the initializing ints as member
data, you probably want to instantiate an object of type C<int, int,
int> whether or not it is initialized with lvalues or rvalues. You probably
also want to have a forwarding constructor if C might be instantiated
for non-fundamental types as well as ints. The normal way to do both
of those is to make the constructor a template function so that it can
do forwarding.

> Although, in a sense, there may be a special meaning to a templated
> function in particular, when it uses an "rvalue-reference" syntax in
> its parameters. In this situation the templated type is automatically
> made to be of reference or non-reference type depending on whether the
> parameter you are giving to it is an lvalue or an rvalue. This happens
> automatically when you don't explicitly specify the template parameter
> types. With the class you have to explicitly specify the template parameter
> types, so it becomes more complicated. (Is it even possible?)

I don't understand your question. With C++11/14 you do have to
explicitly state the types for which a class template is instantiated.
With C++17 there is the option of class template argument deduction.

[Öö Tiib]

On Thursday, 27 September 2018 13:02:31 UTC+3, Chris Vine wrote:
>
> The overarching issue is that for there to be a forwarding reference,
> the function doing the forwarding must be a template function so that
> reference-collapsing can take place. That is basically it: so there is
> some "special magic". One consequence is that virtual functions cannot
> use forwarding. I think it was a mistake to implement forwarding
> through template function type deduction and reference-collapsing: that
> seems to me to be an unnecessary mixing of concerns. But it is what C++
> does.

Technically compiler generates a family of overloads there, family
size grows in exponential relation to number of forwarded parameters
and if that is virtual function then all are "used" and each must
exist and needs entry in vtable of each derived class. Invisible
drain of resources. So we are required to explicitly write all
overloads of virtual functions that we need.

[Chris Vine]

I agree that that problem lies behind the reason why virtual functions
cannot be templated.[1] And you make a good point that because
of that, virtual functions could not have been used for generic
forwarding anyway.

> So we are required to explicitly write all overloads of virtual
> functions that we need.

If a function has multiple arguments any of which could be rvalues or
lvalues there is of course an explosion of lvalue/rvalue permutations
which you have to write overloads for. I have wondered whether we
could not have had a '&&&' qualifier to indicate something which could
be an lvalue or rvalue along these lines:

void foo(Car&&&, Bus&&&, Train&&&); // lvalues or rvalues

and to have a compiler intrinsic to enable the actual
lvalue/rvalue-ness to be obtained further down the call chain to
determine whether the objects could be moved from. This would seem to
allow multi-argument specific-type forwarding to be used with virtual
functions as well. For generic forwarding the equivalent could have
been this, with no need for special template deduction rules or
reference collapsing rules:

template <typename T>
void foo(T&&&); // lvalue or rvalue

template <typename T>
void foo(T&&); // rvalue only

template <typename T>
void foo(T&); // lvalue only

There may be problems with this - I imagine it was thought about - but
anyway it won't now happen because C++ has gone down a different
route. With C++, for perfect forwarding either you use explicit
overloading or you use template functions and reference collapsing.

Chris

[1] In fact because in the template function signature 'template
<typename T> foo(T)' the number of types which can match T is
unbounded, the complexity for a templated virtual function would I think
be infinite without the compiler carrying out whole program analysis.

[Alf P. Steinbach]

That's an argument against templating of virtual functions, not against
argument forwarding in itself. It seems to have an assumption of /static
type checking/. But if one is satisfied with dynamic type checking, then
argument forwarding is quite doable for a virtual function.

To examine it more closely I think we should be clear on what argument
forwarding MEANS in this context.

Is it just forwarding a set of actual arguments, e.g. a virtual function
that gets an argument pack as arguments, and calls an overloaded free
function F with those arguments? And that can be overridden to e.g. also
log this call? That's not so difficult with dynamic type checking, one
can e.g. use a sequence of `std::any` instances.


--------------------------------------------------------------------------
#include <any>
using std::any, std::any_cast;

#include <iostream>
using std::cout, std::endl;

#include <string>
using std::string;

#include <iterator>
using std::begin, std::end;

template< class T > using p_ = T*;
template< class T > using r_ = T&;

class Base
{
public:
// Here nothing is known about types of arguments, or their number, in
// a derived class override.
virtual void foo( p_<const any> first, p_<const any> beyond ) const
= 0;

template< class... Args >
void pass_to_foo( const Args&... args ) const
{
const any arg_list[]{ args... };
foo( begin( arg_list ), end( arg_list ) );
}
};

class Derived:
public Base
{
public:
void foo( p_<const any> first, p_<const any> beyond ) const override
{
for( auto p = first; p != beyond; ++p )
{
if( p->type() == typeid( int ) )
{
cout << "int arg: " << any_cast<int>( *p ) << "." << endl;
}
else if( p->type() == typeid( string ) )
{
cout << "string arg: \"" << any_cast<string>( *p ) <<
"\"." << endl;
}
else
{
throw "Uh oh, an argument of a type not supported in
Derived.";
}
}
}
};

auto main()
-> int
{
using namespace std::literals;
r_<const Base> o = Derived();
o.pass_to_foo( "The answer is "s, 0b0101010 );
}
--------------------------------------------------------------------------


But, do you also or alternatively require that the top-level call's
expression category - lvalue, xvalue or rvalue - for each argument,
should be recreated? More involved & difficult, I'd say. Because even
the standard library's statically type checked forwarding doesn't manage
to do that 100%:


--------------------------------------------------------------------------
#include <iostream>
using std::cout, std::endl;

#include <string>
using std::string;

#include <utility>
using std::forward;

template< class E >
struct Expression_category{ static auto s() -> string { return
"prvalue"; } };

template< class E >
struct Expression_category<E&>{ static auto s() -> string { return
"lvalue"; } };

template< class E >
struct Expression_category<E&&>{ static auto s() -> string { return
"xvalue"; } };

#define EC( e ) Expression_category<decltype(( e ))>::s()

template< class... Args >
void foo( Args&&... args )
{
const string msgs[] = { (args + " as an " + EC( forward<Args>( args
) ) )... };
for( string const& m : msgs )
{
cout << m << endl;
}
}

auto lval( const string& s ) -> const string& { return s; }
auto xval( string&& s ) -> string&& { return move( s ); }
auto prval( string s ) -> string { return move( s ); }

auto main()
-> int
{
string l = "An lvalue expression!";
string x = "An xvalue expression!";
string p = "A prvalue expression!";

cout << EC( lval( l ) ) << endl;
cout << EC( xval( move( x ) ) ) << endl;
cout << EC( prval( p ) ) << endl;
cout << endl;
cout << "Calling foo:" << endl;
foo( lval( l ), xval( move( x ) ), prval( p ) );
}
--------------------------------------------------------------------------

Output with MinGW g++ 7.3:


lvalue
xvalue
prvalue

Calling foo:
An lvalue expression! as an lvalue
An xvalue expression! as an xvalue
A prvalue expression! as an xvalue


... where the last two lines are the same expression category.

Not sure how to deal with that even on its own.

Cheers!,

- Alf

[Öö Tiib]

On Thursday, 27 September 2018 16:28:38 UTC+3, Alf P. Steinbach wrote:
> On 27.09.2018 13:39, Öö Tiib wrote:
> > On Thursday, 27 September 2018 13:02:31 UTC+3, Chris Vine wrote:
> >>
> >> The overarching issue is that for there to be a forwarding reference,
> >> the function doing the forwarding must be a template function so that
> >> reference-collapsing can take place. That is basically it: so there is
> >> some "special magic". One consequence is that virtual functions cannot
> >> use forwarding. I think it was a mistake to implement forwarding
> >> through template function type deduction and reference-collapsing: that
> >> seems to me to be an unnecessary mixing of concerns. But it is what C++
> >> does.
> >
> > Technically compiler generates a family of overloads there, family
> > size grows in exponential relation to number of forwarded parameters
> > and if that is virtual function then all are "used" and each must
> > exist and needs entry in vtable of each derived class. Invisible
> > drain of resources. So we are required to explicitly write all
> > overloads of virtual functions that we need.
>
> That's an argument against templating of virtual functions, not against
> argument forwarding in itself. It seems to have an assumption of /static
> type checking/. But if one is satisfied with dynamic type checking, then
> argument forwarding is quite doable for a virtual function.
>
> To examine it more closely I think we should be clear on what argument
> forwarding MEANS in this context.

With forwarding I meant we hypothetically want to accept any arguments
(lvalue, xvalue, or prvalue) to our callable. It matters to us are
those safe to move from, unsafe to move from (but mutable) or impossible
to move from (since immutable). Often we just want to forward that to
things that our hypothetical callable calls. And all the point of
perfection is to do least amount of copies or moves on the way.

> Is it just forwarding a set of actual arguments, e.g. a virtual function
> that gets an argument pack as arguments, and calls an overloaded free
> function F with those arguments? And that can be overridden to e.g. also
> log this call? That's not so difficult with dynamic type checking, one
> can e.g. use a sequence of `std::any` instances.

I suspect that this throws our baby (efficiency of templates) away with
washing water (any) here. But on the other hand option to have JIT
compiling and mutable hashtables of members instead of fixed vtable
of class for to have virtual function templates but not to pay
for what we don't use is also perhaps too lot to request. ;)

> But, do you also or alternatively require that the top-level call's
> expression category - lvalue, xvalue or rvalue - for each argument,
> should be recreated? More involved & difficult, I'd say. Because even
> the standard library's statically type checked forwarding doesn't manage
> to do that 100%:

Yes, but that is likely not a problem when prvalue (movable from) is
turned into xvalue (also movable from) by forwarding.

[Öö Tiib]

On Thursday, 27 September 2018 15:55:59 UTC+3, Chris Vine wrote:
> On Thu, 27 Sep 2018 04:39:03 -0700 (PDT)
> Öö Tiib <oot...@hot.ee> wrote:
> > On Thursday, 27 September 2018 13:02:31 UTC+3, Chris Vine wrote:
> > >
> > > The overarching issue is that for there to be a forwarding reference,
> > > the function doing the forwarding must be a template function so that
> > > reference-collapsing can take place. That is basically it: so there is
> > > some "special magic". One consequence is that virtual functions cannot
> > > use forwarding. I think it was a mistake to implement forwarding
> > > through template function type deduction and reference-collapsing: that
> > > seems to me to be an unnecessary mixing of concerns. But it is what C++
> > > does.
> >
> > Technically compiler generates a family of overloads there, family
> > size grows in exponential relation to number of forwarded parameters
> > and if that is virtual function then all are "used" and each must
> > exist and needs entry in vtable of each derived class. Invisible
> > drain of resources.
>
> I agree that that problem lies behind the reason why virtual functions
> cannot be templated.[1] And you make a good point that because
> of that, virtual functions could not have been used for generic
> forwarding anyway.

Yes, my point was that even when implementation does full program
analysis of all potential yo-yo of virtual calls then what we
face can be still quite a combination explosion. Also I can't
imagine manually writing unit tests for such stuff. :D

I have thought about it in other way. On one hand the "by RVO"
("OUT") parameter passing is actually unavailable, we have really
to return into that hidden parameter. On other hand there are by
value ("SINK-IN"), by reference ("IN-OUT"), by rvalue reference
("MOVE-IN") and by reference to const ("IMMUTABLE-IN") parameters.
So if we could leave the choice between "SINK-IN", "MOVE-IN" and
"IMMUTABLE-IN" parameters up to implementation to choose and optimize
then we would just have two types of parameters ("IN" and "IN-OUT")
left and all the concern about move, copy or forward gone. ;)