lambdas and perfect forwarding

[Eric Niebler]

I'm trying to do something pretty simple, I think: write a lambda that
perfectly forwards its arguments. Something like:

template<class... T>
void g(T &&... t) {}

template<class... T>
std::function<void()> f(T &&... t)
{
return [t...]()->void{g(std::forward<T>(t)...);}
}

Obviously, I want the rvalue-ness of arguments passed to g() to be the
same as that passed to f(). The above doesn't work because the lambda
captures the arguments by value. I could capture by reference, but
that's equally wrong, and likely to create dangling references besides.

The problem becomes obvious when I try to capture a move-only type:

struct moveable
{
moveable() = default;
moveable(moveable const &) = delete;
moveable(moveable &&) = default;
};

template<typename T>
void foo(T && t)
{
auto fun = [t]() mutable {return std::move(t);};
}

int main()
{
foo(moveable());
}

There doesn't seem to be any combination of capture behavior or mutable
lambdas that works. The above fails (on clang-trunk) because it tries to
copy t into the lambda closure instead of moving it.

Is this possible or not? Shouldn't it be? It seems reasonable. I
couldn't find anything in the active core issues list.

Thanks,
Eric

[Ville Voutilainen]

On 30 August 2012 02:47, Eric Niebler <eric.n...@gmail.com> wrote:
> There doesn't seem to be any combination of capture behavior or mutable
> lambdas that works. The above fails (on clang-trunk) because it tries to
> copy t into the lambda closure instead of moving it.

There is no capture-by-move, and you can't initialize the captures by
forwarding them. See
http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2863.html#JP9
That comment talks about move-captures, but perhaps we indeed want
a way to do generic initialization of the captures somehow.

> Is this possible or not? Shouldn't it be? It seems reasonable. I
> couldn't find anything in the active core issues list.

I think it should be, but it's an extension and thus requires a design paper,
and is probably something to target for C++1y.

[Alberto Ganesh Barbati]

Ville has correctly summarized the current status. This is on my wish list too. One informal proposal that was briefly discussed on the C++ reflectors was about allowing this kind of syntax:

[x {expression}] { /* lambda body */ }

which would directly initialize the capture x with the given expression. This would allow capture-by-move by writing:

[x {std::move(x)}] { /* lambda body */ }

However, as Ville noted, there is still no design paper and a few details are not as obvious as they might be (for example: how to deduce the type of x?).

Ganesh

[Eric Niebler]

On 8/29/2012 5:18 PM, Ville Voutilainen wrote:

> On 30 August 2012 02:47, Eric Niebler wrote:
>> There doesn't seem to be any combination of capture behavior or
>> mutable lambdas that works. The above fails (on clang-trunk)
>> because it tries to copy t into the lambda closure instead of
>> moving it.
>
> There is no capture-by-move, and you can't initialize the captures by
> forwarding them. See
> http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2863.html#JP9

Thanks, Ville. Can you help me understand this (from n2863):

> REJECTED
>
> This would be a dangerous feature, allowing "moving" from named
> variables. It would also contradict the change made by paper N2844,
> prohibiting the binding of rvalue references to lvalues, which was
> adopted at the March, 2009 meeting.

Moving from named variables is ok with explicit syntax, like
std::move(x) or casting to T&&. This syntax would also be explicit. The
user has to specify && in the capture list. What exactly is the objection?

I also don't see where in the proposed solution an rvalue ref would bind
to an lvalue. What am I missing?

> That comment talks about move-captures, but perhaps we indeed want a
> way to do generic initialization of the captures somehow.

Interesting. I'm not opposed.

>> Is this possible or not? Shouldn't it be? It seems reasonable. I
>> couldn't find anything in the active core issues list.
>
> I think it should be, but it's an extension and thus requires a
> design paper, and is probably something to target for C++1y.

It's always the trouble: finding somebody qualified and motivated enough
to write a paper. I'm not sure I am. :-P

Thanks,
Eric

[Ville Voutilainen]

On 30 August 2012 22:10, Eric Niebler <eric.n...@gmail.com> wrote:
> Thanks, Ville. Can you help me understand this (from n2863):
>> REJECTED
>> This would be a dangerous feature, allowing "moving" from named
>> variables. It would also contradict the change made by paper N2844,
>> prohibiting the binding of rvalue references to lvalues, which was
>> adopted at the March, 2009 meeting.
> Moving from named variables is ok with explicit syntax, like
> std::move(x) or casting to T&&. This syntax would also be explicit. The
> user has to specify && in the capture list. What exactly is the objection?

The only bit of explicit syntax in the proposed change was allowing && in
the capture list, there's no place where the move() would happen. That
was apparently deemed not explicit enough. The proposal was extension-ish
as a CD1 comment, the CWG was up to its eyeballs in issues, and EWG
was closed. We have learned from that experience that we shouldn't close
EWG even when we are supposedly just bug-fixing a draft rather than
dealing with new ideas.

> I also don't see where in the proposed solution an rvalue ref would bind
> to an lvalue. What am I missing?

The problem would be that

void foo()
{
MovableObject x;
[&&]() { do_things_with(x);}();
}

would move from x. That's as explicit as it would be, and was deemed
too implicit,
keeping in mind the problems we had with rvalue references binding to lvalues
in other contexts.

>> That comment talks about move-captures, but perhaps we indeed want a
>> way to do generic initialization of the captures somehow.
> Interesting. I'm not opposed.

It was mentioned in [c++std-ext-13023] by Alberto Ganesh Barbati, and I later
pointed out that it would handle capture-by-move too. Several other people
have agreed that we should strive for supporting generic
transformations/initializations
of captures rather than just supporting by-ref/by-copy/by-move.

>> I think it should be, but it's an extension and thus requires a
>> design paper, and is probably something to target for C++1y.
> It's always the trouble: finding somebody qualified and motivated enough
> to write a paper. I'm not sure I am. :-P

I expect this issue to be raised again at some point. I boldly expect my
motivation and qualification to be sufficient, but having the sufficient
bandwidth is another matter, and an implementation wouldn't hurt either,
which is even more taxing bandwidth-wise.

[Dave Abrahams]

on Wed Aug 29 2012, Eric Niebler <eric.niebler-AT-gmail.com> wrote:

> I'm trying to do something pretty simple, I think: write a lambda that
> perfectly forwards its arguments.

Seems to me that as long as lambdas are monomorphic, there's no way to
get the special deduction we'd need to make forward<> work. The
non-uniformity vs regular functions of other suggestions I've seen, like
capture-by-move, bother me.

--
Dave Abrahams
BoostPro Computing Software Development Training
http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

[Ville Voutilainen]

On 17 September 2012 15:53, Dave Abrahams <da...@boostpro.com> wrote:
> on Wed Aug 29 2012, Eric Niebler <eric.niebler-AT-gmail.com> wrote:
>> I'm trying to do something pretty simple, I think: write a lambda that
>> perfectly forwards its arguments.
> Seems to me that as long as lambdas are monomorphic, there's no way to
> get the special deduction we'd need to make forward<> work. The
> non-uniformity vs regular functions of other suggestions I've seen, like
> capture-by-move, bother me.

I think there's also a question of whether you want to perfect-forward just
arguments, or also captures. I think being able to capture-by-move would
move lambdas more in the direction of uniformity rather than further away
from it. Currently there are things that I can do with arguments that I can't
do with captures, to me that's lack of uniformity and increased complexity
from the point of view of understanding the rules and their rationale.

[victor.ni...@gmail.com]

Since now (C++14) we have capture by move, is there any reason why the code below should not compile?

template<typename F, typename... T>
auto apply(F&& f, T&&...t)
{
    return [p = std::forward<F>(f), q... = std::forward<T>(t)...](){ p(q...); };
}


Note that I am trying to perfectly forward the arguments of 'apply' to the returned expression for them to be stored either by copy or by move.

Unfortunately, neither g++ 4.9, nor clang++ 3.5 compile this successfully.

A slight modification makes the code compile on both:

template<typename F, typename... T>
auto apply(F&& f, T&&...t)
{
    return [p = std::forward<F>(f), t...](){ p(t...); };
}


But now I am (potentially) paying for an unnecessary copy (, or a lot of them). Do I have to (according to the standard)? We have perfect forwarding for one parameter (f to p), why not for the parameter pack as well?

[Edward Catmur]

On Tuesday, 30 September 2014 10:58:31 UTC+1, victor.ni...@gmail.com wrote:

template<typename F, typename... T>
auto apply(F&& f, T&&...t)
{
    return [p = std::forward<F>(f), q... = std::forward<T>(t)...](){ p(q...); };
}

 

Yes, a pack expansion can't generate variables, whether in a declaration-initialization or in an init-capture. You can achieve perfect forwarding by capturing into a tuple of lvalue/rvalue references and then applying it using the tuple apply from n3915:

template<typename F, typename... T>
auto apply(F&& f, T&&...t)
{

    return [f = std::forward<F>(f),
            t = std::forward_as_tuple<T...>(std::forward<T>(t)...)
            ]() mutable { return std::apply(std::forward<F>(f), t); };
}

Full example: http://coliru.stacked-crooked.com/a/f99e163bc2c5532d

[victor.ni...@gmail.com]

Thank you for your suggestion for a workaround! Capturing as tuple does achieve the behaviour I was talking about, but I was more interested was the following part:

On Wednesday, October 1, 2014 4:17:03 PM UTC+3, Edward Catmur wrote:

Yes, a pack expansion can't generate variables, whether in a declaration-initialization or in an init-capture.

Why does a pack expansion generate variables when default-captured by value, but not when it is init-captured? More importantly, why should we end up with a pack expression if we capture by value and with a tuple when doing perfect forwarding? Is there some reasoning behind why we couldn't have what I believe to be a more consistent behaviour, or was this just overlooked when the c++14 generalized-capture was defined and would call for a proposal?

Thank you again for your input!

[Louis Dionne]

Being able to init-capture a pack with perfect forwarding would make it possible 

to implement std::tuple in a much more compile-efficient and terse way, as explained

in [1]. I would _really_ like to see this in the language.

Louis

[1]: https://groups.google.com/forum/#!topic/comp.lang.c++.moderated/9Fa2Fzlvlxg

[Edward Catmur]

There's a conceptual difference; with a simple-capture it's the same identifier within the lambda body as outside, just with a different referent, while an init-capture introduces a new identifier possibly shadowing one in the enclosing scope. Currently templates (function, class, alias) are the only syntactic constructs that can introduce variadic identifiers. A proposal would definitely be worthwhile.

Some issues you might consider:

What is allowed on the rhs of a variadic init-capture; if it has to be a pack expansion, are other comma sequences allowed?

Alternatively, would it be better to omit the ellipsis on the lhs and use an unexpanded pack on the rhs?

[Edward Catmur]

On 3 Oct 2014 14:26, "Edward Catmur" <eca...@googlemail.com> wrote:
> What is allowed on the rhs of a variadic init-capture; if it has to be a pack expansion, are other comma sequences allowed?

That should have been: does it have to be a pack expansion, or are other comma sequences allowed?