lambdas as stateful function objects

[Paul]

The following code outputs 1 2 2.
I wouldn't have expected this.
1 1 1 seems to make more sense.
Also 1 2 3 would be my second guess.
Apparently the explanation has to do with "stateful function objects"
but I don't understand what this means.

Can anyone explain?

Thanks a lot.

Paul


int main()
{int x = 0;
auto foo = [=] () mutable {

x++;

return x;
};

std::cout << foo() << " ";
auto bar = foo;

std::cout << foo() << " ";


std::cout << bar() << " ";
}

[Alf P. Steinbach]

First call of foo increments its x to 1 and returns that.

Copying foo to bar copies the 1.

Second call to foo increments its x to 2 and returns that.

First call of bar increments its x to 2 and returns that.

Not sure how you could think anything else could make sense?


Cheers & hth.,

- Alf

[Paul]

I still don't get it. In what sense does foo contain an x?
Many people clearly think (erroneously) like me since this
code features in a multiple choice quiz with several options.

If the right output was obvious to everyone, no one would
design such a quiz.

For example, no one would write a c++ quiz question such as the below:

int main()
{
std::cout << 2;
}

Which of these is printed by the above code?

A) -3000000000
B) 2
C) Hello World
D) 34134343434343

Paul

[Alf P. Steinbach]

In the sense of actually containing an x.

You can rewrite the lambda as a class:

class Foo_lambda
{
private:
int x;
public:
auto operator()()
-> int
{
x++;
return x;
}

Foo_lambda( const int x_value )
: x{ x_value }
{}
};

The lambda expression produces an instance of this class, or of a class
pretty much like it.

That instance contains a data member called `x`, which due to the `=`
capture by value, the constructor initializes with the value of the
outer scope's `x`.

If you had not declared the lambda `mutable` then the `operator()` would
effectively have been `const` and couldn't have modified the data
member. I write “effectively” because I don't know the details of the
rules, and e.g. modification of `this` is prevented by treating it as an
rvalue expression instead of treating it as `const`. But whatever the
standard's formal mechanism is, `const` or rvalue or whatever, without
C++17 `mutable` you're prevented from modifying.

[Chris Vine]

I would be surprised if many people think like you. This is a standard
lexical closure and most people will have come across them in python or
javascript (which acquired them from the functional languages) if not
via C++. You might want to look up closures on wikipedia. They can be
confusing to begin with but they have become ubiquitous.

In C++, lambda functions are a function object and the closure is
effected via its capture list. In terms of implementation, in C++ the
function object keeps the variables in its lexical environment which
feature in the capture list as private members of the function object.
You will find that C++ has its quirks, particularly about the way a
lambda expression in a non-static member function captures a datum which
is a member of the class - in effect, by default it takes it by
reference, by capturing the 'this' pointer. For such a case, the
default behaviour would result in you getting the "1 2 3" which was your
second expectation, and _that_ is what normally surprises and confuses
newcomers.

[Paavo Helde]

On 26.02.2017 14:11, Paul wrote:
> The following code outputs 1 2 2.
> I wouldn't have expected this.
> 1 1 1 seems to make more sense.
> Also 1 2 3 would be my second guess.
> Apparently the explanation has to do with "stateful function objects"
> but I don't understand what this means.
>
> Can anyone explain?
>
> Thanks a lot.
>
> Paul
>
>
> int main()
> {int x = 0;
> auto foo = [=] () mutable {

Here, you need to change [=] to [&] in order to get the 1 2 3 output. HTH.

[Chris Vine]

On Sun, 26 Feb 2017 14:26:09 +0000
Chris Vine <chris@cvine--nospam--.freeserve.co.uk> wrote:
[snip]

> I would be surprised if many people think like you. This is a
> standard lexical closure and most people will have come across them
> in python or javascript (which acquired them from the functional
> languages) if not via C++. You might want to look up closures on
> wikipedia. They can be confusing to begin with but they have become
> ubiquitous.
>
> In C++, lambda functions are a function object and the closure is
> effected via its capture list. In terms of implementation, in C++ the
> function object keeps the variables in its lexical environment which
> feature in the capture list as private members of the function object.
> You will find that C++ has its quirks, particularly about the way a
> lambda expression in a non-static member function captures a datum
> which is a member of the class - in effect, by default it takes it by
> reference, by capturing the 'this' pointer. For such a case, the
> default behaviour would result in you getting the "1 2 3" which was
> your second expectation, and _that_ is what normally surprises and
> confuses newcomers.

It might be worth my making the additional point that you got "1 2 2"
because in C++ objects are copied by value, and that includes function
objects. This means that the captured variables held by the lambda
function object are also copied by value when you copied the lambda.
Languages in which objects (including functions) are copied by
reference, such as javascript, will give "1 2 3". As another poster
has already pointed out, you would get a similar effect in C++ if you
captured the variables concerned by reference rather than by value.

[Ben Bacarisse]

I think it's a stretch to call this a standard lexical closure. Whilst
the lambda part itself looks pretty standard., C++'s lambdas are
peculiar in they can capture objects in the environment by copying, and
these copies objectare a copied on assignment. That is perfectly
natural in the context of C++, but you can't easily write that in the
other languages you cite.

> In C++, lambda functions are a function object and the closure is
> effected via its capture list. In terms of implementation, in C++ the
> function object keeps the variables in its lexical environment which
> feature in the capture list as private members of the function object.
> You will find that C++ has its quirks, particularly about the way a
> lambda expression in a non-static member function captures a datum which
> is a member of the class - in effect, by default it takes it by
> reference, by capturing the 'this' pointer. For such a case, the
> default behaviour would result in you getting the "1 2 3" which was your
> second expectation, and _that_ is what normally surprises and confuses
> newcomers.

I have little experience anymore of what surprises newcomers, but 1 2 3
is what you get from writing the seemingly equivalent code in
ECMAScript:

var x = 0;
var foo = function () { x++; return x; };
console.log(foo());
var bar = foo;
console.log(foo());
console.log(bar());

If you try to emulate the "capture by copying" that [=] is doing in C++
you get 1 1 1:

var x = 0;
var foo = function () { var lx = x; lx++; return lx; };
console.log(foo());
var bar = foo;
console.log(foo());
console.log(bar());

And now you are stuck because there is no obvious way to get the
assignment of bar to copy the local variable lx 'inside' foo.

Python is slightly less help as a model of what C++ is doing because you
can't even write x += 1; return x in a lambda. You can write a function
returning function but you might as well just name foo directly:

x = 0;
def foo(): x += 1; return x
print(foo())
bar = foo
print(foo())
print(bar())

This gives a run-time error because x is referenced before being
assigned. You have to either state that you want the global one:

x = 0;
def foo(): global x; x += 1; return x
print(foo())
bar = foo
print(foo())
print(bar())

(giving 1 2 3) or you need to copy x and you go back to getting 1 1 1:

x = 0;
def foo(): lx = x; lx += 1; return lx
print(foo())
bar = foo
print(foo())
print(bar())

In short, experience of other languages with mutable state and function
values will not help much in understanding the C++ code in the question.

--
Ben.

[Chris Vine]

I don't agree. A lexical closure is exactly what it is. The main
distinguishing feature of C++ is that the free variables do not
automatically close over the entire lexical environment: they must be
specified in the capture list (which can of course be made to close
over the entire environment with '=').

Closures work by copying the captured variables, because they outlast
the lifetime of the surrounding environment (and which is why
overriding this by capturing free variables by reference in C++ is so
dangerous). What is meant by "copy" depends on the language.
Javascript for example copies simple numbers by value, and arrays,
objects and functions by reference (strings are immutable so there is
no difference), and most garbage collected languages do something
similar.

> [snip]

> In short, experience of other languages with mutable state and
> function values will not help much in understanding the C++ code in
> the question.

The important issue for a learner to understand is how free variables
in lambdas capture their lexical environment.

After that, the differences in behavior when copying the lambda (and how
often do you do that?) stem only from whether the lambda object, and
any free variables, are copied by reference or by value; this
difference being prompted in part by C++'s lifetime rules for automatic
objects. Copy by reference of lambda function objects can trivially
be implemented in C++ in the OP's code by changing 'auto bar = foo;' to
'auto& bar = foo;', which would then print "1 2 3".

The reverse (copying functions by value) in javascript would require
cloning the function object and all its captured variables. Java (as
opposed to javascript) and python do not permit mutation of free
variables at all. But all these different languages work, with C++, by
lexical closure by free variables.