parameters passed by "expression": lazy/delayed evaluation of parameters

562 views
Skip to first unread message

floria...@gmail.com

unread,
May 14, 2018, 9:16:07 AM5/14/18
to ISO C++ Standard - Future Proposals
Hello everyone,

I would want to propose a new way to pass parameters to functions.
Currently, we have 2 way to pass parameters: by value, and by reference.
(Passing by pointer is really passing by value the address)

However, none of those allow delayed evaluation of parameters.
But this would be useful in some cases: for example overloading operator&& and operator|| (and also operator?: when it will be overloadable).
It could also be used to implement a select function defined like this:
float select(int i, float a, float b, float c) {
 
switch (i) {
   
case 0:
     
return a;

   
case 1:
     
return b;

   
case 2:
     
return c;

 
}
}

The delayed evaluation is important to allow a parameter with side effects not to be executed if not needed, avoiding undefined behaviors:
float *p;

if (p && *p > 0) { ... } // this is valid

bool AND(bool a, bool b) {
  return a && b;
}

if (AND(p, *p > 0)) { ... } // This is not


I thought of 4 ways to express that:

Type qualifier + implicit evaluation:
float select(int i, float @ a, float @ b, float @ c) {
 
switch (i) {
   
case 0:
     
return a;

   
case 1:
     
return b;

   
case 2:
     
return c;

 
}
}

Type qualifier + operator:
float select(int i, float @ a, float @ b, float @ c) {
 
switch (i) {
   
case 0:
     
return @a;

   
case 1:
     
return @b;

   
case 2:
     
return @c;

 
}
}

Type qualifier + callable:
float select(int i, float @ a, float @ b, float @ c) {
 
switch (i) {
   
case 0:
     
return a();

   
case 1:
     
return b();

   
case 2:
     
return c();

 
}
}

STL type:
float select(int i, std::expression<float> a, std::expression<float> b, std::expression<float> c) {
 
switch (i) {
   
case 0:
     
return a();

   
case 1:
     
return b();

   
case 2:
     
return c();

 
}
}


In all the 3 cases, when the select function is called, a functor-like object is created computing the expression, and this object is then passed to the function.
float* p;
int i;

float r = select(i, 1.f, *p, std::sqrt(*p));

// equivalent to:

float r;
switch (i) {
  case 0:
    r = 1.f;
    break;

  case 1:
    r = *p;
    break;

  case 2:
    r = std::sqrt(*p);
    break;
}

I really have the impression this could be very interesting in some corner cases where there is currently no alternatives to do that safely (and efficiently).

Would this be sensible? Were there any attempt to formalize passing by expression?
Which way of expressing this should be preferred?

I would be glad to answer any question.

Florian

Nicol Bolas

unread,
May 14, 2018, 9:41:53 AM5/14/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
On Monday, May 14, 2018 at 9:16:07 AM UTC-4, floria...@gmail.com wrote:
Hello everyone,

I would want to propose a new way to pass parameters to functions.
Currently, we have 2 way to pass parameters: by value, and by reference.
(Passing by pointer is really passing by value the address)

However, none of those allow delayed evaluation of parameters.

P0927 covers this general idea. I'm not sure if it was discussed at Jacksonville.

Nicolas Lesser

unread,
May 14, 2018, 10:02:14 AM5/14/18
to std-pr...@isocpp.org
It wasn't, there wasn't enough time. It will probably be discussed in Raspperwil.

Dilip Ranganathan

unread,
May 14, 2018, 10:07:36 AM5/14/18
to std-pr...@isocpp.org
It was also discussed at length in this forum a while ago:
https://groups.google.com/a/isocpp.org/d/msg/std-proposals/DnxRfo_uQl8/anZ8bFXi4QwJ

floria...@gmail.com

unread,
May 14, 2018, 10:49:18 AM5/14/18
to ISO C++ Standard - Future Proposals
Thank you for the links.
The proposal is exactly what I had in mind (except for the slightly different syntax).
I'm glad to see a well written proposal.

The previous discussion is also interesting.

Corentin

unread,
May 14, 2018, 11:49:37 AM5/14/18
to std-pr...@isocpp.org
While it is a more involved work ( as it depends on reflection and code injection - aka the underlying mechanisms of the meta class proposal), I do think that use-site parameter evaluation can be achieved more generally by having a combination of generalized reflection on arbitrary expression and named code fragments - neither of these things having been proposed yet.
They would however be implementable reasonably easily on top of the code injection facilities the metaclasses proposal calls for,  as we are basically just shuffling and copying AST nodes around.

I did not write a paper, because I'm trying to see if I can apply the design to more macro usages - which is a pain because macro are worse than token soup - but I hope to present that work someday depending on how the reflection and code injections papers evolve.I do however have a blog post about it https://cor3ntin.github.io/posts/undef_preprocessor/#6-to-get-rid-of-macros-we-need-a-new-kind-of-macros

It might also be interesting to see how languages such as rust solve these kind of problems. (Rust uses a full blown token tree for their macros - do not mind the word "macro", it's a loaded word in the C++ world but there are all kind of macros in other languages that do not involve a preprocessor).

Note that this approach would be different than lazy evaluation in that it would be a compile time mechanism whereas your solution has a runtime cost. Both approaches have advantages and draw backs(compile time code expansion leads to bigger binaries, but you don't pay for an extra call at runtime (which might be inline in lot of cases)) , so both approach are certainly worth exploring.

However,  it might be interesting that a lambda-based solution be applied to more than this particular issue.
This paper for a short-lambda syntax ( that unfortunately failed to meet its audience if i remember correctly) might be a good place to start wg21.link/p0573






--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/7fb13c92-2d15-44a0-8aa1-848539f8658d%40isocpp.org.

floria...@gmail.com

unread,
May 17, 2018, 4:12:31 AM5/17/18
to ISO C++ Standard - Future Proposals
After reading the P0927 proposal, it seems that this mechanism + RVO would supersede the piecewise_construct for tuples (and maps):
The construction of the elements will be delayed until the actual destination address is known and where RVO will be applied.

Am I correct with this interpretation?

mihailn...@gmail.com

unread,
May 26, 2018, 4:52:07 AM5/26/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
 - How are these lazy arguments different then taking std::function/function_ref? 

 - On the call site I really doubt not be able to see you are NOT constructing the argument on function call is a good idea - it will make the code misleading
In a way we will repeat the problem with non-const references, where one is not able to tell if the argument is passed by reference or not, without looking at the function interface!

 - But there is also a more serious issue - proliferation of anonymous classes!

Where right now the user will have write clumsy lambda

SaveAction([this]{ return _filepath; });
CopyAction([this]{ return _filepath; });
MoveAction([this]{ return _filepath; });

Sooner or later the user will see the obvious solution to use a variable for all cases

std::function<Filepath()> _filepathSource = [this]{ return _filepath; };

SaveAction(_filepathSource);


The user will NEVER do that if he has transparent lambdas creation, which ultimately will lead to code bloat and/or more optimizer work.

Florian Lemaitre

unread,
May 27, 2018, 12:19:31 PM5/27/18
to mihailn...@gmail.com, ISO C++ Standard - Future Proposals

Lazy arguments are compatible with terse lambda. Having one will not remove the need for the other.

You seem to miss one crucial point with this proposal (P0927): it will not create anonymous classes at all: it will always use a function pointer like this: T (*)(void*)

If I give an example with the proposal syntax, this will be:

int foo([] -> int i) {
 
return i();
}

int bar(int j) {
 
return foo(j+2);
}
int foobar(int j) {
 
return foo(2*j);
}

With lambdas, you will have two instantiation of foo, but in this proposal, you will have a single symbol for foo, which would be equivalent to int foo(int (*)(void*), void*).

So the previous example will be strictly equivalent to:
int foo(int (*i)(void*), void* stack_p) {
 
return i(stack_p);
}

int bar(int j) {
 
int closure(void* stack_p) {
   
int j = *static_cast<int*>(stack_p + /* offset j */);
   
return j+2;
 
}
 
return foo(&closure, __builtin_frame_address(0));
}

int foobar(int j) {
 
int closure(void* stack_p) {
   
int j = *static_cast<int*>(stack_p + /* offset j */);
   
return 2*j;
 
}
 
return foo(&closure, __builtin_frame_address(0));
}

This will not lead to code bloat, nor create any anonymous types.
It will create anonymous functions, sure, but with a known and fixed type. And those functions will be created anyway if you want any form of lazyness delayed evaluation. Even if the call site is aware that it is creating those functions.

As you can see, it is even mush simpler and lighter than creating a std::function std::function_ref. There is no extra burden on the compiler to optimize this: it is just inlining a function pointer when this pointer is known: no heap allocation!


Now, that's true it will be the same call syntax when it is passed by value/ref as when it is passed lazily, giving potential side-effects not be executed.
Guess what? That's already the case in template code when using && and ||: those operators are short-circuiting when used on bools, but maybe overrided by any custom class losing their short-circuiting property.
In fact, this proposal would solve this last problem by allowing implementations of custom classes to also be short-circuiting.

mihailn...@gmail.com

unread,
May 27, 2018, 1:08:29 PM5/27/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com, floria...@gmail.com
I admit I skimmed a bit the part it is just a function pointer. I am also allergic to "lets use lambda for this and that".
Of course the impl. limits their uses compared to normal functors, but I guess that's normal, and that's makes them different.
I personally use late evaluation A LOT so I will keep an eye where something like this could be used in my case, as I tend to sore the functors which evaluate, not unlike the action example above.

Also the fact that we are still on the same stack alleviates the need to be explicit about late evaluation on call site. 

I will think about it more and write back if I have something to comment more. 

The topic of callable objects is very interesting to me. I also believe entire function/functors system needs COMPLETE overhaul and be made core language. 
I have written some "mad scientist" ideas here if you have nothing better to do https://groups.google.com/a/isocpp.org/d/msg/std-proposals/Do4dMUSlS7Q/i5Pp6EKlBAAJ


Edward Catmur

unread,
May 29, 2018, 10:09:43 AM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com, floria...@gmail.com
This seems inefficient; if I have n lazy parameters, the address of the enclosing frame is passed n times when it needs to be passed at most once and (for most ABIs) it does not need to be passed at all, since the called function can just save the stack pointer on entry. Would it not be better to just pass a bare code pointer?

I suppose this would complicate taking the address of such lazy types, but perhaps it would be better if this operation were also forbidden? We don't allow taking the address of a reference per se, after all. If one needs to pass a lazy argument around, one can pass it as another lazy argument, or capture it into a closure type:

int bar([] -> int i) { return i(); }
int baz(std::function_ref<int()> f) { return f(); }

int foo([] -> int i) {

   
return bar(i()) + baz([&] { return i(); };
}

Indeed (and perhaps controversially), if the parentheses-suffixed form is the only way to use the identifier of the parameter, then the parentheses are redundant and can (should) be omitted, such that any use of the identifier (in evaluated context) represents an evaluation of the argument expression (and thus a call through the code pointer):

int bar([] -> int i) { return i; }
int baz(std::function_ref<int()> f) { return f(); }

int foo([] -> int i) {

   
return bar(i) + baz([&] { return i; };
}

The address-of operation would return the address of the result of the argument expression, if the lazy parameter has a reference type, and would otherwise be ill-formed (since a [] -> T lazy parameter would be a prvalue):

int* foo([] -> int& i) { return &i; } // OK
int* bar([] -> int i) { return &i; } // ill-formed, taking address of a prvalue

Again, this would be similar to references, where any use of an identifier of reference type in evaluated context represents a load or store to the bound object and the address-of operation gives the address of the bound object rather than the reference itself.

This would certainly make translation from existing code simple, but maybe it is a step too far?
Message has been deleted

mihailn...@gmail.com

unread,
May 29, 2018, 10:40:53 AM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com, floria...@gmail.com
Question: Why would you ever want to use [] -> int instead of std::function_ref<int()> as a parameter? 

Why limit the interface to that just that callable? 
I am not against the callable itself, but it does not make good interface, does it? 

Edward Catmur

unread,
May 29, 2018, 10:44:34 AM5/29/18
to std-pr...@isocpp.org
On Tue, May 29, 2018 at 3:36 PM, <mihailn...@gmail.com> wrote:


On Tuesday, May 29, 2018 at 5:09:43 PM UTC+3, Edward Catmur wrote:
Question: Why would you ever want to use [] -> int instead of std::function_ref<int()> as a parameter? 

[] -> int is transparent at the call site; std::function_ref<int()> requires the calling code to write a lambda. It is also lighter weight.

Nicol Bolas

unread,
May 29, 2018, 10:48:43 AM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com, floria...@gmail.com
On Tuesday, May 29, 2018 at 10:40:53 AM UTC-4, mihailn...@gmail.com wrote:
Question: Why would you ever want to use [] -> int instead of std::function_ref<int()> as a parameter? 

Because I don't really want to call a function to get a value. I want you to give me an expression that I will evaluate later to produce a value of that type. That this happens through "a function" is an implementation detail, one that I have no interest in as the writer of that function.

Hyman Rosen

unread,
May 29, 2018, 10:52:05 AM5/29/18
to std-pr...@isocpp.org, floria...@gmail.com
On Sat, May 26, 2018 at 4:52 AM <mihailn...@gmail.com> wrote:
 - On the call site I really doubt not be able to see you are NOT constructing the argument on function call is a good idea

This is the way it's done by the language, and this facility should therefore imitate it.
When I say a() && b(), I do not know at the call site whether b() will be evaluated,
and if this construct is inside generic code, I have no way of figuring it out - if the type
of a() is user-defined, b() will always be evaluated, and if it is built-in, that evaluation
will be conditional.  If this proposal does not allow user-defined implementations of
operator&& and operator|| that work just like the built-in ones, then it is a failure.

Constructs that conditionally evaluate are as old as the hills, like Lisp's cond,
(cond (c1 e1) (c2 e2) (c3 e3)), in which at most one of the en is ever
evaluated.  Confusion is averted by naming or familiarity.

mihailn...@gmail.com

unread,
May 29, 2018, 10:53:45 AM5/29/18
to ISO C++ Standard - Future Proposals


On Tuesday, May 29, 2018 at 5:44:34 PM UTC+3, Edward Catmur wrote:
On Tue, May 29, 2018 at 3:36 PM, <mihailn...@gmail.com> wrote:


On Tuesday, May 29, 2018 at 5:09:43 PM UTC+3, Edward Catmur wrote:
...

Question: Why would you ever want to use [] -> int instead of std::function_ref<int()> as a parameter? 

[] -> int is transparent at the call site; std::function_ref<int()> requires the calling code to write a lambda. It is also lighter weight.

What if the user decides to upgrade the call and write a lambda? What if the user wants to pass a callable instance variable? Overloads? 

Hyman Rosen

unread,
May 29, 2018, 11:00:29 AM5/29/18
to std-pr...@isocpp.org, mihailn...@gmail.com, floria...@gmail.com

It may be an implementation detail, but the function is the implementation!
If we're providing the possibility of unevaluated parameters, with the function
in charge of whether they're evaluated, it seems valuable to let the evaluation
look deliberate.  It also sidesteps the question of whether an unevaluated
parameter can be evaluated at most once or more than once, by making
evaluations manifest in the code.

The worst case would be to to have the compiler decide behind the scenes
whether to do the evaluation based on its detection of the first use of the
parameter.

Nicol Bolas

unread,
May 29, 2018, 11:03:29 AM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com
What happens when a user decides to stop using `std::string` and start trying to pass a `std::vector<char>`? They get a compile error for not providing the thing they were told by the API to provide.

How is that any different? You said "give me an expression that results in an integer". You instead tries to give a lambda. That's not "an expression that results in an integer", so you get a compile error.

You can turn any function-that-results-in-an-integer into an expresion-that-results-in-an-integer simply by calling it in-situ:

func([]() -> int{...}());

That makes it clear to everyone that the function call is the expression being captured, not the function itself.

Edward Catmur

unread,
May 29, 2018, 11:12:51 AM5/29/18
to std-pr...@isocpp.org
On Tue, May 29, 2018 at 4:00 PM, Hyman Rosen <hyman...@gmail.com> wrote:
If we're providing the possibility of unevaluated parameters, with the function
in charge of whether they're evaluated, it seems valuable to let the evaluation
look deliberate.  It also sidesteps the question of whether an unevaluated
parameter can be evaluated at most once or more than once, by making
evaluations manifest in the code.

The worst case would be to to have the compiler decide behind the scenes
whether to do the evaluation based on its detection of the first use of the
parameter.

Thanks, I think those are pretty strong arguments in favor of `identifier()` as the invocation syntax and against bare `identifier`. 

(I'd still hope that the committee would consider and reject the bare syntax for posterity, but I recognise that may not be the best use of time.)

Nicol Bolas

unread,
May 29, 2018, 11:33:28 AM5/29/18
to ISO C++ Standard - Future Proposals
On Tuesday, May 29, 2018 at 11:12:51 AM UTC-4, Edward Catmur wrote:
On Tue, May 29, 2018 at 4:00 PM, Hyman Rosen <hyman...@gmail.com> wrote:
If we're providing the possibility of unevaluated parameters, with the function
in charge of whether they're evaluated, it seems valuable to let the evaluation
look deliberate.  It also sidesteps the question of whether an unevaluated
parameter can be evaluated at most once or more than once, by making
evaluations manifest in the code.

The worst case would be to to have the compiler decide behind the scenes
whether to do the evaluation based on its detection of the first use of the
parameter.

Thanks, I think those are pretty strong arguments in favor of `identifier()` as the invocation syntax and against bare `identifier`.

They're arguments for an "evaluation syntax"; not for `identifier()` as that syntax. Yes, we need a way to tell the difference between talking about the captured expression (which you might defer to some other function) and evaluating it. And maybe `identifier()` would be OK.

But I don't like the idea of allowing the user to use `identifier` as if it were a function or functor. Or even as if it were an object. It should be a completely different order of things, fundamentally distinct from anything else in C++. Much like braced-init-lists aren't expressions, captured expressions identifiers should not be objects or functions.

Hyman Rosen

unread,
May 29, 2018, 11:44:22 AM5/29/18
to std-pr...@isocpp.org
On Tue, May 29, 2018 at 11:33 AM Nicol Bolas <jmck...@gmail.com> wrote:
But I don't like the idea of allowing the user to use `identifier` as if it were a function or functor.
Or even as if it were an object. It should be a completely different order of things, fundamentally
distinct from anything else in C++. Much like braced-init-lists aren't expressions, captured
expressions identifiers should not be objects or functions.

 "Thing that can be invoked to execute a block of code and get back a value"
is not a completely different order of things.  It's a function.  Using function-call
syntax therefore makes sense.

mihailn...@gmail.com

unread,
May 29, 2018, 12:12:12 PM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com


On Tuesday, May 29, 2018 at 6:03:29 PM UTC+3, Nicol Bolas wrote:


On Tuesday, May 29, 2018 at 10:53:45 AM UTC-4, mihailn...@gmail.com wrote:


On Tuesday, May 29, 2018 at 5:44:34 PM UTC+3, Edward Catmur wrote:


On Tue, May 29, 2018 at 3:36 PM, <mihailn...@gmail.com> wrote:


On Tuesday, May 29, 2018 at 5:09:43 PM UTC+3, Edward Catmur wrote:
...

Question: Why would you ever want to use [] -> int instead of std::function_ref<int()> as a parameter? 

[] -> int is transparent at the call site; std::function_ref<int()> requires the calling code to write a lambda. It is also lighter weight.

What if the user decides to upgrade the call and write a lambda? What if the user wants to pass a callable instance variable? Overloads? 

What happens when a user decides to stop using `std::string` and start trying to pass a `std::vector<char>`? They get a compile error for not providing the thing they were told by the API to provide.

And that is exactly why we are moving away from std::string as a param to string_view! 
You want a good, general interface. Accepting a lazy arg alone is not one.
 

How is that any different? You said "give me an expression that results in an integer". You instead tries to give a lambda. That's not "an expression that results in an integer", so you get a compile error.

You can turn any function-that-results-in-an-integer into an expresion-that-results-in-an-integer simply by calling it in-situ:
 

And both bind to function_ref hence function_ref is the better interface.

The problem is, the user knows the code is late evaluated - he has seen the declaration. 
Second, for the user there is no practical difference b/w "function-that-results-in-an-integer" and "expresion-that-results-in-an-integer"

Because of this the user will want to pass callables as well. 

With lazy arg interface alone this will be both clumsy and awkward (to say the least).

floria...@gmail.com

unread,
May 29, 2018, 12:41:28 PM5/29/18
to ISO C++ Standard - Future Proposals
You are missing the point: lazy parameters is not about  passing callables to a function, it is about delaying the construction of the parameters in order to skip the construction of not needed ones, or allow some Return Value Optimizations by knowing the final destination of the object.
If the call site is not transparent to this, then you cannot overload properly operator|| for instance (or operator?: if this gets accepted), and makes most use cases harder to write for the user for no good reason.
An "expression-that-results-in-an-integer" has no syntax overhead.

You could argue that an expression is implicitly convertible (without executing it) into a std::function_ref, for instance. This would allow you to have a real object to play with while allowing transparent call site.
But this would make the type system clumsy.
Consider trying to pass a std::function_ref<T()> as a lazy parameter to a function that takes any kind of input:
template <class T>
T foo
(std::function_ref<T()> f) { return f(); }

foo
(1) // ok automatic conversion from 1 to std::function<int()>: lazy evaluation

foo
(std::function_ref<int()>([](){ return 1; })) // std::function_ref<int()> is copied, not lazy evaluated

So expressions implicitly convertible to std::function_ref: bad idea!

Edward Catmur

unread,
May 29, 2018, 1:05:02 PM5/29/18
to std-pr...@isocpp.org
On Tue, May 29, 2018 at 5:12 PM, <mihailn...@gmail.com> wrote:
On Tuesday, May 29, 2018 at 6:03:29 PM UTC+3, Nicol Bolas wrote:
What happens when a user decides to stop using `std::string` and start trying to pass a `std::vector<char>`? They get a compile error for not providing the thing they were told by the API to provide.

And that is exactly why we are moving away from std::string as a param to string_view! 
You want a good, general interface. Accepting a lazy arg alone is not one.

std::string_view is strictly better than std::string as a parameter; it has a smaller footprint than a std::string passed by value, and it requires less indirection than a std::string passed by reference. In addition, the relevant template is instantiated a limited number of times: at most once for basic_string_view<char> per translation unit.

Contra this, std::function_ref must be instantiated once per function type and its constructor must be instantiated once per callable type - once per call, if the callables are closures. Its footprint (two pointers) is at least as large as either a closure type (0-1 pointers, for a capture-by-reference default) or a lazy parameter (1-2 pointers, depending on implementation) and requires a closure to be reified which might otherwise not occur. Its call pointer may result in double-indirection if not inlined, and bloats the symbol table, particularly if debug symbols are enabled. Again, if debugging is enabled or if inlining fails, its call operator pollutes the call stack with irrelevant frames.


How is that any different? You said "give me an expression that results in an integer". You instead tries to give a lambda. That's not "an expression that results in an integer", so you get a compile error.

You can turn any function-that-results-in-an-integer into an expresion-that-results-in-an-integer simply by calling it in-situ:
 

And both bind to function_ref hence function_ref is the better interface.

Using function_ref requires wrapping an expression in a closure - `expr` becomes at minimum `[&] { return expr; }` or quite possibly `[&]() -> decltype(auto) { return expr; }` - a large amount of unnecessary boilerplate. Passing a callable as a lazy parameter requires appending a pair of parentheses.
 
The problem is, the user knows the code is late evaluated - he has seen the declaration. 
Second, for the user there is no practical difference b/w "function-that-results-in-an-integer" and "expresion-that-results-in-an-integer"

Because of this the user will want to pass callables as well. 

Why? How likely are they have callables lying around with precisely the semantics that are required?
 
With lazy arg interface alone this will be both clumsy and awkward (to say the least).

Two characters `()` is clumsy and awkward?


func([]() -> int{...}());

That makes it clear to everyone that the function call is the expression being captured, not the function itself.

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposals+unsubscribe@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.

Nicol Bolas

unread,
May 29, 2018, 1:08:46 PM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com


On Tuesday, May 29, 2018 at 12:12:12 PM UTC-4, mihailn...@gmail.com wrote:


On Tuesday, May 29, 2018 at 6:03:29 PM UTC+3, Nicol Bolas wrote:


On Tuesday, May 29, 2018 at 10:53:45 AM UTC-4, mihailn...@gmail.com wrote:


On Tuesday, May 29, 2018 at 5:44:34 PM UTC+3, Edward Catmur wrote:


On Tue, May 29, 2018 at 3:36 PM, <mihailn...@gmail.com> wrote:


On Tuesday, May 29, 2018 at 5:09:43 PM UTC+3, Edward Catmur wrote:
...

Question: Why would you ever want to use [] -> int instead of std::function_ref<int()> as a parameter? 

[] -> int is transparent at the call site; std::function_ref<int()> requires the calling code to write a lambda. It is also lighter weight.

What if the user decides to upgrade the call and write a lambda? What if the user wants to pass a callable instance variable? Overloads? 

What happens when a user decides to stop using `std::string` and start trying to pass a `std::vector<char>`? They get a compile error for not providing the thing they were told by the API to provide.

And that is exactly why we are moving away from std::string as a param to string_view! 

`vector<char>` is not convertible to `string_view`, nor should it be. If you want to convert them, then you should do it yourself.

Same goes for functions vs. captured expressions. After all, what happens if you want to capture an expression that evaluates to a function? How do you tell the difference if a function is automatically turned into a captured expression?

mihailn...@gmail.com

unread,
May 29, 2018, 1:47:45 PM5/29/18
to ISO C++ Standard - Future Proposals


On Tuesday, May 29, 2018 at 8:05:02 PM UTC+3, Edward Catmur wrote:
On Tue, May 29, 2018 at 5:12 PM, <mihailn...@gmail.com> wrote:
On Tuesday, May 29, 2018 at 6:03:29 PM UTC+3, Nicol Bolas wrote:
What happens when a user decides to stop using `std::string` and start trying to pass a `std::vector<char>`? They get a compile error for not providing the thing they were told by the API to provide.

And that is exactly why we are moving away from std::string as a param to string_view! 
You want a good, general interface. Accepting a lazy arg alone is not one.

std::string_view is strictly better than std::string as a parameter; it has a smaller footprint than a std::string passed by value, and it requires less indirection than a std::string passed by reference. In addition, the relevant template is instantiated a limited number of times: at most once for basic_string_view<char> per translation unit.

Contra this, std::function_ref must be instantiated once per function type and its constructor must be instantiated once per callable type - once per call, if the callables are closures. Its footprint (two pointers) is at least as large as either a closure type (0-1 pointers, for a capture-by-reference default) or a lazy parameter (1-2 pointers, depending on implementation) and requires a closure to be reified which might otherwise not occur. Its call pointer may result in double-indirection if not inlined, and bloats the symbol table, particularly if debug symbols are enabled. Again, if debugging is enabled or if inlining fails, its call operator pollutes the call stack with irrelevant frames.


How is that any different? You said "give me an expression that results in an integer". You instead tries to give a lambda. That's not "an expression that results in an integer", so you get a compile error.

You can turn any function-that-results-in-an-integer into an expresion-that-results-in-an-integer simply by calling it in-situ:
 

And both bind to function_ref hence function_ref is the better interface.

Using function_ref requires wrapping an expression in a closure - `expr` becomes at minimum `[&] { return expr; }` or quite possibly `[&]() -> decltype(auto) { return expr; }` - a large amount of unnecessary boilerplate. Passing a callable as a lazy parameter requires appending a pair of parentheses.
 
The problem is, the user knows the code is late evaluated - he has seen the declaration. 
Second, for the user there is no practical difference b/w "function-that-results-in-an-integer" and "expresion-that-results-in-an-integer"

Because of this the user will want to pass callables as well. 

Why? How likely are they have callables lying around with precisely the semantics that are required?
 
With lazy arg interface alone this will be both clumsy and awkward (to say the least).

Two characters `()` is clumsy and awkward?


Ok, give me a sales pitch. When to choose func_ref and when lazy if I what to present my class to the world. 

On one hand is func_ref which binds to everything but must introduce an explicit lambda to write code inline
, on the other is lazy, in which inline code is the default, but is single single expression only and does not bind to anything (inline only). 

For me, personally func_ref is the better interface as it is the most general, but I am obviously missing something (performance aside) as you all are so exited.

When I will want to use lazy? What are the rules of thumb? What is "the killer app"? 

This is an honest question as I use callables a lot, and will gladly find its place if I can. 
 


func([]() -> int{...}());

That makes it clear to everyone that the function call is the expression being captured, not the function itself.

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.

inkwizyt...@gmail.com

unread,
May 29, 2018, 2:23:21 PM5/29/18
to ISO C++ Standard - Future Proposals
I agree that it could look as function but I think it should not be function, it could have even different ABI than normal function.
Another thing is how many times you can call it, image simple case:
foo(bar(baz()));
`foo` have this lazy parameter, then where should be temporary crated by `baz` stored? and what will happens when you call lazy parameter again?
You already use storage for it. Or lifetime of temporaries in lazy parameters will be different than normal parameters?

mihailn...@gmail.com

unread,
May 29, 2018, 2:40:33 PM5/29/18
to ISO C++ Standard - Future Proposals
 everything b/w foo brackets is a single expression. baz is in the same scope as bar an the call to bar is normal as in []{ bar(baz()); } normal
See one of the post of Florian Lemaitre above for tear down of the process. 

inkwizyt...@gmail.com

unread,
May 29, 2018, 3:14:23 PM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com
 Ok, but then behavior of `5 && foo(bar())` and `myType{} && foo(bar())` is different because value returned by `bar` in second case will be destroyed before whole expression will end.

Bengt Gustafsson

unread,
May 29, 2018, 3:20:47 PM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com
There is a huge difference between passing a callable as parameter (for instance as a std::function) and calling by lazy: A functor will be called each time, producing its value, while the lazy is evaluated at most once, but maybe never. This means that there has to be a flag indicating whether the evaluation of the parameter was done. This flag must be created and pre-cleared by the caller to allow for lazy forwarding. I don't see how the called function could handle this as it doesn't know if the caller created the flag or just passed forwarded by lazy from its caller.

Note that even for types without constructor there must be a flag as the function to be called to evaluate the lazy parameter may be arbitrarily complex, including having side effects and requiring construction of temporaries. These temporareis, at least in the case of a by reference parameter type, will have to live until the outermost function getting the lazy parameter returns. Obviously only the caller knows the nature of the expression so only it can destroy the temporaries (and the returned value) in case the callee actually evaluated it.

Thus it seems that for each lazy parameter the caller must push a pointer to a "lazy block" containing the flag, the function pointer and the resulting data. This lazy block is created on the stack before calling the function. In case the function forwards the lazy parameter lazily it just passes the pointer it was given. When all of this returns the ultimate caller checks the flag to see if it needs to destroy the value and any temporaries.

Exception handling could be interesting to implement, but should not be intractable as we have the flag to check. If there is an exception in the expression itself it unwinds what it did so far and does not set the flag. The caller responsible for destruction must of course check the flag also in an unwinding situation and that seems to be just about it.

Given the fact that evaluation occurs just once I am suspicious whether the operator() is really appropriate, it does indicate that evaluation occurs each time. I would argue that just like for by reference vs. by value parameters we will get used to not having to annotate the parameter usages.

It would be reasonable to let the "evaluation function" be responsible for the flag checking but it is not possible to just pass the function pointer as has been indicated in this thread as the lazy forwarding prevents the stack offset from being knowable when the function gets called. Well maybe some kind of thunk could be introduced to add an offset to the stack pointer for each forwarding level but that would be an ABI definition issue, not a language issue.

floria...@gmail.com

unread,
May 29, 2018, 4:20:14 PM5/29/18
to ISO C++ Standard - Future Proposals
 The "evaluation function" needs to access the stack frame (or related) of the caller anyway, so adding the flag here and let the "evaluation function" set it seems reasonable.

I like the idea to put temporaries into the caller stack frame instead of the "evaluation function" stack frame. If all temporaries have a trivial destructor, you could even say that there is no flag for its destruction.

If we want to optimize lazy parameter forwarding, we need to pass the stack frame (or related) from the caller for each lazy parameter.
Otherwise, we could pass the stack frame pointer once to the callee, and recreate a new "evaluation function" to forward a lazy parameter that could just tail call the previous one.
You could even say that you don't need to pass the stack frame explicitly, and let the callee retrieve it with the equivalent to __builtin_frame_address(1).
Of course, this is an ABI and an optimization problem, not a language one as Bengt said.

Nicol Bolas

unread,
May 29, 2018, 4:21:21 PM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com
On Tuesday, May 29, 2018 at 1:47:45 PM UTC-4, mihailn...@gmail.com wrote:
Ok, give me a sales pitch. When to choose func_ref and when lazy if I what to present my class to the world. 

Stop thinking about what is going on literally between the caller and callee, and focus on the meaning of the API. You use lazy evaluation if you want a function to capture an expression. You use function_ref if you want a function to call a given function.

It's that simple.

`std::sort` does not take an expression to be evaluated; it takes a function to be called. A callback system does not take an expression to be evaluated; it takes a function to be called at the designated time. And so forth.

By contrast, a lazy `operator&&` does not take a function to be called; it takes an expression to be conditionally evaluated. That's what `&&` does; it acts on two expressions. The person who gives you those expressions thinks of them as expressions, not functions that evaluate to a value.

`lazy_emplace` does not take a function which returns `T`; it takes an expression whose evaluation ultimately generates a prvalue of type `T`, which can be shoved into the container. Indeed, in the latter case, we could declare that a braced-init-list can be used in such an expression capture (as long as the expression capture's type is explicitly stated), thus allowing you to do `vector::lazy_emplace({value1, value2})`. Or even with designated initializers for aggregates: `lazy_emplace({.x = value1, .y = value2})`. The compiler has enough information to know what's going on, all without having to mention the type redundantly at the cite of the call.

I cannot conceive of an interface where you would confuse one with the other. Where you want to take something that is either a function to be called or an expression to be evaluated. In the mind of the person writing the API, there is a clear idea of how the function will be used, and to me, it's always one or the other.

On one hand is func_ref which binds to everything but must introduce an explicit lambda to write code inline
, on the other is lazy, in which inline code is the default, but is single single expression only and does not bind to anything (inline only). 

For me, personally func_ref is the better interface as it is the most general, but I am obviously missing something (performance aside) as you all are so exited.

When I will want to use lazy? What are the rules of thumb? What is "the killer app"?

The two examples I gave above are the "killer app". Transparent lazy evaluation equivalent to C++'s `&&` and `||` behavior, and the ability to have `lazy_emplace` work. Neither of these cases are things for which passing a function is the natural interface.

The rule of thumb is that you use a lazy expression when it makes sense based on what you're doing. `lazy_emplace` does it because it is the most natural way to make it work. The natural low-level user code is:

new(container.get_mem_for_new_item()) auto(expr);

So we invert that by having `container.lazy_emplace` apply `expr` internally. Same thing, just done without having to expose the user to the low-level guts. And since `expr` could throw exceptions, those now happen within the purview of `container`'s member functions, which can abort the object creation properly.

Nicol Bolas

unread,
May 29, 2018, 4:25:16 PM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com
Some other rules of thumb:

1. You can't affect how lazy expressions evaluate. That is, you can't give them values; they are simply a thing that generates a value. That's another reason `std::sort` and most algorithms wouldn't use them.

2. You can't evaluate them more than once. UB ought to result if you try.

3. You can't pass a lazy expression outside of your call stack. Nor can you return one.

So if you ever need to do any of these things, you are clearly using the wrong tool.

Bengt Gustafsson

unread,
May 29, 2018, 6:03:12 PM5/29/18
to ISO C++ Standard - Future Proposals, mihailn...@gmail.com
@florian: Yes, good point: The fact that you don't need a "already evaluated" flag for simple expressions clearly points to the fact that the caller side should handle the flag, i.e. the callee should just call the function in case it needs the value. The function can elect to use a flag or re-evalutate if it can figure out that runtime cost is low, no side effects occur and no temporary destructors to call. Using thunks to adjust stack frame offsets is probably wise as it is going to be fairly rare that forwarding occurs (especially in multiple levels).

@Nicol: Prohibiting using a lazy parameter more than once seems overly restrictive and very error prone, especially if no diagnostic is required. Furthermore the caller must keep track of whether the function was called anyway to handle temporary destruction properly.

Edward Catmur

unread,
May 29, 2018, 7:22:11 PM5/29/18
to std-pr...@isocpp.org


On Tue, 29 May 2018, 23:03 Bengt Gustafsson, <bengt.gu...@beamways.com> wrote:
@florian: Yes, good point: The fact that you don't need a "already evaluated" flag for simple expressions clearly points to the fact that the caller side should handle the flag, i.e. the callee should just call the function in case it needs the value. The function can elect to use a flag or re-evalutate if it can figure out that runtime cost is low, no side effects occur and no temporary destructors to call. Using thunks to adjust stack frame offsets is probably wise as it is going to be fairly rare that forwarding occurs (especially in multiple levels).

@Nicol: Prohibiting using a lazy parameter more than once seems overly restrictive and very error prone, especially if no diagnostic is required. Furthermore the caller must keep track of whether the function was called anyway to handle temporary destruction properly.

In the proposal under discussion, there is no caching of the result of the evaluation of the passed in expression and so no flag required and no destructor call. Multiple use results in multiple evaluation of the initializer expression, which may result in UB if the evaluation of the expression falsifies its own preconditions (e.g. moving out a container or smart pointer that is required to be non empty). 


Den tisdag 29 maj 2018 kl. 22:25:16 UTC+2 skrev Nicol Bolas:
On Tuesday, May 29, 2018 at 4:21:21 PM UTC-4, Nicol Bolas wrote:
On Tuesday, May 29, 2018 at 1:47:45 PM UTC-4, mihailn...@gmail.com wrote:
On one hand is func_ref which binds to everything but must introduce an explicit lambda to write code inline
, on the other is lazy, in which inline code is the default, but is single single expression only and does not bind to anything (inline only). 

For me, personally func_ref is the better interface as it is the most general, but I am obviously missing something (performance aside) as you all are so exited.

When I will want to use lazy? What are the rules of thumb? What is "the killer app"?

The two examples I gave above are the "killer app". Transparent lazy evaluation equivalent to C++'s `&&` and `||` behavior, and the ability to have `lazy_emplace` work. Neither of these cases are things for which passing a function is the natural interface.

The rule of thumb is that you use a lazy expression when it makes sense based on what you're doing. `lazy_emplace` does it because it is the most natural way to make it work. The natural low-level user code is:

new(container.get_mem_for_new_item()) auto(expr);

So we invert that by having `container.lazy_emplace` apply `expr` internally. Same thing, just done without having to expose the user to the low-level guts. And since `expr` could throw exceptions, those now happen within the purview of `container`'s member functions, which can abort the object creation properly.

Some other rules of thumb:

1. You can't affect how lazy expressions evaluate. That is, you can't give them values; they are simply a thing that generates a value. That's another reason `std::sort` and most algorithms wouldn't use them.

2. You can't evaluate them more than once. UB ought to result if you try.

3. You can't pass a lazy expression outside of your call stack. Nor can you return one.

So if you ever need to do any of these things, you are clearly using the wrong tool.

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

Nicol Bolas

unread,
May 29, 2018, 8:00:44 PM5/29/18
to ISO C++ Standard - Future Proposals
On Tuesday, May 29, 2018 at 7:22:11 PM UTC-4, Edward Catmur wrote:
On Tue, 29 May 2018, 23:03 Bengt Gustafsson, <bengt.gu...@beamways.com> wrote:
@florian: Yes, good point: The fact that you don't need a "already evaluated" flag for simple expressions clearly points to the fact that the caller side should handle the flag, i.e. the callee should just call the function in case it needs the value. The function can elect to use a flag or re-evalutate if it can figure out that runtime cost is low, no side effects occur and no temporary destructors to call. Using thunks to adjust stack frame offsets is probably wise as it is going to be fairly rare that forwarding occurs (especially in multiple levels).

@Nicol: Prohibiting using a lazy parameter more than once seems overly restrictive and very error prone, especially if no diagnostic is required. Furthermore the caller must keep track of whether the function was called anyway to handle temporary destruction properly.

In the proposal under discussion, there is no caching of the result of the evaluation of the passed in expression and so no flag required and no destructor call. Multiple use results in multiple evaluation of the initializer expression, which may result in UB if the evaluation of the expression falsifies its own preconditions (e.g. moving out a container or smart pointer that is required to be non empty). 

My issue with allowing multiple evaluation to be well-defined in some cases is that there's no way to ensure that at the compiler level. There's no way to write a concept or a `static_assert` or something that would cause compilation failure if the user does the wrong thing.

It would be safer to just declare it UB period.

floria...@gmail.com

unread,
May 30, 2018, 4:40:45 AM5/30/18
to ISO C++ Standard - Future Proposals


Le mercredi 30 mai 2018 01:22:11 UTC+2, Edward Catmur a écrit :


On Tue, 29 May 2018, 23:03 Bengt Gustafsson, <bengt.gu...@beamways.com> wrote:
@florian: Yes, good point: The fact that you don't need a "already evaluated" flag for simple expressions clearly points to the fact that the caller side should handle the flag, i.e. the callee should just call the function in case it needs the value. The function can elect to use a flag or re-evalutate if it can figure out that runtime cost is low, no side effects occur and no temporary destructors to call. Using thunks to adjust stack frame offsets is probably wise as it is going to be fairly rare that forwarding occurs (especially in multiple levels).

@Nicol: Prohibiting using a lazy parameter more than once seems overly restrictive and very error prone, especially if no diagnostic is required. Furthermore the caller must keep track of whether the function was called anyway to handle temporary destruction properly.

In the proposal under discussion, there is no caching of the result of the evaluation of the passed in expression and so no flag required and no destructor call. Multiple use results in multiple evaluation of the initializer expression, which may result in UB if the evaluation of the expression falsifies its own preconditions (e.g. moving out a container or smart pointer that is required to be non empty).
 
The problem here is not caching the result. If you want to evaluate the expression multiple times, then passing an actual callable object would be preferable. So caching is not needed.
The real problem is temporaries lifetime. Let me give you an example:
void print(std::string_view sv) { std::cout << sv << std::endl; }
void print_lazy([] -> std::string_view sv) { std::cout << sv() << std::endl; }

void foo0() {
 
print(std::string("foo")); // ok: temporary string is kept alive until after the execution of print
}
void foo1() {
  print_lazy
(std::string("foo")); // std::string is generated within "evaluation function" and cannot be kept alive until after the execution of print_lazy
}
void foo2() {
 
print([]() -> std::string_view { return std::string("foo"); }) // same issue as above, but made explicit for better understanding
}

If temporaries are destroyed by the "evaluation function", print and print_lazy cannot be semantically equivalent, and lazy parameters would bring many lifetime issues that are not desirable.
If foo0 is correct (and it is), then foo1 should also be correct, but this cannot be if temporary lifetime is bound to the "evaluation function"

The idea was the temporaries use the caller storage (in the caller stack frame), and are destroyed by the caller after the callee execution.
Here we would need to store a flag for the caller to know if the temporaries need to be destroyed or not.
Here could be how the compiler generates print_lazy and foo1:
void print_lazy(std::string_view (*sv_expr)(void*), void* stack_p) {
  std
::cout << sv_expr(stack_p) << std::endl; // no lifetime management here
}

void foo1() {
 
// lazy parameter storage
 
bool __lazy_executed = false;
  alignas
(alignof(std::string)) std::byte __lazy_str_storage[sizeof(std::string)];

 
// lazy parameter executor
  std
::string_view __lazy(void* stack_p) {
    std
::string* p = new(stack_p + /* __lazy_str_storage offset */) std::string("foo");

   
// ensure the string is deleted if std::string_view construction throws    
   
auto str_deleter = std::make_scope_exit([p]{ p->~std::string(); })

    std
::string_view sv = std::string_view(*p); // no exception guarantee for this constructor

   
// if creation of the string throws, the object is not created and don't need to be destroyed
   
// The flag will not be set in that case as the exception is not caught
   
*static_cast<bool*>(stack_p + /* __lazy_executed offset */) = true;
   
   
return sv; // "NRVO" is mandatory here
 
}

 
// call to print_lazy
  print_lazy
(&__lazy, __builtin_stack_address(0));

 
// destroy the temporary string:
 
if (__lazy_executed) {
   
static_cast<std::string*>(__lazy_str_storage)->~std::string();
 
}
}
Of course, this would be generated by the compiler and it would not need to be achievable with the same efficiency if written by hand (the NRVO here is trivial to do for the compiler in this case).
Also, this is an implementation example to show it is possible, but other implementations with the same guarantees would also be possible.

I hope this little explanation will make things clearer...

floria...@gmail.com

unread,
May 30, 2018, 4:50:53 AM5/30/18
to ISO C++ Standard - Future Proposals
I screwed up the exception handling, scope_exit will destroy  std::string even if it doesn't throw, which is not what we want.
Here is the example fixed:
void print_lazy(std::string_view (*sv_expr)(void*), void* stack_p) {
  std
::cout << sv_expr(stack_p) << std::endl; // no lifetime management here
}

void foo1() {
 
// lazy parameter storage
 
bool __lazy_executed = false;
  alignas
(alignof(std::string)) std::byte __lazy_str_storage[sizeof(std::string)];

 
// lazy parameter executor
  std
::string_view __lazy(void* stack_p) {
    std
::string* p = new(stack_p + /* __lazy_str_storage offset */) std::string("foo");

   
// ensure the string is deleted if std::string_view construction throws
   
try {
     
std::string_view sv = std::string_view(*p); // no exception guarantee for this constructor
   
} catch (...) {
      p
->~std::string();
     
throw;
   
}

Edward Catmur

unread,
May 30, 2018, 5:45:02 AM5/30/18
to std-pr...@isocpp.org


On Wed, 30 May 2018, 09:40 , <floria...@gmail.com> wrote:


Le mercredi 30 mai 2018 01:22:11 UTC+2, Edward Catmur a écrit :


On Tue, 29 May 2018, 23:03 Bengt Gustafsson, <bengt.gu...@beamways.com> wrote:
@florian: Yes, good point: The fact that you don't need a "already evaluated" flag for simple expressions clearly points to the fact that the caller side should handle the flag, i.e. the callee should just call the function in case it needs the value. The function can elect to use a flag or re-evalutate if it can figure out that runtime cost is low, no side effects occur and no temporary destructors to call. Using thunks to adjust stack frame offsets is probably wise as it is going to be fairly rare that forwarding occurs (especially in multiple levels).

@Nicol: Prohibiting using a lazy parameter more than once seems overly restrictive and very error prone, especially if no diagnostic is required. Furthermore the caller must keep track of whether the function was called anyway to handle temporary destruction properly.

In the proposal under discussion, there is no caching of the result of the evaluation of the passed in expression and so no flag required and no destructor call. Multiple use results in multiple evaluation of the initializer expression, which may result in UB if the evaluation of the expression falsifies its own preconditions (e.g. moving out a container or smart pointer that is required to be non empty).
 
The problem here is not caching the result. If you want to evaluate the expression multiple times, then passing an actual callable object would be preferable. So caching is not needed.
The real problem is temporaries lifetime. Let me give you an example:
void print(std::string_view sv) { std::cout << sv << std::endl; }
void print_lazy([] -> std::string_view sv) { std::cout << sv() << std::endl; }

void foo0() {
 
print(std::string("foo")); // ok: temporary string is kept alive until after the execution of print
}
void foo1() {
  print_lazy
(std::string("foo")); // std::string is generated within "evaluation function" and cannot be kept alive until after the execution of print_lazy
}
void foo2() {
 
print([]() -> std::string_view { return std::string("foo"); }) // same issue as above, but made explicit for better understanding
}

If temporaries are destroyed by the "evaluation function", print and print_lazy cannot be semantically equivalent, and lazy parameters would bring many lifetime issues that are not desirable.
If foo0 is correct (and it is), then foo1 should also be correct, but this cannot be if temporary lifetime is bound to the "evaluation function"

The idea was the temporaries use the caller storage (in the caller stack frame), and are destroyed by the caller after the callee execution.
Here we would need to store a flag for the caller to know if the temporaries need to be destroyed or not.

Thanks, that's very clear. So the same ABI can still be used (the callee does not need to know whether the expression involves temporaries), but because an evaluation may use stack space in the caller, the callee is strictly forbidden from evaluating the expression more than once. This would be UB so a check would be emitted in debug builds and omitted in optimized release builds. 

If multiple evaluation were really desired, a different ABI would be required such that the callee could supply scratch stack space to the evaluation and invoke code to destroy temporaries. But I'm not aware of any use cases for multiple evaluation. 


--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

Nicol Bolas

unread,
May 30, 2018, 9:29:38 AM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
On Wednesday, May 30, 2018 at 4:40:45 AM UTC-4, floria...@gmail.com wrote:
Le mercredi 30 mai 2018 01:22:11 UTC+2, Edward Catmur a écrit :


On Tue, 29 May 2018, 23:03 Bengt Gustafsson, <bengt.gu...@beamways.com> wrote:
@florian: Yes, good point: The fact that you don't need a "already evaluated" flag for simple expressions clearly points to the fact that the caller side should handle the flag, i.e. the callee should just call the function in case it needs the value. The function can elect to use a flag or re-evalutate if it can figure out that runtime cost is low, no side effects occur and no temporary destructors to call. Using thunks to adjust stack frame offsets is probably wise as it is going to be fairly rare that forwarding occurs (especially in multiple levels).

@Nicol: Prohibiting using a lazy parameter more than once seems overly restrictive and very error prone, especially if no diagnostic is required. Furthermore the caller must keep track of whether the function was called anyway to handle temporary destruction properly.

In the proposal under discussion, there is no caching of the result of the evaluation of the passed in expression and so no flag required and no destructor call. Multiple use results in multiple evaluation of the initializer expression, which may result in UB if the evaluation of the expression falsifies its own preconditions (e.g. moving out a container or smart pointer that is required to be non empty).
 
The problem here is not caching the result. If you want to evaluate the expression multiple times, then passing an actual callable object would be preferable. So caching is not needed.
The real problem is temporaries lifetime. Let me give you an example:
void print(std::string_view sv) { std::cout << sv << std::endl; }
void print_lazy([] -> std::string_view sv) { std::cout << sv() << std::endl; }

void foo0() {
 
print(std::string("foo")); // ok: temporary string is kept alive until after the execution of print
}
void foo1() {
  print_lazy
(std::string("foo")); // std::string is generated within "evaluation function" and cannot be kept alive until after the execution of print_lazy
}
void foo2() {
 
print([]() -> std::string_view { return std::string("foo"); }) // same issue as above, but made explicit for better understanding
}

If temporaries are destroyed by the "evaluation function", print and print_lazy cannot be semantically equivalent, and lazy parameters would bring many lifetime issues that are not desirable.
If foo0 is correct (and it is), then foo1 should also be correct, but this cannot be if temporary lifetime is bound to the "evaluation function"

Functions that use lazily evaluated expressions are not intended to be equivalent to their non-lazy counterparts. That's why we give them lazily evaluated expressions; we don't want them to be equivalent.

Yes, temporaries manifested in a lazy expression will not be extended to the lifetime in which the original expression was passed. This is a good thing. In your example, there is no lifetime issue at all. Temporaries generated by `print_lazy`'s lazy evaluation will be destroyed after `std::cout` is executed, in accord with C++'s usual lifetime rules.

This is a great example of why I say we shouldn't think of lazy evaluation as a function call. Because it isn't a function call and shouldn't act like one. A normal C++ function cannot spawn a temporary whose lifetime is extended to the end of an expression.

A lazy expression must be able to do this. Evaluating one ought to be exactly identical to doing a copy-and-paste of the expression at the point of evaluation. Otherwise, there's just no point in bothering.

Edward Catmur

unread,
May 30, 2018, 10:14:02 AM5/30/18
to std-pr...@isocpp.org
On Wed, May 30, 2018 at 2:29 PM, Nicol Bolas <jmck...@gmail.com> wrote:
Functions that use lazily evaluated expressions are not intended to be equivalent to their non-lazy counterparts. That's why we give them lazily evaluated expressions; we don't want them to be equivalent.

Yes, temporaries manifested in a lazy expression will not be extended to the lifetime in which the original expression was passed. This is a good thing. In your example, there is no lifetime issue at all. Temporaries generated by `print_lazy`'s lazy evaluation will be destroyed after `std::cout` is executed, in accord with C++'s usual lifetime rules.

This is a great example of why I say we shouldn't think of lazy evaluation as a function call. Because it isn't a function call and shouldn't act like one. A normal C++ function cannot spawn a temporary whose lifetime is extended to the end of an expression.

A lazy expression must be able to do this. Evaluating one ought to be exactly identical to doing a copy-and-paste of the expression at the point of evaluation. Otherwise, there's just no point in bothering.

OK, so you're saying that in

void print_lazy([] -> std::string_view sv) { std::cout << sv(); std::cout << std::endl; }
//                                                            ^ #1                      ^ #2
void foo1() { print_lazy(std::string("foo")); }

the temporary std::string should be destructed at #1, not at #2. 

I agree that that would be a better semantic, but it implies a different ABI to that proposed (in passing) in P0927; rather than a single code pointer code whose invocation evaluates the expression, we need two code pointers with the second destructing temporaries, or a single code pointer with signature T(void* caller_stack, enum class action { evaluate, destruct_temporaries}). The caller would still need to reserve stack space for any temporaries in the lazily evaluated expression, but would not need to maintain a flag indicating whether temporaries had been constructed, as calling the code pointer to perform cleanup would be the responsibility of the callee.

Nicol Bolas

unread,
May 30, 2018, 10:31:05 AM5/30/18
to ISO C++ Standard - Future Proposals


On Wednesday, May 30, 2018 at 10:14:02 AM UTC-4, Edward Catmur wrote:
On Wed, May 30, 2018 at 2:29 PM, Nicol Bolas <jmck...@gmail.com> wrote:
Functions that use lazily evaluated expressions are not intended to be equivalent to their non-lazy counterparts. That's why we give them lazily evaluated expressions; we don't want them to be equivalent.

Yes, temporaries manifested in a lazy expression will not be extended to the lifetime in which the original expression was passed. This is a good thing. In your example, there is no lifetime issue at all. Temporaries generated by `print_lazy`'s lazy evaluation will be destroyed after `std::cout` is executed, in accord with C++'s usual lifetime rules.

This is a great example of why I say we shouldn't think of lazy evaluation as a function call. Because it isn't a function call and shouldn't act like one. A normal C++ function cannot spawn a temporary whose lifetime is extended to the end of an expression.

A lazy expression must be able to do this. Evaluating one ought to be exactly identical to doing a copy-and-paste of the expression at the point of evaluation. Otherwise, there's just no point in bothering.

OK, so you're saying that in

void print_lazy([] -> std::string_view sv) { std::cout << sv(); std::cout << std::endl; }
//                                                            ^ #1                      ^ #2
void foo1() { print_lazy(std::string("foo")); }

the temporary std::string should be destructed at #1, not at #2. 

Yes.

I agree that that would be a better semantic, but it implies a different ABI to that proposed (in passing) in P0927; rather than a single code pointer code whose invocation evaluates the expression, we need two code pointers with the second destructing temporaries, or a single code pointer with signature T(void* caller_stack, enum class action { evaluate, destruct_temporaries}). The caller would still need to reserve stack space for any temporaries in the lazily evaluated expression, but would not need to maintain a flag indicating whether temporaries had been constructed, as calling the code pointer to perform cleanup would be the responsibility of the callee.

ABI is essentially irrelevant, because the only way you can have lazy expressions work correctly is if the function(s) that uses them are inlined, relative to the function that provided the expression.

The only cross-ABI way to make lazy expressions work is to make them an actual function, with the expected C++ semantics. At which point... they're not lazy expressions anymore. They're just a transparent way to turn an expression into a (lighter-weight) lambda.

Basically, functions that capture lazy expressions have to be treated like template functions to some degree.

floria...@gmail.com

unread,
May 30, 2018, 10:46:39 AM5/30/18
to ISO C++ Standard - Future Proposals
I have the impression you misunderstood my point, so let me rephrase it.

In my example, there is 3 expressions we need to consider:
print_lazy(std::string("foo")): call expression from the caller
std::string("foo"): lazy expression defined in the caller, executed in the callee
std::cout << sv_expr() << std::endl: expression in the callee triggering the execution of the lazy expression

To be fair, the lazy expression returns a std::string_view, so should be implicitly transformed into std::string_view(std::string("foo"))

Now my point is, the temporary std::string object created by the lazy expression must outlive the callee expression (std::cout << sv_expr() << std::endl).
But if you follow naively lifetime rules, this temporary will be destroyed after the std::string_view is created and before std::cout << sv_expr() is executed.
And the program will try to access unallocated data.
This is the issue I pointed out and that need to be solved.
If a proposal makes this code undefined behavior, then the whole feature is broken from the beginning.

Actually, I even think the lazy expression should outlive the whole execution of the caller expression (print_lazy(std::string("foo"))).
This is what we expect from a regular expression, I don't see why it should be different with lazy expressions.

What I proposed is to extend the lifetime of the std::string temporary until the caller expression (print_lazy(std::string("foo"))) is completely done by letting the caller the responsibility to destroy it.
I also proposed a possible implementation of this solution.

Just a sidenote: when I talk about function here and I show passing functions, it's only a way for me to show a possible implementation to make the concept easier to understand.
Those functions should not appear at the language level.



Le mercredi 30 mai 2018 16:31:05 UTC+2, Nicol Bolas a écrit :


On Wednesday, May 30, 2018 at 10:14:02 AM UTC-4, Edward Catmur wrote:
On Wed, May 30, 2018 at 2:29 PM, Nicol Bolas <jmck...@gmail.com> wrote:
Functions that use lazily evaluated expressions are not intended to be equivalent to their non-lazy counterparts. That's why we give them lazily evaluated expressions; we don't want them to be equivalent.

Yes, temporaries manifested in a lazy expression will not be extended to the lifetime in which the original expression was passed. This is a good thing. In your example, there is no lifetime issue at all. Temporaries generated by `print_lazy`'s lazy evaluation will be destroyed after `std::cout` is executed, in accord with C++'s usual lifetime rules.

This is a great example of why I say we shouldn't think of lazy evaluation as a function call. Because it isn't a function call and shouldn't act like one. A normal C++ function cannot spawn a temporary whose lifetime is extended to the end of an expression.

A lazy expression must be able to do this. Evaluating one ought to be exactly identical to doing a copy-and-paste of the expression at the point of evaluation. Otherwise, there's just no point in bothering.

OK, so you're saying that in

void print_lazy([] -> std::string_view sv) { std::cout << sv(); std::cout << std::endl; }
//                                                            ^ #1                      ^ #2
void foo1() { print_lazy(std::string("foo")); }

the temporary std::string should be destructed at #1, not at #2. 

Yes.

I don't think this is reasonable as it is completely different from current lifetime rules, without any benefit.
 

I agree that that would be a better semantic, but it implies a different ABI to that proposed (in passing) in P0927; rather than a single code pointer code whose invocation evaluates the expression, we need two code pointers with the second destructing temporaries, or a single code pointer with signature T(void* caller_stack, enum class action { evaluate, destruct_temporaries}). The caller would still need to reserve stack space for any temporaries in the lazily evaluated expression, but would not need to maintain a flag indicating whether temporaries had been constructed, as calling the code pointer to perform cleanup would be the responsibility of the callee.

ABI is essentially irrelevant, because the only way you can have lazy expressions work correctly is if the function(s) that uses them are inlined, relative to the function that provided the expression.

That's not true, there are many ways to implement this to have the correct behavior without inlining. Edward's solution is one.
And as I said before, I don't what you call "the correct behavior" is what we want.
 

The only cross-ABI way to make lazy expressions work is to make them an actual function, with the expected C++ semantics. At which point... they're not lazy expressions anymore. They're just a transparent way to turn an expression into a (lighter-weight) lambda.

Yes, so what? C++ is ABI agnostic, you repeat this often enough. And now you want to specify language constructions to solve an implementation/ABI issue?
Remember that the concept of inlining itself is not part of C++. This is a useful optimization that makes sense only at the assembly level.
 

Basically, functions that capture lazy expressions have to be treated like template functions to some degree.

This is where you did not follow the proposal. There is no need to create a template like function to capture lazy expressions.
And my example implementation proves it. That's why I wrote it, to explain how it could be possible to not have template-like nor lambda-like constructions for this to work.
It doesn't even rely on inlining to be efficient. Inlining will make faster though, but I see no reason for it to be required.

Nicol Bolas

unread,
May 30, 2018, 11:25:44 AM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
On Wednesday, May 30, 2018 at 10:46:39 AM UTC-4, floria...@gmail.com wrote:
I have the impression you misunderstood my point, so let me rephrase it.

In my example, there is 3 expressions we need to consider:
print_lazy(std::string("foo")): call expression from the caller
std::string("foo"): lazy expression defined in the caller, executed in the callee
std::cout << sv_expr() << std::endl: expression in the callee triggering the execution of the lazy expression

To be fair, the lazy expression returns a std::string_view, so should be implicitly transformed into std::string_view(std::string("foo"))

Now my point is, the temporary std::string object created by the lazy expression must outlive the callee expression (std::cout << sv_expr() << std::endl).

Why "must" it? Just because it would have if the expression weren't lazily evaluated?

But if you follow naively lifetime rules, this temporary will be destroyed after the std::string_view is created and before std::cout << sv_expr() is executed.

That's only true if you think of `sv_expr()` as a function call.

If you think of it as being the exact equivalent to `std::cout << std::string_view(std::string("foo"));`, then it works just fine. Evaluating that expression will manifest a temporary. And per C++ rules, by the way that temporary is used within the evaluated expression, the lifetime of that temporary will be the lifetime of the whole expression in which it is used. And the "whole expression" includes `std::cout <<`.

This is why thinking of lazy expressions as functions is a bad thing. They aren't functions; they don't act like functions, and they can do things functions cannot normally do.

And the program will try to access unallocated data.
This is the issue I pointed out and that need to be solved.
If a proposal makes this code undefined behavior, then the whole feature is broken from the beginning.

Actually, I even think the lazy expression should outlive the whole execution of the caller expression (print_lazy(std::string("foo"))).
This is what we expect from a regular expression, I don't see why it should be different with lazy expressions.

I don't see why we should expect lazy expressions to behave exactly like non-lazy ones.

Nicol Bolas

unread,
May 30, 2018, 11:35:47 AM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
On Wednesday, May 30, 2018 at 10:46:39 AM UTC-4, floria...@gmail.com wrote:
 

I agree that that would be a better semantic, but it implies a different ABI to that proposed (in passing) in P0927; rather than a single code pointer code whose invocation evaluates the expression, we need two code pointers with the second destructing temporaries, or a single code pointer with signature T(void* caller_stack, enum class action { evaluate, destruct_temporaries}). The caller would still need to reserve stack space for any temporaries in the lazily evaluated expression, but would not need to maintain a flag indicating whether temporaries had been constructed, as calling the code pointer to perform cleanup would be the responsibility of the callee.

ABI is essentially irrelevant, because the only way you can have lazy expressions work correctly is if the function(s) that uses them are inlined, relative to the function that provided the expression.

That's not true, there are many ways to implement this to have the correct behavior without inlining. Edward's solution is one.
And as I said before, I don't what you call "the correct behavior" is what we want.
 
The only cross-ABI way to make lazy expressions work is to make them an actual function, with the expected C++ semantics. At which point... they're not lazy expressions anymore. They're just a transparent way to turn an expression into a (lighter-weight) lambda.

Yes, so what? C++ is ABI agnostic, you repeat this often enough. And now you want to specify language constructions to solve an implementation/ABI issue?

No, I want to specify lazy expressions so that they're expressions, not function calls. So that they work exactly as if you had copy-and-pasted the expression at the site of use.

The only way to do that is to allow the compiler to "instantiate" every function that captures a lazy expression. That is, there can't be some generic interface between caller and callee; the compiler must generate special-case code at every capture of an expression.

What you want makes lazy expressions into function calls.

Remember that the concept of inlining itself is not part of C++.

Tell that to `constexpr` functions. Those are required to be inlined (in the sense that a TU that uses a `constexpr` function must be able to see its definition). The same goes for template functions; you can't instantiate a template without having its definition visible to you.

That's what I'm talking about here.

mihailn...@gmail.com

unread,
May 30, 2018, 12:04:45 PM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com


On Wednesday, May 30, 2018 at 6:35:47 PM UTC+3, Nicol Bolas wrote:
On Wednesday, May 30, 2018 at 10:46:39 AM UTC-4, floria...@gmail.com wrote:
 

I agree that that would be a better semantic, but it implies a different ABI to that proposed (in passing) in P0927; rather than a single code pointer code whose invocation evaluates the expression, we need two code pointers with the second destructing temporaries, or a single code pointer with signature T(void* caller_stack, enum class action { evaluate, destruct_temporaries}). The caller would still need to reserve stack space for any temporaries in the lazily evaluated expression, but would not need to maintain a flag indicating whether temporaries had been constructed, as calling the code pointer to perform cleanup would be the responsibility of the callee.

ABI is essentially irrelevant, because the only way you can have lazy expressions work correctly is if the function(s) that uses them are inlined, relative to the function that provided the expression.

That's not true, there are many ways to implement this to have the correct behavior without inlining. Edward's solution is one.
And as I said before, I don't what you call "the correct behavior" is what we want.
 
The only cross-ABI way to make lazy expressions work is to make them an actual function, with the expected C++ semantics. At which point... they're not lazy expressions anymore. They're just a transparent way to turn an expression into a (lighter-weight) lambda.

Yes, so what? C++ is ABI agnostic, you repeat this often enough. And now you want to specify language constructions to solve an implementation/ABI issue?

No, I want to specify lazy expressions so that they're expressions, not function calls. So that they work exactly as if you had copy-and-pasted the expression at the site of use.

Any particular reason to want them to behave exactly like that? Any pointer or examples of why this is required?  

To this point I understood layzies are like this (correct me if I am wrong)
void f()
{
  call(string("hello"));
}

becomes

void f()
{
 {
  auto s = string("hello");
  call(s);
 }
}

However the auto s = string("hello") is executed not before the function call, but from within the function call, going back to the original context with all the locals visible and so on.

That model seems to me pretty "natural" and easy to understand. 

What are the benefits of the model you are talking about, how easy it is to be implemented and at what cost?
What are the downsides? 

floria...@gmail.com

unread,
May 30, 2018, 12:14:49 PM5/30/18
to ISO C++ Standard - Future Proposals


Le mercredi 30 mai 2018 17:25:44 UTC+2, Nicol Bolas a écrit :
On Wednesday, May 30, 2018 at 10:46:39 AM UTC-4, floria...@gmail.com wrote:
I have the impression you misunderstood my point, so let me rephrase it.

In my example, there is 3 expressions we need to consider:
print_lazy(std::string("foo")): call expression from the caller
std::string("foo"): lazy expression defined in the caller, executed in the callee
std::cout << sv_expr() << std::endl: expression in the callee triggering the execution of the lazy expression

To be fair, the lazy expression returns a std::string_view, so should be implicitly transformed into std::string_view(std::string("foo"))

Now my point is, the temporary std::string object created by the lazy expression must outlive the callee expression (std::cout << sv_expr() << std::endl).

Why "must" it? Just because it would have if the expression weren't lazily evaluated?

Because you just said (below in your reply) so: it should behave as if the lazy expression were copied into the callee expression:
so in that case: std::cout << sv_expr() << std::endl; should be equivalent to std::cout << std::string_view(std::string("foo")) << std::endl;

If you consider the latter, the temporary std::string is not destroyed before the execution of the whole expression is complete: it outlives the expression (even if it is destroyed just before the next expression).
That is exactly what I said. Even though, I think it is not enough.


But if you follow naively lifetime rules, this temporary will be destroyed after the std::string_view is created and before std::cout << sv_expr() is executed.

That's only true if you think of `sv_expr()` as a function call.

If you think of it as being the exact equivalent to `std::cout << std::string_view(std::string("foo"));`, then it works just fine. Evaluating that expression will manifest a temporary. And per C++ rules, by the way that temporary is used within the evaluated expression, the lifetime of that temporary will be the lifetime of the whole expression in which it is used. And the "whole expression" includes `std::cout <<`.

This is why thinking of lazy expressions as functions is a bad thing. They aren't functions; they don't act like functions, and they can do things functions cannot normally do.

I don't think lazy expressions as functions, but it is not a replacement either. If you want replacement, this is not lazy parameters you want, but some kind of macros.
 

And the program will try to access unallocated data.
This is the issue I pointed out and that need to be solved.
If a proposal makes this code undefined behavior, then the whole feature is broken from the beginning.

Actually, I even think the lazy expression should outlive the whole execution of the caller expression (print_lazy(std::string("foo"))).
This is what we expect from a regular expression, I don't see why it should be different with lazy expressions.

I don't see why we should expect lazy expressions to behave exactly like non-lazy ones.

To be consistent. To avoid random bugs. To ease the understandability of the concept. To avoid dangling references because of some very arcane rules.
Choose the one you prefer.
For me the difference is: it can be delayed or not executed at all. But the rest should be identical.

Apart from syntax at the callee site, there should be no difference between a call without lazy parameters, and a call with lazy parameters where those have not side effect and are actually used by the callee.
This is possible only if the temporaries outlives the caller expression.

For instance:
void print(std::string_view sv) {
  std
::string_view sv2 = sv;
  std
::cout << sv2 << std::endl;

  std
::cerr << sv2 << std::endl;

}

void print_lazy([] -> std::string_view sv_expr) {
  std
::string_view sv2 = sv();
  std
::cout << sv2 << std::endl;

  std
::cerr << sv2 << std::endl;

}

void foo0() { print(std::string("foo")); } // definitely correct
void foo1() { print_lazy(std::string("foo")); } // Is this correct?
void foo2() { print([] -> std::string_view { return std::string("foo"); }()); } // not correct
foo0() is correct, there is no doubt about it.
foo2() is not correct, and I think that's perfectly fine.

But with what you say, foo1() is not correct (as std::string("foo") has been destroyed before printing).
What I'm saying is: foo1() should also be correct.


 
...
 
The only cross-ABI way to make lazy expressions work is to make them an actual function, with the expected C++ semantics. At which point... they're not lazy expressions anymore. They're just a transparent way to turn an expression into a (lighter-weight) lambda.

Yes, so what? C++ is ABI agnostic, you repeat this often enough. And now you want to specify language constructions to solve an implementation/ABI issue?
No, I want to specify lazy expressions so that they're expressions, not function calls. So that they work exactly as if you had copy-and-pasted the expression at the site of use.

This is not lazy expressions, this is macros...
 

The only way to do that is to allow the compiler to "instantiate" every function that captures a lazy expression. That is, there can't be some generic interface between caller and callee; the compiler must generate special-case code at every capture of an expression.

Again, no: Edward's implementation gave you exactly the effect you want.
The compiler needs to generate code for the lazy expression at every call site, of course.
But there is no need to "instantiate" the function that will use the lazy expression: this has been proved many times now.

And there is no problem about generating piece of code on the fly at each call site, otherwise templates would be dead...
 

What you want makes lazy expressions into function calls.

That's definitely not what I want. Otherwise I wouldn't propose to extend lifetime of temporaries...
 

Remember that the concept of inlining itself is not part of C++.

Tell that to `constexpr` functions. Those are required to be inlined (in the sense that a TU that uses a `constexpr` function must be able to see its definition). The same goes for template functions; you can't instantiate a template without having its definition visible to you.

constexpr functions are not required to be inlined. They are required to have the definition visible I agree. But a compiler not inlining a call to a constexpr function is still compliant.
The inline keyword is not about inlining. It is about multiple definitions in multiple TUs...

Back to point: even what you say is not needed: it is possible to call a function taking lazy parameters without knowing the definition of that function. My implementation should make it pretty clear.

Nicol Bolas

unread,
May 30, 2018, 12:42:06 PM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com, mihailn...@gmail.com
On Wednesday, May 30, 2018 at 12:04:45 PM UTC-4, mihailn...@gmail.com wrote:
On Wednesday, May 30, 2018 at 6:35:47 PM UTC+3, Nicol Bolas wrote:
On Wednesday, May 30, 2018 at 10:46:39 AM UTC-4, floria...@gmail.com wrote:
 

I agree that that would be a better semantic, but it implies a different ABI to that proposed (in passing) in P0927; rather than a single code pointer code whose invocation evaluates the expression, we need two code pointers with the second destructing temporaries, or a single code pointer with signature T(void* caller_stack, enum class action { evaluate, destruct_temporaries}). The caller would still need to reserve stack space for any temporaries in the lazily evaluated expression, but would not need to maintain a flag indicating whether temporaries had been constructed, as calling the code pointer to perform cleanup would be the responsibility of the callee.

ABI is essentially irrelevant, because the only way you can have lazy expressions work correctly is if the function(s) that uses them are inlined, relative to the function that provided the expression.

That's not true, there are many ways to implement this to have the correct behavior without inlining. Edward's solution is one.
And as I said before, I don't what you call "the correct behavior" is what we want.
 
The only cross-ABI way to make lazy expressions work is to make them an actual function, with the expected C++ semantics. At which point... they're not lazy expressions anymore. They're just a transparent way to turn an expression into a (lighter-weight) lambda.

Yes, so what? C++ is ABI agnostic, you repeat this often enough. And now you want to specify language constructions to solve an implementation/ABI issue?

No, I want to specify lazy expressions so that they're expressions, not function calls. So that they work exactly as if you had copy-and-pasted the expression at the site of use.

Any particular reason to want them to behave exactly like that? Any pointer or examples of why this is required?  

To this point I understood layzies are like this (correct me if I am wrong)
void f()
{
  call(string("hello"));
}

becomes

void f()
{
 {
  auto s = string("hello");
  call(s);
 }
}

However the auto s = string("hello") is executed not before the function call, but from within the function call, going back to the original context with all the locals visible and so on.

That model seems to me pretty "natural" and easy to understand.

A function in C++ is isolated from any non-global state outside of its parameters. That's a basic aspect of how functions work.

The model you're talking about breaks this. A parameter will start manipulating memory outside of that function's scope and outside of the scope of that function's parameters. It will create temporaries to objects outside of that function's scope. It will begin the lifetime of objects whose lifetime ends well outside of its own scope. That now makes it difficult to reason about the results of that expression.

For example, consider the case where one of the sub-expressions of a captured expression throws, and the function that evaluates the expression catches it, so that the exception doesn't get back to the function that provided the expression. Well, what happens to the temporaries created by the captured expression prior to the throw?

In normal expressions (and in copy-and-paste lazy evaluation), the way this works is normal. Any temporaries created are destroyed by stack unwinding. Those temporaries were created in the current scope, so when the scope exits, those temporaries are destroyed.

But with your way, temporaries are manifested outside of the scope they're created in. They are supposed to live beyond that scope. So if you catch the exception before the scope leaves... what happens to them? When do they get destroyed? What is the right answer?

If those temporaries are not destroyed, then now you have a situation where some of the temporaries were successfully created and some were not. How does that work when you leave the capturing function? And if they are destroyed... how does that make sense? You haven't unwound the stack past their scope, so there's no reason for them to be destroyed.

So I would say that copy-and-paste evaluation seems more "natural" and "easy to understand".

floria...@gmail.com

unread,
May 30, 2018, 12:47:14 PM5/30/18
to ISO C++ Standard - Future Proposals
All those questions have been answered and solved by the example implementation I gave.
Execptions look natural.

The lazy evaluation scope is a bit different from other scopes in the sense that it is not nested with other scopes, it is in "parallel". But lifetime of temporaries within this scope follow classical rules.

floria...@gmail.com

unread,
May 30, 2018, 1:00:10 PM5/30/18
to ISO C++ Standard - Future Proposals
If you want more complete answer, here I go:

If an exception is thrown by the lazy expression, the lazy expression code destroys the successfully created objects, and lets the flag as it is (so the caller will not try to destroy any of them).
After that, the exception is forwarded to the callee with classical means and the callee can catch the exception if needed.

And what if the lazy evaluation doesn't throw, but the callee does:
The caller will destroy the lazy scope as usual... (this is something I forgot in my implementation, but trivially fixable with a std::scope_exit for instance)


Also, you seem to mix scope and storage: the scope of the lazy evaluation is completely and nicely defined: it is the same scope as if it were not lazy.
The storage used depends on the ABI (how the stack frames are handled), but that's completely implementation defined.

Nicol Bolas

unread,
May 30, 2018, 1:39:38 PM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
I don't think this answered the question I asked, so allow me to provide some code:

void foo([]->std::string str)
{
 
try { auto str2 = str(); }
 
catch(...)
 
{
 
}
 
//bottom of code
}

foo
(std::string("foo") + func_that_throws() + std::string("bar"));

OK, the captured expression contains several temporaries.

Let's say that the evaluation order for + for this implementation does things purely left-to-right. So it creates a temporary `string("foo")`. It then gets a temporary `string` from `func_that_throws()` and does `operator+` on them, resulting in a prvalue. That prvalue will then manifest a temporary to be added to the temporary `string("bar")`.

Now of course, `func_that_throws` is so named because it throws. So my question is this: given the above evaluation order, which temporaries are still alive when you reach that `catch` statement?

The only correct answer I can see is "none of them". And if that's the case, why is it OK for those temporaries to be destroyed if you reach "bottom of code" through the `catch` statement and not through non-exception execution?

And if some of those temporaries are alive and some are not... how does that even work? Is there a boolean for each temporary in the captured expression to say if it's alive?

And what if the lazy evaluation doesn't throw, but the callee does:
The caller will destroy the lazy scope as usual... (this is something I forgot in my implementation, but trivially fixable with a std::scope_exit for instance)


Also, you seem to mix scope and storage: the scope of the lazy evaluation is completely and nicely defined: it is the same scope as if it were not lazy.

But it's not the same scope. Scope for automatic storage duration objects start when they are constructed. That happens when the expression is evaluated. And when the block that creates an automatic storage duration object ends, that object will be destroyed (temporaries may be destroyed earlier, but they never outlive the block that created them).

Your scoping is breaking those rules. My version doesn't need to.

Edward Catmur

unread,
May 30, 2018, 2:01:05 PM5/30/18
to std-pr...@isocpp.org
Similarly, in the scheme I describe, the EH routine for the lazy expression where the exception is thrown is invoked first; it cleans up its own temporaries. The callee's EH routine is next; it invokes the temporary destruction routines of those lazy expressions that have already been invoked successfully in the current full-expression, just prior to cleaning up the respective return values. So destructors are invoked in the correct order and as appropriate, with no changes needed to the exception handling system. 

Hyman Rosen

unread,
May 30, 2018, 2:01:44 PM5/30/18
to std-pr...@isocpp.org
Can't we specify that given void foo([] -> T parm);, a call foo(expr) is treated as foo([&] -> T { return expr; }), and then let the semantics flow from that?  (With a touch of magic so that foo isn't generic, but that should be fine for a parameterless lambda returning a specific type, right?)

Edward Catmur

unread,
May 30, 2018, 2:09:07 PM5/30/18
to std-pr...@isocpp.org


On Wed, 30 May 2018, 19:01 Hyman Rosen, <hyman...@gmail.com> wrote:
Can't we specify that given void foo([] -> T parm);, a call foo(expr) is treated as foo([&] -> T { return expr; }), and then let the semantics flow from that?  (With a touch of magic so that foo isn't generic, but that should be fine for a parameterless lambda returning a specific type, right?)


That would be a very bad idea if T is std::string_view and expr is std::string("foo"). 

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

floria...@gmail.com

unread,
May 30, 2018, 2:21:57 PM5/30/18
to ISO C++ Standard - Future Proposals
@Hyman: that would imply to many lifetime issues:
void print(std::string_view sv) { std::cout << sv << std::endl; }
void print_lazy([] -> std::string_view sv_expr) { std::cout << sv_expr << std::endl; }
template <class F> void print_fun(F&& f) { std::cout << std::forward<F>(f)() << std::endl; }

print(std::string("foo")); // This is correct
print_fun
([] -> std::string_view { return std::string("foo"); } // temporary string will be destroyed before being printed
print_lazy
(std::string("foo")); // What do we want?

For the lazy one, my point of view is: we want it to be correct, but with your suggestion, it will be equivalent to print_fun that will not be correct as the string is destroyed too early.

@Nicol: Edward made a good and quick summary.
So in your example, when the throwing function throws, the lazy expression destroys the two strings already constructed (with classical exception handling for example), does not set the magic flag to true, and forward the exception to the callee (foo in your example).
So at the beginning of the catch, no temporaries are alive.

Then when the callee has finished, the caller will check the flag: it will still be false as it has not been set to true by the lazy expression, and will therefore not destroy the temporaries.

After the execution of the lazy expression (but in still in the callee), either all temporaries are alive (and the caller will need to destroy them), or none.


Concerning the scope, I agree with you that in the lazy case, the scope will start after, but it will still be the same (access to the same data, same behavior regarding exceptions) as the non-lazy expression scope.
The difference is: a lazy expression scope might/will outlive the scope that triggered its execution, but the rest is exactly the same.

I hope things are clearer now, but if that's not the case, I would be glad to answer more of your questions.

Nicol Bolas

unread,
May 30, 2018, 2:53:01 PM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
On Wednesday, May 30, 2018 at 2:21:57 PM UTC-4, floria...@gmail.com wrote:
@Nicol: Edward made a good and quick summary.
So in your example, when the throwing function throws, the lazy expression destroys the two strings already constructed (with classical exception handling for example), does not set the magic flag to true, and forward the exception to the callee (foo in your example).
So at the beginning of the catch, no temporaries are alive.

Then when the callee has finished, the caller will check the flag: it will still be false as it has not been set to true by the lazy expression, and will therefore not destroy the temporaries.

After the execution of the lazy expression (but in still in the callee), either all temporaries are alive (and the caller will need to destroy them), or none.

So, why is it OK for those temporaries to be destroyed if there's an exception, but they must be extended to outside of the evaluator if there is no exception? That's my point: if it's OK in one case for the temporaries to no longer exist, then it ought to be OK in general.

If the goal is to make lazy expression evaluation work like non-lazy evaluation, then this design fails. If an expression throws and it wasn't lazily captured, the caller is the first one who sees it (since they're the ones who evaluated it). If it was lazily captured, the function evaluating the expression can swallow it, such that the caller never sees it.

That is, the same logic you use to say that the code providing the expression ought to be providing storage for it is the same logic that could be used to say that the throw ought to originate in the caller, just like it would otherwise appear to. That is, if a lazy expression throws, the code between the evaluator and the provider of the expression unrolls immediately, and `catch` statements are only considered from the site of the one who originally provided the expression.

Concerning the scope, I agree with you that in the lazy case, the scope will start after, but it will still be the same (access to the same data, same behavior regarding exceptions) as the non-lazy expression scope.
The difference is: a lazy expression scope might/will outlive the scope that triggered its execution, but the rest is exactly the same.

Everything is the same... except for the parts where it is different. But that means it's not the same. This would be the first time that the creation of an automatic object extends beyond the block that invokes that creation.

You can't say that isn't new.

inkwizyt...@gmail.com

unread,
May 30, 2018, 3:06:12 PM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com


On Wednesday, May 30, 2018 at 7:39:38 PM UTC+2, Nicol Bolas wrote:
>
>    On Wednesday, May 30, 2018 at 1:00:10 PM UTC-4, floria...@gmail.com wrote:
>
>   >      Le mercredi 30 mai 2018 18:47:14 UTC+2, floria...@gmail.com a écrit :
>
>
>    I don't think this answered the question I asked, so allow me to provide some code:
>
>    void foo([]->std::string str)
>    {
>      try { auto str2 = str(); }
>      catch(...)
>      {
>      }
>      //bottom of code
>    }
>
>   foo(std::string("foo") + func_that_throws() + std::string("bar"));
>
>   OK, the captured expression contains several temporaries.
>
>    Let's say that the evaluation order for + for this implementation does things purely left-to-right. So it creates a temporary `string("foo")`. It then gets a temporary `string` from `func_that_throws()` and does `operator+` on them, resulting in a prvalue. That prvalue will then manifest a temporary to be added to the temporary `string("bar")`.
>
>    Now of course, `func_that_throws` is so named because it throws. So my question is this: given the above evaluation order, which temporaries are still alive when you reach that `catch` statement?
>
>    The only correct answer I can see is "none of them". And if that's the case, why is it OK for those temporaries to be destroyed if you reach "bottom of code" through the `catch` statement and not through non-exception execution?
>
>    And if some of those temporaries are alive and some are not... how does that even work? Is there a boolean for each temporary in the captured expression to say if it's alive?


IMHO behavior of this two lines (with `z1` and `z2`) should be indistinguishable:
int foo(bool t, []->int a1, []->int a2){ return t ? a1() : a2(); }

auto z1 = p ? bar() : baz();
auto z2 = foo(p, bar(), baz());
or
bool test([]->bool a1, []->bool a2, []->bool a3) { return a1() && a2() && a3(); }

auto z1 = foo() && bar() && baz();
auto z2 = test(foo(), bar(), baz());

Overall I think that parameter could be multiple times evaluated but it will return same value every time, storage for it will be from caller and caller will store flag that will check if paramere was aready evaluated.
This value is already needed today because if you have `&&` you can create temporaries that need be destroyed after this expression.
Exception will leave parameter in uninitialized state and try unwind current function, you can catch this exception and try call it again:
T try_catch([]->T try_, []->T catch_)
{
   
try
   
{
       
return try_();
   
}
   
catch(...)
   
{
       
return catch_();
   
}
}

If `try_` throw then from caller perspective `try_` was not call at all. If we call any other parameter before then clean up will be responsible of caller.
One parameter can have "block" of temporales guarded by one flag. Exception in it will destroy rest that was already created leaving whole block uninitlalized.

T try_again
(int i, []->T try_)
{
   
while(i--)
   
{
       
try
       
{
           
return try_();
       
}
       
catch(...)
       
{
       
}
   
}
   
return T();
}

We try again to call this param, first fail leave parameter unevaluated, then second will proceed if there was no previous call.

Again in current C++ this behavior need be implemented to support `?:` and `&&` we simply need only to expose part of this to programmers.

floria...@gmail.com

unread,
May 30, 2018, 3:16:24 PM5/30/18
to ISO C++ Standard - Future Proposals


Le mercredi 30 mai 2018 20:53:01 UTC+2, Nicol Bolas a écrit :
On Wednesday, May 30, 2018 at 2:21:57 PM UTC-4, floria...@gmail.com wrote:
@Nicol: Edward made a good and quick summary.
So in your example, when the throwing function throws, the lazy expression destroys the two strings already constructed (with classical exception handling for example), does not set the magic flag to true, and forward the exception to the callee (foo in your example).
So at the beginning of the catch, no temporaries are alive.

Then when the callee has finished, the caller will check the flag: it will still be false as it has not been set to true by the lazy expression, and will therefore not destroy the temporaries.

After the execution of the lazy expression (but in still in the callee), either all temporaries are alive (and the caller will need to destroy them), or none.

So, why is it OK for those temporaries to be destroyed if there's an exception, but they must be extended to outside of the evaluator if there is no exception? That's my point: if it's OK in one case for the temporaries to no longer exist, then it ought to be OK in general.

That's exactly like a non lazy expression: temporaries are destroyed at the end of the scope. If there is an exception, the scope ends earlier.
I think I realise what your problem is: the lazy scope can outlive the scope triggering its execution. And yes, that's new.
 

If the goal is to make lazy expression evaluation work like non-lazy evaluation, then this design fails. If an expression throws and it wasn't lazily captured, the caller is the first one who sees it (since they're the ones who evaluated it). If it was lazily captured, the function evaluating the expression can swallow it, such that the caller never sees it.

True, the exception will not be seen by the callee if the expression is not lazy. And if the expression is lazy, the callee can swallow the exception. But that would be the case with any proposal allowing some kind of lazy/delayed/deferred expression.
Even your proposal to make it as if it was copy pasted into the callee will change the exception handler order. Or with callables.
 

That is, the same logic you use to say that the code providing the expression ought to be providing storage for it is the same logic that could be used to say that the throw ought to originate in the caller, just like it would otherwise appear to. That is, if a lazy expression throws, the code between the evaluator and the provider of the expression unrolls immediately, and `catch` statements are only considered from the site of the one who originally provided the expression.

That would be a possibility, but I'm not sure we want that. We would need to find some use cases for both way to handle exceptions.
The problem I see with that is: in a function taking a lazy parameter, you would not be able to write an exception handler without relying on destructors (or std::scope_exit).
It would not be possible to use a try-catch even if the catch catches anything and forwards the exception.
 

Concerning the scope, I agree with you that in the lazy case, the scope will start after, but it will still be the same (access to the same data, same behavior regarding exceptions) as the non-lazy expression scope.
The difference is: a lazy expression scope might/will outlive the scope that triggered its execution, but the rest is exactly the same.

Everything is the same... except for the parts where it is different. But that means it's not the same. This would be the first time that the creation of an automatic object extends beyond the block that invokes that creation.

You can't say that isn't new.

I agree, it's new. but it's not as different as you might think. In the implementation, it is very different, but coneptually (at the language level) it is not that different.
For me, as I already said, the difference is the destruction can outlive the scope where it has been started.
Another way to see it is: the scope start point is delayed and will be triggered by another (inner) scope.

floria...@gmail.com

unread,
May 30, 2018, 3:32:43 PM5/30/18
to ISO C++ Standard - Future Proposals


Le mercredi 30 mai 2018 21:06:12 UTC+2, Marcin Jaczewski a écrit :


On Wednesday, May 30, 2018 at 7:39:38 PM UTC+2, Nicol Bolas wrote:
>
>    On Wednesday, May 30, 2018 at 1:00:10 PM UTC-4, floria...@gmail.com wrote:
>
>   >      Le mercredi 30 mai 2018 18:47:14 UTC+2, floria...@gmail.com a écrit :
>
>
>    I don't think this answered the question I asked, so allow me to provide some code:
>
>    void foo([]->std::string str)
>    {
>      try { auto str2 = str(); }
>      catch(...)
>      {
>      }
>      //bottom of code
>    }
>
>   foo(std::string("foo") + func_that_throws() + std::string("bar"));
>
>   OK, the captured expression contains several temporaries.
>
>    Let's say that the evaluation order for + for this implementation does things purely left-to-right. So it creates a temporary `string("foo")`. It then gets a temporary `string` from `func_that_throws()` and does `operator+` on them, resulting in a prvalue. That prvalue will then manifest a temporary to be added to the temporary `string("bar")`.
>
>    Now of course, `func_that_throws` is so named because it throws. So my question is this: given the above evaluation order, which temporaries are still alive when you reach that `catch` statement?
>
>    The only correct answer I can see is "none of them". And if that's the case, why is it OK for those temporaries to be destroyed if you reach "bottom of code" through the `catch` statement and not through non-exception execution?
>
>    And if some of those temporaries are alive and some are not... how does that even work? Is there a boolean for each temporary in the captured expression to say if it's alive?


IMHO behavior of this two lines (with `z1` and `z2`) should be indistinguishable:
int foo(bool t, []->int a1, []->int a2){ return t ? a1() : a2(); }

auto z1 = p ? bar() : baz();
auto z2 = foo(p, bar(), baz());
or
bool test([]->bool a1, []->bool a2, []->bool a3) { return a1() && a2() && a3(); }

auto z1 = foo() && bar() && baz();
auto z2 = test(foo(), bar(), baz());

Yes, if those are different, then the feature has failed its purpose.
 
Overall I think that parameter could be multiple times evaluated but it will return same value every time, storage for it will be from caller and caller will store flag that will check if paramere was aready evaluated.

I don't think it is a good idea to allow multiple evaluations of the expression. If you need to reuse the result, you can store it in a local variable and reuse it as many times as you want.
The problem I see with this automatic local variable stored/cached is: how would you enforce RVO to construct the object in-place (one of the biggest feature of this proposal).

I would say we should not enforce a single call by checking a flag for instance, but make it undefined behaviour.
This would usually be constructing the object multiple times without properly destroying them.

inkwizyt...@gmail.com

unread,
May 30, 2018, 4:13:21 PM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
I use wrong words, when you call `p()` and then you again call `p()` then code will be call only once. This will have cost but only one `if`, value that store this flag is already need in current C++.
If function get inlined this cost could be removed because in some cases we could infer from context what temporaries live. This is probably small cost compare to UB.

I think that for `[]->int a` we should have `&a() == &a()` this will be same temporary variable that is allocated on caller stack.

Only problem I see is order in with temporaries will be destroyed, probably it will be fixed one not depending on order in with params was call. Adding info about order will only increase implementation complexity and cost of each call of this parameter. This part I would leave UB.

floria...@gmail.com

unread,
May 30, 2018, 4:59:26 PM5/30/18
to ISO C++ Standard - Future Proposals
I understood what you said, and I keep thinking that's not a good idea.
std::array<int, 1000>* create_big([] -> std::array<int, 1000> a) {
 
return new std::array<int, 1000>(a()); //copy ellision: a() is a prvalue
}

auto *p = create_big({1, 2, ..., 1000});
Lazy parameter here allows to defer the creation of the array until the point where its final storage is known, and "RVO" can be applied to ellide the copy as the lazy expression "returns" a prvalue.

But now, if you do:
std::array<int, 1000>* create_big([] -> std::array<int, 1000> a) {
  std
::array<int, 1000> b(a()); // copy ellision
 
return new std::array<int, 1000>(b); // no copy ellision here, b is not a prvalue
}

auto *p = create_big({1, 2, ..., 1000});
The copy is not ellided as b is not a prvalue nor can it be converted into one.
What you propose is forcing the second scenario, even when the code is actually written like the first.
With this, the lazy "expression" cannot return a prvalue.

And I also don't want the compiler to choose between the two scenarii depending on the number of uses of the lazy expression.
In fact, it will not be possible in general to know statically if the expression will be used several times.

If you say undefined behaviour if used more than once, you can guarantee that the lazy expression "returns" a prvalue and copy ellision will be applied in the first case in every single situation, while still allowing people to store it in a local variable and reuse this local variable instead.
As a consequence, you will not be able to use &a() if the parameter is [] -> int a. But you will be able to do that if it is defined as [] -> const int& a.

Also, if all temporaries of the lazy expression are trivial (trivially destructible is enough), you could even save the flag telling it has already been executed, and skip proper destruction of the temporaries. This will be a big gain if the construction of the returned object can throw, for instance.
This point is possible only if you don't define multiple evaluations of the lazy expression.


I think that for `[]->int a` we should have `&a() == &a()` this will be same temporary variable that is allocated on caller stack.

if the lazy exporession "returns" a prvalue, you will not be able to call &a() at all.
 

Only problem I see is order in with temporaries will be destroyed, probably it will be fixed one not depending on order in with params was call. Adding info about order will only increase implementation complexity and cost of each call of this parameter. This part I would leave UB.


Actually, this part is not a problem and can be fully defined: if the lazy expression throws the constructed temporaries will be destroyed in reverse order, and then exception is forwarded to the callee.
If there are other lazy parameters, either they have not been executed already and no object needs to be destroyed, or they have been successfully executed and their temporaries will be destroyed by the caller (not the calle) in reverse order (that is known by the caller) per lazy parameter.

The destruction order for different lazy parameter could be unspecified. I don't see any reason we should try to force an order between completely unrelated objects.

inkwizyt...@gmail.com

unread,
May 30, 2018, 6:20:07 PM5/30/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
Good point. Then to have best performance we will need have UB. With this for simplicity my "retry" feature should be scraped too because it to close to UB situation even if in itself could be easy implemented.
Rule will be simple: parameter can be only called onece or less.
If UB is not desired then exception could be thrown as alternative solution for repeated calls.

Edward Catmur

unread,
May 30, 2018, 6:37:13 PM5/30/18
to std-pr...@isocpp.org
And with the coroutines TS we will have far more complicated overlapping scopes allocating stack space for themselves. At least here the stack requirement of the innermost scope is known allowing it to use fixed space allocated by the outermost scope. 

You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

Bengt Gustafsson

unread,
Jun 1, 2018, 6:52:34 PM6/1/18
to ISO C++ Standard - Future Proposals
Considering forwarding of lazy parameters to functions that may or may not use them it becomes very hard to use them if they can only be evaluated once, or the code must make copies just in case, which defeats the purpose of the feature. I don't think that avoiding a copy during initialization is a motivating purpose which should be allowed to rule the definition, instead we should focus on being able to mimic the shortcut operators and operator? and the classical logging scenario. Here is a nasty example:

bool andand([]->bool lhs, []->bool rhs)
{
   
Log("Performing andand on", lhs, " and ", rhs);   // Variadic log with lazy parameters only evaluated if logging is enabled.
   
return lhs() && rhs();    // This is UB if logging is enabled!
}



Apart from showing the problems with evaluate once principle this example rasies a couple of syntactical issues:

1) How do we declare a variadic template with lazy parameters?
2) Is is logical to forward-by-lazy by not using postfix(). If so, how many of us will add the () by mistake when calling Log, accidentally (but silently) evaluating the lazy parameter and forwarding the resulting value as a (very simple) lazy expression to Log?
3) Could it be considered to use postfix [] (empty bracket) to "derefer" a lazy parameter to make it stand out more from function application?

Personally I still think that you should not have to annotate the use of a lazy parameter in the called function, just as the value/reference distinction this is something that the function author will have close to the top of her head when writing the code. This would nullify the 2) issue above. Maybe it could be complemented by a template function std::eager() which makes sure evaluation is done even if forwarding to a function that may take a lazy parameter. I don't have an example where this would be important though, it seems a bit backwards.

I think the best would be to view this as a new kind of reference, possibly using the | token to signify it. That is, it works like a reference, except that before the first use it is evaluated (using the flag to check this). As for other references there is no indicating at the usage site, as other (lvalue) references it is transparently forwarded to subordinate functions (for example Log in my example). The issue of how to write a template function has an obvious answer, and, I daresay, there should be no syntax issues as we have the example of using the & character which as an operator is parallel to | to indicate a type of reference already.

// Variadic Log with lazy parameters only evaluated if logging is enabled.
template<typename T, typename... Ts> Log(T| head, Ts| tail)
{
   
if (log_enabled) {
        cout
<< head;
       
Log(tail...);
   
}
}
bool andand(bool| lhs, bool| rhs)
{
   
Log("Performing andand on", lhs, " and ", rhs);
   
return lhs && rhs;
}


Looking at the big-array-copy scenario, I understand this as a case where a non-inlined function gets called and you want to redirect the constructor call of the lazy thunk (creating the by value "return" value) to do its construction directly in the receiving variable, presuming that, as the variable type is "big" the ABI will have the caller of the lazy thunk provide an address to construct the value in. The lazy thunk can use different constructors at different call sites which is nice in combination with not having to see the callee source code. This has significant performance advantage in the case that the value is large and does not have a lightweight move possibility. Apart from std::array all standard containers have cheap moves  so the usefulness is further reduced compared to a regular by value parameter which is moved into the destination except for objects with large C or std arrays. Again: Is this functionality really so important that it warrants hijacking the original feature at the cost of making it less useful for its original purpose?



inkwizyt...@gmail.com

unread,
Jun 1, 2018, 7:47:47 PM6/1/18
to ISO C++ Standard - Future Proposals
Florian show examples where allowing multiple usage of lazy parameters cause suboptimal code.
Only way I see to avoid performance degradation and UB is ban multiple usage of parameter in "dumb" code paths:
void foo(bool t, []->int a, []->int b, []->int c)
{
   
if (t)
   
{
      a
();
   
}
   
else
   
{
      a
(); //ok
   
}
   
if (t)
   
{
      b
();
   
}
   
if (!t) //compiler is dumb and do not try figure out that this path can't be taken
   
{
      b
(); //error!
   
}
   
do
   
{
      c
(); //error again, even if this can be call only once.
   
}
   
while(false);
}


Something like this is done by C# compiler when checking is variable unassigned or `out` parameters. Because this is new feature that never existed in language before we can be lot of more restrictive than normal.
This need be done on "dumb" level because is impossible to check it in general case.

Another thing is that lazy parameters should not leak from function body if you want pass them to another function you need evaluate them first.

With this two requirements we could even drop `()` from syntax because lazy parameter symbol can only appear once in function body (with small exception for completely separate code paths).

As bonus, with `co_await` and `co_wait` we will have another limitation that you can use parameters before first `co_` operation.

Nicol Bolas

unread,
Jun 2, 2018, 12:21:10 AM6/2/18
to ISO C++ Standard - Future Proposals
On Friday, June 1, 2018 at 6:52:34 PM UTC-4, Bengt Gustafsson wrote:
Considering forwarding of lazy parameters to functions that may or may not use them it becomes very hard to use them if they can only be evaluated once, or the code must make copies just in case, which defeats the purpose of the feature.

I disagree. We encounter this scenario in other forms. A function which takes a parameter by rvalue reference may move from it... or it may not. Can you use the object afterwards? And if so, how? The same goes for forwarding parameters with `std::forward`. You're not supposed to use them later; you don't know what state they're in or if they've been moved from or not.

This is just one more case of where you need to be careful.

Edward Catmur

unread,
Jun 2, 2018, 10:06:49 AM6/2/18
to std-pr...@isocpp.org


On Fri, 1 Jun 2018, 23:52 Bengt Gustafsson, <bengt.gu...@beamways.com> wrote:
Considering forwarding of lazy parameters to functions that may or may not use them it becomes very hard to use them if they can only be evaluated once, or the code must make copies just in case, which defeats the purpose of the feature. I don't think that avoiding a copy during initialization is a motivating purpose which should be allowed to rule the definition, instead we should focus on being able to mimic the shortcut operators and operator? and the classical logging scenario. Here is a nasty example:

bool andand([]->bool lhs, []->bool rhs)
{
   
Log("Performing andand on", lhs, " and ", rhs);   // Variadic log with lazy parameters only evaluated if logging is enabled.
   
return lhs() && rhs();    // This is UB if logging is enabled!
}



Apart from showing the problems with evaluate once principle this example rasies a couple of syntactical issues:

1) How do we declare a variadic template with lazy parameters?
2) Is is logical to forward-by-lazy by not using postfix(). If so, how many of us will add the () by mistake when calling Log, accidentally (but silently) evaluating the lazy parameter and forwarding the resulting value as a (very simple) lazy expression to Log?

No, adding the parentheses continues to defer evaluation. The syntactic function call expression is a subexpression of the initializer passed to Log, so its evaluation is delayed. Inventing a special forwarding operation is unnecessary (although the compiler might need to identify such to generate optimal code). 

I'm not sure that the identifier without postfix parentheses has any semantics. 

3) Could it be considered to use postfix [] (empty bracket) to "derefer" a lazy parameter to make it stand out more from function application?

Personally I still think that you should not have to annotate the use of a lazy parameter in the called function, just as the value/reference distinction this is something that the function author will have close to the top of her head when writing the code. This would nullify the 2) issue above. Maybe it could be complemented by a template function std::eager() which makes sure evaluation is done even if forwarding to a function that may take a lazy parameter. I don't have an example where this would be important though, it seems a bit backwards.

That wouldn't accomplish anything; the call to std::eager would itself be delayed. 


I think the best would be to view this as a new kind of reference, possibly using the | token to signify it. That is, it works like a reference, except that before the first use it is evaluated (using the flag to check this). As for other references there is no indicating at the usage site, as other (lvalue) references it is transparently forwarded to subordinate functions (for example Log in my example). The issue of how to write a template function has an obvious answer, and, I daresay, there should be no syntax issues as we have the example of using the & character which as an operator is parallel to | to indicate a type of reference already.

// Variadic Log with lazy parameters only evaluated if logging is enabled.
template<typename T, typename... Ts> Log(T| head, Ts| tail)
{
   
if (log_enabled) {
        cout
<< head;
       
Log(tail...);
   
}
}
bool andand(bool| lhs, bool| rhs)
{
   
Log("Performing andand on", lhs, " and ", rhs);
   
return lhs && rhs;
}


Looking at the big-array-copy scenario, I understand this as a case where a non-inlined function gets called and you want to redirect the constructor call of the lazy thunk (creating the by value "return" value) to do its construction directly in the receiving variable, presuming that, as the variable type is "big" the ABI will have the caller of the lazy thunk provide an address to construct the value in. The lazy thunk can use different constructors at different call sites which is nice in combination with not having to see the callee source code. This has significant performance advantage in the case that the value is large and does not have a lightweight move possibility. Apart from std::array all standard containers have cheap moves  so the usefulness is further reduced compared to a regular by value parameter which is moved into the destination except for objects with large C or std arrays. Again: Is this functionality really so important that it warrants hijacking the original feature at the cost of making it less useful for its original purpose?

It's not just types with expensive move; there are also types with deleted move and copy constructors. Given that guaranteed RVO enabled use of these types recently, it would be a shame if they were not to work with this new feature. 




--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

floria...@gmail.com

unread,
Jun 2, 2018, 12:20:31 PM6/2/18
to ISO C++ Standard - Future Proposals
I know other people already have answered your questions, but I would like to give my point of view.


Le samedi 2 juin 2018 00:52:34 UTC+2, Bengt Gustafsson a écrit :
Considering forwarding of lazy parameters to functions that may or may not use them it becomes very hard to use them if they can only be evaluated once, or the code must make copies just in case, which defeats the purpose of the feature. I don't think that avoiding a copy during initialization is a motivating purpose which should be allowed to rule the definition, instead we should focus on being able to mimic the shortcut operators and operator? and the classical logging scenario. Here is a nasty example:

bool andand([]->bool lhs, []->bool rhs)
{
   
Log("Performing andand on", lhs, " and ", rhs);   // Variadic log with lazy parameters only evaluated if logging is enabled.
   
return lhs() && rhs();    // This is UB if logging is enabled!
}



If you really want something like that you could imagine doing the following:
bool andand([] -> bool lhs_lazy, [] -> bool rhs_lazy) {
  std
::optional<bool> lhs_v, rhs_v;
 
auto lhs = [&] () -> bool { if (!lhs_v) { lhs_v = lhs_lazy(); } return lhs_v.value(); };
  auto rhs = [&] () -> bool { if (!rhs_v) { rhs_v = rhs_lazy(); } return rhs_v.value(); };
 
  Log("Performing andand on", lhs(), " and ", rhs());
  return lhs() && rhs();
}

This would allow the kind of behavior you want (parameters will be evaluated only if they are needed), while keeping lazy arguments returning pr-values.
We would need to standardize lambda captures relating to lazy parameters, but it would be worth it.

However, if you allow multiple evaluation of the expression: either you will not be able to return a pr-value (with caching), or you will have problems with overlapping lifetime of the same objects...
So you can implement what you want with what I propose, but I cannot implement what I want with what you propose.
 

Apart from showing the problems with evaluate once principle this example rasies a couple of syntactical issues:

1) How do we declare a variadic template with lazy parameters?

Is there any problem with this?
template <class... Args>
void Log([] -> Args... args);

 
2) Is is logical to forward-by-lazy by not using postfix(). If so, how many of us will add the () by mistake when calling Log, accidentally (but silently) evaluating the lazy parameter and forwarding the resulting value as a (very simple) lazy expression to Log?
Lazy parameter forwarding is just an optimization a compiler would be able to do. But there is no reason it should be in the language: if Log takes lazy parameters, when you "evaluate" a lazy parameter to forward to Log, you will create a new lazy expression that will evaluate the first lazy expression when it will be evaluated.
With or without a proper lazy parameter forwarding, the behaviour of the program will be exactly the same.

 
3) Could it be considered to use postfix [] (empty bracket) to "derefer" a lazy parameter to make it stand out more from function application?

I have no opinion on what syntax should be used to "evaluate" a lazy expression. For now the majority seems to prefer postfix ().
 

Personally I still think that you should not have to annotate the use of a lazy parameter in the called function, just as the value/reference distinction this is something that the function author will have close to the top of her head when writing the code. This would nullify the 2) issue above. Maybe it could be complemented by a template function std::eager() which makes sure evaluation is done even if forwarding to a function that may take a lazy parameter. I don't have an example where this would be important though, it seems a bit backwards.

If you want an expression to be executed even when passed as a lazy parameter, just create a local variable constructed from your expression, and pass this variable as the lazy parameter.
bool foo() {
  bool lhs = bar(0); // this expression evaluation is forced here
  return lhs && bar(1); // the evaluation of bar(1) might not happen
}


 

I think the best would be to view this as a new kind of reference, possibly using the | token to signify it. That is, it works like a reference, except that before the first use it is evaluated (using the flag to check this). As for other references there is no indicating at the usage site, as other (lvalue) references it is transparently forwarded to subordinate functions (for example Log in my example). The issue of how to write a template function has an obvious answer, and, I daresay, there should be no syntax issues as we have the example of using the & character which as an operator is parallel to | to indicate a type of reference already.

// Variadic Log with lazy parameters only evaluated if logging is enabled.
template<typename T, typename... Ts> Log(T| head, Ts| tail)
{
   
if (log_enabled) {
        cout
<< head;
       
Log(tail...);
   
}
}
bool andand(bool| lhs, bool| rhs)
{
   
Log("Performing andand on", lhs, " and ", rhs);
   
return lhs && rhs;
}



I fail to see how this new syntax is better. The proposed syntax [] -> has a comprehensive meaning once you have seen a lambda. But this is a weak argument, I have to admit.
 
Looking at the big-array-copy scenario, I understand this as a case where a non-inlined function gets called and you want to redirect the constructor call of the lazy thunk (creating the by value "return" value) to do its construction directly in the receiving variable, presuming that, as the variable type is "big" the ABI will have the caller of the lazy thunk provide an address to construct the value in. The lazy thunk can use different constructors at different call sites which is nice in combination with not having to see the callee source code. This has significant performance advantage in the case that the value is large and does not have a lightweight move possibility. Apart from std::array all standard containers have cheap moves  so the usefulness is further reduced compared to a regular by value parameter which is moved into the destination except for objects with large C or std arrays. Again: Is this functionality really so important that it warrants hijacking the original feature at the cost of making it less useful for its original purpose?


As Edward said, this is not only about big objects with inefficient moves, but also types that are neither copyable nor moveable: for instance std::atomic or std::mutex.
Also, I have the feeling that lazy parameters would be an efficient way to create complex objects without relying on emplace fonctions and std::piecewise_construct.
I asked before if my feeling on this subject was shared, but the discussion drifted away.

Edward Catmur

unread,
Jun 3, 2018, 3:55:18 AM6/3/18
to std-pr...@isocpp.org


On Sat, 2 Jun 2018, 17:20 , <floria...@gmail.com> wrote:
I know other people already have answered your questions, but I would like to give my point of view.

Le samedi 2 juin 2018 00:52:34 UTC+2, Bengt Gustafsson a écrit :
Considering forwarding of lazy parameters to functions that may or may not use them it becomes very hard to use them if they can only be evaluated once, or the code must make copies just in case, which defeats the purpose of the feature. I don't think that avoiding a copy during initialization is a motivating purpose which should be allowed to rule the definition, instead we should focus on being able to mimic the shortcut operators and operator? and the classical logging scenario. Here is a nasty example:

bool andand([]->bool lhs, []->bool rhs)
{
   
Log("Performing andand on", lhs, " and ", rhs);   // Variadic log with lazy parameters only evaluated if logging is enabled.
   
return lhs() && rhs();    // This is UB if logging is enabled!
}



If you really want something like that you could imagine doing the following:
bool andand([] -> bool lhs_lazy, [] -> bool rhs_lazy) {
  std
::optional<bool> lhs_v, rhs_v;
 
auto lhs = [&] () -> bool { if (!lhs_v) { lhs_v = lhs_lazy(); } return lhs_v.value(); };
  auto rhs = [&] () -> bool { if (!rhs_v) { rhs_v = rhs_lazy(); } return rhs_v.value(); };
 
  Log("Performing andand on", lhs(), " and ", rhs());
  return lhs() && rhs();
}

This would allow the kind of behavior you want (parameters will be evaluated only if they are needed), while keeping lazy arguments returning pr-values.
We would need to standardize lambda captures relating to lazy parameters, but it would be worth it.

However, if you allow multiple evaluation of the expression: either you will not be able to return a pr-value (with caching), or you will have problems with overlapping lifetime of the same objects...
So you can implement what you want with what I propose, but I cannot implement what I want with what you propose.
 

It should be possible to implement multiple evaluation without caching by storing lifetime extended temporaries in a conventionally located stack frame and then not restoring (or only partially restoring) the stack pointer on return from evaluation. This should be technically possible on any platform where alloca is implementable. 

The stack would look like: (growing downwards) 

caller
callee
lazy1a temporaries
lazy2 temporaries
lazy1b temporaries
inner function

Here lazy1 is evaluated twice within the same expression, giving two sets of temporaries. The temporaries would be destructed and their stack space reclaimed at the end of the full-expression in the callee. Note that the stack space used is bounded by the size of the expression in the callee. 

This would have the advantage that scope, lifetime and stack location would be more conventionally correlated, and may be technically easier to implement than storing temporaries within the stack frame of the caller. 


--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

Magnus Fromreide

unread,
Jun 3, 2018, 7:16:18 AM6/3/18
to 'Edward Catmur' via ISO C++ Standard - Future Proposals
Hello

I am more than a little concerned about all this talk of multiple evaluation
of lazy arguments.

I presume that everone agrees that given

void f([]->int);

then after

int a = 10;
f(a++);

the value of a is either 10 or 11, no other values are possible.
(BTW, What a great obfuscation tool this would be even with this limitation)

This in turn means that any second evaluation of the lazy expression must
be forbidden unless the compiler can prove it to be side effect free.
> > program will be *exactly* the same.
> > <https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/cc20625c-804f-4ad2-8c38-ce443e7314ec%40isocpp.org?utm_medium=email&utm_source=footer>
> > .
> >
>
> --
> You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAJnLdObwz9LJQGtquyGFsGSQawuLWH9kbAQQZjS9LKrTVxYiNQ%40mail.gmail.com.

floria...@gmail.com

unread,
Jun 3, 2018, 8:20:57 AM6/3/18
to ISO C++ Standard - Future Proposals

Le dimanche 3 juin 2018 13:16:18 UTC+2, Magnus Fromreide a écrit :
Hello

I am more than a little concerned about all this talk of multiple evaluation
of lazy arguments.

I presume that everone agrees that given

void f([]->int);

then after

int a = 10;
f(a++);

the value of a is either 10 or 11, no other values are possible.
(BTW, What a great obfuscation tool this would be even with this limitation)

This example is really great to show mutliple evaluations should not be allowed, period.
BTW, the example I showed with manually caching the result in a lambda gives this behavior: the lazy expression is evaluated once.
 

This in turn means that any second evaluation of the lazy expression must
be forbidden unless the compiler can prove it to be side effect free.

Unless the compiler inline both the caller and the lazy expression, it will not be able to prove this side-effect free property.
And if both are inlined, all technical issues disappear.

@Edward
It should be possible to implement multiple evaluation without caching by storing lifetime extended temporaries in a conventionally located stack frame and then not restoring (or only partially restoring) the stack pointer on return from evaluation. This should be technically possible on any platform where alloca is implementable. 

The stack would look like: (growing downwards) 

caller
callee
lazy1a temporaries
lazy2 temporaries
lazy1b temporaries
inner function

Here lazy1 is evaluated twice within the same expression, giving two sets of temporaries. The temporaries would be destructed and their stack space reclaimed at the end of the full-expression in the callee. Note that the stack space used is bounded by the size of the expression in the callee. 

This would have the advantage that scope, lifetime and stack location would be more conventionally correlated, and may be technically easier to implement than storing temporaries within the stack frame of the caller.

Unfortunatley, if the lazy expression "calls" alloca, the stack memory will be deallocated when the lazy expression ends its scope. That's why allocating this memory in the caller stack frame is much easier.
Also, how would you track down all destructors that would be needed to be called?
If you want multiple scopes, you will need heap memory allocation at some point, which is really not needed to have a very useful feature.
You can always create more complex objects from the simple lazy expression that cannot be evaluated multiple times to achieve your goal.

Also, what if the lazy expression calls new: how do you track down the multiple allocations?
If the lazy expression is very expensive, do you really want do execute the all thing again?
Caching is the only sensible way to allow multiple evaluations, but this forbids pr-values, with all the issues mentionned before.

And as I showed earlier, if a caller needs to cache the result, it can do it. Whereas other proposals do not/cannot solve altogether the pr-value issue, the lifetime issues and the performance issue.
Of course it needs a bit of boilerplate code to actually do the caching, but this use case should be rare enough. And the caching code is really understandable and easy to come up with, especially if you know the lazy expression will always be evaluated.

One solution might be to have both: [] -> lazy parameter without caching  and [=] -> lazy parameter with caching (with no consideration on how it is implemented).

Anyway, if you need multiple evaluations, why do you pass a lazy expression and not a callable? That would make your intent much clearer.

Edward Catmur

unread,
Jun 4, 2018, 1:31:03 AM6/4/18
to std-pr...@isocpp.org


On Sun, 3 Jun 2018, 13:21 , <floria...@gmail.com> wrote:

Le dimanche 3 juin 2018 13:16:18 UTC+2, Magnus Fromreide a écrit :
Hello

I am more than a little concerned about all this talk of multiple evaluation
of lazy arguments.

I presume that everone agrees that given

void f([]->int);

then after

int a = 10;
f(a++);

the value of a is either 10 or 11, no other values are possible.
(BTW, What a great obfuscation tool this would be even with this limitation)

This example is really great to show mutliple evaluations should not be allowed, period.
BTW, the example I showed with manually caching the result in a lambda gives this behavior: the lazy expression is evaluated once.
 


That depends, though. We're perfectly happy to write `while (a++) ...;` and it's not unreasonable to think that this facility might be used for functions with control flow that requires multiple invocation. 



This in turn means that any second evaluation of the lazy expression must
be forbidden unless the compiler can prove it to be side effect free.

Unless the compiler inline both the caller and the lazy expression, it will not be able to prove this side-effect free property.
And if both are inlined, all technical issues disappear.

@Edward
It should be possible to implement multiple evaluation without caching by storing lifetime extended temporaries in a conventionally located stack frame and then not restoring (or only partially restoring) the stack pointer on return from evaluation. This should be technically possible on any platform where alloca is implementable. 

The stack would look like: (growing downwards) 

caller
callee
lazy1a temporaries
lazy2 temporaries
lazy1b temporaries
inner function

Here lazy1 is evaluated twice within the same expression, giving two sets of temporaries. The temporaries would be destructed and their stack space reclaimed at the end of the full-expression in the callee. Note that the stack space used is bounded by the size of the expression in the callee. 

This would have the advantage that scope, lifetime and stack location would be more conventionally correlated, and may be technically easier to implement than storing temporaries within the stack frame of the caller.

Unfortunatley, if the lazy expression "calls" alloca, the stack memory will be deallocated when the lazy expression ends its scope. That's why allocating this memory in the caller stack frame is much easier.

Sorry, I didn't mean to imply that the lazy expression would call alloca. Rather it would behave like alloca, giving the stack pointer a different value on exit than on entry. 

Also, how would you track down all destructors that would be needed to be called?

That would be the job of the lazy expression routine; the callee would invoke the lazy expression routines in destruct mode as many times as they were invoked in evaluate mode and in reverse. 

Recall that you need a destruct mode to be invoked by the callee to ensure that destructors of temporaries are called at the end of the correct statement in the callee. 

If you want multiple scopes, you will need heap memory allocation at some point, which is really not needed to have a very useful feature.

I don't see why? 

You can always create more complex objects from the simple lazy expression that cannot be evaluated multiple times to achieve your goal.

Yes, but then you lose prvalue-ness, as you have pointed out. 


Also, what if the lazy expression calls new: how do you track down the multiple allocations?

Assuming the new-expressions are adopted by a smart pointer, normal RAII would apply. 

If the lazy expression is very expensive, do you really want do execute the all thing again?

Probably not, but that should generally be the user's choice. 

Caching is the only sensible way to allow multiple evaluations, but this forbids pr-values, with all the issues mentionned before.

And as I showed earlier, if a caller needs to cache the result, it can do it. Whereas other proposals do not/cannot solve altogether the pr-value issue, the lifetime issues and the performance issue.
Of course it needs a bit of boilerplate code to actually do the caching, but this use case should be rare enough. And the caching code is really understandable and easy to come up with, especially if you know the lazy expression will always be evaluated.

One solution might be to have both: [] -> lazy parameter without caching  and [=] -> lazy parameter with caching (with no consideration on how it is implemented).

Anyway, if you need multiple evaluations, why do you pass a lazy expression and not a callable? That would make your intent much clearer.

In general, I agree with all of the above. I am concerned that there may be use cases (e.g. current macros) that would benefit from multiple evaluation (non-cached) being defined, and I would not want multiple evaluation to be excluded that I believe I have shown can be avoided. 

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

Edward Catmur

unread,
Jun 4, 2018, 4:13:42 AM6/4/18
to std-pr...@isocpp.org

On Mon, Jun 4, 2018 at 5:53 AM, Edward Catmur <eca...@googlemail.com> wrote:

In general, I agree with all of the above. I am concerned that there may be use cases (e.g. current macros) that would benefit from multiple evaluation (non-cached) being defined, and I would not want multiple evaluation to be excluded that I believe I have shown can be avoided. 

Sorry, missed a few words there: I would not want multiple evaluation to be excluded *on the basis of technical considerations* that I believe I have shown can be avoided. 

floria...@gmail.com

unread,
Jun 4, 2018, 4:24:17 AM6/4/18
to ISO C++ Standard - Future Proposals


Le lundi 4 juin 2018 07:31:03 UTC+2, Edward Catmur a écrit :


On Sun, 3 Jun 2018, 13:21 , <floria...@gmail.com> wrote:

Le dimanche 3 juin 2018 13:16:18 UTC+2, Magnus Fromreide a écrit :
Hello

I am more than a little concerned about all this talk of multiple evaluation
of lazy arguments.

I presume that everone agrees that given

void f([]->int);

then after

int a = 10;
f(a++);

the value of a is either 10 or 11, no other values are possible.
(BTW, What a great obfuscation tool this would be even with this limitation)

This example is really great to show mutliple evaluations should not be allowed, period.
BTW, the example I showed with manually caching the result in a lambda gives this behavior: the lazy expression is evaluated once.
 


That depends, though. We're perfectly happy to write `while (a++) ...;` and it's not unreasonable to think that this facility might be used for functions with control flow that requires multiple invocation. 

For that kind of stuff, I would prefer a "macro-like" approach. I know some people were working on that. It's personal taste here.
 



This in turn means that any second evaluation of the lazy expression must
be forbidden unless the compiler can prove it to be side effect free.

Unless the compiler inline both the caller and the lazy expression, it will not be able to prove this side-effect free property.
And if both are inlined, all technical issues disappear.

@Edward
It should be possible to implement multiple evaluation without caching by storing lifetime extended temporaries in a conventionally located stack frame and then not restoring (or only partially restoring) the stack pointer on return from evaluation. This should be technically possible on any platform where alloca is implementable. 

The stack would look like: (growing downwards) 

caller
callee
lazy1a temporaries
lazy2 temporaries
lazy1b temporaries
inner function

Here lazy1 is evaluated twice within the same expression, giving two sets of temporaries. The temporaries would be destructed and their stack space reclaimed at the end of the full-expression in the callee. Note that the stack space used is bounded by the size of the expression in the callee. 

This would have the advantage that scope, lifetime and stack location would be more conventionally correlated, and may be technically easier to implement than storing temporaries within the stack frame of the caller.

Unfortunatley, if the lazy expression "calls" alloca, the stack memory will be deallocated when the lazy expression ends its scope. That's why allocating this memory in the caller stack frame is much easier.

Sorry, I didn't mean to imply that the lazy expression would call alloca. Rather it would behave like alloca, giving the stack pointer a different value on exit than on entry. 

(alloca is not a function anyway, it is just some arithmetic on pointers)
The problem is: if you give a different stack pointer to the callee, it would need to adjust back the pointer itself to access its data. And what if the callee needs to allocate some stack after evaluating a lazy expression? Where would the new stack frame go? How does the caller know where it would go?

After some thinking, I agree it might be possible to implement, but at what costs?
 

Also, how would you track down all destructors that would be needed to be called?

That would be the job of the lazy expression routine; the callee would invoke the lazy expression routines in destruct mode as many times as they were invoked in evaluate mode and in reverse. 

Recall that you need a destruct mode to be invoked by the callee to ensure that destructors of temporaries are called at the end of the correct statement in the callee. 

I don't recall that. It was suggested, but I think the lazy expression temporaries must outlive the caller expression (not only the callee one).
void print(std::string_view sv) { std::cout << sv << std::endl; }

std
::string_view lazy([] -> std::string s) { return s; }

std
::string_view eager(const std::string & s) { return s; }

print(eager("hello world!"));

print(lazy("hello world!"));
The eager expression works perfectly fine now: the std::string temporary lifetime ends at the semicolon (https://godbolt.org/g/6xV4aM).
I would expect the second to also works for the very same reason.

This can happen only if the caller is the one to destroy the lazy expression temporaries.
 

If you want multiple scopes, you will need heap memory allocation at some point, which is really not needed to have a very useful feature.

I don't see why? 

Maybe not, but very hard. Especially to make the destruction happens in the caller as I explained just above.
 

You can always create more complex objects from the simple lazy expression that cannot be evaluated multiple times to achieve your goal.

Yes, but then you lose prvalue-ness, as you have pointed out. 

I don't see any valid use case where these 2 features might be needed at the same time. If you really want multiple evaluations also allowing prvalues, why callables are not enough?
 


Also, what if the lazy expression calls new: how do you track down the multiple allocations?

Assuming the new-expressions are adopted by a smart pointer, normal RAII would apply. 

You don't need RAII for this code to be correct:
int i = /* ... */;
int* p = i > 0 ? new int(i) : nullptr;
/* ... */
if (p) delete p;
Why would it be different with custom functions, or custom types?

I agree smart pointers are much safer and should be used everywhere a new/delete was used. But this is still valid code.
 

If the lazy expression is very expensive, do you really want do execute the all thing again?

Probably not, but that should generally be the user's choice. 

A user might not be able to tell if an expression is expensive or not. It will be even more true with this because it will be simpler to call complex constructors without even noticing (and that's part of the point).
 

Caching is the only sensible way to allow multiple evaluations, but this forbids pr-values, with all the issues mentionned before.

And as I showed earlier, if a caller needs to cache the result, it can do it. Whereas other proposals do not/cannot solve altogether the pr-value issue, the lifetime issues and the performance issue.
Of course it needs a bit of boilerplate code to actually do the caching, but this use case should be rare enough. And the caching code is really understandable and easy to come up with, especially if you know the lazy expression will always be evaluated.

One solution might be to have both: [] -> lazy parameter without caching  and [=] -> lazy parameter with caching (with no consideration on how it is implemented).

Anyway, if you need multiple evaluations, why do you pass a lazy expression and not a callable? That would make your intent much clearer.

In general, I agree with all of the above. I am concerned that there may be use cases (e.g. current macros) that would benefit from multiple evaluation (non-cached) being defined, and I would not want multiple evaluation to be excluded that I believe I have shown can be avoided. 

For now, I don't see any use case that would really need this syntax boilerplate free at call site feature.
I really believe that this kind of problems should be solved either with callables or "macro"-like constructions.

Also multiple evaluation could also be added afterwards if there is a need for it. As long as we say multiple evaluation is undefined, then extending it would be possible.


Sorry, missed a few words there: I would not want multiple evaluation to be excluded *on the basis of technical considerations* that I believe I have shown can be avoided.

More than technical considerations, it is performance considerations. You want the fast cases being fast: if the lazy expression is just a literal, you really don't want all the implementation boilerplate to deal with multiple evaluations and temporaries bookkeeping.
Your solution makes this use case inefficient because the callee won't know the lazy expression is as simple as that.
In the implementation I proposed, this is as fast as it could get without inlining: just a call to a function returning the literal.

Matthew Woehlke

unread,
Jun 4, 2018, 12:26:42 PM6/4/18
to std-pr...@isocpp.org
On 2018-06-04 00:53, Edward Catmur wrote:
> On Sun, 3 Jun 2018, 13:21 , <floria...@gmail.com> wrote:
>> Le dimanche 3 juin 2018 13:16:18 UTC+2, Magnus Fromreide a écrit :
>>> I presume that everone agrees that given
>>>
>>> void f([]->int);
>>>
>>> then after
>>>
>>> int a = 10;
>>> f(a++);
>>>
>>> the value of a is either 10 or 11, no other values are possible.
>>> (BTW, What a great obfuscation tool this would be even with this
>>> limitation)

As far as obfuscation, this is no worse than macros.

>> This example is really great to show mutliple evaluations should not be
>> allowed, period.
>
> That depends, though. We're perfectly happy to write `while (a++) ...;` and
> it's not unreasonable to think that this facility might be used for
> functions with control flow that requires multiple invocation.

...but control-flow keywords are not much like functions. IMHO, if you
really need multiple evaluation, use a full lambda. The main problems I
see delayed evaluation trying to solve do not need multiple evaluation,
and permitting it adds an additional "gotcha".

--
Matthew

Edward Catmur

unread,
Jun 5, 2018, 8:47:31 PM6/5/18
to std-pr...@isocpp.org


On Mon, 4 Jun 2018, 09:24 , <floria...@gmail.com> wrote:


Le lundi 4 juin 2018 07:31:03 UTC+2, Edward Catmur a écrit :


On Sun, 3 Jun 2018, 13:21 , <floria...@gmail.com> wrote:

Le dimanche 3 juin 2018 13:16:18 UTC+2, Magnus Fromreide a écrit :
Hello

I am more than a little concerned about all this talk of multiple evaluation
of lazy arguments.

I presume that everone agrees that given

void f([]->int);

then after

int a = 10;
f(a++);

the value of a is either 10 or 11, no other values are possible.
(BTW, What a great obfuscation tool this would be even with this limitation)

This example is really great to show mutliple evaluations should not be allowed, period.
BTW, the example I showed with manually caching the result in a lambda gives this behavior: the lazy expression is evaluated once.
 


That depends, though. We're perfectly happy to write `while (a++) ...;` and it's not unreasonable to think that this facility might be used for functions with control flow that requires multiple invocation. 

For that kind of stuff, I would prefer a "macro-like" approach. I know some people were working on that. It's personal taste here.
 

If this could be (part of) that macro-like approach, I'm sure conventions would arise to distinguish between functions taking lazy arguments that are evaluated at most once and those whose lazy arguments may be evaluated multiple times. 




This in turn means that any second evaluation of the lazy expression must
be forbidden unless the compiler can prove it to be side effect free.

Unless the compiler inline both the caller and the lazy expression, it will not be able to prove this side-effect free property.
And if both are inlined, all technical issues disappear.

@Edward
It should be possible to implement multiple evaluation without caching by storing lifetime extended temporaries in a conventionally located stack frame and then not restoring (or only partially restoring) the stack pointer on return from evaluation. This should be technically possible on any platform where alloca is implementable. 

The stack would look like: (growing downwards) 

caller
callee
lazy1a temporaries
lazy2 temporaries
lazy1b temporaries
inner function

Here lazy1 is evaluated twice within the same expression, giving two sets of temporaries. The temporaries would be destructed and their stack space reclaimed at the end of the full-expression in the callee. Note that the stack space used is bounded by the size of the expression in the callee. 

This would have the advantage that scope, lifetime and stack location would be more conventionally correlated, and may be technically easier to implement than storing temporaries within the stack frame of the caller.

Unfortunatley, if the lazy expression "calls" alloca, the stack memory will be deallocated when the lazy expression ends its scope. That's why allocating this memory in the caller stack frame is much easier.

Sorry, I didn't mean to imply that the lazy expression would call alloca. Rather it would behave like alloca, giving the stack pointer a different value on exit than on entry. 

(alloca is not a function anyway, it is just some arithmetic on pointers)
The problem is: if you give a different stack pointer to the callee, it would need to adjust back the pointer itself to access its data. And what if the callee needs to allocate some stack after evaluating a lazy expression? Where would the new stack frame go? How does the caller know where it would go?

After some thinking, I agree it might be possible to implement, but at what costs?
 

It would require the use of an additional register to hold the base address. This wouldn't be particularly expensive; it was after all till relatively recently the convention to hold a base pointer in ebp. 


Also, how would you track down all destructors that would be needed to be called?

That would be the job of the lazy expression routine; the callee would invoke the lazy expression routines in destruct mode as many times as they were invoked in evaluate mode and in reverse. 

Recall that you need a destruct mode to be invoked by the callee to ensure that destructors of temporaries are called at the end of the correct statement in the callee. 

I don't recall that. It was suggested, but I think the lazy expression temporaries must outlive the caller expression (not only the callee one).
void print(std::string_view sv) { std::cout << sv << std::endl; }
std
::string_view lazy([] -> std::string s) { return s; }

std
::string_view eager(const std::string & s) { return s; }

print(eager("hello world!"));

print(lazy("hello world!"));
The eager expression works perfectly fine now: the std::string temporary lifetime ends at the semicolon (https://godbolt.org/g/6xV4aM).
I would expect the second to also works for the very same reason.

This can happen only if the caller is the one to destroy the lazy expression temporaries.
 

Ah, that's an interesting point, thanks. I suppose it makes a difference whether you regard this facility as a different kind of parameter or as a macro replacement, but certainly your example should work, which is a strong argument for destruction handled by the caller. 



If you want multiple scopes, you will need heap memory allocation at some point, which is really not needed to have a very useful feature.

I don't see why? 

Maybe not, but very hard. Especially to make the destruction happens in the caller as I explained just above.
 

You can always create more complex objects from the simple lazy expression that cannot be evaluated multiple times to achieve your goal.

Yes, but then you lose prvalue-ness, as you have pointed out. 

I don't see any valid use case where these 2 features might be needed at the same time. If you really want multiple evaluations also allowing prvalues, why callables are not enough?
 

There's advantages of terseness, and also of lifetime of temporaries - extended into the callee, if not the caller. 




Also, what if the lazy expression calls new: how do you track down the multiple allocations?

Assuming the new-expressions are adopted by a smart pointer, normal RAII would apply. 

You don't need RAII for this code to be correct:
int i = /* ... */;
int* p = i > 0 ? new int(i) : nullptr;
/* ... */
if (p) delete p;
Why would it be different with custom functions, or custom types?

I agree smart pointers are much safer and should be used everywhere a new/delete was used. But this is still valid code.
 

It's fragile if the ellipsized code can throw, or merely return. I prefer smart pointers or scope guards wherever reasonable. 


If the lazy expression is very expensive, do you really want do execute the all thing again?

Probably not, but that should generally be the user's choice. 

A user might not be able to tell if an expression is expensive or not. It will be even more true with this because it will be simpler to call complex constructors without even noticing (and that's part of the point).
 

Caching is the only sensible way to allow multiple evaluations, but this forbids pr-values, with all the issues mentionned before.

And as I showed earlier, if a caller needs to cache the result, it can do it. Whereas other proposals do not/cannot solve altogether the pr-value issue, the lifetime issues and the performance issue.
Of course it needs a bit of boilerplate code to actually do the caching, but this use case should be rare enough. And the caching code is really understandable and easy to come up with, especially if you know the lazy expression will always be evaluated.

One solution might be to have both: [] -> lazy parameter without caching  and [=] -> lazy parameter with caching (with no consideration on how it is implemented).

Anyway, if you need multiple evaluations, why do you pass a lazy expression and not a callable? That would make your intent much clearer.

In general, I agree with all of the above. I am concerned that there may be use cases (e.g. current macros) that would benefit from multiple evaluation (non-cached) being defined, and I would not want multiple evaluation to be excluded that I believe I have shown can be avoided. 

For now, I don't see any use case that would really need this syntax boilerplate free at call site feature.
I really believe that this kind of problems should be solved either with callables or "macro"-like constructions.

Also multiple evaluation could also be added afterwards if there is a need for it. As long as we say multiple evaluation is undefined, then extending it would be possible.

Yes, that's reasonable. Thanks for taking the time to present your ideas and arguments. 



Sorry, missed a few words there: I would not want multiple evaluation to be excluded *on the basis of technical considerations* that I believe I have shown can be avoided.

More than technical considerations, it is performance considerations. You want the fast cases being fast: if the lazy expression is just a literal, you really don't want all the implementation boilerplate to deal with multiple evaluations and temporaries bookkeeping.
Your solution makes this use case inefficient because the callee won't know the lazy expression is as simple as that.
In the implementation I proposed, this is as fast as it could get without inlining: just a call to a function returning the literal.

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/5oUZysJB4HE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

Bengt Gustafsson

unread,
Jun 14, 2018, 3:19:25 AM6/14/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com
I have been pondering this proposal and P0927 for a while to see if I could reconcile the two use cases of P0927 into one decent feature. I failed, and I think that they must be served by different features.

The first use case is to avoid evaluating argument expressions if not needed, which may be a performance optimization such as in a logging case or semantically important as in a shortcut operator. The parameter of std::optional::value_or would also be a good use case if it wasn't too late for that. Here an important property is that the call site does not have to care whether the parameter is lazy and that the argument can be forwarded to another lazy parameter without problems. These two requirements mean that temporaries of the argument expression must be preserved until the callee returns and that the parameter must be allowed to be evaluatated more than once (or forwarding would be ridiculously useless).

The second use case is for a nullary lambda which is usable inside the callee by returning values when called. A primary feature is that the callee should not have to be a template function. Thanks to guaranteed return value elison this can allow initiating non-movable types as if values by that type were magically passed into the callee. This would also benefit from allowing an argument expression to be transformed to such a nullary lambda but this is not a really important property compared to the general need for a non-templated way of passing callables to functions. The proposed std::function_ref class serves this purpose well.

With this in mind I argue for a lazy parameter passing method, maybe using a void func([]->int x) syntax. This is still to be viewed as a parameter by both caller and callee and not as a terse lambda.

The other use case I think is served well enough by std::function_ref. I tried the idea of allowing a function_ref<T()> to have a constructor from a []->T but I couldn't get the semantics intiutive enough due to the requirement that it should be able to return a non-copyable type.


Den måndag 14 maj 2018 kl. 15:16:07 UTC+2 skrev floria...@gmail.com:
Hello everyone,

I would want to propose a new way to pass parameters to functions.
Currently, we have 2 way to pass parameters: by value, and by reference.
(Passing by pointer is really passing by value the address)

However, none of those allow delayed evaluation of parameters.
But this would be useful in some cases: for example overloading operator&& and operator|| (and also operator?: when it will be overloadable).
It could also be used to implement a select function defined like this:
float select(int i, float a, float b, float c) {
 
switch (i) {
   
case 0:
     
return a;

   
case 1:
     
return b;

   
case 2:
     
return c;

 
}
}

The delayed evaluation is important to allow a parameter with side effects not to be executed if not needed, avoiding undefined behaviors:
float *p;

if (p && *p > 0) { ... } // this is valid

bool AND(bool a, bool b) {
  return a && b;
}

if (AND(p, *p > 0)) { ... } // This is not


I thought of 4 ways to express that:

Type qualifier + implicit evaluation:
float select(int i, float @ a, float @ b, float @ c) {
 
switch (i) {
   
case 0:
     
return a;

   
case 1:
     
return b;

   
case 2:
     
return c;

 
}
}

Type qualifier + operator:
float select(int i, float @ a, float @ b, float @ c) {
 
switch (i) {
   
case 0:
     
return @a;

   
case 1:
     
return @b;

   
case 2:
     
return @c;

 
}
}

Type qualifier + callable:
float select(int i, float @ a, float @ b, float @ c) {
 
switch (i) {
   
case 0:
     
return a();

   
case 1:
     
return b();

   
case 2:
     
return c();

 
}
}

STL type:
float select(int i, std::expression<float> a, std::expression<float> b, std::expression<float> c) {
 
switch (i) {
   
case 0:
     
return a();

   
case 1:
     
return b();

   
case 2:
     
return c();

 
}
}


In all the 3 cases, when the select function is called, a functor-like object is created computing the expression, and this object is then passed to the function.
float* p;
int i;

float r = select(i, 1.f, *p, std::sqrt(*p));

// equivalent to:

float r;
switch (i) {
  case 0:
    r = 1.f;
    break;

  case 1:
    r = *p;
    break;

  case 2:
    r = std::sqrt(*p);
    break;
}

I really have the impression this could be very interesting in some corner cases where there is currently no alternatives to do that safely (and efficiently).

Would this be sensible? Were there any attempt to formalize passing by expression?
Which way of expressing this should be preferred?

I would be glad to answer any question.

Florian

floria...@gmail.com

unread,
Jun 14, 2018, 7:37:16 AM6/14/18
to ISO C++ Standard - Future Proposals
I don't clearly get your point.

If in your second use case, you don't want multiple evaluations, then both use cases are possible within a single feature (as I already showed earlier).
If you do, std::function_ref would be the answer.

There is a case that would be improved with
struct Foo {
 
int i;
 
Foo(const Foo&) = delete;
};

template <class T>
struct Bar {
  T
* p = nullptr;
 
~Bar() { delete p; }

 
void set(T const& v) { p = new T(v); }
 
void set_lazy([] -> T v) { p = new T(v()); }
 
void set_func(std::function_ref<T()> f) { return new T(f()); }
 
template <class... Args>
 
void emplace(Args&&... args) { p = new T(std::forward<Args>(args)...); }
};

Bar<Foo> bar;
// We want to construct Foo from { .i = 1 } // designated initializer list
bar
.set({ .i = 1 }); // Foo is not copyable
bar
.set_lazy({ .i = 1 }); // this is fine
bar
.emplace(/* what here? */); // designated initializer list construction is not a constructor
bar
.set_func([]() -> Foo { return { .i = 1 }; }); // name has to be mentionned here

In this case, set cannot work because Foo is non-copyable.
emplace cannot work because it is not a constructor (and it is not type-erased).
And you would need to name the type to make set_func works (even with an hypothetical terse lambda syntax).
While set_lazy would have all the good properties we want here.

Edward Catmur

unread,
Jun 14, 2018, 2:15:25 PM6/14/18
to std-pr...@isocpp.org
On Thu, Jun 14, 2018 at 8:19 AM, Bengt Gustafsson <bengt.gu...@beamways.com> wrote:
I have been pondering this proposal and P0927 for a while to see if I could reconcile the two use cases of P0927 into one decent feature. I failed, and I think that they must be served by different features.

The first use case is to avoid evaluating argument expressions if not needed, which may be a performance optimization such as in a logging case or semantically important as in a shortcut operator. The parameter of std::optional::value_or would also be a good use case if it wasn't too late for that. Here an important property is that the call site does not have to care whether the parameter is lazy and that the argument can be forwarded to another lazy parameter without problems. These two requirements mean that temporaries of the argument expression must be preserved until the callee returns and that the parameter must be allowed to be evaluatated more than once (or forwarding would be ridiculously useless).

I don't see it, why would forwarding require allowing evaluating the parameter more than once?

Bengt Gustafsson

unread,
Jun 15, 2018, 6:21:39 AM6/15/18
to ISO C++ Standard - Future Proposals
I understand that there was hard to see my conclusion here, but I showed an example earlier in this thread where a shortcut operation, as part of its implementation, logs the incoming parameter value by forwarding it to a logging function that may, or may not, actually evaluate the parameter. Thus when log returns, the main callee does not know if the parameter has already been evaluated as it depend on whether logging is enabled. Thus if it requires the parameter value it must be able to do so even if log() also did.

Bengt Gustafsson

unread,
Jun 15, 2018, 6:36:12 AM6/15/18
to ISO C++ Standard - Future Proposals, floria...@gmail.com


Den torsdag 14 juni 2018 kl. 13:37:16 UTC+2 skrev floria...@gmail.com:
I don't clearly get your point.

If in your second use case, you don't want multiple evaluations, then both use cases are possible within a single feature (as I already showed earlier).

The problem is that if the first use case is for a by value parameter and evaluates it multiple times the type must be copyable. So we can't combine allowing multiple usage with  allowing non-copyability.
If you consider the code that the compiler has to generate for the lazy parameter thunk I can't see that we can get both in the same feature. The only possibility would be for the compiler to generate different code depending on whether the
parameter type is copyable. This could be possible but I don't have time right now to think that trhough.

 
If you do, std::function_ref would be the answer.


 

There is a case that would be improved with
struct Foo {
 
int i;
 
Foo(const Foo&) = delete;
};

template <class T>
struct Bar {
  T
* p = nullptr;
 
~Bar() { delete p; }

 
void set(T const& v) { p = new T(v); }
 
void set_lazy([] -> T v) { p = new T(v()); }
 
void set_func(std::function_ref<T()> f) { return new T(f()); }
 
template <class... Args>
 
void emplace(Args&&... args) { p = new T(std::forward<Args>(args)...); }
};

Bar<Foo> bar;
// We want to construct Foo from { .i = 1 } // designated initializer list
bar
.set({ .i = 1 }); // Foo is not copyable
bar
.set_lazy({ .i = 1 }); // this is fine
bar
.emplace(/* what here? */); // designated initializer list construction is not a constructor
bar
.set_func([]() -> Foo { return { .i = 1 }; }); // name has to be mentionned here

In this case, set cannot work because Foo is non-copyable.
emplace cannot work because it is not a constructor (and it is not type-erased).
And you would need to name the type to make set_func works (even with an hypothetical terse lamb 
da syntax). While set_lazy would have all the good properties we want here.
 
Yes, I know. This is the second use case. I don't see it as important at all compared to the first use case. I would like someone to come up with a situation where this is actually interesting. Is it even legal to initialize a class with a constructor (albeit deleted) from a braced init list. If it is: Can you come up with a class which as good reason to be non-movable but where the constructor does absolutely nothing. If so, how can it be important to not have a constructor instead of the braced init list. If there is a constructor it would be much more logical to send the constructor parameters than a "fake" object as the parameter type. After all it is very odd to pass an object of a non-copyable type as a BY VALUE parameter. This feels like a totally academic excecise devised just to prevent a perfectly good feature (lazy parameters) from getting a reasonable and easy to understand definition.

floria...@gmail.com

unread,
Jun 15, 2018, 7:42:59 AM6/15/18
to ISO C++ Standard - Future Proposals
I understand that there was hard to see my conclusion here, but I showed an example earlier in this thread where a shortcut operation, as part of its implementation, logs the incoming parameter value by forwarding it to a logging function that may, or may not, actually evaluate the parameter. Thus when log returns, the main callee does not know if the parameter has already been evaluated as it depend on whether logging is enabled. Thus if it requires the parameter value it must be able to do so even if log() also did.

bool andand([]->bool lhs, []->bool rhs)
{
    
Log("Performing andand on", lhs, " and ", rhs);   // Variadic log with lazy parameters only evaluated if logging is enabled.
    
return lhs() && rhs();    // This is UB if logging is enabled!
}

First, the optional destruction is easy to solve at the call site with a flag the the lazy expression can manipulate.
And this flag can be optimized away if every temporary is trivially destructible (the callee should not be aware of such a flag).

For the multiple evaluation, I showed that allowing lazy parameters to be captured could solve the problem:
bool andand([] -> bool lhs_lazy, [] -> bool rhs_lazy) {
  std
::optional<bool> lhs_v, rhs_v;
 
auto lhs = [&] () -> bool { if (!lhs_v) { lhs_v = lhs_lazy(); } return lhs_v.value(); };
  auto rhs = [&] () -> bool { if (!rhs_v) { rhs_v = rhs_lazy(); } return rhs_v.value(); };
 
  Log("Performing andand on", lhs(), " and ", rhs());
  return lhs() && rhs();
}

You could even say it is possible ta lazy expressions in variables and members and create a wrapper to do that automatically:
template <class T>
class multiple_execution {
 
private:
   
[] -> T lazy;
    std
::optional<T> val;
   
void execute() {
     
if (!val) {
        val
= lazy();
     
}
   
}
 
public:
    multiple_execution
([] -> T lazy) : lazy(lazy) {}
    multiple_execution
() = delete;
    multiple_execution
(const multiple execution&) = delete;
    multiple_execution
& operator=(const multiple_execution&) = delete;
   
~multiple_execution() = default;

    T
& operator()() & {
      execute
();
     
return val.value();
   
}
    T
&& operator()() && {
      execute
();
     
return *std::move(val)
   
}
};

And your logging example would look like:
bool andand([] -> bool lhs_lazy, [] -> bool rhs_lazy) {

  multiple_execution
<bool> lhs(lhs_lazy()), rhs(rhs_lazy());
 
Log("performing andand on ", lhs(), " and ", rhs());
 
return lhs() && rhs();
}



Le vendredi 15 juin 2018 12:36:12 UTC+2, Bengt Gustafsson a écrit :


Den torsdag 14 juni 2018 kl. 13:37:16 UTC+2 skrev floria...@gmail.com:
I don't clearly get your point.

If in your second use case, you don't want multiple evaluations, then both use cases are possible within a single feature (as I already showed earlier).

The problem is that if the first use case is for a by value parameter and evaluates it multiple times the type must be copyable. So we can't combine allowing multiple usage with  allowing non-copyability.
If you consider the code that the compiler has to generate for the lazy parameter thunk I can't see that we can get both in the same feature. The only possibility would be for the compiler to generate different code depending on whether the
parameter type is copyable. This could be possible but I don't have time right now to think that trhough.

You forget and third possibility: recreate the object several times allows both non-copyability and multiple executions. However, I think that shouldn't be a goal for such a feature.
In that case, I would much prefer having an actual callable part of the interface.

We don't want magic that tries to be clever, and has different behaviors depending on what the type supports (copyability, non-throw, moveability...)
The rules should be the same for every types.
My point here was I can address all your issues, but you cannot address all mines. So which approach is more powerful?
A bit more explanation on this (cumbersome) example:
Here Foo is an aggregate, so it can be aggregate initialized: I don't call any constructor in the example.
std::atomic<> is a valid class that is trivially constructible, and that is neither copyable nor movable.
Now, imagine I have this:
struct Point {
 
float x, y, z;
};

Bar<std::atomic<Point>> bar;
bar
.set_lazy({{ .x = 1.f, .y = 1.f, .z = 1.f }});
Here, of course you could make Point having a constructor taking 3 floats, but designated initializer is much clearer (that's why it has been standardize in the first place).


In the end, my approach is easy to understand: if you need multiple evaluations: wrap the lazy expression, or put a callable in the interface (depending on the use case).
The lifetime is the same as regular parameters.
From the caller point of view, the only thing to remember: the evaluation might not appear, but cannot appear more than once.
From the callee point of view: cannot evaluate more than once. If this it is needed: wrapping is possible.
In case of exceptions in the lazy expression, the callee will see (and be able to catch) the exception before the caller can, but that's also what you would expect from a delayed expression.
Really, I fail to see where my approach is unintuitive.

Nicol Bolas

unread,
Jun 15, 2018, 9:25:18 AM6/15/18
to ISO C++ Standard - Future Proposals
I don't understand why this is a good idea. If logging is enabled, then your program has substantially different behavior than if logging isn't enabled. Presumably, the user is using `andand` because they want to ensure that the rhs expression is only evaluated if the lhs expression is true, right? So if `andand`'s logging is active, then `andand` has the wrong behavior.

Remember: short-circuit evaluation usually is necessary because `rhs` is undefined behavior if `lhs` is false. Think `andand(x != nullptr && x->foo())`.

Nicol Bolas

unread,
Jun 15, 2018, 9:35:15 AM6/15/18
to ISO C++ Standard - Future Proposals
Sorry; I meant `andand(x != nullptr, x->foo()` at the end.

Also, if the idea with `andand` is not that we have short-circuit evaluation... then why does it need to use lazy evaluation at all? Why can't it just take 2 bools? Yes, there will be a few times when rhs isn't evaluated, but that's merely an optimization, not an actual part of the function's interface. And it's not an optimization that anyone will be relying on, since the caller must provide an expression with well-defined behavior.

So why is this important to be able to do?

Miguel Ojeda

unread,
Oct 11, 2018, 8:28:40 PM10/11/18
to std-pr...@isocpp.org
Hi,

We also discussed laziness and "expression-parameters" in two more places:

https://groups.google.com/a/isocpp.org/d/msg/std-proposals/jB5TIcRZeic/HV-y43WUAAAJ
https://groups.google.com/a/isocpp.org/d/msg/std-proposals/x5ucKwQI7Bc/y6iFvtQUBwAJ

The way I see it is that lazy arguments (i.e. only evaluated once even
if they appear several times -- what P0927 describes) are simply a
sub-case of "expressions copy-pasting"/"macro-like" etc.; so less
flexible, and more complex to specify.

With actual expressions (rather than lazy arguments), we can get a
much simpler proposal (there is not much to specify: simply say that
it behaves as-if the expression was written in-place). Then, if you
want lazy arguments, you can "implement" them simply by assigning the
expression once, to a variable, e.g.

using log(using auto lazy_arg) {
if (level < warning) {
auto x = lazy_arg;
// use x as usual
}
}

This is the syntax of P1221's "using" parameters that Jason came up
with from my "expression-parameters" suggestion.

Now, you can free more things since now you are not forced to a single
evaluation. But what is more: the caller has more freedom to decide
how to call it. Take P0927's example:

void fn([] -> std::unique_ptr<int> i) { ... }; // assume fn()
evaluates twice i

This would be:

using fn(using auto i) { ... }; // assume fn() evaluates twice i

Then:

fn(make_unique<int>()); // a new object each time
fn(ptr); // some pointer possibly used twice
fn(std::move(ptr)); // the second time possibly nullptr

Having something like this would remove almost all the remaining uses
for macros + gives us lazy arguments too. And it is simpler to
understand for users, no new semantics, etc. The limitation, of
course, is that this is a compile-time thing only (no real functions
-- but if you need that, you can still pass normal lambdas).

Cheers,
Miguel
> --
> You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
> To post to this group, send email to std-pr...@isocpp.org.
> To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/c91b3d77-5797-4abf-84dc-a7e854dafb9b%40isocpp.org.



--
Cheers,
Miguel

floria...@gmail.com

unread,
Oct 12, 2018, 4:29:48 AM10/12/18
to ISO C++ Standard - Future Proposals
Jason's proposal is interesting, and I also thought it could be used to implement lazy parameters, but actually, some stuff are just not possible with it.
The main thing I see is: the code must be visible where it is used.
With lazy parameters, you don't need to have the callee code visible from the caller. It can be compiled within another Translation Unit.
This can seriously decrease binary size (the same way type erasure does).

Also, what about lifetime of temporaries?
As far as I understand, the lifetime of the temporary will be the same as with a macro/code copy-pasted.
And this is something that is simply just not true with lazy parameters.
The lifetime of the lazy parameters is extended as-if it were normal parameters (until the end of the caller expression).
So no surprise for the user.
(If you want more details, you can read the thread again, there are many examples)
And if you want to integrate these lifetime rules within Jason's proposal, then it would become much more complex (maybe more complex than lazy parameters?).

A last point about multiple evaluation, this is tricky for the user on its own, and it's not needed most of the time. So that's fine for "macro-like" approach, but for lazy parameters, I don't think that's a good idea.
Currently in c++ (macros are not C++), there is no expression that could lead to multiple evaluations of sub-expression. But there already are some expression where sub-expressions can be discarded without being evaluated.

Jason's proposal is interesting on its own and has many use-cases, but I think lazy parameters is not one of them.
That's why I think the two should be pursued independently.

Miguel Ojeda

unread,
Oct 12, 2018, 6:03:57 AM10/12/18
to std-pr...@isocpp.org
On Fri, Oct 12, 2018 at 10:29 AM <floria...@gmail.com> wrote:
>
>
> Le vendredi 12 octobre 2018 02:28:40 UTC+2, Miguel Ojeda a écrit :
>>
> Jason's proposal is interesting, and I also thought it could be used to implement lazy parameters, but actually, some stuff are just not possible with it.
> The main thing I see is: the code must be visible where it is used.
> With lazy parameters, you don't need to have the callee code visible from the caller. It can be compiled within another Translation Unit.
> This can seriously decrease binary size (the same way type erasure does).

Indeed, that is what I meant when I said:

>> The limitation, of
>> course, is that this is a compile-time thing only (no real functions
>> -- but if you need that, you can still pass normal lambdas).

In my view, macro-like expansion is something we are 100% missing from
the language (i.e. cannot be done except with macros); while lazy
parameters are something that you can approximate quite nicely today
(e.g. lambdas). Therefore, I value much more the former!

>
> Also, what about lifetime of temporaries?
> As far as I understand, the lifetime of the temporary will be the same as with a macro/code copy-pasted.
> And this is something that is simply just not true with lazy parameters.
> The lifetime of the lazy parameters is extended as-if it were normal parameters (until the end of the caller expression).
> So no surprise for the user.
> (If you want more details, you can read the thread again, there are many examples)
> And if you want to integrate these lifetime rules within Jason's proposal, then it would become much more complex (maybe more complex than lazy parameters?).

Yep, I did, and I am with Nicol here -- there is no point in
integrating "lifetime rules" for the macro-like approach, that is the
beauty of it.

>
> A last point about multiple evaluation, this is tricky for the user on its own, and it's not needed most of the time.

It is tricky in the same ways as macros, so basically already known by
C programmers; i.e. it is an advantage w.r.t. introducing lazy
arguments (which requires a quite more complex proposal, as you said).

> So that's fine for "macro-like" approach, but for lazy parameters, I don't think that's a good idea.

Agreed: I didn't say it was appropriate for the lazy-arguments approach.

> Currently in c++ (macros are not C++), there is no expression that could lead to multiple evaluations of sub-expression. But there already are some expression where sub-expressions can be discarded without being evaluated.

Macros are C++, what do you mean? We try to avoid them because they
are on their own phase of translation which gives them shortcomings --
but that is one of the points of the proposal(s): as we have done
previously many times in C++, we are trying to reduce reliance on
macros).

Now, I agree that it is worth discussing whether using this vs. a
macro would be beneficial or not (i.e. whether the proposal is worth
its disadvantages).

>
> Jason's proposal is interesting on its own and has many use-cases, but I think lazy parameters is not one of them.
> That's why I think the two should be pursued independently.

Isn't the only non-overlapping use case the ability to use declare
functions which use lazy arguments without a definition? (If I am
missing other significant use cases that cannot be provided by a
macro-like proposal, please let me know!).

Cheers,
Miguel

floria...@gmail.com

unread,
Oct 12, 2018, 6:43:23 AM10/12/18
to ISO C++ Standard - Future Proposals

Le vendredi 12 octobre 2018 12:03:57 UTC+2, Miguel Ojeda a écrit :
On Fri, Oct 12, 2018 at 10:29 AM <floria...@gmail.com> wrote:
>
>
> Le vendredi 12 octobre 2018 02:28:40 UTC+2, Miguel Ojeda a écrit :
>>
> Jason's proposal is interesting, and I also thought it could be used to implement lazy parameters, but actually, some stuff are just not possible with it.
> The main thing I see is: the code must be visible where it is used.
> With lazy parameters, you don't need to have the callee code visible from the caller. It can be compiled within another Translation Unit.
> This can seriously decrease binary size (the same way type erasure does).

Indeed, that is what I meant when I said:

>> The limitation, of
>> course, is that this is a compile-time thing only (no real functions
>> -- but if you need that, you can still pass normal lambdas).

In my view, macro-like expansion is something we are 100% missing from
the language (i.e. cannot be done except with macros); while lazy
parameters are something that you can approximate quite nicely today
(e.g. lambdas). Therefore, I value much more the former!

And I agree on that. (I wouldn't say lambdas are a nice approximation of lazy parameters though)
 

>
> Also, what about lifetime of temporaries?
> As far as I understand, the lifetime of the temporary will be the same as with a macro/code copy-pasted.
> And this is something that is simply just not true with lazy parameters.
> The lifetime of the lazy parameters is extended as-if it were normal parameters (until the end of the caller expression).
> So no surprise for the user.
> (If you want more details, you can read the thread again, there are many examples)
> And if you want to integrate these lifetime rules within Jason's proposal, then it would become much more complex (maybe more complex than lazy parameters?).

Yep, I did, and I am with Nicol here -- there is no point in
integrating "lifetime rules" for the macro-like approach, that is the
beauty of it.


I agree: there is no point in integrating special lifetime rules for a macro approach.
But we want those extra lifetime rules for lazy parameters (reread the thread if you want to know why).
 
>
> A last point about multiple evaluation, this is tricky for the user on its own, and it's not needed most of the time.

It is tricky in the same ways as macros, so basically already known by
C programmers; i.e. it is an advantage w.r.t. introducing lazy
arguments (which requires a quite more complex proposal, as you said).

Even the most experienced C programmer can be tricked by multiple evaluations within macros: either because they didn't realize it was a macro, or more subtle because experienced programmers write macros in such a way that parameters are not evaluated more than once (for safety reasons).
 

> So that's fine for "macro-like" approach, but for lazy parameters, I don't think that's a good idea.

Agreed: I didn't say it was appropriate for the lazy-arguments approach.

> Currently in c++ (macros are not C++), there is no expression that could lead to multiple evaluations of sub-expression. But there already are some expression where sub-expressions can be discarded without being evaluated.

Macros are C++, what do you mean? We try to avoid them because they
are on their own phase of translation which gives them shortcomings --
but that is one of the points of the proposal(s): as we have done
previously many times in C++, we are trying to reduce reliance on
macros).

(No, the preprocessor is not part of C++. C++ acknowledges the existence of the preprocessor, but not really more. But that's just being picky...).
Anyway: I completely understand and agree with Jason's proposal.
Even more: I encourage it.
But I say that lazy parameters should be something else, not related to the "macro-like" approach of Jason.
 

Now, I agree that it is worth discussing whether using this vs. a
macro would be beneficial or not (i.e. whether the proposal is worth
its disadvantages).

>
> Jason's proposal is interesting on its own and has many use-cases, but I think lazy parameters is not one of them.
> That's why I think the two should be pursued independently.

Isn't the only non-overlapping use case the ability to use declare
functions which use lazy arguments without a definition? (If I am
missing other significant use cases that cannot be provided by a
macro-like proposal, please let me know!).

You miss the lifetime rules. Those are crucial.
And the binary size reduction (implied by the defintion not being required), but that's marginal I would say.

All in all, I think you misunderstood me:
I'm all for Jason's proposal. But I think it is not adapted to implement lazy parameters because of the lifetime issues (and the "forced inline" policy).
And the 2 are compatible with each other: if you have one, you can have the other later.

Ville Voutilainen

unread,
Oct 12, 2018, 6:50:59 AM10/12/18
to ISO C++ Standard - Future Proposals
On Fri, 12 Oct 2018 at 13:43, <floria...@gmail.com> wrote:
>> > Currently in c++ (macros are not C++), there is no expression that could lead to multiple evaluations of sub-expression. But there already are some expression where sub-expressions can be discarded without being evaluated.
>>
>> Macros are C++, what do you mean? We try to avoid them because they
>> are on their own phase of translation which gives them shortcomings --
>> but that is one of the points of the proposal(s): as we have done
>> previously many times in C++, we are trying to reduce reliance on
>> macros).
>
>
> (No, the preprocessor is not part of C++. C++ acknowledges the existence of the preprocessor, but not really more. But that's just being picky...).

That's not correct; preprocessing is a phase of translation specified
in the C++ standard. Macros are C++, and so is the preprocessor.

floria...@gmail.com

unread,
Oct 12, 2018, 7:26:17 AM10/12/18
to ISO C++ Standard - Future Proposals
I always thought it was not part of C++ standard, but you're actually right.

Corentin

unread,
Oct 12, 2018, 7:52:16 AM10/12/18
to std-pr...@isocpp.org
I do not see the value of a language feature for lazy evaluation. 
Any solution would ultimately look a lot like a lambda, maybe with a shorter syntax but it would essentially look and behave the same, so there would be a little gain.
A more generic solution for that particular use case would be to simplify lambda everywhere (see for example http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0573r2.html )


However, I do think that superseding all or part of the pre-processor macros (token soup injection), by a language level feature is something  that would be incredibly powerful,
And as such, Jason proposal is a much better solution overall.

But I think it's important to think about what we want.
  • That language feature results in visible AST-nodes
  • Scoped names - that seems evident but it's one of the issues with C-macro
  • The definition should be checked
  • The parameters should be checked - All expressions have the nice property of having a type, so parametric expression should be able to constrain on type/concepts
I think it's important to keep in mind that what Jason proposes and similar proposals never call anything or return anything, regarding of whether we end up on a solution building on top of  code injection/expression reflection or not, the result is basically the same: shifting ast-nodes around so name lookup etc is a non-issue - but you can never escape the scope.

(Having the injectee/parametric expression being able to see in each other scope would probably be of little value and a pretty big nightmare) 

A code-injection based system can be extended to be able to inject statements in the current scope.
So it would be possible to have sanitary language-level macros that have expressions 
as parameters and expand to one or several statements.
Herb and Andrew also demonstrated ways to construct identifiers and typename at compile time and all of these pieces can fit nicely together.

However, that does not, unfortunately, replaces all macro use cases.
Namely, I have no idea how to inject partial statements, which is something a lot of test frameworks do, for example

There is no one unique replacement for macros but instead a multitude of solutions.
And I don't think it would be a bad thing to have a gradient of solutions (function, constexpr functions, constexpr parameters, parametric expressions, something else in the future) as long as the whole is cohesive.


My 2 cents


--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

floria...@gmail.com

unread,
Oct 12, 2018, 8:13:25 AM10/12/18
to ISO C++ Standard - Future Proposals
Le vendredi 12 octobre 2018 13:52:16 UTC+2, Corentin a écrit :
I do not see the value of a language feature for lazy evaluation. 
Any solution would ultimately look a lot like a lambda, maybe with a shorter syntax but it would essentially look and behave the same, so there would be a little gain.
A more generic solution for that particular use case would be to simplify lambda everywhere (see for example http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0573r2.html )

This point has already been debated many times on this thread. And the point is: we need lazy evaluation, meaning a way for a function to take something "callable" without any syntax overhead on the call site.
P0573r2 is not about that. But it remains interesting in a world where lazy parameters exist.
The two are orthogonal and independant.
 


However, I do think that superseding all or part of the pre-processor macros (token soup injection), by a language level feature is something  that would be incredibly powerful,
And as such, Jason proposal is a much better solution overall.

Yes, I also do want to prioritize Jason's proposal (or any well thought alternative) over lazy parameters.
But Jason's proposal is not good enough for proper lazy parameters. So I also want lazy parameters in addition.
And again, those proposals are also orthogonal and independant.
 

But I think it's important to think about what we want.
  • That language feature results in visible AST-nodes
  • Scoped names - that seems evident but it's one of the issues with C-macro
  • The definition should be checked
  • The parameters should be checked - All expressions have the nice property of having a type, so parametric expression should be able to constrain on type/concepts
I think it's important to keep in mind that what Jason proposes and similar proposals never call anything or return anything, regarding of whether we end up on a solution building on top of  code injection/expression reflection or not, the result is basically the same: shifting ast-nodes around so name lookup etc is a non-issue - but you can never escape the scope.

And this scope problem is why "macro-like" approach is not enough for proper lazy parameters: we want temporary objects from the lazy expression to outlive the callee.
 

(Having the injectee/parametric expression being able to see in each other scope would probably be of little value and a pretty big nightmare) 

A code-injection based system can be extended to be able to inject statements in the current scope.
So it would be possible to have sanitary language-level macros that have expressions 
as parameters and expand to one or several statements.
Herb and Andrew also demonstrated ways to construct identifiers and typename at compile time and all of these pieces can fit nicely together.

However, that does not, unfortunately, replaces all macro use cases.
Namely, I have no idea how to inject partial statements, which is something a lot of test frameworks do, for example

There is no one unique replacement for macros but instead a multitude of solutions.
And I don't think it would be a bad thing to have a gradient of solutions (function, constexpr functions, constexpr parameters, parametric expressions, something else in the future) as long as the whole is cohesive.


You've just said it: there is no ONE solution. And lazy parameter cannot be solved properly by parametric expressions. That's why we need something else for that.
But we don't need to wait for lazy parameters in order to standardize parametric expressions. Those are orthogonal and can be standardized independently.

Corentin

unread,
Oct 12, 2018, 8:47:10 AM10/12/18
to std-pr...@isocpp.org
I'm not sure no marking at the call site would be a good idea.
You most likely want some syntax notifying "This thing that you think is evaluating may in fact, not be evaluated"
There is a value in being explicit, and not giving the same syntax to multiple constructs 

function_call( [x, y] =>  x + y);   

Similarly, I think we probably want some syntax for jason's proposal  to distinguish "this is evaluating the parameters and invoking a function" from "this expands to some more code"
Rust has the `!` syntax for this reason (printn!("I am a macro"))


--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

floria...@gmail.com

unread,
Oct 12, 2018, 8:55:45 AM10/12/18
to ISO C++ Standard - Future Proposals

Le vendredi 12 octobre 2018 14:47:10 UTC+2, Corentin a écrit :
I'm not sure no marking at the call site would be a good idea.
You most likely want some syntax notifying "This thing that you think is evaluating may in fact, not be evaluated"
There is a value in being explicit, and not giving the same syntax to multiple constructs 

function_call( [x, y] =>  x + y);   

This has also already been discussed here.
One reason we want no syntax overhead at call site is to have shortcircuiting || && operators on custom types (and also ?: )
 

Similarly, I think we probably want some syntax for jason's proposal  to distinguish "this is evaluating the parameters and invoking a function" from "this expands to some more code"
Rust has the `!` syntax for this reason (printn!("I am a macro"))


That is debatable...

Matthew Woehlke

unread,
Oct 12, 2018, 11:11:20 AM10/12/18
to std-pr...@isocpp.org, Corentin
On 12/10/2018 07.52, Corentin wrote:
> However, that does not, unfortunately, replaces all macro use cases.
> Namely, I have no idea how to inject partial statements, which is something
> a lot of test frameworks do, for example

Indeed, and I was just thinking about this very same issue.

Other things for which we don't currently have a replacement:

#define call(x, ...) log(#x, x, __VA_ARGS__) // ?
#define signal protected
#define with(x) if(x; true)

I'm not *entirely* certain if it's possible with reflection to implement
the first as a non-macro. Even if *that* case is possible, I'd have to
question if all possible uses of stringifying can be replaced,
especially those that stringify entire expressions (again, something
often found in test frameworks).

For the second, I doubt any replacement is forthcoming.

For the third, in particular, I'm thinking of macros that expand to a
statement which opens a scope, i.e. `if`, `while`, `for`, ... where the
macro is followed by a statement or an open brace which is associated
with the macro. I suppose we could define a way to make "new control
statements" that would always have this property (in fact, that might
even be a very interesting feature, especially if it incorporates
mechanisms to execute code at the end of the scope and possibly to
decide if the scope should be repeated like a loop). However AFAIK the
current "parametric expressions" proposal would not cover these cases.

--
Matthew

Miguel Ojeda

unread,
Oct 12, 2018, 11:51:29 AM10/12/18
to std-pr...@isocpp.org
On Fri, Oct 12, 2018 at 12:43 PM <floria...@gmail.com> wrote:
>
> Le vendredi 12 octobre 2018 12:03:57 UTC+2, Miguel Ojeda a écrit :
>>
>> On Fri, Oct 12, 2018 at 10:29 AM <floria...@gmail.com> wrote:
>>
>> It is tricky in the same ways as macros, so basically already known by
>> C programmers; i.e. it is an advantage w.r.t. introducing lazy
>> arguments (which requires a quite more complex proposal, as you said).
>
>
> Even the most experienced C programmer can be tricked by multiple evaluations within macros: either because they didn't realize it was a macro, or more subtle because experienced programmers write macros in such a way that parameters are not evaluated more than once (for safety reasons).
>

Anyone can make mistakes with any feature, that is not the really
point... Macros are already understood by all C++ (and C) programmers.
They are not *that* tricky, compared to other C++ features.

>> Macros are C++, what do you mean? We try to avoid them because they
>> are on their own phase of translation which gives them shortcomings --
>> but that is one of the points of the proposal(s): as we have done
>> previously many times in C++, we are trying to reduce reliance on
>> macros).
>
>
> (No, the preprocessor is not part of C++. C++ acknowledges the existence of the preprocessor, but not really more. But that's just being picky...).

No. Please refer to [cpp] or [lex] in the standard.

>> Isn't the only non-overlapping use case the ability to use declare
>> functions which use lazy arguments without a definition? (If I am
>> missing other significant use cases that cannot be provided by a
>> macro-like proposal, please let me know!).
>
>
> You miss the lifetime rules. Those are crucial.

I don't see what the lifetime rules buys you that the macro-like
approach doesn't. Please provide an example.

Cheers,
Miguel

Corentin

unread,
Oct 12, 2018, 12:41:39 PM10/12/18
to std-pr...@isocpp.org
the thing we must be careful with is to keep in mind that some usages for macros are _not_ worth replacing

On Fri, 12 Oct 2018 at 17:11 Matthew Woehlke <mwoehlk...@gmail.com> wrote:
On 12/10/2018 07.52, Corentin wrote:
> However, that does not, unfortunately, replaces all macro use cases.
> Namely, I have no idea how to inject partial statements, which is something
> a lot of test frameworks do, for example

Indeed, and I was just thinking about this very same issue.

Other things for which we don't currently have a replacement:

  #define call(x, ...) log(#x, x, __VA_ARGS__) // ?

This could be possible, and I think it's important indeed.
But if reflection on type give you the type's name, it seems reasonable to expect
that reflecting on expressions could give you its string representation. hopefully, as in the source code.
 
  #define signal protected

Ok, this does not sounds like a legitimate use case ( sorry Qt )
This is needed for moc, but the ultimate goal is is a moc-less Qt.
And indeed this causes actual bug in software if for example you use Qt along boost::signal

A better solution would be [[qt::signal]] void my signal.

I think means to defining dialects should be specifically a non-goal in this problem space
 
  #define with(x) if(x; true)

Yes, this is the big problem.
I'm trying to determine whether is worth caring about it to begin with because that falls into
the "defining dialect" category.
But as you point out, it's kinda required by test frameworks.

But if we manage to only need macros in test code, it would already be a big win !
 

I'm not *entirely* certain if it's possible with reflection to implement
the first as a non-macro. Even if *that* case is possible, I'd have to
question if all possible uses of stringifying can be replaced,
especially those that stringify entire expressions (again, something
often found in test frameworks).

For the second, I doubt any replacement is forthcoming.

For the third, in particular, I'm thinking of macros that expand to a
statement which opens a scope, i.e. `if`, `while`, `for`, ... where the
macro is followed by a statement or an open brace which is associated
with the macro. I suppose we could define a way to make "new control
statements" that would always have this property (in fact, that might
even be a very interesting feature, especially if it incorporates
mechanisms to execute code at the end of the scope and possibly to
decide if the scope should be repeated like a loop). However AFAIK the
current "parametric expressions" proposal would not cover these cases.

--
Matthew

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

Matthew Woehlke

unread,
Oct 12, 2018, 2:30:49 PM10/12/18
to std-pr...@isocpp.org, Corentin
On 12/10/2018 12.41, Corentin wrote:
> the thing we must be careful with is to keep in mind that some usages for
> macros are _not_ worth replacing

If by that, you mean that the preprocessor is not a terrible evil that
must be replaced at all costs... then I'd have to agree :-). Sure, there
are some things that maybe can be done *better* in other ways, but I've
never really understood the anti-PP crowd.

> On Fri, 12 Oct 2018 at 17:11 Matthew Woehlke wrote:
>> Other things for which we don't currently have a replacement:
>>
>> #define call(x, ...) log(#x, x, __VA_ARGS__) // ?
>
> This could be possible, and I think it's important indeed.

Yes, this was probably not the best example. In the case that the macro
argument is definitely a single "thing", then I would hope reflection
can potentially replace such uses.

I'm less convinced we can replace e.g.:

#define EXPECT_EQUAL(a, b) \
::testing::expect_equal(a, b, #a, #b, __FILE__, __LINE__, true)

>> #define signal protected
>
> A better solution would be [[qt::signal]] void my signal.

Plausible, though annoying to have to mark *every* signal and slot. This
makes me wonder if being able to apply an attribute to an access
specifier (which would implicitly "percolate" to everything so affected)
would be desirable?

> I think means to defining dialects should be specifically a non-goal in
> this problem space

...but if that's the case, will you still use the preprocessor? (BTW,
I'm fine with that!)

>> #define with(x) if(x; true)
>
> Yes, this is the big problem.
> I'm trying to determine whether is worth caring about it to begin with
> because that falls into the "defining dialect" category.
> But as you point out, it's kinda required by test frameworks.

Right. And by folks that really, really want a `with` statement ;-).
OTOH, some of the more useful cases of this form are being replaced with
real grammar, e.g. range-based for.

--
Matthew

floria...@gmail.com

unread,
Oct 15, 2018, 6:15:27 AM10/15/18
to ISO C++ Standard - Future Proposals

Le vendredi 12 octobre 2018 17:51:29 UTC+2, Miguel Ojeda a écrit :
On Fri, Oct 12, 2018 at 12:43 PM <floria...@gmail.com> wrote:
>
> Le vendredi 12 octobre 2018 12:03:57 UTC+2, Miguel Ojeda a écrit :
>>
>> On Fri, Oct 12, 2018 at 10:29 AM <floria...@gmail.com> wrote:
>>
>> It is tricky in the same ways as macros, so basically already known by
>> C programmers; i.e. it is an advantage w.r.t. introducing lazy
>> arguments (which requires a quite more complex proposal, as you said).
>
>
> Even the most experienced C programmer can be tricked by multiple evaluations within macros: either because they didn't realize it was a macro, or more subtle because experienced programmers write macros in such a way that parameters are not evaluated more than once (for safety reasons).
>

Anyone can make mistakes with any feature, that is not the really
point... Macros are already understood by all C++ (and C) programmers.
They are not *that* tricky, compared to other C++ features.  

>> Macros are C++, what do you mean? We try to avoid them because they
>> are on their own phase of translation which gives them shortcomings --
>> but that is one of the points of the proposal(s): as we have done
>> previously many times in C++, we are trying to reduce reliance on
>> macros).
>
>
> (No, the preprocessor is not part of C++. C++ acknowledges the existence of the preprocessor, but not really more. But that's just being picky...).

No. Please refer to [cpp] or [lex] in the standard.

You're right. I was fooled by the fact that the preprocessor really looks like a separate language on its own.
But that doesn't really change my point (because the preprocessor looks like a separate language on its own).
 

>> Isn't the only non-overlapping use case the ability to use declare
>> functions which use lazy arguments without a definition? (If I am
>> missing other significant use cases that cannot be provided by a
>> macro-like proposal, please let me know!).
>
>
> You miss the lifetime rules. Those are crucial.

I don't see what the lifetime rules buys you that the macro-like
approach doesn't. Please provide an example.

This example was already provided previously in this thread; but here it is (updated):
void print(std::string_view sv) { std::cout << sv << std::endl; }


std
::string_view eager(const std::string & s) {
  std
::string_view sv{s};
 
return sv;

}
std
::string_view lazy([] -> std::string s) {

  std
::string_view sv{s};
 
return sv;
}
std
::string_view call(std::function<std::string()> f) {
  std
::string_view sv{f()};
 
return sv;
}
using macro(auto s) {
  std
::string_view sv{s};
 
return sv;
}

print(eager(std::string("hello world!"))); // here is fine
print(call([]{ return std::string("hello world!");})); // here, `std::string("hello world")` will be destroyed before calling `print`
print(lazy(std::string("hello world!"))); // here, we want it to have the same lifetime as for `eager`
print(macro(std::string("hello world!"))); // what here ?

Here, with eager parameters, the std::string("hello world!") lifetime ends after the call to print.
The same example with a callable does not work as the string is destroyed before the call to print (so before the string_view is actually used).

In the lazy case, we want the string to also be destroyed after the call to print as in the eager case.
I already showed previously in this thread how to implement such lifetimes with lazy parameters with no overhead on the cases where it is not needed.

Now, what's the lifetime of the std::string in the macro case?
I can think of 2 cases: The string will be destroyed just after the string_view is constructed (so before it is returned).
Or the string is detroyed when the string_view is returned (so before the call to print).

That's the kind of lifetime issues lazy parameters should solve and macro-like approach does not handle (AFAIU).
We can imagine to apply the same special rules we imagined for lazy parameters to the parametric expressions, but as you said, it is probably not worth.

If I misunderstood the lifetime in the parametric expression case, please enlight me.

Cheers,
Florian
It is loading more messages.
0 new messages