Static analysis and the future of C++

[mihailn...@gmail.com]

Hello,

In recent years there is a boom of static analysis - from new languages like Rust and Swift to the powerful tools, provided by C++ compilers.

However I don't see it C++ language mandating any of these. I have a question - is this something the committee is looking into? Is there a working group for this? 

If a tool is a lifesaver, why is it not mandatory - who does not what to save lifes?

I strongly believe bringing more (any?) static analysis into the language is away to keep it modern and relevant, especially, considering C++ very static language and has a lot of compile-time information the tools can work with.

Any answers and thoughts are welcome.

Now on a concrete idea - to use static analysis to make *_view classes safe(er).

Here is a suggestion:

struct A_view

{

  A_view( [[dependent_lifetime]]  A&& arg) : _a(&arg) {}

  A* _a;

};

Now, given the code above using A_view, after ~A() is called on the variable arg points to. will be marked as an error:

A_view ref(A{}); //< OK, no use of ref

A_view (A{}).func(); //< OK , A outlives ref

A_view ref(A{});

ref.func(); //< error/warning, ref used after A is destroyed

I believed this can be enforced by any compiler, as long as these limitations apply.

1) A_view is not heap allocated 

2) We forbid all uses of ref, not just uses involving _a;

These two limitations are there to avoid tracking lifetime through pointers.

Lifetime contract is exclusively b/w ref and *(&arg). 

Considering the compiler is responsible to destroy both of these, it should be possible to give a error/warning if variable ref is used after variable *(&arg) is destroyed.

Copies of ref do not contribute to lifetime tracking - assumed is the view outlives the viewed the same way it is assumed when the view is created from a non-rvalue.

Comments are more then welcome, both on the broader topic about mandating static analysis and the concrete proposal.

Thanks 

Mihail Naydenov

[j c]

This discussion will be pointless until C++ decides what it should do when built-in static analysers discover undefined behaviour.

Right now they use it as an excuse to generate heavily optimised, but ultimately incorrect, programs instead of just bailing out of the compile.

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/0087c1a4-0b36-40ca-8d43-5bfaf0af62ca%40isocpp.org.

[Andrey Semashev]

On 01/14/18 16:40, mihailn...@gmail.com wrote:
> Hello,
> In recent years there is a boom of static analysis - from new languages
> like Rust and Swift to the powerful tools, provided by C++ compilers.
>

> However I don't see it C++ language /mandating/ any of these. I have a

> question - is this something the committee is looking into? Is there a
> working group for this?
> If a tool is a lifesaver, why is it not mandatory - who does not what to
> save lifes?
>
> I strongly believe bringing more (any?) static analysis into the
> language is away to keep it modern and relevant, especially, considering
> C++ very static language and has a lot of compile-time information the
> tools can work with.
>
> Any answers and thoughts are welcome.

How does *mandating* a particular tool help language users? If you can
use a tool right now, why do you want to enforce it on everyone?

Also, what should we do about the platforms where such tools are not
available?

[Michał Dominiak]

On Sun, Jan 14, 2018 at 4:39 PM j c <james.a...@gmail.com> wrote:

This discussion will be pointless until C++ decides what it should do when built-in static analysers discover undefined behaviour.

Right now they use it as an excuse to generate heavily optimised, but ultimately incorrect, programs instead of just bailing out of the compile.

If you find that your implementation generates an incorrect program out of your code, then file a bug against that implementation.

Of course it is meaningless to speak about program correctness when your program violates the rules given by the specification, which includes actually executing through undefined behavior.

To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/0087c1a4-0b36-40ca-8d43-5bfaf0af62ca%40isocpp.org.

--

You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.

To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAFQaeCDQyFB8zoRoBzBJsjQGUi2wJVsW8Yn5LRgzAyc0abLiyQ%40mail.gmail.com.

[Nicol Bolas]

On Sunday, January 14, 2018 at 8:40:26 AM UTC-5, mihailn...@gmail.com wrote:

Hello,

In recent years there is a boom of static analysis - from new languages like Rust and Swift to the powerful tools, provided by C++ compilers.

However I don't see it C++ language mandating any of these.

... why would it?

The standard defines behavior. The standard decides what is valid language and how it behaves.

Static analysis is for looking at what is valid language and deciding whether or not it's likely that the behavior your code gets is the behavior you actually wanted. If you make a change to the language, you aren't "using static analysis"; you've made something linguistically impossible.

It's like the difference between saying "we shouldn't swear" vs. removing swear words from the English language entirely.

Whether compilers include static analysis tools, and how they are used, is out-of-bounds for the standard. Though you could think of contracts as a mechanism to make static analysis easier, that's more a useful additional thing that contracts do.

I have a question - is this something the committee is looking into? Is there a working group for this? 

If a tool is a lifesaver, why is it not mandatory - who does not what to save lifes?

I strongly believe bringing more (any?) static analysis into the language is away to keep it modern and relevant, especially, considering C++ very static language and has a lot of compile-time information the tools can work with.

Any answers and thoughts are welcome.

Now on a concrete idea - to use static analysis to make *_view classes safe(er).

This is the wrong tool for the job. The ideal solution is to make such code ill formed, not to rely on "static analysis" to make compilers give a warning or something.

Or better yet, to change the language to make such code functional. After all, if you're going to go through the effort of tagging a function with this attribute, why not tag it with a keyword that extends the lifetime of the temporary correctly?

A_view(extend A&& arg) : _a(&arg) {}

...

A_view ref(A{}); //A{}'s lifetime is extended to that of the declaration's enclosing scope.

[Jake Arkinstall]

I don't think the idea is to force users to do it, but to force compilers to do it. It's essentially the same as deprecating various forms of undefined behaviour, which in turn forces compilers to become smarter.

My position on this is constantly evolving. On one hand it's nice to have a language that doesn't allow developers to screw up. On the other hand, it's also nice to allow users to have more control (which is why std::unreachable is favourable to me, so that people only get more control when they ask for it).

I do not believe it is reasonable to *demand* compilers check for such things simply because it isn't currently reasonable to demand users to do such things. I do, however, think it'd be very useful for every teacher of C++ (From tutorials to books to actual teachers) to explicitly teach people what tools are available to them. When I first started out with C++, I made so many dumb mistakes that took a long time to figure out. Sure that process made me a better developer, but it would have been a lot easier had my toolset been better than valgrind and gdb. Now we have all of these shiny sanitisers, we should mandate that new users have them drilled into their heads. In fact, I'd go as far as saying we should probably have a set of standards for C++ teaching material (I'm not sure if that's a thing).

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.

To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/3d1ee12f-b97a-f259-b98a-831153b4245e%40gmail.com.

[mihailn...@gmail.com]

On Sunday, January 14, 2018 at 5:51:04 PM UTC+2, Andrey Semashev wrote:

...

How does *mandating* a particular tool help language users? If you can
use a tool right now, why do you want to enforce it on everyone?

Because enforcing it creates better programs for everyone. Not enforcing it leaves it to seasoned developers only - people going to cppcon etc.

Of course it will not be a wildcard, just focused uses of it. 

 

Also, what should we do about the platforms where such tools are not
available?

They will have worse (less) warning messages. Of course it is up to the standard and compiler writers to decide. The bar can be set very low in terms of complexity.

The point is to have something, which is better then nothing.

 

[Richard Hodges]

> This discussion will be pointless until C++ decides what it should do when built-in static analysers discover undefined behaviour.

It's obvious to anyone but the isocpp committee what it should do. It should fail to compile and issue a meaningful diagnostic, whenever possible, for any and all UB unless that UB has been specifically opted-in through pragmas and/or attributes.

Any other course of action is indefensible.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAFQaeCDQyFB8zoRoBzBJsjQGUi2wJVsW8Yn5LRgzAyc0abLiyQ%40mail.gmail.com.

[mihailn...@gmail.com]

On Sunday, January 14, 2018 at 6:03:45 PM UTC+2, Nicol Bolas wrote:

...

Static analysis is for looking at what is valid language and deciding whether or not it's likely that the behavior your code gets is the behavior you actually wanted.

Isn't that precisely the example I gave? A valid language, not doing what one wanted? 

Only difference, the standard should say the implementation should warn the user if this and that.

 

If you make a change to the language, you aren't "using static analysis"; you've made something linguistically impossible.

It's like the difference between saying "we shouldn't swear" vs. removing swear words from the English language entirely.

Whether compilers include static analysis tools, and how they are used, is out-of-bounds for the standard. Though you could think of contracts as a mechanism to make static analysis easier, that's more a useful additional thing that contracts do.

I have a question - is this something the committee is looking into? Is there a working group for this? 

If a tool is a lifesaver, why is it not mandatory - who does not what to save lifes?

I strongly believe bringing more (any?) static analysis into the language is away to keep it modern and relevant, especially, considering C++ very static language and has a lot of compile-time information the tools can work with.

Any answers and thoughts are welcome.

Now on a concrete idea - to use static analysis to make *_view classes safe(er).

This is the wrong tool for the job. The ideal solution is to make such code ill formed, not to rely on "static analysis" to make compilers give a warning or something.

Or better yet, to change the language to make such code functional. After all, if you're going to go through the effort of tagging a function with this attribute, why not tag it with a keyword that extends the lifetime of the temporary correctly?

A_view(extend A&& arg) : _a(&arg) {}

...

A_view ref(A{}); //A{}'s lifetime is extended to that of the declaration's enclosing scope.

This is infinitely more complex to implement however, also makes semantic change to the language (hence it can not be attribute). 

But more impartially, what do you rally need it?

Do you really need the lifetime extended, when one could just decl A before view? We are worried about errors and shots in the food, I doubt we need the extra functionality, considering the complexity. 

(I might be wrong, I also considered [[extends_lifetime]] but discarded because the above reasons)

[Nicol Bolas]

On Sunday, January 14, 2018 at 11:34:42 AM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 6:03:45 PM UTC+2, Nicol Bolas wrote:

...

Static analysis is for looking at what is valid language and deciding whether or not it's likely that the behavior your code gets is the behavior you actually wanted.

Isn't that precisely the example I gave? A valid language, not doing what one wanted? 

Only difference, the standard should say the implementation should warn the user if this and that.

And it's that "difference" that I'm saying shouldn't happen. If you want a static analyzer to find bugs, that's great. But the standard should not require it, nor should the standard be getting in the way of that process.

The standard should be about the actual language.

If you make a change to the language, you aren't "using static analysis"; you've made something linguistically impossible.

It's like the difference between saying "we shouldn't swear" vs. removing swear words from the English language entirely.

Whether compilers include static analysis tools, and how they are used, is out-of-bounds for the standard. Though you could think of contracts as a mechanism to make static analysis easier, that's more a useful additional thing that contracts do.

I have a question - is this something the committee is looking into? Is there a working group for this? 

If a tool is a lifesaver, why is it not mandatory - who does not what to save lifes?

I strongly believe bringing more (any?) static analysis into the language is away to keep it modern and relevant, especially, considering C++ very static language and has a lot of compile-time information the tools can work with.

Any answers and thoughts are welcome.

Now on a concrete idea - to use static analysis to make *_view classes safe(er).

This is the wrong tool for the job. The ideal solution is to make such code ill formed, not to rely on "static analysis" to make compilers give a warning or something.

Or better yet, to change the language to make such code functional. After all, if you're going to go through the effort of tagging a function with this attribute, why not tag it with a keyword that extends the lifetime of the temporary correctly?

A_view(extend A&& arg) : _a(&arg) {}

...

A_view ref(A{}); //A{}'s lifetime is extended to that of the declaration's enclosing scope.

This is infinitely more complex to implement however, also makes semantic change to the language (hence it can not be attribute). 

But more impartially, what do you rally need it?

Do you really need the lifetime extended, when one could just decl A before view? We are worried about errors and shots in the food, I doubt we need the extra functionality, considering the complexity.

You're thinking about it backwards. If people frequently keep writing code like that, then on some level, they want to write it that way. It's natural for them to. So you can either slap them in the face and make them stop, or you can just make it legal and make it do what they expect.

Yes, making it legal properly is infinitely harder than just slapping an attribute on it and calling it a day. But why should C++ keep taking the easy path to "fixing" its problems?

[mihailn...@gmail.com]

On Sunday, January 14, 2018 at 9:05:44 PM UTC+2, Nicol Bolas wrote:

On Sunday, January 14, 2018 at 11:34:42 AM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 6:03:45 PM UTC+2, Nicol Bolas wrote:

...

...

And it's that "difference" that I'm saying shouldn't happen. If you want a static analyzer to find bugs, that's great. But the standard should not require it, nor should the standard be getting in the way of that process.

The standard should be about the actual language.

Yeah, but things like attributes are not about the language. And that is OK. If there is an attribute which will help you enforce correct code, why not have it? 

 

...

You're thinking about it backwards. If people frequently keep writing code like that, then on some level, they want to write it that way. It's natural for them to. So you can either slap them in the face and make them stop, or you can just make it legal and make it do what they expect.

Yes, making it legal properly is infinitely harder than just slapping an attribute on it and calling it a day. But why should C++ keep taking the easy path to "fixing" its problems?

I don't find the possibility of adding lifetime extensions realistic.  I don't mind it, but I really, really doubt it will ever be added. Is there any work on in that direction? Have ever been any work on that?

Not that it is without problems - we break our own rules adding that. Yea, const ref already breaks it, but still. Even semantically is not very accurate - a view is 'slave' to the viewed, should not control it.

Also, what will happen with heap allocated views? Will they fail to compile and be illegal? 

But as I said, I will not fight against lifetime extensions, I just don't see them coming. Ever.

Where with said attribute we can considerably improve the quality of the user code, making the foot gun very loud.

Actually I will not be surprised if implementations start doing that by themselves (warning in cases thy can verify). The question is, why the initiative should always come from them?

[Nicol Bolas]

On Sunday, January 14, 2018 at 3:44:10 PM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 9:05:44 PM UTC+2, Nicol Bolas wrote:

On Sunday, January 14, 2018 at 11:34:42 AM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 6:03:45 PM UTC+2, Nicol Bolas wrote:

...

...

And it's that "difference" that I'm saying shouldn't happen. If you want a static analyzer to find bugs, that's great. But the standard should not require it, nor should the standard be getting in the way of that process.

The standard should be about the actual language.

Yeah, but things like attributes are not about the language. And that is OK. If there is an attribute which will help you enforce correct code, why not have it? 

If attributes aren't "about the language", then why are they defined by the language? And how do you explain P0840?

...

You're thinking about it backwards. If people frequently keep writing code like that, then on some level, they want to write it that way. It's natural for them to. So you can either slap them in the face and make them stop, or you can just make it legal and make it do what they expect.

Yes, making it legal properly is infinitely harder than just slapping an attribute on it and calling it a day. But why should C++ keep taking the easy path to "fixing" its problems?

I don't find the possibility of adding lifetime extensions realistic.  I don't mind it, but I really, really doubt it will ever be added. Is there any work on in that direction? Have ever been any work on that?

Yes there has. But apparently, EWG didn't consider the cost of consistent use of such annotations to be worth the fixing of this problem.

Not that it is without problems - we break our own rules adding that. Yea, const ref already breaks it, but still. Even semantically is not very accurate - a view is 'slave' to the viewed, should not control it.

Also, what will happen with heap allocated views? Will they fail to compile and be illegal? 

But as I said, I will not fight against lifetime extensions, I just don't see them coming. Ever.

Where with said attribute we can considerably improve the quality of the user code, making the foot gun very loud.

 

Except where it doesn't, of course:

auto ptr = make_shared<A_view>(A{});

After a discussion about the "extend me" proposal idea, I investigated the issues with an annotation-based approach. I concluded that the only way to make it work would be to add the equivalent of a third reference type (whether you spell it with `&&&` or some keyword or attribute associated with a reference parameter, the effect is to have a thing which behaves much like a third kind of reference). Which means you'd have to make forwarding references work with such a tool... somehow.

Your attribute-based annotation falls into the same trap: once you leave the immediate scope of the code creating the prvalue, it stops being a prvalue. So unless the immediate scope knows that the parameter is going to be used like this, it can't be annotated properly.

Ultimately, I've come to believe that the only reasonable solution is something like `extend_me`: something which explicitly extends the lifetime of a temporary, which is the responsibility of the writer of the prvalue expression to properly use.

Actually I will not be surprised if implementations start doing that by themselves (warning in cases thy can verify). The question is, why the initiative should always come from them?

Because the interaction between two rules of the standard makes this legal code behave in a way that is unexpected but clearly explicitly specified.

[mihailn...@gmail.com]

On Monday, January 15, 2018 at 1:54:32 AM UTC+2, Nicol Bolas wrote:

On Sunday, January 14, 2018 at 3:44:10 PM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 9:05:44 PM UTC+2, Nicol Bolas wrote:

On Sunday, January 14, 2018 at 11:34:42 AM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 6:03:45 PM UTC+2, Nicol Bolas wrote:

...

...

And it's that "difference" that I'm saying shouldn't happen. If you want a static analyzer to find bugs, that's great. But the standard should not require it, nor should the standard be getting in the way of that process.

The standard should be about the actual language.

Yeah, but things like attributes are not about the language. And that is OK. If there is an attribute which will help you enforce correct code, why not have it? 

If attributes aren't "about the language", then why are they defined by the language? And how do you explain P0840?

...

You're thinking about it backwards. If people frequently keep writing code like that, then on some level, they want to write it that way. It's natural for them to. So you can either slap them in the face and make them stop, or you can just make it legal and make it do what they expect.

Yes, making it legal properly is infinitely harder than just slapping an attribute on it and calling it a day. But why should C++ keep taking the easy path to "fixing" its problems?

I don't find the possibility of adding lifetime extensions realistic.  I don't mind it, but I really, really doubt it will ever be added. Is there any work on in that direction? Have ever been any work on that?

Yes there has. But apparently, EWG didn't consider the cost of consistent use of such annotations to be worth the fixing of this problem.

Not that it is without problems - we break our own rules adding that. Yea, const ref already breaks it, but still. Even semantically is not very accurate - a view is 'slave' to the viewed, should not control it.

Also, what will happen with heap allocated views? Will they fail to compile and be illegal? 

But as I said, I will not fight against lifetime extensions, I just don't see them coming. Ever.

Where with said attribute we can considerably improve the quality of the user code, making the foot gun very loud.

 

Except where it doesn't, of course:

auto ptr = make_shared<A_view>(A{});

You are not fair in your example. I was very specific - the attr will be ignored for heap allocated views (which are not that widely used, if at all). In any case an attribute can be ignored - it is just a hint - it is whole different story if its an language extension.

However, even for heap allocated views it will be possible to give a warning if view is used after A is destroyed, where it is practically impossible to extend the lifetime of A!

 

 

After a discussion about the "extend me" proposal idea, I investigated the issues with an annotation-based approach. I concluded that the only way to make it work would be to add the equivalent of a third reference type (whether you spell it with `&&&` or some keyword or attribute associated with a reference parameter, the effect is to have a thing which behaves much like a third kind of reference). Which means you'd have to make forwarding references work with such a tool... somehow.

Your attribute-based annotation falls into the same trap: once you leave the immediate scope of the code creating the prvalue, it stops being a prvalue. So unless the immediate scope knows that the parameter is going to be used like this, it can't be annotated properly.

You have to give me an example, I am not sure I understand the issue.

My logic is very, very simple - as long as we have automatic variables (aka predictable destruction) the compiler knows everything needed to track improper use, given our specification of improper use.

Temporaries, prvalues these have semantic value to us, but in the end - it is the compiler just rearranging the calls the dtors. All is visible to him - he created it.

If we, in this predictable for the compiler world, want a variable marked so we are informed it is used before another variable from this same predictable world, I am pretty sure the compiler can do this for us.

The point is, not want can't be done (the cases which will not be possible), but what can be done. Covering the trivial cases alone is not only better then nothing, but will actually amount to 95% of the cases needed to be covered in the first place.

 

Ultimately, I've come to believe that the only reasonable solution is something like `extend_me`: something which explicitly extends the lifetime of a temporary, which is the responsibility of the writer of the prvalue expression to properly use.

Actually I will not be surprised if implementations start doing that by themselves (warning in cases thy can verify). The question is, why the initiative should always come from them?

Because the interaction between two rules of the standard makes this legal code behave in a way that is unexpected but clearly explicitly specified.

Does not change the fact the standard not mandating warnings, it is the implementation which must take the initiative - warning for assignment in if statement, warning for no return, for fallthrough, for empty statements, etc etc. These save lifes for pros and juniors alike. 

Ironically all the standard did is a way to turn these warnings off if they do not match the intend. However, how about the standard turning warnings on (in cases impl. agree are implementable).

Granted [[deprecated]] is such an example but, that is way too little, compared to what it is possible.

Sure it is better to make this code illegal, but what if it is not possible or too hard? As the saying (in my language) goes - [It is] better [to have a]sparrow in the hand, then an eagle in the sky.

[Richard Hodges]

> The point is, not want can't be done (the cases which will not be possible), but what can

 be done. Covering the trivial cases alone is not only better then nothing, but will actually amount to 95% of the cases needed to be covered in the first place.

Absolutely, unequivocally, this ^^

It is a travesty and a failure of the language specification that UB is allowed into programs accidentally when it is perfectly possible to detect it at compile time.

It makes a mockery of any claim of "compile-time safety" which we constantly strive for with template constructs such as variant.

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/9ec7a895-af7b-43d3-b47a-b7c95ed8660b%40isocpp.org.

[Nicol Bolas]

On Monday, January 15, 2018 at 4:27:27 AM UTC-5, mihailn...@gmail.com wrote:

On Monday, January 15, 2018 at 1:54:32 AM UTC+2, Nicol Bolas wrote:

On Sunday, January 14, 2018 at 3:44:10 PM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 9:05:44 PM UTC+2, Nicol Bolas wrote:

On Sunday, January 14, 2018 at 11:34:42 AM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 14, 2018 at 6:03:45 PM UTC+2, Nicol Bolas wrote:

...

...

And it's that "difference" that I'm saying shouldn't happen. If you want a static analyzer to find bugs, that's great. But the standard should not require it, nor should the standard be getting in the way of that process.

The standard should be about the actual language.

Yeah, but things like attributes are not about the language. And that is OK. If there is an attribute which will help you enforce correct code, why not have it? 

If attributes aren't "about the language", then why are they defined by the language? And how do you explain P0840?

...

You're thinking about it backwards. If people frequently keep writing code like that, then on some level, they want to write it that way. It's natural for them to. So you can either slap them in the face and make them stop, or you can just make it legal and make it do what they expect.

Yes, making it legal properly is infinitely harder than just slapping an attribute on it and calling it a day. But why should C++ keep taking the easy path to "fixing" its problems?

I don't find the possibility of adding lifetime extensions realistic.  I don't mind it, but I really, really doubt it will ever be added. Is there any work on in that direction? Have ever been any work on that?

Yes there has. But apparently, EWG didn't consider the cost of consistent use of such annotations to be worth the fixing of this problem.

Not that it is without problems - we break our own rules adding that. Yea, const ref already breaks it, but still. Even semantically is not very accurate - a view is 'slave' to the viewed, should not control it.

Also, what will happen with heap allocated views? Will they fail to compile and be illegal? 

But as I said, I will not fight against lifetime extensions, I just don't see them coming. Ever.

Where with said attribute we can considerably improve the quality of the user code, making the foot gun very loud.

 

Except where it doesn't, of course:

auto ptr = make_shared<A_view>(A{});

You are not fair in your example. I was very specific - the attr will be ignored for heap allocated views (which are not that widely used, if at all). In any case an attribute can be ignored - it is just a hint - it is whole different story if its an language extension.

The heap allocation is irrelevant; the problem is indirect initialization of the object. Consider this:

template<typename T, typename ...Args>
auto initialize(Args ...&&args)
{
  return T(std::forward<Args>(args)...);
}

auto var = initialize<A_view>(A{});

Or even simpler:

optional<A_view> ovar(in_place, A{});

Any instance where you initialize `A_view` via forwarding will not properly detect that you're using a temporary, since the forwarding function cannot tell the difference between an xvalue and a prvalue.

That's ultimately why I concluded that annotation-based solutions can't work, that you have to create a "third reference" that preserves the prvalue-ness of the argument. That is, if you want to correctly deal with lifetime issues, you need to be able to tell the difference, not between lvalues and rvalues, but between lvalues, xvalues, and prvalues.

And technically, you need a sixth value category too. One that represents the difference between `prvalue.member`, which is not technically a prvalue but can extend the temporary's lifetime, and a function call on a prvalue that returns an xvalue subobject of the prvalue, which doesn't extend its lifetime.

However, even for heap allocated views it will be possible to give a warning if view is used after A is destroyed, where it is practically impossible to extend the lifetime of A!

 

 

After a discussion about the "extend me" proposal idea, I investigated the issues with an annotation-based approach. I concluded that the only way to make it work would be to add the equivalent of a third reference type (whether you spell it with `&&&` or some keyword or attribute associated with a reference parameter, the effect is to have a thing which behaves much like a third kind of reference). Which means you'd have to make forwarding references work with such a tool... somehow.

Your attribute-based annotation falls into the same trap: once you leave the immediate scope of the code creating the prvalue, it stops being a prvalue. So unless the immediate scope knows that the parameter is going to be used like this, it can't be annotated properly.

You have to give me an example, I am not sure I understand the issue.

My logic is very, very simple - as long as we have automatic variables (aka predictable destruction) the compiler knows everything needed to track improper use, given our specification of improper use.

 

See above. The compiler cannot just look at the prvalue argument and an attribute on the parameter to tell what's going on. `A_view` is not being fed a prvalue; it's being fed an xvalue. And by all rights, that ought to be fine.

Temporaries, prvalues these have semantic value to us, but in the end - it is the compiler just rearranging the calls the dtors. All is visible to him - he created it.

If we, in this predictable for the compiler world, want a variable marked so we are informed it is used before another variable from this same predictable world, I am pretty sure the compiler can do this for us.

The point is, not want can't be done (the cases which will not be possible), but what can be done. Covering the trivial cases alone is not only better then nothing, but will actually amount to 95% of the cases needed to be covered in the first place.

No, half-measures are actually worse than doing nothing. Why?

Because right now, we can teach people "be careful with view types and temporaries". We can teach people about lifetime issues and so forth. Maybe not enough programmers get this lesson, but the information is still out there.

But if you give people the illusion that the compiler can handle this issue, that they'll get a compile error or whatever if they improperly use views and temporaries, then people will use view types and temporaries with abandon. They won't realize that the compiler can't catch all of these kinds of problems. Without the training to avoid this problem, they're more likely to run into it in indirect initialization cases and so forth, because they have faith the compiler will tell them when they're doing it wrong.

If you can't solve this problem, better to do nothing at all. At least then, we'll always be on our toes. Better that than to become complacent.

Ultimately, I've come to believe that the only reasonable solution is something like `extend_me`: something which explicitly extends the lifetime of a temporary, which is the responsibility of the writer of the prvalue expression to properly use.

Actually I will not be surprised if implementations start doing that by themselves (warning in cases thy can verify). The question is, why the initiative should always come from them?

Because the interaction between two rules of the standard makes this legal code behave in a way that is unexpected but clearly explicitly specified.

Does not change the fact the standard not mandating warnings, it is the implementation which must take the initiative - warning for assignment in if statement, warning for no return, for fallthrough, for empty statements, etc etc. These save lifes for pros and juniors alike. 

 

Ironically all the standard did is a way to turn these warnings off if they do not match the intend. However, how about the standard turning warnings on (in cases impl. agree are implementable).

The standard defines behavior. Warnings are not behavior. Thus, it has no right to mandate warnings.

[mihailn...@gmail.com]

On Monday, January 15, 2018 at 7:48:04 PM UTC+2, Nicol Bolas wrote:

On Monday, January 15, 2018 at 4:27:27 AM UTC-5, mihailn...@gmail.com wrote:

...

You are not fair in your example. I was very specific - the attr will be ignored for heap allocated views (which are not that widely used, if at all). In any case an attribute can be ignored - it is just a hint - it is whole different story if its an language extension.

The heap allocation is irrelevant; the problem is indirect initialization of the object. Consider this:

template<typename T, typename ...Args>
auto initialize(Args ...&&args)
{
  return T(std::forward<Args>(args)...);
}

auto var = initialize<A_view>(A{});

Or even simpler:

optional<A_view> ovar(in_place, A{});

Any instance where you initialize `A_view` via forwarding will not properly detect that you're using a temporary, since the forwarding function cannot tell the difference between an xvalue and a prvalue.

That's ultimately why I concluded that annotation-based solutions can't work, that you have to create a "third reference" that preserves the prvalue-ness of the argument. That is, if you want to correctly deal with lifetime issues, you need to be able to tell the difference, not between lvalues and rvalues, but between lvalues, xvalues, and prvalues.

And technically, you need a sixth value category too. One that represents the difference between `prvalue.member`, which is not technically a prvalue but can extend the temporary's lifetime, and a function call on a prvalue that returns an xvalue subobject of the prvalue, which doesn't extend its lifetime.

However, even for heap allocated views it will be possible to give a warning if view is used after A is destroyed, where it is practically impossible to extend the lifetime of A!

 ...

You have to give me an example, I am not sure I understand the issue.

My logic is very, very simple - as long as we have automatic variables (aka predictable destruction) the compiler knows everything needed to track improper use, given our specification of improper use.

 

See above. The compiler cannot just look at the prvalue argument and an attribute on the parameter to tell what's going on. `A_view` is not being fed a prvalue; it's being fed an xvalue. And by all rights, that ought to be fine.

Oh, I see. However, does it matter? Consider:

A a;

A_view ref(std::move(a));

Sure, 'a' is not a temporary, but the same rules apply and still the compiler has all the information it needs.

In the extend case the x/prvalue might have been important, but now they are not.

The only thing that is important is to have visibility over the dtors, to mark the correct variable, and iff ~A is called first, the compiler will check for uses of ref b/w the calls of the two dtors. 

 

Temporaries, prvalues these have semantic value to us, but in the end - it is the compiler just rearranging the calls the dtors. All is visible to him - he created it.

If we, in this predictable for the compiler world, want a variable marked so we are informed it is used before another variable from this same predictable world, I am pretty sure the compiler can do this for us.

The point is, not want can't be done (the cases which will not be possible), but what can be done. Covering the trivial cases alone is not only better then nothing, but will actually amount to 95% of the cases needed to be covered in the first place.

No, half-measures are actually worse than doing nothing. Why?

Because right now, we can teach people "be careful with view types and temporaries". We can teach people about lifetime issues and so forth. Maybe not enough programmers get this lesson, but the information is still out there.

But if you give people the illusion that the compiler can handle this issue, that they'll get a compile error or whatever if they improperly use views and temporaries, then people will use view types and temporaries with abandon. They won't realize that the compiler can't catch all of these kinds of problems. Without the training to avoid this problem, they're more likely to run into it in indirect initialization cases and so forth, because they have faith the compiler will tell them when they're doing it wrong.

If you can't solve this problem, better to do nothing at all. At least then, we'll always be on our toes. Better that than to become complacent.

If you can't put a fence - you put a sigh.

If you can't solve the problem you give a warning (as much as you can).

And also, warnings teach better then UB!

We can teach exactly as before, the same lessons, however we put some safety nets - for all of us. Everyone makes mistakes - I forget return statements at least few times a year.

 

Ultimately, I've come to believe that the only reasonable solution is something like `extend_me`: something which explicitly extends the lifetime of a temporary, which is the responsibility of the writer of the prvalue expression to properly use.

Actually I will not be surprised if implementations start doing that by themselves (warning in cases thy can verify). The question is, why the initiative should always come from them?

Because the interaction between two rules of the standard makes this legal code behave in a way that is unexpected but clearly explicitly specified.

Does not change the fact the standard not mandating warnings, it is the implementation which must take the initiative - warning for assignment in if statement, warning for no return, for fallthrough, for empty statements, etc etc. These save lifes for pros and juniors alike. 

 

Ironically all the standard did is a way to turn these warnings off if they do not match the intend. However, how about the standard turning warnings on (in cases impl. agree are implementable).

The standard defines behavior. Warnings are not behavior. Thus, it has no right to mandate warnings.

I don't know. If there is behavior, then there is incorrect behavior (though legal code). I don't see harm of mandating warnings on incorrect behavior, if possible and if it is not possible for the problem to be solved (make the incorrect illegal).

But we are treading off topic here, as I didn't suggest a wildcard solution.

[Matt Calabrese]

On Sun, Jan 14, 2018 at 10:39 AM, j c <james.a...@gmail.com> wrote:

This discussion will be pointless until C++ decides what it should do when built-in static analysers discover undefined behaviour.

Right now they use it as an excuse to generate heavily optimised, but ultimately incorrect, programs instead of just bailing out of the compile.

1) Any program with UB is broken. Optimizations are not what make them broken.

2) Plenty of compilers do diagnose forms of UB at compile-time in places where it is feasible to deduce or to reasonably assume that the code is intended to be reachable when in a problematic state (simple example that is often diagnosed in warnings is the lack of return statement for functions, though even that is necessarily noisy). Sure, implementations can always do more, but it's not as though developers are saying "here's UB that we can detect at compile-time and that we will intentionally avoid warning about, you know, just to annoy our users". That's a horrible misconception that people need to stop perpetuating.

3) Just because an optimization seemingly takes advantage of UB does not imply that potential execution leading to UB was detected or is even feasibly detectable at compile-time with the language as-is (assumptions that there are no data races, etc.).

[Thomas Köppe]

On a tangential note, the committee has formed a new study group, SG 15, on "Tooling", chaired by Titus Winters. The group is very new and hasn't yet produced any public papers or other kinds of output, but I expect that static analysis and automatic migration will be things in SG 15's scope.

[mihailn...@gmail.com]

For the sake of completeness, in cases when the viewed class creates the view, then the && overload should be marked as [[dependent_lifetime]]

class string
{
  ...
  [[dependent_lifetime]] operator string_view() && { return {c_str(), size()}; } //< (probably not the correct place to set the attribute, but you get the idea)
};

Now the returned value (the variable created from it) depends on the lifetime of *this and should not be used after *this is dtor is called.

Further investigation

Consider that case:

void f()
{
  string_view v;
 
 if(const auto str = getString(); str == "something")
 {
    v = str;
 }
 
 // 'v' is either empty or dangles!
}

The above mistake can happen even to an experienced programmer (if he is used to ref-counted strings for instance). People learning C++ and junior developers are basically guaranteed to fall in there at some point.

Here it is painfully obvious the compiler knows the right usage - he just keeps it shut! We should be able to make him speak.

BTW, that is an interesting case, as it is not about initialization, but assignment. However, does it matter? It is still clear the scope of tracking - from assignment tor dtor.

Even if we have a more involved case

void f( string_view& v)
{
 if(const auto str = getString(); str == "something")
 {
    v = str;
 }
}

int main()
{
 string_view& v;
 f(v);

 return 0;
}

The compiler still knows all that is needed, doesn't it? As long as the view is an automatic variable the compiler can always detected if, during its lifetime, it is used after the pointed-to dtor is called.

Isn't that always the case? Isn't that that case even if the pointed-to is heap allocated?

o_view v = o_p->getView();

The compiler can still tell if *o_p is destroyed before v is destroyed. Am I mistaken here?

Considering view classes are 99.9% of the time used as automatic variables, there is no good reason for them to be unsafe.

(Or maybe there is, I might be missing something)

And lets talk copies.

Nothing is actually preventing this tracking to work for copies of the view as well - the rules are exactly the same - if before view.~view() there is a call to object.~object() there should be no view,access().

How many views are, it does not matter at all as long as they are automatic.

It also does not matter if ~object() was called by the compiler (automatic), the user (delete p_object;) or by a ref counted shared_ptr!

With that in mind.

void f(view<B> vb) //< by copy
{
   destroy_b_unknowingly();

  vb.use(); //< warning, dtor of b was called already
}

Even further investigation

template<class T>
class unique_ptr
{
  ...
  [[dependent_lifetime]] T* get() { return _p; }
  ...
  T* _p;
};

Now the variable of type T* is a view to the unique_ptr. If ~unique_ptr() is called before the (automatic) T* variable, created/assigned by get, goes out of scope, it will be an error to use that variable.

But, lets take this to the next level:

template<class T>
class unique_ptr
{
  ...
  [[dependent_lifetime(*_p)]] T* get() { return _p; }
  ...
  T* _p;
};

Now it is an error to use T* after ~T(), and ~T(),  can come from any place - shared_ptr, manual delete, another, moved into unique_ptr. Everywhere.

And the rules are still the same - no use of this T* variable, after a ~T(), until this T* variable goes out of scope ("as if" it has a dtor called).

The relationship is still b/w an automatic variable and a dtor call.

Am I fooling myself here? Isn't that mighty useful? Isn't that all implementable today? What is the harm? It does not even writes off lifetime extension and what not - it just reinforces safe use of the language, the way this language is right now.
No new semantics, no new rules!

[Richard Hodges]

On the whole I completely agree with you.

However:

auto up1 = std::make_unique<int>(5);

auto p = up1.get();   // dependent...

auto up2 = std::move(up1);  // ...sort of

std::string_view should never have been implicitly convertible from std::string. Unhappily, the committee seems to favour encouraging dangerous coding practices at the moment. Consider:

std::string foo();

std::string_view x = foo();

x.anything(); // boom!

whereas:

std::string const& x = foo();  // perfectly safe

auto x = foo(); // again, safe

auto&& x = foo(); // yet again

IMHO proliferating reference types in the c++ standard library is a grave error which will haunt us for years.

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/6e78f3c4-267d-40c4-bcde-a026e90c3838%40isocpp.org.

[mihailn...@gmail.com]

On Thursday, January 18, 2018 at 4:40:58 PM UTC+2, Richard Hodges wrote:

On the whole I completely agree with you.

However:

auto up1 = std::make_unique<int>(5);

auto p = up1.get();   // dependent...

auto up2 = std::move(up1);  // ...sort of

This is completely defined as per second suggested declaration of get().

[[dependent_lifetime(*_p)]] T* get() { return _p; }

p, still depends on the lifetime of the object returned by the new int(5) call inside make_unique. 

All the fancy pointers and/or classes we delegate the lifetime of the int to, will not hide when the delete on that int is called. If this happens before p is out of scope and we try to use p, then we error out.

 

std::string_view should never have been implicitly convertible from std::string. Unhappily, the committee seems to favour encouraging dangerous coding practices at the moment.

If not with string it will happen with something else - spans, function_view etc - and most of the time it is expected to be able to just call the function the way you normally would. 

Also, making the conversion explicit buy us too little I think - users will need to create views explicitly and still shoot themselves by using temporal objects. 

 

Consider:

std::string foo();

std::string_view x = foo();

x.anything(); // boom!

whereas:

std::string const& x = foo();  // perfectly safe

auto x = foo(); // again, safe

auto&& x = foo(); // yet again

That is why I really hope some work is done to mandate warnings in a well defined manner, instead of waiting the implementations to fill the gap by they own initiative, like always. 

As said, more views are coming!

 

IMHO proliferating reference types in the c++ standard library is a grave error which will haunt us for years.

I personally like them, because of the abstraction - a base type am guaranteed to be able to convert to. It is really a form of abstract base class, but without all the hierarchy requirements and indirection.

Also, a thing like function_view will enable us to finally have functor pointers - cheap almost zero-cost pointers to anything callable, lambda included.

There is no alternative to that now - you either must use a template, or commit to a representation (and conversion) to something like std::function or my_function or whatever. 

 

To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.

[mihailn...@gmail.com]

I just realized this could make observer_ptr finally useful.

Once decorated it will actually observe the dtor/free of the original object is called and warn uses after that.

Observation will be compile-time static analysis, active only in debug mode and/or a compiler flag.

It will not work for heap allocated observers or observers members of heap allocated object and in case the original object is destroyed in a different thread. 

Still, it would be infinitely more useful then today, because it not only reinforces semantics, but help us write correct code.

If someone writes correct code, then it should be opt-out (the attribute should be in a namespace 'debug' or 'analyze'). No need to pay the compile time price if your code is right.

However verification should be "one click away" and targeted (no wide guess by the analyzer what "correct" is).

Now lets consider something interesting.

std::string str = "good";

std::string_view v = str;

str += "or not";

// bad things waiting to happen

That case is not solved neither by my original suggestion nor the life extension proposal!

Both are concerned by the lifetime of the holding object and not what the object holds, however views are most of the time about internal representation, not the object itself (function_view is major exception).

Views about the object themselves are pointers and references - we have them already.

Interestingly this case is trivially solved by a the improved observer_ptr.

class string_view

{

  ...

  void friendly_func() { /*does not use _p*/ }

  std::observer_ptr _p;

  ...

};

class string 

{

  operator string_view () const { return {c_str(), size()}; }

};

Note, string does not need attribute decorations any more!

Also note, now we can call view.friendly_func(), without a warning, even when the observed object is destroyed. (not sure how usefully this is, just an observation) 

Needless to say, the original example case should be now solved as after the addition the compiler/analyzer should detect that the original string must be relocated and observer_ptr will dangle.

Interestingly - Is there something besides observer_ptr actually needed?

Can we "just" define special behavior of observer_ptr (in debug/analyze mode) and solve all cases of lifetime monitoring (under the said limitations) - from smart pointers to function_view?

[Nicol Bolas]

On Friday, January 19, 2018 at 7:30:52 AM UTC-5, mihailn...@gmail.com wrote:

Can we "just" define special behavior of observer_ptr (in debug/analyze mode) and solve all cases of lifetime monitoring (under the said limitations) - from smart pointers to function_view?

That's not "behavior" of the type. There is no actual code in `observer_ptr` that does anything you're talking about. It's simply a compiler or tool that sees you using this particular type and starts looking at things because of it.

It should also be noted that this still doesn't handle perfect forwarding. Not unless it is the caller of the function who wraps the parameter in an `observer_ptr`.

[mihailn...@gmail.com]

On Friday, January 19, 2018 at 10:19:40 PM UTC+2, Nicol Bolas wrote:

On Friday, January 19, 2018 at 7:30:52 AM UTC-5, mihailn...@gmail.com wrote:

Can we "just" define special behavior of observer_ptr (in debug/analyze mode) and solve all cases of lifetime monitoring (under the said limitations) - from smart pointers to function_view?

That's not "behavior" of the type. There is no actual code in `observer_ptr` that does anything you're talking about. It's simply a compiler or tool that sees you using this particular type and starts looking at things because of it.

If the constructor and reset of observer_ptr are decorated with something like [[debug:observe_destruction]] the behavior of observer_ptr will be enforced by the compiler.

The behavior, use and definition of observer_ptr are not changed.  

 

It should also be noted that this still doesn't handle perfect forwarding. Not unless it is the caller of the function who wraps the parameter in an `observer_ptr`.

class A_view

{

  A_view(const A* pa) : _p(pa) {}

  A_view(const A& a) : _p(&a) {}

  A_view(A&& a) : _p(&a) {}

  observer_ptr<const A> _p;

}; 

observer_ptr will monitor addressof(*_p) for ~A() call. The scope of monitoring is ~observer_ptr() or release() call.

Copies of observer_ptr still monitor the same address, but with a different scope - their dtor, release or reset.

That is a major difference from lifetime extension - we can have many-to-one mapping without any semantic or implementation problems. Lifetime extension does not allow copies of a view/ref to extend with the lifetime of the object.

We are also not limited to initialization only - the monitoring can (re)start at the call of reset() instead on constructor call only. Lifetime extension deals with initialization exclusively. 

How can perfect forwarding interfere in all this? 

template<typename T, typename ...Args>

auto init(Args&&... arg)
{
  return T(std::forward<Args>(arg)...); //< observer_ptr starts monitoring &arg
}

int main()
{
  auto v = init<A_view>(A{}); //< both A_view and A allocated. A_view usable before the ';' when ~A is called
}

If copy elision does not kick-in nothing changes - copy is safe and defined, lifetime of A is the same.

If we pass arguments and create A in init() instead ( return  T(A(std::forward<Args>(arg)...)); ), then the observer_ptr is still created with a valid address,

however, after we return will not be able to use A_view in any way, including before the ';'. 

If both there is no elision and A is created when initializing T inside init, then the returned view can be created from a dangling view. 

We could allow views from dangling views to be created, however no dtor will be monitored and all uses of the view will be illegal until destroyed/reset/release-ed.

We could also issue a warning on coping from dangling view without problem.

 

[Richard Hodges]

> Copies of observer_ptr still monitor the same address, but with a different scope - their dtor, release or reset.

It would be nice if this were enough.

consider

auto& exec = some_async_executor();

auto up = std::make_unique<Foo>();

auto p = observer_ptr(up.get());

exec.post([p] { something_with(*p); });    // A

exec.post([up = std::move(up)] { something_with(*up); }); // B

exec.post([p] { something_with(*p); });   // C

Assume that B's lambda invocation may or may not complete before C's lambda invocation.

I can't see how this could be statically analysed. It's an undetectable logic error, since as far as the compiler is concerned, it has no way of knowing that C's p may outlive B's up.

Note that at the point of the construction of each of the 3 lambdas, the pointers captured are all valid (except in the crossing case where there is a context switch after the post at B).

--

You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.

To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/4fa792ec-1f51-4357-a65a-2ce3c10c60c0%40isocpp.org.

[mihailn...@gmail.com]

On Saturday, January 20, 2018 at 4:54:53 PM UTC+2, Richard Hodges wrote:

> Copies of observer_ptr still monitor the same address, but with a different scope - their dtor, release or reset.

It would be nice if this were enough.

consider

auto& exec = some_async_executor();

auto up = std::make_unique<Foo>();

auto p = observer_ptr(up.get());

exec.post([p] { something_with(*p); });    // A

exec.post([up = std::move(up)] { something_with(*up); }); // B

exec.post([p] { something_with(*p); });   // C

Assume that B's lambda invocation may or may not complete before C's lambda invocation.

I can't see how this could be statically analysed. It's an undetectable logic error, since as far as the compiler is concerned, it has no way of knowing that C's p may outlive B's up.

Note that at the point of the construction of each of the 3 lambdas, the pointers captured are all valid (except in the crossing case where there is a context switch after the post at B).

In any case this does heap allocation for the views (the lambdas, inside the exec class) and presumably threading after that. Both are impossible to track.

For both however we have nice tools - shared and week ptr. 

Which is ironic - the complex code is more safe then the trivial one. Yes you could use the heavyweight tools for simple tasks, but that is not zero overhead any more. 

Static analysis is the tool to help us with the common, simple, yet (terribly) unsafe tasks, still keeping zero overhead.

Having said that, there are scenarios where tracking will not give accurate prediction - the destruction might, for instance, depend upon some real time state.

That is the reason, these cannot be errors, only warnings.

The compiler should, however, be able to see all paths leading to destruction even the once behind real time check/uncertainty and warn that "it might dangle" or that "no one can be certain, even you!".

In your example, the compiler should warn it can't predict the observer - has no visibility over its lifetime and/or the observed destruction is behind real time uncertainty. 

He might even suggest alternative like share/weak ptr, because observe_ptr was not the right tool in the first place! The compiler will only reaffirm that fact. 

 

To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.

[Richard Hodges]

> In your example, the compiler should warn it can't predict the observer - has no visibility over its lifetime and/or the observed destruction is behind real time uncertainty. 

> He might even suggest alternative like share/weak ptr, because observe_ptr was not the right tool in the first place! The compiler will only reaffirm that fact. 

That would indeed be a useful feature.

To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.

To post to this group, send email to std-pr...@isocpp.org.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/00d91d4c-4ee7-4dbd-a547-3dd844f5685d%40isocpp.org.

[Nicol Bolas]

On Saturday, January 20, 2018 at 6:12:09 AM UTC-5, mihailn...@gmail.com wrote:

On Friday, January 19, 2018 at 10:19:40 PM UTC+2, Nicol Bolas wrote:

On Friday, January 19, 2018 at 7:30:52 AM UTC-5, mihailn...@gmail.com wrote:

Can we "just" define special behavior of observer_ptr (in debug/analyze mode) and solve all cases of lifetime monitoring (under the said limitations) - from smart pointers to function_view?

That's not "behavior" of the type. There is no actual code in `observer_ptr` that does anything you're talking about. It's simply a compiler or tool that sees you using this particular type and starts looking at things because of it.

If the constructor and reset of observer_ptr are decorated with something like [[debug:observe_destruction]] the behavior of observer_ptr will be enforced by the compiler.

The behavior, use and definition of observer_ptr are not changed.  

 

It should also be noted that this still doesn't handle perfect forwarding. Not unless it is the caller of the function who wraps the parameter in an `observer_ptr`.

class A_view

{

  A_view(const A* pa) : _p(pa) {}

  A_view(const A& a) : _p(&a) {}

  A_view(A&& a) : _p(&a) {}

  observer_ptr<const A> _p;

}; 

observer_ptr will monitor addressof(*_p) for ~A() call.

How would it know? If all the compiler in that translation unit sees is:

class A_view
{
public:
  A_view(const A &p);

private:
  observer_ptr<A> p_;
};

How could the compiler possibly know that this code is invalid:

A_view v(A{});

We're not talking about runtime detection here; this is purely compile-time. And the compiler doesn't necessarily see everything. If `A_view` is in another translation unit that isn't visible to static analysis, this doesn't work.

[mihailn...@gmail.com]

On Saturday, January 20, 2018 at 9:31:22 PM UTC+2, Nicol Bolas wrote:

...

How would it know? If all the compiler in that translation unit sees is:

class A_view
{
public:
  A_view(const A &p);

private:
  observer_ptr<A> p_;
};

How could the compiler possibly know that this code is invalid:

A_view v(A{});

We're not talking about runtime detection here; this is purely compile-time. And the compiler doesn't necessarily see everything. If `A_view` is in another translation unit that isn't visible to static analysis, this doesn't work.

You have to be more specific, might be my knowledge failing me, but what would prevent the compiler seeing in the case when are the dtros of both variables called?  

[Nicol Bolas]

In order for the compiler to be able to associate the prvalue temporary with `A_view`, it must be able to look at the code and see that `A_view` contains a pointer/reference to that temporary. While the compiler can see that `A_view`'s constructor is being given a pointer to a temporary, it does not know what that constructor is doing. It cannot associate the constructor parameter with `A_view::p_`, because there is no code in this translation unit that associates the constructor parameter with that member variable.

Yes, the compiler can see that two destructors are happening. But there is no evident association between these two objects. And without that knowledge, the compiler has no right to declare that this code is problematic.

This is why you need the annotation to be part of the declaration. Because the declaration may be the only thing the compiler will ever see.

[mihailn...@gmail.com]

I see, you mean that only the A_view ctor is visible as a single function call and nothing beyond that.

If that is the case, then both view (its ctor) and observer_ptr must be inlined. Luckily, that covers practically all views and observer_ptr is a template already.

Surely it would be a noticeable limitation, but will not make the feature useless. 

However, that does not have to be the case. I was assuming some sort of "entire program" analysis.

In that case the analyzer sees observer_ptr as part of the A_view declaration in the header, then it sees the decorated constructor of observer_ptr,

after that it goes and "recompiles" the A_view object file to see how the association is made.

As I said, I was assuming "entire program" analysis, because otherwise copies of views would be impossible to implement.

Not just copies, but code like this

void f(observer_ptr<P> pv)

{

  kill_p_by_accident(); //< some function using p, not in the TU

 // warn of using pv after potential kill

}

The quotes in "entire program" are important. The whole point of decorations is to have targeted analysis - real "entire program" analysis are too slow and too noisy.

If the analyzer have all the source, to all the translation units, and knows what to look for, then it should be able to do reasonably fast and reasonably accurate predictions.

I am not insisting of "entire program", though, just speculating what would be possible and useful!

I would be happy with anything, any scaled back version of this (or something similar), because now, out of the box, we have nothing.

And what we have, the external tools, have to do a ton of guess work simply because C++ deliberately ignores their existence.

 

 

[Nicol Bolas]

On Sunday, January 21, 2018 at 8:44:07 AM UTC-5, mihailn...@gmail.com wrote:

On Sunday, January 21, 2018 at 3:45:37 AM UTC+2, Nicol Bolas wrote:

On Saturday, January 20, 2018 at 3:25:14 PM UTC-5, mihailn...@gmail.com wrote:

On Saturday, January 20, 2018 at 9:31:22 PM UTC+2, Nicol Bolas wrote:

...

How would it know? If all the compiler in that translation unit sees is:

class A_view
{
public:
  A_view(const A &p);

private:
  observer_ptr<A> p_;
};

How could the compiler possibly know that this code is invalid:

A_view v(A{});

We're not talking about runtime detection here; this is purely compile-time. And the compiler doesn't necessarily see everything. If `A_view` is in another translation unit that isn't visible to static analysis, this doesn't work.

You have to be more specific, might be my knowledge failing me, but what would prevent the compiler seeing in the case when are the dtros of both variables called? 

In order for the compiler to be able to associate the prvalue temporary with `A_view`, it must be able to look at the code and see that `A_view` contains a pointer/reference to that temporary. While the compiler can see that `A_view`'s constructor is being given a pointer to a temporary, it does not know what that constructor is doing. It cannot associate the constructor parameter with `A_view::p_`, because there is no code in this translation unit that associates the constructor parameter with that member variable.

Yes, the compiler can see that two destructors are happening. But there is no evident association between these two objects. And without that knowledge, the compiler has no right to declare that this code is problematic.

This is why you need the annotation to be part of the declaration. Because the declaration may be the only thing the compiler will ever see.

I see, you mean that only the A_view ctor is visible as a single function call and nothing beyond that.

If that is the case, then both view (its ctor) and observer_ptr must be inlined. Luckily, that covers practically all views and observer_ptr is a template already.

Surely it would be a noticeable limitation, but will not make the feature useless. 

But that wishy-washy-ness is exactly what makes this not a feature of the standard. It's a compiler tool or a static analysis tool, not something the standard can deal with.

And quite frankly, if you have a static analysis tool that can do whole-program analysis, I'd bet that it could be smart enough to figure out that `observer_ptr` is not destroying the pointer it's given just by analyzing its member functions. So you don't even need that attribute to tell such a tool what's going on; just let it figure out which types are "observer" types.

[mihailn...@gmail.com]

Without decoration it will not know to look in that TU. Actually the other way around - TU will be ignored unless a type it exports (or uses) is decorated. Even if all units use an observer pointer, not all pointers or objects will be an observer or decorated.

Looking for specific pointers should be faster then looking for all pointers, among a bunch of other things as well. 

About wishy-washy-ness.

The implementers will say what it is feasible, much like it is today with other features. They can also say what is required to improve their work - what sort of decorations, where. 

Considering C++ already defined an interface to communicate with the implementations - the attributes - I don't see a reason not to use that interface to improve both the implementation and the language (as experience, not semantics).

But, OK, fine, do not define these in "The C++ standard", define them as part of "C++ Guidelines" - it is, after all, for the correct use of the language.

The point is, ignoring and/or not helping the tools is a game where everybody loses.