Is there really no path forward for definition checking of templates?

[Faisalv]

I just glanced through Matt Calabrese's interesting opinion/response paper on concepts lite ..

I have two questions (admittedly as someone far less seasoned at generic programming compared to Matt's credentials on the topic):
1) For Andrew: Is there really no clear path forward?
2) For all you seasoned generic programmers: is definition checking of templates important ?

Thanks!

[Bjarne Stroustrup]

On 2/18/2016 1:58 PM, Faisalv wrote:
> I just glanced through Matt Calabrese's interesting opinion/response paper on concepts lite ..
>
> I have two questions (admittedly as someone far less seasoned at generic programming compared to Matt's credentials on the topic):
> 1) For Andrew: Is there really no clear path forward?

I'm not Andrew, but there is a path forward. Gaby has done definition
checking based on predicated in his model language Liz. It works. There
is a snag, though, as several people have pointed out: if you do, you
can't do data collection, tracing, debugging using debug statement,
telemetry, etc. without interface statements.

We have been quite consistent about this for several years.

> 2) For all you seasoned generic programmers: is definition checking of templates important ?

It is important, but not essential and not without its own problems. It
is a huge benefit in its own right.

Matt's paper is making a lot of assumptions about the knowledge and
opinions about people wanting concepts now based on the current model.
Most seem flat wrong. He seems to have missed the last 5 years progress
in the standards committee, or maybe 10.

>
> Thanks!
>
>

[Andrew Sutton]

> I have two questions (admittedly as someone far less seasoned at generic programming compared to Matt's credentials on the topic):

> 1) For Andrew: Is there really no clear path forward?

See Bjarne's response. It's better.

> 2) For all you seasoned generic programmers: is definition checking of templates important ?

I don't miss it.

Andrew

[Ville Voutilainen]

On 18 February 2016 at 20:58, Faisalv <fai...@gmail.com> wrote:
> I just glanced through Matt Calabrese's interesting opinion/response paper on concepts lite ..
>
> I have two questions (admittedly as someone far less seasoned at generic programming compared to Matt's credentials on the topic):
> 1) For Andrew: Is there really no clear path forward?

There are many possible ways forward. Making current Concepts
automatically begin
checking definitions is not one of them, mostly because what Bjarne explained.

> 2) For all you seasoned generic programmers: is definition checking of templates important ?

It's far less important than the facilities in the current Concepts
TS, and contrary
to claims that it makes writing constrained templates easier, it can
make writing them
harder, MUCH harder. Again, mostly because what Bjarne explained, but
for other reasons
too - there are cases where invoking an unconstrained duck-typed template from
a constrained template is just fine for every specialization, but
C++0x concepts adamantly
forbade such uses.

Definition checking has the potential to divide templates into two
incompatible camps
of constrained and unconstrained templates. That may lead to all sorts
of problems,
see
http://open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2919.html

If we decide that definition checking must be solved before any sort
of concepts can
ship in an IS, we're talking next decade. If we decide that definition
checking is a mandatory
part of every constrained template, we'll never get any sort of concepts.

[Bjarne Stroustrup]

On 2/18/2016 2:34 PM, Ville Voutilainen wrote:
> On 18 February 2016 at 20:58, Faisalv <fai...@gmail.com> wrote:
>> I just glanced through Matt Calabrese's interesting opinion/response paper on concepts lite ..
>>
>> I have two questions (admittedly as someone far less seasoned at generic programming compared to Matt's credentials on the topic):
>> 1) For Andrew: Is there really no clear path forward?
> There are many possible ways forward. Making current Concepts
> automatically begin
> checking definitions is not one of them, mostly because what Bjarne explained.
>
>> 2) For all you seasoned generic programmers: is definition checking of templates important ?
> It's far less important than the facilities in the current Concepts
> TS, and contrary
> to claims that it makes writing constrained templates easier, it can
> make writing them
> harder, MUCH harder. Again, mostly because what Bjarne explained, but
> for other reasons
> too - there are cases where invoking an unconstrained duck-typed template from
> a constrained template is just fine for every specialization, but
> C++0x concepts adamantly
> forbade such uses.

With C++0x Concepts, we had two worlds checked/constrained and
traditional/unconstrained. The facilities for merging those two worlds
never meshed very well. And - especially if we check definitions -
conversion to a wholly constrained world could take a looong time.
Concepts "Lite" has a smooth integration; "typename" is simply the
weakest concept, so template metaprogramming still works as before. Add
concept maps, and we never figured out how to make the system completely
type safe.

I do not want to go back to C++0x concepts, but I might want definition
checking if we can solved the debugging and data collection problems
mentioned. We plan to explore (further).

>
> Definition checking has the potential to divide templates into two
> incompatible camps
> of constrained and unconstrained templates. That may lead to all sorts
> of problems,
> see
> http://open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2919.html
>
> If we decide that definition checking must be solved before any sort
> of concepts can
> ship in an IS, we're talking next decade. If we decide that definition
> checking is a mandatory
> part of every constrained template, we'll never get any sort of concepts.
>

Exactly! and if we don't insist on those, we are ready now.

[Herb Sutter]

> I do not want to go back to C++0x concepts, but I might want definition checking
> if we can solved the debugging and data collection problems mentioned. We
> plan to explore (further).

For reference, could someone please link to a reasonably current concise summary of definition checking and the debug/datacollection issues? Some examples would be a good refresher here.

Thanks,

Herb

[Jason McKesson]

Have you not just proved Matt's point? Namely, that if we get Concepts TS into C++17, we will never see definition checking. Just look at what you said.

You say that definition checking would require effort through to "next decade". It would require so much work that "we'll never get any sort of concepts". Well, the only reason we would "never" get the feature is if everyone working on it decided that it wasn't worth the effort. After all, constraints is apparently very much worth a great deal of effort, considering the fact that we're on our second attempt. But if adding checking onto that requires so much more effort that even getting the rest of constraints doesn't make it worth doing, then... what's the chance that people will ever see checking alone as being worth the effort?

Matt is very concerned that, if Concepts TS goes into C++17, then in 2018, SG8 will dissolve and most of the committee will be decidedly uninterested in doing anything more with it. And some of the responses in this thread seem to bear this out. Andrew Sutton, one of the architects of Concepts TS, has already made it clear in his post that checking is optional in his mind, nothing that he'll miss. That doesn't exactly sound like he's chomping at the bit to start working on it.

I do not find that Matt's fear is unreasonable.

[Bjarne Stroustrup]

What I said was that Gaby has done experiments that show that we can do definition checking and that is isn't all that hard.

I'm much more worried that definition checking could become a straightjacket that prevented things we wanted: debug statements, data gathering, etc.

You say that definition checking would require effort through to "next decade". It would require so much work that "we'll never get any sort of concepts". Well, the only reason we would "never" get the feature is if everyone working on it decided that it wasn't worth the effort. After all, constraints is apparently very much worth a great deal of effort, considering the fact that we're on our second attempt. But if adding checking onto that requires so much more effort that even getting the rest of constraints doesn't make it worth doing, then... what's the chance that people will ever see checking alone as being worth the effort?

It isn't all that much effort, but it's not certain that we actually want it.

Matt is very concerned that, if Concepts TS goes into C++17, then in 2018, SG8 will dissolve and most of the committee will be decidedly uninterested in doing anything more with it. And some of the responses in this thread seem to bear this out. Andrew Sutton, one of the architects of Concepts TS, has already made it clear in his post that checking is optional in his mind, nothing that he'll miss. That doesn't exactly sound like he's chomping at the bit to start working on it.

I do not find that Matt's fear is unreasonable.

I have seen the argument lately that we shouldn't provide a improvement because we should wait for something much better. I don't believe in that argument. We need steady improvement.

If definition checking is such a big deal as Matt claims, we'll get it. If not, as Andrew notes, maybe not.

Could it be that Matt is stuck in the conceptual framework of C++0x concepts?

--
You received this message because you are subscribed to the Google Groups "SG8 - Concepts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concepts+u...@isocpp.org.
To post to this group, send email to conc...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/concepts/.

[Ville Voutilainen]

On 19 February 2016 at 03:57, Jason McKesson <jmck...@gmail.com> wrote:
>> If we decide that definition checking must be solved before any sort
>> of concepts can
>> ship in an IS, we're talking next decade. If we decide that definition
>> checking is a mandatory
>> part of every constrained template, we'll never get any sort of concepts.
> Have you not just proved Matt's point? Namely, that if we get Concepts TS

No.

> into C++17, we will never see definition checking. Just look at what you
> said.

The timing of definition checking is not a consequence of whether we do or do
not get Concepts into C++17-

> You say that definition checking would require effort through to "next
> decade". It would require so much work that "we'll never get any sort of
> concepts". Well, the only reason we would "never" get the feature is if

I didn't say that it requires so much work that we'll never get concepts.
I said that mandating it for every constrained template means we'll never
get concepts. The next decade is, surprise surprise, 2020 or later.

> everyone working on it decided that it wasn't worth the effort. After all,
> constraints is apparently very much worth a great deal of effort,
> considering the fact that we're on our second attempt. But if adding
> checking onto that requires so much more effort that even getting the rest
> of constraints doesn't make it worth doing, then... what's the chance that
> people will ever see checking alone as being worth the effort?

That chance is questionable, and has been for quite some time.

> Matt is very concerned that, if Concepts TS goes into C++17, then in 2018,
> SG8 will dissolve and most of the committee will be decidedly uninterested
> in doing anything more with it. And some of the responses in this thread
> seem to bear this out. Andrew Sutton, one of the architects of Concepts TS,
> has already made it clear in his post that checking is optional in his mind,
> nothing that he'll miss. That doesn't exactly sound like he's chomping at
> the bit to start working on it.
> I do not find that Matt's fear is unreasonable.

I don't suppose his fear is unreasonable, but I don't see that as motivation
to postpone Concepts.

[Andrew Sutton]

> Andrew Sutton, one of the architects of Concepts TS,
> has already made it clear in his post that checking is optional in his mind,
> nothing that he'll miss. That doesn't exactly sound like he's chomping at
> the bit to start working on it.

I'll work on the problem because I think it's interesting. That's
unrelated to whether or not I think it's essential for generic code.

Andrew

[Matt Calabrese]

On Thursday, February 18, 2016 at 11:05:07 AM UTC-8, Bjarne Stroustrup wrote:

I'm not Andrew, but there is a path forward. Gaby has done definition
checking based on predicated in his model language Liz. It works.

With all due respect, in order for definition-checking to really be meaningful and do what the programmer expects regarding C++0x concepts (and realistically for any language-level concepts approach), we'd need to make sure that the functions specified in the constraints really are the ones that are being called when in the body (i.e. not overloads). A subtle example of this is that when passing results of your associated functions that are mentioned in constraints which return "convertible to <type>..." to another associated function that expects "<type>", it better both succeed and pick up the function that is described by the constraint. Similarly, using something like && or || with a result that is merely convertible to bool.

In C++0x concepts (given that it actually solved this problem, I think it's reasonable to refer to), this was accomplished by altering lookup from within the constrained template and going through an indirection that exposed and preferred the associated functions of that name. In this way, users could write their constrained templates in a very simple manner and get the correct behavior -- note that this is exceedingly difficult in current C++ and a lot of generic code is incorrect because of it. If you do not do this or something similar, then definition checking cannot realistically deliver on the promise of preventing compile-time errors during instantiation -- or perhaps worse, you can end up with something that compiles and *looks* like it is using the associated functions but actually is not, whether because of ADL or other overloads that are in scope. Without constrained templates being instantiated in a way that is different from an unconstrained template (such as altering lookup), how are these problems supposed to be solved? If the answer really is to eventually alter lookup, then that's a huge breaking change that is unlikely to make it through, and even if it did, we're stuck with the likelihood that most standard and user-defined constraints or definitions were written incorrectly and so all of that code would break when we "turn the switch" (this is not an unreasonable assessment given that it was even true for the C++0x conceptified standard library when it was actually implemented -- the vast majority of the specified constraints were incorrect and the compiler thankfully caught that).

We can try to convince ourselves that the problem is simpler than it really is and claim that we'll manage to do definition checking without drastically changing things and/or breaking an unreasonable amount of code, but that's either an argument based out of ignorance (which I clearly don't think is true in your case, nor in the case of anyone more deeply involved with Concepts, but is true of laymen in the committee who are interested in Concepts), or otherwise it is unintentionally misleading. This is *not* a solved problem with respect to the Concepts TS and there is not a clear and direct route to a solution. This problem is *not* easy in a language like C++, regardless of the approach to concepts. It *is* hard and it *does* require consideration during the design of a constraint system in order to make the problem actually tractable.

On Thursday, February 18, 2016 at 11:05:07 AM UTC-8, Bjarne Stroustrup wrote: 

There
is a snag, though, as several people have pointed out: if you do, you
can't do data collection, tracing, debugging using debug statement,
telemetry, etc. without interface statements.

You can use something like late_check or a similar kind of block-level mechanism. I do not personally see that as a serious burden. Instead of just writing a debug statement, you wrap it in the equivalent of a late_check-ed call/block.

On Thursday, February 18, 2016 at 11:05:07 AM UTC-8, Bjarne Stroustrup wrote: 

It is important, but not essential and not without its own problems. It
is a huge benefit in its own right.

I'm glad we are at least in agreement that it is important. I know that a lot of people in the community don't consider it important and that's been somewhat disappointing to me.

I also fear that some people are taking my paper as something like an attack. I want to really stress here that I do not intend it to be. I don't know the entirety of the more personal controversies surrounding C++0x concepts, and I'm somewhat thankfully a level of indirection removed from all of it since I think it allows me to have an untarnished perspective of things. That said, I am convinced that Concepts as-is would pretty much preclude us from getting definition checking in the language, and I am not alone in this assessment. In addition to my assessment, Doug Gregor agrees (his opinion still at least should mean something), as well as others who have absolutely no ties to C++0x concepts. For what it's worth, when I talked to Doug it was explicitly for his perspective on Concepts. I hadn't actually spoken to him for several years up until that point (I contacted him the day before the mailing deadline) and his personal assessment wasn't even known to me up until that point. We've all come to this conclusion independently, which I think is important to recognize.

I really value your views on this but I can't take it on faith that definition checking will be here eventually when everything that I've come to understand upon my own investigation and through my own comprehension of the problem disagrees. I want concepts in C++ today as much as anyone here, but I cannot endorse Concepts in its current state. If people involved can produce an implementation that properly does definition checking and addresses the many subtleties that C++0x concepts successfully tackled, then please do.

On Thursday, February 18, 2016 at 11:05:07 AM UTC-8, Bjarne Stroustrup wrote: 

Matt's paper is making a lot of assumptions about the knowledge and
opinions about people wanting concepts now based on the current model.
Most seem flat wrong. He seems to have missed the last 5 years progress
in the standards committee, or maybe 10.

That's comes off as a little bit rude, though I'm sure it wasn't intentional. What are the assumptions you are talking about? I haven't missed the progress that has been going on, I just feel that with that progress we have also unnecessarily gone backwards in certain respects, and have done so in a way that will make getting definition checking so difficult that we will likely never even see it (or at the very least that it won't come for another decade).

[Ville Voutilainen]

On 20 February 2016 at 02:11, Matt Calabrese

<metaprogram...@gmail.com> wrote:
> With all due respect, in order for definition-checking to really be
> meaningful and do what the programmer expects regarding C++0x concepts (and
> realistically for any language-level concepts approach), we'd need to make

Well, it certainly seems that different programmers have very
different expectations.

> In C++0x concepts (given that it actually solved this problem, I think it's
> reasonable to refer to), this was accomplished by altering lookup from
> within the constrained template and going through an indirection that
> exposed and preferred the associated functions of that name. In this way,
> users could write their constrained templates in a very simple manner and
> get the correct behavior -- note that this is exceedingly difficult in
> current C++ and a lot of generic code is incorrect because of it. If you do

Also, a lot of generic code is correct despite of this concern, but will
become incorrect if you suddenly enable definition checking on it.

> not do this or something similar, then definition checking cannot
> realistically deliver on the promise of preventing compile-time errors
> during instantiation -- or perhaps worse, you can end up with something that

I find it non-obvious and debatable whether concepts should prevent
all errors during instantiation. And, remember,

1) The Concepts TS allows me to write constrained templates that are not
definition-checked, and we might get definition checking later
2) C++0x concepts don't allow me to write constrained templates that are
not definition-checked. There was never even any plan to allow that.

(1) happens to be of utmost importance. I don't need nor want all my
constrained templates to be definition-checked. I can't have them all
be definition checked, some of them can never be definition-checked.
There's no full migration path to a world where all constrained templates
are definition-checked.

The choice between (1) and (2) is easy and crystal clear to me; (2)
is practically useless.

> compiles and *looks* like it is using the associated functions but actually
> is not, whether because of ADL or other overloads that are in scope. Without
> constrained templates being instantiated in a way that is different from an
> unconstrained template (such as altering lookup), how are these problems
> supposed to be solved? If the answer really is to eventually alter lookup,
> then that's a huge breaking change that is unlikely to make it through, and
> even if it did, we're stuck with the likelihood that most standard and
> user-defined constraints or definitions were written incorrectly and so all
> of that code would break when we "turn the switch" (this is not an
> unreasonable assessment given that it was even true for the C++0x
> conceptified standard library when it was actually implemented -- the vast
> majority of the specified constraints were incorrect and the compiler
> thankfully caught that).

Look - if you expect that every constrained template that is currently
constrained
with traits, and/or later constrained with Concepts TS -concepts will
become definition-checked
by "turning a switch", you should stop expecting that. That will not
happen. It's
completely infeasible. So it seems to me that the concerns about definition
checking are based on Concepts-TS-constrained templates being
(potentially) incompatible
with definition checking. That seemingly leads to suggesting that we
should hold it off to have the
Concept-TS-constrained templates be readily compatible with definition checking.
That horse has left the barn before Concepts of any kind entered any
working draft. The whole
premise is mistaken.

I can currently change lots of my constrained templates to use Concepts instead
of traits. I don't need to touch the definition to do so. Those
constrained templates
are *already* incompatible with definition checking. The Concepts TS
didn't bring
that about. There's *tons* of such code.

> We can try to convince ourselves that the problem is simpler than it really
> is and claim that we'll manage to do definition checking without drastically
> changing things and/or breaking an unreasonable amount of code, but that's

I do expect that definition checking must be explicitly asked for in a
constrained template.
You may consider that a drastic change.

>> There
>> is a snag, though, as several people have pointed out: if you do, you
>> can't do data collection, tracing, debugging using debug statement,
>> telemetry, etc. without interface statements.
> You can use something like late_check or a similar kind of block-level
> mechanism. I do not personally see that as a serious burden. Instead of just
> writing a debug statement, you wrap it in the equivalent of a late_check-ed
> call/block.

That means I would need to do drastic changes. Those changes are not
backwards-compatible, whereas switching from traits to not-definition-checked
constraints can be, because I don't need to touch the definitions. I can do
traits/concepts conditionally in interfaces.

[Herb Sutter]

A few comments:

First, please always remember the principle that it's important to prioritize callers over callees because (a) callers outnumber callees, usually by orders of magnitude, and (b) callees are much more scoped and contained, including they can much more readily be unit-tested using current techniques.

In this context, that means it is far more important to check callers (template uses) than callees (template definitions). Of course, if we can get both we should do that; but if we can only get one, or get one first, then we absolutely must prioritize callers over callees. This is a fundamental principle.

> > In C++0x concepts (given that it actually solved this problem,

Wait, no: C++0x concepts failed to solve any problem, because C++0x concepts Did Not Work in the real world.

In particular, C++0x concepts were never shown to be implementable with sufficient performance to actually be usable in a commercial compiler and user project. The prototype ConceptGCC implementation never overcame a multi-factor (IIRC, 6x) slowdown in recompiling existing code against a concept-ized STL, which is way beyond the "unadoptable" line; that would need to come closer to 20% to even consider again just on performance grounds. -- And there were other serious problems that were never addressed, such as that C++0x concepts bifurcated the template system between constrained and unconstrained templates, which was a major contributor to the fact that after several years we still could never figure out how to actually use them correctly.

Note that the part of C++0x concepts I thin Matt most wants is separate checking, which relied on concept maps, and that is exactly the part of the design that caused the excessive overheads and was never shown to be realistically implementable.

Fortunately, what we have now is a concrete design that is promising (including actually addressing the showstopper performance and usability problems with C++0x concepts, where compile times appear to be comparable to non-concepts compile times), and as a bonus is not known to have actually closed the door on separate checking later. And that's why we continue pursuing it for ISO C++, whether in or after C++17 (we'll see how the committee feels in Jacksonville) -- it's the only concepts proposal ever brought forward that doesn't have showstopper problems, therefore the only one ever brought forward that is viable.

Herb

> --
> You received this message because you are subscribed to the Google Groups
> "SG8 - Concepts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to concepts+u...@isocpp.org.
> To post to this group, send email to conc...@isocpp.org.
> Visit this group at

> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgroups.go
> ogle.com%2fa%2fisocpp.org%2fgroup%2fconcepts%2f.&data=01%7c01%7chs
> utter%40microsoft.com%7cc733542d922a44352afb08d339915330%7c72f98
> 8bf86f141af91ab2d7cd011db47%7c1&sdata=%2b535W1CM6Vm4cFebgVnT%
> 2f1mHzz8v3B8limypR%2fEWJO4%3d

[Matt Calabrese]

On Friday, February 19, 2016 at 5:01:11 PM UTC-8, Ville Voutilainen wrote:

Also, a lot of generic code is correct despite of this concern, but will
become incorrect if you suddenly enable definition checking on it.

We seem to have very different definitions of correct. If you enable definition checking and it then fails, then the code was certainly *not* correct. Either the developer specified incorrect constraints, or the developer accidentally used more functionality than the properly-specified constraints, or the developer wrote calls in a way that fail for certain instantiations but not all due to the incredible subtleties of C++ (such as when making unqualified calls).

On Friday, February 19, 2016 at 5:01:11 PM UTC-8, Ville Voutilainen wrote: 

I find it non-obvious and debatable whether concepts should prevent
all errors during instantiation. And, remember,

1) The Concepts TS allows me to write constrained templates that are not
definition-checked, and we might get definition checking later
2) C++0x concepts don't allow me to write constrained templates that are
not definition-checked. There was never even any plan to allow that.

C++0x concepts did allow that. If you made a late_check block inside of a constrained function, the body of the block was unconstrained and ignored the requirements. Putting the full body of a constrained template in late_check effectively made an unconstrained template with a constrained interface, which is all that the Concepts TS currently provides.

Anyway, I don't want this to turn into a battle between C++0x Concepts and the Concepts TS. All I care is about is concepts as a language feature, no matter what it's called or who makes it. I'm not saying that we should revive C++0x concepts, nor that it was perfect, only that the concerns I have are valid ones that shouldn't be so easily hand-waved away. The fact that C++0x concepts handled these concerns and that both the specification of constraints and name lookup within constrained template bodies were affected in order to support definition checking shows that the problems are tractable with that approach and that getting definition checking can seriously impact the fundamentals of the design regarding even what the body of a constrained template *means*. I have seen nothing that convinces me that the Concepts TS as-is can evolve to something that adds definition checking without making similar changes, and if Concepts are added to C++17, I find it extremely unlikely that we'd ever manage to make those changes because of the amount of code that would be affected (effectively all constrained templates).

On Friday, February 19, 2016 at 5:01:11 PM UTC-8, Ville Voutilainen wrote: 

Look - if you expect that every constrained template that is currently
constrained
with traits, and/or later constrained with Concepts TS -concepts will
become definition-checked
by "turning a switch", you should stop expecting that. That will not
happen. It's
completely infeasible.

I agree that it is not feasible for SFINAE-constrained templates, which is one of the reasons why we need language-level concepts. I disagree that it is the case that we should not expect a language-level constraint facility to be able to properly check the definition of a constrained template, though, and as a byproduct guarantee that you really are invoking the associated functions that you intend. It's also very clearly not impossible given that C++0x concepts effectively could do this -- just wrap/unwrap the definition in late_check (not that I would have personally recommend doing this except during migration).

On Friday, February 19, 2016 at 5:01:11 PM UTC-8, Ville Voutilainen wrote: 

I do expect that definition checking must be explicitly asked for in a
constrained template.
You may consider that a drastic change.

Even if you explicitly ask for it, it is unlikely that the current specification of constraints would be sufficient to provide proper checking.

On Friday, February 19, 2016 at 5:01:11 PM UTC-8, Ville Voutilainen wrote: 

> You can use something like late_check or a similar kind of block-level
> mechanism. I do not personally see that as a serious burden. Instead of just
> writing a debug statement, you wrap it in the equivalent of a late_check-ed
> call/block.

That means I would need to do drastic changes. Those changes are not
backwards-compatible, whereas switching from traits to not-definition-checked
constraints can be, because I don't need to touch the definitions. I can do
traits/concepts conditionally in interfaces.

I'm not sure I follow -- how would writing "late_check" be a drastic change? You're already re-specifying the constraints -- is a single additional keyword really a problem? That seems like an incredible stretch. Or am I misunderstanding your point here?

[Ville Voutilainen]

On 20 February 2016 at 04:13, Matt Calabrese

<metaprogram...@gmail.com> wrote:
> On Friday, February 19, 2016 at 5:01:11 PM UTC-8, Ville Voutilainen wrote:
>>
>> Also, a lot of generic code is correct despite of this concern, but will
>> become incorrect if you suddenly enable definition checking on it.
>
>
> We seem to have very different definitions of correct. If you enable
> definition checking and it then fails, then the code was certainly *not*
> correct. Either the developer specified incorrect constraints, or the

Such code exists today, it compiles and runs and works as required.
I'm willing to let you call that code incorrect by your definition, but I have
no trouble disagreeing with that view.

> developer accidentally used more functionality than the properly-specified
> constraints, or the developer wrote calls in a way that fail for certain

Or the developer very deliberately used more functionality than the
"properly-specified
constraints", for debatable definition of "properly".

>> I find it non-obvious and debatable whether concepts should prevent
>> all errors during instantiation. And, remember,
>> 1) The Concepts TS allows me to write constrained templates that are not
>> definition-checked, and we might get definition checking later
>> 2) C++0x concepts don't allow me to write constrained templates that are
>> not definition-checked. There was never even any plan to allow that.
> C++0x concepts did allow that. If you made a late_check block inside of a
> constrained function, the body of the block was unconstrained and ignored

Yeah, except for the cases where it wasn't allowed, like base-specifiers.

> Anyway, I don't want this to turn into a battle between C++0x Concepts and
> the Concepts TS. All I care is about is concepts as a language feature, no
> matter what it's called or who makes it. I'm not saying that we should
> revive C++0x concepts, nor that it was perfect, only that the concerns I
> have are valid ones that shouldn't be so easily hand-waved away. The fact

I don't know what hand-waving you're talking about. What I have written
is no hand-waving.

> that C++0x concepts handled these concerns and that both the specification
> of constraints and name lookup within constrained template bodies were
> affected in order to support definition checking shows that the problems are
> tractable with that approach and that getting definition checking can
> seriously impact the fundamentals of the design regarding even what the body
> of a constrained template *means*. I have seen nothing that convinces me
> that the Concepts TS as-is can evolve to something that adds definition

I don't know what you mean by "as-is". It still sounds like you're striving
for "turning a switch".

>> Look - if you expect that every constrained template that is currently
>> constrained
>> with traits, and/or later constrained with Concepts TS -concepts will
>> become definition-checked
>> by "turning a switch", you should stop expecting that. That will not
>> happen. It's
>> completely infeasible.
> I agree that it is not feasible for SFINAE-constrained templates, which is
> one of the reasons why we need language-level concepts. I disagree that it

That's not the point. The point is about whether the definition of a constrained
template needs to be changed when the constraint mechanism is updated
from legacy mechanisms to concepts.

> is the case that we should not expect a language-level constraint facility
> to be able to properly check the definition of a constrained template,

Oh, you can certainly expect it to be able to "properly" check the definition,
and you can also expect it to either turn well-formed correct code into
ill-formed code, or require changing the definition.

> though, and as a byproduct guarantee that you really are invoking the
> associated functions that you intend. It's also very clearly not impossible
> given that C++0x concepts effectively could do this -- just wrap/unwrap the
> definition in late_check (not that I would have personally recommend doing
> this except during migration).

C++0x concepts could not do this. They suffered from massive migration
problems.

>> I do expect that definition checking must be explicitly asked for in a
>> constrained template.
>> You may consider that a drastic change.
> Even if you explicitly ask for it, it is unlikely that the current
> specification of constraints would be sufficient to provide proper checking.

Meaning what? The current specification of constraints, or the current
specification of a constrained function, which is lacking any lookup changes?

>> > You can use something like late_check or a similar kind of block-level
>> > mechanism. I do not personally see that as a serious burden. Instead of
>> > just
>> > writing a debug statement, you wrap it in the equivalent of a
>> > late_check-ed
>> > call/block.
>> That means I would need to do drastic changes. Those changes are not
>> backwards-compatible, whereas switching from traits to
>> not-definition-checked
>> constraints can be, because I don't need to touch the definitions. I can
>> do
>> traits/concepts conditionally in interfaces.
> I'm not sure I follow -- how would writing "late_check" be a drastic change?
> You're already re-specifying the constraints -- is a single additional
> keyword really a problem? That seems like an incredible stretch. Or am I
> misunderstanding your point here?

It requires changing the definition. That *IS* a drastic change. Re-specifying
the constraints with the Concepts TS concepts doesn't require touching
the definition.

[Matt Calabrese]

On Friday, February 19, 2016 at 5:59:14 PM UTC-8, Herb Sutter wrote:

A few comments:

First, please always remember the principle that it's important to prioritize callers over callees because (a) callers outnumber callees, usually by orders of magnitude, and (b) callees are much more scoped and contained, including they can much more readily be unit-tested using current techniques.

In this context, that means it is far more important to check callers (template uses) than callees (template definitions). Of course, if we can get both we should do that; but if we can only get one, or get one first, then we absolutely must prioritize callers over callees. This is a fundamental principle.

Though I personally agree with this principle, understand that not having definition checking really does affect callers. Even with people properly using SFINAE exploits today (to the extent that anyone might consider SFINAE exploits "proper"), we still see compile errors with nasty template backtraces when in the bodies of constrained templates. This is, in part, because it's both extremely difficult to properly specify constraints and perhaps even more difficult to correctly and unambiguously refer to the associated functions. Whether people are using SFINAE hacks or concepts at the interface boundary won't actually change that.

On Friday, February 19, 2016 at 5:59:14 PM UTC-8, Herb Sutter wrote: 

Wait, no: C++0x concepts failed to solve any problem, because C++0x concepts Did Not Work in the real world. 

In particular, C++0x concepts were never shown to be implementable with sufficient performance to actually be usable in a commercial compiler and user project. The prototype ConceptGCC implementation never overcame a multi-factor (IIRC, 6x) slowdown in recompiling existing code against a concept-ized STL, which is way beyond the "unadoptable" line; that would need to come closer to 20% to even consider again just on performance grounds.

ConceptsGCC was implemented both as a prototype and also at a time when things like specialization-lookup in GCC was not constant. IIRC, though this is based purely on memory, Doug described certain operations as being unnecessarily N^2 when they could have been constant. While there, of course, would be compile-time implications due to checking, ConceptGCC was not directly indicative of what a modern implementation would see.

On Friday, February 19, 2016 at 5:59:14 PM UTC-8, Herb Sutter wrote: 

-- And there were other serious problems that were never addressed, such as that C++0x concepts bifurcated the template system between constrained and unconstrained templates, which was a major contributor to the fact that after several years we still could never figure out how to actually use them correctly.

There were solutions to many concerns in the form of late_check and there was progress being made towards having the interaction between constrained and unconstrained code more approachable, however those ideas were never fully explored due to the... change in course.

On Friday, February 19, 2016 at 5:59:14 PM UTC-8, Herb Sutter wrote: 

Note that the part of C++0x concepts I thin Matt most wants is separate checking, which relied on concept maps, and that is exactly the part of the design that caused the excessive overheads and was never shown to be realistically implementable.

It was shown to be implementable. The prototype was just slow, as one might expect from a prototype that was built on a compiler that, at the time, was not particularly suited for the task.

For what it's worth, I am not alone in my desire for separate checking. As mentioned, there are various people both inside and outside of the committee who are or were under the impression that definition-checking will be coming, with the misconception that there is a very clear path towards getting it. At the very least, those misconceptions need to either be dispelled before a vote or otherwise the ability to do definition-checking should be proven. If there is consensus that we are really willing to likely give up on getting definition checking in a truly meaningful sense, then I'm fine with that even though I personally disagree. People at least need to be aware of this in order to make an informed decision.

[Matt Calabrese]

On Friday, February 19, 2016 at 6:40:07 PM UTC-8, Ville Voutilainen wrote:

On 20 February 2016 at 04:13, Matt Calabrese
<metaprogram...@gmail.com> wrote:
> On Friday, February 19, 2016 at 5:01:11 PM UTC-8, Ville Voutilainen wrote:
>>
>> Also, a lot of generic code is correct despite of this concern, but will
>> become incorrect if you suddenly enable definition checking on it.
>
>
> We seem to have very different definitions of correct. If you enable
> definition checking and it then fails, then the code was certainly *not*
> correct. Either the developer specified incorrect constraints, or the

Such code exists today, it compiles and runs and works as required.
I'm willing to let you call that code incorrect by your definition, but I have
no trouble disagreeing with that view.

So if the standard specified constraints that didn't match what was actually necessarily for implementation, you don't think this would be considered a defect in the standard? Or similarly, if a standard library implementation depended on functionality that wasn't a part of the specified constraints, you wouldn't consider this an invalid implementation? I understand that many users may not encounter the issues and that the issues can be very subtle, but it is precisely that subtlety that makes the checking so important, in my opinion. The subtlest issues are the ones that require the most aid at identifying.

On Friday, February 19, 2016 at 6:40:07 PM UTC-8, Ville Voutilainen wrote:

Or the developer very deliberately used more functionality than the
"properly-specified
constraints", for debatable definition of "properly".

... I don't understand how you can even state that this is debatable. If the developer depends on functionality that is not a part of the constraints, then those constraints are under-specified. If you are referring to things like debug statements, etc. then you can use facilities like what was late_check of the C++0x concepts world, though that is somewhat analogous to a const_cast (again, only using C++0x as an example).

On Friday, February 19, 2016 at 6:40:07 PM UTC-8, Ville Voutilainen wrote:

>> Look - if you expect that every constrained template that is currently 

>> constrained
>> with traits, and/or later constrained with Concepts TS -concepts will
>> become definition-checked
>> by "turning a switch", you should stop expecting that. That will not
>> happen. It's
>> completely infeasible.
> I agree that it is not feasible for SFINAE-constrained templates, which is
> one of the reasons why we need language-level concepts. I disagree that it

That's not the point. The point is about whether the definition of a constrained
template needs to be changed when the constraint mechanism is updated
from legacy mechanisms to concepts.

Why? It is expected that code written prior to concepts being introduced as a language facility is often incorrect regarding its constraint specification or implementation, and we have experience that shows that. I would personally be grateful if such a process found bugs in my code and specifications, forcing me to fix them. I'm much less concerned, however, with simply replacing things like enable_if with Concepts constraints.

On Friday, February 19, 2016 at 6:40:07 PM UTC-8, Ville Voutilainen wrote:

> is the case that we should not expect a language-level constraint facility
> to be able to properly check the definition of a constrained template,

Oh, you can certainly expect it to be able to "properly" check the definition,
and you can also expect it to either turn well-formed correct code into
ill-formed code, or require changing the definition.

 

Of course you're probably going to have to change the definition when a proper language-level facility is introduced. It's finding problems with your code whether you choose to acknowledge them as such or not.

On Friday, February 19, 2016 at 6:40:07 PM UTC-8, Ville Voutilainen wrote:

>> I do expect that definition checking must be explicitly asked for in a
>> constrained template.
>> You may consider that a drastic change.
> Even if you explicitly ask for it, it is unlikely that the current
> specification of constraints would be sufficient to provide proper checking.

Meaning what? The current specification of constraints, or the current
specification of a constrained function, which is lacking any lookup changes?

Both would likely need to be changed. On the constraint side, you'd probably need the constraints to effectively be isomorphic with pseudo-signatures for them to be useful, including the semantics of how things like return value and parameter types are treated from within a constrained definition (i.e. the requirements are actually convertible-to-T, but in the constrained definition are only seen as exactly T, etc.).

On Friday, February 19, 2016 at 6:40:07 PM UTC-8, Ville Voutilainen wrote:

> I'm not sure I follow -- how would writing "late_check" be a drastic change?
> You're already re-specifying the constraints -- is a single additional
> keyword really a problem? That seems like an incredible stretch. Or am I
> misunderstanding your point here?

It requires changing the definition. That *IS* a drastic change. Re-specifying
the constraints with the Concepts TS concepts doesn't require touching
the definition.

Right, because in those cases it's a very superficial change without much hope of getting beyond that. That's the problem.

[Jonathan Wakely]

On 20 February 2016 at 03:30, Matt Calabrese wrote:
> So if the standard specified constraints that didn't match what was actually
> necessarily for implementation, you don't think this would be considered a
> defect in the standard? Or similarly, if a standard library implementation
> depended on functionality that wasn't a part of the specified constraints,
> you wouldn't consider this an invalid implementation? I understand that many
> users may not encounter the issues and that the issues can be very subtle,
> but it is precisely that subtlety that makes the checking so important, in
> my opinion. The subtlest issues are the ones that require the most aid at
> identifying.

But should such subtle, rarely encountered issues really prevent us
putting the Concepts TS into the standard?

Are they a problem? Yes. Are they a big problem? No.

Should we halt progress until we solve every little problem?

[Ville Voutilainen]

On 20 February 2016 at 05:30, Matt Calabrese

<metaprogram...@gmail.com> wrote:
>> >> Also, a lot of generic code is correct despite of this concern, but
>> >> will
>> >> become incorrect if you suddenly enable definition checking on it.
>> > We seem to have very different definitions of correct. If you enable
>> > definition checking and it then fails, then the code was certainly *not*
>> > correct. Either the developer specified incorrect constraints, or the
>> Such code exists today, it compiles and runs and works as required.
>> I'm willing to let you call that code incorrect by your definition, but I
>> have
>> no trouble disagreeing with that view.
> So if the standard specified constraints that didn't match what was actually
> necessarily for implementation, you don't think this would be considered a
> defect in the standard? Or similarly, if a standard library implementation

I wasn't talking about standard library code, for what it's worth.

> depended on functionality that wasn't a part of the specified constraints,
> you wouldn't consider this an invalid implementation? I understand that many

That very much depends on whether the code can produce a hard
error(*). In addition
to that, there are cases where the standard allows implementation freedom for
whether an implementation diagnoses certain requirements violations. So
no, I wouldn't necessarily consider that an invalid implementation.

(*) A violation of "proper constraints" doesn't necessarily produce a
hard error.

> users may not encounter the issues and that the issues can be very subtle,
> but it is precisely that subtlety that makes the checking so important, in
> my opinion. The subtlest issues are the ones that require the most aid at
> identifying.

On the other hand, bugs that never manifest do not need to be fixed.
Except if definition checking is in effect, in which case they have to be.
Which is why definition checking cannot be mandatory.

>> Or the developer very deliberately used more functionality than the
>> "properly-specified
>> constraints", for debatable definition of "properly".
> ... I don't understand how you can even state that this is debatable. If the
> developer depends on functionality that is not a part of the constraints,
> then those constraints are under-specified. If you are referring to things

I can easily find cases where that's not true. A call chain from a constrained
template to an unconstrained template to a constrained template is such
a case, if the unconstrained template doesn't violate the constraints on
the end points, and those constraints are compatible.

>> That's not the point. The point is about whether the definition of a
>> constrained
>> template needs to be changed when the constraint mechanism is updated
>> from legacy mechanisms to concepts.
> Why? It is expected that code written prior to concepts being introduced as
> a language facility is often incorrect regarding its constraint
> specification or implementation, and we have experience that shows that. I

Maybe, but we also have experience that shows that code written prior
to concepts
being introuced as a language facility is correct regarding its
constraint specification,
even if it violates strict definition checking.

> would personally be grateful if such a process found bugs in my code and
> specifications, forcing me to fix them. I'm much less concerned, however,

I wouldn't be grateful for a language facility breaking my code for no reason.

>> Oh, you can certainly expect it to be able to "properly" check the
>> definition,
>> and you can also expect it to either turn well-formed correct code into
>> ill-formed code, or require changing the definition.
> Of course you're probably going to have to change the definition when a
> proper language-level facility is introduced. It's finding problems with
> your code whether you choose to acknowledge them as such or not.

It's not a "choice to acknowledge". I can prove that the code I have
is correct as written
but incorrect under definition checking.

>> Meaning what? The current specification of constraints, or the current
>> specification of a constrained function, which is lacking any lookup
>> changes?
> Both would likely need to be changed. On the constraint side, you'd probably

"Likely"? You seem to use that word a lot.

> need the constraints to effectively be isomorphic with pseudo-signatures for
> them to be useful, including the semantics of how things like return value

I do not know what that means. You continue to use very vague descriptions.

> and parameter types are treated from within a constrained definition (i.e.
> the requirements are actually convertible-to-T, but in the constrained
> definition are only seen as exactly T, etc.).

For example?

>> > I'm not sure I follow -- how would writing "late_check" be a drastic
>> > change?
>> > You're already re-specifying the constraints -- is a single additional
>> > keyword really a problem? That seems like an incredible stretch. Or am I
>> > misunderstanding your point here?
>> It requires changing the definition. That *IS* a drastic change.
>> Re-specifying
>> the constraints with the Concepts TS concepts doesn't require touching
>> the definition.
> Right, because in those cases it's a very superficial change without much
> hope of getting beyond that. That's the problem.

Well, we obviously disagree on whether it's a superficial change and whether
there is a problem with that.

[Matt Calabrese]

If the design implies that those problems -- both subtle and less subtle, mind you -- may not actually be realistically solvable without redesigning the feature and/or re-specifying its uses, then the issue should certainly be addressed, in my opinion, especially if that problem is considered important to many of those who want the facility. We understand the subtleties of the issue and even know more-or-less the fundamentals of what is required to get around it, so lets not ignore that.

Note that we are in a very different situation here from other features that permit more incremental progress. With many other aspects of the language, we do not have such a problem. I am not conservative by nature. If we shipped std::list without something like emplacement facilities, for example, we could have always added it in the future without any harm done. The concern here is that the current design may very well prevent us from making such incremental progress with language-level concepts. I'd rather not do that, and given that we don't even have a conceptified standard library, it's just one more reason on top of the others for why we should hold off.

There is nothing wrong with Concepts remaining a TS for now and not being a part of C++17. It's not as though it would be dead if it doesn't immediately make it into the next standard. Please, let's just not risk seriously messing this up because we want it directly in the language a couple of years early.

[Nevin Liber]

On 19 February 2016 at 22:04, Matt Calabrese <metaprogram...@gmail.com> wrote:

The concern here is that the current design may very well prevent us from making such incremental progress with language-level concepts. I'd rather not do that, and given that we don't even have a conceptified standard library, it's just one more reason on top of the others for why we should hold off.

We all want unicorns. 

It's been seven years since Concepts were pulled from C++0x.  Where is an updated prototype with reasonable compile times?  Who is committing to doing that investigative work?  Are we supposed to wait indefinitely?  What makes progress on this?

You haven't even provided concrete examples of why this is so critical to resolve first.  Not here on the mailing list, and not in your paper.  Lots of hand waving, but nothing people can evaluate.  That is pretty much the definition of FUD.

 

There is nothing wrong with Concepts remaining a TS for now and not being a part of C++17. It's not as though it would be dead if it doesn't immediately make it into the next standard. Please, let's just not risk seriously messing this up because we want it directly in the language a couple of years early.

The committee has been working on it for thirteen years.  This is not early.  I don't find "may not be compatible with some as yet to be designed future feature" to be a compelling reason to delay it; there is no bound or convergence on that.

--

 Nevin ":-)" Liber  <mailto:ne...@eviloverlord.com>  +1-847-691-1404

[Matt Calabrese]

On Friday, February 19, 2016 at 7:57:42 PM UTC-8, Ville Voutilainen wrote:

On 20 February 2016 at 05:30, Matt Calabrese
<metaprogram...@gmail.com> wrote:
>> >> Also, a lot of generic code is correct despite of this concern, but
>> >> will
>> >> become incorrect if you suddenly enable definition checking on it.
>> > We seem to have very different definitions of correct. If you enable
>> > definition checking and it then fails, then the code was certainly *not*
>> > correct. Either the developer specified incorrect constraints, or the
>> Such code exists today, it compiles and runs and works as required.
>> I'm willing to let you call that code incorrect by your definition, but I
>> have
>> no trouble disagreeing with that view.
> So if the standard specified constraints that didn't match what was actually
> necessarily for implementation, you don't think this would be considered a
> defect in the standard? Or similarly, if a standard library implementation

I wasn't talking about standard library code, for what it's worth.

That distinction is arbitrary. If it's not correct for standard library code, why would one consider it "correct" for user-defined code. In either case, the specification is incorrect.

 

On Friday, February 19, 2016 at 7:57:42 PM UTC-8, Ville Voutilainen wrote:

> depended on functionality that wasn't a part of the specified constraints,
> you wouldn't consider this an invalid implementation? I understand that many

That very much depends on whether the code can produce a hard
error(*). In addition
to that, there are cases where the standard allows implementation freedom for
whether an implementation diagnoses certain requirements violations. So
no, I wouldn't necessarily consider that an invalid implementation.

(*) A violation of "proper constraints" doesn't necessarily produce a
hard error.

There is a difference between a compiler not being strictly required to diagnose an error and that error not existing. For example, compilers often cannot diagnose UB, but your code is still incorrect if you would encounter it. Regardless of whether or not a requirement violation or something like UB produces a hard error either at compile time or at runtime doesn't change the fact that it's still a mistake.

On Friday, February 19, 2016 at 7:57:42 PM UTC-8, Ville Voutilainen wrote:

> users may not encounter the issues and that the issues can be very subtle,
> but it is precisely that subtlety that makes the checking so important, in
> my opinion. The subtlest issues are the ones that require the most aid at
> identifying.

On the other hand, bugs that never manifest do not need to be fixed.
Except if definition checking is in effect, in which case they have to be.
Which is why definition checking cannot be mandatory.

This is starting to sound like conversations I have with people working in dynamically typed languages. Not that I'm saying your argument is erroneous because of it, as it's a valid point of view, I just want to point that out. Just as it is not particularly burdensome to expect people to properly specify types and function signatures, I do not see it as particularly burdensome to expect people to properly specify concepts and constraints.

My perspective is that code that is written should be logically sound and it is also in my opinion that this is part of what formally specifying concepts and constraints, and using those associated operations is about. It may be that code can be written that isn't actually correct and yet happens to work for some subset of cases that you are encountering (by analogy, this kind of thing happens all the time in dynamically typed languages, for better or for worse), but I'd still be grateful if the language identified my latent mistakes before they manifest themselves in a more subtle way.

Ultimately, I think that some of your concerns about constrained template checking being... on by default or however you want to call it, are a bit unwarranted. If you write some code that goes outside of the constraints then the compiler should tell you and you should adjust your constraints accordingly (or not rely on the function, or in the case of constrained templates use something like late_check). This is not particularly more difficult to diagnose or even to fix than when being expected to strictly use the declared member functions of a type in a plain old function. It's not unreasonable to expect constraints to be properly specified and for functions to be written properly according to those constraints.

On Friday, February 19, 2016 at 7:57:42 PM UTC-8, Ville Voutilainen wrote:

>> Or the developer very deliberately used more functionality than the
>> "properly-specified
>> constraints", for debatable definition of "properly".
> ... I don't understand how you can even state that this is debatable. If the
> developer depends on functionality that is not a part of the constraints,
> then those constraints are under-specified. If you are referring to things

I can easily find cases where that's not true. A call chain from a constrained
template to an unconstrained template to a constrained template is such
a case, if the unconstrained template doesn't violate the constraints on
the end points, and those constraints are compatible.

We talked about this in Kona and I know that both Dave A. and yourself independently (or perhaps unintentionally dependently) had thoughts on how to make this more approachable. I have my own views on this topic as well. Regardless of approach, you're always going to have to expel at least some effort when dancing between a tightly checked world and a less-tightly-checked world if you want to try to maintain some level of checking.

On Friday, February 19, 2016 at 7:57:42 PM UTC-8, Ville Voutilainen wrote:

It's not a "choice to acknowledge". I can prove that the code I have
is correct as written
but incorrect under definition checking.

If you are talking about things like the change in lookup affecting what you originally wanted to do in the unconstrained world, that's because there is a (in my opinion, necessary) change in lookup. Or are you talking about something else?

On Friday, February 19, 2016 at 7:57:42 PM UTC-8, Ville Voutilainen wrote:

>> Meaning what? The current specification of constraints, or the current
>> specification of a constrained function, which is lacking any lookup
>> changes?
> Both would likely need to be changed. On the constraint side, you'd probably

"Likely"? You seem to use that word a lot.

"Likely" stemming from the fact that the working implementation of definition checking that we had depended on this and there's not a thoroughly proven approach that wouldn't have to do similar things. It's also the point of view of certain experts on the subject, which certainly isn't authoritative but should at least not be written off. If you aren't fond of the word "likely" I'll say it's "debatable" from now on, though I really do personally feel that's a little too generous. If we're getting serious about language usage, I also take issue with people stating that there is a clear path forward for definition checking. That is very misleading, especially to laymen.

On Friday, February 19, 2016 at 7:57:42 PM UTC-8, Ville Voutilainen wrote:

> need the constraints to effectively be isomorphic with pseudo-signatures for
> them to be useful, including the semantics of how things like return value

I do not know what that means. You continue to use very vague descriptions.

Sorry. In a simplified example, consider a pseudo-signature that "returns" T, keeping in mind that the function it matches with doesn't necessarily return exactly T, but rather, just something convertible to T. Note that this is a common kind of constraint in practice. Now, in your constrained template, you make a call to some function "foo" that takes a T (maybe a constrained function or not, depending on if T is dependent -- it doesn't really matter). In the current world, one can naively, though not obviously so, invoke "foo" directly with the result of the call to the associated function... only this is not strictly correct. This can either call a different function or be ambiguous, or cause other kinds of issues. As a byproduct of definition checking (or more precisely, in the C++0x world at least, by going through a map), this code definitively calls the foo that takes T and the writer of the generic code does not have to be a language lawyer to understand the subtleties to do it properly. It works in the way a reasonable programmer would expect it to.

While this is a very simplified version of one of the problems, it manifests itself in subtle ways in higher level code, and if you want definition checking that actually accomplishes one of the goals that it intends to solve (preventing errors during instantiation), then it needs to be addressed in some way. We knew how to address it in a way that made sense in the C++0x world and I'm extremely skeptical that the problem can be solved without at least some similar kinds of methods (or alternatively, the other methods should be explained and not just be presumed to exist or taken on faith).

[Ville Voutilainen]

On 20 February 2016 at 08:34, Matt Calabrese

<metaprogram...@gmail.com> wrote:
>> >> >> Also, a lot of generic code is correct despite of this concern, but
>> >> >> will
>> >> >> become incorrect if you suddenly enable definition checking on it.
>> >> > We seem to have very different definitions of correct. If you enable
>> >> > definition checking and it then fails, then the code was certainly
>> >> > *not*
>> >> > correct. Either the developer specified incorrect constraints, or the
>> >> Such code exists today, it compiles and runs and works as required.
>> >> I'm willing to let you call that code incorrect by your definition, but
>> >> I
>> >> have
>> >> no trouble disagreeing with that view.
>> > So if the standard specified constraints that didn't match what was
>> > actually
>> > necessarily for implementation, you don't think this would be considered
>> > a
>> > defect in the standard? Or similarly, if a standard library
>> > implementation
>>
>> I wasn't talking about standard library code, for what it's worth.
> That distinction is arbitrary. If it's not correct for standard library
> code, why would one consider it "correct" for user-defined code. In either
> case, the specification is incorrect.

For user-defined code, there are more cases where it's correct, because
it may be so that it's architecturally ensured that you won't encounter
an error at instantiation time. For standard code, if you can't get an
instantiation-time
error out of it for a Concepts-TS-constrained function, I consider the
constraints
correct, even though they aren't under definition checking.

>> On the other hand, bugs that never manifest do not need to be fixed.
>> Except if definition checking is in effect, in which case they have to be.
>> Which is why definition checking cannot be mandatory.
> This is starting to sound like conversations I have with people working in
> dynamically typed languages. Not that I'm saying your argument is erroneous
> because of it, as it's a valid point of view, I just want to point that out.
> Just as it is not particularly burdensome to expect people to properly
> specify types and function signatures, I do not see it as particularly
> burdensome to expect people to properly specify concepts and constraints.

Well, I have tried "properly specifying" definition-checked
constraints for existing code,
and I found it burdensome. You mentioned late_check being analogous to
a const_cast.
For the cases I'm talking about, it's analogous to a const_cast that
adds const to something
that was already const, in language that requires a const_cast for such cases to
avoid the language diagnosing false positives.

> My perspective is that code that is written should be logically sound and it

You're more than welcome to do that with explicitly definition-checked
constrained
functions. You're not at all welcome to force that dogma on code that doesn't
want definition checking.

> Ultimately, I think that some of your concerns about constrained template
> checking being... on by default or however you want to call it, are a bit
> unwarranted. If you write some code that goes outside of the constraints

Unwarranted as in testing it on existing code?

> then the compiler should tell you and you should adjust your constraints
> accordingly (or not rely on the function, or in the case of constrained
> templates use something like late_check). This is not particularly more
> difficult to diagnose or even to fix than when being expected to strictly
> use the declared member functions of a type in a plain old function. It's
> not unreasonable to expect constraints to be properly specified and for
> functions to be written properly according to those constraints.

I'm eagerly waiting for rich volunteers who are willing to provide the budget
for such pointless changes in large-scale applications.

[Tomasz]

W dniu sobota, 20 lutego 2016 07:34:50 UTC+1 użytkownik Matt Calabrese napisał:

Sorry. In a simplified example, consider a pseudo-signature that "returns" T, keeping in mind that the function it matches with doesn't necessarily return exactly T, but rather, just something convertible to T. Note that this is a common kind of constraint in practice. Now, in your constrained template, you make a call to some function "foo" that takes a T (maybe a constrained function or not, depending on if T is dependent -- it doesn't really matter). In the current world, one can naively, though not obviously so, invoke "foo" directly with the result of the call to the associated function... only this is not strictly correct. This can either call a different function or be ambiguous, or cause other kinds of issues. As a byproduct of definition checking (or more precisely, in the C++0x world at least, by going through a map), this code definitively calls the foo that takes T and the writer of the generic code does not have to be a language lawyer to understand the subtleties to do it properly. It works in the way a reasonable programmer would expect it to.

From the other side, allowing the code to take other overload of foo() that takes exactly was was returned from your function and not T precisely, is the think that allows certain desirable optimizations, like use of expression templates in numeric libraryies. Lets consider following template:
template<T>
T evaluate_polynomial(coefficients<T> const& coefs, T val)
{
  T result{0};
   for (T const& coef : coefs)
      result = coef * val + result;
   return result;
}
I assume that the constrain for this function, may looks like follows:
Multiplicative: T const& * T const& -> T
Additive: T const& + T const& -> T
ZeroConstructAdditionNeutralElement: 0 -> T

So with the implementation checking model, the operation:
coef * val + result;
Will be interpreted (if I understand correctly):
T(T(coef * val) + result)
If we assume that T is type of square matrix with same size, this will incur cost of creating the temporary.

While for exactly the same implementation for uncostrained template (and even constrained without implementation checking)
coef * val + result;
may produced expression template that will use optimized/vectorized multiply and add instruction.

[Matt Calabrese]

On Saturday, February 20, 2016 at 1:31:36 PM UTC-8, Tomasz wrote:

W dniu sobota, 20 lutego 2016 07:34:50 UTC+1 użytkownik Matt Calabrese napisał:

Sorry. In a simplified example, consider a pseudo-signature that "returns" T, keeping in mind that the function it matches with doesn't necessarily return exactly T, but rather, just something convertible to T. Note that this is a common kind of constraint in practice. Now, in your constrained template, you make a call to some function "foo" that takes a T (maybe a constrained function or not, depending on if T is dependent -- it doesn't really matter). In the current world, one can naively, though not obviously so, invoke "foo" directly with the result of the call to the associated function... only this is not strictly correct. This can either call a different function or be ambiguous, or cause other kinds of issues. As a byproduct of definition checking (or more precisely, in the C++0x world at least, by going through a map), this code definitively calls the foo that takes T and the writer of the generic code does not have to be a language lawyer to understand the subtleties to do it properly. It works in the way a reasonable programmer would expect it to.

From the other side, allowing the code to take other overload of foo() that takes exactly was was returned from your function and not T precisely, is the think that allows certain desirable optimizations, like use of expression templates in numeric libraryies.

If you want the behavior as in your example then the concept you are using wouldn't be a concept operating on T with operators specified to return T. It would be a concept operating on T that has operators that return associated types, where those associated types are convertible to T. Those types model a less-refined version of the concept (assuming expression templates, where the associated types likely wouldn't provide most of the mutating operations that T provides). Note that a concept such as this would also match a type where the operators directly return T such as if T were a double or int.

Note too that if you wrote your code against the simpler concept that returns T from each operator, the code works with expression templates without subtlety (even though it will more strictly evaluate). Consider for example, assuming right now that Numeric specifies operators that return T (i.e. it is not the more sophisticated concept with associated types):

template<Numeric T>

void foo(T a, T b, T c) {

  auto d = a + b + c;

  bar(d); 

}

What happens in the above code where T is something like "int"? decltype(d) would be int. bar is then called with that int. Everything is fine. Now, similarly, what happens if T is a type that yields expression templates from operators? Since the Numeric concept is the more simple one, this code still works, too, even if the result of something like (a+b)+c might normally produce an object that holds on to a temporary operand by reference. This is because the concept specifies that the associated operator + yields T. Sure it's not optimal because the concept used was not a close match, however because decltype(d) ends up being T, the code is safe and works. This is a good thing, since if this did not happen and d ended up as something like an expression-template that held on to operands by reference, we'd now have dangling references and a subtle run-time bug that may not even be caught at compile time.

In practice, if you write generic code that you expect to work both with expression-templates and with basic types, that generic code has to be aware of this, otherwise the code just might not work or may work purely out of coincidence or may appear to work but has bugs that won't show up until run-time if they are even diagnosed at all. This is not an uncommon kind of problem when working with expression template libraries today -- people pass instances of their types and expressions along to templates that were written without expression templates in mind and the mistake is not always obvious. If, in a world with checking, the developer of the generic code used the simple kind of Numeric concept here, the code would be correct.

On a tangent, the proposed operator auto would have also be nice for these cases, even when not inside of a template, but *shrugs*.

[Herb Sutter]

> > In particular, C++0x concepts were never shown to be implementable with
> > sufficient performance to actually be usable in a commercial compiler and user
> > project. The prototype ConceptGCC implementation never overcame a multi-
> > factor (IIRC, 6x) slowdown in recompiling existing code against a concept-ized
> > STL, which is way beyond the "unadoptable" line; that would need to come
> > closer to 20% to even consider again just on performance grounds.
>
> ConceptsGCC was implemented both as a prototype and also at a time when
> things like specialization-lookup in GCC was not constant. IIRC, though this is
> based purely on memory, Doug described certain operations as being
> unnecessarily N^2 when they could have been constant. While there, of course,
> would be compile-time implications due to checking, ConceptGCC was not
> directly indicative of what a modern implementation would see.

I understand that. I talked about this with Doug a number of times last decade, and I know he believes it can be better and so I strongly encouraged him to demonstrate that as an essential work item to keep the proposal alive. It was never done, and until it is the only data we have is that C++0x concepts "were never shown to be implementable with sufficient performance."

OTOH, the implementation of the current proposal has demonstrated that it does bring compile times to roughly the same as unconstrained templates. That was very important for the committee to even look at concepts again.

> For what it's worth, I am not alone in my desire for separate checking. As
> mentioned, there are various people both inside and outside of the committee
> who are or were under the impression that definition-checking will be coming,
> with the misconception that there is a very clear path towards getting it. At the
> very least, those misconceptions need to either be dispelled before a vote or
> otherwise the ability to do definition-checking should be proven. If there is
> consensus that we are really willing to likely give up on getting definition
> checking in a truly meaningful sense, then I'm fine with that even though I
> personally disagree. People at least need to be aware of this in order to make an
> informed decision.

That's true, and we'll do our best to address that question in Jacksonville.

Matt, here's my attempt to distill the essence of what I think you’re saying: We shouldn't put concepts into the standard until definitions can be separately checked, ideally including that template definitions should be checked by default. Is that a fair summary?

Herb

[Matt Austern]

Other Matt here... My interpretation of what he says isn't quite that we should delay concepts until we have definition-side checking, but that we should delay concepts until we are confident that there is a workable upgrade path to definition side checking.

--
You received this message because you are subscribed to the Google Groups "SG8 - Concepts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concepts+u...@isocpp.org.
To post to this group, send email to conc...@isocpp.org.

Visit this group at https://groups.google.com/a/isocpp.org/group/concepts/.

[Matt Calabrese]

On Saturday, February 20, 2016 at 5:47:43 PM UTC-8, austern wrote:

Other Matt here... My interpretation of what he says isn't quite that we should delay concepts until we have definition-side checking, but that we should delay concepts until we are confident that there is a workable upgrade path to definition side checking.

 

Yes, that's pretty accurate. Based on independent assessments by myself, by the developer of ConceptGCC, and by others, it appears unlikely that we will be able to add definition checking incrementally and that if we push Concepts into the language as-is then we probably will not get definition checking at all. This is a very worthwhile point to make as there are people voting under the (in my opinion) mistaken impression that this approach is compatible with definition checking. If they were to vote believing that, then they'd be voting based on a misconception and I don't think anyone wants that.

That said, while I do personally feel that definition checking is very important and I will always push for it and defend it, I'm fully aware that this is not at all a unanimous view. I called that out in my paper, as people I have incredible respect for do not feel as such. If consensus is that people do not care, which may be the case, then this doesn't much matter. However, if the claim is that definition checking realistically will come and people are voting based on that because they consider definition checking to be important, then the ability for Concepts as-is to incrementally add checking needs to be shown, since it is not at all self-evident and it goes against the independent assessment of those with experience. It's too important to be taken on faith. C++0x concepts had sensible definition checking because the very way that constraints were specified was not at all incidental -- that you plan to do checking influences how constraints need to be specified in the language and what those constraints really mean.

[Tomasz]

W dniu niedziela, 21 lutego 2016 00:18:33 UTC+1 użytkownik Matt Calabrese napisał:

If you want the behavior as in your example then the concept you are using wouldn't be a concept operating on T with operators specified to return T. It would be a concept operating on T that has operators that return associated types, where those associated types are convertible to T. Those types model a less-refined version of the concept (assuming expression templates, where the associated types likely wouldn't provide most of the mutating operations that T provides). Note that a concept such as this would also match a type where the operators directly return T such as if T were a double or int.

The point is that this is not the caller that want the specific behavior from the template, but the user (means both callee and the class author). With the proposed approach, if the template author will not foresee every possible use of expression template in their constrains, then it would be not possible to employ such optimization. Or the class author will need to provide another template, with possibly same code.
 

What happens in the above code where T is something like "int"? decltype(d) would be int. bar is then called with that int. Everything is fine. Now, similarly, what happens if T is a type that yields expression templates from operators? Since the Numeric concept is the more simple one, this code still works, too, even if the result of something like (a+b)+c might normally produce an object that holds on to a temporary operand by reference. This is because the concept specifies that the associated operator + yields T. Sure it's not optimal because the concept used was not a close match, however because decltype(d) ends up being T, the code is safe and works. This is a good thing, since if this did not happen and d ended up as something like an expression-template that held on to operands by reference, we'd now have dangling references and a subtle run-time bug that may not even be caught at compile time.

So I am expected to be able to precisely specify return type of each operation, including the possibility of expression template, but I will not be able to spell it inside my function body and will rely on template interference in my actual implementation. That argumentation seems to be a bit contradictory, especially when new concept will need to introduced for newly created generic algorithm.
 

In practice, if you write generic code that you expect to work both with expression-templates and with basic types, that generic code has to be aware of this, otherwise the code just might not work or may work purely out of coincidence or may appear to work but has bugs that won't show up until run-time if they are even diagnosed at all. This is not an uncommon kind of problem when working with expression template libraries today -- people pass instances of their types and expressions along to templates that were written without expression templates in mind and the mistake is not always obvious. If, in a world with checking, the developer of the generic code used the simple kind of Numeric concept here, the code would be correct.

And will be less generic and optimal, compared to exactly same code using non-checked template. Your argumentation is based on the assumption, that user will provide overloads of the same function with different meaning (certainly an design bug) and calling overload specialized for exact return will lead to incorrect behavior. My point is that for well-designed code, the call of specific overload for return will provide the same effect more effectively. Shorter example, if I have following expression in my code:
indentifier(t1) == identifier(t2)
And the identifier function is declared to return std::string const& by constrain. Now if MyClass returns sting_view from this functions, is the conversion necessary or even expected?

[Paul Fultz II]

Matt's paper is making a lot of assumptions about the knowledge and
opinions about people wanting concepts now based on the current model.
Most seem flat wrong. He seems to have missed the last 5 years progress
in the standards committee, or maybe 10.

Oh please. Concepts lite has ignored advances in modern C++ in the last 5
years, which is why we we still need to use type traits and wrap `concept
bool` in a `bool_constant`. Just as the need to use `typename` and `template`
keywords for disambiguation has caused much confusion and headache for new
users, concepts lite combined with modern C++ programming will suffer a
similar fate. However, this is a solvable problem if the committee would take
the time to consider these things.

[Thorsten Ottosen]

Hi Matt,

As I recall there were other problems with the old proposal.

A. ADL based customization points
B. Type checking generic lambda bodies

It seems to me that if we need to return to c++0x concepts and complete that design, then we could have to a decade before anything happens, while risking we can't complete the design satisfactory or implement it efficiently enough.

Can we afford that?

It was quite an eye opener when I compared the specification of the old concepts to the new. The old specification was huge. That level of complexity is just scaring.

There is another route to getting definition checking though. Instead of wiring definition checking into the compiler, it could be a separate tool, perhaps done by using clang. That way the definition checking could be done once in a while, or during nightly builds, without affecting compilation or debug performance for the developer.

Of course, it still requires that definition checking is possible in some form (we can probably live with some obscure cases not being perfectly diagnosed).

Just my two cents

Kind regards

Thorsten

[Paul Fultz II]

On Monday, February 22, 2016 at 1:37:27 PM UTC-6, Thorsten Ottosen wrote:

Hi Matt,

As I recall there were other problems with the old proposal.

A. ADL based customization points
B. Type checking generic lambda bodies

I believe Larisse Voufo has done research with "weak"-binding that helps solve those issues.
 

It seems to me that if we need to return to c++0x concepts and complete that design,

The desire to have semantic-based concepts doesn't mean we have to return to C++0x concepts, nor does a design for semantic-based needs to be complete for concepts to be accepted, just that there is a possible path forward(ideally without deprecating `concept bool`).

 

then we could have to a decade before anything happens, while risking we can't complete the design satisfactory or implement it efficiently enough.

 Can we afford that?

It was quite an eye opener when I compared the specification of the old concepts to the new. The old specification was huge. That level of complexity is just scaring.

However, the scope of C++0x was much larger than the current Concept TS. Furthermore, Doug Gregor had proposed a simplified form of semantic-based concepts for C++ that simplified archetype instantiations and removed concept maps, refinements, axioms, and explicit concepts. It was actually much simpler than the current Concept TS:

https://docs.google.com/viewer?a=v&pid=forums&srcid=MDIyMDc3NjUwMTczOTM0Mjk3NjABMDY0MzIwNzAxMTY2ODcxNjg5ODABTXN4MjR4YjZ5Q01KATAuMQFpc29jcHAub3JnAXYy
 

There is another route to getting definition checking though. Instead of wiring definition checking into the compiler, it could be a separate tool, perhaps done by using clang. That way the definition checking could be done once in a while, or during nightly builds, without affecting compilation or debug performance for the developer.

Of course, it still requires that definition checking is possible in some form (we can probably live with some obscure cases not being perfectly diagnosed).

But how do we know these are really obscure cases without more experience and perhaps a conceptified standard library? Given a little more time with a conceptified STL would help iron-out these issues, and show that definition checking is good enough.
 

[Nevin Liber]

On 22 February 2016 at 14:48, Paul Fultz II <pful...@gmail.com> wrote:

But how do we know these are really obscure cases without more experience and perhaps a conceptified standard library?

I must have missed that paper in the current mailing.  Where can I find this conceptified standard library proposal?

[Paul Fultz II]

On Monday, February 22, 2016 at 3:01:55 PM UTC-6, Nevin Liber wrote:

On 22 February 2016 at 14:48, Paul Fultz II <pful...@gmail.com> wrote:

But how do we know these are really obscure cases without more experience and perhaps a conceptified standard library?

I must have missed that paper in the current mailing.  Where can I find this conceptified standard library proposal?

Well here:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4382.pdf

But I don't know if thats the latest.
 

[Matt Austern]

On Mon, Feb 22, 2016 at 1:42 PM, Paul Fultz II <pful...@gmail.com> wrote:

On Monday, February 22, 2016 at 3:01:55 PM UTC-6, Nevin Liber wrote:

On 22 February 2016 at 14:48, Paul Fultz II <pful...@gmail.com> wrote:

But how do we know these are really obscure cases without more experience and perhaps a conceptified standard library?

I must have missed that paper in the current mailing.  Where can I find this conceptified standard library proposal?

Well here:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4382.pdf

But I don't know if thats the latest.

It does concern me that it's only partial: it covers a conceptified version of clauses 24 and 25, but not clause 23. I don't care very much about the concepts that containers satisfy (defining a good Sequence concept is a research project, and not really a crucial one in the absence of a whole library of Sequence algorithms), but I do care about the concepts that T has to satisfy in order for vector<T> or map<T,U> to be valid. I think it's likely that attempting to constrain containers' arguments will reveal interesting things.

And yes, I do realize I've elided something in what I wrote above.

                             --Matt

[Thorsten Ottosen]

Sendt fra min iPhone

Den 22. feb. 2016 kl. 21.48 skrev Paul Fultz II <pful...@gmail.com>:

On Monday, February 22, 2016 at 1:37:27 PM UTC-6, Thorsten Ottosen wrote:

Hi Matt,

As I recall there were other problems with the old proposal.

A. ADL based customization points
B. Type checking generic lambda bodies

I believe Larisse Voufo has done research with "weak"-binding that helps solve those issues.
 

Ok. Another issue was degrading run-time performance. Using concepts should not mean any negative change to code generation.

It seems to me that if we need to return to c++0x concepts and complete that design,

The desire to have semantic-based concepts doesn't mean we have to return to C++0x concepts, nor does a design for semantic-based needs to be complete for concepts to be accepted, just that there is a possible path forward(ideally without deprecating `concept bool`).

 

then we could have to a decade before anything happens, while risking we can't complete the design satisfactory or implement it efficiently enough.

 Can we afford that?

It was quite an eye opener when I compared the specification of the old concepts to the new. The old specification was huge. That level of complexity is just scaring.

However, the scope of C++0x was much larger than the current Concept TS. Furthermore, Doug Gregor had proposed a simplified form of semantic-based concepts for C++ that simplified archetype instantiations and removed concept maps, refinements, axioms, and explicit concepts. It was actually much simpler than the current Concept TS:

https://docs.google.com/viewer?a=v&pid=forums&srcid=MDIyMDc3NjUwMTczOTM0Mjk3NjABMDY0MzIwNzAxMTY2ODcxNjg5ODABTXN4MjR4YjZ5Q01KATAuMQFpc29jcHAub3JnAXYy
 

I forgot about that. Though I can't tell how large its core language specification would be, I'm willing to believe it would be much smaller. 

Still we don't have a working demo of that + the other research showing no overhead in compile and runtime. And it would still require the wierd late check feature. 

There is another route to getting definition checking though. Instead of wiring definition checking into the compiler, it could be a separate tool, perhaps done by using clang. That way the definition checking could be done once in a while, or during nightly builds, without affecting compilation or debug performance for the developer.

Of course, it still requires that definition checking is possible in some form (we can probably live with some obscure cases not being perfectly diagnosed).

But how do we know these are really obscure cases without more experience and perhaps a conceptified standard library? Given a little more time with a conceptified STL would help iron-out these issues, and show that definition checking is good enough.

I agree it would be good with such an stl if we don't have that already.

I remember a meeting more than 10 years ago. The debate was fierce, and circled around whether the two proposals at that time had certain theoretical properties, specifically about definition checking. I wouldn't say it ended in a fist fight, but it was close ;) I believe there is a difference, but I can't image that the difference can be very large. That just seem implausible ... If the the difference is large it must be easy to show code examples of common code that can't be definition checked. 

Regards

-Thorsten

 

Just my two cents

Kind regards

Thorsten

[Paul Fultz II]

There is another route to getting definition checking though. Instead of wiring definition checking into the compiler, it could be a separate tool, perhaps done by using clang. That way the definition checking could be done once in a while, or during nightly builds, without affecting compilation or debug performance for the developer.

Of course, it still requires that definition checking is possible in some form (we can probably live with some obscure cases not being perfectly diagnosed).

But how do we know these are really obscure cases without more experience and perhaps a conceptified standard library? Given a little more time with a conceptified STL would help iron-out these issues, and show that definition checking is good enough.

I agree it would be good with such an stl if we don't have that already.

I remember a meeting more than 10 years ago. The debate was fierce, and circled around whether the two proposals at that time had certain theoretical properties, specifically about definition checking. I wouldn't say it ended in a fist fight, but it was close ;) I believe there is a difference, but I can't image that the difference can be very large. That just seem implausible ... If the the difference is large it must be easy to show code examples of common code that can't be definition checked. 

The theoretical model for semantic-based concepts has been widely implemented across many languages(such as haskell, scala, rust, swift, etc), so there is strong evidence to its practicality and successfulness. However, I don't know of a language that implements the property model used in Concepts TS(perhaps one exists but I am unaware of it). This is why there is a lot of caution about adding such a feature to the language, since there is still many unanswered questions(especially related to type-safety and definition checking). Whereas the difficult part with semantic-based concepts is retrofitting it in C++, rather than concerns of boxing ourselves in a corner with no path forward.

One major difference between semantic-based concepts and Concept TS, is the pseudo signatures vs usage patterns. Trying to check template expression with usage patterns that are a composition(such as `f(g(x))`) can be quite complicated(perhaps impossible?). Furthermore, usage patterns complicate future language features as well(such as virtual concepts aka type erasure).

Paul
 

[Thorsten Ottosen]

>
> One major difference between semantic-based concepts and Concept TS, is the pseudo signatures vs usage patterns. Trying to check template expression with usage patterns that are a composition(such as `f(g(x))`) can be quite complicated(perhaps impossible?). Furthermore, usage patterns complicate future language features as well(such as virtual concepts aka type erasure).

So what is it that makes this so hard?

I assume we can check g in isolation and f in isolation. Why is the composition hard or impossible?

Just trying to understand.

Regards

Thorsten

[Bjarne Stroustrup]

Please note that - as we have said repeatedly - we know how to do definition checking based on predicates. Gaby Dos Reis did it in his C++-style model language Liz several years ago: Gabriel Dos Reis. A System for Axiomatic Programming. In proceedings of the 2012 Conferences on Intelligent Computer Mathemathics. Bremen, German; July 2012; Springer. Liz is still in academic/research use. This again harks back to the ideas documented in our POPL paper: Gabriel Dos Reis, Bjarne Stroustrup. Specifying C++ Concepts. In International Symposium on Principles of Programming Languages (POPL 2006). Charleston (South Carolina), USA; January 2006. It's not all that hard or expensive in compile time.

For some people the question is not whether we can check definitions against concepts specified in the declaration, it's whether we want to. The usual questions are about telemetry, use-counts, tracing, etc. Should those require interface changes?

[Paul Fultz II]

There is a lot that happens in the transition from `g(x)` to `f`. There is copying, moving, passing by reference, implicit and explicit conversions. And how deeply should the compiler check through conversion and assignments to declare the definition correct?

Then if we want to virtualize the concepts, it becomes even more complicated, because a function table needs to be generated for all these operations, or the user needs to define an additional psuedo-signature to clarify what is to happen at the transition. I saw some initial work on virtual concepts by Zach Laine and the details got pretty hairy with these transitions.
 

[Paul Fultz II]

On Tuesday, February 23, 2016 at 8:38:01 PM UTC-6, Bjarne Stroustrup wrote:

Please note that - as we have said repeatedly - we know how to do definition checking based on predicates. Gaby Dos Reis did it in his C++-style model language Liz several years ago: Gabriel Dos Reis. A System for Axiomatic Programming. In proceedings of the 2012 Conferences on Intelligent Computer Mathemathics. Bremen, German; July 2012; Springer. Liz is still in academic/research use. This again harks back to the ideas documented in our POPL paper: Gabriel Dos Reis, Bjarne Stroustrup. Specifying C++ Concepts. In International Symposium on Principles of Programming Languages (POPL 2006). Charleston (South Carolina), USA; January 2006. It's not all that hard or expensive in compile time.

The Liz language ignores things like implicit conversions, so mapping that directly to C++ maybe more difficult than it seems.
 

For some people the question is not whether we can check definitions against concepts specified in the declaration, it's whether we want to.

If we want user-friendly error messages than we want to check definitions. It weakens the type safety by not checking the usage.
 

[Thorsten Ottosen]

There is a lot that happens in the transition from `g(x)` to `f`. There is copying, moving, passing by reference, implicit and explicit conversions. And how deeply should the compiler check through conversion and assignments to declare the definition correct?

Well, g must satisfy some concept. This include the concept or type of its return type. Similar f must have a type or concept for its argument. I don't see how copying, moving or pass by ref can be a problem. For example if f needs a non-const reference argument, I would expect the concept to say that. If g doesn't guarantee a ref return, then we have an error. 

Anyway, I can see its not trivial. It would still be nice to see a concrete example of code that could not be checked.

Then if we want to virtualize the concepts, it becomes even more complicated, because a function table needs to be generated for all 

It seems to me that we already have a good mechanism for that: normal OO.  Yes, it's rigid and all, but has definition checking built right into it. And it works. If we want to wait for all kinds of such proposals to be done before we can move forward on concepts, I predict that we will never get concepts. 

Regards

Thorsten 

[Thorsten Ottosen]

 

For some people the question is not whether we can check definitions against concepts specified in the declaration, it's whether we want to.

If we want user-friendly error messages than we want to check definitions. It weakens the type safety by not checking the usage.

As Bjarne said, the cost is high for that. We can't insert debug statements. The world of templates will be divided into two separate worlds. 

That's why I suggested that definition checking should be a tool, not wired into the compiler.

Regards

Thorsten 

[Vicente J. Botet Escriba]

If we consider that check definition is not good for some cases, maybe the language proposal should consider that, and let the user add their traces and disable the check definition in some way, by block, statement, expression ....

Nevertheless, I believe that we would want check definition most of the time, but I could change my point of view, if checking them will slow a lot the compilation time.

Just my 2cts
Vicente

[Tony V E]

In addition to non-checked blocks, etc, you want classes and types that are invisible to checking - ie the logger class. 

So the template author doesn't need to do anything special to use the logger. 

Maybe.  

Sent from my BlackBerry portable Babbage Device

From: Vicente J. Botet Escriba Sent: Wednesday, February 24, 2016 6:50 PM To: conc...@isocpp.org Reply To: conc...@isocpp.org Subject: Re: [concepts] Is there really no path forward for definition checking of templates?

[Bjarne Stroustrup]

Don't change your point of view quite yet. We know we can do checking. I actually think that it wouldn't be that expensive, but we need more experience. Some people think in terms of opt-in, some in terms of opt-out, and some in terms of defining specific loop-holes. It's next on some to-do lists. The most important point is that we provide concepts as a general, flexible, precise, easy-to-use specification tool. Please note that what I say is nothing new.

For now, we can enjoy good specification and call-interface checking.

Just my 2cts
Vicente

[Vicente J. Botet Escriba]

Le 25/02/2016 01:28, Bjarne Stroustrup a écrit :

On 2/24/2016 6:50 PM, Vicente J. Botet Escriba wrote:

Le 24/02/2016 23:02, Thorsten Ottosen a écrit :

 

For some people the question is not whether we can check definitions against concepts specified in the declaration, it's whether we want to.

If we want user-friendly error messages than we want to check definitions. It weakens the type safety by not checking the usage.

As Bjarne said, the cost is high for that. We can't insert debug statements. The world of templates will be divided into two separate worlds. 

That's why I suggested that definition checking should be a tool, not wired into the compiler.

If we consider that check definition is not good for some cases, maybe the language proposal should consider that, and let the user add their traces and disable the check definition in some way, by block, statement, expression ....

Nevertheless, I believe that we would want check definition most of the time, but I could change my point of view, if checking them will slow a lot the compilation time.

Don't change your point of view quite yet. We know we can do checking. I actually think that it wouldn't be that expensive, but we need more experience. Some people think in terms of opt-in, some in terms of opt-out, and some in terms of defining specific loop-holes. It's next on some to-do lists. The most important point is that we provide concepts as a general, flexible, precise, easy-to-use specification tool. Please note that what I say is nothing new.

My point was that I want definition-checking defined by the language, not by an external tool as Thorsten was suggesting.

I said *most of the time*, as I could say, in general we want definition checking. If definition-checking is expensive we will want it disabled by default instead of enable by default. If it is not expensive we will need to opt-out in some rare and specific cases. 


Vicente

[Bjarne Stroustrup]

On 2/27/2016 1:46 PM, Vicente J. Botet Escriba wrote:

Le 25/02/2016 01:28, Bjarne Stroustrup a écrit :

On 2/24/2016 6:50 PM, Vicente J. Botet Escriba wrote:

Le 24/02/2016 23:02, Thorsten Ottosen a écrit :

 

For some people the question is not whether we can check definitions against concepts specified in the declaration, it's whether we want to.

If we want user-friendly error messages than we want to check definitions. It weakens the type safety by not checking the usage.

As Bjarne said, the cost is high for that. We can't insert debug statements. The world of templates will be divided into two separate worlds. 

That's why I suggested that definition checking should be a tool, not wired into the compiler.

If we consider that check definition is not good for some cases, maybe the language proposal should consider that, and let the user add their traces and disable the check definition in some way, by block, statement, expression ....

Nevertheless, I believe that we would want check definition most of the time, but I could change my point of view, if checking them will slow a lot the compilation time.

Don't change your point of view quite yet. We know we can do checking. I actually think that it wouldn't be that expensive, but we need more experience. Some people think in terms of opt-in, some in terms of opt-out, and some in terms of defining specific loop-holes. It's next on some to-do lists. The most important point is that we provide concepts as a general, flexible, precise, easy-to-use specification tool. Please note that what I say is nothing new.

My point was that I want definition-checking defined by the language, not by an external tool as Thorsten was suggesting.

I agree. *If* I want definition checking, I want to compiler to do it.

I said *most of the time*, as I could say, in general we want definition checking. If definition-checking is expensive we will want it disabled by default instead of enable by default. If it is not expensive we will need to opt-out in some rare and specific cases. 

I don't think that we yet know exactly what "we" want. That's a discussion for next year or so, and a topic for experimentation.

[John Spicer]

On Feb 27, 2016, at 2:30 PM, Bjarne Stroustrup <bja...@stroustrup.com> wrote:

On 2/27/2016 1:46 PM, Vicente J. Botet Escriba wrote:

Le 25/02/2016 01:28, Bjarne Stroustrup a écrit :

On 2/24/2016 6:50 PM, Vicente J. Botet Escriba wrote:

Le 24/02/2016 23:02, Thorsten Ottosen a écrit :

 

For some people the question is not whether we can check definitions against concepts specified in the declaration, it's whether we want to.

If we want user-friendly error messages than we want to check definitions. It weakens the type safety by not checking the usage.

As Bjarne said, the cost is high for that. We can't insert debug statements. The world of templates will be divided into two separate worlds. 

That's why I suggested that definition checking should be a tool, not wired into the compiler.

If we consider that check definition is not good for some cases, maybe the language proposal should consider that, and let the user add their traces and disable the check definition in some way, by block, statement, expression ....

Nevertheless, I believe that we would want check definition most of the time, but I could change my point of view, if checking them will slow a lot the compilation time.

Don't change your point of view quite yet. We know we can do checking. I actually think that it wouldn't be that expensive, but we need more experience. Some people think in terms of opt-in, some in terms of opt-out, and some in terms of defining specific loop-holes. It's next on some to-do lists. The most important point is that we provide concepts as a general, flexible, precise, easy-to-use specification tool. Please note that what I say is nothing new.

My point was that I want definition-checking defined by the language, not by an external tool as Thorsten was suggesting.

I agree. *If* I want definition checking, I want to compiler to do it.

I said *most of the time*, as I could say, in general we want definition checking. If definition-checking is expensive we will want it disabled by default instead of enable by default. If it is not expensive we will need to opt-out in some rare and specific cases. 

I don't think that we yet know exactly what "we" want. That's a discussion for next year or so, and a topic for experimentation.

Agreed.

I think, in principal, we’d all like definition checking, but in the end that depends on what form that might take.

We tried to go for the whole ball of wax in C++11 concepts.   If the implementation of that could have continued to support the evolving definition that was in the working draft, we may or may not have come up with something workable.   But if there is any lesson from that effort it would be that definition checking is a *very* difficult problem.

I think the current concepts TS solves the most serious problems that users encounter using template libraries today.    Today you have to follow the documentation about what your types are supposed to do, and if you fail you get some obscure error N levels down in someone else’s template code.   The concepts TS puts the burden on the library provider to get the interface description right, and if they fail to do so, you can still end up with that obscure error case.   But we at least have the ability to detect most problems, and library providers can tweak their concepts if they discover holes.

My main concern is making sure we have the interface checking part right.

John.