Questions on N4430 (Core Issue 1776: Replacement of class objects containing reference members)

[Kazutoshi Satoda]

I have some questions on the paper.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4430.html

How did it taken by the committee?
Rejected or accepted? I couldn't find any mention about this paper
even in this very detailed trip report.
https://botondballo.wordpress.com/2015/06/05/trip-report-c-standards-meeting-in-lenexa-may-2015/
I'm feeling like that the change in the paper will make new
incompatibilities between C and C++, and then that would be pointed out
by the committee.

What the changes do for reference members?
There is a change addressing the issue with const members (Change in 3.8
(basic.life) paragraph 9). But it seems that replacement of class
objects containing reference members (the title issue) is still not
allowed because 3.8 [basic.life] paragraph 7 is not changed. Changes on
pointer comparisons seems like irrelevant to the issue.

--
k_satoda

[Jens Maurer]

On 06/27/2015 08:32 AM, Kazutoshi Satoda wrote:
> I have some questions on the paper.
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4430.html
>
> How did it taken by the committee?
> Rejected or accepted? I couldn't find any mention about this paper
> even in this very detailed trip report.

The paper was discussed in the core working group. It needs a bit
more polishing before it can go to adoption by the full committee.
My feeling was that the general sentiment in the room (NOT the full
committee) was positive about the direction. This paper essentially
codifies that you can't access a complete object using a pointer
that pointed originally to some other complete object, which is
very useful for optimizers.

If an update comes along for Kona, I'd expect the paper to be
adopted there and then.

> https://botondballo.wordpress.com/2015/06/05/trip-report-c-standards-meeting-in-lenexa-may-2015/
> I'm feeling like that the change in the paper will make new
> incompatibilities between C and C++, and then that would be pointed out
> by the committee.

There was no mention of C incompatibilities (that I remember). In practice,
reasonable programs will be entirely unaffected by this change.

> What the changes do for reference members?
> There is a change addressing the issue with const members (Change in 3.8
> (basic.life) paragraph 9). But it seems that replacement of class
> objects containing reference members (the title issue) is still not
> allowed because 3.8 [basic.life] paragraph 7 is not changed. Changes on
> pointer comparisons seems like irrelevant to the issue.

I've e-mailed the paper author about that.

Thanks,
Jens

[Kazutoshi Satoda]

On 2015/06/28 3:36, Jens Maurer wrote:
> On 06/27/2015 08:32 AM, Kazutoshi Satoda wrote:
>> I have some questions on the paper.
>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4430.html
>>
>> How did it taken by the committee?

...

> The paper was discussed in the core working group. It needs a bit
> more polishing before it can go to adoption by the full committee.
> My feeling was that the general sentiment in the room (NOT the full
> committee) was positive about the direction. This paper essentially
> codifies that you can't access a complete object using a pointer
> that pointed originally to some other complete object, which is
> very useful for optimizers.

I think it's fine on that essential point.

> If an update comes along for Kona, I'd expect the paper to be
> adopted there and then.

...

>> I'm feeling like that the change in the paper will make new
>> incompatibilities between C and C++, and then that would be pointed out
>> by the committee.
>
> There was no mention of C incompatibilities (that I remember). In practice,
> reasonable programs will be entirely unaffected by this change.

Then I'll add some more details about my concerns bellow.


First concern with C incompatibility is that the paper says "malloc
alone is not sufficient to create an object" as "the status quo". But
in C, malloc (or other allocation functions) is the only way to create
objects with dynamic storage duration. If changes (proposed and
foreseen) based on that "status quo" are accepted, many C programs will
be seen invalid with C++ compiler and may be broken silently.


Second one is that the paper seems banning casts between pointer to a
class and pointer to its first subobject. There seems to be some C
programs that emulate class inheritance with such casts. For example:
https://stackoverflow.com/questions/415452/object-orientation-in-c/415536#415536
> struct base
> {
> /* base class members */
> };
>
> struct derived
> {
> struct base super;
> /* derived class members */
> };
>
> struct derived d;
> struct base *base_ptr = (struct base *)&d; // upcast
> struct derived *derived_ptr = (struct derived *)base_ptr; // downcast

I think these are valid C program, and C++ also allow them with some
special rules to keep compatibility with C. The paper seems like
removing one of them.

"Change in 3.9.2 (basic.compound) paragraph 3" deletes this part of
the current standard.
> If an object of type T is located at an address A, a pointer of type cv
> T* whose value is the address A is said to point to that object,
> regardless of how the value was obtained.

"Change in 9.2 class.mem paragraph 19" adds a note that is presumably
derived from the above deletion.
> If a standard-layout class object has any non-static data members, its
> address is the same as the address of its first non-static data member.
...
> [ Note Nonetheless, if a pointer to the object is cast to a pointer to
> the type of its first subobject (or vice versa), the resulting pointer does
> not point to the subobject and can not be used to access it. —end note ]

With these changes, the result of "upcast"/"downcast" in the above
quoted code no longer "points to" the subobject nor the enclosing object,
thus unusable to access them.


I remember some points like these were discussed in SG12, but I can't
find the conclusion.
http://www.open-std.org/pipermail/ub/2014-January/thread.html
Did the discussion continued in other place and reached to a conclusion?

>> What the changes do for reference members?

...

> I've e-mailed the paper author about that.

Thank you.

Is it good to send CC to the e-mail address in paper to post about a
paper? (I think it is good, but I forgot to do that this time.)


--
k_satoda

[Richard Smith]

On Sat, Jun 27, 2015 at 8:52 PM, Kazutoshi Satoda <k_sa...@f2.dion.ne.jp> wrote:

On 2015/06/28 3:36, Jens Maurer wrote:
> On 06/27/2015 08:32 AM, Kazutoshi Satoda wrote:
>> I have some questions on the paper.
>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4430.html

Regarding 3.8/7: I don't see that any changes are required there. It continues to be the case that replacing an object of a class with reference or const members invalidates all pointers, references, and names that refer to the original object, in the sense that they cannot be used to access the new object. Accessing the new object starting from an old pointer/reference/name requires use of std::launder.

This paper does not change anything here. In C++, malloc has never created an object because [intro.object]/1 says that an object is *only* created by a new-expression, declaration, or when an implementation creates a temporary object (malloc is none of these). I agree that this is a problem, and it's very common to see programs do this:

  struct TrivialThing { int a, b, c; };

  TrivialThing *p = reinterpret_cast<TrivialThing*>(malloc());

  p->a = 0; // UB, no object of type TrivialThing here

 

I'll reiterate what the paper says: there is no change to the status quo here; the standard previously said this was UB, and still does. Fixing this (if the committee agrees that it is a bug) should be done by a separate paper.

Second one is that the paper seems banning casts between pointer to a
class and pointer to its first subobject. There seems to be some C
programs that emulate class inheritance with such casts. For example:
https://stackoverflow.com/questions/415452/object-orientation-in-c/415536#415536
> struct base
> {
>     /* base class members */
> };
>
> struct derived
> {
>     struct base super;
>     /* derived class members */
> };
>
> struct derived d;
> struct base *base_ptr = (struct base *)&d;  // upcast
> struct derived *derived_ptr = (struct derived *)base_ptr;  // downcast

I think these are valid C program, and C++ also allow them with some
special rules to keep compatibility with C. The paper seems like
removing one of them.

This was the subject of some debate. The previous draft of the paper intended to allow such casts in the cases where the standard guaranteed the addresses are the same (indeed, that seemed to be the purpose of such guarantees). We backed off this when it was observed that an existing C++ optimizer was already optimizing on the basis of such casts being invalid, based on a (reasonable) reading of the current wording of the standard.

This probably warrants more debate by CWG. If possible, I'd like to guarantee that casting between a pointer-to-standard-layout-class and its first element gives a usable pointer value, and likewise for casting between a pointer to an array and its first element and for casting between a pointer to a union and any element type. This seems to be the intent of the current standard wording, even if a strict interpretation of the words could give another conclusion.

This author reads std-proposals :) But yes, as general advice, you should contact the author of the paper when providing feedback; that's why the papers contain an email address for the author.

[Kazutoshi Satoda]

On 2015/06/28 16:11, Richard Smith wrote:
> Regarding 3.8/7: I don't see that any changes are required there. It
> continues to be the case that replacing an object of a class with reference
> or const members invalidates all pointers, references, and names that refer
> to the original object, in the sense that they cannot be used to access the
> new object. Accessing the new object starting from an old
> pointer/reference/name requires use of std::launder.

OK I got the idea of that. I should re-read the paper. Thank you.

If possible, I want to see a newer version of the paper or list of
feedbacks already given.

>> First concern with C incompatibility is that the paper says "malloc
>> alone is not sufficient to create an object" as "the status quo". But

...

> This paper does not change anything here. In C++, malloc has never created
> an object because [intro.object]/1 says that an object is *only* created by
> a new-expression, declaration, or when an implementation creates a
> temporary object (malloc is none of these). I agree that this is a problem,
> and it's very common to see programs do this:
>
> struct TrivialThing { int a, b, c; };
> TrivialThing *p = reinterpret_cast<TrivialThing*>(malloc());
> p->a = 0; // UB, no object of type TrivialThing here
>
> I'll reiterate what the paper says: there is no change to the status quo
> here; the standard previously said this was UB, and still does. Fixing this
> (if the committee agrees that it is a bug) should be done by a separate
> paper.

I'm seeing one more problem that the above code can be said to have
defined behavior based on 3.8 [basic.life] p1 (as discussed in SG12).

But yes, if the approval of this paper doesn't affect on that, it is a
separate issue. (#1116 <http://wg21.cmeerw.net/cwg/issue1116>, or new
one...)

>> Second one is that the paper seems banning casts between pointer to a
>> class and pointer to its first subobject. There seems to be some C

...

> This was the subject of some debate. The previous draft of the paper
> intended to allow such casts in the cases where the standard guaranteed the
> addresses are the same (indeed, that seemed to be the purpose of such
> guarantees). We backed off this when it was observed that an existing C++
> optimizer was already optimizing on the basis of such casts being invalid,
> based on a (reasonable) reading of the current wording of the standard.
>
> This probably warrants more debate by CWG. If possible, I'd like to
> guarantee that casting between a pointer-to-standard-layout-class and its
> first element gives a usable pointer value, and likewise for casting
> between a pointer to an array and its first element and for casting between
> a pointer to a union and any element type. This seems to be the intent of
> the current standard wording, even if a strict interpretation of the words
> could give another conclusion.

This is a news for me. Could you please point the relevant wording, and
hopefully an example with such optimizer (compiler and version, etc)?

>> Is it good to send CC to the e-mail address in paper to post about a
>> paper? (I think it is good, but I forgot to do that this time.)
>
> This author reads std-proposals :) But yes, as general advice, you should
> contact the author of the paper when providing feedback; that's why the
> papers contain an email address for the author.

OK. I'll do that next time.

--
k_satoda

[Richard Smith]

On Sun, Jun 28, 2015 at 7:37 AM, Kazutoshi Satoda <k_sa...@f2.dion.ne.jp> wrote:

On 2015/06/28 16:11, Richard Smith wrote:
> Regarding 3.8/7: I don't see that any changes are required there. It
> continues to be the case that replacing an object of a class with reference
> or const members invalidates all pointers, references, and names that refer
> to the original object, in the sense that they cannot be used to access the
> new object. Accessing the new object starting from an old
> pointer/reference/name requires use of std::launder.

OK I got the idea of that. I should re-read the paper. Thank you.

If possible, I want to see a newer version of the paper or list of
feedbacks already given.

The committee's internal discussions on a paper are not public; this is deliberate as it allows us to talk more freely about issues that (for corporate, personal or legal reasons) we would not wish to disclose more widely (and it allows us to express views and opinions without fear of being publicly embarrassed when we inevitably make mistakes!).

I'm not sure exactly when I'll produce a newer version of the wording; I expect there'll be at least one core drafting review teleconference between now and Kona, and I should have revised wording for that review. I'll try to remember to post the wording here in advance of that.

I'll check with the person from whom I received the information whether it's OK to make that public.

>> Is it good to send CC to the e-mail address in paper to post about a
>> paper? (I think it is good, but I forgot to do that this time.)
>
> This author reads std-proposals :) But yes, as general advice, you should
> contact the author of the paper when providing feedback; that's why the
> papers contain an email address for the author.

OK. I'll do that next time.

--
k_satoda

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[Myriachan]

On Sunday, June 28, 2015 at 12:11:49 AM UTC-7, Richard Smith wrote:

On Sat, Jun 27, 2015 at 8:52 PM, Kazutoshi Satoda <k_sa...@f2.dion.ne.jp> wrote:

First concern with C incompatibility is that the paper says "malloc
alone is not sufficient to create an object" as "the status quo". But
in C, malloc (or other allocation functions) is the only way to create
objects with dynamic storage duration. If changes (proposed and
foreseen) based on that "status quo" are accepted, many C programs will
be seen invalid with C++ compiler and may be broken silently.

This paper does not change anything here. In C++, malloc has never created an object because [intro.object]/1 says that an object is *only* created by a new-expression, declaration, or when an implementation creates a temporary object (malloc is none of these). I agree that this is a problem, and it's very common to see programs do this:

  struct TrivialThing { int a, b, c; };

  TrivialThing *p = reinterpret_cast<TrivialThing*>(malloc());

  p->a = 0; // UB, no object of type TrivialThing here

 

I'll reiterate what the paper says: there is no change to the status quo here; the standard previously said this was UB, and still does. Fixing this (if the committee agrees that it is a bug) should be done by a separate paper.

Why is that undefined behavior in the current Standard?  [basic.life]/1:

The lifetime of an object is a runtime property of the object.  An object is said to have non-vacuous initialization if it is of a class or aggregate type and it or one of its members is initialized by a constructor other than a trivial default constructor.  [Note: initialization by a trivial copy/move constructor is non-vacuous initialization. — end note]  The lifetime of an object of type T begins when:

— storage with the proper alignment and size for type T is obtained; and
— if the object has non-vacuous initialization, its initialization is complete.

TrivialThing has vacuous initialization.  malloc(sizeof(TrivialThing)) allocates storage of the proper size.  Only alignment remains as a concern, but without an alignas attribute, it isn't possible for a standard-layout structure to require an alignment greater than that guaranteed by malloc()'s definition.

If the argument is then about it not being "an object" until a new expression, that interpretation leads to a contradiction: how can the Standard talk about an object before its existence?  It'd imply a contradiction in the Standard.


Looking at this, this proposal looks extremely bad for the future of C++, unless the intent is to fork C++ into a new managed language.  The ability to do some of the pointer manipulation banned by the proposal is relied upon by way too much code.  In fact, C++ programmers are used to using these mechanisms to get work done, because it's sometimes the best tool for the job.

The above is extremely broken in particular: it means that many C libraries with such statements in macros or inline functions in header files can't be used in C++ programs anymore.  If you try to use one of these C functions, the compiler will just willy-nilly delete your code from existence without warning.

Then there is the matter of pool allocators: with such strict rules on pointer manipulation, it's impossible to write a meaningful pool allocator or similar construction.  Unless you want to write such things in C since C++ would be too strict to allow it.

Optimization is not the only goal — usability, versatility, intelligibility and self-consistency are also goals.

> struct derived d;
> struct base *base_ptr = (struct base *)&d;  // upcast
> struct derived *derived_ptr = (struct derived *)base_ptr;  // downcast

I think these are valid C program, and C++ also allow them with some
special rules to keep compatibility with C. The paper seems like
removing one of them.

This was the subject of some debate. The previous draft of the paper intended to allow such casts in the cases where the standard guaranteed the addresses are the same (indeed, that seemed to be the purpose of such guarantees). We backed off this when it was observed that an existing C++ optimizer was already optimizing on the basis of such casts being invalid, based on a (reasonable) reading of the current wording of the standard.

This probably warrants more debate by CWG. If possible, I'd like to guarantee that casting between a pointer-to-standard-layout-class and its first element gives a usable pointer value, and likewise for casting between a pointer to an array and its first element and for casting between a pointer to a union and any element type. This seems to be the intent of the current standard wording, even if a strict interpretation of the words could give another conclusion.

 

Without support for this, struct sockaddr does not work.

Melissa

[Richard Smith]

On Thu, Jul 9, 2015 at 12:25 PM, Myriachan <myri...@gmail.com> wrote:

On Sunday, June 28, 2015 at 12:11:49 AM UTC-7, Richard Smith wrote:

On Sat, Jun 27, 2015 at 8:52 PM, Kazutoshi Satoda <k_sa...@f2.dion.ne.jp> wrote:

First concern with C incompatibility is that the paper says "malloc
alone is not sufficient to create an object" as "the status quo". But
in C, malloc (or other allocation functions) is the only way to create
objects with dynamic storage duration. If changes (proposed and
foreseen) based on that "status quo" are accepted, many C programs will
be seen invalid with C++ compiler and may be broken silently.

This paper does not change anything here. In C++, malloc has never created an object because [intro.object]/1 says that an object is *only* created by a new-expression, declaration, or when an implementation creates a temporary object (malloc is none of these). I agree that this is a problem, and it's very common to see programs do this:

  struct TrivialThing { int a, b, c; };

  TrivialThing *p = reinterpret_cast<TrivialThing*>(malloc());

  p->a = 0; // UB, no object of type TrivialThing here

 

I'll reiterate what the paper says: there is no change to the status quo here; the standard previously said this was UB, and still does. Fixing this (if the committee agrees that it is a bug) should be done by a separate paper.

Why is that undefined behavior in the current Standard?  [basic.life]/1:

The lifetime of an object is a runtime property of the object.  An object is said to have non-vacuous initialization if it is of a class or aggregate type and it or one of its members is initialized by a constructor other than a trivial default constructor.  [Note: initialization by a trivial copy/move constructor is non-vacuous initialization. — end note]  The lifetime of an object of type T begins when:

— storage with the proper alignment and size for type T is obtained; and
— if the object has non-vacuous initialization, its initialization is complete.

TrivialThing has vacuous initialization.

"Vacuous initialization" is a property of an object, not a type, per the text you quoted. The above code does not create an object of type TrivialType.

malloc(sizeof(TrivialThing)) allocates storage of the proper size.  Only alignment remains as a concern, but without an alignas attribute, it isn't possible for a standard-layout structure to require an alignment greater than that guaranteed by malloc()'s definition.

If the argument is then about it not being "an object" until a new expression, that interpretation leads to a contradiction: how can the Standard talk about an object before its existence?  It'd imply a contradiction in the Standard.

There's only a contradiction if you ignore [basic.life]/1 and imagine that an object is created in the above case. To reiterate what I said before: I too think this is a problem, but it's a pre-existing problem.

Looking at this, this proposal looks extremely bad for the future of C++, unless the intent is to fork C++ into a new managed language.

I think you are mistaken about the effect of the change. C++ has always had stronger semantic rules than many people imagined; there are a lot of constructs that implementations intentionally permit despite them not having defined behavior (and more that are permitted simply because that is the current emergent behavior of the implementations in question). Some of these rules should probably be weakened, but that is orthogonal to this proposal.

Fundamentally, I think you're right to complain, but your complaint is misdirected: you shouldn't be against this proposal simply because it has brought a pre-existing problem to your attention. The right thing to do here is to work out what the rules should be, and to work towards allowing the reasonable constructs that people currently assume should work but the standard disallows (following the example of n3751).

The ability to do some of the pointer manipulation banned by the proposal is relied upon by way too much code.  In fact, C++ programmers are used to using these mechanisms to get work done, because it's sometimes the best tool for the job.

The above is extremely broken in particular: it means that many C libraries with such statements in macros or inline functions in header files can't be used in C++ programs anymore.  If you try to use one of these C functions, the compiler will just willy-nilly delete your code from existence without warning.

Then there is the matter of pool allocators: with such strict rules on pointer manipulation, it's impossible to write a meaningful pool allocator or similar construction.  Unless you want to write such things in C since C++ would be too strict to allow it.

Optimization is not the only goal — usability, versatility, intelligibility and self-consistency are also goals.

> struct derived d;
> struct base *base_ptr = (struct base *)&d;  // upcast
> struct derived *derived_ptr = (struct derived *)base_ptr;  // downcast

I think these are valid C program, and C++ also allow them with some
special rules to keep compatibility with C. The paper seems like
removing one of them.

This was the subject of some debate. The previous draft of the paper intended to allow such casts in the cases where the standard guaranteed the addresses are the same (indeed, that seemed to be the purpose of such guarantees). We backed off this when it was observed that an existing C++ optimizer was already optimizing on the basis of such casts being invalid, based on a (reasonable) reading of the current wording of the standard.

This probably warrants more debate by CWG. If possible, I'd like to guarantee that casting between a pointer-to-standard-layout-class and its first element gives a usable pointer value, and likewise for casting between a pointer to an array and its first element and for casting between a pointer to a union and any element type. This seems to be the intent of the current standard wording, even if a strict interpretation of the words could give another conclusion.

 

Without support for this, struct sockaddr does not work.

Melissa

--

[Kazutoshi Satoda]

On 2015/06/29 5:17, Richard Smith wrote:
> The committee's internal discussions on a paper are not public; this is
> deliberate as it allows us to talk more freely about issues that (for
> corporate, personal or legal reasons) we would not wish to disclose more
> widely (and it allows us to express views and opinions without fear of
> being publicly embarrassed when we inevitably make mistakes!).

Fair enough. But I believe decisions from such discussions and their
rationales will be recored in public papers.

The current version (N4430) seems too quiet on that. I expect revised
one will include answers to given questions and possibly some more which
were already solved by yourself or in the committee.

>>>> Second one is that the paper seems banning casts between pointer to a
>>>> class and pointer to its first subobject. There seems to be some C
...

>>> guarantees). We backed off this when it was observed that an existing C++
>>> optimizer was already optimizing on the basis of such casts being invalid,
>>> based on a (reasonable) reading of the current wording of the standard.

...

>> This is a news for me. Could you please point the relevant wording, and
>> hopefully an example with such optimizer (compiler and version, etc)?
>
> I'll check with the person from whom I received the information whether
> it's OK to make that public.

Could you please just point the relevant wording in the standard?

I understand compiler information would be non-public, but wordings in
the standard should not be a problem.

I suspect that wording could be a defect and better to fix, meaning to
ban the above mentioned relatively unknown optimization, instead of
banning already wide spread use of the casts.


I also wonder if it is possible or not to plug the effect of
std::launder() to some use of built-in cast operators and union member
access which are expected to be already written in lifetime
manipulations like optional<> do. That will reduce some chance of
optimization, but sounds safer direction and encouraging people to avoid
such cast for efficiency.

--
k_satoda

[Kazutoshi Satoda]

Ping?

On 2015/07/11 16:44, Kazutoshi Satoda wrote:
...

> On 2015/06/29 5:17, Richard Smith wrote:

...

>>>>> Second one is that the paper seems banning casts between pointer to a
>>>>> class and pointer to its first subobject. There seems to be some C
> ...
>>>> guarantees). We backed off this when it was observed that an existing C++
>>>> optimizer was already optimizing on the basis of such casts being invalid,
>>>> based on a (reasonable) reading of the current wording of the standard.
> ...
>>> This is a news for me. Could you please point the relevant wording, and
>>> hopefully an example with such optimizer (compiler and version, etc)?
>>
>> I'll check with the person from whom I received the information whether
>> it's OK to make that public.
>
> Could you please just point the relevant wording in the standard?
>
> I understand compiler information would be non-public, but wordings in
> the standard should not be a problem.
>
> I suspect that wording could be a defect and better to fix, meaning to
> ban the above mentioned relatively unknown optimization, instead of
> banning already wide spread use of the casts.
>
>
> I also wonder if it is possible or not to plug the effect of
> std::launder() to some use of built-in cast operators and union member
> access which are expected to be already written in lifetime
> manipulations like optional<> do. That will reduce some chance of
> optimization, but sounds safer direction and encouraging people to avoid
> such cast for efficiency.

One more question: Where std::launder() is expected to be written?
Is it written in library function (operator* of optional, operator[] of
vector, ...) or user code which use such functions with concerned types?

--
k_satoda