[std-discussion]why placement new not allowed in constexpr?

[Larry Evans]

The following code fails to compile for reasons cited
in the comments. However, since placement new only
affects a non-static member variable, it's unclear why
the placement new use is not allowed.

Could someone please help me understand why
placement new is not allowed here?

TIA.

-regards,
Larry

-{--cut here--
//Purpose:
// Demonstrate limitations of using placement new
// in a constexpr.
//References:
// [expr.const.placenew]
// Paper:
// https://isocpp.org/files/papers/N3797.pdf
// Section:
// 5.19 Constant Expresssions [expr.const]
// Paragraph:
// 2
// Bullet starting with:
// a new-expression (5.3.4)
#define USE_PLACEMENT_NEW
#ifdef USE_PLACEMENT_NEW
#include <new>
#endif
template<typename T>
struct place_new
{
T t_p[1];
constexpr place_new(T val)
: t_p{0}
{
#ifdef USE_PLACEMENT_NEW
new(t_p) T(val+1);//violates [expr.const.placenew]
#else
*t_p=val+1;
#endif
}
constexpr T get()const
{
return t_p[0];
}
};
template<int Val>
struct int_const
{
static int const value=Val;
};
int main()
{
constexpr place_new<int> pn(1);
using ic=int_const<pn.get()>;//test if really a constexpr
return ic::value;
}

-}--cut here--

[Richard Smith]

On Mon, Apr 13, 2015 at 2:35 AM, Larry Evans <cpplj...@suddenlink.net> wrote:

The following code fails to compile for reasons cited
in the comments.  However, since placement new only
affects a non-static member variable, it's unclear why
the placement new use is not allowed.

Placement new is not allowed because it would allow constant expression evaluation to interact with the underlying storage of objects (which need not exist and need not be modeled the same way as at runtime).

 

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org.
To post to this group, send email to std-dis...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-discussion/.

[David Krauss]

On 2015–04–13, at 6:16 PM, Richard Smith <ric...@metafoo.co.uk> wrote:

On Mon, Apr 13, 2015 at 2:35 AM, Larry Evans <cpplj...@suddenlink.net> wrote:

The following code fails to compile for reasons cited
in the comments.  However, since placement new only
affects a non-static member variable, it's unclear why
the placement new use is not allowed.

Placement new is not allowed because it would allow constant expression evaluation to interact with the underlying storage of objects (which need not exist and need not be modeled the same way as at runtime).

Put another way, new( char_ptr ) Thing intrinsically includes an operation like reinterpret_cast< Thing * >( char_ptr ). The cast is illegal because you can still access the char_ptr array, and new is illegal by implication.

I wonder if it would be feasible to poison the array to sidestep the issue. Something like, in a constant evaluation context:

1. Casting to void* is not allowed except from a char* value referencing the first element of an array which is a constant expression in the current context. (Note, the relevant reinterpret_cast is defined in terms of static_casts to and from void*.)

2. Casting from cv void* is not allowed unless the value was produced by a cast from char*.

3. Accessing an array with a subobject (character) that has participated in such a cast in not allowed.

4. Casting the array again to a different type is not allowed.

Also allowing the opposite cast, from an object pointer to a char* invisibly associated with its type, would provide consistency and enable constexpr std::addressof, which is currently a minor hole.

However, having constructed such an object, you wouldn’t be able to destroy it because there are no constexpr explicit destructor calls. (Pedantically, the standard only says pseudo-destructor calls (§5.2.4) are not allowed, and those are a different animal than explicit destructor calls (§12.4/13), but that’s certainly a defect.) Those would need to be allowed as well, and then the interpreter has to keep a bit of state for each object to remember whether it’s already been destroyed.

I think just a few pieces of additional state would be sufficient to get this working:

1. For the representation of char array constants, remember what object (if any) it’s been cast to or from. This reference could be stored in place of the array value. For extra generosity, provide for a sequence of objects at offsets within the array.

2. For all other constant, complete objects, store a potential reference to the (inaccessible, stubbed-out) char array of its object representation.

3. Provide some means of navigating from subobjects to complete objects.

4. Let each void * constant value have an underlying char * value.

5. Allow char * constant values to refer to any target, not necessarily a char.

Support for all this would be somewhat useful alone, and a step toward predynamic storage. The added complexity is substantial, but still less than emulating all object representation, and nothing to kill performance. Unless I’ve missed something.

[Richard Smith]

On Mon, Apr 13, 2015 at 9:56 AM, David Krauss <pot...@gmail.com> wrote:

On 2015–04–13, at 6:16 PM, Richard Smith <ric...@metafoo.co.uk> wrote:

On Mon, Apr 13, 2015 at 2:35 AM, Larry Evans <cpplj...@suddenlink.net> wrote:

The following code fails to compile for reasons cited
in the comments.  However, since placement new only
affects a non-static member variable, it's unclear why
the placement new use is not allowed.

Placement new is not allowed because it would allow constant expression evaluation to interact with the underlying storage of objects (which need not exist and need not be modeled the same way as at runtime).

Put another way, new( char_ptr ) Thing intrinsically includes an operation like reinterpret_cast< Thing * >( char_ptr ). The cast is illegal because you can still access the char_ptr array, and new is illegal by implication.

I wonder if it would be feasible to poison the array to sidestep the issue. Something like, in a constant evaluation context:

1. Casting to void* is not allowed except from a char* value referencing the first element of an array which is a constant expression in the current context. (Note, the relevant reinterpret_cast is defined in terms of static_casts to and from void*.)

2. Casting from cv void* is not allowed unless the value was produced by a cast from char*.

3. Accessing an array with a subobject (character) that has participated in such a cast in not allowed.

4. Casting the array again to a different type is not allowed.

Also allowing the opposite cast, from an object pointer to a char* invisibly associated with its type, would provide consistency and enable constexpr std::addressof, which is currently a minor hole.

I think the easiest way to get a constexpr std::addressof is just for the library to mark it constexpr. Implementations can easily support it as a builtin (Clang has had a constexpr __builtin_addressof for quite some time).

However, having constructed such an object, you wouldn’t be able to destroy it because there are no constexpr explicit destructor calls. (Pedantically, the standard only says pseudo-destructor calls (§5.2.4) are not allowed, and those are a different animal than explicit destructor calls (§12.4/13), but that’s certainly a defect.)

An explicit destructor call would be an invocation of a function that is not constexpr, so should be covered by the second bullet in 5.20.

Those would need to be allowed as well, and then the interpreter has to keep a bit of state for each object to remember whether it’s already been destroyed.

I think just a few pieces of additional state would be sufficient to get this working:

1. For the representation of char array constants, remember what object (if any) it’s been cast to or from. This reference could be stored in place of the array value. For extra generosity, provide for a sequence of objects at offsets within the array.

2. For all other constant, complete objects, store a potential reference to the (inaccessible, stubbed-out) char array of its object representation.

3. Provide some means of navigating from subobjects to complete objects.

4. Let each void * constant value have an underlying char * value.

5. Allow char * constant values to refer to any target, not necessarily a char.

The above is not sufficient for Clang's implementation, which represents a pointer or reference as a path (sequence of members, bases, array elements) from a complete object (a declaration or temporary creation, plus optionally a handle to a particular function invocation), and your new object is not reachable in such a manner. I don't know how other implementations represent values during constexpr evaluation, but I wouldn't be surprised if the above is insufficient for them in some way, too.

 

Support for all this would be somewhat useful alone, and a step toward predynamic storage. The added complexity is substantial, but still less than emulating all object representation, and nothing to kill performance. Unless I’ve missed something.

I agree that the added complexity is substantial. There's also added complexity in the conversion of the compile-time representation to a sequence of bytes to emit, and in the new flavors of UB to detect. Also, the language doesn't really let you get away with placement new'ing over a char array subobject of a larger object (it probably should, but see 1.8/6 and 3.8/1.4).

I think the complexity (not just in implementation, but also in specification and user-facing language semantics) is probably too great to support this as a special case. If this could be made to work more generally, that might help to justify the complexity, but there are limits to how far we can push constexpr and retain a strong phase distinction and a simple semantic model.

[David Krauss]

On 2015–04–14, at 7:55 AM, Richard Smith <ric...@metafoo.co.uk> wrote:

I think the easiest way to get a constexpr std::addressof is just for the library to mark it constexpr. Implementations can easily support it as a builtin (Clang has had a constexpr __builtin_addressof for quite some time).

Yes, if we take that problem separately.

However, having constructed such an object, you wouldn’t be able to destroy it because there are no constexpr explicit destructor calls. (Pedantically, the standard only says pseudo-destructor calls (§5.2.4) are not allowed, and those are a different animal than explicit destructor calls (§12.4/13), but that’s certainly a defect.)

An explicit destructor call would be an invocation of a function that is not constexpr, so should be covered by the second bullet in 5.20.

Ah, yes. Also, I forgot that nontrivial destructors still aren’t allowed.

However, if we assume that the result of placement new is trivially destructible, then the explicit destructor call is also unnecessary.

The motivation for the feature gets stronger with predynamic storage, and in that case, destructors would have to be enabled.

Those would need to be allowed as well, and then the interpreter has to keep a bit of state for each object to remember whether it’s already been destroyed.

I think just a few pieces of additional state would be sufficient to get this working:

1. For the representation of char array constants, remember what object (if any) it’s been cast to or from. This reference could be stored in place of the array value. For extra generosity, provide for a sequence of objects at offsets within the array.

2. For all other constant, complete objects, store a potential reference to the (inaccessible, stubbed-out) char array of its object representation.

3. Provide some means of navigating from subobjects to complete objects.

4. Let each void * constant value have an underlying char * value.

5. Allow char * constant values to refer to any target, not necessarily a char.

The above is not sufficient for Clang's implementation, which represents a pointer or reference as a path (sequence of members, bases, array elements) from a complete object (a declaration or temporary creation, plus optionally a handle to a particular function invocation), and your new object is not reachable in such a manner. I don't know how other implementations represent values during constexpr evaluation, but I wouldn't be surprised if the above is insufficient for them in some way, too.

Thought experiments only go so far.

But since, as you mention, the backing storage array must not be a subobject, the new object can be introduced as its peer. It’s as if the placement new result was declared in whatever scope the array was. (Likewise, the backing storage array supporting a reinterpret_cast would be introduced as a peer to its associated complete object.)

Otherwise, an object placed within a buffer could be modeled as another kind of subobject, along with members, bases, and array elements. According to my rules, placing an object within a buffer would destroy its character sequence. Internally, there could be separate types for string-like char arrays and storage-like char arrays, although you would have to support conversion (i.e. replacement) from the former to the latter.

The problem you describe sounds like a solution to #3, because a complete object is always already on hand in case the user takes a reference to a subobject.

I think the complexity (not just in implementation, but also in specification and user-facing language semantics) is probably too great to support this as a special case. If this could be made to work more generally, that might help to justify the complexity, but there are limits to how far we can push constexpr and retain a strong phase distinction and a simple semantic model.

Yes, the motivation I see is more toward supporting predynamic storage. That’s a can of worms, and it’s all still at the theoretical stage of trying to squeeze it into the existing models.

[Gonzalo BG]

I ran into this today and just want to share my use case: implementing a stack allocated std::vector (vector<T, Capacity>) in C++.

I basically had to choose between 2 trade-offs:

1) use a recursive implementation or a std::array and make the container constexpr

    for trivially default constructible and trivially destructible types (but this means all

    elements up to capacity are always initialized)

2) using std::aligned_storage with std::alignment_of, not making the container constexpr,

    but having the same semantics as vector (only elements up to size are initialized)

I went for the second approach because I want semantics as similar to std::vector as possible

(and want to allow customizing the alignment requirements via std::alignment_of for e.g. SIMD types). 

I would rather not have to choose tho. To make the second approach constexpr I need:

  - constexpr reinterpret_cast

  - constexpr placement new

  - constexpr user-defined destructors

These "features" would also simplify the implementation of constexpr friendly 

variants (and thus optional, expected, ...) significantly.

P.S: using a std::vector<T, stack_allocator> is not only suboptimal but highly unreliable due to 

vector's growth factor so that is not a satisfying solution for my use cases.

[Myriachan]

On Monday, April 13, 2015 at 4:55:30 PM UTC-7, Richard Smith wrote:
> On Mon, Apr 13, 2015 at 9:56 AM, David Krauss <pot...@gmail.com> wrote:
> > Also allowing the opposite cast, from an object pointer to a char* invisibly associated with its type, would provide consistency and enable constexpr std::addressof, which is currently a minor hole.
>
>
> I think the easiest way to get a constexpr std::addressof is just for the library to mark it constexpr. Implementations can easily support it as a builtin (Clang has had a constexpr __builtin_addressof for quite some time).

It's not just std::addressof that is affected; std::numeric_limits<FloatType>::infinity() and friends are as well, and those are mandated to be constexpr already. GCC libstdc++ uses compiler magic in the form of __builtin_hugevalf() to do that currently, even though current GCC allows reinterpret_cast and fun with unions in constexpr. No idea about Clang and libcxx, or MSVC 2015, other than that MSVC 2015 lets you get away with constexpr reinterpret_cast shenanigans and Clang rightfully doesn't.

Melissa

[Richard Smith]

On Tue, Aug 25, 2015 at 1:24 AM, Myriachan <myri...@gmail.com> wrote:

On Monday, April 13, 2015 at 4:55:30 PM UTC-7, Richard Smith wrote:
> On Mon, Apr 13, 2015 at 9:56 AM, David Krauss <pot...@gmail.com> wrote:
> > Also allowing the opposite cast, from an object pointer to a char* invisibly associated with its type, would provide consistency and enable constexpr std::addressof, which is currently a minor hole.
>
>
> I think the easiest way to get a constexpr std::addressof is just for the library to mark it constexpr. Implementations can easily support it as a builtin (Clang has had a constexpr __builtin_addressof for quite some time).

It's not just std::addressof that is affected; std::numeric_limits<FloatType>::infinity() and friends are as well, and those are mandated to be constexpr already.  GCC libstdc++ uses compiler magic in the form of __builtin_hugevalf() to do that currently,

infinity() is fundamentally non-portable; why is it a problem that it is implemented (once, in the standard library) with compiler magic? And what change do you suggest to improve the situation?

 

even though current GCC allows reinterpret_cast and fun with unions in constexpr.

Only in fairly limited ways, and it has become stricter in recent releases. If you want to use type punning to get an infinite value, you're already relying on undefined behavior whether or not you try to perform the computation in a constant expression, so you're already doing something non-portable and compiler-specific at best.

 

No idea about Clang and libcxx, or MSVC 2015, other than that MSVC 2015 lets you get away with constexpr reinterpret_cast shenanigans and Clang rightfully doesn't.

Melissa