Why are make_signed and make_unsigned based on sizeof(T)?

[tru...@gmail.com]

Hello,

make_signed and make_unsigned from type_traits, when the argument is not a signed or unsigned integer type, are specified as giving the result based on sizeof(T). Assuming C++ still supports padding bits in integer types, how is this useful? Suppose int, long, and long long all have 64 bits, but they have 32, 16 and 0 padding bits respectively (giving 32, 48 and 64 sign+value bits). Also suppose that no other signed or unsigned integer type has the same size as any of these. Now suppose enum E : long { }. Since sizeof(E) == sizeof(int), make_signed<E>::type is required to be int, and make_unsigned<E>::type is required to be unsigned int. What could you use this for? When would you not want the signed or unsigned type of E to be the signed or unsigned type of E's underlying type, or at the very least something based on the types' ranges, rather than their sizes?

Cheers,
Harald van Dijk

[Daniel Krügler]

I agree that your criticism is justified, the problem is presumably
based on the historic development of the traits - at the time where
they had been suggested (~2007) the wording was influenced by the
compile-time introspection capabilities of that time.
std::underlying_type did not exist at that time (and unfortunately
this trait is even today not defined for non-enumeration types,
although we really would like to get the expected answer for wchar_t
and others through this trait).

But if you want to improve the situation I recommend to write a small
proposal to fix these problems - seriously! I can offer you to review
it and give you suggestions in regard to the "Standardeze", if you
like.

Personally I agree that for enumeration types the outcome of
make_[un]signed is probably misleading in many cases. A drive-in fix
would be to say that for integral types that are not signed integer
types nor unsigned integer types, the result would by obtained by
applying the corresponding trait to the underlying type of this
integral type and apply the same cv-qualification as the original type
to the outcome of that operation. Unfortunately you won't be able to
implement it without compiler-support, so maybe it would make your
proposal more attractive to suggest to extend std::underlying type for
all other integral types and than define make_[un]signed purely based
on that trait.

The question is whether your concerns are serious enough to open a
library issue. The usual criteria (wording is unclear, wording is
contradictory) seem not to apply here, so the problem could be that
the committee could be hesitant to change the specification, but this
is just a guess and I don't want to discourage you.

IMO you *could* try to submit an additional library issue (post it to
the address given in the reply-to field of
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html),
especially if you are careful to describe the situation pointing out
the *inconsistency* as you did above. The good news would be that the
original intended characteristics of the trait - as described in the
original proposing paper

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2255.html#int

should still hold in the revised form.

- Daniel

[tru...@gmail.com]

Thank you. I will make an attempt to come up with improved wording when I have some free time. Note that there is one type that would be left unaddressed by that approach: plain char has no underlying type, is neither a signed integer type nor an unsigned integer type, and has no underlying type. The current wording requires make_signed<char>::type to be signed char, and make_unsigned<char>::type to be unsigned char. (Assuming no extended integer type has a lower rank than char, anyway.) That makes sense to me, and should probably remain that way.

It should require no compiler support beyond what already exists for enumeration types, unless I am misunderstanding: it could be implemented as an implementation-specific specialisation for char16_t, char32_t and wchar_t. I do like your suggestion of extending std::underlying_type.
 

The question is whether your concerns are serious enough to open a
library issue. The usual criteria (wording is unclear, wording is
contradictory) seem not to apply here, so the problem could be that
the committee could be hesitant to change the specification, but this
is just a guess and I don't want to discourage you.

In that case, might it be more likely to be accepted if it based on the types' ranges? On an implementation where INT_MIN == LONG_MIN, when given an enum E : long { }, make_signed<E>::type would remain int. It has the benefit that existing implementations, using sizeof, do continue to be valid for systems without padding bits, and on those implementations, int would be as equally a usable result as long. I will try to come up with two alternative suggested wordings.
 

IMO you *could* try to submit an additional library issue (post it to
the address given in the reply-to field of
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html),
especially if you are careful to describe the situation pointing out
the *inconsistency* as you did above. The good news would be that the
original intended characteristics of the trait - as described in the
original proposing paper

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2255.html#int

should still hold in the revised form.

Thank you, I had not seen that. I am surprised by the invariant mentioned there that numeric_limits<U>::digits == numeric_limits<S>::digits + 1. I know that in C, UINT_MAX is permitted to be equal to INT_MAX. It is also permitted to be greater than 2*INT_MAX+1. I was not aware of any changes in C++ in this area, and cannot find this as a requirement in the standard. The last of the invariants also seems to be incorrectly worded to me: if integral and signed, then (...), else (...) says the else part applies to non-integral types, but it doesn't and doesn't have to. For the rest, it make sense to me, though, and I will make sure not to make any suggestion where any of those invariants that did previously hold, no longer would.

- Daniel

Cheers,
Harald van Dijk

[Daniel Krügler]

2014-02-26 21:20 GMT+01:00 <tru...@gmail.com>:
> On Tuesday, February 25, 2014 10:17:55 PM UTC+1, Daniel Krügler wrote:
> Thank you. I will make an attempt to come up with improved wording when I
> have some free time. Note that there is one type that would be left
> unaddressed by that approach: plain char has no underlying type, is neither
> a signed integer type nor an unsigned integer type, and has no underlying
> type.

Well, but that could be defined for my recommended changes to
std::underlying_type, even if the core language does not define it
that way.

> The current wording requires make_signed<char>::type to be signed
> char, and make_unsigned<char>::type to be unsigned char. (Assuming no
> extended integer type has a lower rank than char, anyway.) That makes sense
> to me, and should probably remain that way.

Yes.

> It should require no compiler support beyond what already exists for
> enumeration types, unless I am misunderstanding: it could be implemented as
> an implementation-specific specialisation for char16_t, char32_t and
> wchar_t. I do like your suggestion of extending std::underlying_type.

Yes, but this specialization is more or less requiring
compiler-support, because it depends on compiler-defined types.
Therefore it would be better to move all of this into a single point,
preferably into underlying_type.

>> The question is whether your concerns are serious enough to open a
>> library issue. The usual criteria (wording is unclear, wording is
>> contradictory) seem not to apply here, so the problem could be that
>> the committee could be hesitant to change the specification, but this
>> is just a guess and I don't want to discourage you.
>
> In that case, might it be more likely to be accepted if it based on the
> types' ranges?

I don't think so, I believe that the route via the underlying type
better reflects the original intentions.

> On an implementation where INT_MIN == LONG_MIN, when given an
> enum E : long { }, make_signed<E>::type would remain int. It has the benefit
> that existing implementations, using sizeof, do continue to be valid for
> systems without padding bits, and on those implementations, int would be as
> equally a usable result as long. I will try to come up with two alternative
> suggested wordings.
>
>>
>> IMO you *could* try to submit an additional library issue (post it to
>> the address given in the reply-to field of
>> http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html),
>> especially if you are careful to describe the situation pointing out
>> the *inconsistency* as you did above. The good news would be that the
>> original intended characteristics of the trait - as described in the
>> original proposing paper
>>
>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2255.html#int
>>
>> should still hold in the revised form.
>
> Thank you, I had not seen that. I am surprised by the invariant mentioned
> there that numeric_limits<U>::digits == numeric_limits<S>::digits + 1.

Well, this is certainly not really guaranteed, but it was true for
Howard's implementation ;-)

- Daniel

[Daniel Krügler]

2014-02-26 21:26 GMT+01:00 Daniel Krügler <daniel....@gmail.com>:
>> It should require no compiler support beyond what already exists for
>> enumeration types, unless I am misunderstanding: it could be implemented as
>> an implementation-specific specialisation for char16_t, char32_t and
>> wchar_t. I do like your suggestion of extending std::underlying_type.
>
> Yes, but this specialization is more or less requiring
> compiler-support, because it depends on compiler-defined types.
> Therefore it would be better to move all of this into a single point,
> preferably into underlying_type.

I should add that this technique requires potentially to apply a
second round-trip because we do allow

enum class ew : wchar_t {};

which returns wchar_t.

The more severe problem of this approach is, that we would now have
the problem that for

enum class eb : bool {};

we need a good answer for make_[un]signed, because bool is not in the
set of supported types, albeit from the current definition of these
traits this enum would have a well-defined type (which seems a bit
questionable, given that we forbid plain bool).

- Daniel

[David Krauss]

On Feb 27, 2014, at 3:12 PM, Daniel Krügler <daniel....@gmail.com> wrote:

we need a good answer for make_[un]signed, because bool is not in the
set of supported types, albeit from the current definition of these
traits this enum would have a well-defined type (which seems a bit
questionable, given that we forbid plain bool).

[Un]signed enumeration types are nonsense for the same reason as with bool. What is the use case for make_[un]signed< E >? Is it plausible that it could be deprecated and migrated to make_[un]signed< underlying_type_t< E > >? And if the user furthermore wants to support bool and the various char types besides signed char and unsigned char, they can code their own special cases, because the semantic meaning is hazy and the transformation is lossy — you don’t recover the original type by applying the “inverse.”

It looks like make_signed and make_unsigned were introduced by N2255:

Two new traits have been added: make_signed and make_unsigned. These traits have been found to be useful in practice and reinvented multiple times. The multiple implementations in the field tend to have different behaviors in the corner cases. This specification standardizes these traits, even for the corner cases.

Invariants for make_signed and make_unsigned:

Let S be make_signed<T>::type and U be make_unsigned<T>::type.

• is_signed<S>::value
• is_unsigned<U>::value
• sizeof(T) == sizeof(S)
• sizeof(T) == sizeof(U)
• !is_same<S, U>::value
• is_convertible<S, U>::value
• is_convertible<U, S>::value

• numeric_limits<U>::digits == numeric_limits<S>::digits + 1

• If is_integral<T>::value and is_signed<T>::value then
  • numeric_limits<T>::digits == numeric_limits<S>::digits, else
  • numeric_limits<T>::digits == numeric_limits<U>::digits.

The last one doesn’t seem to hold for enumeration types, and the preceding one as you noted earlier is bogus.

“Standardizes these traits, even for the corner cases” seems a bit cryptic. Apparently the behavior is adopted from Boost.  Going through several pages of Google and Ohloh search results doesn’t turn up evidence of use with enumeration types, and most of the code is still using Boost anyway.

Perhaps there’s still room to let the standard be more conservative, and hackier users who need Boost-like enum handling can continue using Boost?

Suppose there were a normalize_integral_type metafunction mapping as such:

• bool => unsigned integral type with sizeof(T) == sizeof(bool) which emulates std::underlying_type as if bool were an enumeration.
• char => signed char or unsigned char
• wchar_t, char16_t, char32_t => underlying type per [basic.fundamental]/5
• signed and unsigned integer types => themselves
• anything else => ill-formed

Then underlying_type and normalize_integral_type could be orthogonal and “lossy” operations, and make_signed and make_unsigned could be defined only in the domain of signed and unsigned integer types, such as to only modify a pre-existing signed or unsigned specifier. But I’m just dreaming…

[Matthew Woehlke]

On 2014-02-28 06:42, David Krauss wrote:
> [Un]signed enumeration types are nonsense for the same reason as with bool.

Why? I could have an enumeration (probably weakly-typed) for well-known
values of some field that is of type (e.g.) int.

Actually I do this in real code; I have enumerations for symbolic names
of columns in QTreeView's which takes 'int' as the column index (since
being a template class taking an enum for column index would have oh so
many issues). And that's one example; I could give others.

I don't know that I'd ever be using make_[un]signed on these, but the
enum itself having an associated type, sign and all, is certainly relevant.

--
Matthew

[David Krauss]

On Mar 1, 2014, at 2:18 AM, Matthew Woehlke <mw_t...@users.sourceforge.net> wrote:

On 2014-02-28 06:42, David Krauss wrote:

[Un]signed enumeration types are nonsense for the same reason as with bool.

Why? I could have an enumeration (probably weakly-typed) for well-known values of some field that is of type (e.g.) int.

Actually I do this in real code; I have enumerations for symbolic names of columns in QTreeView's which takes 'int' as the column index (since being a template class taking an enum for column index would have oh so many issues). And that's one example; I could give others.

But, by that logic, bool should also work.

Furthermore, even when using enum types numerically, common practice is now to avoid implicit conversions. So you have a situation where explicit conversion is needed just to get from t to make_unsigned_type<t>, when the underlying type of t is already unsigned. Better to make explicit the step through underlying_type.

I don't know that I'd ever be using make_[un]signed on these, but the enum itself having an associated type, sign and all, is certainly relevant.

Absolutely. Drawing a distinction between bool and an enumeration that acts exactly like bool is weird though. And suggesting that the signed type of lowest rank and same sizeof some enumeration e is that closely related to e is probably inviting the user to dangerous assumptions.

[Matthew Woehlke]

On 2014-02-28 13:30, David Krauss wrote:

> On Mar 1, 2014, at 2:18 AM, Matthew Woehlke wrote:
>> On 2014-02-28 06:42, David Krauss wrote:
>>> [Un]signed enumeration types are nonsense for the same reason as
>>> with bool.
>>
>> Why? I could have an enumeration (probably weakly-typed) for
>> well-known values of some field that is of type (e.g.) int.
>

> But, by that logic, bool should also work.

If you mean that having bool as an underlying type of an enumeration,
then yes. And I could give reasonable examples for that also (in this
case, strongly typed, as a common reason to have an enum with underlying
type bool is for type distinction).

> Furthermore, even when using enum types numerically, common practice
> is now to avoid implicit conversions.

That (IMHO) would be really silly in the example I gave above; the API I
am calling takes an int, not an enum. What would be the point of
explicitly casting the enum to int? There is no type safety gained; just
more verbose code.

--
Matthew

[David Krauss]

On Mar 1, 2014, at 2:58 AM, Matthew Woehlke <mw_t...@users.sourceforge.net> wrote:

That (IMHO) would be really silly in the example I gave above; the API I am calling takes an int, not an enum. What would be the point of explicitly casting the enum to int? There is no type safety gained; just more verbose code.

The safety is gained in code besides that API interface. A number-like enum will only allow use with the operators specifically defined for it.

See for example http://stackoverflow.com/a/18553898/153285 .

[David Krauss]

On Mar 1, 2014, at 2:58 AM, Matthew Woehlke <mw_t...@users.sourceforge.net> wrote:

That (IMHO) would be really silly in the example I gave above; the API I am calling takes an int, not an enum. What would be the point of explicitly casting the enum to int? There is no type safety gained; just more verbose code.

Also note that the implicit conversion isn’t to a given enum’s underlying type or directly to the parameter type, it’s always to int. A numeric enumeration representing sequence indexes, for example, should be based on size_t and should not implicitly convert. A bitset-like enum which could potentially grow beyond 15 places likewise can’t safely be converted to int.

I use unary operator+ as explicit cast shorthand. Implicit conversion functions would be nice.

[tru...@gmail.com]

On Wednesday, February 26, 2014 9:26:57 PM UTC+1, Daniel Krügler wrote:

2014-02-26 21:20 GMT+01:00  <tru...@gmail.com>:
> On Tuesday, February 25, 2014 10:17:55 PM UTC+1, Daniel Krügler wrote:
> Thank you. I will make an attempt to come up with improved wording when I
> have some free time. Note that there is one type that would be left
> unaddressed by that approach: plain char has no underlying type, is neither
> a signed integer type nor an unsigned integer type, and has no underlying
> type.

Well, but that could be defined for my recommended changes to
std::underlying_type, even if the core language does not define it
that way.

Ah, okay, I didn't read it that way. I took your message as suggesting that std::underlying_type could be extended to work for the types for which the standard currently requires an underlying type.

> The current wording requires make_signed<char>::type to be signed
> char, and make_unsigned<char>::type to be unsigned char. (Assuming no
> extended integer type has a lower rank than char, anyway.) That makes sense
> to me, and should probably remain that way.

Yes.

> It should require no compiler support beyond what already exists for
> enumeration types, unless I am misunderstanding: it could be implemented as
> an implementation-specific specialisation for char16_t, char32_t and
> wchar_t. I do like your suggestion of extending std::underlying_type.

Yes, but this specialization is more or less requiring
compiler-support, because it depends on compiler-defined types.
Therefore it would be better to move all of this into a single point,
preferably into underlying_type.

I was under the impression that "compiler support" generally does not mean changes that are restricted to the library, even if the implementation is specific to one specific compiler, but agreed, it cannot be implemented portably.
 

>> The question is whether your concerns are serious enough to open a
>> library issue. The usual criteria (wording is unclear, wording is
>> contradictory) seem not to apply here, so the problem could be that
>> the committee could be hesitant to change the specification, but this
>> is just a guess and I don't want to discourage you.
>
> In that case, might it be more likely to be accepted if it based on the
> types' ranges?

I don't think so, I believe that the route via the underlying type
better reflects the original intentions.

In that case, I do expect strong reluctance to change the specification, since an implementation that would implement my suggested wording would fail to implement the published C++11 standard. But perhaps it's worth the cost.
 

> On an implementation where INT_MIN == LONG_MIN, when given an
> enum E : long { }, make_signed<E>::type would remain int. It has the benefit
> that existing implementations, using sizeof, do continue to be valid for
> systems without padding bits, and on those implementations, int would be as
> equally a usable result as long. I will try to come up with two alternative
> suggested wordings.

Based on your message, I have limited this to one suggestion:

3.9.1 Fundamental types

In any particular implementation, a plain char object can take on either the same values as a signed char or an unsigned char; which one is implementation-defined.
=>
In any particular implementation, plain char has an underlying type of signed char, or unsigned char; which one is implementation-defined. A plain char object can take on the same values as an object of its underlying type.

20.9.7.3 Sign modifications

template <class T>
struct make_signed;

If T names a (possibly cv-qualified) signed integer type (3.9.1) then the member typedef type shall name the type T; otherwise, if T names a (possibly cv-qualified) unsigned integer type then type shall name the corresponding signed integer type, with the same cv-qualifiers as T; otherwise, type shall name the signed integer type with smallest rank (4.13) for which sizeof(T) == sizeof(type), with the same cv-qualifiers as T.
=>
If T names a (possibly cv-qualified) signed integer type (3.9.1) then the member typedef type shall name the type T; otherwise, if T names a (possibly cv-qualified) unsigned integer type then type shall name the corresponding signed integer type, with the same cv-qualifiers as T; otherwise, T has an underlying type, and the result is the same as the result for its underlying type, with the same cv-qualifiers as T.

Requires: T shall be a (possibly cv-qualified) integral type or enumeration but not a bool type.
=>
Requires: T shall be a (possibly cv-qualified) integral type or enumeration but not a bool type or an enumeration with an underlying type of bool.

20.9.7.6 Other transformations

template <class T>
struct underlying_type;

T shall be an enumeration type (7.2)
=>
T shall be char, wchar_t, char16_t, char32_t, or an enumeration type (7.2)

The member typedef type shall name the underlying type of T.

Does this wording seem appropriate? Is anything important still missing? Would it be a problem that this also changes a definition in the core language, and not just the library bits?

Cheers,
Harald van Dijk
 

>>
>> IMO you *could* try to submit an additional library issue (post it to
>> the address given in the reply-to field of
>> http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html),

P.S.: http://cplusplus.github.io/LWG/lwg-active.html seems to be the more up-to-date location.
 

[Daniel Krügler]

2014-03-09 11:23 GMT+01:00 <tru...@gmail.com>:
> In that case, I do expect strong reluctance to change the specification,
> since an implementation that would implement my suggested wording would fail
> to implement the published C++11 standard. But perhaps it's worth the cost.

I'm not denying that there could be some resistance to such a change,
but I would also like to add that current C++ has now reached a state,
that allows to inspect so many implementation details by compile-time
inspection that several currently accepted defects are to some degree
observable compared to the state before.

> Based on your message, I have limited this to one suggestion:
>
> 3.9.1 Fundamental types
>
> In any particular implementation, a plain char object can take on either the
> same values as a signed char or an unsigned char; which one is
> implementation-defined.
> =>
> In any particular implementation, plain char has an underlying type of
> signed char, or unsigned char; which one is implementation-defined. A plain
> char object can take on the same values as an object of its underlying type.

Sounds good to me, but keep in my that such proposed changes cannot
occur in a library issue (because it touches the core language) and
this is a sign to make a small proposal instead. But in such a case it
would still be useful to submit a library issue as well. In fact your
proposal should mention that it will address the corresponding issue
as well.

> 20.9.7.3 Sign modifications
>
> template <class T>
> struct make_signed;
>
> If T names a (possibly cv-qualified) signed integer type (3.9.1) then the
> member typedef type shall name the type T; otherwise, if T names a (possibly
> cv-qualified) unsigned integer type then type shall name the corresponding
> signed integer type, with the same cv-qualifiers as T; otherwise, type shall
> name the signed integer type with smallest rank (4.13) for which sizeof(T)
> == sizeof(type), with the same cv-qualifiers as T.
> =>
> If T names a (possibly cv-qualified) signed integer type (3.9.1) then the
> member typedef type shall name the type T; otherwise, if T names a (possibly
> cv-qualified) unsigned integer type then type shall name the corresponding
> signed integer type, with the same cv-qualifiers as T; otherwise, T has an
> underlying type, and the result is the same as the result for its underlying
> type, with the same cv-qualifiers as T.

IMO the last part starting with ";otherwise, ..." would be clearer,
when expressed in code for, e.g.

"; otherwise T has an underlying type and the member typedef type
shall name the type make_signed<underlying_type<T>::type>::type"

or

"; otherwise T has an underlying type and the member typedef type
shall name the type make_signed_t<underlying_type_t<T>>"

> Does this wording seem appropriate?

Sounds good to me, except for my recommendation mentioned above.

> Would it be a problem that this also changes a definition in the core
> language, and not just the library bits?

This is no fundamental problem, but it points for a need of a paper
that will have to be looked at by both the core language group and the
library group instead of trying to solve this by a plain issue. We had
several such papers before, so this would not be a very unusual
procedure.

- Daniel