UB while dealing with invalid raw pointers, the std::uninitialized_fill case
Francesco S. Carta
as far as I've been able to understand, if a raw pointer contains an
invalid value (that is, it does not point to any valid object of the
type it is a pointer to) then some of the actions performed on these
pointers will lead to UB.
As it seems, two actions in particular should be safe and well defined:
- zeroing the invalid pointer;
- assigning a valid value to the invalid pointer;
One issue that has been recently raised in this group is about storing
invalid raw pointers into a container such as std::vector; the rationale
that led to define it as a potential source of UB is about the lvalue to
rvalue conversion that will be performed on those raw pointers during
internal reallocations of the container.
Since the only significant action that gets performed during the
reallocation is to copy such invalid pointer values from a storage to
another, it should boil down to something equivalent to this:
int* p = new int;
delete p;
Now "p" contains and invalid value.
int* q = p;
During the above assignment, an lvalue to rvalue conversion is performed
on "p", leading to undefined behavior.
Now my question is, would the following test also lead to an lvalue to
rvalue conversion on "p", therefore leading to UB?
int* p = new int;
delete p;
int* q = new int;
if(q != p) {
//...
}
If that's the case, then any uninitialized_fill performed on a storage
area of raw pointers will lead to UB, as the Standard depicts, as
expected effect, the fact of comparing two invalid pointers:
[citation formatted for presentation]
> 20.4.4.2 uninitialized_fill [lib.uninitialized.fill]
>
> template <class ForwardIterator, class T>
> void uninitialized_fill(ForwardIterator first,
> ForwardIterator last,
> const T& x);
>
> 1 Effects:
>
> for (; first != last; ++first)
> new (static_cast<void*>(&*first))
> typename iterator_traits<ForwardIterator>::value_type(x);
Would all the above mean that we shouldn't really worry about UB when
dealing with invalid pointers into standard containers as long as we
don't dereference such invalid pointers, and accordingly, would that
mean that the standard needs to be modified to state these actions
(copying and comparing of invalid pointers) as well-defined?
Thank you for your attention.
--
FSC - http://userscripts.org/scripts/show/59948
http://fscode.altervista.org - http://sardinias.com
Alf P. Steinbach /Usenet
> Hi there,
> as far as I've been able to understand, if a raw pointer contains an invalid
> value (that is, it does not point to any valid object of the type it is a
> pointer to) then some of the actions performed on these pointers will lead to UB.
Hm, well you need to define "invalid" more precisely, e.g. as "is not valid".
;-) Or even more precisely, "can not be dereferenced without UB". For example, 0
is a valid pointer value, as is 1+p where p points to the last element in an array.
Neither C++98 nor C++0x does, as far as I know, define "invalid pointer", but
C++0x defines "valid pointer" as pointing to a byte in memory or being zero, in
C++0x §2.9.2/3. The definition in C++0x is perhaps too permissive. If taken
literally the validity of a pointer would in general not be deducible but would
depend on whether the address in question had been remapped by the HW, e.g. p
would be /valid/ immediately after delete p unless the delete affected the
validity of the address itself (e.g. by changing paging or segment setup).
But OK...
> As it seems, two actions in particular should be safe and well defined:
> - zeroing the invalid pointer;
> - assigning a valid value to the invalid pointer;
>
> One issue that has been recently raised in this group is about storing invalid
> raw pointers into a container such as std::vector; the rationale that led to
> define it as a potential source of UB is about the lvalue to rvalue conversion
> that will be performed on those raw pointers during internal reallocations of
> the container.
>
> Since the only significant action that gets performed during the reallocation is
> to copy such invalid pointer values from a storage to another, it should boil
> down to something equivalent to this:
>
> int* p = new int;
> delete p;
>
> Now "p" contains and invalid value.
>
> int* q = p;
>
> During the above assignment, an lvalue to rvalue conversion is performed on "p",
> leading to undefined behavior.
>
> Now my question is, would the following test also lead to an lvalue to rvalue
> conversion on "p", therefore leading to UB?
>
> int* p = new int;
> delete p;
> int* q = new int;
> if(q != p) {
> //...
> }
Yes, this invokes rvalue conversion and UB.
> If that's the case, then any uninitialized_fill performed on a storage area of
> raw pointers will lead to UB, as the Standard depicts, as expected effect, the
> fact of comparing two invalid pointers:
>
> [citation formatted for presentation]
>> 20.4.4.2 uninitialized_fill [lib.uninitialized.fill]
>>
>> template <class ForwardIterator, class T>
>> void uninitialized_fill(ForwardIterator first,
>> ForwardIterator last,
>> const T& x);
>>
>> 1 Effects:
>>
>> for (; first != last; ++first)
>> new (static_cast<void*>(&*first))
>> typename iterator_traits<ForwardIterator>::value_type(x);
>
Huh, no.
'first' and 'last' here are not invalid pointers: if pointers, then they point
/to/ the area to be filled.
> Would all the above mean that we shouldn't really worry about UB when dealing
> with invalid pointers into standard containers as long as we don't dereference
> such invalid pointers,
Formally you invoke UB when a vector containing invalid pointers is destroyed.
That's because a simplistic implementation may iterate over the vector contents
and do pseudo destructor calls on the pointers (or it can do anything at all).
In practice it's not anything I'd worry about, because leaving a vector with
invalid pointers is common practice, so implementations have to not crash on
that. However, to play nice, I guess one should always zero a pointer in a
vector (or other container) after making it invalid. Just making sure.
> and accordingly, would that mean that the standard needs
> to be modified to state these actions (copying and comparing of invalid
> pointers) as well-defined?
I dont't think so.
If the standard was all too clear about everything then we'd have nothing to
discuss.
Cheers & hth.,
- Alf
--
blog at <url: http://alfps.wordpress.com>
Alf P. Steinbach /Usenet
> * Francesco S. Carta, on 03.09.2010 12:52:
>> Hi there,
>> as far as I've been able to understand, if a raw pointer contains an invalid
>> value (that is, it does not point to any valid object of the type it is a
>> pointer to) then some of the actions performed on these pointers will lead to UB.
>
> Hm, well you need to define "invalid" more precisely, e.g. as "is not valid".
> ;-) Or even more precisely, "can not be dereferenced without UB". For example, 0
> is a valid pointer value, as is 1+p where p points to the last element in an array.
Hm, let me be more clear on that, before someone protests: a zero pointer can be
dereferenced without UB, namely in a typeid expression.
I'm concerned, however, that "can be dereferenced" possibly does not hold for
1+p, so that my "even more precisely" then becomes less precise :-(.
Deref of 1+p was slated to become part of C++0x, for compatibility with C99
rules, but I don't recall exactly what happened and it takes too much time to
find it, especially if it's not there...
So perhaps the most precise that can be said is just "is not valid".
Whatever.
Cheers,
Francesco S. Carta
> * Francesco S. Carta, on 03.09.2010 12:52:
>> Hi there,
>> as far as I've been able to understand, if a raw pointer contains an
>> invalid
>> value (that is, it does not point to any valid object of the type it is a
>> pointer to) then some of the actions performed on these pointers will
>> lead to UB.
>
> Hm, well you need to define "invalid" more precisely, e.g. as "is not
> valid". ;-) Or even more precisely, "can not be dereferenced without
> UB". For example, 0 is a valid pointer value, as is 1+p where p points
> to the last element in an array.
>
> Neither C++98 nor C++0x does, as far as I know, define "invalid
> pointer", but C++0x defines "valid pointer" as pointing to a byte in
> memory or being zero, in C++0x §2.9.2/3. The definition in C++0x is
> perhaps too permissive. If taken literally the validity of a pointer
> would in general not be deducible but would depend on whether the
> address in question had been remapped by the HW, e.g. p would be /valid/
> immediately after delete p unless the delete affected the validity of
> the address itself (e.g. by changing paging or segment setup).
>
> But OK...
OK, I've read your self-follow-up, just for the records. Correctly
defining an invalid pointer seems to be impossible, but we have some
agreed cases of valid and invalid pointer values:
- the null-pointer value is a valid and non-dereferenceable value;
- the address of a valid object is a valid pointer value;
- the address of a valid object becomes an invalid pointer value after
the object gets destroyed;
- the value of an uninitialized pointer is an invalid pointer value and,
according to the following, it also is a singular pointer value:
[lib.iterator.requirements] p. 5
"[...] Iterators can also have singular values that are not associated
with any container. [Example: After the declaration of an uninitialized
pointer x (as with int* x;), x must always be assumed to have a singular
value of a pointer. ] Results of most expressions are undefined for
singular values; the only exception is an assignment of a non-singular
value to an iterator that holds a singular value. [...]"
...and as the above states, the only thing that can be done with a
singular pointer value is to assign a non-singular pointer value to it.
Following the informal reasoning above, we can safely assign zero or the
address of a valid object to that invalid pointer (i.e. that pointer
containing a singular value).
Fast forward now...
OK about the conversion, still not convinced about the UB.
Fast forward once more...
>> If that's the case, then any uninitialized_fill performed on a storage
>> area of
>> raw pointers will lead to UB, as the Standard depicts, as expected
>> effect, the
>> fact of comparing two invalid pointers:
>>
>> [citation formatted for presentation]
>>> 20.4.4.2 uninitialized_fill [lib.uninitialized.fill]
>>>
>>> template <class ForwardIterator, class T>
>>> void uninitialized_fill(ForwardIterator first,
>>> ForwardIterator last,
>>> const T& x);
>>>
>>> 1 Effects:
>>>
>>> for (; first != last; ++first)
>>> new (static_cast<void*>(&*first))
>>> typename iterator_traits<ForwardIterator>::value_type(x);
>>
>
> Huh, no.
>
> 'first' and 'last' here are not invalid pointers: if pointers, then they
> point /to/ the area to be filled.
Here we come to the point, assume this program, which should be
well-defined and well-behaving:
//-------
#include <iostream>
#include <memory>
using namespace std;
int main() {
size_t n = 4;
int* start = static_cast<int*>(
operator new(n * sizeof(int))
);
int* end = start + n;
uninitialized_fill(start, end, 42);
for(int* i = start; i < end; ++i) {
cout << *i << endl;
}
operator delete(start);
return 0;
}
//-------
By the time "start" gets initialized, it points to an uninitialized
storage area big enough to hold an int, but since that storage is
uninitialized, the pointer is currently invalid (we cannot dereference
it without invoking UB). "end" is an invalid pointer too.
Let's now enter the uninitialized_fill template function.
It gets called with this pseudo-signature:
uninitialized_fill<int*, int>(...)
which means that in the "Expected" section cited above, we have:
for (; first != last; ++first)
where "first == start" and "last == end", and all of them are of type
"class ForwardIterator = int*"
Following from all the above, we should have a standard algorithm that
invokes UB by comparing two invalid pointers.
Where is my reasoning flawed?
>> Would all the above mean that we shouldn't really worry about UB when
>> dealing
>> with invalid pointers into standard containers as long as we don't
>> dereference
>> such invalid pointers,
>
> Formally you invoke UB when a vector containing invalid pointers is
> destroyed.
>
> That's because a simplistic implementation may iterate over the vector
> contents and do pseudo destructor calls on the pointers (or it can do
> anything at all).
>
> In practice it's not anything I'd worry about, because leaving a vector
> with invalid pointers is common practice, so implementations have to not
> crash on that. However, to play nice, I guess one should always zero a
> pointer in a vector (or other container) after making it invalid. Just
> making sure.
>
>
>> and accordingly, would that mean that the standard needs
>> to be modified to state these actions (copying and comparing of invalid
>> pointers) as well-defined?
>
> I dont't think so.
>
> If the standard was all too clear about everything then we'd have
> nothing to discuss.
That doesn't really seem a good reason to keep a self-contradicting
standard (if it really is the case). I'd like to think that you're just
kidding :-)
Alf P. Steinbach /Usenet
Yah, mostly that was my reasoning in my old "pointers tutorial" (referenced from
my blog, right hand column somewhere).
I introduced the concept of "RealGood" pointers there.
But unfortunately that term did not catch on.
[snip]
The pointers are not invalid. They can be dereferenced. What you can't do is to
invoke an rvalue conversion on *p, because that would use an indeterminate
value. p itself is valid, *p is a valid reference, (*p)+2, for example, is bad.
[snip]
>>
>>> and accordingly, would that mean that the standard needs
>>> to be modified to state these actions (copying and comparing of invalid
>>> pointers) as well-defined?
>>
>> I dont't think so.
>>
>> If the standard was all too clear about everything then we'd have
>> nothing to discuss.
>
> That doesn't really seem a good reason to keep a self-contradicting standard (if
> it really is the case). I'd like to think that you're just kidding :-)
He he. :-)
Only partially... ;-)
Stuart Redmann
I would rather like it if the standard made it some kind of platform-
dependent. Since nobody can cite some convincing rationale for UB, and
apparently lots of people use deleted pointer in containers, it makes
little sense to say that all those programs show UB.
Regards,
Stuart
Francesco S. Carta
All right... I think I'm starting to understand. The difference between
the above and the following (restored from your previous reply)...
>> Now my question is, would the following test also lead to an lvalue to rvalue
>> conversion on "p", therefore leading to UB?
>>
>> int* p = new int;
>> delete p;
>> int* q = new int;
>> if(q != p) {
>> //...
>> }
>
> Yes, this invokes rvalue conversion and UB.
...is that "start" points to allocated (even if uninitialized) memory,
while "p" points to deallocated memory, is this all the difference that
makes one case as well-defined and the other as UB?
Francesco S. Carta
03/09/2010 13:28:40, wrote:
> Formally you invoke UB when a vector containing invalid pointers is
> destroyed.
>
> That's because a simplistic implementation may iterate over the vector
> contents and do pseudo destructor calls on the pointers (or it can do
> anything at all).
I forgot to ask about this. What is a pseudo destructor call?
Alf P. Steinbach /Usenet
> Alf P. Steinbach /Usenet <alf.p.stein...@gmail.com>, on 03/09/2010
> 13:28:40, wrote:
>
>> Formally you invoke UB when a vector containing invalid pointers is
>> destroyed.
>>
>> That's because a simplistic implementation may iterate over the vector
>> contents and do pseudo destructor calls on the pointers (or it can do
>> anything at all).
>
> I forgot to ask about this. What is a pseudo destructor call?
template< class Type >
void destroy( T& x ) { x.~Type(); }
invoked with say Type as int.
Cheers & hth.,
- Alf (and else-thread: yes)
Francesco S. Carta
> * Francesco S. Carta, on 03.09.2010 17:34:
>> Alf P. Steinbach /Usenet <alf.p.stein...@gmail.com>, on
>> 03/09/2010
>> 13:28:40, wrote:
>>
>>> Formally you invoke UB when a vector containing invalid pointers is
>>> destroyed.
>>>
>>> That's because a simplistic implementation may iterate over the vector
>>> contents and do pseudo destructor calls on the pointers (or it can do
>>> anything at all).
>>
>> I forgot to ask about this. What is a pseudo destructor call?
>
> template< class Type >
> void destroy( T& x ) { x.~Type(); }
>
>
> invoked with say Type as int.
Ah, I see. Though, I cannot figure any implementation of std::vector not
calling the destructor on the types it contains - furthermore as it will
resolve to a no-op for the types that do not have a destructor. I wonder
why you labeled that as a simplistic implementation.
Francesco S. Carta
Alf, I suppose your "(and else-thread: yes)" was about the above.
I wonder why some of you "better knowing" ones need to be "extracted"
the info with the clamps, sometimes ;-)
But I think it's fine, nonetheless. It forced me to dig the issue till I
realized the actual answer: had you pointed it out directly, it would
not have had the same value for my comprehension, so thank you very much
for your replies, Alf.
Bo Persson
> On 3 Sep., Francesco S. Cartawrote:
>>
>> when dealing with invalid pointers into standard containers as
>> long as we don't dereference such invalid pointers, and
>> accordingly, would that mean that the standard needs to be
>> modified to state these actions (copying and comparing of invalid
>> pointers) as well-defined?
>
> I would rather like it if the standard made it some kind of
> platform- dependent. Since nobody can cite some convincing
> rationale for UB, and apparently lots of people use deleted pointer
> in containers, it makes little sense to say that all those programs
> show UB.
>
You can't test for UB, because it IS undefined. "Seems to work" isn't
good enough! :-)
Those of us who used to program with 16-bit segmented memory (80286)
know that a deallocated segment WILL trap if you load a segment
register with a selector for a segment that has been removed from the
descriptor tables.
The hardware is still there for 32-bit x86, but the most popular
operating systems have decided to just use a single segment, which
masks that problem. Is this good enough reason to change the language
definition? Who knows!?
Bo Persson
Andy Champ
>>
>
> You can't test for UB, because it IS undefined. "Seems to work" isn't
> good enough! :-)
>
> Those of us who used to program with 16-bit segmented memory (80286)
> know that a deallocated segment WILL trap if you load a segment
> register with a selector for a segment that has been removed from the
> descriptor tables.
>
> The hardware is still there for 32-bit x86, but the most popular
> operating systems have decided to just use a single segment, which
> masks that problem. Is this good enough reason to change the language
> definition? Who knows!?
>
Hmm, I'm still puzzled. I've worked with architectures where a block of
memory is precisely bounded (fortunately for my sanity, never on a '286
- I've only ever used the MMU to get to bigger addresses, and _that_
gave me a headache) but even in those merely accessing the memory
containing the invalid pointer wouldn't cause a fault.
Copying the pointer - I can see why that could cause a fault, if the
system chose to copy the pointer as a pointer not raw bytes, and loaded
into some kind of pointer register. At that time the HW has the
opportunity to validate it.
All the architectures I've done this on BTW give you the memory you ask
for, and range check it - so de-referencing p+1 where p points to the
last element would fault. (And I've not used C in anger on any of them,
never mind C++)
But I still can't imagine any architecture where copying the bits of an
invalid pointer into an int (via a union) might cause a fault, but any
valid pointer would be fine. Not even the ones where ints are sometimes
BCD - on those valid pointers might fault.
Ah well, these day's I'm on X86/X64 only (or .net :( ) and the
architecture is flat with no protection - so not my problem!
Andy
Pavel
> On 03/09/2010 19:20, Bo Persson wrote:
>>>
>>
>> You can't test for UB, because it IS undefined. "Seems to work" isn't
>> good enough! :-)
>>
>> Those of us who used to program with 16-bit segmented memory (80286)
>> know that a deallocated segment WILL trap if you load a segment
>> register with a selector for a segment that has been removed from the
>> descriptor tables.
>>
>> The hardware is still there for 32-bit x86, but the most popular
>> operating systems have decided to just use a single segment, which
>> masks that problem. Is this good enough reason to change the language
>> definition? Who knows!?
>>
>
> Hmm, I'm still puzzled. I've worked with architectures where a block of
> memory is precisely bounded (fortunately for my sanity, never on a '286
> - I've only ever used the MMU to get to bigger addresses, and _that_
> gave me a headache) but even in those merely accessing the memory
> containing the invalid pointer wouldn't cause a fault.
hardware type -- that is define how it is stored in memory, and (very
probably for a pointer type) how it is stored in registers. Some
architectures have special registers specifically for the pointers, in
particular in 8086/8088 (not even 286), one kind of pointers, namely
"long" pointers (two 16-bit words) had to be loaded into a pair of
16-bit special registers, for example DS:SI or DS:EI. It has some
advantages (as well as drawbacks) to use special registers for the
pointers as opposed to using general registers for both pointers and
integers; in particular, as soon as a special pointer register is
loaded, the hardware can speculate that is probably will be dereferenced
soon, so the hardware can start some background job of reloading shadow
translation tables (as in protected mode of 80286) or updating the
nearby cache lines (as in modern CPUs) etc. You can think of it as a
"pre-fetch" or "proactive hardware-level-dereferencing". It is up to the
hardware designer how to work out the situation when the "hardware
dereferencing" can't be done because the memory pointed to does not
exist -- whether to work around the error or throw it back in the face
of the faulty program.
Why would a compiler load the pointers to the registers if no
dereferencing is required by the program? -- for example to compare them
(a good explanation of why comparing of not valid pointers should be
UB). In real mode of 8086, for example, addresses A000:ABCD and
AA00:0BCD were pointing to the same byte in the memory -- it would be
logical if the hardware registers provided some help in comparison these
for equality and lesser/greater (in reality they did not though as far
as I can remember).
Pointing to non-existing memory is only one example of how uninitialized
pointer can alarm the hardware; others may include special "mode" bits
in the pointers where not every combination makes sense or allowed in
the current CPU mode and what not (although I don't have ready examples
for these).
>
> Copying the pointer - I can see why that could cause a fault, if the
> system chose to copy the pointer as a pointer not raw bytes, and loaded
> into some kind of pointer register. At that time the HW has the
> opportunity to validate it.
>
> All the architectures I've done this on BTW give you the memory you ask
> for, and range check it - so de-referencing p+1 where p points to the
> last element would fault. (And I've not used C in anger on any of them,
> never mind C++)
>
> But I still can't imagine any architecture where copying the bits of an
> invalid pointer into an int (via a union) might cause a fault, but any
> valid pointer would be fine. Not even the ones where ints are sometimes
> BCD - on those valid pointers might fault.
>
> Ah well, these day's I'm on X86/X64 only (or .net :( ) and the
> architecture is flat with no protection - so not my problem!
>
> Andy
-Pavel
Bo Persson
> On 03/09/2010 19:20, Bo Persson wrote:
>>>
>>
>> You can't test for UB, because it IS undefined. "Seems to work"
>> isn't good enough! :-)
>>
>> Those of us who used to program with 16-bit segmented memory
>> (80286) know that a deallocated segment WILL trap if you load a
>> segment register with a selector for a segment that has been
>> removed from the descriptor tables.
>>
>> The hardware is still there for 32-bit x86, but the most popular
>> operating systems have decided to just use a single segment, which
>> masks that problem. Is this good enough reason to change the
>> language definition? Who knows!?
>>
>
> Hmm, I'm still puzzled. I've worked with architectures where a
> block of memory is precisely bounded (fortunately for my sanity,
> never on a '286 - I've only ever used the MMU to get to bigger
> addresses, and _that_ gave me a headache) but even in those merely
> accessing the memory containing the invalid pointer wouldn't cause
> a fault.
> Copying the pointer - I can see why that could cause a fault, if the
> system chose to copy the pointer as a pointer not raw bytes, and
> loaded into some kind of pointer register. At that time the HW has
> the opportunity to validate it.
Right. :-)
And the language standard has chosen not to prescribe how the hardware
should copy a pointer.
>
> All the architectures I've done this on BTW give you the memory you
> ask for, and range check it - so de-referencing p+1 where p points
> to the last element would fault. (And I've not used C in anger on
> any of them, never mind C++)
>
> But I still can't imagine any architecture where copying the bits
> of an invalid pointer into an int (via a union) might cause a
> fault, but any valid pointer would be fine. Not even the ones
> where ints are sometimes BCD - on those valid pointers might fault.
>
> Ah well, these day's I'm on X86/X64 only (or .net :( ) and the
> architecture is flat with no protection - so not my problem!
>
No, but it is language problem. You can very well "limit" the
portability by specifying flat memory, 8-bit bytes, 32-bit ints, and
IEEE floating point for your programs. Java does that for you, and I
guess .NET does too.
C++ wants to be *natively* implementable on a wider range of
platforms. Therefore it cannot specify copying a pointer as an int,
because that might hurt badly on hardware with special address
registers, or CPUs without special integer registers.
http://en.wikipedia.org/wiki/CDC_6600#The_Central_Processor_.28CP.29
http://en.wikipedia.org/wiki/Motorola_68000#Internal_registers
It is true that most current hardware doesn't show these
peculiarities, but why limit a programming language to current
hardware?
Bo Persson
Andy Champ
bits, and they have been propagated across to the more recent
architectures so that they can run '286 code.
>
>>
>> All the architectures I've done this on BTW give you the memory you
>> ask for, and range check it - so de-referencing p+1 where p points
>> to the last element would fault. (And I've not used C in anger on
>> any of them, never mind C++)
>>
>> But I still can't imagine any architecture where copying the bits
>> of an invalid pointer into an int (via a union) might cause a
>> fault, but any valid pointer would be fine. Not even the ones
>> where ints are sometimes BCD - on those valid pointers might fault.
>>
>> Ah well, these day's I'm on X86/X64 only (or .net :( ) and the
>> architecture is flat with no protection - so not my problem!
>>
>
> No, but it is language problem. You can very well "limit" the
> portability by specifying flat memory, 8-bit bytes, 32-bit ints, and
> IEEE floating point for your programs. Java does that for you, and I
> guess .NET does too.
>
> C++ wants to be *natively* implementable on a wider range of
> platforms. Therefore it cannot specify copying a pointer as an int,
> because that might hurt badly on hardware with special address
> registers, or CPUs without special integer registers.
>
> http://en.wikipedia.org/wiki/CDC_6600#The_Central_Processor_.28CP.29
>
> http://en.wikipedia.org/wiki/Motorola_68000#Internal_registers
>
>
> It is true that most current hardware doesn't show these
> peculiarities, but why limit a programming language to current
> hardware?
>
I suspect the language is already limited.
Take this nice simple bit of C:
char* p = malloc(10);
How is the compiler supposed to know that the pointer returned from
malloc is supposed to be a pointer to chars? Some architectures have
different pointer types depending on the target data. That'll give me
10 bytes back, and I suppose the compiler could work out that the cast
from void* to char* should do a pointer conversion to a char type
pointer with a bound of 10.
Then I write
float* fp = (void*)p;
Floats are bigger. Let's say that they have a size of 8 bytes on this
architecture - so is it going to put a range check on the memory pointer
of 1 float? And if it does, and I cast it back to char*, will it
remember the size of 10?
What do you do on those TI graphics processors, where the native address
is a bit address not a byte address? Is sizeof(char) == 8? (We just
wrote _c_a_r_e_f_u_l_l_y_, especially when we were dealing with the
arrays of 3-bit items which were the graphics memory, whose sizeof was
not a good thing to ask!)
Andy
Stuart Redmann
>
> >> Would all the above mean that we shouldn't really worry about UB
> >> when dealing with invalid pointers into standard containers as
> >> long as we don't dereference such invalid pointers, and
> >> accordingly, would that mean that the standard needs to be
> >> modified to state these actions (copying and comparing of invalid
> >> pointers) as well-defined?
Stuart Redmann wrote:
> > I would rather like it if the standard made it some kind of
> > platform- dependent. Since nobody can cite some convincing
> > rationale for UB, and apparently lots of people use deleted pointer
> > in containers, it makes little sense to say that all those programs
> > show UB.
>
On 3 Sep., Bo Persson wrote:
> You can't test for UB, because it IS undefined. "Seems to work" isn't
> good enough! :-)
I don't quite get you. What do you mean by "testing" for UB? "Seems to
work" is good enough for me when the compiler emits the right binary
(if the produced binary is well-formed, I don't need to care whether
the compiled source code is ill-formed).
> Those of us who used to program with 16-bit segmented memory (80286)
> know that a deallocated segment WILL trap if you load a segment
> register with a selector for a segment that has been removed from the
> descriptor tables.
Right. But from my point of view loading a pointer into the segment
registers is only necessary if you want to dereference it. It should
not be a problem to compare two invalid pointers for equality without
loading either of them into the segment selectors. The same goes for
destructing an invalid pointer (which is a no-op even on the 8086
AFAIK).
> The hardware is still there for 32-bit x86, but the most popular
> operating systems have decided to just use a single segment, which
> masks that problem. Is this good enough reason to change the language
> definition? Who knows!?
I wanted to say that the standard could say something like this: Any
use of invalid pointers except dereferencing is platform-dependent,
dereferencing it is UB.
Regards,
Stuart
James Kanze
> On 3 Sep., Francesco S. Cartawrote:
[...]
> > Would all the above mean that we shouldn't really worry about UB when
> > dealing with invalid pointers into standard containers as long as we
> > don't dereference such invalid pointers, and accordingly, would that
> > mean that the standard needs to be modified to state these actions
> > (copying and comparing of invalid pointers) as well-defined?
> I would rather like it if the standard made it some kind of platform-
> dependent. Since nobody can cite some convincing rationale for UB, and
> apparently lots of people use deleted pointer in containers, it makes
> little sense to say that all those programs show UB.
I'd rather the standard just require those programs to be legal
and well defined. There are two issues:
-- lvalue to rvalue conversion of an invalid pointer, and
-- allowing a container to make gratuitous lvalue to rvalue
conversions, for no real reason.
I have no problem with the lvalue to rvalue conversion remaining
undefined behavior, but I do object to the idea that it might
occur when I'm not looking, when there's no reason for it to
occur. If, for example, I have a class which contains an
std::vector<T*> myVector (for some type T), and in the
destructor I write:
for (std::vector<T*>::iterator iter = myVector.begin();
iter != myVector.end();
++ iter) {
delete *iter;
}
there should be no undefined behavior. Similarly in the case
of:
delete myVector[i];
myVector[i] = NULL;
In these two cases (and many others), there's just no reason for
an lvalue to rvalue conversion of the invalid pointer to occur
(and in fact, it doesn't occur in any existing implementation).
--
James Kanze
James Kanze
> On 04/09/2010 13:01, Bo Persson wrote:
[...]
> > C++ wants to be *natively* implementable on a wider range of
> > platforms. Therefore it cannot specify copying a pointer as an int,
> > because that might hurt badly on hardware with special address
> > registers, or CPUs without special integer registers.
> >http://en.wikipedia.org/wiki/CDC_6600#The_Central_Processor_.28CP.29
> >http://en.wikipedia.org/wiki/Motorola_68000#Internal_registers
> > It is true that most current hardware doesn't show these
> > peculiarities, but why limit a programming language to current
> > hardware?
> I suspect the language is already limited.
> Take this nice simple bit of C:
> char* p = malloc(10);
> How is the compiler supposed to know that the pointer returned from
> malloc is supposed to be a pointer to chars?
At that level, it doesn't. All it can do is memorize the upper
and lower bounds.
> Some architectures have different pointer types depending on
> the target data. That'll give me 10 bytes back, and I suppose
> the compiler could work out that the cast from void* to char*
> should do a pointer conversion to a char type pointer with
> a bound of 10.
> Then I write
> float* fp = (void*)p;
> Floats are bigger. Let's say that they have a size of 8 bytes
> on this architecture - so is it going to put a range check on
> the memory pointer of 1 float?
No, but it can easily generate code which will detect that fp+1
will access bytes beyond the address p+10 (if it is
dereferenced).
> And if it does, and I cast it back to char*, will it remember
> the size of 10?
It doesn't have to. All it has to do is remember the upper and
lower bounds.
> What do you do on those TI graphics processors, where the
> native address is a bit address not a byte address? Is
> sizeof(char) == 8?
Or bigger. That's a requirement of the standard.
> (We just wrote _c_a_r_e_f_u_l_l_y_, especially when we were
> dealing with the arrays of 3-bit items which were the graphics
> memory, whose sizeof was not a good thing to ask!)
Most likely, if you were using C, the compiler had some
extensions to support the extra addressing possibilities.
--
James Kanze
Francesco S. Carta
So, if I understand you correctly, the problem is that the standard does
not explicitly forbid lvalue to rvalue conversions during the
destruction of a container, did I understand you correctly?
I suppose you're /not/ including, say, vector reallocation as a case
where the lvalue to rvalue conversion would be gratuitous - I ask
because I cannot conceive any way to reallocate the internals of a
vector without performing such a conversion, at least not under the
current standard.
Francesco S. Carta
Uh? Is that an oversight of yours, James, or you did really mean that?
As I read the standard, sizeof(char) == 1, no more, no less... maybe you
mistaken that line for CHAR_BIT == 8?
If an implementation has sizeof(char) != 1, that would be not
conforming, if I'm not mistaken.
Bo Persson
> On 3 Sep., Francesco S. Cartawrote:
>>
>>>> Would all the above mean that we shouldn't really worry about UB
>>>> when dealing with invalid pointers into standard containers as
>>>> long as we don't dereference such invalid pointers, and
>>>> accordingly, would that mean that the standard needs to be
>>>> modified to state these actions (copying and comparing of invalid
>>>> pointers) as well-defined?
>
> Stuart Redmann wrote:
>>> I would rather like it if the standard made it some kind of
>>> platform- dependent. Since nobody can cite some convincing
>>> rationale for UB, and apparently lots of people use deleted
>>> pointer in containers, it makes little sense to say that all
>>> those programs show UB.
>>
>
> On 3 Sep., Bo Persson wrote:
>> You can't test for UB, because it IS undefined. "Seems to work"
>> isn't good enough! :-)
>
> I don't quite get you. What do you mean by "testing" for UB? "Seems
> to work" is good enough for me when the compiler emits the right
> binary (if the produced binary is well-formed, I don't need to care
> whether the compiled source code is ill-formed).
I mean running unit tests for your code.
Because the UB doesn't have to be consistent, passing the tests
doesn't tell us if the code works, just that it works sometimes.
"Seems to work".
UB is really evil!
Bo Persson