Indeterminate Pointer Values and Trapping

[wil...@wilbur.25thandclement.com]

The standard says that after free, a pointer value becomes indeterminate.
C11 (N1570) 6.2.4p1.

An indeterminate value is "either an unspecified value or a trap
representation". supra 3.19.2p1.

My question is, can a value become a trap representation even if the
bit-value doesn't change? In other words, may an implementation abort the
program if you load or compute a value (not the referenced object) which has
recently become indeterminate?

[Keith Thompson]

Yes. The subset of the 2**N possible representations that represent
valid values of a pointer type is not required to remain the same
throughout the execution of a program.

Is there some reason you think an implementation is not permitted to
behave that way?

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[Kaz Kylheku]

On 2014-04-29, <wil...@wilbur.25thandClement.com> <wil...@wilbur.25thandClement.com> wrote:
> My question is, can a value become a trap representation even if the
> bit-value doesn't change?

Can your lost or stolen credit card be canceled without the plastic being
altered, such that attempted uses of that card are "trapped"?

[James Kuyper]

The standard specifies no details about the representation of pointer
types. Certain pointer types are supposed to have the same
representation as other types, and that's about it. That leaves open the
possibility that the piece of memory pointed at by a pointer containing
a given bit pattern could be time dependent; and in particular, that the
bit pattern might cease to validly represent any memory location. This
is not an abstract possibility, but something that comes up with real
world systems. A pointer can refer to a block of memory that was
available during the lifetime of the object it pointed at, but which was
allowed to become unavailable once that lifetime ended. 6.2.4p1 is what
allows an implementation of C to be fully conforming, despite making use
of that feature.

You talked about computing a value that has recently become
indeterminate, presumably by reason of violating 6.2.4p1. You can't do
that, at least not with behavior that would otherwise be well-defined.
Any attempt to compute such a value has undefined behavior before it
could even qualify as a value that would otherwise violate 6.2.4p1.

You might think you could take a pointer 'p' that points into the
allocated block of memory, and compute 'p+n', where n is an integer
value that leaves the pointer still pointing inside that block. However,
just loading the value of 'p' violates 6.2.4p1, even before you get a
chance to add 'n'. You might think you could take a pointer 'q' that
points somewhere outside of that block, and add an integer value 'm' to
it, such that 'q+m' points into that block. However, all such
calculations run afoul of 6.5.6p8, before the result even has a chance
of qualifying as a violation of 6.2.4p1. The same is true, for several
other different reasons, for any other way of calculating such a
pointer. The only thing that makes it permissible for such calculations
to generate a pointer that might point inside the deallocated block of
memory is the fact that the calculation itself has undefined behavior.

[wil...@wilbur.25thandclement.com]

Keith Thompson <ks...@mib.org> wrote:
> <wil...@wilbur.25thandClement.com> writes:
>> The standard says that after free, a pointer value becomes indeterminate.
>> C11 (N1570) 6.2.4p1.
>>
>> An indeterminate value is "either an unspecified value or a trap
>> representation". supra 3.19.2p1.
>>
>> My question is, can a value become a trap representation even if the
>> bit-value doesn't change? In other words, may an implementation abort the
>> program if you load or compute a value (not the referenced object) which
>> has recently become indeterminate?
>
> Yes. The subset of the 2**N possible representations that represent
> valid values of a pointer type is not required to remain the same
> throughout the execution of a program.
>
> Is there some reason you think an implementation is not permitted to
> behave that way?
>

I didn't think that. But I was having a discussion on another mailing list
and the subject came up. I originally contended that a pointer value is
"invalid"* after a call to free, and portable code shouldn't use such
pointers, regardless of whether they were dereferenced.

Someone else argued that the standard only spoke of trap representations,
and as the representation didn't change after free, it couldn't be a trap
representation.

I was going to respond that the set of trap representation wasn't
necessarily fixed, and that such "invalid" pointers could result in a trap.
At the very least, I knew of at least one implementation** that
instrumented*** code to verify values at every load and store of pointer
addresses, and would exit on such a pointer.

But before I responded I wanted to do a sanity check in this forum.


* I concede that "invalid" is imprecise.
** TinyCC's Memory and Bounds Checker.
*** TinyCC's instrumentation exited on user code that decremented pointers
to iterate buffers, even though (array-1) was only compared and never
dereferenced.

[Robert Wessel]

n x86-16, in protected mode, pointers can become invalid if the
segment pointed to is deallocated by the OS. Most C compilers are
pretty conservative about loading segment registers until it's clear
the address is going to be used to access memory, so they tend to
avoid the trap on the now-invalid selector value.

[glen herrmannsfeldt]

Robert Wessel <robert...@yahoo.com> wrote:

(snip, someone wrote)

>> I was going to respond that the set of trap representation wasn't
>> necessarily fixed, and that such "invalid" pointers could result
>> in a trap.
>> At the very least, I knew of at least one implementation** that
>> instrumented*** code to verify values at every load and store
>> of pointer addresses, and would exit on such a pointer.

> n x86-16, in protected mode, pointers can become invalid if the
> segment pointed to is deallocated by the OS. Most C compilers are
> pretty conservative about loading segment registers until it's clear
> the address is going to be used to access memory, so they tend to
> avoid the trap on the now-invalid selector value.

Note that all the logic is still there, but in IA32 (32 bit x86)
segment lengths are 32 bits. Pretty much all systems allocated
one or two segments.

The Watcom 32 bit C compiler can (or could last time I looked)
generate large model (16 bit selector, 32 bit offset) code.

Otherwise, historically there have been hardware that accessed
all data through descriptors of some kind, and so have similar
restrictions on pointers.

Early MacOS required programs to access system managed data blocks
as doubly indirect. (That is, pointers to pointers.) Instead
of the C compiler doing that all internally, programs had to do
it explicitely. There were some complicated rules on pointers
to keep everything consistent. Pointers could become invalid
on any system call, not just free().

-- glen

[Ken Brody]

Yes.

Consider, for example, the x86 series of CPUs. When running in "protected
mode" (which virtually all modern O/Ses use) and using "far pointers", the
mere act of loading an invalid value into a selector register (far pointers
are selector/offset) will cause a hardware fault.

--
Kenneth Brody

[Ken Brody]

On 4/30/2014 3:17 AM, glen herrmannsfeldt wrote:
[...]

> Early MacOS required programs to access system managed data blocks
> as doubly indirect. (That is, pointers to pointers.) Instead
> of the C compiler doing that all internally, programs had to do
> it explicitely. There were some complicated rules on pointers
> to keep everything consistent. Pointers could become invalid
> on any system call, not just free().

I assume the original pointer to the pointer remained valid?

In any case, this sounds conceptually similar to the old Windows "handles"
to objects. Handles couldn't be dereferenced (they weren't pointers), but
you could "lock" the object it pointed to, which would return a pointer.
When you were done, you would "unlock" the object, and the pointer would
become invalid. (My understanding is that a handle was really a pointer to
an opaque section of memory which maintained the information about the
object itself. However, you couldn't access that memory.)

--
Kenneth Brody

[Kenny McCormack]

In article <ljqtdf$5r3$1...@dont-email.me>,
Ken Brody <kenb...@spamcop.net> wrote:
...

>In any case, this sounds conceptually similar to the old Windows "handles"
>to objects.

Why do you say "old" ?

--
"That's the eternal flame."[7] The single became another worldwide hit.[8]

Hoffs was actually naked when she recorded the song, after being convinced
by Sigerson that Olivia Newton-John got her amazing performances by
recording everything while naked.[9]

(From: http://en.wikipedia.org/wiki/The_Bangles)

[Richard]

gaz...@shell.xmission.com (Kenny McCormack) writes:

> In article <ljqtdf$5r3$1...@dont-email.me>,
> Ken Brody <kenb...@spamcop.net> wrote:
> ...
>>In any case, this sounds conceptually similar to the old Windows "handles"
>>to objects.
>
> Why do you say "old" ?

Its probably used in the vernacular as in "how's tricks old son" etc.



--
"Avoid hyperbole at all costs, its the most destructive argument on
the planet" - Mark McIntyre in comp.lang.c

[Kaz Kylheku]

On 2014-04-30, Richard <rgr...@gmail.com> wrote:
> gaz...@shell.xmission.com (Kenny McCormack) writes:
>
>> In article <ljqtdf$5r3$1...@dont-email.me>,
>> Ken Brody <kenb...@spamcop.net> wrote:
>> ...
>>>In any case, this sounds conceptually similar to the old Windows "handles"
>>>to objects.
>>
>> Why do you say "old" ?
>
> Its probably used in the vernacular as in "how's tricks old son" etc.

Or maybe Old Shatterhand (character in Karl May novels like _Winnetou_).

That Old Shatter-HWND, has PWND his foe, LOL. :)

[Ken Brody]

On 4/30/2014 11:55 AM, Kenny McCormack wrote:
> In article <ljqtdf$5r3$1...@dont-email.me>,
> Ken Brody <kenb...@spamcop.net> wrote:
> ...
>> In any case, this sounds conceptually similar to the old Windows "handles"
>> to objects.
>
> Why do you say "old" ?

Yes, Windows still uses handles for many things. However, you no longer
need to allocate memory in that manner.

--
Kenneth Brody

[Kenny McCormack]

In article <lk0fjq$s7b$1...@dont-email.me>,

Ken Brody <kenb...@spamcop.net> wrote:
>On 4/30/2014 11:55 AM, Kenny McCormack wrote:
>> In article <ljqtdf$5r3$1...@dont-email.me>,
>> Ken Brody <kenb...@spamcop.net> wrote:
>> ...
>>> In any case, this sounds conceptually similar to the old Windows "handles"
>>> to objects.
>>
>> Why do you say "old" ?
>
>Yes, Windows still uses handles for many things. However, you no longer
>need to allocate memory in that manner.

I see. I assumed from your use of the word that it was somehow in the "but
not anymore" category. I actually was asking the question straight (no
implication that you were wrong or anything), since I haven't followed
Windows development that closely (again, no implication that there's
anything wrong with Windows - I use it and think it is a perfectly fine OS
- I just haven't followed the gory details closely in a long time).

Can you say a few things about:

1) Why "you no longer need to allocate memory in that manner" ? What has
changed? FWIW, the original idea seemed pretty sensible to me.

2) In what other ways Windows still uses "handles"? I ask this because
many people use the word "handle" more or less synonymously with
"descripter" - that is, w/o the Windows-specific (AFAIK) meaning of (in
sort of C terms) "pointer to pointer to memory" as opposed to a simple
"pointer to memory".

--
Modern Conservative: Someone who can take time out
from demanding more flag burning laws, more abortion
laws, more drug laws, more obscenity laws, and more
police authority to make warrantless arrests to remind
us that we need to "get the government off our backs".