Testing if a pointer is valid

jacob navia

unread,

Sep 17, 2011, 6:29:38 PM9/17/11

to

In a recent discussion about preconditions, I argued that testing ALL
preconditions would be unpractical. As an example of a precondition
impossible to test I presented the problem to know if a pointer
points to a really readable place in memory.

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

But after doing some research, I found the following equivalent,
written in almost standard C.
---------------------------------------------------------cut here
/* Heavily adapted from:
http://fixunix.com/linux/337646-isbadreadptr-linux.html */
#include <setjmp.h>
#include <signal.h>
static int PtrTestInstalled;
static jmp_buf PtrTestJmpBuf;
static void PtrTestHandler(int nSig)
{
if (PtrTestInstalled)
longjmp(PtrTestJmpBuf, 1);
}

int IsBadReadPtr(void* lp, size_t ObjectSize)
{
size_t i;
volatile unsigned char c;
int r = 1;
void (* PrevHandler)(int);
PtrTestInstalled = 1;
if (setjmp(PtrTestJmpBuf)) {
r = 0;
goto Ret;
}
PrevHandler = signal(SIGSEGV, PtrTestHandler);

for (i = 0; i < ObjectSize; i ++)
c = ((unsigned char*)lp)[i];
Ret:
PtrTestInstalled = 0;
signal(SIGSEGV, PrevHandler);

return r;
}
--------------------------------------------------------cut here

This looks like a promising stuff. The only non-standard
thing here is the SIGSEGV constant. That constant is not
in the standard but defined under windows, linux, and
most Unix variants.
Does anybody see any potentially non-standard stuff here?

Thanks

Harald van Dĳk

unread,

Sep 17, 2011, 6:38:33 PM9/17/11

to

On Sep 18, 12:29 am, jacob navia <ja...@spamsink.net> wrote:
> In a recent discussion about preconditions, I argued that testing ALL
> preconditions would be unpractical. As an example of a precondition
> impossible to test I presented the problem to know if a pointer
> points to a really readable place in memory.
>
> The windows API provides the function
> bool IsBadReadPtr(const void *lp, size_t ucb);
>
> This takes a pointer and a size and returns true if it is OK, zero
> if it fails.
>
> Problem is, that is only present under windows.

That is not its biggest problem. Even on Windows, don't use it. Please
read
<http://blogs.msdn.com/b/oldnewthing/archive/2006/09/27/773741.aspx>
for reasons why the concept of IsBadReadPtr (and IsBadWritePtr even
more so) is fundamentally flawed.

BartC

unread,

Sep 17, 2011, 6:54:47 PM9/17/11

to

"jacob navia" <ja...@spamsink.net> wrote in message
news:j5370i$1nv$1...@speranza.aioe.org...

> In a recent discussion about preconditions, I argued that testing ALL
> preconditions would be unpractical. As an example of a precondition
> impossible to test I presented the problem to know if a pointer
> points to a really readable place in memory.
>
> The windows API provides the function
> bool IsBadReadPtr(const void *lp, size_t ucb);
>
> This takes a pointer and a size and returns true if it is OK, zero
> if it fails.

Even if the pointer is OK, how does it know it points to the right place?

(BTW the MSDN specs say: "Despite its name, it does not guarantee that the
pointer is valid or that the memory pointed to is safe to use.")

--
Bartc

jacob navia

unread,

Sep 17, 2011, 6:57:32 PM9/17/11

to

Le 18/09/11 00:38, Harald van Dĳk a écrit :

They aren't "flawed". They just could interfere with guard pages. The
solution is very easy: You call (before the loop) a virtual memory
status function and see if you are going to touch a guard page. I know
that such an API exists since I used it in the debugger a lot.

I was unable to know where a function pointer was really pointing and
wanted to implement the "step" command. That was easy: I read the memory
rights of the pages of the program and set all the code pages to
NOACCESS. That way the program would crash after the jump, I would get
called (since I am the debugger!)and I would reset the page rights, see
where the program landed, and continue happily the debugging.

jacob navia

unread,

Sep 17, 2011, 6:59:39 PM9/17/11

to

Le 18/09/11 00:54, BartC a écrit :

>
>
> "jacob navia" <ja...@spamsink.net> wrote in message
> news:j5370i$1nv$1...@speranza.aioe.org...
>> In a recent discussion about preconditions, I argued that testing ALL
>> preconditions would be unpractical. As an example of a precondition
>> impossible to test I presented the problem to know if a pointer
>> points to a really readable place in memory.
>>
>> The windows API provides the function
>> bool IsBadReadPtr(const void *lp, size_t ucb);
>>
>> This takes a pointer and a size and returns true if it is OK, zero
>> if it fails.
>
> Even if the pointer is OK, how does it know it points to the right place?
>

Well, that is the second part. You just put a "magic number" in your
structure.

Without this function, trying to access the magic number could trap
Now you can read it safely after you verified that the pointer is readable.

> (BTW the MSDN specs say: "Despite its name, it does not guarantee that
> the pointer is valid or that the memory pointed to is safe to use.")
>

Of course, how could it do that?

Heikki Kallasjoki

unread,

Sep 17, 2011, 7:35:48 PM9/17/11

to

On 2011-09-17, jacob navia <ja...@spamsink.net> wrote:
> The windows API provides the function
> bool IsBadReadPtr(const void *lp, size_t ucb);
>
> This takes a pointer and a size and returns true if it is OK, zero
> if it fails.
>
> Problem is, that is only present under windows.
>
> But after doing some research, I found the following equivalent,
> written in almost standard C.

[snip code calling longjmp() from a signal handler]

> Does anybody see any potentially non-standard stuff here?

The act of calling longjmp() from within a signal handler, while
possible in many circumstances/systems, is not quite completely
standard. The following page summarizes the state:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1318.htm

In particular, C89 has some verbiage to allow it (in certain cases),
which has been removed from C99; and even a C90 defect report response
qualifies that it is required to work only for signal handlers invoked
by raise() or abort(). It does not seem to appear on the lists of
signal-handler-safe functions in any standard.

--
Heikki Kallasjoki

jacob navia

unread,

Sep 17, 2011, 7:47:53 PM9/17/11

to

Le 18/09/11 01:35, Heikki Kallasjoki a écrit :

Yes, I see. Maybe a better approach would be to just read the
virtual memory information directly. Under windows it is easy by
querying the memory information API with VirtualQueryEx and
using that info. But then this is completely system specific and
I do not know the linux equivalent.

Geoff

unread,

Sep 17, 2011, 8:15:22 PM9/17/11

to

On Sun, 18 Sep 2011 00:29:38 +0200, jacob navia <ja...@spamsink.net>
wrote:

>In a recent discussion about preconditions, I argued that testing ALL
>preconditions would be unpractical. As an example of a precondition
>impossible to test I presented the problem to know if a pointer
>points to a really readable place in memory.
>
>The windows API provides the function
>bool IsBadReadPtr(const void *lp, size_t ucb);
>
>This takes a pointer and a size and returns true if it is OK, zero
>if it fails.
>
>Problem is, that is only present under windows.
>

Second problem is it's not recommended:

http://msdn.microsoft.com/en-us/library/aa366713(v=VS.85).aspx

"Important This function is obsolete and should not be used. Despite

its name, it does not guarantee that the pointer is valid or that the

memory pointed to is safe to use. For more information, see Remarks on
this page."

You would be better off using SEH and handling it there.

Максим Фомин

unread,

Sep 18, 2011, 3:43:37 AM9/18/11

to

I was thinking about this problem (and found several examples with
setjmp)
and my conclusion was that creating malloc wrapper function,
(which can validate pointers only to dynamic memory) is best and
portable option.
IMHO this problem araises when callee function uses memory allocated
by caller function
and may be the right approach is to fix the caller function.

Rosario Pulvirenti

unread,

Sep 18, 2011, 4:34:15 AM9/18/11

to

"jacob navia" <ja...@spamsink.net> ha scritto nel messaggio
news:j5370i$1nv$1...@speranza.aioe.org...

> In a recent discussion about preconditions, I argued that testing ALL
> preconditions would be unpractical. As an example of a precondition
> impossible to test I presented the problem to know if a pointer
> points to a really readable place in memory.

i could find [one loop on a list of all pointer released from myMalloc()
function and a couple of subtration] if a pointer point to some
MyMalloc area
if the stack memory is implementd trught myMalloc is possible to see
if one pointer point valid memory something about
O(numberPonters) [linear time]

but i can think on that we want O(1)
so something using hash function

Ike Naar

unread,

Sep 18, 2011, 8:55:41 AM9/18/11

to

On 2011-09-17, jacob navia <ja...@spamsink.net> wrote:

> In a recent discussion about preconditions, I argued that testing
> ALL
> preconditions would be unpractical. As an example of a precondition
> impossible to test I presented the problem to know if a pointer
> points to a really readable place in memory.
>
> The windows API provides the function
> bool IsBadReadPtr(const void *lp, size_t ucb);
>
> This takes a pointer and a size and returns true if it is OK, zero
> if it fails.
>
> Problem is, that is only present under windows.
>
> But after doing some research, I found the following equivalent,
> written in almost standard C.

> [int IsBadReadPtr(void* lp, size_t ObjectSize)]

>
> This looks like a promising stuff. The only non-standard
> thing here is the SIGSEGV constant. That constant is not
> in the standard but defined under windows, linux, and
> most Unix variants.
> Does anybody see any potentially non-standard stuff here?

One problem with IsBadReadPtr is that the behaviour of a program
after continuing from a SIGSEGV interrupt handler is undefined.

Another problem with IsBadReadPtr is that, even if it "works",
it probably detects only a subset of pointers that are invalid.

Using an invalid pointer is not guaranteed to cause a
SIGSEGV fault, other things may happen, for instance:
- a misaligned pointer may cause a SIGBUS fault
- dereferencing a null pointer may yield a zero value
- dereferencing an invalid pointer may yield a garbage value
etcetera.
Even for a given implementation the behaviour may be inconsistent.

As an example from the real world I ran the two programs below
on an amd64 under NetBSD 5.0.1 with gcc 4.1.3:

int main(void) { double p[1]; return p[1297] < 0.0; }

returns succesfully,

int main(void) { double p[1]; return p[1298] < 0.0; }

segfaults.

Kleuskes & Moos

unread,

Sep 18, 2011, 12:30:05 PM9/18/11

to

On Sun, 18 Sep 2011 00:57:32 +0200, jacob navia wrote:

> Le 18/09/11 00:38, Harald van Dĳk a écrit :
>> On Sep 18, 12:29 am, jacob navia<ja...@spamsink.net> wrote:
>>> In a recent discussion about preconditions, I argued that testing ALL
>>> preconditions would be unpractical. As an example of a precondition
>>> impossible to test I presented the problem to know if a pointer points
>>> to a really readable place in memory.
>>>
>>> The windows API provides the function bool IsBadReadPtr(const void
>>> *lp, size_t ucb);
>>>
>>> This takes a pointer and a size and returns true if it is OK, zero if
>>> it fails.
>>>
>>> Problem is, that is only present under windows.
>>
>> That is not its biggest problem. Even on Windows, don't use it. Please
>> read
>> <http://blogs.msdn.com/b/oldnewthing/archive/2006/09/27/773741.aspx>
>> for reasons why the concept of IsBadReadPtr (and IsBadWritePtr even
>> more so) is fundamentally flawed.
>
> They aren't "flawed".

If you need code to check whether or not a pointer is valid, _your code_
is fundamentally flawed.

-------------------------------------------------------------------------------
_________________________________________
< I joined scientology at a garage sale!! >
-----------------------------------------
\
\
___
{~._.~}
( Y )
()~*~()
(_)-(_)
-------------------------------------------------------------------------------

jacob navia

unread,

Sep 18, 2011, 1:25:49 PM9/18/11

to

Le 18/09/11 18:30, Kleuskes & Moos a écrit :

>
> If you need code to check whether or not a pointer is valid, _your code_
> is fundamentally flawed.
>

Suppose a library that receives a bad pointer. Isn't it reasonable to
check its validity before using it?

You should have a way to avoid crashes because of malicious code that
feeds you a bad pointer isn't it?

Besides, my motivation here is that in principle I thought that this
would be impossible, but later on and reflecting a bit, I found out
that it is not that difficult. I am not advocating the usage of this
function everywhere.

Consider too the development environment where it is much more likely
that in the first versions of your software bugs still creep in or
bugs exist that are undetected.

Rather than simply crashing you could print some useful information,
jump to a recovery point and go on.

Ian Collins

unread,

Sep 18, 2011, 3:53:08 PM9/18/11

to

On 09/19/11 05:25 AM, jacob navia wrote:
> Le 18/09/11 18:30, Kleuskes& Moos a écrit :

>>
>> If you need code to check whether or not a pointer is valid, _your code_
>> is fundamentally flawed.
>>
>
> Suppose a library that receives a bad pointer. Isn't it reasonable to
> check its validity before using it?

If the check is reliable, yes.

> You should have a way to avoid crashes because of malicious code that
> feeds you a bad pointer isn't it?

If the check is reliable, yes. There simply isn't a portable, reliable
way to do this.

> Besides, my motivation here is that in principle I thought that this
> would be impossible, but later on and reflecting a bit, I found out
> that it is not that difficult. I am not advocating the usage of this
> function everywhere.

It is impossible to have a portable solution. Say for example you have
some form of copy function:

void copy( const char* src, char* dst, size_t size );

How could you detect something like

int a,b,c;
char buf[4];
int d,e,f;

copy( "this is too big" , buf, 20 );

--
Ian Collins

Kleuskes & Moos

unread,

Sep 18, 2011, 4:00:22 PM9/18/11

to

On Sun, 18 Sep 2011 19:25:49 +0200, jacob navia wrote:

> Le 18/09/11 18:30, Kleuskes & Moos a écrit :
>>
>> If you need code to check whether or not a pointer is valid, _your
>> code_ is fundamentally flawed.
>>
>>
> Suppose a library that receives a bad pointer. Isn't it reasonable to
> check its validity before using it?

Nope. If you feed a badpointer to the library, a crash ensues, since the
programmer screwed up. Masking that by attempting to return a polite
error message masks a serious problem.

> You should have a way to avoid crashes because of malicious code that
> feeds you a bad pointer isn't it?

You should not be uncertain wether or not a pointer is valid or not. If
you are, your design stinks. If you try to compensate by adding checks,
your methodology stinks, too.

> Besides, my motivation here is that in principle I thought that this
> would be impossible, but later on and reflecting a bit, I found out that
> it is not that difficult. I am not advocating the usage of this function
> everywhere.

Good.

> Consider too the development environment where it is much more likely
> that in the first versions of your software bugs still creep in or bugs
> exist that are undetected.

Then the program crashes and it's time you fire up your debugger. If there
is a debugger available, that is.

> Rather than simply crashing you could print some useful information,
> jump to a recovery point and go on.

That's what debuggers are for. Trying to duplicate that functionality in code
is both superfluous *and* dangerous. It tends to mask serious problems in the
design.

-------------------------------------------------------------------------------
_____________________________________
/ Oh, I get it!! "The BEACH goes on", \
\ huh, SONNY?? /

Ian Collins

unread,

Sep 18, 2011, 4:23:50 PM9/18/11

to

On 09/19/11 08:00 AM, Kleuskes & Moos wrote:
> On Sun, 18 Sep 2011 19:25:49 +0200, jacob navia wrote:
>
>> You should have a way to avoid crashes because of malicious code that
>> feeds you a bad pointer isn't it?
>
> You should not be uncertain wether or not a pointer is valid or not. If
> you are, your design stinks. If you try to compensate by adding checks,
> your methodology stinks, too.

That's untrue. How can a library designer know what a user will pass to
one of their functions?

--
Ian Collins

Kleuskes & Moos

unread,

Sep 18, 2011, 4:32:12 PM9/18/11

to

That's the users responsibility, not one the author of the library should
concern himself with. If the user passes crap to the library, a crash (or
at least a SIGSEGV) is what's called for.

The alternative is requiring complete checking of any pointer passed to any
library, thus penalizing those programs that pass valid pointers for the
benefit of programmers who can't be bothered to ensure their pointers are
valid.

That's not the proper way to do it.

-------------------------------------------------------------------------------
___________________________
< Yow! Am I having fun yet? >
---------------------------
\
\

Nobody

unread,

Sep 18, 2011, 6:34:40 PM9/18/11

to

On Sun, 18 Sep 2011 19:25:49 +0200, jacob navia wrote:

>> If you need code to check whether or not a pointer is valid, _your code_
>> is fundamentally flawed.
>
> Suppose a library that receives a bad pointer. Isn't it reasonable to
> check its validity before using it?

Only if such a check exists. Usually it doesn't.

It isn't the pointers which point at unmapped address space that are the
problem; those produce a nice clean segfault. The real problem is with
pointers which point at something they're not supposed to be pointing at,
which end up silently trashing data. And there's no way to detect that.

If you want that level of hand-holding, you need to use a high-level
language which simply doesn't admit the possibility of an invalid pointer.

Alan Curry

unread,

Sep 18, 2011, 6:47:04 PM9/18/11

to

In article <j559ir$np9$1...@speranza.aioe.org>,

jacob navia <ja...@jspamsink.org> wrote:
>Le 18/09/11 18:30, Kleuskes & Moos a écrit :
>>
>> If you need code to check whether or not a pointer is valid, _your code_
>> is fundamentally flawed.
>>
>
>Suppose a library that receives a bad pointer. Isn't it reasonable to
>check its validity before using it?
>
>You should have a way to avoid crashes because of malicious code that
>feeds you a bad pointer isn't it?

Not if you're implementing a library interface. There is no security
boundary between a library function and its caller. Within a single
process, if any part of it is malicious, the whole thing is malicious.

If you want to be able to classify one piece of code as an attacker and
another as its potential victim, they need to be in separate processes.
With different user IDs.

--
Alan Curry

Richard Harter

unread,

Sep 18, 2011, 7:21:56 PM9/18/11

to

On Sun, 18 Sep 2011 16:30:05 +0000 (UTC), Kleuskes & Moos
<kle...@somewhere.else.net> wrote:

>On Sun, 18 Sep 2011 00:57:32 +0200, jacob navia wrote:
>
>> Le 18/09/11 00:38, Harald van Dĳk a écrit :
>>> On Sep 18, 12:29 am, jacob navia<ja...@spamsink.net> wrote:
>>>> In a recent discussion about preconditions, I argued that testing ALL
>>>> preconditions would be unpractical. As an example of a precondition
>>>> impossible to test I presented the problem to know if a pointer points
>>>> to a really readable place in memory.
>>>>
>>>> The windows API provides the function bool IsBadReadPtr(const void
>>>> *lp, size_t ucb);
>>>>
>>>> This takes a pointer and a size and returns true if it is OK, zero if
>>>> it fails.
>>>>
>>>> Problem is, that is only present under windows.
>>>
>>> That is not its biggest problem. Even on Windows, don't use it. Please
>>> read
>>> <http://blogs.msdn.com/b/oldnewthing/archive/2006/09/27/773741.aspx>
>>> for reasons why the concept of IsBadReadPtr (and IsBadWritePtr even
>>> more so) is fundamentally flawed.
>>
>> They aren't "flawed".
>
>If you need code to check whether or not a pointer is valid, _your code_
>is fundamentally flawed.

Any language in which it is possible for a pointer to be invalid but
which provides no way to test for validity is fundamentally flawed.

Richard Harter

unread,

Sep 18, 2011, 7:28:19 PM9/18/11

to

On Sun, 18 Sep 2011 20:32:12 +0000 (UTC), Kleuskes & Moos
<kle...@somewhere.else.net> wrote:

>On Mon, 19 Sep 2011 08:23:50 +1200, Ian Collins wrote:
>
>> On 09/19/11 08:00 AM, Kleuskes & Moos wrote:
>>> On Sun, 18 Sep 2011 19:25:49 +0200, jacob navia wrote:
>>>
>>>> You should have a way to avoid crashes because of malicious code that
>>>> feeds you a bad pointer isn't it?
>>>
>>> You should not be uncertain wether or not a pointer is valid or not. If
>>> you are, your design stinks. If you try to compensate by adding checks,
>>> your methodology stinks, too.
>>
>> That's untrue. How can a library designer know what a user will pass to
>> one of their functions?
>
>That's the users responsibility, not one the author of the library should
>concern himself with. If the user passes crap to the library, a crash (or
>at least a SIGSEGV) is what's called for.
>
>The alternative is requiring complete checking of any pointer passed to any
>library, thus penalizing those programs that pass valid pointers for the
>benefit of programmers who can't be bothered to ensure their pointers are
>valid.
>
>That's not the proper way to do it.

If you don't mind I would prefer not to use libraries you write.

Ben Pfaff

unread,

Sep 18, 2011, 7:38:57 PM9/18/11

to

Richard, you seem to believe that a library should validate all
the pointers passed into it. Can you explain how you believe
that a library should actually do this? I don't believe that it
is practical.

An example would be helpful. For example, how would you
implement memcpy()?
--
"In My Egotistical Opinion, most people's C programs should be indented six
feet downward and covered with dirt." -- Blair P. Houghton

Keith Thompson

unread,

Sep 18, 2011, 8:32:55 PM9/18/11

to

c...@tiac.net (Richard Harter) writes:
[...]

> Any language in which it is possible for a pointer to be invalid but
> which provides no way to test for validity is fundamentally flawed.

What existing language has pointers and is not "fundamentally flawed"?

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

Joe Wright

unread,

Sep 18, 2011, 11:10:54 PM9/18/11

to

On 9/18/2011 20:32, Keith Thompson wrote:
> c...@tiac.net (Richard Harter) writes:
> [...]
>> Any language in which it is possible for a pointer to be invalid but
>> which provides no way to test for validity is fundamentally flawed.
>
> What existing language has pointers and is not "fundamentally flawed"?
>

Indeed. Testing a pointer by simple inspection can tell you whether it is
null or not. If it is null it is not valid. If not you simply can't tell.

--
Joe Wright
"If you rob Peter to pay Paul you can depend on the support of Paul."

Kenny McCormack

unread,

Sep 18, 2011, 11:34:25 PM9/18/11

to

In article <H7idnUSmaqgcL-vT...@giganews.com>,

Joe Wright <joeww...@comcast.net> wrote:
>On 9/18/2011 20:32, Keith Thompson wrote:
>> c...@tiac.net (Richard Harter) writes:
>> [...]
>>> Any language in which it is possible for a pointer to be invalid but
>>> which provides no way to test for validity is fundamentally flawed.
>>
>> What existing language has pointers and is not "fundamentally flawed"?
>>
>Indeed. Testing a pointer by simple inspection can tell you whether it is
>null or not. If it is null it is not valid. If not you simply can't tell.

We just keep going around and around on this.

It is perfectly obvious that it is possible to design and implement a
language in which pointers (or something resembling what we call pointers in
C) are safe. But that language won't be C.

--
Faced with the choice between changing one's mind and proving that there is
no need to do so, almost everyone gets busy on the proof.

- John Kenneth Galbraith -

Richard Harter

unread,

Sep 19, 2011, 12:27:43 AM9/19/11

to

On Sun, 18 Sep 2011 16:38:57 -0700, b...@cs.stanford.edu (Ben Pfaff)
wrote:

Mon ami, you miscomprehend. I did not claim nor do I suppose that I
can determine whether every pointer is valid. That is not the issue.
It is the viewpoint. Let me quote:

That's the users responsibility, not one the author of the library
should concern himself with. If the user passes crap to the
library, a crash (or at least a SIGSEGV) is what's called for.

The issue is: As a library author do you validate your input as best
you can and provide a graceful response to input error - again as best
as you can? Or do you, perhaps, take the view that if "the user
passes crap" it is their lookout.

If the library author takes the view that the onus is on the crap
passing user and therefore doesn't have to care whether about
validating input or about designing the library software to be robust.
Given my druthers, I'd rather not use software written by such an
author.

To be fair I believe that K&M is an able programmer who checks inputs
carefully and who writes robust software. Much of what K&M posts is
of high quality. However I was dismayed at the attitude expressed in
the post to which I responded.

Joe Pfeiffer

unread,

Sep 19, 2011, 12:32:57 AM9/19/11

to

gaz...@shell.xmission.com (Kenny McCormack) writes:

> In article <H7idnUSmaqgcL-vT...@giganews.com>,
> Joe Wright <joeww...@comcast.net> wrote:
>>On 9/18/2011 20:32, Keith Thompson wrote:
>>> c...@tiac.net (Richard Harter) writes:
>>> [...]
>>>> Any language in which it is possible for a pointer to be invalid but
>>>> which provides no way to test for validity is fundamentally flawed.
>>>
>>> What existing language has pointers and is not "fundamentally flawed"?
>>>
>>Indeed. Testing a pointer by simple inspection can tell you whether it is
>>null or not. If it is null it is not valid. If not you simply can't tell.
>
> We just keep going around and around on this.
>
> It is perfectly obvious that it is possible to design and implement a
> language in which pointers (or something resembling what we call pointers in
> C) are safe. But that language won't be C.

And in which those "pointers" have semantics somewhat different from
what C calls pointers. And, I expect, semantics somewhat different from
what most readers of this newsgroup infer from the word pointers.

Ben Pfaff

unread,

Sep 19, 2011, 12:48:36 AM9/19/11

to

c...@tiac.net (Richard Harter) writes:

> Mon ami, you miscomprehend. I did not claim nor do I suppose that I
> can determine whether every pointer is valid. That is not the issue.
> It is the viewpoint. Let me quote:
>
> That's the users responsibility, not one the author of the library
> should concern himself with. If the user passes crap to the
> library, a crash (or at least a SIGSEGV) is what's called for.
>
> The issue is: As a library author do you validate your input as best
> you can and provide a graceful response to input error - again as best
> as you can? Or do you, perhaps, take the view that if "the user
> passes crap" it is their lookout.
>
> If the library author takes the view that the onus is on the crap
> passing user and therefore doesn't have to care whether about
> validating input or about designing the library software to be robust.
> Given my druthers, I'd rather not use software written by such an
> author.

Personally, I believe that the author of a library should
document his assumptions on the input to a library, including the
degree to which the library tolerates deviations. Usually, I
would rather carefully use a *fast* library that does not verify
its input, instead of a slower library that does. The former I
can transform into the latter with wrapper functions, if
necessary; the latter I'm stuck with.
--
"The expression isn't unclear *at all* and only an expert could actually
have doubts about it"
--Dan Pop

Richard Harter

unread,

Sep 19, 2011, 1:14:53 AM9/19/11

to

On Sun, 18 Sep 2011 17:32:55 -0700, Keith Thompson <ks...@mib.org>
wrote:

>c...@tiac.net (Richard Harter) writes:
>[...]
>> Any language in which it is possible for a pointer to be invalid but
>> which provides no way to test for validity is fundamentally flawed.
>
>What existing language has pointers and is not "fundamentally flawed"?

Please do not snip the relevant text, i.e.

>>>If you need code to check whether or not a pointer is valid, _your code_
>>>is fundamentally flawed.
>>

>> Any language in which it is possible for a pointer to be invalid but
>> which provides no way to test for validity is fundamentally flawed.

I was making a (mildly sarcastic) point about the statement I was
commenting on.

That said, I dunno off hand about languages that have pointers and
aren't fundamentally flawed. It depends on what counts as pointers.
If "pointer" means "address" you're out of luck. Once you get into
the world of smart pointers and references it's a different matter.

The interesting thing is that it isn't hard for a language to have
references that can be tested for validity.

Richard Harter

unread,

Sep 19, 2011, 1:38:36 AM9/19/11

to

On Sun, 18 Sep 2011 21:48:36 -0700, b...@cs.stanford.edu (Ben Pfaff)
wrote:

>c...@tiac.net (Richard Harter) writes:

Fair enough, if those are your only choices. Quite often, however,
the cost of verification is insignificant. Choosing the *fast*
library could be a premature optimization. Now if I actually had two
libraries that did the same thing *and* I was really really concerned
with speed *and* I did performance measurements on both *and* there a
measurable difference, then I might go with the one that doesn't
verify its inputs. How often does that happen?

Ian Collins

unread,

Sep 19, 2011, 2:02:15 AM9/19/11

to

That's normally a case of prevention by controlling what can be be used
to initialise a reference rather than by detection.

--
Ian Collins

Richard Harter

unread,

Sep 19, 2011, 2:18:52 AM9/19/11

to

On Mon, 19 Sep 2011 18:02:15 +1200, Ian Collins <ian-...@hotmail.com>
wrote:

That may be - I think that's what fortran does. But that's only half
of it; you need to be able to detect whether a reference is stale.
That's a trifle more complicated.

Kleuskes & Moos

unread,

Sep 19, 2011, 2:39:20 AM9/19/11

to

On Sun, 18 Sep 2011 23:10:54 -0400, Joe Wright wrote:

> On 9/18/2011 20:32, Keith Thompson wrote:
>> c...@tiac.net (Richard Harter) writes: [...]
>>> Any language in which it is possible for a pointer to be invalid but
>>> which provides no way to test for validity is fundamentally flawed.
>>
>> What existing language has pointers and is not "fundamentally flawed"?
>>
> Indeed. Testing a pointer by simple inspection can tell you whether it
> is null or not. If it is null it is not valid. If not you simply can't
> tell.

I actually consider NULL a valid value for a pointer, and of course
checking for NULL pointers, if called for, is not what i had in mind.

--
-------------------------------------------------------------------------------
_______________________________________
/ If I had a Q-TIP, I could prevent th' \
\ collapse of NEGOTIATIONS!! /

Malcolm McLean

unread,

Sep 19, 2011, 2:52:32 AM9/19/11

to

On Sep 19, 2:21 am, c...@tiac.net (Richard Harter) wrote:
>
> Any language in which it is possible for a pointer to be invalid but
> which provides no way to test for validity is fundamentally flawed.
>

That's the problem with pointers, of course.

What you really mean is "too low level" rather than "fundamentally
flawed". At some point you've got to write to raw addresses. C allows
you to pass these around, and makes it difficult to tag them with
range and type information, either at the level of the compiler or in
client code. That's the feature that makes C what it is. If you move
to C++ you can use controlled sequences, and essentially eliminate
pointers. But experience shows that this brings its own problems.
--
Fuzzy logic trees, now in Apple ebooks
http://www.malcolmmclean.site11.com/www

jacob navia

unread,

Sep 19, 2011, 3:14:24 AM9/19/11

to

Le 19/09/11 07:38, Richard Harter a écrit :

>
> Fair enough, if those are your only choices. Quite often, however,
> the cost of verification is insignificant.

Exactly

Choosing the *fast*
> library could be a premature optimization. Now if I actually had two
> libraries that did the same thing *and* I was really really concerned
> with speed *and* I did performance measurements on both *and* there a
> measurable difference, then I might go with the one that doesn't
> verify its inputs. How often does that happen?
>

Suppose you mark your data with a 64 bit magic number at the start
of the data structure.

To know if you have a valid pointer you test its validity for
reading 64 bits (8 bytes). If the pointer is valid it takes you
several assignments, 2 calls to signal and 1 call to setjmp.

You can fuse the testing for reading with the reading of the
magic number, so, in fact I would say that at most you are spending
several hundred cycles in testing pointer validty, what in
modern machines is nothing.

If your magic numbers are cleverly built, you can also prevent
the much more common error of passing a valid pointer to a
DIFFERENT kind of object. For instance when writing the serializing
part of the container library I write a GUUID (128 bytes) of
magic number, what allows me to avoid trying to restore a list
container from a file that was actually a saved file from the
vector container!

It can be argued that people that do not make mistakes are
penalized by this, but I think nobody will even notice
that I am reading 128 bytes more and making a memcmp.

Kleuskes & Moos

unread,

Sep 19, 2011, 3:14:38 AM9/19/11

to

If your standards are such that you would insist on having impractical, nay,
impossible checks, your choice of software to use may be rather limited. If
I'm correctly informed not even (g)libc does that kind of checking.

Validating input is something different that what we're discussing
here, which specifically talked about bogus pointers. If there's input from
some unreliable source, it should be checked thoroughly, but if the user of
a library is considered unreliable, your well into the paranoid mode of
defensive programming.

> To be fair I believe that K&M is an able programmer who checks inputs
> carefully and who writes robust software. Much of what K&M posts is of
> high quality. However I was dismayed at the attitude expressed in the
> post to which I responded.

Why, thank you. May i add that my usual neck of the woods is real-time
embedded software, which may help you understand the origins of my attitude.
I rather _do_ expect a fellow programmer to know what he's doing.

If you're writing software (and that's the only way i can think of to pass
bogus pointers to a library), you should be able to tell whether or not a
pointer is valid or not at a given point. If you're not, *something* is
seriously wrong.

Note: I'm using "you are" in the sense of "one is", I'm not implying anything
about you personally.

-------------------------------------------------------------------------------
_________________________________________
/ Was my SOY LOAF left out in th'RAIN? It \
\ tastes REAL GOOD!! /
-----------------------------------------
\
\

BartC

unread,

Sep 19, 2011, 5:10:57 AM9/19/11

to

"jacob navia" <ja...@spamsink.net> wrote in message
news:j56q4f$ljc$1...@speranza.aioe.org...

> Le 19/09/11 07:38, Richard Harter a écrit :

> Choosing the *fast*
>> library could be a premature optimization. Now if I actually had two
>> libraries that did the same thing *and* I was really really concerned
>> with speed *and* I did performance measurements on both *and* there a
>> measurable difference, then I might go with the one that doesn't
>> verify its inputs. How often does that happen?
>>
>
> Suppose you mark your data with a 64 bit magic number at the start
> of the data structure.

This is impractical for much of the low-level programming that is done in C.

And when the library code is written by someone else, this requires the
caller to maintain the magic number, unless you also insist that
*everything* to do with the data structure is handled by the library.

There is also the risk of stray magic numbers lurking in memory, unless care
is taken in deleting them when no longer needed.

It sounds like something that would be only be suitable for a high-level,
self-contained package such as your Containers thing.

What *would* be nice is libraries checking for NULL pointers, which would
cause a crash otherwise, as currently happens with many standard functions
(and requiring me to write wrappers around them to make them better
behaved).

--
Bartc

tom st denis

unread,

Sep 19, 2011, 7:50:19 AM9/19/11

to

On Sep 18, 1:25 pm, jacob navia <ja...@spamsink.net> wrote:
> Le 18/09/11 18:30, Kleuskes & Moos a écrit :
>
>
>
> > If you need code to check whether or not a pointer is valid, _your code_
> > is fundamentally flawed.
>
> Suppose a library that receives a bad pointer. Isn't it reasonable to
> check its validity before using it?

Not really. Besides checking for NULL [*] pointers it's up to the
caller to pass valid pointers since they're not really easily testable
[hence this thread] in a portable fashion.

> You should have a way to avoid crashes because of malicious code that
> feeds you a bad pointer isn't it?
>
> Besides, my motivation here is that in principle I thought that this
> would be impossible, but later on and reflecting a bit, I found out
> that it is not that difficult. I am not advocating the usage of this
> function everywhere.

Raising a signal is not the best of ideas because you could be in a
debugger in which signals are masked to the user process and throw
debugging problems.

[*] I check for NULL pointers since I tend to use memset/calloc and it
lets me find any uninitialized pointers easily.

Tom

Malcolm McLean

unread,

Sep 19, 2011, 8:07:34 AM9/19/11

to

On Sep 19, 10:14 am, jacob navia <ja...@spamsink.net> wrote:
>
> If your magic numbers are cleverly built, you can also prevent
> the much more common error of passing a valid pointer to a
> DIFFERENT kind of object. For instance when writing the serializing
> part of the container library I write a GUUID (128 bytes) of
> magic number, what allows me to avoid trying to restore a list
> container from a file that was actually a saved file from the
> vector container!
>

That's OK now, when terabyte disks are cheap but most datasets are in
the megabyte range.

However it might not always be OK. Terabyte disks are unlikely to
suddenly become expensive again, but datasets could balloon, so your
library is saving many hundreds of millions of lists, or cheaper but
lower capacity storage could come into vogue.

Ben Bacarisse

unread,

Sep 19, 2011, 9:24:08 AM9/19/11

to

If it was a really clever scheme you'd have run-time type checking. If
the language were really clever, you could do this checking at compile
time. It's one reason people choose to program in other languages. I
am not entirely sure why all the people who program in C do so, but some
do so because they want minimal run-time overhead so the case for doing
what amounts to some limited run-time type-checking in C is not clear to
me. There are languages that do it better, both at compile time and at
run time. I think you might be aiming at a niche market.

<snip>
--
Ben.

jacob navia

unread,

Sep 19, 2011, 10:10:52 AM9/19/11

to

Le 19/09/11 15:24, Ben Bacarisse a écrit :

Well, that is run time type checking obviously. But ONLY when you want
and where you want, not everywhere.

> If
> the language were really clever, you could do this checking at compile
> time.

In most cases this is done in C. In most cases this checking is
unnecessary. But for life critical applications it could be worth
to spend some cycles avoiding crashing you see?

> It's one reason people choose to program in other languages.

Look, there are NO languages that will eliminate bugs. There is NO
silver bullet here.

I
> am not entirely sure why all the people who program in C do so, but some
> do so because they want minimal run-time overhead

Exactly, and the run time overhead of a few pointer verifications at
a critical interface do not change that C is very fast anyway.

> so the case for doing
> what amounts to some limited run-time type-checking in C is not clear to
> me.

There could be a few applications, specially when debugging.

> There are languages that do it better, both at compile time and at
> run time.

No. I think C is quite good.

> I think you might be aiming at a niche market.
>
> <snip>

I am not "aiming" at any market. I am just discussing a function for
checking pointers if needed. I am not selling anything either. We are
so swamped by marketing that in a simple conversation about pointer
checking we need to do a market research first!

Please Ben, let's keep the discussion sane. I proposed a function for
pointer checking in its own rights, because I was surprised that it
could be so simple. Now, is it the panacea for avoiding bugs?

Surely not.

Joel C. Salomon

unread,

Sep 19, 2011, 10:29:07 AM9/19/11

to

On 09/17/2011 06:29 PM, jacob navia wrote:
> In a recent discussion about preconditions, I argued that testing ALL
> preconditions would be unpractical. As an example of a precondition
> impossible to test I presented the problem to know if a pointer

unread,

Sep 19, 2011, 11:50:09 AM9/19/11

to

"jacob navia" <ja...@spamsink.net> wrote in message
news:j56q4f$ljc$1...@speranza.aioe.org...

> Suppose you mark your data with a 64 bit magic number at the start
> of the data structure.
>
> To know if you have a valid pointer you test its validity for
> reading 64 bits (8 bytes). If the pointer is valid it takes you
> several assignments, 2 calls to signal and 1 call to setjmp.
>
> You can fuse the testing for reading with the reading of the
> magic number, so, in fact I would say that at most you are spending
> several hundred cycles in testing pointer validty, what in
> modern machines is nothing.

At that level of application, perhaps you can dispense with raw pointers,
and switch to handles.

A handle could be as simple as an index into a linear table, but the table
now contains the actual pointer, and any verification necessary, rather than
sticking magic numbers into user data. And the table itself is in trusted
memory. This sounds faster than calling some dodgy OS function which may or
may not work.

But that still won't stop the caller passing a handle B instead of A, both
equally valid, causing incorrect results if not a crash.

--
Bartc

August Karlstrom

unread,

Sep 19, 2011, 11:50:45 AM9/19/11

to

On 2011-09-19 02:32, Keith Thompson wrote:
> c...@tiac.net (Richard Harter) writes:
> [...]
>> Any language in which it is possible for a pointer to be invalid but
>> which provides no way to test for validity is fundamentally flawed.
>
> What existing language has pointers and is not "fundamentally flawed"?

Oberon comes to mind.

August

--
The competent programmer is fully aware of the limited size of his own
skull. He therefore approaches his task with full humility, and avoids
clever tricks like the plague. --Edsger Dijkstra

Richard Harter

unread,

Sep 19, 2011, 12:13:55 PM9/19/11

to

On Mon, 19 Sep 2011 07:14:38 +0000 (UTC), Kleuskes & Moos

Nice strawman. I make no such insistance, just as I don't insist that
all software I use be written by somebody named Fred.

>If I'm correctly informed not even (g)libc does that kind of checking.

So? I would be highly surprised if they did.

>Validating input is something different that what we're discussing
>here, which specifically talked about bogus pointers. If there's input from
>some unreliable source, it should be checked thoroughly, but if the user of
>a library is considered unreliable, your well into the paranoid mode of
>defensive programming.

In general, users of libraries are unreliable. Maybe your users write
error free code and never have dangling pointers. If so, I am happy
for you.

The issue is, what can you do about being passed bad pointers, if
anything. The trouble I have with your post is that you present a
false dichotomy - either you accept no responsibility for handling bad
input pointers (crap passing users are fair game) or else you go into
paranoid mode. ("You" being a library author, not you personally.)

What you do or even can do depends upon the specific situation. We
all (I hope) understand that C pointers are inherently unsafe and
there is no general way to check for pointer validity. That said,
sometimes there are things that can be done. One can at least ask the
question, "is there anything meaningful I can do".

And sometimes "paranoid defensive programming" is right and proper.
As it happens I am working with some software in which the possibility
of stale *pointers* is part of the spec. The answer is not to use raw
C pointers but rather to use "safe references" that can be and are
checked for validity.

>
>> To be fair I believe that K&M is an able programmer who checks inputs
>> carefully and who writes robust software. Much of what K&M posts is of
>> high quality. However I was dismayed at the attitude expressed in the
>> post to which I responded.
>
>Why, thank you. May i add that my usual neck of the woods is real-time
>embedded software, which may help you understand the origins of my attitude.
>I rather _do_ expect a fellow programmer to know what he's doing.

Having done a bit of real-time programming myself, I quite understand.

>
>If you're writing software (and that's the only way i can think of to pass
>bogus pointers to a library), you should be able to tell whether or not a
>pointer is valid or not at a given point. If you're not, *something* is
>seriously wrong.

Eventually, of course, you *should* be able to tell. I notice,
though, that I know of no large software product that is know to be
error free. Finding the dangling pointer after the rocket has crashed
is not the happiest of outcomes.

>
>Note: I'm using "you are" in the sense of "one is", I'm not implying anything
>about you personally.

Likewise.

Richard Harter

unread,

Sep 19, 2011, 12:39:11 PM9/19/11

to

On Mon, 19 Sep 2011 08:11:10 -0700, Ben Pfaff <b...@cs.stanford.edu>

Of course. Was anyone proposing some iron clad rule that everyone
must follow? Is common sense not on the table?

[snip]

Ben Pfaff

unread,

Sep 19, 2011, 12:48:52 PM9/19/11

to

c...@tiac.net (Richard Harter) writes:

I guess that common sense is on the table, based on your
response. Your original statement was very broad and, to my
mind, overreaching. Now that you've qualified it, it makes more
sense.
--
"Some people *are* arrogant, and others read the FAQ."
--Chris Dollin

Keith Thompson

unread,

Sep 19, 2011, 1:06:03 PM9/19/11

to

Kleuskes & Moos <kle...@somewhere.else.net> writes:
> On Sun, 18 Sep 2011 23:10:54 -0400, Joe Wright wrote:
>> On 9/18/2011 20:32, Keith Thompson wrote:
>>> c...@tiac.net (Richard Harter) writes: [...]
>>>> Any language in which it is possible for a pointer to be invalid but
>>>> which provides no way to test for validity is fundamentally flawed.
>>>
>>> What existing language has pointers and is not "fundamentally flawed"?
>>>
>> Indeed. Testing a pointer by simple inspection can tell you whether it
>> is null or not. If it is null it is not valid. If not you simply can't
>> tell.
>
> I actually consider NULL a valid value for a pointer, and of course
> checking for NULL pointers, if called for, is not what i had in mind.

The phrase "valid pointer" is context-dependent. NULL is not valid
as the operand of the unary "*" operator; it's perfectly valid as
the argument of free().

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

Keith Thompson

unread,

Sep 19, 2011, 1:14:23 PM9/19/11

to

"BartC" <b...@freeuk.com> writes:
> "Ben Pfaff" <b...@cs.stanford.edu> wrote in message
> news:87zki0w...@blp.benpfaff.org...
>
>> It's pretty easy to come up with examples of functions that are
>> trivial if you don't check the arguments and nontrivial if you
>> do. An excerpt from a library of mine is below. Each of these
>> functions is only one or two machine instructions when inlined.
>
> The body of these functions is so trivial that one wonders why they aren't
> written in-line anyway, or at least wrapped in a macro.

Writing them in-line would be more error-prone. Making them functions
lets you give each operation a name.

How would a macro be better than an inline function?

[...]

Kleuskes & Moos

unread,

Sep 19, 2011, 1:36:57 PM9/19/11

to

No they aren't and neither am i. I think, however, i can assume both know
how to use a debugger. Besides, i don't usually write libraries.

> The issue is, what can you do about being passed bad pointers, if
> anything. The trouble I have with your post is that you present a false
> dichotomy - either you accept no responsibility for handling bad input
> pointers (crap passing users are fair game) or else you go into paranoid
> mode. ("You" being a library author, not you personally.)

I don't see much in the middle, except replacing pointers with something
more robust, like your solution below. The only trick i sometimes employ
is crude range-checking, but that depends on know where in memory a
pointer should (or rather, should not) point and/or checking alignment if
required, but these do not end up in production code.

> What you do or even can do depends upon the specific situation. We all
> (I hope) understand that C pointers are inherently unsafe and there is
> no general way to check for pointer validity. That said, sometimes
> there are things that can be done. One can at least ask the question,
> "is there anything meaningful I can do".

Not much, I'm afraid.

> And sometimes "paranoid defensive programming" is right and proper.

Aye. But that's usually restricted to having proper asserts in place
and various other sanity checks. Don't get me wrong, I'm all for that,
if it's possible to use them.

> As
> it happens I am working with some software in which the possibility of
> stale *pointers* is part of the spec. The answer is not to use raw C
> pointers but rather to use "safe references" that can be and are checked
> for validity.

Thus avoiding the problem. Clever. But it doesn't really address being
passed bad pointers.

>>> To be fair I believe that K&M is an able programmer who checks inputs
>>> carefully and who writes robust software. Much of what K&M posts is
>>> of high quality. However I was dismayed at the attitude expressed in
>>> the post to which I responded.
>>
>>Why, thank you. May i add that my usual neck of the woods is real-time
>>embedded software, which may help you understand the origins of my
>>attitude. I rather _do_ expect a fellow programmer to know what he's
>>doing.
>
> Having done a bit of real-time programming myself, I quite understand.

:-)

>>If you're writing software (and that's the only way i can think of to
>>pass bogus pointers to a library), you should be able to tell whether or
>>not a pointer is valid or not at a given point. If you're not,
>>*something* is seriously wrong.
>
> Eventually, of course, you *should* be able to tell. I notice, though,
> that I know of no large software product that is know to be error free.

There's a bet on TeX, IIRC. Problem is, by the time the software has matured,
the product is obsolete.

> Finding the dangling pointer after the rocket has crashed is not the
> happiest of outcomes.

Nope. And unless you're talking manned missions, there are worse possible
outcomes, but the solutions suggested in this thread do not strike me as
being "the way to go".

>>Note: I'm using "you are" in the sense of "one is", I'm not implying
>>anything about you personally.
>
> Likewise.

-------------------------------------------------------------------------------
_________________________________
< YOU PICKED KARL MALDEN'S NOSE!! >

Kenneth Brody

unread,

Sep 19, 2011, 1:42:00 PM9/19/11

to

On 9/18/2011 1:25 PM, jacob navia wrote:
> Le 18/09/11 18:30, Kleuskes & Moos a écrit :
>>
>> If you need code to check whether or not a pointer is valid, _your code_
>> is fundamentally flawed.
>>
>
> Suppose a library that receives a bad pointer. Isn't it reasonable to
> check its validity before using it?

Well, it depends on your definition of "valid". The IsBadReadPtr() and
IsBadWritePtr() just check that, at the moment the function is called, there
exists readable/writable memory at the address specified. It doesn't (and
can't) determine whether that memory is the correct place to read/write.

> You should have a way to avoid crashes because of malicious code that
> feeds you a bad pointer isn't it?

Doesn't crashing a program avoid executing malicious code?

> Besides, my motivation here is that in principle I thought that this
> would be impossible, but later on and reflecting a bit, I found out
> that it is not that difficult. I am not advocating the usage of this
> function everywhere.

Okay.

> Consider too the development environment where it is much more likely
> that in the first versions of your software bugs still creep in or
> bugs exist that are undetected.
>
> Rather than simply crashing you could print some useful information,
> jump to a recovery point and go on.

Personally, I prefer debugging a program that crashes at the actual point of
failure.

--
Kenneth Brody

Kenneth Brody

unread,

Sep 19, 2011, 1:51:57 PM9/19/11

to

On 9/18/2011 7:28 PM, Richard Harter wrote:
> On Sun, 18 Sep 2011 20:32:12 +0000 (UTC), Kleuskes& Moos

> <kle...@somewhere.else.net> wrote:
>
>> On Mon, 19 Sep 2011 08:23:50 +1200, Ian Collins wrote:
>>

>>> On 09/19/11 08:00 AM, Kleuskes& Moos wrote:
>>>> On Sun, 18 Sep 2011 19:25:49 +0200, jacob navia wrote:
>>>>

>>>>> You should have a way to avoid crashes because of malicious code that
>>>>> feeds you a bad pointer isn't it?
>>>>

>>>> You should not be uncertain wether or not a pointer is valid or not. If
>>>> you are, your design stinks. If you try to compensate by adding checks,
>>>> your methodology stinks, too.
>>>
>>> That's untrue. How can a library designer know what a user will pass to
>>> one of their functions?
>>
>> That's the users responsibility, not one the author of the library should
>> concern himself with. If the user passes crap to the library, a crash (or
>> at least a SIGSEGV) is what's called for.
>>
>> The alternative is requiring complete checking of any pointer passed to any
>> library, thus penalizing those programs that pass valid pointers for the
>> benefit of programmers who can't be bothered to ensure their pointers are
>> valid.
>>
>> That's not the proper way to do it.
>
> If you don't mind I would prefer not to use libraries you write.

Some might respond that they would prefer that you not use their libraries,
either. (After all, if you can't be bothered to check that you aren't using
garbage pointers, go somewhere else.)

I'm not saying on which side I sit.

--
Kenneth Brody

jacob navia

unread,

Sep 19, 2011, 2:30:42 PM9/19/11

to

Le 19/09/11 19:42, Kenneth Brody a écrit :

>
> Personally, I prefer debugging a program that crashes at the actual
> point of failure.
>

Well, it depends where the point of failure is.

Suppose your library is expecting a pointer to "a" and is passed a
pointer to "b", a similar syructure but with slightly different layout.

1) The program crashes deep inside the library when processing the
pointer data. You have a stack trace that tells you:

0x488776AAD()
0x446655FF4()
MyFunction() myfunction.c 476
main() main.c 876

Now what?

2) When you pass a bad pointer the library opens a dialog box:
"Invalid argument to function Library_input(), argument two"

You see?

A crash isn't always a good thing.

jacob navia

unread,

Sep 19, 2011, 2:36:22 PM9/19/11

to

Le 19/09/11 09:14, Kleuskes & Moos a écrit :

>
> If you're writing software (and that's the only way i can think of to pass
> bogus pointers to a library), you should be able to tell whether or not a
> pointer is valid or not at a given point. If you're not, *something* is
> seriously wrong.
>

But you can write in your code

LibraryFunction(handle1,pdata);

instead of the correct

LibraryFunction(pdata,handle1);

you see, because you are (like many other beings)
a HUMAN, not a machine.

And now you do not get any warning because both are
pointers to opaque types, i.e. eventually void *.

Or, you have two similarly named variables and instead of
writing h1 you wrote h2...

etc, the possibilities (as you very well know) are endless.

Shit happens...

In a debug setting, a clear error message from the library would be
very welcome. You are spared starting the debugger, driving the program
to the crash point and debugging!!!

August Karlstrom

unread,

Sep 19, 2011, 2:45:54 PM9/19/11

to

On 2011-09-19 17:50, August Karlstrom wrote:
> On 2011-09-19 02:32, Keith Thompson wrote:
>> c...@tiac.net (Richard Harter) writes:
>> [...]
>>> Any language in which it is possible for a pointer to be invalid but
>>> which provides no way to test for validity is fundamentally flawed.
>>
>> What existing language has pointers and is not "fundamentally flawed"?
>
> Oberon comes to mind.

Never mind. I misread Richard's statement as something along the lines
of "any language with non-safe pointers is fundamentally flawed." The
best solution is of course if the language can prevent invalid pointers
in the first place. In Oberon the only invalid pointers are
uninitialized local pointer variables but a compiler will typically
issue a warning if such a variable is used before being initialized.

BartC

unread,

Sep 19, 2011, 2:56:36 PM9/19/11

to

"Keith Thompson" <ks...@mib.org> wrote in message
news:lnmxe05...@nuthaus.mib.org...

> "BartC" <b...@freeuk.com> writes:
>> "Ben Pfaff" <b...@cs.stanford.edu> wrote in message
>> news:87zki0w...@blp.benpfaff.org...
>>
>>> It's pretty easy to come up with examples of functions that are
>>> trivial if you don't check the arguments and nontrivial if you
>>> do. An excerpt from a library of mine is below. Each of these
>>> functions is only one or two machine instructions when inlined.
>>
>> The body of these functions is so trivial that one wonders why they
>> aren't
>> written in-line anyway, or at least wrapped in a macro.
>
> Writing them in-line would be more error-prone. Making them functions
> lets you give each operation a name.
>
> How would a macro be better than an inline function?

Looking at the original functions, they are just applying some
abstraction/hiding, so are doing exactly what macros are for.

Written as functions, they are not guaranteed to be inline. And some
overzealous maintainer might come along and decide the pointer arguments
need to be validated...

--
Bartc

ImpalerCore

unread,

Sep 19, 2011, 3:49:15 PM9/19/11

to

The only pointer check I consider useful at the library level is
whether NULL pointers are semantically valid as arguments to function
parameters. In your functions defined above, passing a NULL pointer
in as 'node' will introduce a memory access violation. One could
consider inserting a check at the module (library) level to verify
that the pointer argument is semantically valid.

\code

/* Returns the position of the first 1-bit in NODE. */
static inline unsigned long int
range_set_node_get_start (const struct range_set_node *node)
{

check_pointer_semantics (node != NULL);
return node->start;
}
\endcode

Unfortunately, when one wants to insert checking at the library level,
they enforce behavior on the client that is not satisfactory in all
scenarios. In development and debugging, 'check_pointer_semantics'
would prefer an assert like function. In production, crashing the
application due to passing a NULL pointer argument in
'range_set_node_get_start' may not be the preferred option. It might
be nice to save the current program's progress if it's a long
algorithm, or perhaps inform the user in a nicer fashion.

The best option I can come up with is to try to abstract the outcome
of a precondition violation. The check of a precondition can be split
into three phases: detection, reporting, and response. The detection
is evaluation of the precondition expression 'node != NULL'. The
reporting step may be a display of the file and line of the error like
'Precondition violation: node != NULL, file lib_source.c, line 123'.
The response can be to abort like 'assert', or to early-return with a
error value, or invoke a global procedure to save and shutdown, or pop
a dialog with an option to report the error. The standard library
'assert' can be represented as a combination of the three phases,
where the report is to print file and line information, and the
response is to 'abort'. The advantage is that when the constraint is
enforced at the library level, every call to
'range_set_node_get_start' is validated irregardless of the knowledge
and competency of the developers calling it.

The key to acceptance is providing an architecture that allows the
client (the application developer using the module) to sculpt the
report and response to these precondition violations in a manner that
is convenient. To help satisfy these requirements, I use macros that
reference a global function table that contains function pointers to
'report' and 'response' functions.

\code snippet
static struct c_constraint_violation_ftable
gc_private_constraint_violation_ftable =
{
gc_default_report_handler,
NULL
};

void c_constraint_violation_set_report_handler( void (*report_fn)
( const char*, const char*, int ) );
void c_constraint_violation_set_response_handler( void (*response_fn)
( void ) );
\endcode

The report handler 'gc_default_report_handler' would look like the
following.

\code snippet
void gc_default_report_handler( const char* expr, const char* file,
int line )
{
fprintf( stderr, "%s, file %s, line %d\n", expr, file, line );
fflush( stderr );
}
\endcode

The response and report handlers are called from a generic constraint
violation handler.

\code
void gc_constraint_violation( const char* expr, const char* file, int
line )
{
if ( gc_private_constraint_violation_ftable.report ) {
(*gc_private_constraint_violation_ftable.report)( expr, file,
line );
}
if ( gc_private_constraint_violation_ftable.response ) {
(*gc_private_constraint_violation_ftable.response)();
}
}
\endcode

Finally, I have a macros to evaluate the constraint and add any
additional response that cannot be encapsulated in the 'response'
function pointer.

\code snippet
#define c_return_value_if_fail( expr, val ) \
do \
{ \
if ( expr ) {} \
else \
{ \
g_constraint_violation( "Constraint violation: " #expr, \
__FILE__, __LINE__ ); \
return (val); \
} \
} while (0)
\endcode

There is also a 'c_return_if_fail' that replace the 'return (val);'
with 'return;' statement, and a c_assert_if_fail that doesn't have any
return so it acts as a pass through for cases when an 'early-exit'
response isn't required (an example would be putting a constraint that
informs the user when strlcat truncates its result, as a truncation is
likely a serious error, but not fatal to the program since truncation
is defined as valid semantics for the function).

Just as 'assert' is disabled by NDEBUG, one can define a preprocessor
symbol to disable evaluation of these macros at library level, or at
source level with a granularity that depends on how many source files
one uses to implement a module. If 'range_set_node_get_start',
'range_set_node_get_end', and 'range_set_node_get_width' are all
defined in different source .c files, one can use the build system to
control which functions get compiled with or without constraint
checking (in my case, by defining C_NO_CONSTRAINTS). It's even
possible to do it at run-time if the 'report' or 'response' function
pointer check a global status value.

The result is that more of the library's assumptions can be enforced
within the framework of a single architecture with more flexibility
offered that by using 'assert' or manual defensive programming
techniques. If one wants to use this system in a design-by-contract
style, simply wire the 'response' function pointer to 'abort', which
ensures that a 'c_return_value_if_fail' will stop in its tracks. Or
one can place a breakpoint in an empty response function pointer to
stop at each constraint violation while still maintaining the
defensive style of 'c_return_value_if_fail' for production
environments.

For example, here is my version of strlcat that I use, that validates
that the source and destination strings reference memory, and warns if
the truncation occurs.

size_t c_strlcat( char* dst, const char* src, size_t dst_size )
{
size_t dst_length;
char* d = dst;
const char* s = src;
size_t n = dst_size;
size_t dlen;

c_return_value_if_fail( dst != NULL, 0 )
c_return_value_if_fail( src != NULL, 0 );

/* Find the end of dst and adjust bytes left but don't go past end.
*/
while ( *d != '\0' && n-- != 0 ) {
++d;
}
dlen = d - dst;
n = dst_size - dlen;

if ( n == 0 ) {
return dlen + strlen( s );
}

while ( *s != '\0' )
{
if ( n != 1 )
{
*d++ = *s;
--n;
}
++s;
}
*d = '\0';

/* count does not include NUL character */
dst_length = dlen + (s - src);

c_assert_if_fail( dst_length < dst_size );

return dst_length;
}
\endcode

If I get a constraint violation, I get a message like the following.

\result
Constraint violation: dst_length < dst_size, file strops.c, line 326
\end result

If one wanted to annotate these constraints, I simply add a '&&
"constraint annotation"' to the expression.

\code snippet
c_assert_if_fail( dst_length < dst_size && "buffer truncation" );
\endcode

The point is that it's possible to create an architecture that
enforces many of the module's constraints in a *consistent* manner
that is useful (since as the library writer, you're the one with the
best knowledge and the most control). Even if it doesn't satisfy all
the people, all the time, one can still write wrappers around these
functions and hard-define C_NO_CONSTRAINTS to reduce the error
checking to nothing which allows people the freedom to completely
customize their error handling behavior absent of any constraint
checking the library provides. In the ideal case, it should reduce
the support needed to help the general user-base interface with the
module.

So for your functions, I would have no problem writing them like the
following (the value of '0' in the examples could be a named constant
if more applicable).

\code snippet

/* A node in the range set. */
struct range_set_node
{
struct bt_node bt_node; /* Binary tree node. */
unsigned long int start; /* Start of region. */
unsigned long int end; /* One past end of region. */
};

/* Returns the position of the first 1-bit in NODE. */
static inline unsigned long int
range_set_node_get_start (const struct range_set_node *node)
{

c_return_value_if_fail( node != NULL, 0 );

return node->start;
}

/* Returns one past the position of the last 1-bit in NODE. */
static inline unsigned long int
range_set_node_get_end (const struct range_set_node *node)
{

c_return_value_if_fail( node != NULL, 0 );

return node->end;
}

/* Returns the number of contiguous 1-bits in NODE. */
static inline unsigned long int
range_set_node_get_width (const struct range_set_node *node)
{

c_return_value_if_fail( node != NULL, 0 );

return node->end - node->start;
}

\endcode

Even though there may be some conflict where the value '0' is a valid
semantic value, the report portion of a constraint violation is often
enough to trigger the developer to quickly fix the error and move on,
or at least recognize that something wrong may be going on and the
result may be invalid. It's more informative to the developer and the
user than just crashing the program. And if one wants blistering
speed, define C_NO_CONSTRAINTS to disable constraint checking and
throw caution to the wind. It's also better than pure defensive
programming, especially scenarios where an error return value shares
the same domain as valid return values. For example, one can return 0
as a result of a NULL pointer constraint violation detected in
strlcat, but it doesn't distinguish it from a valid value of 0 when
copying an empty string "".

It's important to recognize that these macros often can just verify a
partial view of a module's preconditions. That's why it's important
to document explicit preconditions (depending on the formal-ness of
the code) as well as any constraints one can verify in code. Even if
one cannot validate all properties of the preconditions of a module,
checking a subset of those conditions can still be very powerful.

Most of the time, I place constraint checking on all arguments when
applicable to public module functions, but not on private functions.
But, the more complicated a private function is, the more likely I
will add constraints to it (typically of the 'c_assert_if_fail'
variety).

I've been "simulated annealing" on this for quite a while, but I
recognize that there is no one best solution. And I'm sure there are
rocks that one could throw at this setup (like the lack of __func__ to
allow function names; it's still in the annealing phase). All I can
say is that from my personal experience, this scheme's value exceeds
that of 'assert' and the GLib g_return[_val]_if_fail macros. It
allows one to switch quite easily from the defensive programming style
to the contract programming style with a minimal change in its
'response' function pointer. It's as close to the best of both worlds
that I have come up with.

In Navia's case, for any pointer validation beyond semantic checking
for NULL, I would highly recommend not placing it within any library
API other than an allocator that centers on creating "heavy
pointers". If one wants to decorate pointers to dynamically allocated
memory blocks with its allocated size to track how much allocated
memory the application currently uses, or fence checking to place
magic numbers to try to detect memory corruption, determine ALIGN_MAX
to properly align and stack the decorations on the pointer and use
custom allocator routines. Or rely on a memory debugger framework
like valgrind, dmalloc, ... life's too short to reinvent this wheel.

Best regards,
John D.

Ian Collins

unread,

Sep 19, 2011, 4:05:53 PM9/19/11

to

On 09/20/11 06:36 AM, jacob navia wrote:
> Le 19/09/11 09:14, Kleuskes& Moos a écrit :

But the examples you used are precisely the common case where run time
pointer validity checking achieves nothing: both pointers are valid.

--
Ian Collins

Richard Harter

unread,

Sep 19, 2011, 4:09:13 PM9/19/11

to

On Mon, 19 Sep 2011 17:36:57 +0000 (UTC), Kleuskes & Moos
<kle...@somewhere.else.net> wrote:

>On Mon, 19 Sep 2011 16:13:55 +0000, Richard Harter wrote:

[snip]

>> In general, users of libraries are unreliable. Maybe your users write
>> error free code and never have dangling pointers. If so, I am happy for
>> you.
>
>No they aren't and neither am i. I think, however, i can assume both know
>how to use a debugger. Besides, i don't usually write libraries.

Much of my work has been in contexts where relying on a debugger often
isn't an option.

>
>> The issue is, what can you do about being passed bad pointers, if
>> anything. The trouble I have with your post is that you present a false
>> dichotomy - either you accept no responsibility for handling bad input
>> pointers (crap passing users are fair game) or else you go into paranoid
>> mode. ("You" being a library author, not you personally.)
>
>I don't see much in the middle, except replacing pointers with something
>more robust, like your solution below. The only trick i sometimes employ
>is crude range-checking, but that depends on know where in memory a
>pointer should (or rather, should not) point and/or checking alignment if
>required, but these do not end up in production code.

You've probably never thought about the issue. Then again, neither
have I. :-) Let's see. One can use Jacob's self-identifying data
scheme. It's not a universal solution, but it has its place. One can
an error handler that traps on what would otherwise be a crash and
writes an error file. If you are into instrumented code that can be
quite informative. Again, not a universal solution, but it has its
place. And so on. It's all about deciding in advance what your error
handling strategy is going to be rather than tacking it on as an after
thought.

>
>> What you do or even can do depends upon the specific situation. We all

[snip]

>
>> As
>> it happens I am working with some software in which the possibility of
>> stale *pointers* is part of the spec. The answer is not to use raw C
>> pointers but rather to use "safe references" that can be and are checked
>> for validity.
>
>Thus avoiding the problem. Clever. But it doesn't really address being
>passed bad pointers.

Granted.
[snip]

>> Finding the dangling pointer after the rocket has crashed is not the
>> happiest of outcomes.
>
>Nope. And unless you're talking manned missions, there are worse possible
>outcomes, but the solutions suggested in this thread do not strike me as
>being "the way to go".

True enough. There is a difference between "getting it right" and
"lowering the probability of error".

Ian Collins

unread,

Sep 19, 2011, 4:09:40 PM9/19/11

to

On 09/20/11 06:30 AM, jacob navia wrote:
> Le 19/09/11 19:42, Kenneth Brody a écrit :
>>
>> Personally, I prefer debugging a program that crashes at the actual
>> point of failure.
>>
>
> Well, it depends where the point of failure is.
>
> Suppose your library is expecting a pointer to "a" and is passed a
> pointer to "b", a similar syructure but with slightly different layout.

How do your checking function help in this case? They can't.

> 1) The program crashes deep inside the library when processing the
> pointer data. You have a stack trace that tells you:
>
> 0x488776AAD()
> 0x446655FF4()
> MyFunction() myfunction.c 476
> main() main.c 876
>
> Now what?
>
>
> 2) When you pass a bad pointer the library opens a dialog box:
> "Invalid argument to function Library_input(), argument two"
>
> You see?

Again, your example uses an incorrect, but otherwise good, pointer.

--
Ian Collins

Richard Harter

unread,

Sep 19, 2011, 4:11:56 PM9/19/11

to

Snicker. I'm happy to go along with your hypothetical library author.
If their thinking is that muddled, why should I trust their software?

Ben Pfaff

unread,

Sep 19, 2011, 4:20:36 PM9/19/11

to

"BartC" <b...@freeuk.com> writes:

> "Keith Thompson" <ks...@mib.org> wrote in message
> news:lnmxe05...@nuthaus.mib.org...
>> "BartC" <b...@freeuk.com> writes:
>>> "Ben Pfaff" <b...@cs.stanford.edu> wrote in message
>>> news:87zki0w...@blp.benpfaff.org...
>>>
>>>> It's pretty easy to come up with examples of functions that are
>>>> trivial if you don't check the arguments and nontrivial if you
>>>> do. An excerpt from a library of mine is below. Each of these
>>>> functions is only one or two machine instructions when inlined.
>>>
>>> The body of these functions is so trivial that one wonders why they
>>> aren't
>>> written in-line anyway, or at least wrapped in a macro.
>>
>> Writing them in-line would be more error-prone. Making them functions
>> lets you give each operation a name.
>>
>> How would a macro be better than an inline function?
>
> Looking at the original functions, they are just applying some
> abstraction/hiding, so are doing exactly what macros are for.

I would not describe abstraction or hiding as one of the main
reasons for macros, so I'd not use macros in this case.

I don't think that the "get_width" function could be portably
defined as a macro without expanding the node expression twice.

One of the reasons that I defined these as functions is because I
wasn't sure at first that I was going to use this
representation. For example, I could have used start and width
instead of start and end, or I could have used a different
representation that only defined start and end implicitly.
--
char a[]="\n .CJacehknorstu";int putchar(int);int main(void){unsigned long b[]
={0x67dffdff,0x9aa9aa6a,0xa77ffda9,0x7da6aa6a,0xa67f6aaa,0xaa9aa9f6,0x11f6},*p
=b,i=24;for(;p+=!*p;*p/=4)switch(0[p]&3)case 0:{return 0;for(p--;i--;i--)case+
2:{i++;if(i)break;else default:continue;if(0)case 1:putchar(a[i&15]);break;}}}

Richard Harter

unread,

Sep 19, 2011, 4:18:15 PM9/19/11

to

On Mon, 19 Sep 2011 20:36:22 +0200, jacob navia <ja...@spamsink.net>
wrote:

>Le 19/09/11 09:14, Kleuskes & Moos a écrit :
>>
>> If you're writing software (and that's the only way i can think of to pass
>> bogus pointers to a library), you should be able to tell whether or not a
>> pointer is valid or not at a given point. If you're not, *something* is
>> seriously wrong.
>>
>
>
>But you can write in your code
>
>LibraryFunction(handle1,pdata);
>
>instead of the correct
>
>LibraryFunction(pdata,handle1);
>
>you see, because you are (like many other beings)
>a HUMAN, not a machine.
>
>
>And now you do not get any warning because both are
>pointers to opaque types, i.e. eventually void *.

Just as a note, I have found that using a void * pointer as a handle
is not all that great an idea. Better is to use a small struct that
is type specific. Then you can type check and, with a small bit of
thought, do type specific sanity checks. YMMV.

ImpalerCore

unread,

Sep 19, 2011, 4:01:02 PM9/19/11

to

On Sep 19, 1:14 pm, Keith Thompson <ks...@mib.org> wrote:
> "BartC" <b...@freeuk.com> writes:
> > "Ben Pfaff" <b...@cs.stanford.edu> wrote in message
> >news:87zki0w...@blp.benpfaff.org...
>
> >> It's pretty easy to come up with examples of functions that are
> >> trivial if you don't check the arguments and nontrivial if you
> >> do. An excerpt from a library of mine is below. Each of these
> >> functions is only one or two machine instructions when inlined.
>
> > The body of these functions is so trivial that one wonders why they aren't
> > written in-line anyway, or at least wrapped in a macro.
>
> Writing them in-line would be more error-prone. Making them functions
> lets you give each operation a name.

Right, encapsulating the operation as a function assigns a semantic
name to an underlying operation. It is a textbook case of information
hiding. The value of the routine is that if the internal structure
name changes for any reason, the name change only needs to be done to
the access routines, not everywhere in the code. It can provide more
readability in the general case where you may find 'struct->what->does-
>this->mean'. The compiler is typically clever enough in most cases
to optimize the function call away.

> How would a macro be better than an inline function?
>
> [...]

I find a macro to be more beneficial when one wants compiler
granularity to enable/disable the function. For access routines to a
data structure, definitely not.

Best regards,
John D.

jacob navia

unread,

Sep 19, 2011, 4:38:26 PM9/19/11

to

Le 19/09/11 22:09, Ian Collins a écrit :

>
> Again, your example uses an incorrect, but otherwise good, pointer.
>

Ian, the context is the proposed solution with
a 64 bit magic number. You read 8 bytes and test for your
magic number. If you crash or the magic number is not there
then it is a bad pointer you see?

I proposed that a few messages above, maybe you missed it.

jacob

jacob navia

unread,

Sep 19, 2011, 6:59:53 PM9/19/11

to

Le 20/09/11 00:43, Ike Naar a écrit :

Most versions of free() destroy the memory contents. If they don't they
should be replaced ASAP at least in a debug setting.

>
> or perhaps this situation:
>
> SomeType a[1], b[2]; /* suppose a+2 and b+1 are
> the same memory location */
> b[1] = SomeGoodValue(); /* good value and magic number at b[1] */
> a[2]; /* dereferencing bad pointer, but will that be detected?
> after all, there's good data and a good checksum there */

Sorry but I do not have a solution for fixing all possible bugs. If I
would I would be richer than the owners of Google.

What do you expect? Case two is not really a library problem: you
pass it a correct pointer to a correct type, it can't possible
fix that bug!

pete

unread,

Sep 19, 2011, 7:17:42 PM9/19/11

to

Joe Wright wrote:

>
> On 9/18/2011 20:32, Keith Thompson wrote:
> > c...@tiac.net (Richard Harter) writes:

> > [...]
> >> Any language in which it is possible
> >> for a pointer to be invalid but
> >> which provides no way to test for validity is fundamentally flawed.
> >
> > What existing language has pointers
> > and is not "fundamentally flawed"?
> >

> Indeed. Testing a pointer by simple inspection
> can tell you whether it is null or not.
> If it is null it is not valid.
> If not you simply can't tell.

"null" isn't the last word as to whether or not a pointer is valid.
"validity" is context dependent.

The most invalid pointer value, is an indeterminate pointer value.
You can't do anything with an indeterminate pointer value.

Next on the list, is a null pointer value.
There's nothing wrong with
int *a = 0;
int *b = a;
if (b == a) free(a);

Next on the list, is the one past pointer.
You can do pointer arithmetic with the one past pointer.
int a;
int *one_past = &a + 1;
if (one_past - 1 == &a) one_past[-1] = 5;

And there are also operators, such as greater than (>)
which have their own specifications
as to what's valid and what isn't.
Given:
int a;
int b;
int *one_past = &a + 1;

This is valid:
if (one_past > &a) ;

This isn't valid:
if (&b > &a) ;

--
pete

Ben Bacarisse

unread,

Sep 19, 2011, 7:31:26 PM9/19/11

to

jacob navia <ja...@spamsink.net> writes:

> Le 19/09/11 23:30, Ben Bacarisse a écrit :
>> jacob navia<ja...@spamsink.net> writes:
>>
>>> Le 19/09/11 22:50, Ike Naar a écrit :
>>>> On 2011-09-19, jacob navia<ja...@spamsink.net> wrote:
>>>>> Ian, the context is the proposed solution with
>>>>> a 64 bit magic number. You read 8 bytes and test for your
>>>>> magic number. If you crash or the magic number is not there
>>>>> then it is a bad pointer you see?
>>>>
>>>> But if it does not crash and the magic number is there, you
>>>> still cannot be sure that the pointer was good.
>>>
>>>
>>> Well, the chances of hitting a 64 bit number by chance for a well
>>> chosen magic number ar 1 in 2^64...
>>
>> I doubt that very much. Computers are predictable beasts, and you will
>> have these numbers dotted around in memory. They could get copied by
>> buggy code or they may simply be left lying around in deceptive places:
>>
>> mytype *f(void)
>> {
>> mytype t;
>> /* ... */
>> return&t;
>> }
>>
>> A function that uses the return from f may well find the magic number
>> still in place.
>
> Sure, but most compilers nowadays emit a warning when they see such a
> code.

No compiler can emit a warning in all such cases. Obviously I did not
want to suggest that this simple function was a problem -- it's just
illustrative of the fact that the chance of seeing a valid magic number
is not anything like as low as 1 in 2**64.

> Look I do NOT have a solution for finding ALL bugs. If I would
> I would be multimillionaifre already and wouldn't be here...

Quite. I was commenting on your suggestion about the probabilities, and
making no comment about it not finding all bugs.

<snip>
--
Ben.

Jorgen Grahn

unread,

Sep 19, 2011, 7:37:46 PM9/19/11

to

(Or "incomplete types", or whatever they are called, when you deal with
pointers to 'struct Foo' without having seen the definition of struct
Foo.)

Yes -- I'd be much more annoyed by a library which has a weak
interface with respect to types and constness, than by a library
which doesn't check that my pointers go somewhere meaningful.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

Ian Collins

unread,

Sep 19, 2011, 7:50:06 PM9/19/11

to

On 09/20/11 10:59 AM, jacob navia wrote:
>
> Most versions of free() destroy the memory contents. If they don't they
> should be replaced ASAP at least in a debug setting.

Really? I've yet to see one that does, the overhead would be dramatic
for large blocks. Not to mention knowing how to link a different free
from the standard library for debugging.

I did write an allocator that invalidated the memory segment on free,
which was an extremely effective (and efficient) debug aid. It was
limited by the 386 LDT to 4096 concurrent allocations, but that was
plenty in the application in question.

--
Ian Collins

unread,

Sep 20, 2011, 3:03:01 AM9/20/11

to

On Mon, 19 Sep 2011 10:06:03 -0700, Keith Thompson wrote:

> Kleuskes & Moos <kle...@somewhere.else.net> writes:

>> On Sun, 18 Sep 2011 23:10:54 -0400, Joe Wright wrote:
>>> On 9/18/2011 20:32, Keith Thompson wrote:
>>>> c...@tiac.net (Richard Harter) writes: [...]
>>>>> Any language in which it is possible for a pointer to be invalid but
>>>>> which provides no way to test for validity is fundamentally flawed.
>>>>
>>>> What existing language has pointers and is not "fundamentally
>>>> flawed"?
>>>>
>>> Indeed. Testing a pointer by simple inspection can tell you whether it
>>> is null or not. If it is null it is not valid. If not you simply can't
>>> tell.
>>

>> I actually consider NULL a valid value for a pointer, and of course
>> checking for NULL pointers, if called for, is not what i had in mind.
>
> The phrase "valid pointer" is context-dependent.

Aye. But "valid value for a pointer" is not.

> NULL is not valid as
> the operand of the unary "*" operator; it's perfectly valid as the
> argument of free().

True, but that's not what i said. 0 is a valid value for an integer, while
it's an invalid (rhs) operand for '/' and '%'.

-------------------------------------------------------------------------------
______________________________________
/ over in west Philadelphia a puppy is \
\ vomiting ... /
--------------------------------------
\
\
___
{~._.~}
( Y )
()~*~()
(_)-(_)
-------------------------------------------------------------------------------

Kleuskes & Moos

unread,

Sep 20, 2011, 3:16:44 AM9/20/11

to

On Mon, 19 Sep 2011 20:30:42 +0200, jacob navia wrote:

> Le 19/09/11 19:42, Kenneth Brody a écrit :
>>
>> Personally, I prefer debugging a program that crashes at the actual
>> point of failure.
>>
>>
> Well, it depends where the point of failure is.
>
> Suppose your library is expecting a pointer to "a" and is passed a
> pointer to "b", a similar syructure but with slightly different layout.
>
> 1) The program crashes deep inside the library when processing the
> pointer data. You have a stack trace that tells you:
>
> 0x488776AAD()
> 0x446655FF4()
> MyFunction() myfunction.c 476
> main() main.c 876
>
> Now what?

Point the debugger (by JTAG if need be) at the stackframe containing
'MyFunction' and check the arguments used against the parameters required *).

> 2) When you pass a bad pointer the library opens a dialog box:
> "Invalid argument to function Library_input(), argument two"

If the library already invoked undefined behavior (by crashing)
the state of your program is also undefined and opening a dialog
box may well result in another crash, thus hiding the details of
the original crash and preventing debugging most effectively.

I would, personally, object to using any library that does that.

> You see?

Nope.

> A crash isn't always a good thing.

A crash is never a good thing, but it's preferable to continuing
in an undefined state.

*) simple... The _real_ nasties require whipping out logic-analyzer
and squinting at the traffic on the address and data buses and/or
other data-lines, but they don't usually involve prefab libs. If you're
_really_ unlucky, there's only an old-fashioned oscilloscope or even two
leds available. Been there, done that and earned a decent living doing it.

-------------------------------------------------------------------------------
_________________________________________
/ ... If I had heart failure right now, I \
\ couldn't be a more fortunate man!! /
-----------------------------------------

Message has been deleted

Rosario Pulvirenti

unread,

Sep 20, 2011, 3:28:09 AM9/20/11

to

"Rosario Pulvirenti" <NoInUse...@non.usare.invalid> ha scritto nel messaggio
news:4e75ac80$0$15668$4faf...@reader2.news.tin.it...
>
> "jacob navia" <ja...@spamsink.net> ha scritto nel messaggio
> news:j5370i$1nv$1...@speranza.aioe.org...
>> In a recent discussion about preconditions, I argued that testing ALL
>> preconditions would be unpractical. As an example of a precondition
>> impossible to test I presented the problem to know if a pointer
>> points to a really readable place in memory.
>
> i could find [one loop on a list of all pointer released from myMalloc()
> function and a couple of subtration] if a pointer point to some
> MyMalloc area
> if the stack memory is implementd trught myMalloc is possible to see
> if one pointer point valid memory something about
> O(numberPonters) [linear time]

in the everyday case it could be O(1) because memory for functions
is released as stack
could be patological O(numberPonters) cases for global memory
or if the memory not follow the stack model

> but i can think on that we want O(1)
> so something using hash function

so wrong above

Ike Naar

unread,

Sep 20, 2011, 3:26:44 AM9/20/11

to

On 2011-09-19, jacob navia <ja...@spamsink.net> wrote:
> Sorry but I do not have a solution for fixing all possible bugs. If I
> would I would be richer than the owners of Google.

That sounds a lot more realistic than believing 64-bit
magic numbers reduce the chances of false positives to 2^-64.

jacob navia

unread,

Sep 20, 2011, 3:50:12 AM9/20/11

to

Le 20/09/11 09:16, Kleuskes & Moos a écrit :

> On Mon, 19 Sep 2011 20:30:42 +0200, jacob navia wrote:
>
>> Le 19/09/11 19:42, Kenneth Brody a écrit :
>>>
>>> Personally, I prefer debugging a program that crashes at the actual
>>> point of failure.
>>>
>>>
>> Well, it depends where the point of failure is.
>>
>> Suppose your library is expecting a pointer to "a" and is passed a
>> pointer to "b", a similar syructure but with slightly different layout.
>>
>> 1) The program crashes deep inside the library when processing the
>> pointer data. You have a stack trace that tells you:
>>
>> 0x488776AAD()
>> 0x446655FF4()
>> MyFunction() myfunction.c 476
>> main() main.c 876
>>
>> Now what?
>
> Point the debugger (by JTAG if need be) at the stackframe containing
> 'MyFunction' and check the arguments used against the parameters required *).
>

Yes, sure. You see two arguments. Are the pointers bad? The first comes
from an argument to this function, where were that created?

Up the stack.

Get the docs for the call to the library. Ahhh the two arguments
look ok... Maybe this is the consequence of a bug I corrected yesterday.

Let's come back to the situation before those changes.

Pull that version. Ahh Crash also, it wasn't that.
Let's look better at those documentation for that dammed library.

Ahhhh the arguments are inversed damm it!

I found it after 1 lost day. Well, that is developing.

>> 2) When you pass a bad pointer the library opens a dialog box:
>> "Invalid argument to function Library_input(), argument two"
>
> If the library already invoked undefined behavior (by crashing)
> the state of your program is also undefined and opening a dialog
> box may well result in another crash, thus hiding the details of
> the original crash and preventing debugging most effectively.
>

No. There isn't (precisely) any crash. The library cleanly reports a
failure in your program.

> I would, personally, object to using any library that does that.
>
>> You see?
>
> Nope.
>

Of course, you like software that crashes like that. OK.

>> A crash isn't always a good thing.
>
> A crash is never a good thing, but it's preferable to continuing
> in an undefined state.
>

Maybe, maybe not. It depends on the importance of the library in the
software.

For instance, if you are doing a GPS appplication, a crash in the
graphics module that scrolls the map is not that important,
the GPS can continue to work... The main application is a module
that warns the driver if he/she goes beyond the speed limit for the
road. The map is just "to be nice".

Kleuskes & Moos

unread,

Sep 20, 2011, 6:38:44 AM9/20/11

to

On Tue, 20 Sep 2011 09:50:12 +0200, jacob navia wrote:

> Le 20/09/11 09:16, Kleuskes & Moos a écrit :
>> On Mon, 19 Sep 2011 20:30:42 +0200, jacob navia wrote:
>>
>>> Le 19/09/11 19:42, Kenneth Brody a écrit :
>>>>
>>>> Personally, I prefer debugging a program that crashes at the actual
>>>> point of failure.
>>>>
>>>>
>>> Well, it depends where the point of failure is.
>>>
>>> Suppose your library is expecting a pointer to "a" and is passed a
>>> pointer to "b", a similar syructure but with slightly different
>>> layout.
>>>
>>> 1) The program crashes deep inside the library when processing the
>>> pointer data. You have a stack trace that tells you:
>>>
>>> 0x488776AAD()
>>> 0x446655FF4()
>>> MyFunction() myfunction.c 476
>>> main() main.c 876
>>>
>>> Now what?
>>
>> Point the debugger (by JTAG if need be) at the stackframe containing
>> 'MyFunction' and check the arguments used against the parameters
>> required *).
>>
>>
> Yes, sure. You see two arguments. Are the pointers bad? The first comes
> from an argument to this function, where were that created?

How should I know? It's your example and I'm no psychic.

> Up the stack.
>
> Get the docs for the call to the library. Ahhh the two arguments look
> ok... Maybe this is the consequence of a bug I corrected yesterday.

In that case you haven't fixed it.

> Let's come back to the situation before those changes.
>
> Pull that version. Ahh Crash also, it wasn't that. Let's look better at
> those documentation for that dammed library.
>
> Ahhhh the arguments are inversed damm it!
>
> I found it after 1 lost day. Well, that is developing.

True, and if that took you a full day, don't apply for a job in the
firm i work for.

>>> 2) When you pass a bad pointer the library opens a dialog box:
>>> "Invalid argument to function Library_input(), argument two"
>>
>> If the library already invoked undefined behavior (by crashing) the
>> state of your program is also undefined and opening a dialog box may
>> well result in another crash, thus hiding the details of the original
>> crash and preventing debugging most effectively.
>>
>>
> No. There isn't (precisely) any crash. The library cleanly reports a
> failure in your program.

The premisse of your example was "The program crashes deep inside the library when processing the
pointer data". You're shifting goalposts, unless you have secretly invented some mathemagical way
of detecting bad pointers. I'm sure everybody would be greatly interested.

> > I would, personally, object to using any library that does that.
> >
> >> You see?
> >
> > Nope.
> >
> >
>
> Of course, you like software that crashes like that. OK.

Yes, for the reasons already mentioned upthread several times.

> >> A crash isn't always a good thing.
> >
> > A crash is never a good thing, but it's preferable to continuing in
> > an undefined state.
> >
> >
> Maybe, maybe not. It depends on the importance of the library in the
> software.

Always. The one thing worse than a crash is an undefined state in which
your software might do *anything*. In one of the applications i worked at,
it could kill an unsuspecting person, while a crash would cause a
watchdog-timer to trigger, restarting the app and signaling the problem.

> For instance, if you are doing a GPS appplication, a crash in the
> graphics module that scrolls the map is not that important, the GPS can
> continue to work... The main application is a module that warns the
> driver if he/she goes beyond the speed limit for the road. The map is
> just "to be nice".

So? Place the GPS-code in one process and the map in another, use a
watchdog-timer to restart. Software separated logically, problem solved.

Now consider your approach in a pace-maker going haywire due to your continuing
in an undefined state and placing random pulses on the atriums and ventricles.
I'm very sure neither the patient, nor the MD, your boss or even you will be
very happy with the result*).

So, no. You approach is anything but desirable.

*) in fact, there's a last ditch electronic defense prohibiting pulses being sent
too fast, but that's no absolute guarantee. Ill timed pulses may still provoke a
cardiac arrest or other serious problems.

-------------------------------------------------------------------------------
___________________________________
/ HELLO KITTY gang terrorizes town, \
\ family STICKERED to death! /

Malcolm McLean

unread,

Sep 20, 2011, 7:29:28 AM9/20/11

to

On Sep 20, 1:38 pm, Kleuskes & Moos <kleu...@somewhere.else.net>
wrote:

>
> Always. The one thing worse than a crash is an undefined state in which
> your software might do *anything*. In one of the applications i worked at,
> it could kill an unsuspecting person, while a crash would cause a
> watchdog-timer to trigger, restarting the app and signaling the problem.
>

That's not always true. There's no point deliberately crashing a video
game, for instance.

The hard problem is whether or not to allow an emergency save. If you
don't allow it, the user will lose all the work he did before the last
save. If you do allow it, you might save corrupted data which could
cause far more serious problems further down the line.

Kleuskes & Moos

unread,

Sep 20, 2011, 8:04:02 AM9/20/11

to

On Tue, 20 Sep 2011 04:29:28 -0700, Malcolm McLean wrote:

> On Sep 20, 1:38 pm, Kleuskes & Moos <kleu...@somewhere.else.net> wrote:
>>
>> Always. The one thing worse than a crash is an undefined state in which
>> your software might do *anything*. In one of the applications i worked
>> at, it could kill an unsuspecting person, while a crash would cause a
>> watchdog-timer to trigger, restarting the app and signaling the
>> problem.
>>
> That's not always true. There's no point deliberately crashing a video
> game, for instance.

Better than continuing in an undefined state and opening your platform up
for possible exploits (e.g. buffer overrun). Besides, "deliberately crashing"
differs from "crashing on a bad pointer", which was the subject at hand.

Apart from that, there are few persons who would consider a video-game to be
critical for anything. Worst that can happen is a frustrated user demolishing
an innocent keyboard and making an ass of himself on YouTube.

> The hard problem is whether or not to allow an emergency save. If you
> don't allow it, the user will lose all the work he did before the last
> save. If you do allow it, you might save corrupted data which could
> cause far more serious problems further down the line.

The latter is sufficient reason _not_ to allow that, unless you can guarantee
the data to be saved isn't corrupted. The proper way to do things _like_ that
is to generate a dump _in_a_separate_file_, which might later be used to
restore at least some of the previous state.

But things like that only come into play if your app isn't really critical to
anyone and you have a damned good reason for doing so.

-------------------------------------------------------------------------------
________________________________________
/ My nose feels like a bad Ronald Reagan \
\ movie ... /
----------------------------------------

Malcolm McLean

unread,

Sep 20, 2011, 8:25:21 AM9/20/11

to

On Sep 20, 3:04 pm, Kleuskes & Moos <kleu...@somewhere.else.net>
wrote:

> On Tue, 20 Sep 2011 04:29:28 -0700, Malcolm McLean wrote:
>
> > The hard problem is whether or not to allow an emergency save. If you
> > don't allow it, the user will lose all the work he did before the last
> > save. If you do allow it, you might save corrupted data which could
> > cause far more serious problems further down the line.
>
> The latter is sufficient reason _not_ to allow that, unless you can guarantee
> the data to be saved isn't corrupted. The proper way to do things _like_ that
> is to generate a dump _in_a_separate_file_, which might later be used to
> restore at least some of the previous state.
>
> But things like that only come into play if your app isn't really critical to
> anyone and you have a damned good reason for doing so.
>

Often you don't know how the app will be used. For instance a
spreadsheet might be used for an undergraduate research project, or it
might hold details of cancer patients in a drugs trial. The
requirements are the same.
Now lets say the program detects an invalid pointer. If it crashes
out, it destroys maybe six hours work by the undergraduate. That's a
major irritation. He won't use that package again and you've lost a
customer. If you offer to save, of course you'll warn "this data may
be corrupted". One would hope that the cancer trial people have
procedures. But a harassed clerk has just spent half an hour entering
new drug data. She chooses the save option, looks at the data, it
seems OK. 99% of the time, she'll be right to accept it. Will the
cancer trial people really catch that one? Is it your responsibility,
as spreadsheet programmer?

Kenny McCormack

unread,

Sep 20, 2011, 10:18:59 AM9/20/11

to

In article <1bipop9...@pfeifferfamily.net>,
Joe Pfeiffer <pfei...@cs.nmsu.edu> wrote:
...
>And in which those "pointers" have semantics somewhat different from
>what C calls pointers. And, I expect, semantics somewhat different from
>what most readers of this newsgroup infer from the word pointers.

Quite so.

I know of a language that has "pointers" - they call them pointers and use
the same "*" and "&" syntax that C uses - but these pointers are entirely safe
to use.

--
Is God willing to prevent evil, but not able? Then he is not omnipotent.
Is he able, but not willing? Then he is malevolent.
Is he both able and willing? Then whence cometh evil?
Is he neither able nor willing? Then why call him God?
~ Epicurus

ImpalerCore

unread,

Sep 20, 2011, 10:55:04 AM9/20/11

to

Yeah, but sometimes "poor man's templates" can be still pretty good,
at least in C. While the macro technique sure doesn't beat C++
templates with its compile-time type verification, you can still
abstract a lot of the container grunt work that you see over and over
again that doesn't depend on type. From my experiments, using a macro
API to abstract access to type information was miles better than
sprinkling manual type-casting of void* pointers. It allows one to
have an opportunity to do some semblance of type-checking.

For the auto-resizing array, you can provide some type checking since
the container is localized in a single struct. Using a macro, one can
do something like the following.

\code snippet
struct c_array* gc_array_alloc( size_t sz, size_t n, const char*
name )
{
struct c_array* array = NULL;
...

/* alloc and assign 'name' to 'typename' struct member */
}

void* gc_array_front( struct c_array* array, const char* name )

{
void* p = NULL;

c_return_value_if_fail( array != NULL, NULL );

c_return_value_if_fail( strcmp( name, array->typename ) == 0,

NULL );

if ( array->size ) {
p = array->buffer;
}

return p;
}

#define c_array_new( type, n ) \
(gc_array_alloc( sizeof (type), (n), #type ))

#define c_array_front( array, type ) \

( (type*)(gc_array_front( (array), #type )) )
\endcode

I found this kind of setup to be quite fragile and computationally
expensive. You can try to improve upon strcmp with a string
comparison that ignores spaces, but it still can't deal with
typedefs. For this reason, I settled on using 'sizeof' as a hack to
verify that the type argument matches the array's internal type (at
least in size).

If there was a compiler extension that assigned a unique id to a
compiler type (kind of like a compile-time typeid), and it handled
typedefs properly as well, one could architect low cost run-time type
checking constraints into the generic array interface fairly easily,
ideally solving most of the type-mismatch issues with 'void*'. It
might even be cheap enough to use it in distributed containers like
linked lists and trees, provided users would use a macro API instead
of manual casting.

Best regards,
John D.