free
Tor Rustad
deallocated, that is, made available for further allocation."
Can implementations (free function) release allocated space
back to the system, or is there a guarantee here, that the
freeing program has the deallocated space available for
further allocation later?
In my opinion, the standard gives no such guarantee, and
I cannot see why implementations should be restricted
from optimisations, where e.g. the allocated space is
released back to the system.
--
Tor <torust AT online DOT no>
"To this day, many C programmers believe that 'strong typing' just means
pounding extra hard on the keyboard". PvdL
Francis Glassborow
<tor...@online.no.spam> writes
>"The free function causes the space pointed to by ptr to be
>deallocated, that is, made available for further allocation."
>
>Can implementations (free function) release allocated space
>back to the system, or is there a guarantee here, that the
>freeing program has the deallocated space available for
>further allocation later?
>
>In my opinion, the standard gives no such guarantee, and
>I cannot see why implementations should be restricted
>from optimisations, where e.g. the allocated space is
>released back to the system.
Why do you think otherwise? What happens to a resource once it is
released is no concern of the programmer, just as where allocated space
comes from is not. Some implementations may provide such information in
their documentation but even that is not required.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
Tor Rustad
> In article <DE_p8.19651$eJ6.3...@news2.ulv.nextra.no>, Tor Rustad
> <tor...@online.no.spam> writes
> >"The free function causes the space pointed to by ptr to be
> >deallocated, that is, made available for further allocation."
> >
> >Can implementations (free function) release allocated space
> >back to the system, or is there a guarantee here, that the
> >freeing program has the deallocated space available for
> >further allocation later?
> >
> >In my opinion, the standard gives no such guarantee, and
> >I cannot see why implementations should be restricted
> >from optimisations, where e.g. the allocated space is
> >released back to the system.
>
> Why do you think otherwise?
Because, some of the c.l.c regulars don't share my view.
> What happens to a resource once it is
> released is no concern of the programmer,
It should be of concern to someone implementing a conforming
allocator. From an application programmers viewpoint, it is of
interest to know, if malloc'ed space may be released back to
the system or not.
Paul Eggert
> It should be of concern to someone implementing a conforming
> allocator. From an application programmers viewpoint, it is of
> interest to know, if malloc'ed space may be released back to
> the system or not.
The standard says that it can. That's obvious. It has always been obvious.
Francis Glassborow
<tor...@online.no.spam> writes
>> Why do you think otherwise?
>
>Because, some of the c.l.c regulars don't share my view.
Then they should be asked to quote the relevant sub-clause(s) of the
Standard. However I think someone is getting confused, there is no
requirement that it either be retained or returned to the OS. I believe
either is equally correct behaviour.
Eric Sosman
>
> "Francis Glassborow" <francis.g...@ntlworld.com> wrote in message
> > In article <DE_p8.19651$eJ6.3...@news2.ulv.nextra.no>, Tor Rustad
> > <tor...@online.no.spam> writes
> > >"The free function causes the space pointed to by ptr to be
> > >deallocated, that is, made available for further allocation."
> > >
> > >Can implementations (free function) release allocated space
> > >back to the system, or is there a guarantee here, that the
> > >freeing program has the deallocated space available for
> > >further allocation later?
> > >
> > >In my opinion, the standard gives no such guarantee, and
> > >I cannot see why implementations should be restricted
> > >from optimisations, where e.g. the allocated space is
> > >released back to the system.
> >
> > Why do you think otherwise?
>
> Because, some of the c.l.c regulars don't share my view.
My two cents' worth: The phrase "made available for
further allocation" doesn't constrain the implementation
in any significant way, because the program can't prove
that freed memory *isn't* "available."
The program could remember the pointer to and size of
the freed memory and could check whether subsequent malloc()
calls returned pointers into this region, but this has at
least two problems:
- Since the values of pointers into a dynamically-
allocated object become indeterminate when the
memory is freed, any post-free() use of these
values invokes undefined behavior. Hence, the
checks cannot be made safely.
- Even if the implementation defines the U.B. above
(as most do), the program still can't prove the
negative: it can notice when malloc() returns a
pointer into the freed region, but if malloc()
has not done so after some finite number of calls
the program cannot conclude that malloc() would
*never* do so under "proper circumstances."
In short, since the program cannot (reliably) detect
non-compliance with "made available," the phrase embodies
no enforceable constraint.
> It should be of concern to someone implementing a conforming
> allocator. From an application programmers viewpoint, it is of
> interest to know, if malloc'ed space may be released back to
> the system or not.
malloc()ed space must *not* be released; my view is that
free()d space can be. Or it can be filled with 0xDEADBEEF and
never used again, or be zapped with magic rays and caused to
disappear from the system, or ... The program can't tell, so
the program can't complain.
<off-topic>
Lots of people have written memory managers that return
released memory to the system; I have done so. My experience
suggests that the exercise is not worth while; use of the term
"optimization" to describe such an implementation is probably
a misnomer. In order for the practice to be beneficial, all
of the following must hold:
- A program holding on to stale memory imposes a large
performance penalty on the system as a whole (in most
systems there is a penalty, but a small one).
- The system must be able to devote the released memory
to get some other activity to run better (it doesn't
matter who has ownership of unneeded memory).
- Phase I of the program must allocate a large amount
of memory, and then Phase II must need only a little
(releasing only a few KB isn't going to help much).
- Phase II must run for a long time (releasing several
GB and then terminating the program a millisecond
later isn't going to help much).
It's my opinion that situations where all four of these
conditions are met exist, but are pretty rare. Some embedded
applications certainly fall into this category, but they would
probably benefit from a more specialized memory management than
bare-bones malloc(). Some long-running "daemon" programs fit
this profile, but it might be better to re-architect them so the
memory-hungry Phase I runs as a separate program only when needed.
A run-of-the-mill program won't help a run-of-the-mill system by
releasing free()d memory, or not very much at any rate.
In short: Go ahead and release the memory if it makes you
feel happy, but the likely outcome is a slight slowdown, not
a dramatic speed-up.
</off-topic>
Witless
> In article <DE_p8.19651$eJ6.3...@news2.ulv.nextra.no>, Tor Rustad
> <tor...@online.no.spam> writes
> >"The free function causes the space pointed to by ptr to be
> >deallocated, that is, made available for further allocation."
> >
> >Can implementations (free function) release allocated space
> >back to the system, or is there a guarantee here, that the
> >freeing program has the deallocated space available for
> >further allocation later?
> >
> >In my opinion, the standard gives no such guarantee, and
> >I cannot see why implementations should be restricted
> >from optimisations, where e.g. the allocated space is
> >released back to the system.
>
> Why do you think otherwise? What happens to a resource once it is
> released is no concern of the programmer, just as where allocated space
> comes from is not. Some implementations may provide such information in
> their documentation but even that is not required.
There are at least two reasons why an application programmer might be
legitimately interested in he answer to this question. First, if there
were a guarantee that memory once allocated would remain available after
being freed would also guarantee that at least that much space is known to
be available. I.e., a multi-pass program that succeeded in completing a
first, read-only pass could confidently proceed on later, write-enabled
passes that used less than the upper bound established during the first
pass without risk of malloc failure (the possibility of which might require
some kind of commit logic).
Additionally, if it was known that freed memory was released or not would
influence the desired lifetime of a process with a two-phase lifetime. For
example, a process that performs a large amount of processing might use a
considerable amount of temporary memory during the calculations. If the
results of the calculations were needed for an extended period of time it
would be tempting to permit the process to remain in existence (a la
"zombie state") in order to make the results available on demand. However,
if the freed memory is not released such a zombie might be unacceptably
expensive.
Eric Sosman
>
> Francis Glassborow wrote:
>
> >
> > Why do you think otherwise? What happens to a resource once it is
> > released is no concern of the programmer, just as where allocated space
> > comes from is not. Some implementations may provide such information in
> > their documentation but even that is not required.
>
> There are at least two reasons why an application programmer might be
> legitimately interested in he answer to this question. First, if there
> were a guarantee that memory once allocated would remain available after
> being freed would also guarantee that at least that much space is known to
> be available. I.e., a multi-pass program that succeeded in completing a
> first, read-only pass could confidently proceed on later, write-enabled
> passes that used less than the upper bound established during the first
> pass without risk of malloc failure (the possibility of which might require
> some kind of commit logic).
The fact that a certain chunk of memory has been made
"available" by free()ing does not imply that it will still
be "available" by the time a subsequent malloc() comes along.
It's certainly allowable for functions like fopen() to call
malloc() internally; what if an fopen() chews up part of the
chunk in question? And how about variable-length arrays?
Call some innocuous-seeming function that just happens to
use a VLA, and your "available" chunk may well get bitten
into. For that matter, the Standard doesn't prescribe how
to keep track of the call chain or where to allocate `auto'
variables; what if some implementation uses dynamically-
allocated memory for such purposes? A simple call to sqrt()
might chew up the supposedly-available chunk. (I've never
encountered an implementation that did this, but I have used
machines which *could* have done so efficiently.)
Even without the possibility of "competing" allocations
from library functions, VLAs, and oddball implementations,
there's still the problem that freeing 100 bytes doesn't
mean you can later allocate two 50-byte objects. Nor does
freeing two 50-byte objects imply that a 100-byte allocation
will necessarily succeed. In short, the "guarantee" on which
this multi-pass program relies is not reliable.
Dan Pop
>Tor Rustad wrote:
>>
>> "Francis Glassborow" <francis.g...@ntlworld.com> wrote in message
>> > In article <DE_p8.19651$eJ6.3...@news2.ulv.nextra.no>, Tor Rustad
>> > <tor...@online.no.spam> writes
>> > >"The free function causes the space pointed to by ptr to be
>> > >deallocated, that is, made available for further allocation."
>> > >
>> > >Can implementations (free function) release allocated space
>> > >back to the system, or is there a guarantee here, that the
>> > >freeing program has the deallocated space available for
>> > >further allocation later?
>> > >
>> > >In my opinion, the standard gives no such guarantee, and
>> > >I cannot see why implementations should be restricted
>> > >from optimisations, where e.g. the allocated space is
>> > >released back to the system.
>> >
>> > Why do you think otherwise?
>>
>> Because, some of the c.l.c regulars don't share my view.
>
> My two cents' worth: The phrase "made available for
>further allocation" doesn't constrain the implementation
>in any significant way, because the program can't prove
>that freed memory *isn't* "available."
Of course it can: the program allocates a memory block of a certain
size, then immediately free's it. It does nothing for a while, then
it tries to allocate the same amount of memory again, then immediately
free's it. And so on, in a loop.
If the first malloc call has succeeded, the standard guarantees that the
subsequent ones will succeed, too, because the required amount of memory
was made available for further allocation.
Now, another program will run in a tight loop, allocating as much memory
as possible, and never calling free(). If the memory free'd by the
first program is given back to the system, the second program will
eventually grab it. At some point the second program will have taken
all the available memory and the first program will see the failure of a
malloc call that was guaranteed to succeed.
> The program could remember the pointer to and size of
>the freed memory and could check whether subsequent malloc()
>calls returned pointers into this region, but this has at
>least two problems:
Then, don't do it like this :-)
> In short, since the program cannot (reliably) detect
>non-compliance with "made available," the phrase embodies
>no enforceable constraint.
What exactly is unreliable in my scenario?
>> It should be of concern to someone implementing a conforming
>> allocator. From an application programmers viewpoint, it is of
>> interest to know, if malloc'ed space may be released back to
>> the system or not.
>
> malloc()ed space must *not* be released; my view is that
>free()d space can be. Or it can be filled with 0xDEADBEEF and
>never used again, or be zapped with magic rays and caused to
>disappear from the system, or ... The program can't tell, so
>the program can't complain.
The program can tell and it can complain, too.
The intent of the standard is, to me, at least, clear enough: the
free'd memory is not supposed to disappear. Otherwise, I can't see any
good reason for including the words "that is, made available for further
allocation". The first phrase of the relevant paragraph would have
looked like this:
[#2] The free function causes the space pointed to by ptr to
be deallocated.
Why did the committee feel the need to make it any longer than that?
Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Dan...@ifh.de
Barry Margolin
>The intent of the standard is, to me, at least, clear enough: the
>free'd memory is not supposed to disappear. Otherwise, I can't see any
>good reason for including the words "that is, made available for further
>allocation". The first phrase of the relevant paragraph would have
>looked like this:
>
> [#2] The free function causes the space pointed to by ptr to
> be deallocated.
>
>Why did the committee feel the need to make it any longer than that?
ISTM that it's simply trying to clarify what "deallocated" means. If a
block hasn't been freed, you're guaranteed that another call to malloc()
will not return a value that's == to that block (or any address within the
block), but once a block has been freed it's possible for that address to
be returned by a future allocation.
But there's no guarantee or requirement that it *will* ever be returned. I
don't think the intent was to distinguish freed memory from the rest of the
heap, which is also available for allocation.
--
Barry Margolin, bar...@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Tak-Shing Chan
> If the first malloc call has succeeded, the standard guarantees that the
> subsequent ones will succeed, too,
Chapter and verse, please?
> because the required amount of memory
> was made available for further allocation.
Irrelevant. The availability clause is under ``free'', not
under ``malloc'', i.e. availability != guaranteed malloc success.
Tak-Shing
Eric Sosman
>
> In <3CA8B499...@sun.com> Eric Sosman <Eric....@sun.com> writes:
>
> > My two cents' worth: The phrase "made available for
> >further allocation" doesn't constrain the implementation
> >in any significant way, because the program can't prove
> >that freed memory *isn't* "available."
>
> Of course it can: the program allocates a memory block of a certain
> size, then immediately free's it. It does nothing for a while, then
> it tries to allocate the same amount of memory again, then immediately
> free's it. And so on, in a loop.
>
> If the first malloc call has succeeded, the standard guarantees that the
> subsequent ones will succeed, too, because the required amount of memory
> was made available for further allocation.
For one thing, I don't perceive a guarantee in the text you
mention -- but we needn't quibble over it, because I believe the
technique you describe won't work anyhow.
If I understand correctly, the test you propose is
char *p, *q;
p = malloc(1000); /* assume success here */
free (p);
...
q = malloc(1000);
assert (q != NULL); /* the supposed "guarantee" */
If the assertion fails you might be able to conclude that free()
didn't make the memory "available." But what if it succeeds?
The program cannot determine whether the `q' request has reused
the memory once pointed to by `p' or whether some entirely
different chunk of memory was used. (As mentioned elsewhere,
any use of the value of `p' after the free() is unreliable.)
So: was the memory made "available" or was it not? This test
might be able to give a negative answer, but I don't see how it
could give an affirmative.
> Now, another program will run in a tight loop, allocating as much memory
> as possible, and never calling free(). If the memory free'd by the
> first program is given back to the system, the second program will
> eventually grab it. At some point the second program will have taken
> all the available memory and the first program will see the failure of a
> malloc call that was guaranteed to succeed.
The argument seems to embody a rather old-fashioned notion
of "memory," dating from some point before the introduction of
virtual memory in the early 1960s (Burroughs) and perhaps even
earlier (ICL?). There certainly are environments today where
"memory" is the sort of fixed-size immutable resource the above
argument presumes, but it is equally true that other environments
exist in which "memory" is not a sort of zero-sum game between
one program and its peers. The scenario you describe may hold
in some implementations, but is surely not universal.
> The intent of the standard is, to me, at least, clear enough: the
> free'd memory is not supposed to disappear. Otherwise, I can't see any
> good reason for including the words "that is, made available for further
> allocation". The first phrase of the relevant paragraph would have
> looked like this:
>
> [#2] The free function causes the space pointed to by ptr to
> be deallocated.
>
> Why did the committee feel the need to make it any longer than that?
Cheap shot: The adage about committees and camels.
Seriously: Only the committee members can answer your question.
I've regarded the phrase as a clarification of the word "deallocated,"
which is not elsewhere defined in the Standard (unless it was added
after the N869 draft I'm looking at). Extra text for purposes of
clarification is found throughout the Standard; I welcome it and
don't question why it was put there.
P.J. Plauger
> > The intent of the standard is, to me, at least, clear enough: the
> > free'd memory is not supposed to disappear. Otherwise, I can't see any
> > good reason for including the words "that is, made available for further
> > allocation". The first phrase of the relevant paragraph would have
> > looked like this:
> >
> > [#2] The free function causes the space pointed to by ptr to
> > be deallocated.
> >
> > Why did the committee feel the need to make it any longer than that?
>
> Cheap shot: The adage about committees and camels.
>
> Seriously: Only the committee members can answer your question.
> I've regarded the phrase as a clarification of the word "deallocated,"
> which is not elsewhere defined in the Standard (unless it was added
> after the N869 draft I'm looking at). Extra text for purposes of
> clarification is found throughout the Standard; I welcome it and
> don't question why it was put there.
Actually, I argued for an even more economical statement:
: The free function invalidates ptr.
But various people argued that ``deallocation'' was what everyone expected,
and others argued that ``reuse'' was implied. Thus the more loquacious
rendering, even though it brings no additional semantic guarantees to the
party.
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Witless
I think you've overstated your case. There are certainly reasons such as those
you've mentioned to avoid relying upon such a guarantee. However, your statement
seems to claim that it is impossible to write a program that _could_ rely upon
such a guarantee. The logic appears to be similar to the claim that a conforming
translation unit might not be because in some other piece of code invoked by a
call to an external function might deference a NULL or perform another of my
namesake actions.
Given the code fragment:
void * f(void)
{
void * p = malloc( 100 );
if ( !p ) /* complain and abort */;
free( p );
return malloc( 100 );
}
... can this function return NULL?
Douglas A. Gwyn
> Can implementations (free function) release allocated space
Yes, and some do.
However, this isn't a C standards issue since there is no way for
a strictly conforming program to detect or take advantage of it.
James Kuyper Jr.
>
> Eric Sosman wrote:
...
> I think you've overstated your case. There are certainly reasons such as those
> you've mentioned to avoid relying upon such a guarantee. However, your statement
> seems to claim that it is impossible to write a program that _could_ rely upon
> such a guarantee. The logic appears to be similar to the claim that a conforming
> translation unit might not be because in some other piece of code invoked by a
> call to an external function might deference a NULL or perform another of my
> namesake actions.
Dereferencing a NULL doesn't make a program non-conforming. It has
undefined behavior, but that's not sufficient to make a program
non-conforming. In fact, there's precious few things that will make a
program non-conforming (I'm not sure that there are any).
If, instead, you meant "not strictly conforming", or some other
conformance category that is actually useful, then such logic is
perfectly valid. Only entire programs can be strictly conforming; any
particular piece of code, no matter how free of defects the code itself
is, might be part of a program that is not strictly conforming. The
program as a whole might fail to be strictly conforming solely because
of an interaction between the otherwise perfectly legal code in two
different translation units.
> Given the code fragment:
>
> void * f(void)
> {
> void * p = malloc( 100 );
> if ( !p ) /* complain and abort */;
> free( p );
> return malloc( 100 );
> }
>
> ... can this function return NULL?
Sure. Another program running concurrently with yours could perform the
"further allocation" made possible by your call to free(), making a
"further allocation" within your own program fail.
Thad Smith
>
> In article <DE_p8.19651$eJ6.3...@news2.ulv.nextra.no>, Tor Rustad
> <tor...@online.no.spam> writes
> >"The free function causes the space pointed to by ptr to be
> >deallocated, that is, made available for further allocation."
> >
> >Can implementations (free function) release allocated space
> >back to the system, or is there a guarantee here, that the
> >freeing program has the deallocated space available for
> >further allocation later?
> Why do you think otherwise? What happens to a resource once it is
> released is no concern of the programmer, just as where allocated space
> comes from is not. Some implementations may provide such information in
> their documentation but even that is not required.
I think the question here is whether the standard requires that the feed
space be available for later reallocation to the program, when returning
it to the system might make it unavailable for reuse by the same program
because of other use in the system.
Thad
Dan Pop
The whole idea is that, in my scenario, an implementation which gives
back the memory to the system will make the assert fail.
>> Now, another program will run in a tight loop, allocating as much memory
>> as possible, and never calling free(). If the memory free'd by the
>> first program is given back to the system, the second program will
>> eventually grab it. At some point the second program will have taken
>> all the available memory and the first program will see the failure of a
>> malloc call that was guaranteed to succeed.
>
> The argument seems to embody a rather old-fashioned notion
>of "memory," dating from some point before the introduction of
>virtual memory in the early 1960s (Burroughs) and perhaps even
>earlier (ICL?). There certainly are environments today where
>"memory" is the sort of fixed-size immutable resource the above
>argument presumes, but it is equally true that other environments
>exist in which "memory" is not a sort of zero-sum game between
>one program and its peers. The scenario you describe may hold
>in some implementations, but is surely not universal.
Please provide a concrete example of a (real) system where my scenario
may not work. It is trivial to modify your code so that systems with
lazy swap space allocation (which are non-conforming as C implementations,
anyway) are properly handled: just write every byte of memory, after
allocating the memory block, in both programs.
If memory, either real or virtual is a finite resource, it will be
eventually exhuasted in my scenario.
>> The intent of the standard is, to me, at least, clear enough: the
>> free'd memory is not supposed to disappear. Otherwise, I can't see any
>> good reason for including the words "that is, made available for further
>> allocation". The first phrase of the relevant paragraph would have
>> looked like this:
>>
>> [#2] The free function causes the space pointed to by ptr to
>> be deallocated.
>>
>> Why did the committee feel the need to make it any longer than that?
>
> Cheap shot: The adage about committees and camels.
>
> Seriously: Only the committee members can answer your question.
>I've regarded the phrase as a clarification of the word "deallocated,"
>which is not elsewhere defined in the Standard (unless it was added
>after the N869 draft I'm looking at). Extra text for purposes of
>clarification is found throughout the Standard; I welcome it and
>don't question why it was put there.
And yet, you argue that an implementor is free to ignore it. Since the
text is normative (it's not in a footnote or in an example), and since
a strictly conforming program can tell the difference, I have to disagree.
Eric Sosman
>
> In <3CA9F4EF...@sun.com> Eric Sosman <Eric....@sun.com> writes:
>
> >Dan Pop wrote:
> >>
> >> Now, another program will run in a tight loop, allocating as much memory
> >> as possible, and never calling free(). If the memory free'd by the
> >> first program is given back to the system, the second program will
> >> eventually grab it. At some point the second program will have taken
> >> all the available memory and the first program will see the failure of a
> >> malloc call that was guaranteed to succeed.
> >
> > The argument seems to embody a rather old-fashioned notion
> >of "memory," dating from some point before the introduction of
> >virtual memory in the early 1960s (Burroughs) and perhaps even
> >earlier (ICL?). There certainly are environments today where
> >"memory" is the sort of fixed-size immutable resource the above
> >argument presumes, but it is equally true that other environments
> >exist in which "memory" is not a sort of zero-sum game between
> >one program and its peers. The scenario you describe may hold
> >in some implementations, but is surely not universal.
>
> Please provide a concrete example of a (real) system where my scenario
> may not work.
Solaris, for one. (Open)VMS, for another. And probably AIX,
OS/400, MVS (or whatever they call it these days), Linux, FreeBSD,
and MacOS, although I can't vouch for these personally.
Any system that provides more "memory" (RAM + swap + whatever
else the platform uses) than a single program can address falsifies
the test you propose. A program with 32-bit pointers cannot possibly
address more than 4GB of memory; configure a system with, say, 10GB
of swap and your two programs will run right up to their maxima
without trouble.
> If memory, either real or virtual is a finite resource, it will be
> eventually exhuasted in my scenario.
... but other limits may kick in well before that occurs.
> >> [#2] The free function causes the space pointed to by ptr to
> >> be deallocated.
> >>
> >> Why did the committee feel the need to make it any longer than that?
> >
> > Cheap shot: The adage about committees and camels.
> >
> > Seriously: Only the committee members can answer your question.
> >I've regarded the phrase as a clarification of the word "deallocated,"
> >which is not elsewhere defined in the Standard (unless it was added
> >after the N869 draft I'm looking at). Extra text for purposes of
> >clarification is found throughout the Standard; I welcome it and
> >don't question why it was put there.
>
> And yet, you argue that an implementor is free to ignore it. Since the
> text is normative (it's not in a footnote or in an example), and since
> a strictly conforming program can tell the difference, I have to disagree.
I'll refer you to P.J. Plauger's contribution to this thread,
wherin he writes that the phrase in question "brings no additional
semantic guarantees to the party."
As for the distinction between wording in normative and non-
normative sections, it seems to me absurd to treat every single
word in a normative section as implying a constraint. Normative
sections contain, here and there, material that is obviously just
background, clarification, and illumination. For example, consider
1. Scope
2/ This International Standard does not specify
[list omitted]
Now, what exactly is the constraint or specification communicated
by this paragraph? What instruction does it give to implementors
and to users? What features of the C language does it delineate?
It's background material, it's clarification, it's amplification,
it's the Committee trying to be helpful to the reader. And, I
maintain, it's not unique.
Al Grant
The C standards issue is whether free() makes the store "available
for further allocation" to the extent that a strictly conforming
program can assume that an immediately following malloc() of the
same size returns non-NULL. We can assume that all normative text
means something with respect to strictly conforming programs, and
if "further allocation" doesn't mean that, what _does_ it mean?
(If free() releases store to the system then it becomes non-trivial
to meet such a guarantee.)
Eric Sosman
>
> Eric Sosman wrote:
> >
> > mean you can later allocate two 50-byte objects. Nor does
> > freeing two 50-byte objects imply that a 100-byte allocation
> > will necessarily succeed. In short, the "guarantee" on which
> > this multi-pass program relies is not reliable.
>
> I think you've overstated your case. There are certainly reasons such as those
> you've mentioned to avoid relying upon such a guarantee. However, your statement
> seems to claim that it is impossible to write a program that _could_ rely upon
> such a guarantee.
It comes down to what you mean by "rely." If it means "hope
for the best," I'll concede your point. If it means something
stronger, I remain unconvinced.
> The logic appears to be similar to the claim that a conforming
> translation unit might not be because in some other piece of code invoked by a
> call to an external function might deference a NULL or perform another of my
> namesake actions.
I don't see the similarity. The notion of conformance is
defined in terms of an entire program, not of an isolated
translation unit. One cannot ask whether strlen() is conforming;
one instead must ask whether a given program calls strlen() with
an invalid argument.
> void * f(void)
> {
> void * p = malloc( 100 );
> if ( !p ) /* complain and abort */;
> free( p );
> return malloc( 100 );
> }
>
> ... can this function return NULL?
I believe it can. Rhetorical question: would you as a
professional programmer check for a NULL return, or would
you blithely dereference the returned value? If you had
free()d a megabyte, for how many subsequent malloc(100)
calls would you omit a NULL check?
Douglas A. Gwyn
> The C standards issue is whether free() makes the store "available
> for further allocation" to the extent that a strictly conforming
> program can assume that an immediately following malloc() of the
> same size returns non-NULL.
There is never any circumstance under which the standard guarantees
that malloc() will return a non-null pointer.
The "further allocation" is a warning that the same pointer value
might or might not be valid later, although it is not valid
immediately after the free().
Witless
Of course.
> , or would you blithely dereference the returned value?
Not ever.
But these questions are orthogonal to the issue at hand. I might have answered your
questions as I did because I fear non-standard environments, not because a
standard-compliant system might be a problem.
> If you had free()d a megabyte, for how many subsequent malloc(100)
> calls would you omit a NULL check?
By the above logic, none. By the logic of the original thesis, only one because the
overhead per allocation could (in theory only) exceed the remaining known-available
space.
These issues are distractions. On the original C system the definition of
malloc/free/realloc permitted a program to free a block with the guarantee not only
that the space would be available, and not only would the very same space be available,
but the contents of the freed/reallocated block would be intact.
I see little value in those additional guarantees. However, simply knowing that the
same amount of space is available can be extremely valuable. For example, if the
program requires a large amount of memory with respect to the platform, and will also
run for a long time, then knowing that it is not going to die from memory starvation is
important.
Rather than write my own memory allocator in such a situation (which BTW I've done
multiple times), I'd prefer to perform trial allocations to establish whether
sufficient memory was available for the computation to complete. If I can't allocate
the memory there's little purpose in burning up the MIPS.
Of course this requirement could be met with an availability description that had, for
example, total available, and a measure of the per-allocation overhead (which I know
may not be simple, but can be estimated simply). Lacking such a platform description
the program above is easiest to design by relying upon the non-release of memory that
has been freed.
Dan Pop
>Dan Pop wrote:
>>
>> In <3CA9F4EF...@sun.com> Eric Sosman <Eric....@sun.com> writes:
>>
>> >Dan Pop wrote:
>> >>
>> >> Now, another program will run in a tight loop, allocating as much memory
>> >> as possible, and never calling free(). If the memory free'd by the
>> >> first program is given back to the system, the second program will
>> >> eventually grab it. At some point the second program will have taken
>> >> all the available memory and the first program will see the failure of a
>> >> malloc call that was guaranteed to succeed.
>> >
>> > The argument seems to embody a rather old-fashioned notion
>> >of "memory," dating from some point before the introduction of
>> >virtual memory in the early 1960s (Burroughs) and perhaps even
>> >earlier (ICL?). There certainly are environments today where
>> >"memory" is the sort of fixed-size immutable resource the above
>> >argument presumes, but it is equally true that other environments
>> >exist in which "memory" is not a sort of zero-sum game between
>> >one program and its peers. The scenario you describe may hold
>> >in some implementations, but is surely not universal.
>>
>> Please provide a concrete example of a (real) system where my scenario
>> may not work.
>
> Solaris, for one. (Open)VMS, for another. And probably AIX,
>OS/400, MVS (or whatever they call it these days), Linux, FreeBSD,
>and MacOS, although I can't vouch for these personally.
Why?
> Any system that provides more "memory" (RAM + swap + whatever
>else the platform uses) than a single program can address falsifies
>the test you propose. A program with 32-bit pointers cannot possibly
>address more than 4GB of memory; configure a system with, say, 10GB
>of swap and your two programs will run right up to their maxima
>without trouble.
This is trivial to work around: run more "memory eaters" in parallel.
No matter how much memory you have, it is possible to use all of it.
>> If memory, either real or virtual is a finite resource, it will be
>> eventually exhuasted in my scenario.
>
> ... but other limits may kick in well before that occurs.
>
>> >> [#2] The free function causes the space pointed to by ptr to
>> >> be deallocated.
>> >>
>> >> Why did the committee feel the need to make it any longer than that?
>> >
>> > Cheap shot: The adage about committees and camels.
>> >
>> > Seriously: Only the committee members can answer your question.
>> >I've regarded the phrase as a clarification of the word "deallocated,"
>> >which is not elsewhere defined in the Standard (unless it was added
>> >after the N869 draft I'm looking at). Extra text for purposes of
>> >clarification is found throughout the Standard; I welcome it and
>> >don't question why it was put there.
>>
>> And yet, you argue that an implementor is free to ignore it. Since the
>> text is normative (it's not in a footnote or in an example), and since
>> a strictly conforming program can tell the difference, I have to disagree.
>
> I'll refer you to P.J. Plauger's contribution to this thread,
>wherin he writes that the phrase in question "brings no additional
>semantic guarantees to the party."
Then, why is it there?
> As for the distinction between wording in normative and non-
>normative sections, it seems to me absurd to treat every single
>word in a normative section as implying a constraint. Normative
>sections contain, here and there, material that is obviously just
>background, clarification, and illumination.
The standard would be a completely useless document if each implementor
were free to decide for himself which parts of the normative text are
normative and which are "obviously just background, clarification,
and illumination". For the latter purpose, the standard uses footnotes
and examples. Since they are perfectly adequate for this purpose, and
the standard makes extensive use of them, there is no need to randomly
spread such text inside the normative paragraphs.
>For example, consider
>
> 1. Scope
> 2/ This International Standard does not specify
> [list omitted]
>
>Now, what exactly is the constraint or specification communicated
>by this paragraph? What instruction does it give to implementors
>and to users? What features of the C language does it delineate?
>It's background material, it's clarification, it's amplification,
>it's the Committee trying to be helpful to the reader. And, I
>maintain, it's not unique.
It's more than that: it explicitly specifies what aspects of an
implementation are outside the scope of that standard.
Dan Pop
>There is never any circumstance under which the standard guarantees
>that malloc() will return a non-null pointer.
There are only two circumstances where the standard (7.20.3#1 in N869)
gives malloc a licence to return a null pointer:
1. The memory cannot be allocated.
2. The size of the requested space is zero.
>The "further allocation" is a warning that the same pointer value
>might or might not be valid later, although it is not valid
>immediately after the free().
What is the purpose of this "warning"? The pointer value is indeterminate
after free was called and will remain indeterminate until the end of the
program:
The value of a pointer
that refers to freed space is indeterminate.
Tak-Shing Chan
> In <3CAB47C1...@null.net> "Douglas A. Gwyn" <DAG...@null.net> writes:
>
>>There is never any circumstance under which the standard guarantees
>>that malloc() will return a non-null pointer.
>
> There are only two circumstances where the standard (7.20.3#1 in N869)
> gives malloc a licence to return a null pointer:
>
> 1. The memory cannot be allocated.
Irrelevant to the argument, because memory allocation on the
execution environment is outside the scope of C (Clause 1).
> 2. The size of the requested space is zero.
Irrelevant here.
Tak-Shing
Tak-Shing Chan
> This is trivial to work around: run more "memory eaters" in parallel.
> No matter how much memory you have, it is possible to use all of it.
This is trivial to work around: set limits.
> It's more than that: it explicitly specifies what aspects of an
> implementation are outside the scope of that standard.
It is far from explicit: for example it does not explicitly
specify that the name of the company producing the implemention
is outside the scope of the standard.
Tak-Shing
Barry Margolin
>In <3CAB2648...@sun.com> Eric Sosman <Eric....@sun.com> writes:
>> I'll refer you to P.J. Plauger's contribution to this thread,
>>wherin he writes that the phrase in question "brings no additional
>>semantic guarantees to the party."
>
>Then, why is it there?
As numerous people have already said in this thread, it's there to clarify
the meaning of the word "deallocate", since it's never formally defined.
Perhaps it should say "make potentially available for further allocation."
The point of the statement is to make it clear that the memory is no longer
in reserve, like allocated memory. It could be used for future
allocations, but that doesn't mean it *will* be. There could be a variety
of reasons why the implementation might not reuse it, but the one excuse it
can't use is "because it's already allocated."
Douglas A. Gwyn
> 1. The memory cannot be allocated.
The implementation gets to determine when that occurs.
Just because some region has been free()d does not mean it
is allocatable. Some allocators, e.g. those based on "buddy system"
algorithms, may reasonably refuse to allocate a requested amount of
storage even if there is a contiguous free chunk big enough, if it
doesn't meet other requirements such as a specific block alignment.
> The pointer value is indeterminate after free was called and will
> remain indeterminate until the end of the program:
> The value of a pointer that refers to freed space is indeterminate.
That's a short way for the standard to ensure that s.c. programs
cannot misuse the pointer to freed storage. However, the pointer
variable can be later assigned a determinate pointer value that
happens to compare equal to a previous freed value. While a s.c.
program cannot test for that, not every program is s.c. and so it
is appropriate to caution the programmer that it could happen;
otherwise there are algorithms that could benefit from the knowledge
that every successful allocation returns a unique handle.
Dan Pop
>In article <a8hdhl$42c$1...@sunnews.cern.ch>, Dan Pop <Dan...@ifh.de> wrote:
>>In <3CAB2648...@sun.com> Eric Sosman <Eric....@sun.com> writes:
>>> I'll refer you to P.J. Plauger's contribution to this thread,
>>>wherin he writes that the phrase in question "brings no additional
>>>semantic guarantees to the party."
>>
>>Then, why is it there?
>
>As numerous people have already said in this thread, it's there to clarify
>the meaning of the word "deallocate", since it's never formally defined.
And then, the same people argue that the implementor is free to
*completely* ignore the clarification. Hence my question: why provide the
clarification in the first place?
>Perhaps it should say "make potentially available for further allocation."
>The point of the statement is to make it clear that the memory is no longer
>in reserve, like allocated memory. It could be used for future
>allocations, but that doesn't mean it *will* be. There could be a variety
>of reasons why the implementation might not reuse it, but the one excuse it
>can't use is "because it's already allocated."
What would be a *good* reason not to reuse the memory made available
for further allocation, in the scenario I have already described?
Dan Pop
>Dan Pop wrote:
>> 1. The memory cannot be allocated.
>
>The implementation gets to determine when that occurs.
>Just because some region has been free()d does not mean it
>is allocatable. Some allocators, e.g. those based on "buddy system"
>algorithms, may reasonably refuse to allocate a requested amount of
>storage even if there is a contiguous free chunk big enough, if it
>doesn't meet other requirements such as a specific block alignment.
In my scenario, the memory block in question was already allocated
once to the program. So, after being deallocated, it has all the
required properties to satisfy an *identical* allocation request.
Therefore, the subsequent malloc calls cannot fail as long as this
block was deallocated. Otherwise, either malloc or free have failed to
meet the requirements imposed by the standard.
If the relevant text of the standard is supposed to mean something else
than it actually says, then it was poorly rewritten and should be fixed.
Svante
> In <3CAD1198...@null.net> "Douglas A. Gwyn" <DAG...@null.net> writes:
>
> >Dan Pop wrote:
> >> 1. The memory cannot be allocated.
> >
> >The implementation gets to determine when that occurs.
> >Just because some region has been free()d does not mean it
> >is allocatable. Some allocators, e.g. those based on "buddy system"
> >algorithms, may reasonably refuse to allocate a requested amount of
> >storage even if there is a contiguous free chunk big enough, if it
> >doesn't meet other requirements such as a specific block alignment.
>
> In my scenario, the memory block in question was already allocated
> once to the program. So, after being deallocated, it has all the
> required properties to satisfy an *identical* allocation request.
> Therefore, the subsequent malloc calls cannot fail as long as this
> block was deallocated. Otherwise, either malloc or free have failed to
> meet the requirements imposed by the standard.
There are no properties specified by the standard for when an allocation
rqeuest is successful or not. It only says that an allocation request can
return a null pointer when the space requested cannot be allocated. The
circumstances which decides when it cannot be allocated is not something
the standard poses any requirements about.
>
> If the relevant text of the standard is supposed to mean something else
> than it actually says, then it was poorly rewritten and should be fixed.
No - it's perfectly understandable as it is. The space free()'d is made
available for further allocation. That's perfectly clear. To say that
because you have once managed to allocated N bytes and the free()'d them
that the standard requires a subsequent malloc() of N bytes to succeed
from that statement is of course just plain wrong.
Variable length arrays, function calls, library routines, internal functions
used by the implementation, or external to the implementation, or just
about anything could affect who actually get's that chunk of memory, if anyone,
and how.
From the standard:
"The malloc function returns either a null pointer or a pointer to the
allocated space."
"... Undefined behavior is otherwise indicated in this International
Standard by the words 荘undefined behavior鋳 or by the omission of any
explicit definition of behavior. ..."
Note the "...omission of any explicit...".
You are assuming free()/alloc() work from a heap with fixed determinable
size reserved for that use. There is _nothing_ in the standard that says
or implies this is so. malloc() can return a null pointer or a pointer to
the allocated space. The standard poses no requirements on when or how
either of those cases happens, except to say "when it cannot be allocated".
An implementation that returns a null pointer for every other malloc request
would be a conforming implementation, if the implementation for some internal
reason decides that it cannot allocate the space at those points. Not very
useful except for stress testing of course. But still conforming.
--
/Svante
http://axcrypt.sourceforge.net
Free AES Point'n'Click File Encryption for Windows 9x/ME/2K/XP
Barry Margolin
>What would be a *good* reason not to reuse the memory made available
>for further allocation, in the scenario I have already described?
Because it has been returned to the OS, for possible use by other
processes, shared memory, etc.
Douglas A. Gwyn
> So, after being deallocated, it has all the
> required properties to satisfy an *identical* allocation request.
No, because that could depend on whether there are free neighbors,
whether splitting would leave an unusable fragment, etc.
> If the relevant text of the standard is supposed to mean something
> else than it actually says, then it was poorly rewritten and should
> be fixed.
Experience has shown that no amount of wordsmithing can produce a
specification that cannot be willfully misread. Therefore our
criterion is whether a *reasonable* reader could misunderstand the
intention of the specification. Occasionally we discover that that
occurs, and we then work to fix it. In this particular case, I'm
not convinced that the misunderstanding you're promoting is
reasonable.
Christian Bau
>
> Dan Pop wrote:
> > So, after being deallocated, it has all the
> > required properties to satisfy an *identical* allocation request.
>
> No, because that could depend on whether there are free neighbors,
> whether splitting would leave an unusable fragment, etc.
An example what a "nice" implementation could do: Consider this code
fragment.
char* p;
char* q;
p = malloc (1000000); free (p);
q = malloc (1000000); free (p); /* <- Bug. Should have been free (q) */
Undefined behaviour, but in a typical implementation it would be
relatively likely that the value returned in q happens to be exactly the
same as the one still stored in p, so free () wouldn't have a chance of
finding this bug. A clever malloc () implementation could deliberately
refuse to return a pointer that would compare equal to one that has
recently been free()'d. That would have the advantage that the common
bug of free()ing the same pointer twice is more likely to be caught.
In such an implementation, if there was exactly enough memory to malloc
() 1000000 bytes before the first call, then you could expect that the
first allocation succeeds, but the second doesn't because there is not
enough space to allocate 1000000 bytes with a different starting
address.
Al Grant
> In article <a8js0f$aj1$1...@sunnews.cern.ch>, Dan Pop <Dan...@ifh.de> wrote:
> >What would be a *good* reason not to reuse the memory made available
> >for further allocation, in the scenario I have already described?
>
> Because it has been returned to the OS, for possible use by other
> processes, shared memory, etc.
But that's not within the scope of the standard. When the standard
requires free to make the space available "for further allocation",
that must be in some sense within the scope of the standard.
I don't buy the argument that "the free function causes the space...
to be... made available for further allocation" is really a warning
about pointer values. It's normative text and must be read as such,
and read within the scope of the standard. What it _says_ is that
free makes the space available for further allocation. What, exactly,
is the meaning of that requirement on a C implementation?
If there is really no requirement, then it would be better to word
the definition of free as something like "after a call to free the
pointer and pointers derived from it become invalid", leaving it
unspecified what happens to the space.
Dan Pop
>In article <a8js0f$aj1$1...@sunnews.cern.ch>, Dan Pop <Dan...@ifh.de> wrote:
>>What would be a *good* reason not to reuse the memory made available
>>for further allocation, in the scenario I have already described?
>
>Because it has been returned to the OS, for possible use by other
>processes, shared memory, etc.
In this case, it was not made available for further allocation to the
program, which was precisely my point.
Dan Pop
>Dan Pop wrote:
>> So, after being deallocated, it has all the
>> required properties to satisfy an *identical* allocation request.
>
>No, because that could depend on whether there are free neighbors,
>whether splitting would leave an unusable fragment, etc.
Not in my scenario, where there are no other allocation/deallocation
requests involved.
>> If the relevant text of the standard is supposed to mean something
>> else than it actually says, then it was poorly rewritten and should
>> be fixed.
>
>Experience has shown that no amount of wordsmithing can produce a
>specification that cannot be willfully misread. Therefore our
>criterion is whether a *reasonable* reader could misunderstand the
>intention of the specification. Occasionally we discover that that
>occurs, and we then work to fix it. In this particular case, I'm
>not convinced that the misunderstanding you're promoting is
>reasonable.
So, it is unreasonable to interpret "made available for further
allocation" as "made available for further allocation", but it is
perfectly reasonable to interpret it as "do whatever the implementor
wants, including nothing at all".
Francis Glassborow
<alg...@myrealbox.com> writes
>If there is really no requirement, then it would be better to word
>the definition of free as something like "after a call to free the
>pointer and pointers derived from it become invalid", leaving it
>unspecified what happens to the space.
Actually some such wording would, among other things, help with making
some debugging implementations conforming (I know of at least one that
in the most extreme form never recycles memory so as to allow detection
of attempts to use dead objects)
--
Francis Glassborow ACCU
64 Southfield Rd
Oxford OX4 1PA +44(0)1865 246490
All opinions are mine and do not represent those of any organisation
P.J. Plauger
> In article <5765b025.02040...@posting.google.com>, Al Grant
> <alg...@myrealbox.com> writes
> >If there is really no requirement, then it would be better to word
> >the definition of free as something like "after a call to free the
> >pointer and pointers derived from it become invalid", leaving it
> >unspecified what happens to the space.
That was the point I made nearly 20 years ago, but the C committee chose
to say more. I just can't think of any test program that determines
whether the freed storage is made available for reuse.
> Actually some such wording would, among other things, help with making
> some debugging implementations conforming (I know of at least one that
> in the most extreme form never recycles memory so as to allow detection
> of attempts to use dead objects)
And how is this non-conforming?
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Francis Glassborow
Plauger <p...@dinkumware.com> writes
>> Actually some such wording would, among other things, help with making
>> some debugging implementations conforming (I know of at least one that
>> in the most extreme form never recycles memory so as to allow detection
>> of attempts to use dead objects)
>
>And how is this non-conforming?
Well I never thought it was, but certainly free in such a system does
not make the memory available for any further use.
P.J. Plauger
> In article <3cb191eb$0$11112$4c41...@reader1.ash.ops.us.uu.net>, P.J.
> Plauger <p...@dinkumware.com> writes
> >> Actually some such wording would, among other things, help with making
> >> some debugging implementations conforming (I know of at least one that
> >> in the most extreme form never recycles memory so as to allow detection
> >> of attempts to use dead objects)
> >
> >And how is this non-conforming?
>
> Well I never thought it was, but certainly free in such a system does
> not make the memory available for any further use.
I can't think of any conforming program whose behavior depends on whether
freed memory is actually made available for future use. The C Standard
doesn't say when you can count on seeing the resurrected storage. If you
can't tell that freed memory is actually made available, then you can't
tell if it isn't. By the AS IF rule, a malloc/free that doesn't get around
to reusing memory is conforming.
Barry Margolin
Francis Glassborow <fran...@robinton.demon.co.uk> wrote:
>In article <3cb191eb$0$11112$4c41...@reader1.ash.ops.us.uu.net>, P.J.
>Plauger <p...@dinkumware.com> writes
>>> Actually some such wording would, among other things, help with making
>>> some debugging implementations conforming (I know of at least one that
>>> in the most extreme form never recycles memory so as to allow detection
>>> of attempts to use dead objects)
>>
>>And how is this non-conforming?
>
>Well I never thought it was, but certainly free in such a system does
>not make the memory available for any further use.
I guess that depends on what "use" means. It's using it to detect bugs in
the program, rather than to satisfy future malloc() calls.
And regardless of that, just because memory is available it doesn't mean it
has to be used.
Francis Glassborow
<bar...@genuity.net> writes
>I guess that depends on what "use" means. It's using it to detect bugs in
>the program, rather than to satisfy future malloc() calls.
>
>And regardless of that, just because memory is available it doesn't mean it
>has to be used.
Even when there isn't any other memory available?
Barry Margolin
Francis Glassborow <fran...@robinton.demon.co.uk> wrote:
>In article <9nks8.7$rj4...@paloalto-snr1.gtei.net>, Barry Margolin
><bar...@genuity.net> writes
>>I guess that depends on what "use" means. It's using it to detect bugs in
>>the program, rather than to satisfy future malloc() calls.
>>
>>And regardless of that, just because memory is available it doesn't mean it
>>has to be used.
>
>Even when there isn't any other memory available?
Sure. The standard doesn't say how an implementation decides which memory
to use to satisfy a particular call to malloc(). Being available is
necessary, but may not be sufficient.
Douglas A. Gwyn
> Al Grant <alg...@myrealbox.com> writes
> >If there is really no requirement, then it would be better to word
> >the definition of free as something like "after a call to free the
> >pointer and pointers derived from it become invalid", leaving it
> >unspecified what happens to the space.
I thought we already had such wording. However, without some
mention of possible later reallocation, people like the nit-pickers
in this newsgroup might claim that all successful allocations are
required to return distinct pointers, which is certainly not what
is desired.
> Actually some such wording would, among other things, help with making
> some debugging implementations conforming (I know of at least one that
> in the most extreme form never recycles memory so as to allow detection
> of attempts to use dead objects)
There is nothing nonconforming about that. (Although one wonders
where all the addresses come from; doesn't it eventually run out?)
Douglas A. Gwyn
> Even when there isn't any other memory available?
malloc() is not required to succeed even if one somehow knows
that there is a big enough contiguous chunk somewhere in the
resources available to the program. It could well be that the
chunk wouldn't have correct properties for the allocation
scheme. E.g. in the buddy system there could be two adjacent
power-of-two-sized blocks that together would be large enough,
but not individually on the boundaries imposed by the algorithm/
data structure.
Douglas A. Gwyn
> So, it is unreasonable to interpret "made available for further
> allocation" as "made available for further allocation", but it is
> perfectly reasonable to interpret it as "do whatever the implementor
> wants, including nothing at all".
No, what is unreasonable is your particular interpretation of
the phrase. Others have explained what is meant, but as usual
you stubbornly insist on your own foolish notion. I have a
hard time believing that you actually believe in it; it seems
to me your motivation is to proclaim your cleverness, not to
address real usability issues with the C standard.
Francis Glassborow
<DAG...@null.net> writes
Yes, I know that. However, wearing a language lawyer hat:
int main(){
char * cp = malloc(100);
free (cp);
cp = malloc(100); // line A
return 0;
}
I believe that we intend to allow line A to fail. A reading of the
standard (and not a pathological one) indicates that it cannot fail
because the resource is 'provably' available.
This seems like a genuine DR, and one that can be simply addressed.
Francis Glassborow
<bar...@genuity.net> writes
>>Even when there isn't any other memory available?
>
>Sure. The standard doesn't say how an implementation decides which memory
>to use to satisfy a particular call to malloc(). Being available is
>necessary, but may not be sufficient.
I think you are missing the point of the argument that has been going
on. Basically if I have successfully acquired a block of memory with
malloc and then freed it, the contention is that a later call of malloc
for a block of memory not larger than that freed must succeed unless
some other, internal to the program, has acquired that memory.
Personally I do not think we intended to give such a guarantee but the
current wording does lend itself to such an interpretation. Note that we
do not need such a restrictive interpretation because programs that want
can acquire and manage their own memory resources. Well they could if
OS's did not use lazy allocation mechanisms.
Francis Glassborow
<DAG...@null.net> writes
Sorry, but on this occasion I must support Dan's view. 'Made available
for re-allocation' must be within the context of the C Standard which
addresses C programs, not what happens outside them. free() should be
specified as releasing the memory resources so that the program no
longer owns them and whatever else happens to those resource the program
has no reason to expect to be able to re-allocate them. They are gone,
lost from the program and what now happens to them is beyond the control
of the C Standard.
Accusing all those who make interpretations that you do not like as
being 'nit-pickers' is unhelpful because it allows the genuine
nit-pickers to simply dismiss your arguments. Even those who frequently
pick nits can sometimes hit upon something with more substance. The
argument that everyone knows does not hold up when clearly everyone does
not know.
Tony Finch
>
>I think you are missing the point of the argument that has been going
>on. Basically if I have successfully acquired a block of memory with
>malloc and then freed it, the contention is that a later call of malloc
>for a block of memory not larger than that freed must succeed unless
>some other, internal to the program, has acquired that memory.
That would disallow implementations like GNU and BSD malloc which
use different areas of memory to satisfy allocations of different
sizes. If memory is full then freeing a 256 byte object will not
allow a subsequent 128 byte allocation to succeed.
If you said "the same size as" rather than "not larger than" then
the question remains.
Tony.
--
f.anthony.n.finch <d...@dotat.at>
IRISH SEA SHANNON: NORTHEASTERLY 3 OR 4. FAIR. GOOD.
Tak-Shing Chan
> int main(){
> char * cp = malloc(100);
> free (cp);
> cp = malloc(100); // line A
> return 0;
> }
>
> I believe that we intend to allow line A to fail. A reading of the
> standard (and not a pathological one) indicates that it cannot fail
> because the resource is 'provably' available.
>
> This seems like a genuine DR, and one that can be simply addressed.
If this is the correct view, then the following program will
always return success:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
rename("old", "new");
return !fopen("old", "r") ? EXIT_SUCCESS : EXIT_FAILURE;
}
I fail to see how this is guaranteed by the standard.
Specifically, in between rename and fopen, the standard does not
prohibit the creation of a file named "old" from another
terminal. Likewise for malloc and free, I cannot see any
guarantee in the standard stating that memory allocation is a
zero-sum game.
Tak-Shing
James Kuyper Jr.
>
> Francis Glassborow wrote:
> > Al Grant <alg...@myrealbox.com> writes
> > >If there is really no requirement, then it would be better to word
> > >the definition of free as something like "after a call to free the
> > >pointer and pointers derived from it become invalid", leaving it
> > >unspecified what happens to the space.
>
> I thought we already had such wording. However, without some
> mention of possible later reallocation, people like the nit-pickers
> in this newsgroup might claim that all successful allocations are
> required to return distinct pointers, which is certainly not what
> is desired.
There is wording that makes the pointers invalid. The way in which this
suggestion differs from the current standard, is in it's suggestion that
pointer invalidation should be all that the standard says about the
matter - that it shouldn't say anything about availability of the freed
memory for future allocations.
...
> > Actually some such wording would, among other things, help with making
> > some debugging implementations conforming (I know of at least one that
> > in the most extreme form never recycles memory so as to allow detection
> > of attempts to use dead objects)
> There is nothing nonconforming about that. (Although one wonders
> where all the addresses come from; doesn't it eventually run out?)
Of course. Hence the potential problem, in the eyes of those who
interpret the "available for future allocations" as a guarantee that
malloc() will succeed, in certain circumstances. I disagree with that
interpretation, but I can't say that it's a completely unreasonable one.
Francis Glassborow
Chan <es...@city.ac.uk> writes
But what has that got to do with the view I am giving some credence to.
Nowhere does the Standard suggest you can do that. However I think there
is a not unreasonable reading of the Standard with regards to free that
does seem to suggest that I should not get failure from malloc if I have
released an identically sized block type and have not used malloc in
between. I think we should improve the wording to make it clear that a
programmer has no right to such an expectation, i.e. free can return
memory to the OS for reallocation outside the program.
Francis Glassborow
<kuy...@wizard.net> writes
>Of course. Hence the potential problem, in the eyes of those who
>interpret the "available for future allocations" as a guarantee that
>malloc() will succeed, in certain circumstances. I disagree with that
>interpretation, but I can't say that it's a completely unreasonable one.
Exactly. It is not a deliberately malicious reading of the Standard,
just not one we intended. That is reason for a DR.
Eric Sosman
>
> Yes, I know that. However, wearing a language lawyer hat:
>
> int main(){
> char * cp = malloc(100);
> free (cp);
> cp = malloc(100); // line A
> return 0;
> }
>
> I believe that we intend to allow line A to fail. A reading of the
> standard (and not a pathological one) indicates that it cannot fail
> because the resource is 'provably' available.
>
> This seems like a genuine DR, and one that can be simply addressed.
To prove availability you'd have to prove that no agency
whatsoever could allocate memory between the free() and the
subsequent malloc(). As far as I can tell, the Standard does
not forbid the implementation from allocating dynamic memory
for its own purposes at any moment when it has the whim. For
example, could an implementation raise an asynchronous signal
between the free() and the malloc(), and might the signal-
delivery mechanisms allocate dynamic memory? An implementation
that worked this way might not be wisely constructed, but I
don't think that it would contravene the Standard.
Barry Margolin
Tony Finch <d...@dotat.at> wrote:
>Francis Glassborow <fran...@robinton.demon.co.uk> wrote:
>>
>>I think you are missing the point of the argument that has been going
>>on. Basically if I have successfully acquired a block of memory with
>>malloc and then freed it, the contention is that a later call of malloc
>>for a block of memory not larger than that freed must succeed unless
>>some other, internal to the program, has acquired that memory.
>
>That would disallow implementations like GNU and BSD malloc which
>use different areas of memory to satisfy allocations of different
>sizes. If memory is full then freeing a 256 byte object will not
>allow a subsequent 128 byte allocation to succeed.
>
>If you said "the same size as" rather than "not larger than" then
>the question remains.
I can imagine designs where even that expectation need not be met. For
instance, at the time you free the 256 byte object, the memory manager
could notice that it's low on 128 byte blocks, so it splits it up and
assigns them to the 128 byte pool. Or it could merge it with an adjacent
256 byte space and assign it to the 512 byte pool.
I think the phrase we're debating is just a very poorly worded definition
of "deallocate". Trying to derive precise requirements from this is a
perverse activity -- I don't think the author of that phrase ever intended
it to be taken so literally.
Francis Glassborow
<Eric....@sun.com> writes
> To prove availability you'd have to prove that no agency
>whatsoever could allocate memory between the free() and the
>subsequent malloc(). As far as I can tell, the Standard does
>not forbid the implementation from allocating dynamic memory
>for its own purposes at any moment when it has the whim. For
>example, could an implementation raise an asynchronous signal
>between the free() and the malloc(), and might the signal-
>delivery mechanisms allocate dynamic memory? An implementation
>that worked this way might not be wisely constructed, but I
>don't think that it would contravene the Standard.
Look, I do not want to change what most think is reasonable, I just want
the wording to make it clear that free does NOT make memory available to
the program it simply releases the program's use of memory. That is,
AFAICS is exactly what most of us think happens now.
Francis Glassborow
<bar...@genuity.net> writes
>
>I think the phrase we're debating is just a very poorly worded definition
>of "deallocate". Trying to derive precise requirements from this is a
>perverse activity -- I don't think the author of that phrase ever intended
>it to be taken so literally.
I agree, so let us change the wording to remove the problem (which I
think was meant to be helpful but results in over specification at least
in the eyes of some)
Tak-Shing Chan
> In article <Pine.GSO.4.21.0204091234560.17944-100000@exeter>, Tak-Shing
> Chan <es...@city.ac.uk> writes
>> If this is the correct view, then the following program will
>>always return success:
>>
>> #include <stdio.h>
>> #include <stdlib.h>
>>
>> int main(void)
>> {
>> rename("old", "new");
>> return !fopen("old", "r") ? EXIT_SUCCESS : EXIT_FAILURE;
>> }
>>
>> I fail to see how this is guaranteed by the standard.
>>Specifically, in between rename and fopen, the standard does not
>>prohibit the creation of a file named "old" from another
>>terminal. Likewise for malloc and free, I cannot see any
>>guarantee in the standard stating that memory allocation is a
>>zero-sum game.
>
> But what has that got to do with the view I am giving some credence to.
> Nowhere does the Standard suggest you can do that.
And nowhere does the Standard suggest that malloc will
return success if you have just freed an identically sized block.
It is crystal clear that the ``alternative'' interpretation is
not supported by the Standard at all. According to the Standard,
implementation details such as memory capacity are outside the
scope of the Standard, clause 1 paragraph 2. In theory, you
could shrink the memory size on-the-fly and still have a
conforming implementation, provided that the malloc'ed blocks are
transparently moved (the as-if rule).
Tak-Shing
AMSRL-CI-CN
> int main(){
> char * cp = malloc(100);
> free (cp);
> cp = malloc(100); // line A
> return 0;
> }
> I believe that we intend to allow line A to fail.
Absolutely.
> A reading of the standard (and not a pathological one) indicates
> that it cannot fail because the resource is 'provably' available.
I would like to see such a "proof" exhibited. I think it would
turn up some unwarranted assumptions.
> This seems like a genuine DR, and one that can be simply addressed.
Frankly I don't think we need to allocate scarce committee resources
to this, since the most likely resolution would be that the standard is
clear enough as it stands and attempts to tweak the wording are likely
to make things worse.
Francis Glassborow
Chan <es...@city.ac.uk> writes
> And nowhere does the Standard suggest that malloc will
>return success if you have just freed an identically sized block.
>It is crystal clear that the ``alternative'' interpretation is
>not supported by the Standard at all.
Sorry, but as a native English speaker with over a decade of experience
in computer language standards I disagree. It is NOT crystal clear, the
statement can be worded more clearly.
Al Grant
> In article <U3e*0t...@news.chiark.greenend.org.uk>,
> Tony Finch <d...@dotat.at> wrote:
> >That would disallow implementations like GNU and BSD malloc which
> >use different areas of memory to satisfy allocations of different
> >sizes. If memory is full then freeing a 256 byte object will not
> >allow a subsequent 128 byte allocation to succeed.
> >
> >If you said "the same size as" rather than "not larger than" then
> >the question remains.
>
> I can imagine designs where even that expectation need not be met. For
> instance, at the time you free the 256 byte object, the memory manager
> could notice that it's low on 128 byte blocks, so it splits it up and
> assigns them to the 128 byte pool. Or it could merge it with an adjacent
> 256 byte space and assign it to the 512 byte pool.
The fact that such a heap design may be sensible has no bearing
on an interpretation question.
> I think the phrase we're debating is just a very poorly worded definition
> of "deallocate". Trying to derive precise requirements from this is a
> perverse activity -- I don't think the author of that phrase ever intended
> it to be taken so literally.
How else does one write an international standard?
Francis Glassborow
writes
>> This seems like a genuine DR, and one that can be simply addressed.
>
>Frankly I don't think we need to allocate scarce committee resources
>to this, since the most likely resolution would be that the standard is
>clear enough as it stands and attempts to tweak the wording are likely
>to make things worse.
I disagree that it takes much time to provide alternative wording which
will be clear. The time that can be wasted is interminable arguments as
to why it is already clear enough. From the program side we need not say
anything about releasing resources. All that we need to specify is that
after applying free to a pointer that pointed to memory allocated by a
prior call to malloc, calloc or realloc attempts to use memory based on
the value returned by such call will be undefined.
Douglas A. Gwyn
> I disagree that it takes much time to provide alternative wording which
> will be clear. The time that can be wasted is interminable arguments as
> to why it is already clear enough. From the program side we need not say
> anything about releasing resources. All that we need to specify is that
> after applying free to a pointer that pointed to memory allocated by a
> prior call to malloc, calloc or realloc attempts to use memory based on
> the value returned by such call will be undefined.
We already have that. Further, the range of pointer values within the
storage prior to the free ("based on" the argument pointer value) become
trap representations upon the free. But this doesn't address the issue
that the original wording covers, namely that at some later date the
same range of pointer values might be returned via malloc. There is a
s.c. way to detect that, by saving pointers after conversion to
intptr_t; if those integer values compare equal then perforce the
pointers in some sense "have the same value" and thus the pointed-to
storage "shares the same location". It might be important to realize
that this can happen, e.g. to prevent the application from being
written on the assumption that persistent unique object IDs can be
derived from their locations.
This is exactly the kind of issue that upon readdressing we eventually
discover that the original wording was worked out carefully to strike
the right balance among the many contributing factors.
Francis Glassborow
<DAG...@null.net> writes
>This is exactly the kind of issue that upon readdressing we eventually
>discover that the original wording was worked out carefully to strike
>the right balance among the many contributing factors.
I was not there so I cannot confirm or deny that but evidence suggests
that such care is not always taken and that those close to a problem
sometimes deceive themselves into believing they have covered all the
ground.
James Kuyper Jr.
...
> s.c. way to detect that, by saving pointers after conversion to
> intptr_t; if those integer values compare equal then perforce the
> pointers in some sense "have the same value" and thus the pointed-to
> storage "shares the same location". It might be important to realize
There's nothing in the standard to suggest that.
Douglas A. Gwyn
Sure there is. It's basic logic applied to properties of integers.
If an integer representation at some point isn't a trap rep. then it
never is one (when accessed as the same time, during the lifetime of
the integer object, etc.). Since *all* the information needed to
reconstruct an equivalent pointer value is captured in the intptr_t,
a later conversion of some pointer to intptr_t that results in an
equivalent value must perforce correspond to equivalent pointer
values. The reason I used "quotes" is that the "quoted" phrases
were colloquial for the more exact property. Also note that due to
virtual memory "the same location" has to be taken in context; it
seems to be the same location because the same address is used to
access it.
James Kuyper Jr.
There's actually two different relevant maps: the map between intptr_t
and memory addresses, and the map between memory addresses and physical
memory locations. I don't see any provision of the standard which would
be violated by an implementation which changed either of those mappings,
or even both of them, every time free() was called, so long as the
portions of those maps which changed have no effect on the use of
currently-valid pointer values.
Douglas A. Gwyn
> There's actually two different relevant maps: the map between intptr_t
> and memory addresses, and the map between memory addresses and physical
> memory locations. I don't see any provision of the standard which would
> be violated by an implementation which changed either of those mappings,
> or even both of them, every time free() was called, so long as the
> portions of those maps which changed have no effect on the use of
> currently-valid pointer values.
There is also nothing that prevents different physical pages being
assigned to the process's virtual addresses at different times
during execution, even if malloc/free isn't being used at all.
When we warn that the same region of storage may be returned by later
allocations, we mean storage having the same virtual address,
thus the same pointer value(s).
James Kuyper Jr.
...
> allocations, we mean storage having the same virtual address,
> thus the same pointer value(s).
And the mapping between a given intptr_t and a given virtual address can
change anytime free() is called for the block containing that address.
This isn't the most natural implementation of the conversion, but
neither is it a non-conforming one. Any implementation that uses heavy
pointers (pointers that contain more information than just the memory
address) may even have a good reason to do this.