Memory Leak ??? PROBLEM with MALLOC and FREE ????
Ramesh Natarajan
I have a problem with using malloc and free
I try to allocate memory using malloc and freed it up.
The code worked perfectly well. When I checked with
purify for memory leaks, it showed no leak.
But my problem is, the process's "data" size increases with
every malloc but does not get decreased when freed. I came
to know about this when i monitored the process by using TOP command.
Also when i used "truss" on the executable produced with malloc &
subsequently freeing it ,I found an unusual thing.
MAlloc allocates memory using brk system call, but the corresponding
free does not call any system call to reduce the memory and hence
the data size increases with every malloc.
Is this a bug in malloc and free implementation?
I am using solaris 2.6 and gcc compiler.
Any reason for this behaviour?
Ramesh
Tom St Denis
"Ramesh Natarajan" <nra...@miel.mot.com> wrote in message
news:3AC5EFDC...@miel.mot.com...
Perhaps, or maybe you need to shrink your heap. In windows it's done
automagically (not in MS-DOS though).
Essentially you allocate mem, which in turn allocates heap from the OS. You
then "free()" it and that changes the free mem in your programs heap. Your
libc must not resize the heap though...
I dunno, that might not be it but a good guess.
Tom
George Grant
>But my problem is, the process's "data" size increases with
>every malloc but does not get decreased when freed. I came
>to know about this when i monitored the process by using TOP command.
...
>MAlloc allocates memory using brk system call, but the corresponding
>free does not call any system call to reduce the memory and hence
>the data size increases with every malloc.
...
>Ramesh
Yes, this does happen. It's by design. I'm sure one of the Gurus on here will be
able to tell you why it's "A Good Idea", but I don't really know why.
On AIX, you can use "disclaim" to give the system memory back, I'm not sure what
it would be called on other kinds of UNIX though.
-George
Alain Magloire
> In article <3AC5EFDC...@miel.mot.com>, Ramesh Natarajan says...
>>But my problem is, the process's "data" size increases with
>>every malloc but does not get decreased when freed. I came
>>to know about this when i monitored the process by using TOP command.
> ...
>>MAlloc allocates memory using brk system call, but the corresponding
>>free does not call any system call to reduce the memory and hence
>>the data size increases with every malloc.
> ...
>>Ramesh
> Yes, this does happen. It's by design. I'm sure one of the Gurus on here will be
> able to tell you why it's "A Good Idea", but I don't really know why.
It is costly?
For example an allocator in platform XXX can get new memory by calling
mmap() which return memory in pagesize of 4k.
So if you ask for 20, 30 45, 56 78 12, 14, 13, 12, 12, 24 bytes, the allocator
will slice the 4k. So unless you free the entire memory and let the allocator
coalesce the space it can not unmap() and give the memory back.
In this case it is not about "A Good Idea" but whether it is actually
possible/practicle to do it. Solaris and Linux for big chunks try
to coalesce and give back the memory. See you manual pages for
specific platforms.
> On AIX, you can use "disclaim" to give the system memory back, I'm not sure what
> it would be called on other kinds of UNIX though.
> -George
--
alain
David Schwartz
Chris Thompson wrote:
> >But my problem is, the process's "data" size increases with
> >every malloc but does not get decreased when freed. I came
> >to know about this when i monitored the process by using TOP command.
The process' "data" size is a measure of how much *virtual* memory it's
using. Normally virtual memory is not a scarce resource, so you
shouldn't care.
> >Also when i used "truss" on the executable produced with malloc &
> >subsequently freeing it ,I found an unusual thing.
> >
> >MAlloc allocates memory using brk system call, but the corresponding
> >free does not call any system call to reduce the memory and hence
> >the data size increases with every malloc.
Yes, if malloc uses 'brk', then this will happen.
> >Is this a bug in malloc and free implementation?
No. Again, virtual memory is normally not a scarce resource.
> No. It's characteristic of just about all malloc implementations
> on every Unix system there has ever been.
Not so. There are plenty of malloc implementations that use 'mmap' to
get memory instead of 'sbrk'.
> The memory returned by free() is made available to satisfy subsequent
> malloc() requests, but it is not returned to the kernel.
It was never taken from the kernel so it needn't be returned. The
kernel always controls all physical memory unless you take specific
steps to lock it in place.
DS
Lawrence Kirby
nra...@miel.mot.com "Ramesh Natarajan" writes:
>HI Friends,
>
>I have a problem with using malloc and free
>
>I try to allocate memory using malloc and freed it up.
>The code worked perfectly well. When I checked with
>purify for memory leaks, it showed no leak.
>
>But my problem is, the process's "data" size increases with
>every malloc but does not get decreased when freed. I came
>to know about this when i monitored the process by using TOP command.
This is quite normal. The C language says that freed memory is made
available for further allocation to the program. Some malloc()
implementations can release freed memory to the OS in some circumstances,
some never do. Some peop,e would argue that the latter are more correct
in terms of the C language specification. What should happen is that
freed memory may be allosed again in a later *alloc() call if there
is enough contiguous memory available and it meets any other requirements
of the allocator's strategy.
>Also when i used "truss" on the executable produced with malloc &
>subsequently freeing it ,I found an unusual thing.
>
>MAlloc allocates memory using brk system call, but the corresponding
>free does not call any system call to reduce the memory and hence
>the data size increases with every malloc.
Again that is quite normal.
>Is this a bug in malloc and free implementation?
No. You could argue that there was a bug if something like this
kept on eating more and more memory:
for (i = 0; i < 1000000; i++) {
void *p = malloc(10000);
free(p);
}
>I am using solaris 2.6 and gcc compiler.
>
>
>Any reason for this behaviour?
It is just the way many malloc libraries are implemented, and it is
valid.
--
-----------------------------------------
Lawrence Kirby | fr...@genesis.demon.co.uk
Wilts, England | 7073...@compuserve.com
-----------------------------------------
Dan Pop
>In comp.lang.c George Grant <George...@newsranger.com> wrote:
>> In article <3AC5EFDC...@miel.mot.com>, Ramesh Natarajan says...
>
>>>But my problem is, the process's "data" size increases with
>>>every malloc but does not get decreased when freed. I came
>>>to know about this when i monitored the process by using TOP command.
>> ...
>>>MAlloc allocates memory using brk system call, but the corresponding
>>>free does not call any system call to reduce the memory and hence
>>>the data size increases with every malloc.
>> ...
>>>Ramesh
>
>> Yes, this does happen. It's by design. I'm sure one of the Gurus on here will be
>> able to tell you why it's "A Good Idea", but I don't really know why.
>
>It is costly?
>For example an allocator in platform XXX can get new memory by calling
>mmap() which return memory in pagesize of 4k.
>So if you ask for 20, 30 45, 56 78 12, 14, 13, 12, 12, 24 bytes, the allocator
>will slice the 4k. So unless you free the entire memory and let the allocator
>coalesce the space it can not unmap() and give the memory back.
>
>In this case it is not about "A Good Idea" but whether it is actually
>possible/practicle to do it. Solaris and Linux for big chunks try
>to coalesce and give back the memory. See you manual pages for
>specific platforms.
Giving back the memory to the OS creates a conformance problem.
The free function causes the space pointed to by ptr to be
deallocated, that is, made available for further allocation.
When reading this statement from the C standard, keep in mind the fact
that the C standard describes the behaviour of a single program, not of
the whole system.
If the memory freed by the program is given back to the OS, the OS might
give it to another program, so the program that deallocated it in the
first place may not be able to reuse it, although the above quote seems
to guarantee that it could reuse it.
Dan
--
Dan Pop
CERN, IT Division
Email: Dan...@cern.ch
Mail: CERN - IT, Bat. 31 1-014, CH-1211 Geneve 23, Switzerland
Micah Cowan
In that case, I doubt there is any such thing as a conforming
implementation.
Micah
--
A gentle answer turns away wrath, but a harsh word stirs up anger.
Proverbs 15:1
Logan Shaw
>Giving back the memory to the OS creates a conformance problem.
>
> The free function causes the space pointed to by ptr to be
> deallocated, that is, made available for further allocation.
>
>When reading this statement from the C standard, keep in mind the fact
>that the C standard describes the behaviour of a single program, not of
>the whole system.
>
>If the memory freed by the program is given back to the OS, the OS might
>give it to another program, so the program that deallocated it in the
>first place may not be able to reuse it, although the above quote seems
>to guarantee that it could reuse it.
So, you're saying the standard says that in this code
void *p;
p = malloc (10000);
free (p);
p = malloc (10000);
that the second malloc should never fail under any circumstance.
(Assuming that the code isn't threaded or something.)
While I agree that what you quoted could be read to say that, I'm not
sure I believe that's the only valid interpretation. In fact, I'd say
the text you quoted is just plain ambiguous. It does not specify to
*what* it's made available for further allocation.
In case anyone cares, before virtual memory was commonplace, it was a
pretty common thing for malloc() and free() to work against a memory
pool that was common to all processes. This was true even on systems
that supported multitasking. (At least, it was true on the Amiga.)
- Logan
--
whose? my your his her our their _its_
who's? I'm you're he's she's we're they're _it's_
Dan Pop
>In article <9a64h6$mfr$1...@sunnews.cern.ch>, Dan Pop <Dan...@cern.ch> wrote:
>>Giving back the memory to the OS creates a conformance problem.
>>
>> The free function causes the space pointed to by ptr to be
>> deallocated, that is, made available for further allocation.
>>
>>When reading this statement from the C standard, keep in mind the fact
>>that the C standard describes the behaviour of a single program, not of
>>the whole system.
>>
>>If the memory freed by the program is given back to the OS, the OS might
>>give it to another program, so the program that deallocated it in the
>>first place may not be able to reuse it, although the above quote seems
>>to guarantee that it could reuse it.
>
>So, you're saying the standard says that in this code
>
> void *p;
>
> p = malloc (10000);
> free (p);
> p = malloc (10000);
>
>that the second malloc should never fail under any circumstance.
>(Assuming that the code isn't threaded or something.)
>
>While I agree that what you quoted could be read to say that, I'm not
>sure I believe that's the only valid interpretation. In fact, I'd say
>the text you quoted is just plain ambiguous. It does not specify to
>*what* it's made available for further allocation.
As I've already explained, the standard describes the behaviour of a
C program, not of a whole system. Therefore, there is no ambiguity.
Dan Pop
>Dan...@cern.ch (Dan Pop) writes:
>
>> Giving back the memory to the OS creates a conformance problem.
>>
>> The free function causes the space pointed to by ptr to be
>> deallocated, that is, made available for further allocation.
>>
>> When reading this statement from the C standard, keep in mind the fact
>> that the C standard describes the behaviour of a single program, not of
>> the whole system.
>>
>> If the memory freed by the program is given back to the OS, the OS might
>> give it to another program, so the program that deallocated it in the
>> first place may not be able to reuse it, although the above quote seems
>> to guarantee that it could reuse it.
>
>In that case, I doubt there is any such thing as a conforming
>implementation.
Why? Implementations where nothing is returned back to the OS until
program termination are quite common.
Logan Shaw
>In <9a6hkf$qj3$1...@boomer.cs.utexas.edu> lo...@cs.utexas.edu (Logan Shaw) writes:
>
>>In article <9a64h6$mfr$1...@sunnews.cern.ch>, Dan Pop <Dan...@cern.ch> wrote:
>>>Giving back the memory to the OS creates a conformance problem.
>>>
>>> The free function causes the space pointed to by ptr to be
>>> deallocated, that is, made available for further allocation.
>>>
>>>When reading this statement from the C standard, keep in mind the fact
>>>that the C standard describes the behaviour of a single program, not of
>>>the whole system.
>>While I agree that what you quoted could be read to say that, I'm not
>>sure I believe that's the only valid interpretation. In fact, I'd say
>>the text you quoted is just plain ambiguous. It does not specify to
>>*what* it's made available for further allocation.
>As I've already explained, the standard describes the behaviour of a
>C program, not of a whole system. Therefore, there is no ambiguity.
It's not because I haven't read your argument that I don't agree; it's
because I wasn't convinced by it.
Languages must be implemented on real systems. Languages place certain
constraints on real systems. Therefore, language standards make
statements about not only the language but also about the real systems
on which they might be implemented.
Furthermore, the authors of the standard would (hopefully) have the
good sense to understand that programs written in a language will be
interacting with the rest of the system. To neglect this reality would
not be helpful. (Imagine I am writing a standard describing how pilots
should behave, and I say, "after your airplane has cleared the area,
the runway is made available for further use.")
When reading/interpreting a standard about something that interacts
with other things, I would operate under the assumption that the
standard would make statements about both its main topic and the things
that it would interact with.
Does the C language standard explicitly state that I should do
otherwise? If so, is it unambiguous that it applies to the statement
you originally quoted? If so, then you are right. If not, then I
maintain that it's ambiguous.
David Schwartz
Dan Pop wrote:
> Giving back the memory to the OS creates a conformance problem.
>
> The free function causes the space pointed to by ptr to be
> deallocated, that is, made available for further allocation.
>
> When reading this statement from the C standard, keep in mind the fact
> that the C standard describes the behaviour of a single program, not of
> the whole system.
>
> If the memory freed by the program is given back to the OS, the OS might
> give it to another program, so the program that deallocated it in the
> first place may not be able to reuse it, although the above quote seems
> to guarantee that it could reuse it.
Hahahaha! That's really funny. This is the same as the argument that
any system that implements 'kill -9' isn't conformant since the standard
doesn't say your program can just randomly stop running.
DS
Ramesh Natarajan
This leads to another question:
assume i allocate 4k bytes using malloc
and I free it up.
As per what i can comprehend the os would have allocated a
segment of some predefined k ( say 4k) to the process due to
this malloc operation. Free would have freed that 4k bytes, but
still the process has that region attached to it.
So can I assume my second and subsequent mallocs ( totalling < 4k) will
always not fail?
Ramesh
--
Kaz Kylheku
wrote:
No, because you don't know how much actual space is required for a given
allocation request. If a set of N requests is made whose size_t parameters add
up to 4096 bytes, the allocator will possibly, even quite likely, introduce
overhead which is some function of N. Thus even if the allocator tries to
satisfy these requests by breaking up the previously freed 4 kilobyte zone, it
may run out of space in that zone before all the requests can be met. Of
course there is no requirement that the allocator delve into that zone.
Consider an allocator which keeps free lists in buckets that are indexed by
size. Freeing the four kilobyte object puts it into, say, a bucket of
objects in the range 2 to 4K. A request for 16 bytes will look into
a bucket of smaller chunks, totally ignoring the 4K chunk.
Dan Pop
>When reading/interpreting a standard about something that interacts
>with other things, I would operate under the assumption that the
>standard would make statements about both its main topic and the things
>that it would interact with.
>
>Does the C language standard explicitly state that I should do
>otherwise? If so, is it unambiguous that it applies to the statement
>you originally quoted? If so, then you are right. If not, then I
>maintain that it's ambiguous.
American National Standard Programming Language C specifies the
syntax and semantics of programs written in the C programming
language. It specifies the C program's interactions with the
execution environment via input and output data. It also specifies
restrictions and limits imposed upon conforming implementations of C
language translators.
Do you still see any ambiguities?
Mark McIntyre
>In article <9a6i2p$rh7$1...@sunnews.cern.ch>, Dan Pop <Dan...@cern.ch> wrote:
>>In <9a6hkf$qj3$1...@boomer.cs.utexas.edu> lo...@cs.utexas.edu (Logan Shaw) writes:
>>
>>As I've already explained, the standard describes the behaviour of a
>>C program, not of a whole system. Therefore, there is no ambiguity.
>
>It's not because I haven't read your argument that I don't agree; it's
>because I wasn't convinced by it.
>
>Languages must be implemented on real systems. Languages place certain
>constraints on real systems.
errr ? you have that backwards. Systems impose constraints upon
languages.
>Therefore, language standards make
>statements about not only the language but also about the real systems
>on which they might be implemented.
They limit the number systems which can support a conformant
implementation, yes. But this remark does not follow from your first
one, and thus the word "therefore" is misleading.
>Furthermore, the authors of the standard would (hopefully) have the
>good sense to understand that programs written in a language will be
>interacting with the rest of the system. To neglect this reality would
>not be helpful.
Indeed, and hte C standard defines a number of interactions.
> (Imagine I am writing a standard describing how pilots
>should behave, and I say, "after your airplane has cleared the area,
>the runway is made available for further use.")
But that would be part of the implementation of "runway" not of
"pilot". Once he's left the runway the pilot cares not whats done with
it. Hence your words are informative only, and an indication of what
one particular implementation might do. Another might roll up the
runway and mail it to your destination, so that you could land.
--
Mark McIntyre
CLC FAQ <http://www.eskimo.com/~scs/C-faq/top.html>
Philip Brown
>...
>As per what i can comprehend the os would have allocated a
>segment of some predefined k ( say 4k) to the process due to
>this malloc operation. Free would have freed that 4k bytes, but
>still the process has that region attached to it.
>
>So can I assume my second and subsequent mallocs ( totalling < 4k) will
>always not fail?
it is always a bug to *assume* that malloc will succeed.
But it is quite LIKELY that it will succeed. Which is why some programs
that know they need large chunks of memory in the future, will start up by
mallocing some large amount of mem which then they immediately "free".
--
[Trim the no-bots from my address to reply to me by email!]
[ Do NOT email-CC me on posts. Pick one or the other.]
S.1618 http://thomas.loc.gov/cgi-bin/bdquery/z?d105:SN01618:@@@D
The word of the day is mispergitude
Logan Shaw
>In <9a807b$sfo$1...@boomer.cs.utexas.edu> lo...@cs.utexas.edu (Logan Shaw) writes:
>>Does the C language standard explicitly state that I should do
>>otherwise? If so, is it unambiguous that it applies to the statement
>>you originally quoted? If so, then you are right. If not, then I
>>maintain that it's ambiguous.
>
> American National Standard Programming Language C specifies the
> syntax and semantics of programs written in the C programming
> language. It specifies the C program's interactions with the
> execution environment via input and output data. It also specifies
> restrictions and limits imposed upon conforming implementations of C
> language translators.
>
>Do you still see any ambiguities?
Yes, definitely. But I don't think we're getting anywhere, so this
will be my last post in this thread.
Dan Pop
This is downright idiotic. The standard does not guarantee that a
program's execution cannot be interrupted by external factors, like
cutting the power or pressing on the Reset button. kill -9 is in the
same category.
OTOH, the standard gives no licence to external factors to "eat" from
the program's malloc arena.
Phil Tregoning
Dan Pop <Dan...@cern.ch> wrote in article
<9a6i2p$rh7$1...@sunnews.cern.ch>...
I'm not sure I buy that. Consider the following code:
void *p;
FILE *fp;
p = malloc(10000);
if (p) {
free(p);
fp = fopen("AFILE","r");
p = malloc(10000);
assert(p != NULL);
}
Is the assert allowed to fail? I would say yes, because the
fopen() function is allowed to call malloc(). Even if there were
10000 bytes guaranteed available after the free(), there might
not be after the fopen().
Surely the same argument can equally be applied to malloc().
Before the user requested memory is allocated, is malloc()
allowed to allocate memory for its own use (which remains
allocated until program termination)? This could perhaps be
part of a memory checking extension where malloc() keeps track
(in allocated memory) of when it was called, so malloc() would
itself on occasion call realloc().
Where is the guarantee that malloc(), free(), or any other
library function, behave as if they do /not/ call
malloc()/realloc()?
Phil T
Chris Thompson
David Schwartz <dav...@webmaster.com> wrote:
>
>Chris Thompson wrote:
[...]
>> No. It's characteristic of just about all malloc implementations
>> on every Unix system there has ever been.
>
> Not so. There are plenty of malloc implementations that use 'mmap' to
>get memory instead of 'sbrk'.
All right, that was a bit OTT. Perhaps I should have said "this is the
way that malloc has traditionally been implemented in Unix systems".
Chris Thompson
Email: cet1 [at] cam.ac.uk
Dan Pop
>I'm not sure I buy that. Consider the following code:
>
> void *p;
> FILE *fp;
> p = malloc(10000);
> if (p) {
> free(p);
> fp = fopen("AFILE","r");
> p = malloc(10000);
> assert(p != NULL);
> }
>
>Is the assert allowed to fail?
Yup. The standard explicitly mention buffer allocation and deallocation
issues.
>Surely the same argument can equally be applied to malloc().
Only if you show us some wording from the standard supporting this
assertion.
>Before the user requested memory is allocated, is malloc()
>allowed to allocate memory for its own use (which remains
>allocated until program termination)? This could perhaps be
>part of a memory checking extension where malloc() keeps track
>(in allocated memory) of when it was called, so malloc() would
>itself on occasion call realloc().
>
>Where is the guarantee that malloc(), free(), or any other
>library function, behave as if they do /not/ call
>malloc()/realloc()?
They can do whatever they want, provided the program can reallocate
a chunk of memory it has freed. If it cannot, then the memory freed
was not made available for further allocation. Which is what the
standard requires.
Dan Mercer
Ramesh Natarajan <nra...@miel.mot.com> writes:
> HI Friends,
>
> I have a problem with using malloc and free
>
> I try to allocate memory using malloc and freed it up.
> The code worked perfectly well. When I checked with
> purify for memory leaks, it showed no leak.
>
> But my problem is, the process's "data" size increases with
> every malloc but does not get decreased when freed. I came
> to know about this when i monitored the process by using TOP command.
>
> Also when i used "truss" on the executable produced with malloc &
> subsequently freeing it ,I found an unusual thing.
>
> MAlloc allocates memory using brk system call, but the corresponding
> free does not call any system call to reduce the memory and hence
> the data size increases with every malloc.
This is incorrect. On Unix systems, malloc allocates memory
from the heap. If the heap is exhausted, sbrk will be used to
try to grow it. There are limits to the heap size set in the
kernel (maxdsiz) - on HP-UX the default is 64 meg. If you exceed
that size, sbrk will fail. It may also fail if you exhaust
virtual memory.
A free does not "return" memory to the OS. It merely deallocates
the storage on the heap. Unix systems do not typically shrink the heap
size on free. To do so would require additional bookkeeping and
it would typically offer little in the way of performance. If you
needed that space once, typically you will need it again.
If you malloc,free,malloc the same size, with no intervening
space allocations, sbrk will not be called on the second malloc.
Iff your heap size continues to grow despite freeing data, then
it's probable you are not freeing what you think you are freeing.
--
Dan Mercer
dame...@mmm.com
>
> Is this a bug in malloc and free implementation?
>
> I am using solaris 2.6 and gcc compiler.
>
>
> Any reason for this behaviour?
>
>
>
> Ramesh
Opinions expressed herein are my own and may not represent those of my employer.
R Pradeep Chandran
:In <01c0bc39$c500dde0$4b53...@ptregoni.dev.esoc.esa.de>
: "Phil Tregoning" <ptre...@esoc.esa.de> writes:
<snip>
:>
:>Where is the guarantee that malloc(), free(), or any other
:>library function, behave as if they do /not/ call
:>malloc()/realloc()?
:
:They can do whatever they want, provided the program can reallocate
:a chunk of memory it has freed. If it cannot, then the memory freed
:was not made available for further allocation. Which is what the
:standard requires.
:
:Dan
I have some doubts about the requirements.
Consider the following.
<snippet>
void *pOne;
void *pTwo;
pOne = malloc( 1000 );
if( pOne )
{
free( pOne );
pOne = NULL;
/*
1) At this point is it required that any allocation upto 1000
should succeed?
2) Also, If an allocation of N bytes, where N < 1000, is made,
is there any guarantee about a subsequent allocation of M
bytes where ( M + N ) < 1000 ?
*/
}
pOne = malloc( 1000 );
if( pOne )
{
pTwo = malloc( 1000 );
if( pTwo )
{
free( pOne );
pOne = NULL;
free( pTwo );
pTwo = NULL;
/*
3) At this point, is it required that,
a) any allocation upto 2000 should succeed?
OR
b) two allocations, each upto 1000 should succeed?
*/
}
}
</snippet>
From what I understand, the answers are,
1) Yes, It is required.
2) No, Subsequent allocations are not guranteed to succeed.
3) (b) is correct.
Am I right ?
Have a nice day,
Pradeep
--
R Pradeep Chandran pradeep DOT chandran AT sisl.co.in
All opinions are mine and do not represent those of my employer.
Phil Tregoning
Dan Pop <Dan...@cern.ch> wrote in article
<9aciql$raj$1...@sunnews.cern.ch>...
> In <01c0bc39$c500dde0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
<ptre...@esoc.esa.de> writes:
>
> >I'm not sure I buy that. Consider the following code:
> >
> > void *p;
> > FILE *fp;
> > p = malloc(10000);
> > if (p) {
> > free(p);
> > fp = fopen("AFILE","r");
> > p = malloc(10000);
> > assert(p != NULL);
> > }
> >
> >Is the assert allowed to fail?
>
> Yup. The standard explicitly mention buffer allocation and deallocation
> issues.
Unless I have missed something, the only file operations
that the C standard says can allocate memory are setbuf()
and setvbuf() (which would then be deallocated by fclose()).
The description of fopen() does not say it has a license
to allocate any memory.
Where does it say fopen() can allocate memory (C&V)?
> >Surely the same argument can equally be applied to malloc().
>
> Only if you show us some wording from the standard supporting this
> assertion.
>
> >Before the user requested memory is allocated, is malloc()
> >allowed to allocate memory for its own use (which remains
> >allocated until program termination)? This could perhaps be
> >part of a memory checking extension where malloc() keeps track
> >(in allocated memory) of when it was called, so malloc() would
> >itself on occasion call realloc().
> >
> >Where is the guarantee that malloc(), free(), or any other
> >library function, behave as if they do /not/ call
> >malloc()/realloc()?
>
> They can do whatever they want, provided the program can reallocate
> a chunk of memory it has freed. If it cannot, then the memory freed
> was not made available for further allocation. Which is what the
> standard requires.
>
But what the standard does /not/ say is that freed memory is
"is made available for further allocation _by_malloc()_".
Freed memory is therefore available for allocation by anything
that is allowed to allocate memory (not just malloc). This
could include fopen(), setvbuf(), automatic variables, stack
frames, atexit() registrations and other things I have not
thought of.
So although in principle the memory should be available, in
practice a program cannot know when it allocates memory, so
cannot rely on freed memory being available for malloc.
Some further examples:
void *a, *b = malloc(10000);
if (b) {
free(b);
a = malloc(5000);
b = malloc(5000);
assert(a != NULL && b != NULL);
}
I believe typical implementations of malloc() actually
allocate a bit more memory than requested for housekeeping
purposes. So here the first malloc() might actually allocate
10004 bytes, and the second two together 10008 bytes. A more
extreme example would be:
int i;
void *a[10000], *b = malloc(10000);
if (b) {
free(b);
for (i = 0 ; i < 10000 ; i++) {
a[i] = malloc(1);
assert(a[i] != NULL);
}
}
If these asserts fail, then the 10000 bytes that you say should
have been guaranteed available for allocation by malloc weren't.
Your argument seems to exclude implementations of malloc that ever
attempt to allocate even a single byte more than was asked for.
All in all, I believe that it is not possible to say from the
C standard exactly what operations can allocate memory (and how
much). Because of this, the promise that freed memory is
"available for allocation" is worthless, as a program cannot
control (or predict) when memory allocations occur.
Phil T
Dan Pop
>
>Dan Pop <Dan...@cern.ch> wrote in article
><9aciql$raj$1...@sunnews.cern.ch>...
>> In <01c0bc39$c500dde0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
><ptre...@esoc.esa.de> writes:
>>
>> >I'm not sure I buy that. Consider the following code:
>> >
>> > void *p;
>> > FILE *fp;
>> > p = malloc(10000);
>> > if (p) {
>> > free(p);
>> > fp = fopen("AFILE","r");
>> > p = malloc(10000);
>> > assert(p != NULL);
>> > }
>> >
>> >Is the assert allowed to fail?
>>
>> Yup. The standard explicitly mention buffer allocation and deallocation
>> issues.
>
>Unless I have missed something, the only file operations
>that the C standard says can allocate memory are setbuf()
>and setvbuf() (which would then be deallocated by fclose()).
>The description of fopen() does not say it has a license
>to allocate any memory.
>
>Where does it say fopen() can allocate memory (C&V)?
In the description of fclose:
4.9.5.1 The fclose function
Synopsis
#include <stdio.h>
int fclose(FILE *stream);
Description
The fclose function causes the stream pointed to by stream to be
flushed and the associated file to be closed. Any unwritten buffered
data for the stream are delivered to the host environment to be
written to the file; any unread buffered data are discarded. The
stream is disassociated from the file. If the associated buffer was
automatically allocated, it is deallocated.
>But what the standard does /not/ say is that freed memory is
>"is made available for further allocation _by_malloc()_".
>Freed memory is therefore available for allocation by anything
>that is allowed to allocate memory (not just malloc). This
>could include fopen(), setvbuf(), automatic variables, stack
>frames, atexit() registrations and other things I have not
>thought of.
But all these things still belong to the program itself.
>So although in principle the memory should be available, in
>practice a program cannot know when it allocates memory, so
>cannot rely on freed memory being available for malloc.
It depends on the program. The following strictly conforming program
can tell if an external entity has been "eating" the memory it made
available for further allocation:
#include <stdlib.h>
#define SIZE 1000000
int main()
{
char *p = malloc(SIZE);
if (p == NULL) return 0;
while (1)
{
free(p);
/* burn cpu cycles for some time */
p = malloc(SIZE);
if (p == NULL) {
puts("This implementation is non-conforming");
return EXIT_FAILURE;
}
}
}
Suppose that we have a second program that tries to allocate as much
memory as possible, in 100k chunks and keeps trying even after the first
malloc call has failed. If free() returns the memory back to the OS,
this program will eventually "hijack" the memory freed by the first
program, which will be able to detect this fact.
Phil Tregoning
Dan Pop <Dan...@cern.ch> wrote in article
<9afh10$cru$1...@sunnews.cern.ch>...
> In <01c0bd02$ea2436c0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
<ptre...@esoc.esa.de> writes:
> >
> >Where does it say fopen() can allocate memory (C&V)?
>
> In the description of fclose:
>
> 4.9.5.1 The fclose function
>
> Synopsis
>
> #include <stdio.h>
> int fclose(FILE *stream);
>
> Description
>
> The fclose function causes the stream pointed to by stream to be
> flushed and the associated file to be closed. Any unwritten buffered
> data for the stream are delivered to the host environment to be
> written to the file; any unread buffered data are discarded. The
> stream is disassociated from the file. If the associated buffer was
> automatically allocated, it is deallocated.
Ah-ha. I assume that was from C89 or the draft C99. The final
version of C99 is different. There, 7.9.15.1.2 says:
Description
A successful call to the fclose function causes the stream
pointed to by stream to be flushed and the associated file
to be closed. Any unwritten buffered data for the stream
are delivered to the host environment to be written to the
file; any unread buffered data are discarded. Whether or
not the call succeeds, the stream is disassociated from
the file and any buffer set by the setbuf or setvbuf
function is disassociated from the stream (and deallocated
if it was automatically allocated).
So I read C99 as saying fclose() only deallocates memory that
was automatically allocated by calling set(v)buf, not fopen().
> >But what the standard does /not/ say is that freed memory is
> >"is made available for further allocation _by_malloc()_".
> >Freed memory is therefore available for allocation by anything
> >that is allowed to allocate memory (not just malloc). This
> >could include fopen(), setvbuf(), automatic variables, stack
> >frames, atexit() registrations and other things I have not
> >thought of.
>
> But all these things still belong to the program itself.
Your argument relies on certain operations being guaranteed
not to allocate memory. You should therefore be able to
split all library functions into two categories - those that
are allowed to take memory from the malloc pool, and those
that aren't. The only functions that I could find that the
standard explicitly says allocate memory are:
malloc()
calloc()
realloc()
setbuf() (possibly)
setvbuf()
Functions that I don't think the standard says can allocate
memory, but I consider might on a reasonable implementation
are:
fopen() (unless you find wording to the contrary)
atexit()
asctime() and others returning pointers to static data.
I included functions like asctime() that return pointers
to static memory because they could conceivably reduce
the memory available to malloc(). Some implementations
keep standard library functions in shared memory, and
have a copy-on-write strategy for static writeable data.
If this copy reduces the memory available to future
mallocs() (for example, using up virtual memory or page
file space), it would presumably be non-conforming.
Mind you, such systems will also typically implement
allocate-on-write for malloced data as well. This can allow
hugh sparse arrays, and can spread the allocation cost over
more code, but is also non-conforming.
> >So although in principle the memory should be available, in
> >practice a program cannot know when it allocates memory, so
> >cannot rely on freed memory being available for malloc.
>
> It depends on the program. The following strictly conforming program
> can tell if an external entity has been "eating" the memory it made
> available for further allocation:
>
[Snip program to continually allocate and free 1000000 bytes]
>
> Suppose that we have a second program that tries to allocate as much
> memory as possible, in 100k chunks and keeps trying even after the first
> malloc call has failed. If free() returns the memory back to the OS,
> this program will eventually "hijack" the memory freed by the first
> program, which will be able to detect this fact.
>
Just to be clear, I assume that you do /not/ recommend
allocating and freeing a big chunk up front as a technique
for avoiding the necessity of checking that later
malloc calls succeed.
This is one of those areas that I would prefer a compiler
that does what is most efficient and natural for that
particular platform/OS (even if it is technically
non-conforming), rather than have the compiler writers
stick to the letter of the standard.
Phil T
Dan Pop
>Dan Pop <Dan...@cern.ch> wrote in article
><9afh10$cru$1...@sunnews.cern.ch>...
>> In <01c0bd02$ea2436c0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
><ptre...@esoc.esa.de> writes:
>> >
>> >Where does it say fopen() can allocate memory (C&V)?
>>
>> In the description of fclose:
>>
>> 4.9.5.1 The fclose function
>>
>> Synopsis
>>
>> #include <stdio.h>
>> int fclose(FILE *stream);
>>
>> Description
>>
>> The fclose function causes the stream pointed to by stream to be
>> flushed and the associated file to be closed. Any unwritten buffered
>> data for the stream are delivered to the host environment to be
>> written to the file; any unread buffered data are discarded. The
>> stream is disassociated from the file. If the associated buffer was
>> automatically allocated, it is deallocated.
>
>Ah-ha. I assume that was from C89 or the draft C99.
C89, the standard actually implemented.
>The final
>version of C99 is different. There, 7.9.15.1.2 says:
>
> Description
>
> A successful call to the fclose function causes the stream
> pointed to by stream to be flushed and the associated file
> to be closed. Any unwritten buffered data for the stream
> are delivered to the host environment to be written to the
> file; any unread buffered data are discarded. Whether or
> not the call succeeds, the stream is disassociated from
> the file and any buffer set by the setbuf or setvbuf
> function is disassociated from the stream (and deallocated
> if it was automatically allocated).
>
>So I read C99 as saying fclose() only deallocates memory that
>was automatically allocated by calling set(v)buf, not fopen().
So, the C99 wording is poorer than that of C89, because it does not
specify what happens with the memory allocated for the buffer by fopen.
>> >But what the standard does /not/ say is that freed memory is
>> >"is made available for further allocation _by_malloc()_".
>> >Freed memory is therefore available for allocation by anything
>> >that is allowed to allocate memory (not just malloc). This
>> >could include fopen(), setvbuf(), automatic variables, stack
>> >frames, atexit() registrations and other things I have not
>> >thought of.
>>
>> But all these things still belong to the program itself.
>
>Your argument relies on certain operations being guaranteed
>not to allocate memory. You should therefore be able to
>split all library functions into two categories - those that
>are allowed to take memory from the malloc pool, and those
>that aren't. The only functions that I could find that the
>standard explicitly says allocate memory are:
>
> malloc()
> calloc()
> realloc()
> setbuf() (possibly)
> setvbuf()
>
>Functions that I don't think the standard says can allocate
>memory, but I consider might on a reasonable implementation
>are:
>
> fopen() (unless you find wording to the contrary)
I have already posted it. I don't have a C99 implementation, and I don't
know of any C99-based Unix platform, so the quote I've posted is relevant.
> atexit()
> asctime() and others returning pointers to static data.
>
>I included functions like asctime() that return pointers
>to static memory because they could conceivably reduce
>the memory available to malloc().
Bad logic. If asctime uses malloc, it may fail to allocate its buffer if
the malloc call fails. But the asctime function *cannot* fail!
The same argument applies to atexit, which may not be able to support the
registration of a min 32 functions if its implementation relies on
malloc.
>> >So although in principle the memory should be available, in
>> >practice a program cannot know when it allocates memory, so
>> >cannot rely on freed memory being available for malloc.
>>
>> It depends on the program. The following strictly conforming program
>> can tell if an external entity has been "eating" the memory it made
>> available for further allocation:
>
>[Snip program to continually allocate and free 1000000 bytes]
>
>> Suppose that we have a second program that tries to allocate as much
>> memory as possible, in 100k chunks and keeps trying even after the first
>> malloc call has failed. If free() returns the memory back to the OS,
>> this program will eventually "hijack" the memory freed by the first
>> program, which will be able to detect this fact.
>
>Just to be clear, I assume that you do /not/ recommend
>allocating and freeing a big chunk up front as a technique
>for avoiding the necessity of checking that later
>malloc calls succeed.
I did not recommend anything at all. I merely presented a strictly
conforming program that can detect a non-conforming malloc/free
implementation. Which is the actual topic of this discussion.
>This is one of those areas that I would prefer a compiler
>that does what is most efficient and natural for that
>particular platform/OS (even if it is technically
>non-conforming), rather than have the compiler writers
>stick to the letter of the standard.
This is entirely irrelevant to this discussion. But, since you mentioned
it, I also prefer certain non-conforming behaviours of a C implementation,
like lazy swap space allocation, even if it removes the guarantee that
*any* form of memory allocation actually works.
Phil Tregoning
[Snip lots of discussion between me and Dan Pop
about freed memory being available for future allocation]
My stance boiled down to the question:
Are standard library functions (and arbitrary
expressions in general) that the standard does
not explicitly say can allocate memory allowed
to take memory from the malloc pool?
I had assumed yes, but you say no (I think - forgive me
for putting words in your mouth).
I cannot find anything definitive in the standard one way
or the other, so I will take your word for it.
Phil T
Dan Pop
If a standard library function or an arbitrary expression in general
needs memory from the malloc pool, but none is available, what happens?
Scott MacKay
behaviour' and gone OT to the fact you were looking probably looking for
help to get it to not chew all your memory away...if you were looking
for deep philosophy I guess you could continue to read the long
thread...
You can always check sunsolve.sun.com for Solaris 2.6 and any malloc
issues. I think I saw a patch ID 105210 (public patch access) had some
memory type bugs fixed, but did not check the details.
You could also check out http://gcc.gnu.org/ and see if you have the
most up to date gcc.
In general, I would believe that your app should not keep growing in
size. It might if you keep allocating larger and larger blocks of
memory and never smaller ones (I don't know if the end memory management
scheme will 'combine' allocated chunks and make them into 1 larger one
if needed [Doubt it]. Don't know and quite frankly don't care).
I would:
1) Check the above for patches around mememory management
2) Check your allocation/deallocation sizes. Always bigger?
Sorry for the comparitively terse suggestions; It's Friday and I am not
about to look up any C89 C99 or other standard before I go home :)
-Scott
Ramesh Natarajan wrote:
>
> HI Friends,
>
> I have a problem with using malloc and free
>
> I try to allocate memory using malloc and freed it up.
> The code worked perfectly well. When I checked with
> purify for memory leaks, it showed no leak.
>
> But my problem is, the process's "data" size increases with
> every malloc but does not get decreased when freed. I came
> to know about this when i monitored the process by using TOP command.
>
> Also when i used "truss" on the executable produced with malloc &
> subsequently freeing it ,I found an unusual thing.
>
> MAlloc allocates memory using brk system call, but the corresponding
> free does not call any system call to reduce the memory and hence
> the data size increases with every malloc.
>
> Is this a bug in malloc and free implementation?
>
> I am using solaris 2.6 and gcc compiler.
>
> Any reason for this behaviour?
>
> Ramesh
--
-----
"This posting reflects the views of the poster and does not reflect the
views of the company."
Gergo Barany
> In <01c0be99$3426b6c0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning" <ptre...@esoc.esa.de> writes:
> > Are standard library functions (and arbitrary
> > expressions in general) that the standard does
> > not explicitly say can allocate memory allowed
> > to take memory from the malloc pool?
>
> needs memory from the malloc pool, but none is available, what happens?
It chooses a slower algorithm that doesn't need dynamically
allocated storage.
Gergo
--
PL/I -- "the fatal disease" -- belongs more to the problem set than
to the solution set.
-- Edsger W. Dijkstra, SIGPLAN Notices, Volume 17, Number 5
Dan Pop
>Dan Pop <Dan...@cern.ch> wrote:
>> In <01c0be99$3426b6c0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning" <ptre...@esoc.esa.de> writes:
>> > Are standard library functions (and arbitrary
>> > expressions in general) that the standard does
>> > not explicitly say can allocate memory allowed
>> > to take memory from the malloc pool?
>>
>> If a standard library function or an arbitrary expression in general
>> needs memory from the malloc pool, but none is available, what happens?
>
>It chooses a slower algorithm that doesn't need dynamically
>allocated storage.
You missed Phil's point. He was talking about functions that allocate
memory dynamically *without* releasing it before returning. Like fopen
and setvbuf *may* do.
fopen and setvbuf are special cases, however, because:
1. fclose is required to deallocate the memory allocated by them.
2. They are both allowed to fail if they can't allocate the memory.
Phil Tregoning
Dan Pop <Dan...@cern.ch> wrote in article
<9akoj0$3ff$1...@sunnews.cern.ch>...
> In <01c0be99$3426b6c0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
<ptre...@esoc.esa.de> writes:
>
>
> >[Snip lots of discussion between me and Dan Pop
> >about freed memory being available for future allocation]
> >
> >My stance boiled down to the question:
> >
> > Are standard library functions (and arbitrary
> > expressions in general) that the standard does
> > not explicitly say can allocate memory allowed
> > to take memory from the malloc pool?
> >
> >I had assumed yes, but you say no (I think - forgive me
> >for putting words in your mouth).
> >
> >I cannot find anything definitive in the standard one way
> >or the other, so I will take your word for it.
>
> If a standard library function or an arbitrary expression in general
> needs memory from the malloc pool, but none is available, what happens?
>
The same thing that happens if there are not enough
resources to call the function in the first place.
If you are lucky you might get some sort of signal
raised, or just an abnormal termination.
Phil T
Dan Pop
But we are talking here about functions that are NOT allowed to fail in
a conforming implementation. Such functions use statically allocated
memory to be sure that they have the memory resources needed available
when they are called.
E.g. if the strictly conforming program crashes when I make my tenth
atexit call, the implementation is non-conforming, period. OTOH, if
atexit starts calling malloc *after* the first 32 invocations and my
program crashes at its 33rd atexit call, everything is OK, because my
program is no longer strictly conforming.
Phil Tregoning
Dan Pop <Dan...@cern.ch> wrote in article
<9asfip$ipi$1...@sunnews.cern.ch>...
> In <01c0c0e5$d5f01a20$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
<ptre...@esoc.esa.de> writes:
>
>
> >
> >Dan Pop <Dan...@cern.ch> wrote in article
> ><9akoj0$3ff$1...@sunnews.cern.ch>...
> >>
> >> If a standard library function or an arbitrary expression in general
> >> needs memory from the malloc pool, but none is available, what
happens?
> >
> >The same thing that happens if there are not enough
> >resources to call the function in the first place.
> >If you are lucky you might get some sort of signal
> >raised, or just an abnormal termination.
>
> But we are talking here about functions that are NOT allowed to fail in
> a conforming implementation. Such functions use statically allocated
> memory to be sure that they have the memory resources needed available
> when they are called.
>
In real life, any function can fail. A simple untested case
would be:
#include <stdio.h>
#include <time.h>
void foo(unsigned long n)
{
time_t tm;
if (n != 0 && (tm = time()) != -1) {
ctime(&tm);
foo(n-1);
}
}
int main(void)
{
foo(-1);
printf("Bye.\n");
return 0;
}
This code has a fair chance of failing on the ctime() call
(assuming no optimization). I expected something in the
Environment Limits section of the standard about a minimum
guaranteed function call depth, but couldn't find it.
If my implementation doesn't print "Bye." is it broken?
Phil T
Dan Pop
>In real life, any function can fail. A simple untested case
>would be:
>
> #include <stdio.h>
> #include <time.h>
>
> void foo(unsigned long n)
> {
> time_t tm;
> if (n != 0 && (tm = time()) != -1) {
> ctime(&tm);
> foo(n-1);
> }
> }
>
> int main(void)
> {
> foo(-1);
> printf("Bye.\n");
> return 0;
> }
>
>This code has a fair chance of failing on the ctime() call
>(assuming no optimization). I expected something in the
>Environment Limits section of the standard about a minimum
>guaranteed function call depth, but couldn't find it.
Because it is supposed to return the address of a statically allocated
buffer (of 26 characters) that contains the string. Check the
description of the asctime function. A new ctime call will overwrite
the string produced by the previous call.
>If my implementation doesn't print "Bye." is it broken?
Nope. It is the quasi-infinite recursion that may kill your program,
unless the compiler optimises it away. ctime() is a red herring in your
example program.
Phil Tregoning
Dan Pop <Dan...@cern.ch> wrote in article
<9atd2b$3dr$1...@sunnews.cern.ch>...
> In <01c0c109$ea16c0c0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
<ptre...@esoc.esa.de> writes:
>
> >In real life, any function can fail. A simple untested case
> >would be:
> >
[Snip my deeply recursive function that calls ctime()]
> >
> >This code has a fair chance of failing on the ctime() call
> >(assuming no optimization). I expected something in the
> >Environment Limits section of the standard about a minimum
> >guaranteed function call depth, but couldn't find it.
>
> Because it is supposed to return the address of a statically allocated
> buffer (of 26 characters) that contains the string. Check the
> description of the asctime function. A new ctime call will overwrite
> the string produced by the previous call.
Some implementations may in some sense allocate that on
the first call. I believe mine does, because ctime is in
shared-read, copy-on-write memory. That copy-on-write can
fail if there is no space to copy to.
> >If my implementation doesn't print "Bye." is it broken?
>
> Nope. It is the quasi-infinite recursion that may kill your program,
> unless the compiler optimises it away. ctime() is a red herring in your
> example program.
>
I was trying to demonstrate that any given function call
is allowed to fail if there are not enough resources.
Whether those resources are enough space for another
stack frame (assuming you have a stack of course), or
enough memory in the malloc pool is irrelevant. All
functions can fail, and a function that somehow
dynamically allocates memory is no different from one
that doesn't.
If this is correct, then having standard functions
that allocate memory (and could fail because of it)
is still conforming.
It's just that line in the description of free that
provides a way of telling if functions are mallocing
memory (If functions are allocating memory from
somewhere else, then there is no way to tell). Hence
my question about what functions are allowed to take
memory /from the malloc pool/.
Phil T
Micah Cowan
> > If a standard library function or an arbitrary expression in general
> > needs memory from the malloc pool, but none is available, what happens?
> >
>
> The same thing that happens if there are not enough
> resources to call the function in the first place.
> If you are lucky you might get some sort of signal
> raised, or just an abnormal termination.
>
> Phil T
>
Which, as Dan is indirectly (as usual) pointing out, voilates the
C Standard. Which is the governing document for the newsgroup from
which I'm posting (comp.lang.c).
Micah
--
A gentle answer turns away wrath, but a harsh word stirs up anger.
Proverbs 15:1
Dan Pop
>
>Dan Pop <Dan...@cern.ch> wrote in article
><9atd2b$3dr$1...@sunnews.cern.ch>...
>> In <01c0c109$ea16c0c0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
><ptre...@esoc.esa.de> writes:
>>
>> >In real life, any function can fail. A simple untested case
>> >would be:
>> >
>
>[Snip my deeply recursive function that calls ctime()]
>
>> >
>> >This code has a fair chance of failing on the ctime() call
>> >(assuming no optimization). I expected something in the
>> >Environment Limits section of the standard about a minimum
>> >guaranteed function call depth, but couldn't find it.
>>
>> Because it is supposed to return the address of a statically allocated
>> buffer (of 26 characters) that contains the string. Check the
>> description of the asctime function. A new ctime call will overwrite
>> the string produced by the previous call.
>
>Some implementations may in some sense allocate that on
>the first call. I believe mine does, because ctime is in
>shared-read, copy-on-write memory. That copy-on-write can
>fail if there is no space to copy to.
Then your implementation is non-conforming (unless the program fails
before starting its execution). Once the function main has been
called, all the statically allocated memory is supposed to be available.
>> >If my implementation doesn't print "Bye." is it broken?
>>
>> Nope. It is the quasi-infinite recursion that may kill your program,
>> unless the compiler optimises it away. ctime() is a red herring in your
>> example program.
>>
>
>I was trying to demonstrate that any given function call
>is allowed to fail if there are not enough resources.
All you have demonstrated is that any *program* is allowed to fail if
there are not enough resources.
OTOH, once my program has allocated all the resources it needs, it has
no licence to fail afterwards.
>If this is correct, then having standard functions
>that allocate memory (and could fail because of it)
>is still conforming.
ONLY if the standard *explicitly* gives these functions a licence to
fail. Which is not the case with any of the examples you have presented
in this discussion.
>It's just that line in the description of free that
>provides a way of telling if functions are mallocing
>memory (If functions are allocating memory from
>somewhere else, then there is no way to tell). Hence
>my question about what functions are allowed to take
>memory /from the malloc pool/.
Those which can fail for unspecified reasons.
Brian Inglis
Kylheku) wrote:
>On Mon, 02 Apr 2001 09:51:08 +0530, Ramesh Natarajan <nra...@miel.mot.com>
>wrote:
>>Thanks for the reply.
>>
>>This leads to another question:
>>
>>assume i allocate 4k bytes using malloc
>>and I free it up.
>>
>>As per what i can comprehend the os would have allocated a
>>segment of some predefined k ( say 4k) to the process due to
>>this malloc operation. Free would have freed that 4k bytes, but
>>still the process has that region attached to it.
>>
>>So can I assume my second and subsequent mallocs ( totalling < 4k) will
>>always not fail?
>
>No, because you don't know how much actual space is required for a given
>allocation request. If a set of N requests is made whose size_t parameters add
>up to 4096 bytes, the allocator will possibly, even quite likely, introduce
>overhead which is some function of N. Thus even if the allocator tries to
>satisfy these requests by breaking up the previously freed 4 kilobyte zone, it
>may run out of space in that zone before all the requests can be met. Of
>course there is no requirement that the allocator delve into that zone.
>Consider an allocator which keeps free lists in buckets that are indexed by
>size. Freeing the four kilobyte object puts it into, say, a bucket of
>objects in the range 2 to 4K. A request for 16 bytes will look into
>a bucket of smaller chunks, totally ignoring the 4K chunk.
Allocation requests on certain versions of a popular platform
allocate a minimum of 48 bytes, increasing by multiples of 16
over a certain threshold.
Some systems (used to?) use a "buddy allocation" scheme where
requests were rounded up to the next higher power of two
required, then allocated at the first available address boundary
which was a multiple of that size.
Consider the implications of allocation request and overhead
sizes under either of those schemes.
Also, a function in your program could require a few chunks of
memory on each iteration thru a loop, but because it does not
know which chunks some lower level functions may need later,
decides to keep track of its allocations in its own list and not
free any of the allocated memory until the function exits.
Compare that to a function which knows what its callees require
and can free the memory at the end of each iteration, or just
reuse the same chunks for the next iteration, and free only those
few chunks at termination.
Both time and space considerations have to be taken into account
when designing code using dynamic allocation -- everything
depends on what you need or can do and when.
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada
--
Brian....@CSi.com (Brian dot Inglis at SystematicSw dot ab dot ca)
fake address use address above to reply
tos...@aol.com ab...@aol.com ab...@yahoo.com ab...@hotmail.com ab...@msn.com ab...@sprint.com ab...@earthlink.com ab...@cadvision.com ab...@ibsystems.com
spam traps
Lawrence Kirby
>In article <3AC5EFDC...@miel.mot.com>,
>Ramesh Natarajan <nra...@miel.mot.com> writes:
>> HI Friends,
>>
>> I have a problem with using malloc and free
>>
>> I try to allocate memory using malloc and freed it up.
>> The code worked perfectly well. When I checked with
>> purify for memory leaks, it showed no leak.
>>
>> But my problem is, the process's "data" size increases with
>> every malloc but does not get decreased when freed. I came
>> to know about this when i monitored the process by using TOP command.
>>
>> Also when i used "truss" on the executable produced with malloc &
>> subsequently freeing it ,I found an unusual thing.
>>
>> MAlloc allocates memory using brk system call, but the corresponding
>> free does not call any system call to reduce the memory and hence
>> the data size increases with every malloc.
>
>This is incorrect. On Unix systems, malloc allocates memory
>from the heap.
More advanced implementations use mmap().
If the heap is exhausted, sbrk will be used to
>try to grow it. There are limits to the heap size set in the
>kernel (maxdsiz) - on HP-UX the default is 64 meg. If you exceed
>that size, sbrk will fail. It may also fail if you exhaust
>virtual memory.
>
>A free does not "return" memory to the OS.
On some implementations (especially the mmap() ones) free() can cause
memory to be released to the OS.
> It merely deallocates
>the storage on the heap. Unix systems do not typically shrink the heap
>size on free. To do so would require additional bookkeeping and
>it would typically offer little in the way of performance. If you
>needed that space once, typically you will need it again.
It isn't about performance, it is about efficient use of memory.
>If you malloc,free,malloc the same size, with no intervening
>space allocations, sbrk will not be called on the second malloc.
>Iff your heap size continues to grow despite freeing data, then
>it's probable you are not freeing what you think you are freeing.
That's probably true, but not for certain. If the "heap size" keeps
growing indefinitely then there is most likely a problem.
--
-----------------------------------------
Lawrence Kirby | fr...@genesis.demon.co.uk
Wilts, England | 7073...@compuserve.com
-----------------------------------------
Lawrence Kirby
lo...@cs.utexas.edu "Logan Shaw" writes:
>In article <9a64h6$mfr$1...@sunnews.cern.ch>, Dan Pop <Dan...@cern.ch> wrote:
>>Giving back the memory to the OS creates a conformance problem.
>>
>> The free function causes the space pointed to by ptr to be
>> deallocated, that is, made available for further allocation.
>>
>>When reading this statement from the C standard, keep in mind the fact
>>that the C standard describes the behaviour of a single program, not of
>>the whole system.
>>
>>If the memory freed by the program is given back to the OS, the OS might
>>give it to another program, so the program that deallocated it in the
>>first place may not be able to reuse it, although the above quote seems
>>to guarantee that it could reuse it.
>
>So, you're saying the standard says that in this code
>
> void *p;
>
> p = malloc (10000);
> free (p);
> p = malloc (10000);
>
>that the second malloc should never fail under any circumstance.
The standard makes no such guarantee even if the first call to malloc()
succeeds. The standard does say that freed memory is a avilable for
reallocation but it makes no requirements about the conditions under which
it will be made available.
>(Assuming that the code isn't threaded or something.)
>
>While I agree that what you quoted could be read to say that, I'm not
>sure I believe that's the only valid interpretation. In fact, I'd say
>the text you quoted is just plain ambiguous. It does not specify to
>*what* it's made available for further allocation.
To the program.
>In case anyone cares, before virtual memory was commonplace, it was a
>pretty common thing for malloc() and free() to work against a memory
>pool that was common to all processes. This was true even on systems
>that supported multitasking. (At least, it was true on the Amiga.)
This is potentially non-conforming if you can produce a strictly
conforming program that can show that the implemenbtation is in violation
of the standard. I suggest that this is not possible.
Lawrence Kirby
dav...@webmaster.com "David Schwartz" writes:
>
>
>Dan Pop wrote:
>
>> Giving back the memory to the OS creates a conformance problem.
>>
>> The free function causes the space pointed to by ptr to be
>> deallocated, that is, made available for further allocation.
>>
>> When reading this statement from the C standard, keep in mind the fact
>> that the C standard describes the behaviour of a single program, not of
>> the whole system.
>>
>> If the memory freed by the program is given back to the OS, the OS might
>> give it to another program, so the program that deallocated it in the
>> first place may not be able to reuse it, although the above quote seems
>> to guarantee that it could reuse it.
>
> Hahahaha! That's really funny. This is the same as the argument that
>any system that implements 'kill -9' isn't conformant since the standard
>doesn't say your program can just randomly stop running.
The standard only guarantess that the implementation must be able to
translate and execute one program that contains an example of each of
the implementation limits that it specifies. Any other program could
just randomly stop running. Even "the one" program doesn't have to
complete execution every time it is run, the implementation just has to
show it is able to translate and execute it.
Lawrence Kirby
ptre...@esoc.esa.de "Phil Tregoning" writes:
>Dan Pop <Dan...@cern.ch> wrote in article
><9aciql$raj$1...@sunnews.cern.ch>...
>> In <01c0bc39$c500dde0$4b53...@ptregoni.dev.esoc.esa.de> "Phil Tregoning"
><ptre...@esoc.esa.de> writes:
>>
>> >I'm not sure I buy that. Consider the following code:
>> >
>> > void *p;
>> > FILE *fp;
>> > p = malloc(10000);
>> > if (p) {
>> > free(p);
>> > fp = fopen("AFILE","r");
>> > p = malloc(10000);
>> > assert(p != NULL);
>> > }
>> >
>> >Is the assert allowed to fail?
>>
>> Yup. The standard explicitly mention buffer allocation and deallocation
>> issues.
>
>Unless I have missed something, the only file operations
>that the C standard says can allocate memory are setbuf()
>and setvbuf() (which would then be deallocated by fclose()).
>The description of fopen() does not say it has a license
>to allocate any memory.
Nor does it say it doesn't. The action of malloc() does not involve any
side-effects; it creates an object but it does not read or write to
the object. Also there's no way for a strictly conforming program to
tell whether malloc() has been called behind the scenes so there is
nothing to prevent an implementation from doing so. Because of this the
standard doesn't need to give any explicit license for an implementation
to make use of malloc() internally, so it doesn't.
>Where does it say fopen() can allocate memory (C&V)?
See above. *Any* standard library function can call malloc() if it
...
>All in all, I believe that it is not possible to say from the
>C standard exactly what operations can allocate memory (and how
>much). Because of this, the promise that freed memory is
>"available for allocation" is worthless, as a program cannot
Right. It doesn't really guarantee anything even without any intervening
function calls. It is more a statement of intent than anything else.
Lawrence Kirby
...
>If a standard library function or an arbitrary expression in general
>needs memory from the malloc pool, but none is available, what happens?
If it has a failure return value available it can use that, otherwise
it must cause an abnormal program termination.
Dan Pop
>In article <9akoj0$3ff$1...@sunnews.cern.ch> Dan...@cern.ch "Dan Pop" writes:
>
>...
>
>>If a standard library function or an arbitrary expression in general
>>needs memory from the malloc pool, but none is available, what happens?
>
>If it has a failure return value available it can use that, otherwise
>it must cause an abnormal program termination. ^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Chapter and verse, please.
Lawrence Kirby
>In <988206...@genesis.demon.co.uk> fr...@genesis.demon.co.uk (Lawrence Kirby)
> writes:
>
>>In article <9akoj0$3ff$1...@sunnews.cern.ch> Dan...@cern.ch "Dan Pop" writes:
>>
>>...
>>
>>>If a standard library function or an arbitrary expression in general
>>>needs memory from the malloc pool, but none is available, what happens?
>>
>>If it has a failure return value available it can use that, otherwise
>>it must cause an abnormal program termination. ^^^^^^^^^
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>Chapter and verse, please.
It is a matter of elimination - there is nothing else that it can do
that doesn't violate the standard in some way. OK, perhaps it could
just go into an infinite loop and never return but even that is
questionable because it is not an accurate simulation of the
abstract machine.
David Schwartz
Dan Pop wrote:
> >>If a standard library function or an arbitrary expression in general
> >>needs memory from the malloc pool, but none is available, what happens?
> >If it has a failure return value available it can use that, otherwise
> >it must cause an abnormal program termination. ^^^^^^^^^
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Chapter and verse, please.
You need chapter and verse for a claim that is equivalent to "defined
operations must not produce undefined behavior"?
DS
Dan Pop
Yup, you got it right :-)
Dan Pop
>In article <9c7f6r$fl3$1...@sunnews.cern.ch> Dan...@cern.ch "Dan Pop" writes:
>
>>In <988206...@genesis.demon.co.uk> fr...@genesis.demon.co.uk (Lawrence Kirby)
>> writes:
>>
>>>In article <9akoj0$3ff$1...@sunnews.cern.ch> Dan...@cern.ch "Dan Pop" writes:
>>>
>>>...
>>>
>>>>If a standard library function or an arbitrary expression in general
>>>>needs memory from the malloc pool, but none is available, what happens?
>>>
>>>If it has a failure return value available it can use that, otherwise
>>>it must cause an abnormal program termination. ^^^^^^^^^
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>Chapter and verse, please.
>
>It is a matter of elimination
It's a matter of broken elimination.
>- there is nothing else that it can do
>that doesn't violate the standard in some way. OK, perhaps it could
>just go into an infinite loop and never return but even that is
>questionable because it is not an accurate simulation of the
>abstract machine.
There is at least one better alternative: allocate all the resources
needed by such functions and expressions before calling main(). If those
resources cannot be allocated, refuse to execute the program.
For any implementation with a non-null QoI factor, a function that cannot
fail or an expression with well defined behaviour cannot cause abnormal
program termination because the standard gives them no licence to do so.
The attempt to call a function may fail (because this requires a
dynamically allocated resource, usually stack space), but once the
function has been successfully called, it *must* return. If the function
cannot return a failure indication, the function must *always* suceed.
David Schwartz
Dan Pop wrote:
> For any implementation with a non-null QoI factor, a function that cannot
> fail or an expression with well defined behaviour cannot cause abnormal
> program termination because the standard gives them no licence to do so.
A license to fail is not needed. If it was, any UNIX that implemented
'kill' would be violating numerous standards. If you can't do what
you've been asked to do, and you have no good way to report that, the
only thing you can legally do is terminate the program.
> The attempt to call a function may fail (because this requires a
> dynamically allocated resource, usually stack space), but once the
> function has been successfully called, it *must* return. If the function
> cannot return a failure indication, the function must *always* suceed.
Nothing can guarantee that. If SuSv2 added a new 'read' function with
no failure returns, what do you do if the disk fails?
DS
Dan Pop
>Dan Pop wrote:
>
>> For any implementation with a non-null QoI factor, a function that cannot
>> fail or an expression with well defined behaviour cannot cause abnormal
>> program termination because the standard gives them no licence to do so.
>
> A license to fail is not needed. If it was, any UNIX that implemented
>'kill' would be violating numerous standards.
How is Unix' kill -9 different from a power failure or pressing the reset
button, from the killed program's point of view? I have already explained
you, in this very thread (a few weeks ago, it's true) that your kill
argument is idiotic.
>> The attempt to call a function may fail (because this requires a
>> dynamically allocated resource, usually stack space), but once the
>> function has been successfully called, it *must* return. If the function
>> cannot return a failure indication, the function must *always* suceed.
>
> Nothing can guarantee that. If SuSv2 added a new 'read' function with
>no failure returns, what do you do if the disk fails?
There is no read function, as far as the C standard is concerned. If
nobody added such a function to either Unix or Standard C, there must
be a reason :-)
Peter Seebach
Sure it is.
>If it was, any UNIX that implemented
>'kill' would be violating numerous standards.
Only when someone uses kill. ;) But yes, if you kill a C program, the
program was not run in a conforming environment.
>If you can't do what
>you've been asked to do, and you have no good way to report that, the
>only thing you can legally do is terminate the program.
"legally"?
Anyway, if it doesn't say you can terminate, what makes you think termination
is acceptable?
For a nice concrete example, consider tmpnam(). The C89 standard *DOES NOT*
allow for it to fail on the first call! So, what do you do?
* abort
* wait around hoping something will change so you can create such a name
* return a null pointer and hope someone checked for it
Option 3 is the one I saw used in a few systems, and I believe C99 blesses
it.
> Nothing can guarantee that. If SuSv2 added a new 'read' function with
>no failure returns, what do you do if the disk fails?
Most likely, never return. Such a specification is ill-considered, but from
a conformance standpoint, if a function with no specified failure modes can
fail, the implementation is broken. However, at that point, it doesn't
*MATTER* what you do, except as a quality of implementation issue.
-s
--
Copyright 2001, all wrongs reversed. Peter Seebach / se...@plethora.net
C/Unix wizard, Pro-commerce radical, Spam fighter. Boycott Spamazon!
Consulting & Computers: http://www.plethora.net/
Richard Bos
> In article <3AE9F23E...@webmaster.com>,
> David Schwartz <dav...@webmaster.com> wrote:
> > A license to fail is not needed.
>
> Sure it is.
>
> >If it was, any UNIX that implemented
> >'kill' would be violating numerous standards.
>
> Only when someone uses kill. ;) But yes, if you kill a C program, the
> program was not run in a conforming environment.
So what you're saying is that until your program has completely finished
and returned its status to the hosting environment, your system is about
as conforming as Schroedinger's cat is alive? Doesn't that make the
whole issue of conformity kind of useless?
One could never claim that a certain system is conforming, because power
may fail, chips may fry, _any_ system may abort unexpectedly. Therefore,
a system is _never_ conforming; only one particular run of one
particular program can be called "conforming", and that only after the
fact.
Seems to me that this kind of qualification may as well not be given.
You get discussions like "This program may only function correctly in
conforming environments." "Ok, so how do I know what kind of
environments to run it in?" "Run it, and if it functioned correctly,
that instance of execution happened to be conforming." Not very useful,
IYAM.
Richard
Peter Seebach
Richard Bos <in...@hoekstra-uitgeverij.nl> wrote:
>se...@plethora.net (Peter Seebach) wrote:
>> Only when someone uses kill. ;) But yes, if you kill a C program, the
>> program was not run in a conforming environment.
>So what you're saying is that until your program has completely finished
>and returned its status to the hosting environment, your system is about
>as conforming as Schroedinger's cat is alive? Doesn't that make the
>whole issue of conformity kind of useless?
No. An implementation which *can* be run in a conforming way is still useful.
It's a difficult distinction to draw, certainly, and I'm not sure how to make
it useful.
Dan Pop
>se...@plethora.net (Peter Seebach) wrote:
>
>> In article <3AE9F23E...@webmaster.com>,
>> David Schwartz <dav...@webmaster.com> wrote:
>> > A license to fail is not needed.
>>
>> Sure it is.
>>
>> >If it was, any UNIX that implemented
>> >'kill' would be violating numerous standards.
>>
>> Only when someone uses kill. ;) But yes, if you kill a C program, the
>> program was not run in a conforming environment.
>
>So what you're saying is that until your program has completely finished
>and returned its status to the hosting environment, your system is about
>as conforming as Schroedinger's cat is alive? Doesn't that make the
>whole issue of conformity kind of useless?
>One could never claim that a certain system is conforming, because power
>may fail, chips may fry, _any_ system may abort unexpectedly. Therefore,
>a system is _never_ conforming; only one particular run of one
>particular program can be called "conforming", and that only after the
>fact.
The standard has "carefully" [*] taken care of that:
2.2.4.1 Translation limits
The implementation shall be able to translate and execute at least
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
one program that contains at least one instance of every one of the
^^^^^^^^^^^
following limits:
So, an implementation can still be conforming, even if the kill command
exists, even if power can fail, even if the system has a Reset button
that can be accidentally pressed (and I've seen some PC's made by Vobis
where such an accident could happen incredibly easy).
[*] In the absence of QoI considerations, the above quote renders the
rest of the C standard utterly useless.
David Schwartz
Peter Seebach wrote:
> > Nothing can guarantee that. If SuSv2 added a new 'read' function with
> >no failure returns, what do you do if the disk fails?
>
> Most likely, never return. Such a specification is ill-considered, but from
> a conformance standpoint, if a function with no specified failure modes can
> fail, the implementation is broken. However, at that point, it doesn't
> *MATTER* what you do, except as a quality of implementation issue.
As a quality of implementation issue, what can you do that's
conforming? Is never returning conforming? Is terminating the
application conforming? If not, is literally nothing conforming? If so,
almost no implementation conforms to any standard.
DS
Peter Seebach
David Schwartz <dav...@webmaster.com> wrote:
> As a quality of implementation issue, what can you do that's
>conforming? Is never returning conforming? Is terminating the
>application conforming? If not, is literally nothing conforming? If so,
>almost no implementation conforms to any standard.
Never returning is probably the closest, since after all, there's no
speed committments in the standard.
However, QoI goes beyond conformance; QoI includes "how gracefully do you
handle undefined behavior" or "how useful is your interpretation to the
user".
Rory Foster
in middleware for C/C++, so free() is space that's typically returned to
the C Runtime Library. ( An alternate library example would be an allocator
replacement that provides higher performance and automatic memory
management. )
Middleware still runs in that same address space as the process, so when
objects are freed or deleted they are still available in the same address
space managed by the allocator. This space that the allocator manages
is called heap memory.
Now, giving back memory to the OS is something that the middleware
allocator might do. So some of the virtual to physical memory mapping
would become invalid. The application should be able to request that
that segment of the address space that was unmapped be remapped
at a latter time. The notion of giving that space to another process
would only take place in physical memory not in virtual memory. The
address space of the process remains 4Gb in the case of a 32bit process.
The only difference would be some of the addresses no longer have a
virtual to physical mapping. If the address were unmapped, the HW
would fault, the OS would try to use its data structures to translate. Once
it could not translate, the OS would delive some type of segmentation
violation, or bus error, because of the "bad" address.
I may have missed some of this thread, but it seems like the line
the separates the application at the system call level has been
blurred. What happens with malloc(), new(), free(), delete() occurs
above the system call. Giving pages back to the OS, say with mmap(),
happens below the system call level.
Lastly, was there an original problem with leaking process memory
for a C/C++ application. If so, I'd recommend automatic memory
management. Just fix the leaks automatically. That way application
code as well as third-party libraries would benefit from leak
remediation automatically.
--
Mark McIntyre
<rfos...@austin.rr.com> wrote:
>Let me make sure I understand. The allocator is actually implemented
>in middleware for C/C++,
what the heck are you talking about ?
>so free() is space that's typically returned to
>the C Runtime Library. ( An alternate library example would be an allocator
>replacement that provides higher performance and automatic memory
>management. )
free() releases the memory from use by your program. What the OS does
with it is of no interest.
>Middleware still runs in that same address space as the process, so when
>objects are freed or deleted they are still available in the same address
>space managed by the allocator. This space that the allocator manages
>is called heap memory.
What ?
Stop posting offtopic babble to CLC please !!!
--
Mark McIntyre
CLC FAQ <http://www.eskimo.com/~scs/C-faq/top.html>
Arthur H. Gold
>
> Let me make sure I understand. The allocator is actually implemented
> in middleware for C/C++, so free() is space that's typically returned to
> the C Runtime Library. ( An alternate library example would be an allocator
> replacement that provides higher performance and automatic memory
> management. )
itself.
>
> Middleware still runs in that same address space as the process, so when
> objects are freed or deleted they are still available in the same address
> space managed by the allocator. This space that the allocator manages
> is called heap memory.
>
> Now, giving back memory to the OS is something that the middleware
> allocator might do. So some of the virtual to physical memory mapping
> would become invalid. The application should be able to request that
> that segment of the address space that was unmapped be remapped
> at a latter time. The notion of giving that space to another process
> would only take place in physical memory not in virtual memory. The
> address space of the process remains 4Gb in the case of a 32bit process.
> The only difference would be some of the addresses no longer have a
> virtual to physical mapping. If the address were unmapped, the HW
> would fault, the OS would try to use its data structures to translate. Once
> it could not translate, the OS would delive some type of segmentation
> violation, or bus error, because of the "bad" address.
>
> I may have missed some of this thread, but it seems like the line
> the separates the application at the system call level has been
> blurred. What happens with malloc(), new(), free(), delete() occurs
> above the system call. Giving pages back to the OS, say with mmap(),
> happens below the system call level.
>
the C library (or whatever replacement allocator one might use). In
userland (i.e. the same address space as the application).
The system calls related to memory management are brk() (which sets the
pointer to the `top' of the heap, adding/deleting pages to the usable
virtual address space in the process), mmap() (which adds pages to the
address space, optionally filling them with the contents of some file),
munmap() (which deletes pages from the virtual address space) and
(possibly, depends on system) mremap() (which can `move stuff around' in
the address space.
A typical malloc() works (something like) this:
Do I have a suitable chunk of memory available?
If yes, return it, and mark it as being in use.
Otherwise, call brk() for more heap space, set it up
and satisfy the request.
If _that_ didn't work, return a NULL.
free(), OTOH:
Mark this memory as available for future use.
In some implementations it might also say:
Hey, do I have a bunch of stuff at the top of the
heap that's not in use (and, perhaps, adequate free
space in reserve? Yes? Call brk() to make the heap
smaller, returning that memory to the system.
Some implementations may enhance this by saying:
Hmmm, that's a pretty big block of stuff you want. Why
don't I just mmap() some pages for it so I can return it
back to the system later more easily.
> Lastly, was there an original problem with leaking process memory
> for a C/C++ application. If so, I'd recommend automatic memory
> management. Just fix the leaks automatically. That way application
> code as well as third-party libraries would benefit from leak
> remediation automatically.
Automatic memory management, a.k.a. garbage collection, is a very
valuable tool, though one that comes at a cost. Since C and C++ are
philosophically `only pay for it if you use it' languages, you're not
likely to see it there. This is not to disparage GC by any means;
languages such as those from the LISP family and functional languages in
general could not exist without it. For some languages, its inherent in
the design (Java). It can, in some cases though, not be the best
approach.
Any garbage collector works on the premise that iff you can't reach it,
it's garbage. Unfortunately, there are many cases where a piece of
memory may be reachable, but due to the program's semantics, the
programmer knows it will not be acceessed -- with manual memory
management it can be released. [This often comes up in pure functional
programming, whose (typically) call-by-need semantics often obscure
program flow -- causing the occasional heap explosion. Naturally, there
are ways of `programming around' such situations, but doing so often
requires either a very deep understanding of the operational semantics
involved or, at least, a knowledge of the sorts of idioms that produce
the problem and how to avoid them.]
One other note:
What often looks like a memory leak from the outside (address space
keeps growing and growing) is a matter of memory fragmentation.
Consider the following program:
#include <stdlib.h>
int main( void ) {
int i = 1;
char * buf;
while (buf = malloc(i)) {
i+=i;
free( buf );
}
}
On many implementations, the heap will grow to a larger size than you
might expect.
HTH,
--ag
[x-post to c.l.c snipped]
--
Artie Gold, Austin, TX (finger the cs.utexas.edu account for more info)
mailto:ag...@bga.com or mailto:ag...@cs.utexas.edu
--
The King is dead. But the Boogie Chillun live.
Nils O. Selåsdal
"> #include <stdlib.h>
> int main( void ) {
> int i = 1;
> char * buf;
> while (buf = malloc(i)) {
> i+=i;
> free( buf );
> }
> }
#include <stdlib.h>
int
main (void)
{
int i = 1;
char *buf;
{
buf=malloc(i);
if (buf == NULL)
{
puts ("Out of Memory");
}
else
{
i += i;
free (buf);
sleep (1);
}
}
}
On my RH machine, why does malloc fail pretty soon, while top/ps
shows that the process uses only about 4-500kb memory?
Rory Foster
> > Lastly, was there an original problem with leaking process memory
> > for a C/C++ application. If so, I'd recommend automatic memory
> > management. Just fix the leaks automatically. That way application
> > code as well as third-party libraries would benefit from leak
> > remediation automatically.
>
>Automatic memory management, a.k.a. garbage collection, is a very
>valuable tool, though one that comes at a cost. Since C and C++ are
>philosophically `only pay for it if you use it' languages, you're not
>likely to see it there. This is not to disparage GC by any means;
>languages such as those from the LISP family and functional languages in
>general could not exist without it. For some languages, its inherent in
>the design (Java). It can, in some cases though, not be the best
>approach.
>
>Any garbage collector works on the premise that iff you can't reach it,
>it's garbage. Unfortunately, there are many cases where a piece of
>memory may be reachable, but due to the program's semantics, the
>programmer knows it will not be acceessed -- with manual memory
>management it can be released. [This often comes up in pure functional
>programming, whose (typically) call-by-need semantics often obscure
>program flow -- causing the occasional heap explosion. Naturally, there
>are ways of `programming around' such situations, but doing so often
>requires either a very deep understanding of the operational semantics
>involved or, at least, a knowledge of the sorts of idioms that produce
>the problem and how to avoid them.]
Actually, there are freeware and commercially available automatic
memory managers that do honor manual frees and deletes as well
as remediate leaks. What you describe is sometimes called application
"bloat" where the memory can be reached, but for whatever reason
will never be used. And, yes, automatic memory management will
not help with bloat.
>
>One other note:
>What often looks like a memory leak from the outside (address space
>keeps growing and growing) is a matter of memory fragmentation.
>Consider the following program:
[snip]
Most all allocator replacements (either GC or simply a faster allocator)
use size-specific allocators to greatly reduce fragmentation.
Again, I'm not sure of the orginal post, but if someone is looking
for a solution for their memory leaks and perhaps the ability to
improve their performance, I still recommend an allocator replacement.
By that I mean a new library. Even though the C Runtime Library is
a system library instead of a third-party library, it's still aka
middleware.