malloc
ramu
what happens when i run the below code?
main()
{
int *p;
while(1)
p= (int *)malloc(1000);
}
Do i get segmentation fault?
Regards
dco...@connx.com
Likely, you will get memory allocated until the OS+compiler will give
you no more, after which the malloc() calls will return null pointers
(resulting in more or less a spin lock). Of course, the call to
malloc() is lacking a prototype and malloc() does not return int.
Therefore, the program can do whatever it likes.
>From the ANSI/ISO C standard, section 6.3.2.3 Pointers:
"An integer may be converted to any pointer type. Except as previously
specified, the result is implementation-defined, might not be correctly
aligned, might not point to an entity of the referenced type, and might
be a trap representation.56)"
footnote 56): The mapping functions for converting a pointer to an
integer or an integer to a pointer are intended to be consistent with
the addressing structure of the execution environment.
james of tucson
> Do i get segmentation fault?
Eventually, you get NULL from the malloc call, which you do not check,
and it repeats forever. Since you don't access the pointer, why would
it segfault?
In practice, I doubt you get a segmentation fault. You probably just go
into an endless loop getting null from malloc forever. On my system,
the user starts getting NULL from malloc long before the system runs out
of virtual memory (because it is limited per-user). So the effect is
the same as calling any other function that returns NULL in an endless
loop; there is nothing special about malloc in this regard, and really
no reason for it to cause the system to segfault.
Barry Schwarz
>Hi,
> what happens when i run the below code?
>
> main()
> {
> int *p;
> while(1)
> p= (int *)malloc(1000);
You invoke undefined behavior during the first call to malloc. Even
if size_t is defined as unsigned int on your system, you also invoke
undefined behavior upon the first return from malloc. Once you invoke
undefined behavior, your program can do anything the operating system
lets it get away with.
If you correct the undefined behavior by removing the cast and
including stdlib.h, your program will repeatedly call malloc until you
or the operating system terminate it. On a typical desk top system,
the first umpteen calls will succeed and all subsequent calls will
fail. Since malloc fails gracefully (by returning NULL), these
failures should not interfere with your program continuing on its
merry way. The fact that you have created umpteen memory leaks, while
poor programming practice, should also not interfere with your
program's execution.
> }
> Do i get segmentation fault?
>
Why would you expect a segmentation fault? The usual causes are
accessing memory you don't own or attempting to do something
unauthorized with memory you do own (such as writing to read only
memory). You never attempt to access the allocated memory at all. In
fact, the only memory you access other than the code for main and
malloc is the memory reserved for p. Since this is an automatic
variable inside your function, nothing should hamper this access.
Remove del for email
Sourcerer
news:1165899429....@80g2000cwy.googlegroups.com...
Why speculate when you can test?
I made the program in VS. What it does is it allocates all your memory, then
malloc returns NULL. When I stopped the program, my VS crashed and my memory was
freed.
That's about it.
--
"It is easy in the world to live after the world's oppinion; it easy in solitude
to live after our own; but the great man is he who in the midst of the crowd
keeps with perfect sweetness the independence of solitude."
Ralph Waldo Emerson, Self-reliance 1841
http://pinpoint.wordpress.com/
Richard Heathfield
> "ramu" <ramu...@gmail.com> wrote in message
> news:1165899429....@80g2000cwy.googlegroups.com...
>> Hi,
>> what happens when i run the below code?
>>
>> main()
>> {
>> int *p;
>> while(1)
>> p= (int *)malloc(1000);
>> }
>> Do i get segmentation fault?
>
> Why speculate when you can test?
Because the behaviour is undefined, so a test will only indicate what
happens on that particular platform under that particular implementation
with those particular compilation options on that particular occasion.
So he was right to ask, and the right answer is "anything".
To fix the program, he needs to add:
#include <stdlib.h>
At that point, his program's behaviour becomes well-defined, albeit rather
pointless and wasteful.
--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.
Sourcerer
news:GpWdnQIRluL...@bt.com...
> Sourcerer said:
>
>> "ramu" <ramu...@gmail.com> wrote in message
>> news:1165899429....@80g2000cwy.googlegroups.com...
>>> Hi,
>>> what happens when i run the below code?
>>>
>>> main()
>>> {
>>> int *p;
>>> while(1)
>>> p= (int *)malloc(1000);
>>> }
>>> Do i get segmentation fault?
>>
>> Why speculate when you can test?
>
> To fix the program, he needs to add:
>
> #include <stdlib.h>
>
One has to love your sense of humor. :)
Richard Heathfield
> "Richard Heathfield" <r...@see.sig.invalid> wrote in message
> news:GpWdnQIRluL...@bt.com...
<snip>
>> To fix the program, he needs to add:
>>
>> #include <stdlib.h>
>>
>> <snip>
>
> One has to love your sense of humor. :)
Undoubtedly. I can be an extraordinarily diverting and entertaining chap at
times. Nevertheless, my reply was perfectly serious. The omission of the
header causes a serious problem, because it means we don't have a prototype
for malloc. Under that circumstance, the compiler is obliged to assume that
malloc returns int, whereas in fact it returns void *. So the compiler may
generate code based on that forced assumption (e.g. it may fetch a random
int from a register it dedicates to integer return values, instead of
getching the correct pointer value from a pointer register), resulting in
an integer value being assigned to a pointer, with unpredictable results.
Were it not for the stupid pointless cast, the compiler would be obliged to
diagnose the mismatch between the return type it has been forced to assume
for the function, and the type of the object into which the return value is
being placed. The stupid pointless cast removes the obligation on the
compiler to issue its diagnostic message, but of course does not fix the
underlying problem. The proper course for the programmer is to provide the
prototype, and the best way to do that is to include the appropriate header
for the malloc function.
Sourcerer
True, but I assumed the author omitted the inclusion of the header for the sake
of brevity. Those of us humans, who are knowledgeable of the C language, know
that malloc's prototype is provided in stdlib.h, whereas a compiler does not.
The reason I also found it funny is because you stated something that I took as
obvious. For some reason, that never fails to make me smile. Don't get me wrong,
though; I find it admirable when someone has such full grasp of the concepts in
programming languages as well as in other things in life.
Chris Dollin
> True, but I assumed the author omitted the inclusion of the header for the sake
> of brevity.
Sadly, this isn't a safe assumption. So we assume that what isn't presented,
isn't there.
--
Chris "Perikles triumphant" Dollin
"People are part of the design. It's dangerous to forget that." /Star Cops/
goose
Barry Schwarz wrote:
<snipped>
> or the operating system terminate it. On a typical desk top system,
> the first umpteen calls will succeed and all subsequent calls will
> fail. Since malloc fails gracefully (by returning NULL), these
Not *all* typical desktops perform the same; ISTR that FreeBSD
will happily overcommit knowing that all the memory (virtual + real)
will never ever be needed at the same time. When all processes
*do* attempt to run concurrently the OS picks a process and kills
it, so all malloc calls *always* return a pointer to memory.
I further seem to recall that this was tunable with a FreeBSD
kernel knob. If my memory serves me correctly, the concensus
on clc the last time this came up was that such an implementation
could not be considered conforming ...
Or else all of the conversation that I remember happened on a
different newsgroup :-) (My memory really isn't what it used to be).
goose,
Keith Thompson
> Sourcerer wrote:
>> True, but I assumed the author omitted the inclusion of the header
>> for the sake of brevity.
>
> Sadly, this isn't a safe assumption. So we assume that what isn't presented,
> isn't there.
Indeed. We see plenty of code that calls library functions without
including the required header. In many cases, the poster omitted the
#include out of ignorance, not for brevity.
Unfortunately, such code happens to "work" under many implementations,
so the compiler might not sufficiently encourage the user to break his
bad habits. (Most compilers will probably issue a warning; too many
programmers silence the warning by adding a cast.)
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Random832
goose wrote:
>
> Barry Schwarz wrote:
>
>
> <snipped>
>
>> or the operating system terminate it. On a typical desk top system,
>> the first umpteen calls will succeed and all subsequent calls will
>> fail. Since malloc fails gracefully (by returning NULL), these
>
> Not *all* typical desktops perform the same; ISTR that FreeBSD
> will happily overcommit knowing that all the memory (virtual + real)
> will never ever be needed at the same time. When all processes
> *do* attempt to run concurrently
Um... run concurrently? you mean as opposed to being in swap? Because
that's where the "virtual" part normally comes in.
Random832
Richard Heathfield wrote:
> Under that circumstance, the compiler is obliged to assume that malloc
> returns int,
Under which circumstance, in turn, the compiler is not "obliged" to
anything in particular at all.
Gordon Burditt
>> the first umpteen calls will succeed and all subsequent calls will
>> fail. Since malloc fails gracefully (by returning NULL), these
>
>Not *all* typical desktops perform the same; ISTR that FreeBSD
>will happily overcommit knowing that all the memory (virtual + real)
>will never ever be needed at the same time.
Memory allocation on FreeBSD (examples on i386) can STILL fail:
(1) You run out of address space. malloc((size_t)-1) will fail because
the code required to call malloc() (which exceeds one byte) plus the
memory to be allocated exceed the available address space (2**32).
This will fail regardless of how much swap/page space you have.
(2) You can violate process resource limits, which may or may
not be hit before you get to the point of overcommitting memory.
>When all processes
>*do* attempt to run concurrently the OS picks a process and kills
>it, so all malloc calls *always* return a pointer to memory.
Incorrect. In addition to (1) and (2) above, setting option V will
cause malloc(0) to return NULL, rather than returning a pointer to
a minimal amount of allocated memory. However, setting option X
will cause malloc() to call abort() rather than return NULL (V and X
are incompatable). Among other ways to set options, there is the
MALLOC_OPTIONS environment variable.
The documentation for the configuration of malloc() behavior doesn't
mention anything about overcommitting memory, so I don't know whether
FreeBSD does it or not.
>I further seem to recall that this was tunable with a FreeBSD
>kernel knob. If my memory serves me correctly, the concensus
>on clc the last time this came up was that such an implementation
>could not be considered conforming ...
I'll agree here.
As I recall, program termination for various reasons external to the
program weren't allowed either, including:
- Manual program killing
- Power failure, combined with exhaustion of battery power
- Nuclear detonation in the vicinity of the computer
- Failure to pay the hosting bill
Richard Heathfield
"If the expression that precedes the parenthesized argument list in a
function call consists solely of an identifier, and if no declaration is
visible for this identifier, the identifier is implicitly declared exactly
as if, in the innermost block containing the function call, the declaration
extern int identifier();
appeared." (C89: 3.3.2.2)
Keith Thompson
> Random832 said:
>> 2006-12-12 <7KOdnVNe_ry...@bt.com>,
>> Richard Heathfield wrote:
>>> Under that circumstance, the compiler is obliged to assume that malloc
>>> returns int,
>>
>> Under which circumstance, in turn, the compiler is not "obliged" to
>> anything in particular at all.
>
> "If the expression that precedes the parenthesized argument list in a
> function call consists solely of an identifier, and if no declaration is
> visible for this identifier, the identifier is implicitly declared exactly
> as if, in the innermost block containing the function call, the declaration
>
> extern int identifier();
>
> appeared." (C89: 3.3.2.2)
C99 6.5.2.2p9:
If the function is defined with a type that is not compatible with
the type (of the expression) pointed to by the expression that
denotes the called function, the behavior is undefined.
(C90 has similar wording.)
Since the definition of malloc() has it returning void*, calling it as
if it returned int invokes undefined behavior. (A perverse compiler
could recognize the name "malloc" and generate a "correct" call,
knowing that it actually returns void*.)
Richard Heathfield
<snip>
> Since the definition of malloc() has it returning void*, calling it as
> if it returned int invokes undefined behavior.
Yes, which is why I said, in my first reply in this thread, that the
behaviour of the program is undefined. And even if I hadn't said that, did
you really think I didn't know it?
goose
> >> or the operating system terminate it. On a typical desk top system,
> >> the first umpteen calls will succeed and all subsequent calls will
> >> fail. Since malloc fails gracefully (by returning NULL), these
> >
> >Not *all* typical desktops perform the same; ISTR that FreeBSD
> >will happily overcommit knowing that all the memory (virtual + real)
> >will never ever be needed at the same time.
>
> Memory allocation on FreeBSD (examples on i386) can STILL fail:
> (1) You run out of address space. malloc((size_t)-1) will fail because
> the code required to call malloc() (which exceeds one byte) plus the
> memory to be allocated exceed the available address space (2**32).
> This will fail regardless of how much swap/page space you have.
> (2) You can violate process resource limits, which may or may
> not be hit before you get to the point of overcommitting memory.
>
Agreed. To both.
> >When all processes
> >*do* attempt to run concurrently the OS picks a process and kills
> >it, so all malloc calls *always* return a pointer to memory.
>
> Incorrect. In addition to (1) and (2) above, setting option V will
> cause malloc(0) to return NULL, rather than returning a pointer to
> a minimal amount of allocated memory. However, setting option X
> will cause malloc() to call abort() rather than return NULL (V and X
> are incompatable). Among other ways to set options, there is the
> MALLOC_OPTIONS environment variable.
Um, I wasn't referring to the tuning of malloc options on
FreeBSD, but the tuning of the VM bits of the kernel itself.
FreeBSD famously overcommits swap. Lets say we have a total
of 10Meg swap. 4 processes are running with each using 2Meg
swap. A 5th process starts up and requests 4Meg. FreeBSD
grants the request with a valid pointer secure in the knowledge
that *before* any access to that swap is allowed one of the
first 4 processes must finish running.
Either way, malloc does not "fail gracefully" here; a process
may be targetted for killing in the event that none of the
original 4 processes end within a certain time.
This "feature" is so common that it was actually part of the
answer on an exam I wrote some time back (seems like ages ago)
on operating systems design.
See:
http://lists.freebsd.org/pipermail/freebsd-stable/2003-July/002362.html
ALSO
http://lists.freebsd.org/pipermail/freebsd-stable/2003-July/002265.html
I do not know if the latest FreeBSD has the discussed knob or not.
>
> The documentation for the configuration of malloc() behavior doesn't
> mention anything about overcommitting memory, so I don't know whether
> FreeBSD does it or not.
>
It does, AFAIK.
goose,
goose
> 2006-12-12 <1165956252.8...@j44g2000cwa.googlegroups.com>,
> goose wrote:
> >
> > Barry Schwarz wrote:
> >
> >
> > <snipped>
> >
> >> or the operating system terminate it. On a typical desk top system,
> >> the first umpteen calls will succeed and all subsequent calls will
> >> fail. Since malloc fails gracefully (by returning NULL), these
> >
> > Not *all* typical desktops perform the same; ISTR that FreeBSD
> > will happily overcommit knowing that all the memory (virtual + real)
> > will never ever be needed at the same time. When all processes
> > *do* attempt to run concurrently
>
> Um... run concurrently? you mean as opposed to being in swap? Because
> that's where the "virtual" part normally comes in.
>
Don't attempt to be condescending; you aren't any good at it.
See my reply to Barry Schwarz elsethread.
goose,
It's more common than you think.
Richard Heathfield
<snip>
>
> Don't attempt to be condescending; you aren't any good at it.
But *I* am. I can provide top-quality condescension at competitive rates.
All he has to do is call.
Random832
Was I wrong?
> See my reply to Barry Schwarz elsethread.
Your only reply to him that I can see is the one that I, in turn replied
to. Right above this one.
> goose,
> It's more common than you think.
What is? A program never "releases" memory without ever using it, that'd
be silly. So basically it's gambling that program A will either
"release" some memory or die before program B attempts to write the
memory *that it just allocated* most likely in the next few seconds,
thus allowing it to thwart any code that might be in place for program
B to sanely handle the case where not as much memory as it wanted was
available.
Random832
Maybe FreeBSD does this, but it's Linux that is famous for it.
> Either way, malloc does not "fail gracefully" here; a process
> may be targetted for killing in the event that none of the
> original 4 processes end within a certain time.
Is it a certain time, or is it just whenever the 5th process tries to
access the allocated memory?
incidentally, if it is "within a certain time", that means if program
5 attempts to access the memory before that time, it will block until
that time has passed. Which opens the door for a conforming
implementation that overcommits: just block until the memory is
available, with no such timeout.
Random832
Richard Heathfield wrote:
> Random832 said:
>
>> 2006-12-12 <7KOdnVNe_ry...@bt.com>,
>> Richard Heathfield wrote:
>>> Under that circumstance, the compiler is obliged to assume that malloc
>>> returns int,
>>
>> Under which circumstance, in turn, the compiler is not "obliged" to
>> anything in particular at all.
>
> "If the expression that precedes the parenthesized argument list in a
> function call consists solely of an identifier, and if no declaration is
> visible for this identifier, the identifier is implicitly declared exactly
> as if, in the innermost block containing the function call, the declaration
>
> extern int identifier();
>
> appeared." (C89: 3.3.2.2)
Right. But if identifier is ever used, that's undefined behavior, which
in turn releases the compiler from all obligations.
It's required to work if the function does in fact return int and, if
it's called, it's called with the right types.
It's _not_ required to _not_ work if this is not the case.
Random832
Richard Heathfield wrote:
> Keith Thompson said:
>
> <snip>
>
>> Since the definition of malloc() has it returning void*, calling it as
>> if it returned int invokes undefined behavior.
>
> Yes, which is why I said, in my first reply in this thread, that the
> behaviour of the program is undefined. And even if I hadn't said that, did
> you really think I didn't know it?
Speaking of what the compiler is "obliged" to do in cases of undefined
behavior, though, is misleading and dangerous. It's only not undefined
behavior if the code is never reached.
CBFalconer
> Keith Thompson said:
>
> <snip>
>
>> Since the definition of malloc() has it returning void*, calling
>> it as if it returned int invokes undefined behavior.
>
> Yes, which is why I said, in my first reply in this thread, that
> the behaviour of the program is undefined. And even if I hadn't
> said that, did you really think I didn't know it?
Don't take such clarifications personally. This is a public place,
and there is good reason to correct and/or clarify statements,
whether or not the originator knows better.
--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>
CBFalconer
> goose said:
>
> <snip>
>
>> Don't attempt to be condescending; you aren't any good at it.
>
> But *I* am. I can provide top-quality condescension at competitive
> rates. All he has to do is call.
I assume you have a suitable walk-in freezer and a humid household.
Richard Heathfield
> there is good reason to correct and/or clarify statements,
> whether or not the originator knows better.
What was unclear about my previous contributions to this thread?
Richard Heathfield
> 2006-12-13 <hJGdnfkntPtif-LY...@bt.com>,
> Richard Heathfield wrote:
>> Keith Thompson said:
>>
>> <snip>
>>
>>> Since the definition of malloc() has it returning void*, calling it as
>>> if it returned int invokes undefined behavior.
>>
>> Yes, which is why I said, in my first reply in this thread, that the
>> behaviour of the program is undefined. And even if I hadn't said that,
>> did you really think I didn't know it?
>
> Speaking of what the compiler is "obliged" to do in cases of undefined
> behavior, though, is misleading and dangerous.
Read what I said again, this time more carefully. I was explaining the cause
of the undefined behaviour in terms of what the compiler was obliged to do.
> It's only not undefined
> behavior if the code is never reached.
That's why I said, right at the outset, that the behaviour was undefined.
Kohn Emil Dan
On Mon, 11 Dec 2006, Barry Schwarz wrote:
> On 11 Dec 2006 20:57:09 -0800, "ramu" <ramu...@gmail.com> wrote:
>
>> Hi,
>> what happens when i run the below code?
>>
>> main()
>> {
>> int *p;
>> while(1)
>> p= (int *)malloc(1000);
>
<snip>
>
>> }
>> Do i get segmentation fault?
>>
>
> Why would you expect a segmentation fault?
Speaking in a more practical way, there are pretty good chances to receive
a segmentation fault, but not in *your* program, but in other buggy
programs running at the same time on your system which won't check the
return value of malloc() and/or other system calls or library functions.
Running your program without appropriate process limits will cause it to
hog up the entire system memory. When the entire memory has been hogged
up, malloc() and other library functions which internally allocate memory
will fail. Unfortunately many programs fail to check the return values of
the library functions and ultimately crash with segmentation faults
because of dereferencing NULL pointers.
Bottom line: If all the programs running on your system correctly checked
the return values of all the library functions, you shouldn't receive a
segfault; unfortunately in real life you pretty much have a good chance to
receive one.
Emil
goose
Random832 wrote:
> 2006-12-13 <1166013671.8...@16g2000cwy.googlegroups.com>,
> goose wrote:
> > Random832 wrote:
> >> 2006-12-12 <1165956252.8...@j44g2000cwa.googlegroups.com>,
> >> goose wrote:
> >> >
> >> > Barry Schwarz wrote:
> >> >
> >> >
> >> > <snipped>
> >> >
> >> >> or the operating system terminate it. On a typical desk top system,
> >> >> the first umpteen calls will succeed and all subsequent calls will
> >> >> fail. Since malloc fails gracefully (by returning NULL), these
> >> >
> >> > Not *all* typical desktops perform the same; ISTR that FreeBSD
> >> > will happily overcommit knowing that all the memory (virtual + real)
> >> > will never ever be needed at the same time. When all processes
> >> > *do* attempt to run concurrently
> >>
> >> Um... run concurrently? you mean as opposed to being in swap? Because
> >> that's where the "virtual" part normally comes in.
> >>
> >
> > Don't attempt to be condescending; you aren't any good at it.
>
> Was I wrong?
You weren't even in the same context. It's impossible to be
wrong if you don't even understand the question.
>
> > See my reply to Barry Schwarz elsethread.
>
> Your only reply to him that I can see is the one that I, in turn replied
> to. Right above this one.
>
> > goose,
> > It's more common than you think.
>
> What is? A program never "releases" memory without ever using it, that'd
> be silly.
I'd advise you to read up a little on this before you go on. I've
provided
links to the threaded mailing-list discussion of this. Feel free.
> So basically it's gambling that program A will either
> "release" some memory or die before program B attempts to write the
> memory *that it just allocated* most likely in the next few seconds,
> thus allowing it to thwart any code that might be in place for program
> B to sanely handle the case where not as much memory as it wanted was
> available.
Yes. I think you get the picture now.
goose,
Why yes, I *am* rather tetchy today. Why do you ask?
Random832
goose wrote:
>
> Random832 wrote:
>
>> 2006-12-13 <1166013671.8...@16g2000cwy.googlegroups.com>,
>> goose wrote:
>> > Random832 wrote:
>> >> 2006-12-12 <1165956252.8...@j44g2000cwa.googlegroups.com>,
>> >> goose wrote:
>> >> >
>> >> > Not *all* typical desktops perform the same; ISTR that FreeBSD
>> >> > will happily overcommit knowing that all the memory (virtual + real)
>> >> > will never ever be needed at the same time. When all processes
>> >> > *do* attempt to run concurrently
>> >>
>> >> Um... run concurrently? you mean as opposed to being in swap? Because
>> >> that's where the "virtual" part normally comes in.
>> >>
>> >
>> > Don't attempt to be condescending; you aren't any good at it.
>>
>> Was I wrong?
>
> You weren't even in the same context. It's impossible to be
> wrong if you don't even understand the question.
Regardless of everything else, your use of the term "run concurrently"
was misleading - a program can be blocking/sleeping for hours, entirely
swapped out, and it _still_ ties up every single page of virtual memory
it has ever used; you seemed to suggest this was not the case. If this
wasn't the context you were talking about, your words were wrong for the
context you DID mean.
>> > See my reply to Barry Schwarz elsethread.
>>
>> Your only reply to him that I can see is the one that I, in turn replied
>> to. Right above this one.
>>
>> > goose,
>> > It's more common than you think.
>>
>> What is? A program never "releases" memory without ever using it, that'd
>> be silly.
>
> I've provided links to the threaded mailing-list discussion of this.
You have not. At least, not that I've been able to find. Perhaps
providing these links in an actual reply to one of my messages would be
more useful?
>> So basically it's gambling that program A will either
>> "release" some memory or die before program B attempts to write the
>> memory *that it just allocated* most likely in the next few seconds,
>> thus allowing it to thwart any code that might be in place for program
>> B to sanely handle the case where not as much memory as it wanted was
>> available.
>
> Yes. I think you get the picture now.
When making such a bet, how often does the system win? I'd guess the
odds are probably pretty slim.
goose
Random832 wrote:
I apologise; I stupidly assumed that everyone reading would be
aware of what "concurrently" meant in the context of overcommits.
Rereading it, I see that I was maybe a tad ambigous.
>
> >> > See my reply to Barry Schwarz elsethread.
> >>
> >> Your only reply to him that I can see is the one that I, in turn replied
> >> to. Right above this one.
> >>
> >> > goose,
> >> > It's more common than you think.
> >>
> >> What is? A program never "releases" memory without ever using it, that'd
> >> be silly.
> >
> > I've provided links to the threaded mailing-list discussion of this.
>
> You have not. At least, not that I've been able to find. Perhaps
> providing these links in an actual reply to one of my messages would be
> more useful?
They were in reply to Barry earlier. Nevertheless, if you really wanted
to you could have googled for (without the quotes) "FreeBSD overcommit
malloc"
and taken the second link to read the entire discussion (a pretty big
one, IIRC).
The first link will allow one to find out exactly *how* common this
technique
is. Like I said in a previous post, this is learnt by undergraduates
all over
the world every year - it was mentioned as a technique in my
undergraduate
OS design course.
>
> >> So basically it's gambling that program A will either
> >> "release" some memory or die before program B attempts to write the
> >> memory *that it just allocated* most likely in the next few seconds,
> >> thus allowing it to thwart any code that might be in place for program
> >> B to sanely handle the case where not as much memory as it wanted was
> >> available.
> >
> > Yes. I think you get the picture now.
>
> When making such a bet, how often does the system win? I'd guess the
> odds are probably pretty slim.
Well, if you bet that way, you might lose your money. FreeBSD has
always
(since I remember, anyway) had this, as well as all non System V
unixes.
Since FreeBSD (and other unixes) have had quite a good reputation for
robustness under load, I suspect that this works better than intuition
would
suggest.
The reason might have something to do with the fact that processes
allocate blocks bigger than they need *right now*, and only fill it up
to capacity later, if at all. A common method of using a dynamic
array is to double the size each time you reallocate it - you usually
end
up never accessing the last two-thirds of it under this method. More
than
onje regular here has suggested this method in the past.
goose,
Enough now, this is off-topic as it is, no need to drag it out
further.
Keith Thompson
> CBFalconer said:
>
>> there is good reason to correct and/or clarify statements,
>> whether or not the originator knows better.
>
> What was unclear about my previous contributions to this thread?
Here's the article you wrote, to which I posted my (attempted)
clarification:
| Random832 said:
|
| > 2006-12-12 <7KOdnVNe_ry...@bt.com>,
| > Richard Heathfield wrote:
| >> Under that circumstance, the compiler is obliged to assume that malloc
| >> returns int,
| >
| > Under which circumstance, in turn, the compiler is not "obliged" to
| > anything in particular at all.
|
| "If the expression that precedes the parenthesized argument list in a
| function call consists solely of an identifier, and if no declaration is
| visible for this identifier, the identifier is implicitly declared exactly
| as if, in the innermost block containing the function call, the declaration
|
| extern int identifier();
|
| appeared." (C89: 3.3.2.2)
Your citation of the standard appeared to me to be a refutation of
Random832's statement that
the compiler is not "obliged" to anything in particular at all.
In other words, you seemed (based only on that one followup, without
reference to previous articles in the thread) to be asserting that,
given:
/* Note: no "#include <stdlib.h>" */
int main(void)
{
int *p = (int*)malloc(sizeof(int));
}
the compiler is obliged to treat malloc() as a function returning int
(and the int result will be converted to int*). In fact, as we both
know but others may not, the call invokes undefined behavior, and the
implementation is therefore free to do anything it likes -- including
treating malloc() as a function returning void* if the implementer is
more interested in generating correct results for bad code than in
emitting good diatnostics.
The fact that the call invokes undefined behavior is obvious to both
of us, but possibly not to others. The fact that the undefined
behavior implies that the compiler is allowed to treat malloc() as a
function returning void* is sufficiently subtle that it seemed
plausible you might have missed it -- thus my clarification.
(In fact, if a compiler is clever enough to recognize the undefined
behavior because it know about "malloc" even without the header, it
had *better* issue a decent diagnostic rather than just quietly
generating code consistent with the user's presumed intent, but that's
a quality of implementation issue.)
Random832
Keith Thompson wrote:
> In other words, you seemed (based only on that one followup, without
> reference to previous articles in the thread) to be asserting that,
> given:
>
> /* Note: no "#include <stdlib.h>" */
> int main(void)
> {
> int *p = (int*)malloc(sizeof(int));
>}
>
> the compiler is obliged to treat malloc() as a function returning int
> (and the int result will be converted to int*). In fact, as we both
> know but others may not, the call invokes undefined behavior, and the
> implementation is therefore free to do anything it likes -- including
> treating malloc() as a function returning void* if the implementer is
> more interested in generating correct results for bad code than in
> emitting good diatnostics.
And, conversely, if the implementer is more interested in emitting good
diagnostics, the compiler can emit a diagnostic along the lines of
"warning: malloc used without including <stdlib.h> or a proper
declaration", and convert the line to a call to abort() (along with
a repetition of the diagnostic) so that if it is reached the program
terminates (unfortunately, an implementation is not conforming if it
refuses to translate this piece of crap.)
Harald van Dijk
> 2006-12-13 <ln4przb...@nuthaus.mib.org>,
> Keith Thompson wrote:
> > In other words, you seemed (based only on that one followup, without
> > reference to previous articles in the thread) to be asserting that,
> > given:
> >
> > /* Note: no "#include <stdlib.h>" */
> > int main(void)
> > {
> > int *p = (int*)malloc(sizeof(int));
> >}
> >
> > the compiler is obliged to treat malloc() as a function returning int
> > (and the int result will be converted to int*). In fact, as we both
> > know but others may not, the call invokes undefined behavior, and the
> > implementation is therefore free to do anything it likes -- including
> > treating malloc() as a function returning void* if the implementer is
> > more interested in generating correct results for bad code than in
> > emitting good diatnostics.
>
> And, conversely, if the implementer is more interested in emitting good
> diagnostics, the compiler can emit a diagnostic along the lines of
> "warning: malloc used without including <stdlib.h> or a proper
> declaration", and convert the line to a call to abort() (along with
> a repetition of the diagnostic) so that if it is reached the program
> terminates (unfortunately, an implementation is not conforming if it
> refuses to translate this piece of crap.)
The behaviour would be undefined for each run of the program. In that
case, an implementation is allowed to reject it. That applies even to
/* Note: no "#include <stdlib.h>" */
int main(void)
{
if(0) {
int *p = (int*)malloc(sizeof(int));
}
}
since the behaviour is undefined not just when malloc is called, but
when it is declared inappropriately as well. As IIRC Richard Heathfield
once pointed out, C90's equivalent of C99's 7.1.3 "Reserved
identifiers" does not forbid incompatible declarations, but it still
has the equivalent of C99's 6.2.7p2:
"All declarations that refer to the same object or function shall have
compatible type; otherwise, the behavior is undefined."
and malloc's implicit declaration is incompatible with that inside the
standard library itself.
In C99, by the way, rejection is allowed for more obvious reasons.
Keith Thompson
> goose said:
>
> <snip>
>>
>> Don't attempt to be condescending; you aren't any good at it.
>
> But *I* am. I can provide top-quality condescension at competitive rates.
> All he has to do is call.
Yes, I'm sure you can. No doubt your cute little condescending ways
are among the very very best in the whole wide world! I wouldn't
*dream* of trying to compete with you on that score.
(But my rates are excellent.)
Richard Heathfield
<snip>
> Your citation of the standard appeared to me to be a refutation of
> Random832's statement that
>
> the compiler is not "obliged" to anything in particular at all.
>
> In other words, you seemed (based only on that one followup, without
> reference to previous articles in the thread) to be asserting that,
> given:
>
> /* Note: no "#include <stdlib.h>" */
> int main(void)
> {
> int *p = (int*)malloc(sizeof(int));
> }
>
> the compiler is obliged to treat malloc() as a function returning int
And so it is.
> (and the int result will be converted to int*).
It would be reasonable for the compiler to generate code to provide such a
conversion, yes - although the resulting program's behaviour will be
undefined.
> In fact, as we both
> know but others may not, the call invokes undefined behavior,
Right, but by that stage the compiler's job is done and dusted. The call
happens at runtime, not compile time.
> and the
> implementation is therefore free to do anything it likes -- including
> treating malloc() as a function returning void* if the implementer is
> more interested in generating correct results for bad code than in
> emitting good diatnostics.
Very true, but only tenuously relevant, insofar as it is merely one possible
permutation in the lottery of undefined behaviour.
<snip>
Richard Heathfield
> Richard Heathfield <r...@see.sig.invalid> writes:
>> goose said:
>>
>> <snip>
>>>
>>> Don't attempt to be condescending; you aren't any good at it.
>>
>> But *I* am. I can provide top-quality condescension at competitive rates.
>> All he has to do is call.
>
> Yes, I'm sure you can. No doubt your cute little condescending ways
> are among the very very best in the whole wide world! I wouldn't
> *dream* of trying to compete with you on that score.
Don't give up. It normally takes many years of practice, but you might be
able to accelerate the process a little with an intensive course of study,
such as the one I run (for advanced students only, I'm afraid - there is a
tough entrance examination).
Incidentally, I find myself wondering whether "condescent" would be an
improvement on "condescension".
Joe Wright
> Richard Heathfield wrote:
>> goose said:
>>
>> <snip>
>>
>>> Don't attempt to be condescending; you aren't any good at it.
>> But *I* am. I can provide top-quality condescension at competitive
>> rates. All he has to do is call.
>
> I assume you have a suitable walk-in freezer and a humid household.
>
--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---
Giorgos Keramidas
>Barry Schwarz wrote:
><snipped>
>> or the operating system terminate it. On a typical desk top system,
>> the first umpteen calls will succeed and all subsequent calls will
>> fail. Since malloc fails gracefully (by returning NULL), these
>
> will happily overcommit knowing that all the memory (virtual + real)
> will never ever be needed at the same time. When all processes
> it, so all malloc calls *always* return a pointer to memory.
>
> kernel knob. If my memory serves me correctly, the concensus
> on clc the last time this came up was that such an implementation
> could not be considered conforming ...
<OT>
I can't recall if a kernel tunable for this ever existed, but malloc()
in FreeBSD has options to pre-zero or pre-set the allocated areas to a
magic value, which are mostly useful as debugging features. Quoting the
manpage of malloc(3) from FreeBSD 7.0-CURRENT (similar options exist in
the pre-7.0 implementation of malloc):
J Each byte of new memory allocated by malloc(), realloc() or
reallocf() will be initialized to 0xa5. All memory
returned by free(), realloc() or reallocf() will be
initialized to 0x5a. This is intended for debugging and
will impact performance negatively.
Z Each byte of new memory allocated by malloc(), realloc() or
reallocf() will be initialized to 0. Note that this
initialization only happens once for each byte, so
realloc() and reallocf() calls do not zero memory that was
previously allocated. This is intended for debugging and
will impact performance negatively.
A side-effect of these debugging features is that pages returned by the
kernel will be written to, which will force the 'write' part of ``copy
on write''. It's not really a tunable which can be considered an 100%
disabling of overcommit, but it's reasonably close.
</OT>