Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Request

2 views
Skip to first unread message

jacob navia

unread,
Jan 2, 2007, 6:30:00 PM1/2/07
to
As a consequence of a heavy discussion in another thread,
I wrote this program:
#include <stdlib.h>
int main(void)
{
char *p=calloc(65521,65552);
if (p)
printf("BUG!!!!\n");
}

The multiplication will give an erroneus result
in 32 bit size_t systems. I have tested this
in several systems with several compilers.

Maybe people with other compilers/implementations
would compile this program and send the results?

The results I have so far are:

LINUX
GCC 2.95
HAS THE BUG

GCC-4.02
NO BUGS

GCC-3.x
NO BUGS

WINDOWS
cygwin gcc 3.4.4
NO bugs
pelles c:
HAS THE BUG
Microsoft
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42
for 80x86
NO BUGS (If the multiplication overflows it will trap. Very good
behavior)

AIX
IBM-cc (compiling in 32 bit mode)
NO BUGS
gcc 4.0.0 (compiling in 32 bit mode)
NO BUGS


Thanks in advance for your cooperation

Ben Pfaff

unread,
Jan 2, 2007, 6:42:24 PM1/2/07
to
jacob navia <ja...@jacob.remcomp.fr> writes:

> LINUX
> GCC 2.95
> HAS THE BUG

GCC is not a complete C implementation. In particular, it lacks
a C library. Thus, it doesn't make sense to say that GCC 2.95
has a bug in its calloc function. In fact, when I run your test
program here, using GCC 2.95.4 on top of a Linux kernel, I don't
see the bug you claim to be present. This probably means that we
are using different C libraries.
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

Walter Roberson

unread,
Jan 2, 2007, 6:50:56 PM1/2/07
to
In article <459aeaf9$0$27394$ba4a...@news.orange.fr>,

jacob navia <ja...@jacob.remcomp.fr> wrote:
>As a consequence of a heavy discussion in another thread,
>I wrote this program:
>#include <stdlib.h>
>int main(void)
>{
> char *p=calloc(65521,65552);
> if (p)
> printf("BUG!!!!\n");
>}

You appear to be missing a prototype for printf(), and you
appear to be relying on C99 behaviour upon running off the
end of main().

>The multiplication will give an erroneus result
>in 32 bit size_t systems.

That may or may not be the case, but that isn't what you
are testing.

Your program would claim as a BUG any system
on which size_t was able to accomedate 65521 * 65552 and was
able to allocate that much memory.

The ability to allocate that much memory may depend upon the memory and
swap space installed in the system, and may depend upon any resource
usage limitations that have been put in place. Some of those limits
may be "soft" limits that can be removed with a simple command or two;
some of the limits might be "hard" limits imposed by the systems
administrator on systems that would have otherwise have sufficient
resources.

>I have tested this
>in several systems with several compilers.

>Maybe people with other compilers/implementations
>would compile this program and send the results?

SGI IRIX MipsPro 7.3.1.2:
With -o32 or -n32, size_t is 32 and
the implementation does not allocate the memory. With -64,
size_t is 64 and the particular system I'm running on
has enough resources that the system is able to allocate the
full extent of the memory.

According to your program, that means that the above compiler both
has and does not have the bug (it's the same compiler in all
the cases.)

If I were me, I would conclude that your methodology is flawed.
--
If you lie to the compiler, it will get its revenge. -- Henry Spencer

jacob navia

unread,
Jan 2, 2007, 7:04:12 PM1/2/07
to
Walter Roberson a écrit :
> In article <459aeaf9$0$27394$ba4a...@news.orange.fr>,

>
> You appear to be missing a prototype for printf(), and you
> appear to be relying on C99 behaviour upon running off the
> end of main().
>

Yes, add them at will.

>
>>The multiplication will give an erroneus result
>>in 32 bit size_t systems.
>
>
> That may or may not be the case, but that isn't what you
> are testing.
>
> Your program would claim as a BUG any system
> on which size_t was able to accomedate 65521 * 65552 and was
> able to allocate that much memory.
>

With a 32 bit size_t as I said, it would be interesting that
the system allocates a size_t of 33 bits...

I may be missing something... BUT!!!

> The ability to allocate that much memory may depend upon the memory and
> swap space installed in the system, and may depend upon any resource
> usage limitations that have been put in place. Some of those limits
> may be "soft" limits that can be removed with a simple command or two;
> some of the limits might be "hard" limits imposed by the systems
> administrator on systems that would have otherwise have sufficient
> resources.
>

No, this is impossible in 32 bit size_t systems!!!

> SGI IRIX MipsPro 7.3.1.2:
> With -o32 or -n32, size_t is 32 and
> the implementation does not allocate the memory.

Fine We can add that SGI IRIX has NOT the bug.

Thanks

> With -64,
> size_t is 64 and the particular system I'm running on
> has enough resources that the system is able to allocate the
> full extent of the memory.
>

Excuse me but you haven't a 32 bit size_t but a 64 bit
size_t when compiling in 64 bit mode. If you care to see my table
I compiled explicitely in 32 bit mode in AIX systems!!!

> According to your program, that means that the above compiler both
> has and does not have the bug (it's the same compiler in all
> the cases.)
>

It is not the same size_t!!!

I said:
32 bit size_t!!!

> If I were me, I would conclude that your methodology is flawed.

I am glad that you are you and I am me

:-)

What would happen If i were you? I would try to read
what I am writi,ng and not trying to find inexistent
bugs

Thanks for your input in any case Walter.

Walter Roberson

unread,
Jan 2, 2007, 7:18:47 PM1/2/07
to
In article <459af2fc$0$5080$ba4a...@news.orange.fr>,
jacob navia <ja...@jacob.remcomp.fr> wrote:
>Walter Roberson a écrit :

>>>The multiplication will give an erroneus result
>>>in 32 bit size_t systems.

>> That may or may not be the case, but that isn't what you
>> are testing.

>> Your program would claim as a BUG any system
>> on which size_t was able to accomedate 65521 * 65552 and was
>> able to allocate that much memory.

>With a 32 bit size_t as I said, it would be interesting that
>the system allocates a size_t of 33 bits...

>I may be missing something... BUT!!!

You *are* missing something. What you wrote was that,

"The multiplication will give an erroneus result in 32 bit size_t systems."

You did NOT, however, restrict the testing to systems with size_t
of 32 bits: you merely made an assertion about what would happen
on systems on which size_t is 32 bits. The proof is in the code,
and your code declares as a bug any system and option and
environment combination which is able to allocate the required amount
of memory.

> If you care to see my table
> I compiled explicitely in 32 bit mode in AIX systems!!!

Irrelevant, since you did not restrict the scope of the test
to 32 bit mode.
--
"law -- it's a commodity"
-- Andrew Ryan (The Globe and Mail, 2005/11/26)

user923005

unread,
Jan 2, 2007, 7:26:05 PM1/2/07
to
jacob navia wrote:
> No, this is impossible in 32 bit size_t systems!!!

Why? The calloc() call has two parameters. There is no reason that
the total allocation cannot exceed the value of a size_t.

BTW, there are definitely 32 bit systems that will allocate memory in
excess of 32 bits. For example:
http://support.microsoft.com/kb/274750

>From this, I do not see any restriction that the allocation must fail
if nmemb*size > SIZE_MAX:
"7.20.3.1 The calloc function
Synopsis
1 #include <stdlib.h>
void *calloc(size_t nmemb, size_t size);
Description
2 The calloc function allocates space for an array of nmemb objects,
each of whose size
is size. The space is initialized to all bits zero.252)
Returns
3 The calloc function returns either a null pointer or a pointer to the
allocated space."

Spiros Bousbouras

unread,
Jan 2, 2007, 7:32:44 PM1/2/07
to
jacob navia wrote:
> As a consequence of a heavy discussion in another thread,
> I wrote this program:
> #include <stdlib.h>
> int main(void)
> {
> char *p=calloc(65521,65552);
> if (p)
> printf("BUG!!!!\n");
> }
>

What bug are you claiming exists ? If you're
saying that only 65296 bytes have been allocated
how do you know ?

> The multiplication will give an erroneus result
> in 32 bit size_t systems. I have tested this
> in several systems with several compilers.

How do you know what the calloc implementation does
upon receiving the arguments 65521 and 65552 ? It is
reasonable to assume that it multiplies them but how
do you know that the result of the multiplication is limited
to 32 bits ? I don't see what relevance the width of size_t
has.

> Maybe people with other compilers/implementations
> would compile this program and send the results?

I can't be bothered to be honest because I don't see the
point. As far as I can see , even if calloc returns a non NULL
pointer you can't know how much memory has been allocated
unless you have seen the code of the calloc implementation.
Note that even if you know for sure that the system cannot
allocate 65521*65552 bytes and calloc returns non NULL you
still cannot be sure that it is calloc which has the bug. It may
be that calloc uses some platform specific system call to get
the memory and it is the system call which has the bug.

> RESULTS OF TEST SNIPPED

jacob navia

unread,
Jan 2, 2007, 7:42:19 PM1/2/07
to
Walter Roberson a écrit :

>
> You *are* missing something. What you wrote was that,
>
> "The multiplication will give an erroneus result in 32 bit size_t systems."
>
> You did NOT, however, restrict the testing to systems with size_t
> of 32 bits

I thought that that restriction would be evident from the restriction
of size_t to 32 bits.

OK This is a misunderstanding then. The test is intended only
for 32 bit systems

Updated version:

#include <stdlib.h>
#include <limits.h>
#include <stdio.h> // for printf
int main(void)
{
int s = sizeof(size_t)*CHAR_BIT;
if (s == 32 && NULL != calloc(65521,65552))
printf("BUG!!!!\n");
return 0; // For C89 compilers
}

Jens Thoms Toerring

unread,
Jan 2, 2007, 7:44:46 PM1/2/07
to
jacob navia <ja...@jacob.remcomp.fr> wrote:
> As a consequence of a heavy discussion in another thread,
> I wrote this program:
> #include <stdlib.h>
> int main(void)
> {
> char *p=calloc(65521,65552);
> if (p)
> printf("BUG!!!!\n");
> }

> The multiplication will give an erroneus result
> in 32 bit size_t systems.

I think that assuming that a 32 bit multiplication is the sole
reason is a bit premature. At least I don't see convincing evi-
dence of that.

> I have tested this
> in several systems with several compilers.

> Maybe people with other compilers/implementations
> would compile this program and send the results?

> The results I have so far are:

> LINUX
> GCC 2.95
> HAS THE BUG

> GCC-4.02
> NO BUGS

> GCC-3.x
> NO BUGS

I have 3.3.1 and get the opposite result (with no overcommitment
enabled). Things get even more interesting if I try to print out
the pointers value (of course after including <stdio.h>;-): While
the test of p tells that it's not zero ("BUG!!!!" isn't printed)
a line like

printf( "%p\n", ( void * ) p );

gives me "(nil)", the same result as if I had a NULL pointer. And,
if, on top of it, I try to assign a value to the memory (even the
very first byte of it) then a segmentation fault is the result.

I guess that indicates that a bit more than a simple unsigned 32
bit multiplication that overflows and isn't detected in time is
going on behind the scenes...
Regards, Jens
--
\ Jens Thoms Toerring ___ j...@toerring.de
\__________________________ http://toerring.de

Spiros Bousbouras

unread,
Jan 2, 2007, 7:56:43 PM1/2/07
to

You got it backwards , if "BUG" isn't printed then it is
a NULL pointer.

> a line like
>
> printf( "%p\n", ( void * ) p );
>
> gives me "(nil)", the same result as if I had a NULL pointer. And,
> if, on top of it, I try to assign a value to the memory (even the
> very first byte of it) then a segmentation fault is the result.

Not surprising since you try dereferencing a NULL pointer.

jacob navia

unread,
Jan 2, 2007, 7:59:13 PM1/2/07
to
user923005 a écrit :

> jacob navia wrote:
>
>>No, this is impossible in 32 bit size_t systems!!!
>
>
> Why? The calloc() call has two parameters. There is no reason that
> the total allocation cannot exceed the value of a size_t.
>
> BTW, there are definitely 32 bit systems that will allocate memory in
> excess of 32 bits. For example:
> http://support.microsoft.com/kb/274750
>

This is irrelevant. Those links refer as to how
to use certain flags in the boot.ini to allow
*3GB* if user space. (!!!!!!!)

And 3GB is well below a 32 bit size_t.

With a 32 bit size_t and pointer size how can you
address more than 4GB?

But this is a well known tactic.

Since I present the code, I do the effort of making all those
measurements, it is so easy to just sit by and say

"A system with 32 bit size_t could allocate more than 4GB
memory. A system like that COULD exist."

So WHAT?

Can you please tell me where that system is????

If not, just do not participate. Thanks for your
correction. A system where this test gives a false
positive COULD exist. It is conceivable.

Spiros Bousbouras

unread,
Jan 2, 2007, 8:00:03 PM1/2/07
to
jacob navia wrote:
> Walter Roberson a écrit :
> >
> > You *are* missing something. What you wrote was that,
> >
> > "The multiplication will give an erroneus result in 32 bit size_t systems."
> >
> > You did NOT, however, restrict the testing to systems with size_t
> > of 32 bits
>
> I thought that that restriction would be evident from the restriction
> of size_t to 32 bits.
>
> OK This is a misunderstanding then. The test is intended only
> for 32 bit systems

What do you mean by 32 bit system ? One that cannot
address more than 32 bits of physical memory ? What
if the system is connected to a hard disk which has more
than 2**32 bytes of space. Surely then it would be possible
to somehow address the whole of it even if it meant using
special libraries to do arithmetic on more than 32 bits.

jacob navia

unread,
Jan 2, 2007, 8:04:10 PM1/2/07
to
Spiros Bousbouras a écrit :

> jacob navia wrote:
>
>>Walter Roberson a écrit :
>>
>>>You *are* missing something. What you wrote was that,
>>>
>>>"The multiplication will give an erroneus result in 32 bit size_t systems."
>>>
>>>You did NOT, however, restrict the testing to systems with size_t
>>>of 32 bits
>>
>>I thought that that restriction would be evident from the restriction
>>of size_t to 32 bits.
>>
>>OK This is a misunderstanding then. The test is intended only
>>for 32 bit systems
>
>
> What do you mean by 32 bit system ? One that cannot
> address more than 32 bits of physical memory ?

where sizeof(void *) == sizeof(size_t) == 4 and CHAR_BIT is 8


> What
> if the system is connected to a hard disk which has more
> than 2**32 bytes of space. Surely then it would be possible
> to somehow address the whole of it even if it meant using
> special libraries to do arithmetic on more than 32 bits.
>

File pointers use 64 bit integers this days when it is
impossible to buy a hard disk of 32 GB only :-)

This makes the system not 64 bits!

jacob navia

unread,
Jan 2, 2007, 8:06:08 PM1/2/07
to
Jens Thoms Toerring a écrit :
[snip]

Please see the answer of Mr Spiros Bousburas in this same thread
and try to understand what I want before jumping to conclusions.

Thanks

P.S. if you do not want to participate it is OK but please do not
send noise.

Jens Thoms Toerring

unread,
Jan 2, 2007, 8:08:06 PM1/2/07
to

Grrr;-) Right you are - I played around with it too much (with
disruptions by my cat), loosing track of what I had been trying
out. Time to go to bed...

jacob navia

unread,
Jan 2, 2007, 8:11:54 PM1/2/07
to
Spiros Bousbouras a écrit :

> jacob navia wrote:
>
>>As a consequence of a heavy discussion in another thread,
>>I wrote this program:
>>#include <stdlib.h>
>>int main(void)
>>{
>> char *p=calloc(65521,65552);
>> if (p)
>> printf("BUG!!!!\n");
>>}
>>
>
>
> What bug are you claiming exists ? If you're
> saying that only 65296 bytes have been allocated
> how do you know ?

if size_t is 32 bits, it would be highly surprising that
an allocator returns an object of size 33 bits at least.

You are just saying here that such a system *could* exist.
This doesn't make it any more real.

OK. In those hypothetical systems my test would fail.

>
>
>>The multiplication will give an erroneus result
>>in 32 bit size_t systems. I have tested this
>>in several systems with several compilers.
>
>
> How do you know what the calloc implementation does
> upon receiving the arguments 65521 and 65552 ? It is
> reasonable to assume that it multiplies them but how
> do you know that the result of the multiplication is limited
> to 32 bits ? I don't see what relevance the width of size_t
> has.
>

because if an object of size > 4GB could exist, sizeof
(that returns a size_t) would not work.

>
>>Maybe people with other compilers/implementations
>>would compile this program and send the results?
>
>
> I can't be bothered to be honest because I don't see the
> point.

You can't be bothered to be honest?

I hope you mean

To be honest, I can't be bothered...

:-)

BUG!!!!

> As far as I can see , even if calloc returns a non NULL
> pointer you can't know how much memory has been allocated
> unless you have seen the code of the calloc implementation.
> Note that even if you know for sure that the system cannot
> allocate 65521*65552 bytes and calloc returns non NULL you
> still cannot be sure that it is calloc which has the bug. It may
> be that calloc uses some platform specific system call to get
> the memory and it is the system call which has the bug.

Excuse me but what calloc does is not measured. I just
test the result. If it is the system's fault or whatever
this is not relevant.

Spiros Bousbouras

unread,
Jan 2, 2007, 8:21:45 PM1/2/07
to
jacob navia wrote:
> user923005 a écrit :
> > jacob navia wrote:
> >
> >>No, this is impossible in 32 bit size_t systems!!!
> >
> >
> > Why? The calloc() call has two parameters. There is no reason that
> > the total allocation cannot exceed the value of a size_t.
> >
> > BTW, there are definitely 32 bit systems that will allocate memory in
> > excess of 32 bits. For example:
> > http://support.microsoft.com/kb/274750
> >
>
> This is irrelevant. Those links refer as to how
> to use certain flags in the boot.ini to allow
> *3GB* if user space. (!!!!!!!)
>
> And 3GB is well below a 32 bit size_t.
>
> With a 32 bit size_t and pointer size how can you
> address more than 4GB?
>
> But this is a well known tactic.

A tactic to achieve what ?

>
> Since I present the code, I do the effort of making all those
> measurements, it is so easy to just sit by and say
>
> "A system with 32 bit size_t could allocate more than 4GB
> memory. A system like that COULD exist."
>
> So WHAT?

I don't know "so WHAT". You see the problem here is
that you have not explained what your point is. In
addition to making the effort (actually not that much
effort it seems to me) of making the measurements, it
would have been very nice if you had explained what point
the measurements are meant to illustrate/investigate.

I'll take a guess: your point is that calloc should check
whether the multiplication (if that's what it does) yields
a value so big that the system cannot possibly have that
much memory. Perhaps it should , or perhaps the programmer
should , it seems like a matter of taste to me and I won't
enter into an endless argument about it. But the main problem
as I pointed out in a previous post is that your experiments
even done on a 32 bit system (whatever that means) don't
tell us for sure what checks the calloc implementations on
those systems do. As I said you would need to check the code
of calloc to verify that.

> Can you please tell me where that system is????
>
> If not, just do not participate. Thanks for your
> correction. A system where this test gives a false
> positive COULD exist. It is conceivable.

You cannot set arbitrary rules about when people
cannot participate in the threads you start. If your
point in the above paragraph (see ? I have to *guess*
again what your point is because you don't spell it out)
is that people should only participate if they're going
to address your main point in the thread, then I can kind
of see this as a reasonable requirement but you need to
explain first what your point is and you haven't done that.

user923005

unread,
Jan 2, 2007, 8:24:33 PM1/2/07
to
jacob navia wrote:
> user923005 a écrit :
> > jacob navia wrote:
> >
> >>No, this is impossible in 32 bit size_t systems!!!
> >
> >
> > Why? The calloc() call has two parameters. There is no reason that
> > the total allocation cannot exceed the value of a size_t.
> >
> > BTW, there are definitely 32 bit systems that will allocate memory in
> > excess of 32 bits. For example:
> > http://support.microsoft.com/kb/274750
> >
>
> This is irrelevant. Those links refer as to how
> to use certain flags in the boot.ini to allow
> *3GB* if user space. (!!!!!!!)

That is either 8 GB user space or 32 GB, not 3 GB. I suggest you read
again -- this time with comprehension enabled... Here is a quote from
that page that you may find helpful:
"SQL Server 2000 Enterprise Edition introduces support for the use of
Microsoft Windows 2000 Address Windowing Extensions (AWE) to address
approximately 8 GB of memory for instances that run on Microsoft
Windows 2000 Advanced Server, and approximately 32 GB for instances
that run on Microsoft Windows 2000 Datacenter."

P.S.
These operating systems are 32 bits.

> And 3GB is well below a 32 bit size_t.

You seem to have a bit of a comprehension problem.

> With a 32 bit size_t and pointer size how can you
> address more than 4GB?

There are lots of ways to do it.

> But this is a well known tactic.
>
> Since I present the code, I do the effort of making all those
> measurements, it is so easy to just sit by and say
>
> "A system with 32 bit size_t could allocate more than 4GB
> memory. A system like that COULD exist."
>
> So WHAT?

So that is all that is needed to show that your test is flawed.

> Can you please tell me where that system is????

I mentioned one above that uses 32 bit addressing lines and yet can
address 32 GB or RAM. The way it does it is with memory models at the
low level. Just like back in the bad old days when MS-DOS programmers
had to model with large, medium or small memory models because of 16
bit addressing, in a similar way you can use 32 bit segments and
offsets to address (potentially) 64 GB of RAM or any other scheme you
like.

> If not, just do not participate. Thanks for your
> correction. A system where this test gives a false
> positive COULD exist. It is conceivable.

Your test has no merit that I can imagine.
Systems that will allocate 2^64 pages of ram could return a failure
result if the user space is less than 4GB. So what have we learned
from your code? Nothing.

jacob navia

unread,
Jan 2, 2007, 8:26:25 PM1/2/07
to
Spiros Bousbouras a écrit :

My objective is to

1) make a list of all the buggy systems that
is here in clc and everybody can see.

2) Make people aware of this bug in their implementations.
(if any)

jacob navia

unread,
Jan 2, 2007, 8:27:05 PM1/2/07
to
P.S.
I forgot to spell out clearly the objective of this tests.

Spiros Bousbouras

unread,
Jan 2, 2007, 8:31:30 PM1/2/07
to
jacob navia wrote:
> Spiros Bousbouras a écrit :
> > jacob navia wrote:
> >
> >>Walter Roberson a écrit :
> >>
> >>>You *are* missing something. What you wrote was that,
> >>>
> >>>"The multiplication will give an erroneus result in 32 bit size_t systems."
> >>>
> >>>You did NOT, however, restrict the testing to systems with size_t
> >>>of 32 bits
> >>
> >>I thought that that restriction would be evident from the restriction
> >>of size_t to 32 bits.
> >>
> >>OK This is a misunderstanding then. The test is intended only
> >>for 32 bit systems
> >
> >
> > What do you mean by 32 bit system ? One that cannot
> > address more than 32 bits of physical memory ?
>
> where sizeof(void *) == sizeof(size_t) == 4 and CHAR_BIT is 8

Since calloc returns pointer to void I can see why
sizeof(void *) is relevant but I still don't see the
connection between sizeof(size_t) and what calloc
returns or allocates.

jacob navia

unread,
Jan 2, 2007, 8:34:48 PM1/2/07
to
user923005 a écrit :

> That is either 8 GB user space or 32 GB, not 3 GB. I suggest you read
> again -- this time with comprehension enabled... Here is a quote from
> that page that you may find helpful:
> "SQL Server 2000 Enterprise Edition introduces support for the use of
> Microsoft Windows 2000 Address Windowing Extensions (AWE) to address
> approximately 8 GB of memory for instances that run on Microsoft
> Windows 2000 Advanced Server, and approximately 32 GB for instances
> that run on Microsoft Windows 2000 Datacenter."
>
> P.S.
> These operating systems are 32 bits.
>

So what?

I do not talk about SQL server!

I am talking about calloc, quite a different beast.

Maybe SQL server can allocate a lot of memory but
this is not relevant to calloc I am sure.

>>With a 32 bit size_t and pointer size how can you
>>address more than 4GB?
>
>
> There are lots of ways to do it.
>

No. If you mean using segmented 32:48 bit pointers they
are no longer 32 bit!


> I mentioned one above that uses 32 bit addressing lines and yet can
> address 32 GB or RAM. The way it does it is with memory models at the
> low level. Just like back in the bad old days when MS-DOS programmers
> had to model with large, medium or small memory models because of 16
> bit addressing, in a similar way you can use 32 bit segments and
> offsets to address (potentially) 64 GB of RAM or any other scheme you
> like.
>

But those aren't 32 bit pointers but 48 bit pointers.


>
>>If not, just do not participate. Thanks for your
>>correction. A system where this test gives a false
>>positive COULD exist. It is conceivable.
>
>
> Your test has no merit that I can imagine.

OK


> Systems that will allocate 2^64 pages of ram could return a failure
> result if the user space is less than 4GB. So what have we learned
> from your code? Nothing.

OK

Thanks for your input

Richard Tobin

unread,
Jan 2, 2007, 8:38:29 PM1/2/07
to
In article <1167783965....@v33g2000cwv.googlegroups.com>,
user923005 <dco...@connx.com> wrote:

>BTW, there are definitely 32 bit systems that will allocate memory in
>excess of 32 bits. For example:

There are many 32-bit systems that allow more than 2^32 bytes of
memory to be used, but none that I know of that allow a block that big
to be accessed through calls to C standard allocation functions.

>http://support.microsoft.com/kb/274750

And as far as I can see, this is not a counter-example.

>Why? The calloc() call has two parameters. There is no reason that
>the total allocation cannot exceed the value of a size_t.

Obviously if that were the case, pointers would have to be larger than
32 bits. Even if Jacob is not precise, he obviously means systems
with 32-bit pointers. Constantly attacking for imprecision, when he's
trying to make a serious point about faulty implementations, seems
pointless.

-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.

jacob navia

unread,
Jan 2, 2007, 8:45:18 PM1/2/07
to
Richard Tobin a écrit :

> In article <1167783965....@v33g2000cwv.googlegroups.com>,
> user923005 <dco...@connx.com> wrote:
>
>
>>BTW, there are definitely 32 bit systems that will allocate memory in
>>excess of 32 bits. For example:
>
>
> There are many 32-bit systems that allow more than 2^32 bytes of
> memory to be used, but none that I know of that allow a block that big
> to be accessed through calls to C standard allocation functions.
>
>
>>http://support.microsoft.com/kb/274750
>
>
> And as far as I can see, this is not a counter-example.
>

SQL server surely doesn't use calloc for allocating 32GB of
RAM, since it needs 48 bit pointers (16:32 pointers with
a segmented architecture like in the msdos days of 16:16)

>
>>Why? The calloc() call has two parameters. There is no reason that
>>the total allocation cannot exceed the value of a size_t.
>
>
> Obviously if that were the case, pointers would have to be larger than
> 32 bits. Even if Jacob is not precise, he obviously means systems
> with 32-bit pointers. Constantly attacking for imprecision, when he's
> trying to make a serious point about faulty implementations, seems
> pointless.
>

Thanks. this is precisely what I meant.

sizeof returns a size_t. If an object larger than 4GB
could exist then size_t can't be 32 bits.

Spiros Bousbouras

unread,
Jan 2, 2007, 8:46:44 PM1/2/07
to
jacob navia wrote:
> Spiros Bousbouras a écrit :
> > jacob navia wrote:
> >
> >>As a consequence of a heavy discussion in another thread,
> >>I wrote this program:
> >>#include <stdlib.h>
> >>int main(void)
> >>{
> >> char *p=calloc(65521,65552);
> >> if (p)
> >> printf("BUG!!!!\n");
> >>}
> >>
> >
> >
> > What bug are you claiming exists ? If you're
> > saying that only 65296 bytes have been allocated
> > how do you know ?
>
> if size_t is 32 bits, it would be highly surprising that
> an allocator returns an object of size 33 bits at least.
>
> You are just saying here that such a system *could* exist.
> This doesn't make it any more real.
>
> OK. In those hypothetical systems my test would fail.

But how do you know that the systems that you tried
are not one of those "hypothetical" systems ?

> >>The multiplication will give an erroneus result
> >>in 32 bit size_t systems. I have tested this
> >>in several systems with several compilers.
> >
> >
> > How do you know what the calloc implementation does
> > upon receiving the arguments 65521 and 65552 ? It is
> > reasonable to assume that it multiplies them but how
> > do you know that the result of the multiplication is limited
> > to 32 bits ? I don't see what relevance the width of size_t
> > has.
> >
>
> because if an object of size > 4GB could exist, sizeof
> (that returns a size_t) would not work.

You could not declare an array which has size > 4GB
but it doesn't necessarily impose restrictions on what
calloc can return. But I agree it would be surprising.

>
> >
> >>Maybe people with other compilers/implementations
> >>would compile this program and send the results?
> >
> >
> > I can't be bothered to be honest because I don't see the
> > point.
>
> You can't be bothered to be honest?
>
> I hope you mean
>
> To be honest, I can't be bothered...
>
> :-)

I meant "I can't be bothered , to be honest , ..."
;-)


>
> BUG!!!!
>
> > As far as I can see , even if calloc returns a non NULL
> > pointer you can't know how much memory has been allocated
> > unless you have seen the code of the calloc implementation.
> > Note that even if you know for sure that the system cannot
> > allocate 65521*65552 bytes and calloc returns non NULL you
> > still cannot be sure that it is calloc which has the bug. It may
> > be that calloc uses some platform specific system call to get
> > the memory and it is the system call which has the bug.
>
> Excuse me but what calloc does is not measured. I just
> test the result. If it is the system's fault or whatever
> this is not relevant.

This answers my second point but not my first , that your
test doesn't tell you for sure how much memory has been
allocated.

Richard Tobin

unread,
Jan 2, 2007, 8:54:44 PM1/2/07
to
In article <459aeaf9$0$27394$ba4a...@news.orange.fr>,
jacob navia <ja...@jacob.remcomp.fr> wrote:

>Maybe people with other compilers/implementations
>would compile this program and send the results?

To my surprise, it has the bug on MacOS X Tiger with gcc 4.0.1 (x86)
and 4.0.0 (ppc). It also has the bug on Dragonfly 1.2 with gcc 2.95,
but that's a rather out-of-date system. It behaves correctly on
FreeBSD 6.1 x86 with gcc 3.4.4. I have a vague recollection of seeing
discussions about fixing this very problem in *BSD within the last
year or so.

In all cases I think the version of gcc is irrelevant; rather it is
the version of the C library that matter and the OS version determines
that. (The situation is likely to be more complicated for Linux,
where I believe people mix and match kernel and glibc versions.) All
these systems have sizeof(char *) == 4, sizeof(size_t) == 4, and
CHAR_BIT == 8.

Spiros Bousbouras

unread,
Jan 2, 2007, 8:54:37 PM1/2/07
to
Richard Tobin wrote:
> In article <1167783965....@v33g2000cwv.googlegroups.com>,
> user923005 <dco...@connx.com> wrote:
> >Why? The calloc() call has two parameters. There is no reason that
> >the total allocation cannot exceed the value of a size_t.
>
> Obviously if that were the case, pointers would have to be larger than
> 32 bits. Even if Jacob is not precise, he obviously means systems
> with 32-bit pointers. Constantly attacking for imprecision, when he's
> trying to make a serious point about faulty implementations, seems
> pointless.

Well it wasn't obvious to me from his first post. Even
if some people here were out to get him, so to speak, he
could still try to offer them less possibilities by being
precise and clear.

Richard Tobin

unread,
Jan 2, 2007, 8:57:37 PM1/2/07
to
In article <459b0aad$0$27382$ba4a...@news.orange.fr>,
jacob navia <ja...@jacob.remcomp.fr> wrote:

>sizeof returns a size_t. If an object larger than 4GB
>could exist then size_t can't be 32 bits.

To be pedantic, that doesn't quite follow. You can't apply sizeof to
the block returned by calloc(). I'm not sure whether you can legally
have a system in which calloc() can return an object with more bytes
than size_t can count.

Spiros Bousbouras

unread,
Jan 2, 2007, 9:16:28 PM1/2/07
to

Ok , now we're getting somewhere.

So the central question is:

If some code contains a call of the form calloc(n,m)
where n*m > 2**( sizeof(void*) * CHAR_BIT) then is it
the obligation of the calloc implementation to detect
this and return NULL ?

If the answer to the question is yes then you can say
that the implementations which don't do it are buggy.

>From the practical point of view , if I ever needed to
write something like calloc(n,m) where n*m >= 2**31
on a 32 bits system , alarm bells would start to ring
in my mind and I would wonder if it would work. I
would probably write some small test programme to
see if the allocation would succeed. I'm far from infallible
but my view is that a competent C programmer should
be alert to such issues and not rely on protection from
the calloc implementation *even* if the standard demands
that calloc detects this sort of thing. That's because this
is the kind of thing that an implementor can get wrong.
So I would write a test programme which not only tests
that the allocation succeeded but also assign some random
values to the allocated memory and make sure that everything
went ok.

Keith Thompson

unread,
Jan 2, 2007, 9:29:38 PM1/2/07
to
jacob navia <ja...@jacob.remcomp.fr> writes:
[...]

> But this is a well known tactic.
>
> Since I present the code, I do the effort of making all those
> measurements, it is so easy to just sit by and say
>
> "A system with 32 bit size_t could allocate more than 4GB
> memory. A system like that COULD exist."
>
> So WHAT?
>
> Can you please tell me where that system is????
>
> If not, just do not participate. Thanks for your
> correction. A system where this test gives a false
> positive COULD exist. It is conceivable.

So you acknowledge that a false positive could exist, but when someone
points it out you ask him not to participate in the discussion.

Try being just a little bit less arrogant.

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

user923005

unread,
Jan 2, 2007, 9:33:37 PM1/2/07
to

His code does not demonstrate anything useful.
For instance, it does not show how big a size_t can be or how large of
an allocation is possible or anything of the sort.

Imagine the following:
void *p =calloc(128000, 128000);

Suppose that under the cover, the allocation created an array of 128000
void pointers.
Then, it allocated 128000 separate 128000 byte objects and pointed
those pointers to the objects.
Then it memset each of the 128000 distinct 128000 byte objects with
zero.
We do not need greater than 4GB to cover the stride of the pointer
list.
We do not need greater than 4 GB for any individual object.
Would that implementation be conforming?

We might also use a hash map to associate pointers and allocated ram.

We might allocate in blocksizes of more than one byte. We could use
(for instance) 16 byte pages and still address 4GB*16 bytes with an
unsigned 32 bit integer.

We have lots of 64 bit computers here where I work. I can calloc()
more than 4 GB as su, but if I log on as some other user, I cannot
allocate more than (e.g.) 100 MB. What would his program tell you?

I don't think it shows what he thinks it shows and if it did the way he
thinks it should it still woudn't.

IMO-YMMV.

David T. Ashley

unread,
Jan 2, 2007, 10:32:51 PM1/2/07
to
"jacob navia" <ja...@jacob.remcomp.fr> wrote in message
news:459aeaf9$0$27394$ba4a...@news.orange.fr...

> As a consequence of a heavy discussion in another thread,
> I wrote this program:
> #include <stdlib.h>
> int main(void)
> {
> char *p=calloc(65521,65552);
> if (p)
> printf("BUG!!!!\n");
> }

My system (Red Hat Enterprise Linux 4.X with all errata applied) reports no
bug. The details are below.

Please note that I have subtly modified your test program by adding an
"else" clause. I don't think you'll object.

It isn't immediately clear to me why the call has to fail. 2^16 * 2^16 is
2^32 (4 gigabytes). My system has more virtual memory than that.

Dave.
----------

[nouser@pamc ~]$ cat test2.c
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
char *p=calloc(65521,65552);
if (p)
printf("BUG!!!!\n");

else
printf("NO BUG!!!!\n");
}
[nouser@pamc ~]$ gcc test2.c
[nouser@pamc ~]$ ./a.out
NO BUG!!!!
[nouser@pamc ~]$ rpm -q -a|grep gcc
gcc-c++-3.4.6-3
gcc-objc-3.4.6-3
gcc-3.4.6-3
gcc-java-3.4.6-3
compat-gcc-32-3.2.3-47.3
libgcc-3.4.6-3
gcc-gnat-3.4.6-3
compat-libgcc-296-2.96-132.7.2
gcc-g77-3.4.6-3
[nouser@pamc ~]$


Keith Thompson

unread,
Jan 2, 2007, 10:38:14 PM1/2/07
to
"Spiros Bousbouras" <spi...@gmail.com> writes:
> jacob navia wrote:
>> P.S.
>> I forgot to spell out clearly the objective of this tests.
>>
>> My objective is to
>>
>> 1) make a list of all the buggy systems that
>> is here in clc and everybody can see.
>>
>> 2) Make people aware of this bug in their implementations.
>> (if any)
>
> Ok , now we're getting somewhere.
>
> So the central question is:
>
> If some code contains a call of the form calloc(n,m)
> where n*m > 2**( sizeof(void*) * CHAR_BIT) then is it
> the obligation of the calloc implementation to detect
> this and return NULL ?
>
> If the answer to the question is yes then you can say
> that the implementations which don't do it are buggy.
[...]

I won't try to speak for jacob, but there is a valid point that's
similar to what he's talking about.

I'll use the same notation I introduced here recently: an expression
enclosed in "<<" and ">>" is intended to be interpreted as if all
operations were performed over the full infinite set of integers, with
no overflow or wraparound.

If some code contains a call of the form calloc(n, m), then the system
should either successfully allocate <<n*m>> bytes or return a null
pointer.

If <<n*m>> exceeds SIZE_MAX, and the system is actually able to
allocate that many bytes, that's ok.

There is a bug that some systems seem to have, where calloc()
internally multiplies its two parameters yielding a result of type
size_t, and ignores the possibility that the result could wrap around.
On such systems, for certain values of n and m, a call calloc(n, m)
might successfully allocate <<(n * m) % SIZE_MAX>> bytes, which is
less than <<n * m>>.

jacob's test program (in either version) does not detect this bug with
100% reliability, but if it reports "BUG!!!!" on a given system, it's
probably a good idea to investigate further.

There's a related potential bug, where something like
p = malloc(COUNT * sizeof *p);
can successfully allocate fewer bytes than intended, but in this case
the bug is in the user code; there's not really anything the malloc()
implementation can do about it.

David T. Ashley

unread,
Jan 2, 2007, 10:40:29 PM1/2/07
to
"David T. Ashley" <d...@e3ft.com> wrote in message
news:8cOdneCGIKdOvgbY...@giganews.com...

>
> It isn't immediately clear to me why the call has to fail. 2^16 * 2^16 is
> 2^32 (4 gigabytes). My system has more virtual memory than that.

Well, just out of curiousity, I tried it out to see what the largest
approximate value is. Results below.

54135^2 is going to be on the order of 2.5G. That is a pretty fair hunk of
memory.

---------

[nouser@pamc ~]$ cat test3.c
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
char *p;
int i;

for (i=65535; i>0; i--)
{
if (p = calloc(i,i))
{
printf("%d is apparently the largest integer that will succeed.\n",
i);
break;
}
}
}
[nouser@pamc ~]$ gcc test3.c
[nouser@pamc ~]$ ./a.out
54135 is apparently the largest integer that will succeed.
[nouser@pamc ~]$


Keith Thompson

unread,
Jan 2, 2007, 10:43:00 PM1/2/07
to
ric...@cogsci.ed.ac.uk (Richard Tobin) writes:
> In article <459b0aad$0$27382$ba4a...@news.orange.fr>,
> jacob navia <ja...@jacob.remcomp.fr> wrote:
>
>>sizeof returns a size_t. If an object larger than 4GB
>>could exist then size_t can't be 32 bits.
>
> To be pedantic, that doesn't quite follow. You can't apply sizeof to
> the block returned by calloc().

Agreed.

> I'm not sure whether you can legally
> have a system in which calloc() can return an object with more bytes
> than size_t can count.

I think you can, but such a system would have to have pointers bigger
than size_t. calloc() allocates an object; each byte of that object
must have a unique address of type char* or void*. Making size_t
smaller than void* is probably legal, but I can't think of a good
reason to do it (perhaps the DS9K does this).

Keith Thompson

unread,
Jan 2, 2007, 10:47:50 PM1/2/07
to
"user923005" <dco...@connx.com> writes:
[...]

> Imagine the following:
> void *p =calloc(128000, 128000);
>
> Suppose that under the cover, the allocation created an array of 128000
> void pointers.
> Then, it allocated 128000 separate 128000 byte objects and pointed
> those pointers to the objects.
> Then it memset each of the 128000 distinct 128000 byte objects with
> zero.
> We do not need greater than 4GB to cover the stride of the pointer
> list.
> We do not need greater than 4 GB for any individual object.
> Would that implementation be conforming?

I don't believe so. calloc() must allocate a single contiguous object
(which happens to be an array object). Each byte of this object (or
equivalently, each byte of each element of the array) must have a
unique address.

Nelu

unread,
Jan 2, 2007, 10:51:41 PM1/2/07
to
Keith Thompson wrote:
> ric...@cogsci.ed.ac.uk (Richard Tobin) writes:
>> In article <459b0aad$0$27382$ba4a...@news.orange.fr>,
>> jacob navia <ja...@jacob.remcomp.fr> wrote:
>>
>>> sizeof returns a size_t. If an object larger than 4GB
>>> could exist then size_t can't be 32 bits.
>> To be pedantic, that doesn't quite follow. You can't apply sizeof to
>> the block returned by calloc().
>
> Agreed.
>
>> I'm not sure whether you can legally
>> have a system in which calloc() can return an object with more bytes
>> than size_t can count.
>
> I think you can, but such a system would have to have pointers bigger
> than size_t. calloc() allocates an object; each byte of that object
> must have a unique address of type char* or void*. Making size_t
> smaller than void* is probably legal, but I can't think of a good
> reason to do it (perhaps the DS9K does this).
>

Isn't this similar with the far/near pointers from Turbo C?

--
Ioan - Ciprian Tandau
tandau _at_ freeshell _dot_ org (hope it's not too late)
(... and that it still works...)

Richard Heathfield

unread,
Jan 2, 2007, 11:03:33 PM1/2/07
to
Spiros Bousbouras said:

<snip>


>
> Even
> if some people here were out to get him, so to speak, he
> could still try to offer them less possibilities by being
> precise and clear.

"Fewer", my dear chap - "fewer". :-)

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.

Keith Thompson

unread,
Jan 2, 2007, 11:11:34 PM1/2/07
to
"David T. Ashley" <d...@e3ft.com> writes:
> "jacob navia" <ja...@jacob.remcomp.fr> wrote in message
> news:459aeaf9$0$27394$ba4a...@news.orange.fr...
>> As a consequence of a heavy discussion in another thread,
>> I wrote this program:
>> #include <stdlib.h>
>> int main(void)
>> {
>> char *p=calloc(65521,65552);
>> if (p)
>> printf("BUG!!!!\n");
>> }
>
> My system (Red Hat Enterprise Linux 4.X with all errata applied) reports no
> bug. The details are below.
>
> Please note that I have subtly modified your test program by adding an
> "else" clause. I don't think you'll object.
>
> It isn't immediately clear to me why the call has to fail. 2^16 * 2^16 is
> 2^32 (4 gigabytes). My system has more virtual memory than that.

Really? Is it a 64-bit system? jacob's test program is intended to
apply only to systems with 32-bit size_t.

If your system has more than 4 gigabytes of virtual memory, but only a
32-bit size_t, that might cause some problems. (Or it might not, if
no single object can be bigger than 4 gigabytes.)

user923005

unread,
Jan 2, 2007, 11:40:50 PM1/2/07
to
Seems to me a simpler test might be:

size_t count; /* for calloc() count, eventually */
size_t size; /* for calloc() object size, eventually */
/* the square root of the biggest size_t is a hint that we may be over
the edge */
const size_t sqrtstm = (size_t)sqrt(double(size_t(-1)));

...
if (count > sqrtstm || size > sqrtstm)
{
double dcount = count;
double dsize = size;
double dproduct = dcount * dsize;
if (dproduct > ((size_t)-1))
{
raise_bloody_heck();
}
}

or something of that ilk.
Relying on undocumented (at best) behavior seems a bit strange to me.
I guess that {for instance} passing two (size_t)-1 values into calloc()
will result in undefined behavior. I guess that calloc() could
conceivably be clever enough to catch it, but I am very sure that it
would not be portable.

av

unread,
Jan 3, 2007, 3:19:45 AM1/3/07
to
On 3 Jan 2007 01:54:44 GMT, Richard Tobin wrote:

>In article <459aeaf9$0$27394$ba4a...@news.orange.fr>,
>jacob navia <ja...@jacob.remcomp.fr> wrote:
>
>>Maybe people with other compilers/implementations
>>would compile this program and send the results?
>
>To my surprise, it has the bug on MacOS X Tiger with gcc 4.0.1 (x86)
>and 4.0.0 (ppc). It also has the bug on Dragonfly 1.2 with gcc 2.95,
>but that's a rather out-of-date system. It behaves correctly on
>FreeBSD 6.1 x86 with gcc 3.4.4. I have a vague recollection of seeing
>discussions about fixing this very problem in *BSD within the last
>year or so.

the "problem" is not in calloc or malloc but it is in the mathemaical
model ("modello matematico") used for size_t

av

unread,
Jan 3, 2007, 3:19:50 AM1/3/07
to
On Tue, 02 Jan 2007 19:38:14 -0800, Keith Thompson wrote:
>jacob's test program (in either version) does not detect this bug with
>100% reliability, but if it reports "BUG!!!!" on a given system, it's
>probably a good idea to investigate further.
>
>There's a related potential bug, where something like
> p = malloc(COUNT * sizeof *p);
>can successfully allocate fewer bytes than intended, but in this case
>the bug is in the user code; there's not really anything the malloc()
>implementation can do about it.

there is the same "bug" if there is some "calculus" on a size_t
variable "var" that use the mod definition
and then that variable "var" is passed to some standard c function
that has as parameter size_t (and it seems there are many function
that use size_t)

CBFalconer

unread,
Jan 3, 2007, 1:16:02 AM1/3/07
to
jacob navia wrote:
>
... snip ...

>
> OK This is a misunderstanding then. The test is intended only
> for 32 bit systems
>
> Updated version:
>
> #include <stdlib.h>
> #include <limits.h>
> #include <stdio.h> // for printf
> int main(void)
> {
> int s = sizeof(size_t)*CHAR_BIT;
> if (s == 32 && NULL != calloc(65521,65552))
> printf("BUG!!!!\n");
> return 0; // For C89 compilers
> }

Wrong. A size_t object can include other padding bits. You need
to work with SIZE_MAX or (size_t)-1. Don't you care enough about
your posts to format them properly?

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>


CBFalconer

unread,
Jan 3, 2007, 1:27:37 AM1/3/07
to
user923005 wrote:
> jacob navia wrote:
>
>> No, this is impossible in 32 bit size_t systems!!!
>
> Why? The calloc() call has two parameters. There is no reason
> that the total allocation cannot exceed the value of a size_t.

Yes there is. Fragment follows:

#define N 65600
#define M 65530

long n, m;
char *p;

if (p = calloc(N, M)
for (n = N; n, n--)
for (m = M; m, m--)
if (p[n * m]) puts("BUG")
puts("OK");

if n * m exceeds SIZE_MAX, how is the program going to access that
storage? Since it can't, why even attempt to allocate it.

CBFalconer

unread,
Jan 3, 2007, 2:16:00 AM1/3/07
to
jacob navia wrote:
> Jens Thoms Toerring a écrit :
> [snip]
>
> Please see the answer of Mr Spiros Bousburas in this same thread
> and try to understand what I want before jumping to conclusions.
>
> Thanks
>
> P.S. if you do not want to participate it is OK but please do not
> send noise.

Do you ever read your own posts? I'm sure Herr Toerring is
overjoyed and relieved to have your permission to participate in a
*public* newsgroup.

CBFalconer

unread,
Jan 3, 2007, 2:04:15 AM1/3/07
to
jacob navia wrote:
>
... snip ...
>
> My objective is to
>
> 1) make a list of all the buggy systems that
> is here in clc and everybody can see.
> 2) Make people aware of this bug in their implementations.
> (if any)

Do you really want people to start testing and publicizing bugs in
LCC-win32? Bugs being defined as failure to meet the C99 standard
(which you claim to meet) and/or failure to accurately diagnose
errors. Consider your answer carefully, in the light of having
annoyed many of the most knowledgeable here, whose reports will
probably not be designed to help finding the bug cause.

The results might be the beginning of a GPLd standard C test suite,
after de-obfuscation.

CBFalconer

unread,
Jan 3, 2007, 2:57:59 AM1/3/07
to
"David T. Ashley" wrote:
>
... snip ...

>
> 54135^2 is going to be on the order of 2.5G. That is a pretty
> fair hunk of memory.
>
> ---------
>
> [nouser@pamc ~]$ cat test3.c
> #include <stdio.h>
> #include <stdlib.h>
>
> int main(void)
> {
> char *p;
> int i;
>
> for (i=65535; i>0; i--)
> {
> if (p = calloc(i,i))
> {
> printf("%d is apparently the largest integer that will succeed.\n",
> i);
> break;
> }
> }
> }
> [nouser@pamc ~]$ gcc test3.c
> [nouser@pamc ~]$ ./a.out
> 54135 is apparently the largest integer that will succeed.
> [nouser@pamc ~]$

With DJGPP 2.03 (specifies the library) that crashes immediately in
memset. With nmalloc linked in in place of the library it yields
23093.

Advisory cross-post to comp.os.msdos.djgpp, f'ups set to clc.

av

unread,
Jan 3, 2007, 3:58:35 AM1/3/07
to
On 3 Jan 2007 01:54:44 GMT, Richard Tobin wrote:
>In article <459aeaf9$0$27394$ba4a...@news.orange.fr>,
>jacob navia <ja...@jacob.remcomp.fr> wrote:
>>Maybe people with other compilers/implementations
>>would compile this program and send the results?
>
>To my surprise, it has the bug on MacOS X Tiger with gcc 4.0.1 (x86)
>and 4.0.0 (ppc). It also has the bug on Dragonfly 1.2 with gcc 2.95,
>but that's a rather out-of-date system. It behaves correctly on
>FreeBSD 6.1 x86 with gcc 3.4.4. I have a vague recollection of seeing
>discussions about fixing this very problem in *BSD within the last
>year or so.

the "problem" is not in calloc or malloc but it is in the mathemaical

Richard Heathfield

unread,
Jan 3, 2007, 3:58:47 AM1/3/07
to
av said:

<snip>



> the "problem" is not in calloc or malloc but it is in the mathemaical
> model ("modello matematico") used for size_t

That's not a problem at all. It's simple and well-defined. The problem is in
failing to understand that limits exist.

Florian Xaver

unread,
Jan 3, 2007, 4:03:56 AM1/3/07
to
I cannot help you... :-(

But I found nmalloc on your page and it looks interesting. I am using
latest 2.04 (alpha/beta?) binary and think, that it isn't included there?
Then I would like to use it in my own project.

Bye
--
Florian Xaver

Webdesign, Sword, oZone, personal site: <http://www.flox.at.tf>
Dr-DOS Wiki: <http://www.drdos.org>

av

unread,
Jan 3, 2007, 4:54:17 AM1/3/07
to
On Wed, 03 Jan 2007 08:58:47 +0000, Richard Heathfield wrote:

>av said:
>
><snip>
>
>> the "problem" is not in calloc or malloc but it is in the mathemaical
>> model ("modello matematico") used for size_t
>
>That's not a problem at all. It's simple and well-defined. The problem is in
>failing to understand that limits exist.

not for me

user923005

unread,
Jan 3, 2007, 4:54:50 AM1/3/07
to
CBFalconer wrote:
> user923005 wrote:
> > jacob navia wrote:
> >
> >> No, this is impossible in 32 bit size_t systems!!!
> >
> > Why? The calloc() call has two parameters. There is no reason
> > that the total allocation cannot exceed the value of a size_t.
>
> Yes there is. Fragment follows:
>
> #define N 65600
> #define M 65530
>
> long n, m;
> char *p;
>
> if (p = calloc(N, M)
> for (n = N; n, n--)
> for (m = M; m, m--)
> if (p[n * m]) puts("BUG")
> puts("OK");

if N*M exceed (size_t) -1, I would submit that the program probably
already exhibits undefined behavior.

If the loop runs to completion that is not an indication that the
allcation worked correctly. Only that it might have.

> if n * m exceeds SIZE_MAX, how is the program going to access that
> storage? Since it can't, why even attempt to allocate it.

His program in no way tests to see if n*m exceeds SIZE_MAX, except by
pure accident.

Richard Tobin

unread,
Jan 3, 2007, 6:51:55 AM1/3/07
to
In article <500nimF...@mid.individual.net>,
Nelu <spam...@gmail.com> wrote:

>> I think you can, but such a system would have to have pointers bigger
>> than size_t. calloc() allocates an object; each byte of that object
>> must have a unique address of type char* or void*. Making size_t
>> smaller than void* is probably legal, but I can't think of a good
>> reason to do it (perhaps the DS9K does this).

You wouldn't be able to copy large objects with memcpy(), but that
doesn't prove that it's not legal.

>Isn't this similar with the far/near pointers from Turbo C?

Yes, but no-one suggests that that conforms to the C standard.

Richard Tobin

unread,
Jan 3, 2007, 6:57:31 AM1/3/07
to
In article <459B4CD9...@yahoo.com>,
CBFalconer <cbfal...@maineline.net> wrote:

>#define N 65600
>#define M 65530
>
>long n, m;
>char *p;
>
>if (p = calloc(N, M)
> for (n = N; n, n--)
> for (m = M; m, m--)
> if (p[n * m]) puts("BUG")
>puts("OK");
>
>if n * m exceeds SIZE_MAX, how is the program going to access that
>storage? Since it can't, why even attempt to allocate it.

It can't access it by indexing from a char * pointer to the start
of the block, but consider the more realistic:

struct foo *p = calloc(large_number, sizeof(*p));

where sizeof(struct foo) > 1.

Richard Tobin

unread,
Jan 3, 2007, 7:04:35 AM1/3/07
to
In article <eopmp2pmim2lcar4a...@4ax.com>, av <av@ala.a> wrote:

>>To my surprise, it has the bug on MacOS X Tiger with gcc 4.0.1 (x86)
>>and 4.0.0 (ppc). It also has the bug on Dragonfly 1.2 with gcc 2.95,
>>but that's a rather out-of-date system. It behaves correctly on
>>FreeBSD 6.1 x86 with gcc 3.4.4. I have a vague recollection of seeing
>>discussions about fixing this very problem in *BSD within the last
>>year or so.

>the "problem" is not in calloc or malloc but it is in the mathemaical
>model ("modello matematico") used for size_t

That's true in a sense: the modular arithmetic used for unsigned types
is not appropriate to sizes. But the problem can be fixed in calloc()
without changing the mathematical model used for size_t.

The solution is either to convert the sizes to some other type first,
or to check in some other way that multiplication as size_t would give
the same result as multiplication as (mathematical) integers.

CBFalconer

unread,
Jan 3, 2007, 8:49:35 AM1/3/07
to
Florian Xaver wrote:
>
> I cannot help you... :-(
>
> But I found nmalloc on your page and it looks interesting. I am using
> latest 2.04 (alpha/beta?) binary and think, that it isn't included there?
> Then I would like to use it in my own project.

Go ahead. Under DJGPP copying license there are virtually no
restrictions. I don't know if you can simply replace the malloc
module in the library. But prelinking malloc.o works under DJGPP.
Report any problems to comp.os.msdos.djgpp and/or to me at the
following address (the address in the zip is obsolete). The lack
of memalign may affect compiling of gcc. Don't really know.

Cesar Rabak

unread,
Jan 3, 2007, 2:33:32 PM1/3/07
to
user923005 escreveu:
> jacob navia wrote:
>> user923005 a écrit :

>>> jacob navia wrote:
>>>
>>>> No, this is impossible in 32 bit size_t systems!!!
>>>
>>> Why? The calloc() call has two parameters. There is no reason that
>>> the total allocation cannot exceed the value of a size_t.
>>>
>>> BTW, there are definitely 32 bit systems that will allocate memory in
>>> excess of 32 bits. For example:
>>> http://support.microsoft.com/kb/274750
>>>
>> This is irrelevant. Those links refer as to how
>> to use certain flags in the boot.ini to allow
>> *3GB* if user space. (!!!!!!!)
>
> That is either 8 GB user space or 32 GB, not 3 GB. I suggest you read
> again -- this time with comprehension enabled... Here is a quote from
> that page that you may find helpful:
> "SQL Server 2000 Enterprise Edition introduces support for the use of
> Microsoft Windows 2000 Address Windowing Extensions (AWE) to address
> approximately 8 GB of memory for instances that run on Microsoft
> Windows 2000 Advanced Server, and approximately 32 GB for instances
> that run on Microsoft Windows 2000 Datacenter."
>
> P.S.
> These operating systems are 32 bits.

But in what part of the text (or the page) there is any information how
a programmer can allocate all this memory in a single chunk?

And in this case would that be via a call to calloc?

Keith Thompson

unread,
Jan 3, 2007, 3:37:01 PM1/3/07
to
CBFalconer <cbfal...@yahoo.com> writes:
> user923005 wrote:
>> jacob navia wrote:
>>
>>> No, this is impossible in 32 bit size_t systems!!!
>>
>> Why? The calloc() call has two parameters. There is no reason
>> that the total allocation cannot exceed the value of a size_t.
>
> Yes there is. Fragment follows:
>
> #define N 65600
> #define M 65530
>
> long n, m;
> char *p;
>
> if (p = calloc(N, M)
> for (n = N; n, n--)
> for (m = M; m, m--)
> if (p[n * m]) puts("BUG")
> puts("OK");
>
> if n * m exceeds SIZE_MAX, how is the program going to access that
> storage? Since it can't, why even attempt to allocate it.

There's no *absolute* reason why calloc() can't allocate more than
SIZE_MAX bytes. n and m could be of some type bigger than size_t.
(Array indexing and pointer arithmetic aren't defined in terms of
size_t.)

However, each byte of the allocated object must have a distinct
address, so (ignoring padding bits) char* would have to be bigger than
size_t. I believe this would be legal but silly.

Keith Thompson

unread,
Jan 3, 2007, 3:42:29 PM1/3/07
to
CBFalconer <cbfal...@yahoo.com> writes:
> jacob navia wrote:
>>
> ... snip ...
>>
>> My objective is to
>>
>> 1) make a list of all the buggy systems that
>> is here in clc and everybody can see.
>> 2) Make people aware of this bug in their implementations.
>> (if any)
>
> Do you really want people to start testing and publicizing bugs in
> LCC-win32? Bugs being defined as failure to meet the C99 standard
> (which you claim to meet) and/or failure to accurately diagnose
> errors. Consider your answer carefully, in the light of having
> annoyed many of the most knowledgeable here, whose reports will
> probably not be designed to help finding the bug cause.

I don't believe he actually claims that lcc-win32 fully conforms to
C99 (though some of his statements here could lead to that
conclusion). When I asked in comp.compilers.lcc-win32 a few months
ago, he replied:

| Designated initializers and structure initializers with the
| dot notation are missing.
|
| I am giving priority to the library, that is kind of
| "mostly" finished. I have probably some problems with
| complex numbers, there hasn't been a good testing of that
| part.
|
| Besides the preprocessor is still missing the variable
| arguments feature.

> The results might be the beginning of a GPLd standard C test suite,
> after de-obfuscation.

GPL-ing it would, if I'm not mistaken, require explicit permission by
all contributors. I'm not saying it can't or shouldn't be done, but
it would would have to be planned from the beginning.

Keith Thompson

unread,
Jan 3, 2007, 3:47:45 PM1/3/07
to
Richard Heathfield <r...@see.sig.invalid> writes:
> av said:
> <snip>
>
>> the "problem" is not in calloc or malloc but it is in the mathemaical
>> model ("modello matematico") used for size_t
>
> That's not a problem at all. It's simple and well-defined. The problem is in
> failing to understand that limits exist.

Yes. *Another* problem is the lack of operations on size_t (and
unsigned types in general) that behave in a manner other than the one
chosen by the standard. This inflexibility of the language makes some
useful operations difficult to implement, and in some cases impossible
to implement both portably and efficiently.

On the other hand, "fixing" this "problem" would make the language
more complicated, and it's not entirely sure that it would be worth
it.

Keith Thompson

unread,
Jan 3, 2007, 3:51:29 PM1/3/07
to
ric...@cogsci.ed.ac.uk (Richard Tobin) writes:
> In article <500nimF...@mid.individual.net>,
> Nelu <spam...@gmail.com> wrote:
>
>>> I think you can, but such a system would have to have pointers bigger
>>> than size_t. calloc() allocates an object; each byte of that object
>>> must have a unique address of type char* or void*. Making size_t
>>> smaller than void* is probably legal, but I can't think of a good
>>> reason to do it (perhaps the DS9K does this).

I wrote the above, starting with "I think you can".

> You wouldn't be able to copy large objects with memcpy(), but that
> doesn't prove that it's not legal.
>
>>Isn't this similar with the far/near pointers from Turbo C?
>
> Yes, but no-one suggests that that conforms to the C standard.

Is it similar? On systems that provide far/near pointers, isn't
size_t big enough to cover all possible objects? (I don't know; I've
never used such a system.)

Note that even if you have, say, 64-bit pointers, if the
implementation doesn't allow objects bigger than 2**32 bytes, you can
still have a 32-bit size_t without causing any serious problems. In
the scenario I discussed above, there could be objects too big for
size_t to represent their size; with restricted object sizes, that
wouldn't happen.

Spiros Bousbouras

unread,
Jan 3, 2007, 4:14:18 PM1/3/07
to
user923005 wrote:
> CBFalconer wrote:
> > user923005 wrote:
> > > jacob navia wrote:
> > >
> > >> No, this is impossible in 32 bit size_t systems!!!
> > >
> > > Why? The calloc() call has two parameters. There is no reason
> > > that the total allocation cannot exceed the value of a size_t.
> >
> > Yes there is. Fragment follows:
> >
> > #define N 65600
> > #define M 65530
> >
> > long n, m;
> > char *p;
> >
> > if (p = calloc(N, M)
> > for (n = N; n, n--)
> > for (m = M; m, m--)
> > if (p[n * m]) puts("BUG")
> > puts("OK");
>
> if N*M exceed (size_t) -1, I would submit that the program probably
> already exhibits undefined behavior.

Why would that be ?

Spiros Bousbouras

unread,
Jan 3, 2007, 5:22:09 PM1/3/07
to
Richard Heathfield wrote:
> Spiros Bousbouras said:
>
> <snip>
> >
> > Even
> > if some people here were out to get him, so to speak, he
> > could still try to offer them less possibilities by being
> > precise and clear.
>
> "Fewer", my dear chap - "fewer". :-)

What's wrong with "less" ?

Ian Collins

unread,
Jan 3, 2007, 5:41:34 PM1/3/07
to
"fewer" is used with plural nouns (fewer boxes, fewer possibilities),
"less" refers to an amount (less productive, less motivation).

--
Ian Collins.

christian.bau

unread,
Jan 3, 2007, 5:55:39 PM1/3/07
to
user923005 wrote:

> Relying on undocumented (at best) behavior seems a bit strange to me.
> I guess that {for instance} passing two (size_t)-1 values into calloc()
> will result in undefined behavior. I guess that calloc() could
> conceivably be clever enough to catch it, but I am very sure that it
> would not be portable.

C99 Final Draft says: "The calloc function allocates space for an array
of nmemb objects, each of whose size is size.The space is initialized
to all bits zero. " and later "The calloc function returns either a
null pointer or a pointer to the allocated space."

There is no mention there of any problems in the calculation of nmemb *
size. calloc should either allocate space for nmemb objects and return
a pointer to those objects, or return a null pointer. That
specification is quite clear, and I wouldn't accept any excuses if it
is not followed. If I pass for example nmemb = 0x10001, and size =
0x10001, and in the C implementation that I use 0x10001 * 0x10001 ==
0x20001 (which is quite common), then it should be obvious that
allocating 0x20001 bytes will not give enough space for 0x10001 objects
of 0x10001 bytes, and a calloc implementation doing this would be just
plain broken.

The spec says "nmemb objects, each of whose size is size", not "nmemb *
size bytes".

That said, I wouldn't be surprised by the occasional broken C library
implementation.

christian.bau

unread,
Jan 3, 2007, 6:03:20 PM1/3/07
to
Keith Thompson wrote:
> There's no *absolute* reason why calloc() can't allocate more than
> SIZE_MAX bytes. n and m could be of some type bigger than size_t.
> (Array indexing and pointer arithmetic aren't defined in terms of
> size_t.)
>
> However, each byte of the allocated object must have a distinct
> address, so (ignoring padding bits) char* would have to be bigger than
> size_t. I believe this would be legal but silly.

Implementations where sizeof (char *) == 4, and sizeof (size_t) == 2,
have been quite common in the past. On an 80386 processor, an
implementation with char* = 48 bits and size_t = 32 bits wouldn't have
been completely unreasonable (I have never seen one myself).

Mark McIntyre

unread,
Jan 3, 2007, 6:03:31 PM1/3/07
to
On 2 Jan 2007 17:21:45 -0800, in comp.lang.c , "Spiros Bousbouras"
<spi...@gmail.com> wrote:

>I'll take a guess: your point is that calloc should check
>whether the multiplication (if that's what it does) yields
>a value so big that the system cannot possibly have that
>much memory.

My issue with this idea is that there /is/ no such value. I have a
machine in the office with 128GB memory. I have on on order with 1TB
memory. The only possible way malloc() could check the reqested size
was too large would be to use a system-specific call to ask the OS how
much was free, and compare that to the original request. Any sort of
check based on just looking at the request would be incorrect. And
yes, this machine is capable of running 32-bit apps, even Windows apps
for that matter.

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

Mark McIntyre

unread,
Jan 3, 2007, 6:05:42 PM1/3/07
to
On Wed, 03 Jan 2007 02:26:25 +0100, in comp.lang.c , jacob navia
<ja...@jacob.remcomp.fr> wrote:

>Spiros Bousbouras a écrit :
>> I don't know "so WHAT". You see the problem here is
>> that you have not explained what your point is. >


>My objective is to
>
>1) make a list of all the buggy systems

All buggy systems, in all respects, or just those which have a
specific bug? If the latter, please explain carefully what this bug
is.

Mark McIntyre

unread,
Jan 3, 2007, 6:09:07 PM1/3/07
to
On 2 Jan 2007 18:16:28 -0800, in comp.lang.c , "Spiros Bousbouras"
<spi...@gmail.com> wrote:

>So the central question is:
>
>If some code contains a call of the form calloc(n,m)
>where n*m > 2**( sizeof(void*) * CHAR_BIT) then is it
>the obligation of the calloc implementation to detect
>this and return NULL ?
>
>If the answer to the question is yes then you can say
>that the implementations which don't do it are buggy.

I disagree. Provided the system can actually allocate the requested
memory, why is it a bug to do so?

>>From the practical point of view , if I ever needed to
>write something like calloc(n,m) where n*m >= 2**31
>on a 32 bits system , alarm bells would start to ring

absolutely!

Richard Tobin

unread,
Jan 3, 2007, 6:15:55 PM1/3/07
to
In article <ln1wmbb...@nuthaus.mib.org>,
Keith Thompson <ks...@mib.org> wrote:

>> You wouldn't be able to copy large objects with memcpy(), but that
>> doesn't prove that it's not legal.
>>
>>>Isn't this similar with the far/near pointers from Turbo C?
>>
>> Yes, but no-one suggests that that conforms to the C standard.
>
>Is it similar? On systems that provide far/near pointers, isn't
>size_t big enough to cover all possible objects? (I don't know; I've
>never used such a system.)

Good point, I don't know either.

Spiros Bousbouras

unread,
Jan 3, 2007, 6:12:50 PM1/3/07
to
Mark McIntyre wrote:
> On 2 Jan 2007 17:21:45 -0800, in comp.lang.c , "Spiros Bousbouras"
> <spi...@gmail.com> wrote:
>
> >I'll take a guess: your point is that calloc should check
> >whether the multiplication (if that's what it does) yields
> >a value so big that the system cannot possibly have that
> >much memory.
>
> My issue with this idea is that there /is/ no such value. I have a
> machine in the office with 128GB memory. I have on on order with 1TB
> memory. The only possible way malloc() could check the reqested size
> was too large would be to use a system-specific call to ask the OS how
> much was free, and compare that to the original request.

Indeed. Do you see anything wrong with such an approach ?

Mark McIntyre

unread,
Jan 3, 2007, 6:13:33 PM1/3/07
to
On Wed, 03 Jan 2007 02:45:18 +0100, in comp.lang.c , jacob navia
<ja...@jacob.remcomp.fr> wrote:

>SQL server surely doesn't use calloc for allocating 32GB of
>RAM,

I guess we'd need to ask Microsoft instead of guessing.

>> Obviously if that were the case, pointers would have to be larger than
>> 32 bits.

This doesn't actually follow. I can envisage a segmented architecture
which solved this problem.

>sizeof returns a size_t. If an object larger than 4GB
>could exist then size_t can't be 32 bits.

This doesn't follow either. Just because your tape measure stops at 5
metres, doesn't mean things longer than this can't exist. It does
however mean that a Standard C app cannot measure them accurately.

jacob navia

unread,
Jan 3, 2007, 6:33:43 PM1/3/07
to
christian.bau a écrit :

This is my whole point!

And if you care to see the list, there are implementations like that

Richard Tobin

unread,
Jan 3, 2007, 6:46:29 PM1/3/07
to
In article <fudop2lfal0d9i1rg...@4ax.com>,
Mark McIntyre <markmc...@spamcop.net> wrote:

>>> Obviously if that were the case, pointers would have to be larger than
>>> 32 bits.

>This doesn't actually follow. I can envisage a segmented architecture
>which solved this problem.

A 32-bit pointer can only distinguish 2^32 different values. A
pointer that can point to all the bytes in an object bigger than 2^32
bytes[1] must therefore have more than 32 bits. In a segmented
architecture that allowed objects spanning segments, the segment would
have to be encoded as part of the C pointer, and the pointer would
still have to be bigger than 32 bits.

[1] 2^32-1 if you want to be able to address the byte past the end.

Mark McIntyre

unread,
Jan 3, 2007, 6:50:28 PM1/3/07
to
On 3 Jan 2007 15:12:50 -0800, in comp.lang.c , "Spiros Bousbouras"
<spi...@gmail.com> wrote:

No, and indeed thats what I'd expect calloc to do - check if it could
allocate the requested memory, and return NULL if not. This is after
all what its /required/ to do. How it does it is of course
implementation specific.

Cesar Rabak

unread,
Jan 3, 2007, 6:54:17 PM1/3/07
to
Mark McIntyre escreveu:

> On 2 Jan 2007 17:21:45 -0800, in comp.lang.c , "Spiros Bousbouras"
> <spi...@gmail.com> wrote:
>
>> I'll take a guess: your point is that calloc should check
>> whether the multiplication (if that's what it does) yields
>> a value so big that the system cannot possibly have that
>> much memory.
>
> My issue with this idea is that there /is/ no such value. I have a
> machine in the office with 128GB memory. I have on on order with 1TB
> memory. The only possible way malloc() could check the reqested size
> was too large would be to use a system-specific call to ask the OS how
> much was free, and compare that to the original request.

I cannot see this scheme to see daylight: in order to do this, the
application would need to lock all other apps requests to memory to be
sure the reduction is due its request and not somone's (sum of) request.

Spiros Bousbouras

unread,
Jan 3, 2007, 7:30:11 PM1/3/07
to
Mark McIntyre wrote:
> On 2 Jan 2007 18:16:28 -0800, in comp.lang.c , "Spiros Bousbouras"
> <spi...@gmail.com> wrote:
>
> >So the central question is:
> >
> >If some code contains a call of the form calloc(n,m)
> >where n*m > 2**( sizeof(void*) * CHAR_BIT) then is it
> >the obligation of the calloc implementation to detect
> >this and return NULL ?
> >
> >If the answer to the question is yes then you can say
> >that the implementations which don't do it are buggy.
>
> I disagree. Provided the system can actually allocate the requested
> memory, why is it a bug to do so?

When I wrote "the obligation" I meant according to
the standard. By definition if an implementation does
not do what the standard says it should , then it's
buggy.

If your point was that you see no reason for the standard
to make such a requirement then the reply to that has
been covered more or less already: if a void* does not
have enough bits to be able to refer to any byte in the
allocated memory then the standard is violated.

I feel pretty sure now that an implementation of calloc
should be able to detect this and return NULL.

jacob navia

unread,
Jan 3, 2007, 8:24:36 PM1/3/07
to
Mark McIntyre a écrit :

> On Wed, 03 Jan 2007 02:45:18 +0100, in comp.lang.c , jacob navia
> <ja...@jacob.remcomp.fr> wrote:
>
>
>>SQL server surely doesn't use calloc for allocating 32GB of
>>RAM,
>
>
> I guess we'd need to ask Microsoft instead of guessing.
>
>
>>>Obviously if that were the case, pointers would have to be larger than
>>>32 bits.
>
>
> This doesn't actually follow. I can envisage a segmented architecture
> which solved this problem.
>

Excuse me but then the pointer is bigger than 32 bits!!!

Joe Wright

unread,
Jan 3, 2007, 9:09:42 PM1/3/07
to
Yes. Fewer ice cubes, less ice, etc.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

CBFalconer

unread,
Jan 3, 2007, 8:19:06 PM1/3/07
to

Besides which, it just grates on an educated English speaker.

CBFalconer

unread,
Jan 3, 2007, 8:22:10 PM1/3/07
to

He's right. The longs should have been unsigned longs.

Richard Heathfield

unread,
Jan 3, 2007, 11:38:11 PM1/3/07
to
Spiros Bousbouras said:

Others have answered this whilst I slept. But whether it is "wrong" depends
on whether one is trying to be precise and clear. I would not have
mentioned such a trivial thing in such a newsgroup as this one, but for the
contrast between your sentence's message and its construction.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.

CBFalconer

unread,
Jan 4, 2007, 12:29:09 AM1/4/07
to
alex bodnaru wrote:
> CBFalconer wrote:
>> "David T. Ashley" wrote:
>>
>> ... snip ...
>>
>>> 54135^2 is going to be on the order of 2.5G. That is a pretty
>>> fair hunk of memory.
>>>
>>> ---------
>>>
>>> [nouser@pamc ~]$ cat test3.c
>>> #include <stdio.h>
>>> #include <stdlib.h>
>>>
>>> int main(void)
>>> {
>>> char *p;
>>> int i;
>>>
>>> for (i=65535; i>0; i--)
>>> {
>>> if (p = calloc(i,i))
>>> {
>>> printf("%d is apparently the largest integer that will succeed.\n",
>>> i);
>>> break;
>>> }
>>> }
>>> }
>>> [nouser@pamc ~]$ gcc test3.c
>>> [nouser@pamc ~]$ ./a.out
>>> 54135 is apparently the largest integer that will succeed.
>>> [nouser@pamc ~]$
>>
>> With DJGPP 2.03 (specifies the library) that crashes immediately in
>> memset. With nmalloc linked in in place of the library it yields
>> 23093.
>>
>> Advisory cross-post to comp.os.msdos.djgpp, f'ups set to clc.
>
> even the first malloc is 4gb, and i doubt you have more ;-)

Which should simply fail. The code is looking for the largest
assignable value and for foulups in the calloc multiply operation.
It shouldn't crash. nmalloc fails and returns NULL. DJGPP malloc
claims to succeed, but doesn't, and the memset crashes. This also
shows the need to include calloc in the nmalloc package, to ensure
the same limits are observed.

Why did you override the follow-up I had set? And top-post.
Please go to comp.lang.c for further general discussion of this.
Meanwhile, be aware of the DJGPP bug.

Keith Thompson

unread,
Jan 4, 2007, 1:10:02 AM1/4/07
to

Right. The point I missed (I think I mentioned this later) is that
there has to be a distinct char* value for each byte of each object,
but size_t only has to span a single object (and perhaps only a single
declared object).

In a system with 32-bit char* and 16-bit size_t, I presume that no
single object can exceed 65535 bytes; is that correct? (I'm
accustomed to systems where the theoretical maximum size of a single
object is the same as the size of the entire memory space.)

Chris Torek

unread,
Jan 4, 2007, 6:35:06 AM1/4/07
to
>"christian.bau" <christ...@cbau.wanadoo.co.uk> writes:
>> Implementations where sizeof (char *) == 4, and sizeof (size_t) == 2,
>> have been quite common in the past. On an 80386 processor, an
>> implementation with char* = 48 bits and size_t = 32 bits wouldn't have
>> been completely unreasonable (I have never seen one myself).

In article <lntzz77...@nuthaus.mib.org>


Keith Thompson <ks...@mib.org> wrote:
>Right. The point I missed (I think I mentioned this later) is that
>there has to be a distinct char* value for each byte of each object,
>but size_t only has to span a single object (and perhaps only a single
>declared object).
>
>In a system with 32-bit char* and 16-bit size_t, I presume that no
>single object can exceed 65535 bytes; is that correct?

That is, I think, the only *sensible* way to handle this in C.
(Of course, if you use Vendor-Specific Extensions, all bets are
off.) I am not certain that this is actually required by the
standards, though. (That is, perhaps a compiler can make objects
where sizeof(obj) exceeds SIZE_MAX, under various peculiar conditions.
One constraint would be that the code never actually uses the size
of the object, i.e., if sizeof(obj) appears anywhere in the code,
the value must be discarded.)

>(I'm accustomed to systems where the theoretical maximum size of
>a single object is the same as the size of the entire memory space.)

These are "flat" memory systems. Segmented systems must have
"locally flat" memory, that is, flat within each C-language-level
object, in order to conform to the requirements of the C standards.
(Exceptions can be arranged for objects whose address is never
taken, and perhaps others, as noted above.)
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.

av

unread,
Jan 4, 2007, 6:45:12 AM1/4/07
to
On Wed, 03 Jan 2007 12:47:45 -0800, Keith Thompson wrote:
>Richard Heathfield <r...@see.sig.invalid> writes:
>> av said:
>> <snip>
>>> the "problem" is not in calloc or malloc but it is in the mathemaical
>>> model ("modello matematico") used for size_t
>>
>> That's not a problem at all. It's simple and well-defined. The problem is in
>> failing to understand that limits exist.

>Yes. *Another* problem is the lack of operations on size_t (and
>unsigned types in general) that behave in a manner other than the one
>chosen by the standard. This inflexibility of the language makes some
>useful operations difficult to implement, and in some cases impossible
>to implement both portably and efficiently.
>
>On the other hand, "fixing" this "problem" would make the language
>more complicated, and it's not entirely sure that it would be worth
>it.

so is it not possible to save "capra e cavoli"?

in a pc with 32bits cpu
where is the problem in have 31bits unsigned size_t (all like say the
C standard) and to have the 32th bit that say if there is some
"overflow" (or use mod) in the middle of some computation?

in a pc with 64bits cpu
where is the problem in have 63bits unsigned size_t (all like say the
C standard) and to have the 64th bit that say if there is some
"overflow" (or use mod) in the middle of some computation?

if a function want to see if that some size_t variable has overflow
see the last bit in that variable

than could be added another type that use (-1) for error
and not have "mod" behaviour like below, for computation
that can not overflow in mod
---------------------
>/* it could do size_t multiplication
> for numbers [0..2^32-2]
>*/
>asm{
>multix:
>push edx
> mov eax, [esp+ 8]
> cmp [esp+12], -1
> je .1
> cmp eax, -1
> je .f
> xor edx, edx
> mul dword[esp+12]
> cmp edx, 0
> je .f
>.1:
> mov eax, -1
>.f:
>pop edx
>ret 8
>}
>
>asm{
>addsize:
> mov eax, [esp+ 4]
> cmp [esp+8], -1
> je .1
> cmp eax, -1
> je .f
> add eax, [esp+8]
> jnc .f
>.1:
> mov eax, -1
>.f:
>pop edx
>ret 8
>}
>
>asm{
>subsize:
> mov eax, [esp+ 4]
> cmp [esp+8], -1
> je .1
> cmp eax, -1
> je .f
> sub eax, [esp+8]
> j>= .f
>.1:
> mov eax, -1
>.f:
>pop edx
>ret 8
>}
>
>unsigned _stdcall multix(unsigned a, unsigned b);
uns32 operator*(uns32 x, uns32 y) {return multix(x, y);}

>unsigned _stdcall addsize(unsigned a, unsigned b);
uns32 operator+(uns32 x, uns32 y) {return addsize(x, y);}

>unsigned _stdcall subsize(unsigned a, unsigned b);
uns32 operator-(uns32 x, uns32 y) {return subsize(x, y);}
etc
uns32 result, a, b, c, d, h;
....
result=a*b+c*d-h;

i forget to say if there is some overflow in some place
if(result==-1) error;

the same like above for an signed int type

santosh

unread,
Jan 4, 2007, 7:18:07 AM1/4/07
to

No. The C standard has to apply to more than just PCs. There's no way
to safely detect overflow on all machines for which a C implementation
is possible. The standard can't simply ignore such systems, since one
of the strengths of C is it's portability.

Richard Tobin

unread,
Jan 4, 2007, 8:10:05 AM1/4/07
to
In article <3o6dnYRyCLz...@bt.com>,
Richard Heathfield <r...@see.sig.invalid> wrote:

>> What's wrong with "less" ?

>Others have answered this whilst I slept. But whether it is "wrong" depends
>on whether one is trying to be precise and clear.

I'm all for having plenty of words so that we can make finer
distinctions, but I don't think ambiguity often, if ever, arises from
using "less" instead of "fewer".

As evidence for this, consider that the opposite of both words is "more"
(there's no English word "manier' corresponding to "many"). Have you ever
encountered a case where you felt that "more" was ambiguous?

Richard Heathfield

unread,
Jan 4, 2007, 8:34:49 AM1/4/07
to
Richard Tobin said:

> In article <3o6dnYRyCLz...@bt.com>,
> Richard Heathfield <r...@see.sig.invalid> wrote:
>
>>> What's wrong with "less" ?
>
>>Others have answered this whilst I slept. But whether it is "wrong"
>>depends on whether one is trying to be precise and clear.
>
> I'm all for having plenty of words so that we can make finer
> distinctions, but I don't think ambiguity often, if ever, arises from
> using "less" instead of "fewer".

You're probably right. Neverthefewer, I stand by my reply. :-)

<snip>

> Have you ever encountered a case where you felt that "more" was ambiguous?

Such a case is easy to construct, although in this case "ambiguous"
under-describes the *three* possible meanings:

Many people were oppressed by King Henry VII. More suffered under King Henry
VIII.

Richard Tobin

unread,
Jan 4, 2007, 9:11:32 AM1/4/07
to
In article <poqdnZwEBdOynwDY...@bt.com>,
Richard Heathfield <r...@see.sig.invalid> wrote:

>> Have you ever encountered a case where you felt that "more" was ambiguous?

>Such a case is easy to construct, although in this case "ambiguous"
>under-describes the *three* possible meanings:

>Many people were oppressed by King Henry VII. More suffered under King Henry
>VIII.

I can only see two meanings there ("a greater number of people than under
Henry VII", and "some additional people"), and it's not clear that a
less/fewer type distinction would resolve it (quite likely "manier"
would be idiomatic for both). What is the third?

Richard Heathfield

unread,
Jan 4, 2007, 9:35:12 AM1/4/07
to
Richard Tobin said:

> In article <poqdnZwEBdOynwDY...@bt.com>,
> Richard Heathfield <r...@see.sig.invalid> wrote:
>
>>> Have you ever encountered a case where you felt that "more" was
>>> ambiguous?
>
>>Such a case is easy to construct, although in this case "ambiguous"
>>under-describes the *three* possible meanings:
>
>>Many people were oppressed by King Henry VII. More suffered under King
>>Henry VIII.
>
> I can only see two meanings there ("a greater number of people than under
> Henry VII", and "some additional people"), and it's not clear that a
> less/fewer type distinction would resolve it (quite likely "manier"
> would be idiomatic for both). What is the third?

Cf http://www.luminarium.org/renlit/morebio.htm :-)

Dik T. Winter

unread,
Jan 4, 2007, 9:54:01 AM1/4/07
to
In article <enj1uk$tep$1...@pc-news.cogsci.ed.ac.uk> ric...@cogsci.ed.ac.uk (Richard Tobin) writes:
> In article <poqdnZwEBdOynwDY...@bt.com>,
> Richard Heathfield <r...@see.sig.invalid> wrote:
>
> >> Have you ever encountered a case where you felt that "more" was ambiguous?
>
> >Such a case is easy to construct, although in this case "ambiguous"
> >under-describes the *three* possible meanings:
>
> >Many people were oppressed by King Henry VII. More suffered under King Henry
> >VIII.
>
> I can only see two meanings there ("a greater number of people than under
> Henry VII", and "some additional people"), and it's not clear that a
> less/fewer type distinction would resolve it (quite likely "manier"
> would be idiomatic for both). What is the third?

Sir Thomas.
--
dik t. winter, cwi, kruislaan 413, 1098 sj amsterdam, nederland, +31205924131
home: bovenover 215, 1025 jn amsterdam, nederland; http://www.cwi.nl/~dik/

Richard Tobin

unread,
Jan 4, 2007, 11:19:47 AM1/4/07
to
In article <JBCMq...@cwi.nl>, Dik T. Winter <Dik.W...@cwi.nl> wrote:
>Sir Thomas.

Ah, ok.

rjkem...@yahoo.com

unread,
Jan 4, 2007, 1:11:28 PM1/4/07
to

CBFalconer wrote:

>
> Why did you override the follow-up I had set? And top-post.
> Please go to comp.lang.c for further general discussion of this.
> Meanwhile, be aware of the DJGPP bug.
>

I'd wager he sent a short response to dj...@delorie.com, replying to
something that showed up in his e-mail. Maybe doesn't give a hoot about
the c.l.c. traffic or top-posting cops.

Keith Thompson

unread,
Jan 4, 2007, 2:57:02 PM1/4/07
to
"santosh" <santo...@gmail.com> writes:
[...]

> No. The C standard has to apply to more than just PCs. There's no way
> to safely detect overflow on all machines for which a C implementation
> is possible. The standard can't simply ignore such systems, since one
> of the strengths of C is it's portability.

That's not quite true. It's always possible to detect overflow; if
nothing else, you can perform tests on the operands before performing
the operation. If C required overflow checking on all arithmetic
operations, it *could* be implemented, but the resulting code might be
significantly slower on some systems. Also, compiler technology at
the time the language was developed wasn't as advanced as it is now.
The designers of the language chose to keep the language simpler in
this area.

It's been argued that the effect of this is to get wrong answers more
quickly. There's some truth to that, but it also produces *correct*
answers more quickly, at the cost of putting the burden of avoiding
errors in the first place on the shoulders of the programmer.

Mark McIntyre

unread,
Jan 4, 2007, 6:11:39 PM1/4/07
to
On Thu, 04 Jan 2007 02:24:36 +0100, in comp.lang.c , jacob navia
<ja...@jacob.remcomp.fr> wrote:

>Mark McIntyre a écrit :
>> On Wed, 03 Jan 2007 02:45:18 +0100, in comp.lang.c , jacob navia
>>

>>>>Obviously if that were the case, pointers would have to be larger than
>>>>32 bits.
>>
>> This doesn't actually follow. I can envisage a segmented architecture
>> which solved this problem.
>
>Excuse me but then the pointer is bigger than 32 bits!!!

It still doesn't follow. The pointer returned by malloc need not be a
complete absolute reference. Magic can go on behind the scenes, much
as it did with 16-bit pointers used to reference 20-bit memory.

Of course, for this to work, the pointer needs not to be treated as an
integer...

Mark McIntyre

unread,
Jan 4, 2007, 6:21:03 PM1/4/07
to
On 4 Jan 2007 13:10:05 GMT, in comp.lang.c , ric...@cogsci.ed.ac.uk
(Richard Tobin) wrote:

>In article <3o6dnYRyCLz...@bt.com>,
>Richard Heathfield <r...@see.sig.invalid> wrote:
>
>>> What's wrong with "less" ?
>
>>Others have answered this whilst I slept. But whether it is "wrong" depends
>>on whether one is trying to be precise and clear.
>
>I'm all for having plenty of words so that we can make finer
>distinctions, but I don't think ambiguity often, if ever, arises from
>using "less" instead of "fewer".

I'm all for clarity, but not at the expense of correct English usage.
"less" is correctly applicable only to matters of degree or value,
whereas fewer is applicable to number. Less is also commonly applied
to plural nouns, though this is Bad Form. Thus fewer choices are less
useful.

>As evidence for this, consider that the opposite of both words is "more"

Euh, false logic. Consider "happy" and "smut-free". The opposite of
both is "blue" but that doesn't make them synonyms :-)

Ben Pfaff

unread,
Jan 4, 2007, 6:22:27 PM1/4/07
to
Mark McIntyre <markmc...@spamcop.net> writes:

> On Thu, 04 Jan 2007 02:24:36 +0100, in comp.lang.c , jacob navia
> <ja...@jacob.remcomp.fr> wrote:
>
>>Mark McIntyre a écrit :
>>> On Wed, 03 Jan 2007 02:45:18 +0100, in comp.lang.c , jacob navia
>>>
>>>>>Obviously if that were the case, pointers would have to be larger than
>>>>>32 bits.
>>>
>>> This doesn't actually follow. I can envisage a segmented architecture
>>> which solved this problem.
>>
>>Excuse me but then the pointer is bigger than 32 bits!!!
>
> It still doesn't follow. The pointer returned by malloc need not be a
> complete absolute reference. Magic can go on behind the scenes, much
> as it did with 16-bit pointers used to reference 20-bit memory.

I assume you're speaking of the segmented 16-bit x86 architecture
as featured in MS-DOS compilers. The object pointers used by
these compilers in some modes were 16 bits wide, and they could
only address 64 kB worth of data. You could use
implementation-specific extensions to address more than 16 kB of
memory, or you could switch your compiler to a mode where
pointers were 32 bits long, but there was no way to address more
than 64 kB of objects with 16-bit pointers and without using C
extensions.
--
int main(void){char p[]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz.\
\n",*q="kl BIcNBFr.NKEzjwCIxNJC";int i=sizeof p/2;char *strchr();int putchar(\
);while(*q){i+=strchr(p,*q++)-p;if(i>=(int)sizeof p)i-=sizeof p-1;putchar(p[i]\
);}return 0;}

Richard Tobin

unread,
Jan 4, 2007, 6:34:33 PM1/4/07
to
In article <q72rp2t9p839q3mva...@4ax.com>,
Mark McIntyre <markmc...@spamcop.net> wrote:

>>> This doesn't actually follow. I can envisage a segmented architecture
>>> which solved this problem.

>>Excuse me but then the pointer is bigger than 32 bits!!!

>It still doesn't follow. The pointer returned by malloc need not be a
>complete absolute reference. Magic can go on behind the scenes, much
>as it did with 16-bit pointers used to reference 20-bit memory.

So how can it be used to access all the bytes of the object?

jacob navia

unread,
Jan 4, 2007, 6:39:57 PM1/4/07
to
Richard Tobin a écrit :

> In article <q72rp2t9p839q3mva...@4ax.com>,
> Mark McIntyre <markmc...@spamcop.net> wrote:
>
>
>>>>This doesn't actually follow. I can envisage a segmented architecture
>>>>which solved this problem.
>
>
>>>Excuse me but then the pointer is bigger than 32 bits!!!
>
>
>>It still doesn't follow. The pointer returned by malloc need not be a
>>complete absolute reference. Magic can go on behind the scenes, much
>>as it did with 16-bit pointers used to reference 20-bit memory.
>
>
> So how can it be used to access all the bytes of the object?
>
> -- Richard

You could NOT address more than 64 k with 16 bit pointers.

You could address 640K with a 32 bit pointer composed of
a segment and an offset.

Can't you understand that? There were two halves to
the pointers, a segment part, addressed with the segment
registers es, ds, and ss for extra segment, data segment
and stack segment, and a pointer part, i.e. a 16 bit integer
that addressed the bytes from zero to 65535.

In some cases the addressing was implicit,and declared
in those famous

ASSUME

statements in the assembly code.

Obviously each 16 bit pointer could address only 64K!!!
Mr McIntyre has strong hallucinations when he supposes that
you can have more than 64K with 16 bit pointers!

It is loading more messages.
0 new messages