size_t or int for malloc-type functions?
jacob navia
that asked to know how he could find out the size of
a block allocated with malloc.
As always when I post something, the same group
of people started to try to find possible errors,
a harmless passtime they seem to enjoy.
One of their remarks was that I used "int" instead of
"size_t" for the input of my allocator function.
As I explained, I prefer a signed type to the
unsigned size_t because small negative number will be
confused with huge integers when interpreted as unsigned.
I researched a bit the subject, and I found a type
ssize_t
The name is defined for instance in
http://www.delorie.com/gnu/docs/glibc/libc_239.html
Data Type: ssize_t
This data type is used to represent the sizes of blocks that can be
read or written in a single operation. It is similar to size_t, but must
be a signed type.
Another reference to this type appears in:
http://bochs.sourceforge.net/cgi-bin/lxr/ident?i=ssize_t
with
#define ssize_t long
This concern with the usage of an unsigned type that can be
easily lead to errors (of course only for people that do
make errors like me...) is also expressed in the document
ISO/IEC JTC1 SC22 WG14 N1135 :
"Specification for Safer, More Secure C Library Functions"
where they propose:
Extremely large object sizes are frequently a sign that an object’s size
was calculated incorrectly. For example, negative numbers appear as very
large positive numbers when converted to an unsigned type like size_t.
Also, some implementations do not support objects as large as the
maximum value that can be represented by type size_t.
For those reasons, it is sometimes beneficial to restrict the range of
object sizes to detect programming errors.
They propose having an unsigned rsize_t, but a macro RSIZE_MAX that
would limit the range of the object size.
I post this to document why having an "int" as an argument to a
malloc-type function is not such a bad idea.
Your opinion may differ.
jacob
P.J. Plauger
news:4597b950$0$27413$ba4a...@news.orange.fr...
The problem with int is that it throws away half the address space.
This is a *big* issue with 16-bit address spaces, but of course
such machines are largely relegated to embedded systems, and small
ones at that. Nevertheless, I regularly need to deal with objects
bigger than half the address space even today, perhaps because I
write so much systems code. So I find the choice of slicing the
address space in half arbitrary and potentially dangerous.
That's why I pushed for the notion of an RSIZE_MAX to accompany
the unsigned rsize_t that Microsoft put forth in TR 24731. You
can set it to:
-- (size_t)-1 >> 2 if you want the same protection as a signed
byte count
-- some other value if you know how big objects can really be,
and get maximum protection against silly sizes
-- (size_t)-1 if you want to turn the damned checking off
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
jacob navia
>
> The problem with int is that it throws away half the address space.
> This is a *big* issue with 16-bit address spaces, but of course
> such machines are largely relegated to embedded systems, and small
> ones at that.
Yes. I see this too as a problem, but then, in such small
systems, the amount of ram tends to be very small too, and
the situation remains the same.
In a DSP I used last year the total amount of RAM was
4K, so a sizeof(int) 16 bits I had plenty of space
anyway :-)
> Nevertheless, I regularly need to deal with objects
> bigger than half the address space even today, perhaps because I
> write so much systems code.
In that case it can be useful to have two sets of malloc functions
maybe, one for the small allocations, and another for the "big"
ones.
> So I find the choice of slicing the
> address space in half arbitrary and potentially dangerous.
>
> That's why I pushed for the notion of an RSIZE_MAX to accompany
> the unsigned rsize_t that Microsoft put forth in TR 24731. You
> can set it to:
>
> -- (size_t)-1 >> 2 if you want the same protection as a signed
> byte count
>
> -- some other value if you know how big objects can really be,
> and get maximum protection against silly sizes
>
> -- (size_t)-1 if you want to turn the damned checking off
>
Checking can be a nuisance but it can be an advantage sometimes. It
depends on the situation.
Thanks for your input.
jacob
Richard Heathfield
<snip>
> As I explained, I prefer a signed type to the
> unsigned size_t because small negative number will be
> confused with huge integers when interpreted as unsigned.
Why would you need a small negative number as an argument to malloc? Are you
trying to allocate a negative amount of memory?
>
> I researched a bit the subject, and I found a type
>
> ssize_t
>
> The name is defined for instance in
> http://www.delorie.com/gnu/docs/glibc/libc_239.html
It's a POSIX type, not a C type. Try comp.unix.programmer.
<snip>
--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.
Eric Sosman
> P.J. Plauger a écrit :
>>
>> The problem with int is that it throws away half the address space.
>> This is a *big* issue with 16-bit address spaces, but of course
>> such machines are largely relegated to embedded systems, and small
>> ones at that.
>
> Yes. I see this too as a problem, but then, in such small
> systems, the amount of ram tends to be very small too, and
> the situation remains the same.
It's the same, and yet not the same. The size of a data
structure does not usually scale linearly with the width of
its constituent words. I have not worked with 16-bit machines
for a number of years now, but when I did it was not unusual to
want to allocate >32K to a single object. By contrast, I have
never needed to allocate >2G as a single object on a 32-bit
system.
YMMV, but it seems to me that the need to "use all the
bits" is more pressing on small systems than on large ones.
>> Nevertheless, I regularly need to deal with objects
>> bigger than half the address space even today, perhaps because I
>> write so much systems code.
>
> In that case it can be useful to have two sets of malloc functions
> maybe, one for the small allocations, and another for the "big"
> ones.
Walk that road with caution: I've been down it, and it is
twisty and dangerous, beset with bandits and worse. You need
to exercise a *lot* of discipline to segregate the memory blocks
you get from multiple allocators: If you obtain a block from a
hypothetical lmalloc() and release it with plain free(), chaos
is likely.
But then, you may be considering a different path, like an
smalloc(int) as a sanity-checker wrapped around malloc(size_t):
#include <stdlib.h>
void *smalloc(int bytes) {
return (bytes < 0) ? NULL : malloc(bytes);
}
If so, the hobgoblin of multiple allocators disappears. But
implementing smalloc() is no trick at all; you can do it for
yourself with the tools already provided, if you like. You might
want to consider leaving the argument type as size_t, though, to
give yourself more freedom in setting the failure threshold:
#include <stdlib.h>
#include <limits.h>
#define THRESHOLD ((INT_MAX / 2u + 1) * 3) /* for example */
void *smalloc(size_t bytes) {
return (bytes < THRESHOLD) ? malloc(bytes) : NULL;
}
Returning for a moment to your original motivation for using
a signed argument:
> As I explained, I prefer a signed type to the
> unsigned size_t because small negative number will be
> confused with huge integers when interpreted as unsigned.
I don't see this as a big problem. If the program blunders and
asks for -42 bytes, conversion to size_t turns this into a request
for a very large amount of memory (by the machine's standards,
and assuming a size_t of similar "width" to the addresses). The
request almost certainly fails and returns NULL, with no harm
done. So instead of filtering the argument going into malloc(),
it might be more useful to monitor the result:
#include <stdlib.h>
void *zmalloc(size_t bytes) {
void *new = malloc(bytes);
if (new == NULL)
print_debugging_info();
return new;
}
This would catch absurd arguments (they'll provoke malloc()
failure) and also help track down the slobs who call malloc()
and fail to check the result for NULL. If desired, one could
also monitor the incoming argument for "reasonableness," to
help find code that's making excessively greedy requests.
--
Eric Sosman
eso...@acm-dot-org.invalid
jacob navia
> jacob navia wrote:
>
>> P.J. Plauger a écrit :
>>
>>>
>>> The problem with int is that it throws away half the address space.
>>> This is a *big* issue with 16-bit address spaces, but of course
>>> such machines are largely relegated to embedded systems, and small
>>> ones at that.
>>
>>
>> Yes. I see this too as a problem, but then, in such small
>> systems, the amount of ram tends to be very small too, and
>> the situation remains the same.
>
>
> It's the same, and yet not the same. The size of a data
> structure does not usually scale linearly with the width of
> its constituent words. I have not worked with 16-bit machines
> for a number of years now, but when I did it was not unusual to
> want to allocate >32K to a single object. By contrast, I have
> never needed to allocate >2G as a single object on a 32-bit
> system.
>
> YMMV, but it seems to me that the need to "use all the
> bits" is more pressing on small systems than on large ones.
>
With a bit of more reflection I think you (and Mr Plauger)
have a point here Eric.
Yes, in 16 bit systems all bits may be needed, and this
may be more important than catching a wrong allocation.
jacob
Kenny McCormack
Richard Heathfield <r...@see.sig.invalid> wrote:
>jacob navia said:
>
><snip>
>
>> As I explained, I prefer a signed type to the
>> unsigned size_t because small negative number will be
>> confused with huge integers when interpreted as unsigned.
>
>Why would you need a small negative number as an argument to malloc? Are you
>trying to allocate a negative amount of memory?
Explanation for those (like RH) with limited ability to read between the
lines: What's going on here is the combination of buggy programming
(incompetent programmers) who calculate things and then pass the results
to library functions and buggy implementations (like Linux) that allow
all mallocs to succeed.
You put the two together, and you get chaos.
jacob navia
> jacob navia said:
>
> <snip>
>
>
>>As I explained, I prefer a signed type to the
>>unsigned size_t because small negative number will be
>>confused with huge integers when interpreted as unsigned.
>
>
> Why would you need a small negative number as an argument to malloc? Are you
> trying to allocate a negative amount of memory?
>
Can't you read?
This thread is not for you
I said in my original message:
"... the usage of an unsigned type that can
easily lead to errors (of course only for people that do
make errors like me...)"
You never do any errors heathfield since you are a "competent
C programmer". Please go away. This thread is about errors
and their prevention. You do not need it.
P.J. Plauger
> P.J. Plauger a écrit :
>>
>> The problem with int is that it throws away half the address space.
>> This is a *big* issue with 16-bit address spaces, but of course
>> such machines are largely relegated to embedded systems, and small
>> ones at that.
>
> Yes. I see this too as a problem, but then, in such small
> systems, the amount of ram tends to be very small too, and
> the situation remains the same.
>
> In a DSP I used last year the total amount of RAM was
> 4K, so a sizeof(int) 16 bits I had plenty of space
> anyway :-)
That may be your experience, but mine has been that 16-bit systems
often have >32KB of memory. Anything that interferes with handling
all of memory in one go is sure to cause trouble, sooner or later.
>> Nevertheless, I regularly need to deal with objects
>> bigger than half the address space even today, perhaps because I
>> write so much systems code.
>
> In that case it can be useful to have two sets of malloc functions
> maybe, one for the small allocations, and another for the "big"
> ones.
A nice complement to the malloc(0) discussion elsegroup. My idea
of elegance is to have one function that accepts any size from 0 to
the largest representable object, and do something uniformly sane
with it. But YMMV.
>> So I find the choice of slicing the
>> address space in half arbitrary and potentially dangerous.
>>
>> That's why I pushed for the notion of an RSIZE_MAX to accompany
>> the unsigned rsize_t that Microsoft put forth in TR 24731. You
>> can set it to:
>>
>> -- (size_t)-1 >> 2 if you want the same protection as a signed
>> byte count
>>
>> -- some other value if you know how big objects can really be,
>> and get maximum protection against silly sizes
>>
>> -- (size_t)-1 if you want to turn the damned checking off
>>
>
> Checking can be a nuisance but it can be an advantage sometimes. It
> depends on the situation.
But you *always* have to check if you want to keep your program sane.
The only advantage of a signed byte count is that it makes it slightly
easier to check for a clearly bogus size. And the program had still
better check for a null pointer return from malloc. I find it hard to
conceive of a situation in this day and age where you *wouldn't* want
both malloc and the caller of malloc to check. I can't imagine what
situation would make such checking an avoidable "nuisance".
> Thanks for your input.
CBFalconer
>
> Rcently I posted code in this group, to help a user
> that asked to know how he could find out the size of
> a block allocated with malloc.
>
> As always when I post something, the same group
> of people started to try to find possible errors,
> a harmless passtime they seem to enjoy.
>
> One of their remarks was that I used "int" instead of
> "size_t" for the input of my allocator function.
>
>
> I post this to document why having an "int" as an argument to a
> malloc-type function is not such a bad idea.
>
> Your opinion may differ.
It does. Are you or are you not the implementor of lcc-win32? If
so, you can easily limit the range accepted within the malloc code,
without fouling the specifications of the standard. If not you
shouldn't be fooling with routines that are defined in the
standard.
--
Some informative links:
<http://members.fortunecity.com/nnqweb/> (newusers)
<http://www.catb.org/~esr/faqs/smart-questions.html>
<http://www.caliburn.nl/topposting.html>
<http://www.netmeister.org/news/learn2quote.html>
<http://cfaj.freeshell.org/google/> (taming google)
Richard Heathfield
> Richard Heathfield a écrit :
>> jacob navia said:
>>
>> <snip>
>>
>>
>>>As I explained, I prefer a signed type to the
>>>unsigned size_t because small negative number will be
>>>confused with huge integers when interpreted as unsigned.
>>
>>
>> Why would you need a small negative number as an argument to malloc? Are
>> you trying to allocate a negative amount of memory?
>>
>
> Can't you read?
I can read just fine. Why would you need a small negative number as an
argument to malloc?
> This thread is not for you
This is Usenet. If you want a private discussion, use email.
> I said in my original message:
>
> "... the usage of an unsigned type that can
> easily lead to errors (of course only for people that do
> make errors like me...)"
On the contrary, using an unsigned type as malloc's argument *eliminates*
the possibility of requesting a negative size.
<nonsense snipped>
Stephen Sprunk
> One of their remarks was that I used "int" instead of
> "size_t" for the input of my allocator function.
As long as your allocator isn't named malloc(), calloc(), or realloc(),
that's up to you.
Of course, one must wonder why you'd ever want to allow folks to request
negative amounts of memory and exactly what it means if they do. Making
the argument unsigned (whether size_t or something else) makes it much
more obvious what you intended the proper use to be, and it doubles the
number of legitimate argument values.
> As I explained, I prefer a signed type to the
> unsigned size_t because small negative number will be
> confused with huge integers when interpreted as unsigned.
...
> This concern with the usage of an unsigned type that can be
> easily lead to errors (of course only for people that do
> make errors like me...)
The typical C programmer answer to this is "don't do that". C's
philosophy, at its core, is to give programmers the tools to shoot
themselves in the foot if they so desire. If you frequently shoot
yourself in the foot, then consider using another language, like BASIC,
that doesn't give you that option, or spend more time learning how to
code defensively or how to use your debugger.
> is also expressed in the document ISO/IEC JTC1 SC22 WG14 N1135 :
> "Specification for Safer, More Secure C Library Functions"
> where they propose:
>
> Extremely large object sizes are frequently a sign that an object’s
> size
> was calculated incorrectly. For example, negative numbers appear as
> very
> large positive numbers when converted to an unsigned type like size_t.
>
> Also, some implementations do not support objects as large as the
> maximum value that can be represented by type size_t.
>
> For those reasons, it is sometimes beneficial to restrict the range of
> object sizes to detect programming errors.
>
> They propose having an unsigned rsize_t, but a macro RSIZE_MAX that
> would limit the range of the object size.
There is nothing preventing the implementor from returning NULL if the
request to malloc() et al appears to be erroneous, such as being "a
small negative number" converted to unsigned. We do not need a change
to the standard for this, since the standard already allows the
implementation to return NULL for any reason it wishes -- just like we
don't need a change to the standard to add bounds-checking pointers.
Implementors are free to do all sorts of extra work behind the scenes to
try to prevent problems, if they wish.
Most implementors, however, take the position that it's more important
to give people the freedom to do unexpected things than to treat them
like idiots who need adult supervision. Insulting your customers is not
a sustainable business practice.
S
--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
--
Posted via a free Usenet account from http://www.teranews.com
Richard Tobin
Richard Heathfield <r...@see.sig.invalid> wrote:
>> "... the usage of an unsigned type that can
>> easily lead to errors (of course only for people that do
>> make errors like me...)"
>On the contrary, using an unsigned type as malloc's argument *eliminates*
>the possibility of requesting a negative size.
You seem to be deliberately misunderstanding - surely reading the rest
of the thread makes it clear. The mistake is that you inadvertently
pass an incorrect value to malloc(). Sometimes such incorrect values
will be negative, and if the argument to malloc() was signed, it
could notice this error, rather than treating it as a very large
positive value.
On most current general-purpose computers, such a large positive value
will fail anyway, so it wouldn't be very helpful on those systems.
-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
CBFalconer
> Richard Heathfield a écrit :
>
... snip ...
>>
>> Why would you need a small negative number as an argument to
>> malloc? Are you trying to allocate a negative amount of memory?
>
> Can't you read?
>
> This thread is not for you
This is a *public* newsgroup. If you don't want comments on your
posts, don't post.
--
Merry Christmas, Happy Hanukah, Happy New Year
Joyeux Noel, Bonne Annee.
Chuck F (cbfalconer at maineline dot net)
<http://cbfalconer.home.att.net>
Kenny McCormack
CBFalconer <cbfal...@maineline.net> wrote:
>jacob navia wrote:
>> Richard Heathfield a écrit :
>>
>... snip ...
>>>
>>> Why would you need a small negative number as an argument to
>>> malloc? Are you trying to allocate a negative amount of memory?
>>
>> Can't you read?
>>
>> This thread is not for you
>
>This is a *public* newsgroup. If you don't want comments on your
>posts, don't post.
Are you really that stupid? Are you really so stupid that you don't
follow what Jacob is really saying when he says "This thread is not for
you"?
You
Macho programmers need not apply.
folks
Macho programmers need not apply.
need
Macho programmers need not apply.
to
Macho programmers need not apply.
learn
Macho programmers need not apply.
to
Macho programmers need not apply.
read
Macho programmers need not apply.
between
Macho programmers need not apply.
the
Macho programmers need not apply.
lines.
Kenny McCormack
Stephen Sprunk <ste...@sprunk.org> wrote:
...
>Most implementors, however, take the position that it's more important
>to give people the freedom to do unexpected things than to treat them
>like idiots who need adult supervision. Insulting your customers is not
>a sustainable business practice.
Seems to work just fine for MS. But then again, they're not "most
implementors", I suppose. They're insignificant.
Mark McIntyre
<ja...@jacob.remcomp.fr> wrote:
>This thread is not for you
I'm sorry, where does it say in the Rules of Usenet that some threads
are forbidden to some posters?
(snip collection of sarcastic and gratuitous insults).
--
Mark McIntyre
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Mark McIntyre
(Richard Tobin) wrote:
>The mistake is that you inadvertently
>pass an incorrect value to malloc(). Sometimes such incorrect values
>will be negative,
Thats impossible, if the argument is unsigned.
>and if the argument to malloc() was signed, it
>could notice this error, rather than treating it as a very large
>positive value.
Huh? So you sacrifice half the possible address space, to cater for a
stupid programming error. Actually, this sounds about par for the
course, given the recent string thread.
>On most current general-purpose computers, such a large positive value
>will fail anyway, so it wouldn't be very helpful on those systems.
It works fine on all my general purpose computers. That is to say, it
fails if the attempt is for too much memory, and it works otherwise.
What else would we reasonably expect it to do?
jacob navia
> On Sun, 31 Dec 2006 17:45:42 +0100, in comp.lang.c , jacob navia
> <ja...@jacob.remcomp.fr> wrote:
>
>
>>This thread is not for you
>
>
> I'm sorry, where does it say in the Rules of Usenet that some threads
> are forbidden to some posters?
>
I said in my original post
As always when I post something, the same group
of people started to try to find possible errors,
a harmless passtime they seem to enjoy.
You belong to that group.
jacob navia
> "jacob navia" <ja...@jacob.remcomp.fr> wrote in message
> news:4597cefa$0$27386$ba4a...@news.orange.fr...
>
>
>>P.J. Plauger a écrit :
>>
>>>The problem with int is that it throws away half the address space.
>>>This is a *big* issue with 16-bit address spaces, but of course
>>>such machines are largely relegated to embedded systems, and small
>>>ones at that.
>>
>>Yes. I see this too as a problem, but then, in such small
>>systems, the amount of ram tends to be very small too, and
>>the situation remains the same.
>>
>>In a DSP I used last year the total amount of RAM was
>>4K, so a sizeof(int) 16 bits I had plenty of space
>>anyway :-)
>
>
> That may be your experience, but mine has been that 16-bit systems
> often have >32KB of memory. Anything that interferes with handling
> all of memory in one go is sure to cause trouble, sooner or later.
>
Yes. You are right in this point. For 16 bit systems the
lost of 32K of addressing space is quite a hit. Specially
if you do have the full 64K.
jacob
Kenny McCormack
Mark McIntyre <markmc...@spamcop.net> wrote:
>On 31 Dec 2006 22:25:40 GMT, in comp.lang.c , ric...@cogsci.ed.ac.uk
>(Richard Tobin) wrote:
>
>>The mistake is that you inadvertently
>>pass an incorrect value to malloc(). Sometimes such incorrect values
>>will be negative,
>
>Thats impossible, if the argument is unsigned.
Incorrect. In the C language, if you pass a signed integer (for
example, the number -4) to a function where the formal parameter of that
function is declared unsigned (for example, malloc()), the "default
arithmetic conversions" (something you should read up on, by the way),
will convert that signed number to an unsigned, in order to be
compatible with the formal function declaration. On many systems,
numbers like -4 (negative numbers with small absolute values) will be
converted to very large unsigned numbers.
That's what we are talking about here. Do try to keep up.
>>and if the argument to malloc() was signed, it
>>could notice this error, rather than treating it as a very large
>>positive value.
>
>Huh? So you sacrifice half the possible address space, to cater for a
>stupid programming error. Actually, this sounds about par for the
>course, given the recent string thread.
>
>>On most current general-purpose computers, such a large positive value
>>will fail anyway, so it wouldn't be very helpful on those systems.
>
>It works fine on all my general purpose computers. That is to say, it
>fails if the attempt is for too much memory, and it works otherwise.
>What else would we reasonably expect it to do?
Google for "malloc overcommit Linux OOM" and get back to us, OK?
Richard Tobin
Mark McIntyre <markmc...@spamcop.net> wrote:
>>The mistake is that you inadvertently
>>pass an incorrect value to malloc(). Sometimes such incorrect values
>>will be negative,
>Thats impossible, if the argument is unsigned.
Why don't you take a leaf out of P J Plauger's book and address the
real question, instead of being a tosser and pretending not to
understand?
The value the user supplies is negative. It's converted to a positive
value as part of the function calling process.
>>and if the argument to malloc() was signed, it
>>could notice this error, rather than treating it as a very large
>>positive value.
>Huh? So you sacrifice half the possible address space, to cater for a
>stupid programming error. Actually, this sounds about par for the
>course, given the recent string thread.
Most current operating systems on general-purpose computers do not
allow the user to use the full theoretical address space. It's
common for 32-bit systems to only allow a single process to address
2^31 bytes.
>>On most current general-purpose computers, such a large positive
>>value will fail anyway, so it wouldn't be very helpful on those
>>systems.
>It works fine on all my general purpose computers. That is
>to say, it fails if the attempt is for too much memory, and it works
>otherwise.
And the result of converting, say, -1 to size_t is typically "too much
memory". So malloc(-1) will fail.
Kenny McCormack
Richard Tobin <ric...@cogsci.ed.ac.uk> wrote:
>In article <aalgp2tppn2o65ffq...@4ax.com>,
>Mark McIntyre <markmc...@spamcop.net> wrote:
>
>>>The mistake is that you inadvertently
>>>pass an incorrect value to malloc(). Sometimes such incorrect values
>>>will be negative,
>
>>Thats impossible, if the argument is unsigned.
>
>Why don't you take a leaf out of P J Plauger's book and address the
>real question, instead of being a tosser and pretending not to
>understand?
Because this is clc and that's what they (McIntyre and Heathfield, among
others) do. Get with the program!
>The value the user supplies is negative. It's converted to a positive
>value as part of the function calling process.
Obviously. But where's the fun in that?
Mike Wahler
> Mark McIntyre a écrit :
>> On Sun, 31 Dec 2006 17:45:42 +0100, in comp.lang.c , jacob navia
>> <ja...@jacob.remcomp.fr> wrote:
>>
>>
>>>This thread is not for you
>>
>>
>> I'm sorry, where does it say in the Rules of Usenet that some threads
>> are forbidden to some posters?
>>
> That was of course just an advise.
>
> I said in my original post
>
> As always when I post something, the same group
> of people started to try to find possible errors,
When I post something here, I sincerely hope that
others will attempt to find any errors it might
contain. Because if they do, and point them out,
it's to my (and probably others') benefit.
> a harmless passtime they seem to enjoy.
Not only harmless, but beneficial.
> You belong to that group.
The more members the better.
-Mike
Richard Heathfield
> In article <o_ydndkoXMkLjQXY...@bt.com>,
> Richard Heathfield <r...@see.sig.invalid> wrote:
>
>>> "... the usage of an unsigned type that can
>>> easily lead to errors (of course only for people that do
>>> make errors like me...)"
>
>>On the contrary, using an unsigned type as malloc's argument *eliminates*
>>the possibility of requesting a negative size.
>
> You seem to be deliberately misunderstanding - surely reading the rest
> of the thread makes it clear.
I'm failing to understand how such a mistake is considered so likely and so
undetectable that we suddenly have to change everybody's standard library.
> The mistake is that you inadvertently
> pass an incorrect value to malloc().
>
> Sometimes such incorrect values will be negative,
Um, *what*?
p = malloc(n * sizeof *p);
is a very common idiom and about the most-typed code fragment in this group.
It is a well-known, popular idiom.
n is an object count, so it makes sense for it to be a size_t.
sizeof *p is an object size, and is a size_t.
So we have size_t * size_t - how, precisely, will that produce a negative
number when passed to a function taking size_t as an argument?
<snip>
jacob navia
How to ignore errors. An example of bad programming practices
-------------------------------------------------------------
Mr heathfield says that:
> p = malloc(n * sizeof *p);
>
> is a very common idiom and about the most-typed code fragment in this
group.
> It is a well-known, popular idiom.
Yes, but it is quite dangerous.
Suppose:
struct S {
unsigned n;
double d;
double data[8192];
};
I want to allocate 65520 objects of size 65 552 bytes each.
If you do that in a 32 bit system what you obtain?
i=65520, i*sizeof(*p)=4,294,967,040 as signed number = -256
WE OBTAIN A NEGATIVE NUMBER.
Even WORST.
What happens if you want to allocate just ONE object more?
i=65521, i*sizeof(*p)=65,296 as signed number = 65,296
YOU HAVE AN OVERFLOW and you get a POSITIVE BUT WRONG number!!!
You end allocating space for ONE object and not for
65521!!!!!!
And there is NO WAY a malloc will tell you about any errors
since the request is perfectly normal.
I am aware of this bug because we HAVE DISCUSSED this here and
I realized that the calloc implementation of lcc-win32 furnished
by the windows system had also this BUG. I immediately rewrote
the calloc function that SHOULD ALWAYS TEST FOR OVERFLOW.
What is MACHO programming?
--------------------------
It is an attitude of wrong security and lack of critical distance
to oneself. This attitude is specially pervasive in bright and
experienced programmers. You get into the frame of mind:
"I know what I am doing, I do not need barriers or security
considerations".
This is similar to:
"I am driving since 15 years now, never had an accident.
Why should I use the seat belts???"
Richard Heathfield
<snip>
> How to ignore errors. An example of bad programming practices
> -------------------------------------------------------------
>
> Mr heathfield says that:
>
> > p = malloc(n * sizeof *p);
> >
> > is a very common idiom and about the most-typed code fragment in this
> group.
> > It is a well-known, popular idiom.
>
> Yes, but it is quite dangerous.
>
> Suppose:
> struct S {
> unsigned n;
> double d;
> double data[8192];
> };
>
> I want to allocate 65520 objects of size 65 552 bytes each.
>
> If you do that in a 32 bit system what you obtain?
If by "32 bit system" you mean that size_t is a 32-bit type, then its
maximum value is 65535. If you don't mean that, what do you mean?
> i=65520, i*sizeof(*p)=4,294,967,040 as signed number = -256
>
> WE OBTAIN A NEGATIVE NUMBER.
No, you don't. size_t is an unsigned type.
> Even WORST.
>
> What happens if you want to allocate just ONE object more?
> i=65521, i*sizeof(*p)=65,296 as signed number = 65,296
>
> YOU HAVE AN OVERFLOW and you get a POSITIVE BUT WRONG number!!!
No, you don't get overflow with unsigned types.
<snip>
> I realized that the calloc implementation of lcc-win32 furnished
> by the windows system had also this BUG.
It doesn't surprise me that you found a bug in lcc-win32. That is presumably
your job. But I am surprised that you found *this* bug in lcc-win32,
because it's so easy to get this right. If you have a quantity that cannot
possibly be negative (a count, a size, whatever), use an unsigned type.
> I immediately rewrote the calloc function that SHOULD ALWAYS TEST FOR
> OVERFLOW.
Unsigned types don't overflow.
<nonsense snipped>
Cesar Rabak
like to find it, because I spend a lot of time explaining this to
newcomers in my work...
Richard Tobin
Richard Heathfield <r...@see.sig.invalid> wrote:
>> The mistake is that you inadvertently
>> pass an incorrect value to malloc().
>> Sometimes such incorrect values will be negative,
>Um, *what*?
>
>p = malloc(n * sizeof *p);
>
>is a very common idiom and about the most-typed code fragment in this group.
>It is a well-known, popular idiom.
>
>n is an object count, so it makes sense for it to be a size_t.
>sizeof *p is an object size, and is a size_t.
>
>So we have size_t * size_t - how, precisely, will that produce a negative
>number when passed to a function taking size_t as an argument?
Well obviously it won't. But the idea was to protect against
programmers who make mistakes, and as we all know not all programmers
use that idiom.
As I said, I don't think the proposal would do much good, but surely
you can see what the idea was?
Richard Heathfield
> In article <3ZOdnawJSZo...@bt.com>,
> Richard Heathfield <r...@see.sig.invalid> wrote:
>
<snip>
>>So we have size_t * size_t - how, precisely, will that produce a negative
>>number when passed to a function taking size_t as an argument?
>
> Well obviously it won't.
Quite so.
> But the idea was to protect against
> programmers who make mistakes, and as we all know not all programmers
> use that idiom.
The fix, then, is obvious.
>
> As I said, I don't think the proposal would do much good, but surely
> you can see what the idea was?
It appeared to be an attempt to introduce the additional risk of overflow to
the existing risk of specifying an allocation amount other than the amount
actually required. No, I don't see why this would be of benefit.
Clark S. Cox III
>
> jacob navia said:
>
> > I want to allocate 65520 objects of size 65 552 bytes each.
> >
> > If you do that in a 32 bit system what you obtain?
>
> If by "32 bit system" you mean that size_t is a 32-bit type, then its
> maximum value is 65535. If you don't mean that, what do you mean?
Umm, a 32-bit size_t (with no padding bits, etc.) can hold up to about
4-billion.
--
Clark S. Cox III
clar...@gmail.com
jacob navia
Yes, but that is only logic.
This doesn't count as argument as it seems.
heathfield will answer that unsigned ints
wrap around as specified in the standard, so
that is not an overflow (or a similar nonsense.)
Clark S. Cox III
Oh, don't take my post the wrong way; just because I corrected a single
factual error in his post, I still agree with him; and disagree with you.
There is no way to obtain a negative size_t.
Unsigned types don't overflow.
A signed argument to malloc makes absolutely no sense.
Richard Heathfield
Oh, so it can, provided you use those silly little American billions. :-)
In which case, what's the big deal? 65520 * 65552 is 4294967040 which is a
little smaller than 4294967295. Probably the allocation will fail (because
it's just so colossal) and NULL will be returned, but if the RAM is there,
it may even succeed. I don't see a problem here.
Richard Heathfield
> Clark S. Cox III a écrit :
>> Richard Heathfield wrote:
>>
>>>jacob navia said:
>>>
>>>
>>>>I want to allocate 65520 objects of size 65 552 bytes each.
>>>>
>>>>If you do that in a 32 bit system what you obtain?
>>>
>>>If by "32 bit system" you mean that size_t is a 32-bit type, then its
>>>maximum value is 65535. If you don't mean that, what do you mean?
>>
>>
>> Umm, a 32-bit size_t (with no padding bits, etc.) can hold up to about
>> 4-billion.
>>
>
> Yes, but that is only logic.
Well, I wouldn't have called it that. I'd have called it a mistake on my
part. They happen.
> This doesn't count as argument as it seems.
<shrug> What's to argue? On the matter of 32-bit unsigned integer types, I
made a simple mistake in fact, but the principle remains sound.
> heathfield will answer that unsigned ints
> wrap around as specified in the standard, so
> that is not an overflow
Arithmetic on unsigned integer types is reduced modulo 2^(MAX+1) where MAX
is the maximum value that can be stored in an object of that type.
Therefore, overflow doesn't happen.
> (or a similar nonsense.)
If you want to think of the truth as being nonsense, that's up to you, but
it doesn't stop it from being the truth.
jacob navia
>
>>Why I obtain
>>
>>65521 * 65552 --> 65296 ????
>
>
> Unsigned integer arithmetic is done by reducing the unsigned number to the
> range 0 to MAX, where MAX is the maximum value for the type.
This "arithmetic" has nothing to do with arithmetic
and overflow is just declared normal, leading
to unexpected results.
But it is useless to discuss with you
HINT:
I did not answer to you but to somebody else
Overflow doesn't exist?
65521 *65552 --> 65296 ???
OK.
Then if I want to allocate 65521 objects of
65552 bytes each I obtain a block 65296 bytes long
WONDERFUL!
Then, I can write more than 4GB into 65296 bytes
You should work for RAM producers heathfield.
The would be very inetersted in your
method for writing 4GB into 65296 bytes!!!
:-)
>
>>The whole point of my argumentation is that the "idiom"
>>
>>result = malloc(n * sizeof *p);
>>
>>is a dangerous construct.
>
>
> You have not demonstrated this to be the case.
>
I have demonstrated that the multiplication
65521 * 65552 gives 65296, what is clearly not
enough space to store 65521 objects of size 65552
each. You are free to call that "multiplication that
doesn't overflow".
You can babble as much as you like but that is a fact.
jacob navia
> jacob navia wrote:
>
>>Clark S. Cox III a écrit :
>>
>>>Richard Heathfield wrote:
>>>
>>>
>>>>jacob navia said:
>>>>
>>>>
>>>>
>>>>>I want to allocate 65520 objects of size 65 552 bytes each.
>>>>>
>>>>>If you do that in a 32 bit system what you obtain?
>>>>
>>>>If by "32 bit system" you mean that size_t is a 32-bit type, then its
>>>>maximum value is 65535. If you don't mean that, what do you mean?
>>>
>>>
>>>Umm, a 32-bit size_t (with no padding bits, etc.) can hold up to about
>>>4-billion.
>>>
>>
>>Yes, but that is only logic.
>>
>>This doesn't count as argument as it seems.
>>
>>heathfield will answer that unsigned ints
>>wrap around as specified in the standard, so
>>that is not an overflow (or a similar nonsense.)
>
>
>
> Oh, don't take my post the wrong way; just because I corrected a single
> factual error in his post, I still agree with him; and disagree with you.
His majesty is always right, no matter how much nonsense
he says.
> There is no way to obtain a negative size_t.
Of course not. It is an unsigned number. The whole point is that
when you have
void fn(unsigned arg);
and you write:
fn(-3);
an implicit cast from signed to unsigned is done,
what in all implementations I know leads to no
machine code, but just a change in the way the bits
of the argument are interpreted.
Obviously this is a simplification of a real situation
when the values are not explicitely gibve like in the
example above.
> Unsigned types don't overflow.
This is nonsense. Why I obtain
65521 * 65552 --> 65296 ????
You (and heathfield) are just playing with words. That
overflow is defined for unsigned numbers within C doesn't
mean that the result is mathematically VALID, or that is
the expected result.
The whole point of my argumentation is that the "idiom"
result = malloc(n * sizeof *p);
is a dangerous construct. Use
result = calloc(n, sizeof(*p))
and ensure your implementation of calloc doesn't have the bug!
> A signed argument to malloc makes absolutely no sense.
It would catch the overflow in some cases above, as I explained in
my post.
You ropinion may be different, but it would be helpful if you
tried to advance an argument, just saying
"makes no sense"
makes no sense to anyone but you.
Keith Thompson
[...]
> Can't you read?
>
> This thread is not for you
> You never do any errors heathfield since you are a "competent
> C programmer". Please go away. This thread is about errors
> and their prevention. You do not need it.
Gosh, I didn't know one person could ban someone like that.
Let me try it. navia, this newsgroup is not for you. Please go away.
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Richard Heathfield
> Clark S. Cox III a écrit :
<snip>
>> There is no way to obtain a negative size_t.
>
> Of course not. It is an unsigned number. The whole point is that
> when you have
>
> void fn(unsigned arg);
>
> and you write:
>
> fn(-3);
>
> an implicit cast from signed to unsigned is done,
There is no such thing as an implicit cast. Casts are explicit conversions.
<snip>
>> Unsigned types don't overflow.
>
> This is nonsense.
No, it isn't.
> Why I obtain
>
> 65521 * 65552 --> 65296 ????
Unsigned integer arithmetic is done by reducing the unsigned number to the
range 0 to MAX, where MAX is the maximum value for the type.
> You (and heathfield) are just playing with words. That
> overflow is defined for unsigned numbers within C doesn't
> mean that the result is mathematically VALID, or that is
> the expected result.
Overflow is not defined for unsigned numbers within C, and that's because
overflow is not *possible* for unsigned numbers within C.
The Standard makes this very clear: "A computation involving unsigned
operands can never overflow, because a result that cannot be represented by
the resulting unsigned integer type is reduced modulo the number that is
one greater than the largest value that can be represented by the resulting
unsigned integer type."
And the result is indeed mathematically valid, since unsigned integer types
with N value bits represent a ring with 2^N elements.
> The whole point of my argumentation is that the "idiom"
>
> result = malloc(n * sizeof *p);
>
> is a dangerous construct.
You have not demonstrated this to be the case.
> Use
>
> result = calloc(n, sizeof(*p))
>
> and ensure your implementation of calloc doesn't have the bug!
Better still, use malloc. If your implementation is buggy, get it fixed or
get it changed.
>> A signed argument to malloc makes absolutely no sense.
>
> It would catch the overflow in some cases above, as I explained in
> my post.
You can't overflow an unsigned type, as has been explained ad nauseam.
> You ropinion may be different, but it would be helpful if you
> tried to advance an argument, just saying
> "makes no sense" makes no sense to anyone but you.
This isn't a matter of opinion. See ISO/IEC 9899.
P.J. Plauger
news:3ZOdnawJSZo...@bt.com...
It won't, of course, but it can still be erroneous if the arithmetic
wraps around. This is because malloc is using an unsigned type to
pass a *non-negative* argument value. Wraparound may be valid for
the host type but not for the intented purpose of the conveyed
argument value. An error is an error, even if the compiled code
isn't obliged to catch it.
A robust program would have code something like:
if ((size_t)(-1) / sizeof (*p) < n)
<non-negative overflow will occur>
but it's rare to see such code in real life. Navia is correct
that calloc(n, sizeof *p) robustly passes the problem off to
calloc, which may itself be robust enough to know nonsense
when it sees it. Or not. I've seen C libraries go both ways.
Note that replacing size_t with its equivalent signed type offers
little protection for oversize allocations:
-- A valid allocation of over half of full memory yields a false
positive, since the signed version of the byte count is negative.
-- An invalid allocation of more than full memory has a 50/50
chance of yielding a negative value, or a bogus positive value.
The best of all worlds is probably a calloc that checks as it
should for wraparound, and further checks a non-wrapped byte
count against RSIZE_MAX, a la TR 24731. And this doesn't really
require any change to existing calloc calls -- just better
runtime checking.
But none of this blather about the virtues of signed arithmetic,
or the imperviousness to overflow of unsigned arithmetic,
addresses the true problem of allocating storage sanely and
reporting insane requests properly.
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
jacob navia
> The best of all worlds is probably a calloc that checks as it
> should for wraparound, and further checks a non-wrapped byte
> count against RSIZE_MAX, a la TR 24731. And this doesn't really
> require any change to existing calloc calls -- just better
> runtime checking.
>
I discovered this bug precisely in the windows CRTDLL.DLL
C runtime library provided by the system. The calloc in
there will NOT check if the multiplication overflows.
I replaced it with:
void *calloc(size_t n,size_t s)
{
long long siz = (long long)n * (long long)s;
void *result;
if (siz>>32)
return 0;
result = malloc((unsigned)siz);
if (result)
memset(result,0,(unsigned)siz);
return result;
}
In an implementation with 32 bit unsigned ints (size_t)
and 64 bit long long the multiplication can never
overflow. I just test if the upper 32 bits are different than
zero, what catches all errors of this type.
At least in my opinion. Maybe there is a bug above.
> But none of this blather about the virtues of signed arithmetic,
> or the imperviousness to overflow of unsigned arithmetic,
> addresses the true problem of allocating storage sanely and
> reporting insane requests properly.
>
True.
Mark McIntyre
<ja...@jacob.remcomp.fr> wrote:
>As always when I post something, the same group
>of people started to try to find possible errors,
>a harmless passtime they seem to enjoy.
Don't go giving yourself airs, laddy.
/Anyone/ posting to this group finds their posts scanned for errors.
It'd be pretty poor if mistakes didn't get noticed don't you think.
>You belong to that group.
Glad to hear it.
Mark McIntyre
(Richard Tobin) wrote:
>In article <aalgp2tppn2o65ffq...@4ax.com>,
>Mark McIntyre <markmc...@spamcop.net> wrote:
>
>>>The mistake is that you inadvertently
>>>pass an incorrect value to malloc(). Sometimes such incorrect values
>>>will be negative,
>
>>Thats impossible, if the argument is unsigned.
>
>Why don't you take a leaf out of P J Plauger's book and address the
>real question,
Which is? If the argument is unsigned, you can't pass a -ve value to
it. If you're using an unsigned type throughout you can't even
generate a negative value. Sure, you could somehow pass a huge
positive value to malloc. Where's the problem?
>instead of being a tosser and pretending not to
>understand?
And why don't you swivel on it, since we're being polite?
best regards
Mark McIntyre
<ja...@jacob.remcomp.fr> wrote:
>Clark S. Cox III a écrit :
>>
>> Oh, don't take my post the wrong way; just because I corrected a single
>> factual error in his post, I still agree with him; and disagree with you.
>
>His majesty is always right, no matter how much nonsense
>he says.
No, truth is always right, no matter how much nonsense people try to
conceal it behind.
Mark McIntyre
<ja...@jacob.remcomp.fr> wrote:
>
>I want to allocate 65520 objects of size 65 552 bytes each.
>
>If you do that in a 32 bit system what you obtain?
>i=65520, i*sizeof(*p)=4,294,967,040
>as signed number = -256
So what?
>WE OBTAIN A NEGATIVE NUMBER.
Only if you are stupid enough to copy it into a signed type. Clearly,
since you can't allocate negative memory, that is dopey. Use an
unsigned type, and everything is peachy.
>YOU HAVE AN OVERFLOW
You can't overflow unsigned types.
>and you get a POSITIVE BUT WRONG number!!!
>
>You end allocating space for ONE object and not for
>65521!!!!!!
>
>And there is NO WAY a malloc will tell you about any errors
>since the request is perfectly normal.
Capsitis. any day now, you'll ANNOUNCE that you are a CHAIR.
Richard Heathfield
> Richard Heathfield a écrit :
>>
>>>Why I obtain
>>>
>>>65521 * 65552 --> 65296 ????
>>
>>
>> Unsigned integer arithmetic is done by reducing the unsigned number to
>> the range 0 to MAX, where MAX is the maximum value for the type.
>
> This "arithmetic" has nothing to do with arithmetic
> and overflow is just declared normal, leading
> to unexpected results.
The results are not unexpected to those who know how unsigned integer
arithmetic works. It's not unreasonable to expect people to know a thing or
two about the language before they start using it for stuff like allocating
memory dynamically. Unsigned integer arithmetic is dealt with on page 36 of
K&R, whereas malloc isn't mentioned until page 143. Unsigned integer
arithmetic is primitive, obvious stuff. To call its results "unexpected" is
to betray one's ignorance of the language.
> But it is useless to discuss with you
Yes, and you know why? Because you never *listen*, that's why. That, You
seem to take almost every disagreement as a personal attack.
> HINT:
>
> I did not answer to you but to somebody else
If you want your answer to be read only by a particular individual, use
email. If you publish an article on Usenet, you should be prepared for it
to be read, and replied to, by anybody at all.
> Overflow doesn't exist?
Right.
> 65521 *65552 --> 65296 ???
Before you can know the answer to that question, you need to define your
universe of discourse. In N, the answer is 4295032592. In the ring of
integers modulo 2^b, the answer is 4295032592 modulo 2^b.
> OK.
>
> Then if I want to allocate 65521 objects of
> 65552 bytes each I obtain a block 65296 bytes long
On any given implementation, either size_t is big enough to store 65521 *
65552 or it isn't. If it is, there is no issue. And if it is not, your
request is meaningless, since you're asking for an object bigger than the
system can provide.
<snip>
>>
>>>The whole point of my argumentation is that the "idiom"
>>>
>>>result = malloc(n * sizeof *p);
>>>
>>>is a dangerous construct.
>>
>>
>> You have not demonstrated this to be the case.
>>
>
> I have demonstrated that the multiplication
> 65521 * 65552 gives 65296,
No, assuming you are using unsigned arithmetic it gives 4295032592 modulo
2^b where b is the number of bits in the unsigned integer type you are
using. If b is 33 or more, the answer is 4295032592, not 65296. The width
of size_t is implementation-defined.
> what is clearly not
> enough space to store 65521 objects of size 65552
> each. You are free to call that "multiplication that
> doesn't overflow".
The Standard says it doesn't overflow, and therefore it doesn't overflow
(unless the Standard is wrong, which is something you can take up with ISO
if you like, but I don't rate your chances).
> You can babble as much as you like but that is a fact.
Well, it's a highly selective fact which chooses to ignore all sorts of
other rather important facts, such as what the Standard says, how wide a
size_t is allowed to be, and so on.
Randy Howard
(in article <enb4cv$2vf4$2...@pc-news.cogsci.ed.ac.uk>):
> In article <3ZOdnawJSZo...@bt.com>,
> Richard Heathfield <r...@see.sig.invalid> wrote:
>
>>> The mistake is that you inadvertently
>>> pass an incorrect value to malloc().
>
>>> Sometimes such incorrect values will be negative,
>
>> Um, *what*?
>>
>> p = malloc(n * sizeof *p);
>>
>> is a very common idiom and about the most-typed code fragment in this
>> group.
>> It is a well-known, popular idiom.
>>
>> n is an object count, so it makes sense for it to be a size_t.
>> sizeof *p is an object size, and is a size_t.
>>
>> So we have size_t * size_t - how, precisely, will that produce a negative
>> number when passed to a function taking size_t as an argument?
>
> Well obviously it won't. But the idea was to protect against
> programmers who make mistakes, and as we all know not all programmers
> use that idiom.
>
> As I said, I don't think the proposal would do much good, but surely
> you can see what the idea was?
I'll take a stab it. The proposal is to arbitrarily chop the available
address in space, for those rare cases in which someone wishes to
malloc an amount of memory /larger/ than the available address space on
the processor. I.e., they want a boatload of RAM, so restrict them to
less than half of that they need to make things "safer". That has BS
written all over it, imo.
If the programmer knows how to ask for that much memory, but can't
figure out how to make sure he doesn't ask for more than will fit in
size_t, he needs to be writing DOS .bat files instead, where he can do
less damage. Or maybe he should just go off and write his own pseudo-C
compiler instead.
--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw
jacob navia
> I'll take a stab it. The proposal is to arbitrarily chop the available
> address in space, for those rare cases in which someone wishes to
> malloc an amount of memory /larger/ than the available address space on
> the processor. I.e., they want a boatload of RAM, so restrict them to
> less than half of that they need to make things "safer". That has BS
> written all over it, imo.
>
Not at all. Please just see what the proposal really was
before going to answer to phantasy proposals.
The proposal was discussing the idea of using a signed
type to avoid problems with small negative numbers
that get translated into huge unsigned ones.
THAT was the reason.
After this discussion, and after the remarks of Mr Plauger
I am not so sure that that proposal was actually a very good
idea.
The problem is in this discussions, that everybody is trying to
discuss in such an emotional way, that acknowledging
something in your "adversary" argumentation is seen as
an equivalent of "surrender"...
:-)
Look at you, deforming the proposal in such a way that
it is completely NUTS. Of course THEN it is easy to
say:
IT IS NUTS!!!
Obvious Mr Howard. But if you care to follow a bit the
discussion that wasn't proposed at all.
The argument of Mr Plauger that convinces me is that anyway
a multiplication error like the one I signaled to heathfield
will ANYWAY cause havoc even if we put the signed type in
the argument to malloc, so it is actually not a good idea at all.
> If the programmer knows how to ask for that much memory, but can't
> figure out how to make sure he doesn't ask for more than will fit in
> size_t, he needs to be writing DOS .bat files instead, where he can do
> less damage. Or maybe he should just go off and write his own pseudo-C
> compiler instead.
>
A programmer is a human, and human make errors. This discussion
is not designed for people that do not make errors. It is for
people that want to discuss how can we build safety into software
in such a waythat the consequences of errors are limited, for instance
a malloc that diagnoses a bogus argument returns NULL instead of
crashing the software.
Keith Thompson
> jacob navia said:
[...]
>> what is clearly not
>> enough space to store 65521 objects of size 65552
>> each. You are free to call that "multiplication that
>> doesn't overflow".
>
> The Standard says it doesn't overflow, and therefore it doesn't overflow
> (unless the Standard is wrong, which is something you can take up with ISO
> if you like, but I don't rate your chances).
You're both arguing about the meaning of the word "overflow", which is
IMHO less important than the actual behavior of integer types.
Multiplication of two values of type size_t (or of any unsigned
integer type) can yield a mathematical result that cannot be
represented as a value of type size_t. The standard does not call
this "overflow", but in my opinion "overflow" *would* be a valid term
for it.
The point is this: If a result of an unsigned multiplication cannot be
represented, the result is reduced modulo 2**N (where the maximum
value of the type is 2**N-1 -- "**" denotes exponentiation). If the
result of a signed multiplication cannot be represented, the behavior
is undefined.
Using an unsigned type as the parameter of malloc() means that a
larger range of arguments can be used than if it took the
corresponding signed type. This can be significant for systems with a
16-bit size_t that can allocate more than 32767 bytes, or for systems
with a 32-bit size-t that can allocate more than 2147483647 bytes.
jacob's argument is that using a signed type for the parameter of a
malloc-like function is beneficial, supposedly because if a user
incorrectly passes a very large value as the argument, it is likely to
wrap around to a negative value, which can be detected as an error.
In fact, the standard does not guarantee any such thing. Signed
overflow for an arithmetic operation invokes undefined behavior.
Overflow on conversion to a signed type yields an
implementation-defined result (or, in C99, raises an
implementation-defined signal).
Wraparound to a possibly negative value is not uncommon. However, the
result is just as likely to be positive (and still incorrect). Using
a signed type might catch *some* errors, but in my opinion it's not a
good solution. Unsigned types are tricky; that can't be changed
without changing the language. The only real solution is for the
*programmer* to avoid overflow in the first place, and choosing any
particular type as the parameter to a malloc-like function can't help
much with that. You might as well use size_t for consistency with the
standard library.
At times, it would be very nice to be able to define some different
behavior for unsigned "overflow" (i.e., for operations that yield a
mathematical value outside the range of the type). If I multiply two
unsigned values and get an out-of-range result, the result reduced
modulo 2**N *might* be what I want, but more often it's an error that
I'd like to know about. Ditto for signed and floating-point. But C
doesn't let us do that, at least not portably.
Keith Thompson
> struct S {
> unsigned n;
> double d;
> double data[8192];
> };
>
> I want to allocate 65520 objects of size 65 552 bytes each.
>
> If you do that in a 32 bit system what you obtain?
> i=65520, i*sizeof(*p)=4,294,967,040 as signed number = -256
>
> WE OBTAIN A NEGATIVE NUMBER.
We obtain undefined behavior.
James Daughtry
> A programmer is a human, and human make errors. This discussion
> is not designed for people that do not make errors. It is for
> people that want to discuss how can we build safety into software
> in such a waythat the consequences of errors are limited, for instance
> a malloc that diagnoses a bogus argument returns NULL instead of
> crashing the software.
The real problem is how do you diagnose bogus arguments without making
unwarranted assumptions and severely restricting *everyone* at *every
call* to malloc so that you can save people from a pretty rare mistake?
I think you're focusing on solving the problem at the wrong place. It
should be either a behavior adjustment as some have suggested or a
correctness tool outside of malloc that can toss a warning during
compilation. Neither one is a great solution, and I would favor the
former because it doesn't encourage reliance on tools.
CBFalconer
>
... snip ...
>
> Suppose:
> struct S {
> unsigned n;
> double d;
> double data[8192];
> };
>
> I want to allocate 65520 objects of size 65 552 bytes each.
>
> If you do that in a 32 bit system what you obtain?
> i=65520, i*sizeof(*p)=4,294,967,040 as signed number = -256
>
> WE OBTAIN A NEGATIVE NUMBER.
Very simple. We detect the 'unsigned overflow' by:
if (((n = a * b) < a || (n < b)) overflow()
else alliswell(n);
--
Merry Christmas, Happy Hanukah, Happy New Year
Joyeux Noel, Bonne Annee.
Chuck F (cbfalconer at maineline dot net)
<http://cbfalconer.home.att.net>
CBFalconer
> Richard Tobin said:
>> Richard Heathfield <r...@see.sig.invalid> wrote:
>>
> <snip>
>
>>> So we have size_t * size_t - how, precisely, will that produce a
>>> negative number when passed to a function taking size_t as an
>>> argument?
>>
>> Well obviously it won't.
>
> Quite so.
>
>> But the idea was to protect against programmers who make mistakes,
>> and as we all know not all programmers use that idiom.
>
> The fix, then, is obvious.
>
>> As I said, I don't think the proposal would do much good, but
>> surely you can see what the idea was?
>
> It appeared to be an attempt to introduce the additional risk of
> overflow to the existing risk of specifying an allocation amount
> other than the amount actually required. No, I don't see why this
> would be of benefit.
What this has brought home to me is that calloc should be included
in the nmalloc package, so that the same maximum size criterion
will be applied. I.E:
void *ncalloc(size_t nmemb, size_t size) {
size_t sz;
void *p;
sz = nmemb * size;
if ((sz < nmemb) || (sz < size)) return NULL;
if (p = nmalloc(sz)) memset(p, 0, sz);
return p;
}
Since nmalloc drives a 0 size up to 1, this leaves a problem for
nmemb or size being zero. I don't know whether it is worth
worrying about. I am also having qualms about the overflow test.
There doesn't seem to be a SIZE_T_MAX in limits.h. I am worrying
about something like "p = calloc(7, (SIZE_T_MAX/4 + 1));".
cross-posted to c.std.c to see if there is any opinion on this.
Randy Howard
(in article <4599ae58$0$27387$ba4a...@news.orange.fr>):
> Randy Howard a écrit :
>> I'll take a stab it. The proposal is to arbitrarily chop the available
>> address in space, for those rare cases in which someone wishes to
>> malloc an amount of memory /larger/ than the available address space on
>> the processor. I.e., they want a boatload of RAM, so restrict them to
>> less than half of that they need to make things "safer". That has BS
>> written all over it, imo.
>>
>
> Not at all. Please just see what the proposal really was
> before going to answer to phantasy proposals.
>
> The proposal was discussing the idea of using a signed
> type to avoid problems with small negative numbers
> that get translated into huge unsigned ones.
>
> THAT was the reason.
There was a typo (brain fart) in my original quoted above, which was to
read "chop the address space in half"...
Well, that's exactly what your proposal does. No thank you. size_t
exists for a very good reason. I am willing to take the risk that if
if I manage to somehow attempt to compute a value that is larger than
size_t variables can hold that problems will occur. That is far and
away better than chopping the range of mallocs in half for a but of
questionable safety.
> After this discussion, and after the remarks of Mr Plauger
> I am not so sure that that proposal was actually a very good
> idea.
I agree.
> The problem is in this discussions, that everybody is trying to
> discuss in such an emotional way, that acknowledging
> something in your "adversary" argumentation is seen as
> an equivalent of "surrender"...
The only one exhibiting any obvious sign of emotionalism is you. I
have no idea why.
> Look at you, deforming the proposal in such a way that
> it is completely NUTS. Of course THEN it is easy to
> say:
>
> IT IS NUTS!!!
Well, sorry. It pretty much matches the actual /results/ of what your
proposed to the letter. Maybe not the intent, but that's what comes of
it. And those results are nuts. You seem to recognize that yourself
now, so kudos to you.
> The argument of Mr Plauger that convinces me is that anyway
> a multiplication error like the one I signaled to heathfield
> will ANYWAY cause havoc even if we put the signed type in
> the argument to malloc, so it is actually not a good idea at all.
bingo.
>> If the programmer knows how to ask for that much memory, but can't
>> figure out how to make sure he doesn't ask for more than will fit in
>> size_t, he needs to be writing DOS .bat files instead, where he can do
>> less damage. Or maybe he should just go off and write his own pseudo-C
>> compiler instead.
>>
>
> A programmer is a human, and human make errors.
Yes, and when they do, they debug their programs and attempt to make
those errors go away. Here's a dirty little secret for you: Computers
make errors too. They are not even digital on the inside, contrary to
popular mythology. They're little confused layers upon layers of
antennae that pick up noise and xtalk and do /not/ always put out the
correct 0/1 result for a given set of inputs. Google for "Simultaneous
Switching Output Noise" for a good example of one of the more hairy
forms of this problem. they're analog devices that sometimes do a good
job of simulating a binary computer.
> This discussion is not designed for people that do not make errors.
No doubt, since I've been searching for several decades for such a
person, and have yet to find one.
> It is for
> people that want to discuss how can we build safety into software
> in such a waythat the consequences of errors are limited, for instance
> a malloc that diagnoses a bogus argument returns NULL instead of
> crashing the software.
malloc() can not, and will not read minds. It will not know if you
really /meant/ to try and malloc() 2GB+40K of RAM, or if it was an
accident. All it needs to do is malloc that amount, or return NULL.
It's up to you to decide what to do afterward. Even better, you would
know the range of size_t on your platform and check your numbers
/before/ you call malloc. Plenty of people like to use malloc wrapper
functions, this seems like a candidate for such a thing if you want to
make programming safer, not hacking on the libc implementation.
Richard Heathfield
> Richard Heathfield <r...@see.sig.invalid> writes:
>> jacob navia said:
> [...]
>>> what is clearly not
>>> enough space to store 65521 objects of size 65552
>>> each. You are free to call that "multiplication that
>>> doesn't overflow".
>>
>> The Standard says it doesn't overflow, and therefore it doesn't overflow
>> (unless the Standard is wrong, which is something you can take up with
>> ISO if you like, but I don't rate your chances).
> [...]
>
> You're both arguing about the meaning of the word "overflow",
No, there is no argument here. The Standard is quite clear on the matter.
That Mr Navia cannot understand this does not mean that the matter is open
to dispute; it merely means that he cannot understand it.
> which is
> IMHO less important than the actual behavior of integer types.
Again, this is made very clear by the Standard.
> Multiplication of two values of type size_t (or of any unsigned
> integer type) can yield a mathematical result that cannot be
> represented as a value of type size_t.
No, it cannot. Unsigned integer arithmetic is clearly described as being
performed modulo (2 to the power of the number of value bits in the type);
in other words, the unsigned integers representable by the type form a
ring. Mathematically, the multiplication of two integers in a ring yields
another integer in that ring. So, when you multiply two values of size_t
(which are indeed in such a ring, with 2 to the power of b elements where b
is the number of value bits in a size_t), you get another value in the
ring, so it must be representable in a size_t.
> The standard does not call
> this "overflow", but in my opinion "overflow" *would* be a valid term
> for it.
Much as I respect your opinion, it does not take precedence over the
terminology used by the Standard.
> The point is this: If a result of an unsigned multiplication cannot be
> represented, the result is reduced modulo 2**N (where the maximum
> value of the type is 2**N-1 -- "**" denotes exponentiation). If the
> result of a signed multiplication cannot be represented, the behavior
> is undefined.
It is true that introducing signed ints into the mix exposes the process of
dynamically allocating memory to yet another risk of undefined behaviour,
yes - and since this is supposedly all about protecting the ignorant or
careless programmer from his mistakes, I'm not convinced that giving him
another way of screwing up constitutes protecting him.
> Using an unsigned type as the parameter of malloc() means that a
> larger range of arguments can be used than if it took the
> corresponding signed type. This can be significant for systems with a
> 16-bit size_t that can allocate more than 32767 bytes, or for systems
> with a 32-bit size-t that can allocate more than 2147483647 bytes.
Right.
> jacob's argument is that using a signed type for the parameter of a
> malloc-like function is beneficial, supposedly because if a user
> incorrectly passes a very large value as the argument, it is likely to
> wrap around to a negative value, which can be detected as an error.
I agree that that is his argument, but I cannot see that it has any merit,
since there is no way for the compiler to distinguish between a user who
accidentally makes a large memory request (because his program is broken)
and a user who deliberately makes a large memory request (because he needs
lots of memory).
<snip>
Harald van Dijk
If nmalloc() and ncalloc() are to be used as replacements for or
implementations of malloc() and calloc(), keep in mind that whether
malloc(0) returns NULL is implementation-defined. I don't believe there
is anything preventing the implementation from defining that malloc(0)
and calloc(0, 0) are different from calloc(1, 0) and calloc(0, 1) in
this regard, but this would need to be documented accurately. I believe
it's very easy to avoid this problem by simply replacing your check
with:
if ((sz < nmemb) && (sz < size))
but I may be overlooking something.
> I am also having qualms about the overflow test.
> There doesn't seem to be a SIZE_T_MAX in limits.h.
There's SIZE_MAX, but even without it, you could convert -1 to size_t.
Harald van Dijk
> I believe
> it's very easy to avoid this problem by simply replacing your check
> with:
> if ((sz < nmemb) && (sz < size))
> but I may be overlooking something.
I was, of course. Sorry, and please don't do that, it won't work.
CBFalconer
> CBFalconer wrote:
>
... snip ...
>>
>> What this has brought home to me is that calloc should be included
>> in the nmalloc package, so that the same maximum size criterion
>> will be applied. I.E:
>>
>> void *ncalloc(size_t nmemb, size_t size) {
>> size_t sz;
>> void *p;
>>
>> sz = nmemb * size;
>> if ((sz < nmemb) || (sz < size)) return NULL;
>> if (p = nmalloc(sz)) memset(p, 0, sz);
>> return p;
>> }
>>
>> Since nmalloc drives a 0 size up to 1, this leaves a problem for
>> nmemb or size being zero. I don't know whether it is worth
>> worrying about.
>>
>> There doesn't seem to be a SIZE_T_MAX in limits.h.
>
> There's SIZE_MAX, but even without it, you could convert -1 to size_t.
>
>> I am worrying
>> about something like "p = calloc(7, (SIZE_T_MAX/4 + 1));".
>>
>> cross-posted to c.std.c to see if there is any opinion on this.
>
> If nmalloc() and ncalloc() are to be used as replacements for or
> implementations of malloc() and calloc(), keep in mind that whether
> malloc(0) returns NULL is implementation-defined. I don't believe there
> is anything preventing the implementation from defining that malloc(0)
> and calloc(0, 0) are different from calloc(1, 0) and calloc(0, 1) in
> this regard, but this would need to be documented accurately. I believe
> it's very easy to avoid this problem by simply replacing your check
> with:
> if ((sz < nmemb) && (sz < size))
> but I may be overlooking something.
I think the test detects a zero field already. The problem is that
that is legitimate, and so it should procede to nmalloc, which
handles the zero allocation test already. The test should probably
be:
if ((nmemb && size) && (sz < nmemb) || (sz < size)) return NULL;
but that doesn't handle the calloc call case I mentioned above.
Peter Nilsson
> We detect the 'unsigned overflow' by:
>
> if (((n = a * b) < a || (n < b)) overflow()
> else alliswell(n);
No. Say UINT_MAX is 65535, observe that 257 * 257 == 65536 + 513.
The simple test for whether two size_t variables can be multiplied
is...
if (a <= ((size_t) -1) / b)
/* good */;
else
/* bad */;
--
Peter
av
>On Mon, 01 Jan 2007 20:01:54 +0100, in comp.lang.c , jacob navia
><ja...@jacob.remcomp.fr> wrote:
>
>>Clark S. Cox III a écrit :
>>>
>>> Oh, don't take my post the wrong way; just because I corrected a single
>>> factual error in his post, I still agree with him; and disagree with you.
>>
>>His majesty is always right, no matter how much nonsense
>>he says.
>
>No, truth is always right, no matter how much nonsense people try to
>conceal it behind.
"experimental truth" or "pragmatic truth" is better than just "truth"
i begin to think size_t can lead to bugs for the overflow allowance
(integer overflow for c library functions arguments is not good)
and the functions in the standard that use it are wrong because it is
sufficient an overflow in their arguments for doing ugly errors
that is difficult to find because are only for some value its argumet
in the other side, if a function use int
i can see if the arg is ok with the instruction
if(paremater<0) return error;
av
>No, it cannot. Unsigned integer arithmetic is clearly described as being
>performed modulo (2 to the power of the number of value bits in the type);
>in other words, the unsigned integers representable by the type form a
>ring. Mathematically, the multiplication of two integers in a ring yields
>another integer in that ring. So, when you multiply two values of size_t
>(which are indeed in such a ring, with 2 to the power of b elements where b
>is the number of value bits in a size_t), you get another value in the
>ring, so it must be representable in a size_t.
are you joking?
if there is an "unsigned overflow" (if you can not follow me, the
value that "c" has in each instruction "c=a*b;" or "c=a+3;"
size_t a=-1, b=2, c=a*b;
c=a+3;
)
if that "unsigned overflow" happen to the parameter of some function
that use "size_t" is error.
that error is difficult to find because
it is find only for some values of "a","b", "c" but people in
bugtraq appear to know these values :))
first to speak do you know how many errors of that kind there are in
bugtraq?
ha yes you don't do errors....
Richard Heathfield
> On Tue, 02 Jan 2007 03:32:59 +0000, Richard Heathfield wrote:
>>No, it cannot. Unsigned integer arithmetic is clearly described as being
>>performed modulo (2 to the power of the number of value bits in the type);
>>in other words, the unsigned integers representable by the type form a
>>ring. Mathematically, the multiplication of two integers in a ring yields
>>another integer in that ring. So, when you multiply two values of size_t
>>(which are indeed in such a ring, with 2 to the power of b elements where
>>b is the number of value bits in a size_t), you get another value in the
>>ring, so it must be representable in a size_t.
>
> are you joking?
No.
av
>"experimental truth" or "pragmatic truth" is better than just "truth"
>
>i begin to think size_t can lead to bugs for the overflow allowance
>(integer overflow for c library functions arguments is not good)
>and the functions in the standard that use it are wrong because it is
>sufficient an overflow in their arguments for doing ugly errors
>that is difficult to find because are only for some value its argumet
>
>in the other side, if a function use int
>i can see if the arg is ok with the instruction
>if(paremater<0) return error;
it should be good to do some new type
struct NewSize_t_{
int ovf;
size_t n;
};
typedef NewSize_t_ NewSize_t;
with +-*/ defined on NewSize_t
and use it like parameter for function that now use size_t
if some overflow occur "ovf==1" and the function know parameter is not
ok
but that unsigned type above could be useful for all others
calculation that has not need overflows
i think it is possible to change size_t to that struct
and use the oveflow flag for see all is ok
and this resolve to the root the problem
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Richard Heathfield
<snip>
> it should be good to do some new type
>
> struct NewSize_t_{
> int ovf;
> size_t n;
> };
>
> typedef NewSize_t_ NewSize_t;
>
> with +-*/ defined on NewSize_t
> and use it like parameter for function that now use size_t
>
> if some overflow occur
As has been explained ad nauseam, no overflow occurs.
<snip>
> and this resolve to the root the problem
What problem? I don't see one that can't be solved much more simply by
behavioural change than by adding a new type to the language and then
waiting 20 years for it to get implemented across the board.
av
>On Tue, 02 Jan 2007 10:10:36 +0100, av wrote:
>>"experimental truth" or "pragmatic truth" is better than just "truth"
>>
>>i begin to think size_t can lead to bugs for the overflow allowance
>>(integer overflow for c library functions arguments is not good)
>>and the functions in the standard that use it are wrong because it is
>>sufficient an overflow in their arguments for doing ugly errors
>>that is difficult to find because are only for some value its argumet
>>
>>in the other side, if a function use int
>>i can see if the arg is ok with the instruction
>>if(paremater<0) return error;
>
>it should be good to do some new type
>
>struct NewSize_t_{
>int ovf;
>size_t n;
>};
>
>typedef NewSize_t_ NewSize_t;
in other words i say
struct size_t_{
unsigned n;
int ovf;
};
typedef size_t_ size_t;
with +-*/ defined on size_t
and use it like parameter for functions
if some overflow occur "ovf==1" and the function know parameter is not
ok
but that unsigned type above could be useful for all others
calculation that has not need overflows
if there is a complex calculation i can see if there is some overflow
in the result
Keith Thompson
> Keith Thompson said:
>
>> Richard Heathfield <r...@see.sig.invalid> writes:
>>> jacob navia said:
>> [...]
>>>> what is clearly not
>>>> enough space to store 65521 objects of size 65552
>>>> each. You are free to call that "multiplication that
>>>> doesn't overflow".
>>>
>>> The Standard says it doesn't overflow, and therefore it doesn't overflow
>>> (unless the Standard is wrong, which is something you can take up with
>>> ISO if you like, but I don't rate your chances).
>> [...]
>>
>> You're both arguing about the meaning of the word "overflow",
>
> No, there is no argument here. The Standard is quite clear on the matter.
> That Mr Navia cannot understand this does not mean that the matter is open
> to dispute; it merely means that he cannot understand it.
I think when jacob talks about "overflow" of unsigned multiplication,
he's referring to an operation that, if it were carried out using true
mathematical integers, would yield a result that cannot be represented
in the unsigned type. (C, as we all know, deals with this by reducing
the mathematical result modulo 2**N.) I think that the fact that the
standard doesn't refer to this as "overflow" can lead to some
confusion.
>> which is
>> IMHO less important than the actual behavior of integer types.
>
> Again, this is made very clear by the Standard.
Yes, of course.
>> Multiplication of two values of type size_t (or of any unsigned
>> integer type) can yield a mathematical result that cannot be
>> represented as a value of type size_t.
>
> No, it cannot.
Yes, it can, but I probably didn't state it clearly enough.
What I meant by "mathematical result" is the result of the
multiplication viewed as an operation on the infinite set of
mathematical integers. For example, suppose we're dealing with a
16-bit unsigned int. 1000U * 1000U yields a "mathematical result" of
1000000 (which is reduced modulo 65536 to 16960U).
> Unsigned integer arithmetic is clearly described as being
> performed modulo (2 to the power of the number of value bits in the type);
> in other words, the unsigned integers representable by the type form a
> ring. Mathematically, the multiplication of two integers in a ring yields
> another integer in that ring. So, when you multiply two values of size_t
> (which are indeed in such a ring, with 2 to the power of b elements where b
> is the number of value bits in a size_t), you get another value in the
> ring, so it must be representable in a size_t.
Right.
But the standard *could* have called this "overflow". It could have
said that an unsigned operation that "overflows" causes the result to
be reduced modulo 2**N. And this would have described exactly the
same language.
The standard says this isn't overflow, but referring to it as overflow
isn't entirely nonsensical; it's merely inconsistent with the wording
of the standard.
I'm trying to interpret what jacob has written. One of my goals is to
encourage him to be more careful with terminology.
>> The standard does not call
>> this "overflow", but in my opinion "overflow" *would* be a valid term
>> for it.
>
> Much as I respect your opinion, it does not take precedence over the
> terminology used by the Standard.
Of course not. Probably writing "would have been" rather than "would
be" would have expressed my meaning more clearly.
[snip]
>> jacob's argument is that using a signed type for the parameter of a
>> malloc-like function is beneficial, supposedly because if a user
>> incorrectly passes a very large value as the argument, it is likely to
>> wrap around to a negative value, which can be detected as an error.
>
> I agree that that is his argument, but I cannot see that it has any merit,
> since there is no way for the compiler to distinguish between a user who
> accidentally makes a large memory request (because his program is broken)
> and a user who deliberately makes a large memory request (because he needs
> lots of memory).
Agreed. Having a function detect errors is a good thing, but it
should do so only when it knows better than I do (more generally,
better than the caller) what's an error and what isn't.
The problem is that malloc(X * Y) very likely does not behave properly
if the result of X * Y does not match what I call the "mathematical
result" of X * Y. But the problem, if any, is in the evaluation of
X * Y itself. It can't be avoided in malloc(), because by then it's
too late.
Keith Thompson
[...]
> Very simple. We detect the 'unsigned overflow' by:
>
> if (((n = a * b) < a || (n < b)) overflow()
> else alliswell(n);
Not necessarily. Consider 16-bit unsigned int, a == 20000, b ==
20000. a * b is 400000000; reducing modulo 2**16 yields 33792, which
is greater than both a and b. It also fails if exactly one of a or b
is zero.
I think your method works for addition, but not for multiplication.
I *think* that checking whether n / a == b will work (but it's likely
to be more expensive).
Richard Heathfield
> Richard Heathfield <r...@see.sig.invalid> writes:
>> Keith Thompson said:
<snip>
>
>>> Multiplication of two values of type size_t (or of any unsigned
>>> integer type) can yield a mathematical result that cannot be
>>> represented as a value of type size_t.
>>
>> No, it cannot.
>
> Yes, it can, but I probably didn't state it clearly enough.
>
> What I meant by "mathematical result" is the result of the
> multiplication viewed as an operation on the infinite set of
> mathematical integers.
There are more things in mathematics, Horatio... and I'm sure I need not
complete the misquote for you. :-)
<snip>
> But the problem, if any, is in the evaluation of X * Y itself.
"If any" being the key words there.
> It can't be avoided in malloc(), because by then it's too late.
Agreed.
av
>av said:
><snip>
>> it should be good to do some new type
>>
>> struct NewSize_t_{
>> int ovf;
>> size_t n;
>> };
>>
>> typedef NewSize_t_ NewSize_t;
>>
>> with +-*/ defined on NewSize_t
>> and use it like parameter for function that now use size_t
>>
>> if some overflow occur
>
>As has been explained ad nauseam, no overflow occurs.
then i do for you the definition for overflow that i see
you have 3 unsigned integers type variabiles
of XXbits of name "a" "b" "c" in the computer
than you do c= b (operation) a; in the computer
where operation is + or - or *
in the computer
than you immagine that have 3 signed integers type variabiles in math
(do you know the set of natural number N? and the set Z of signed
integers?)
bb=b,aa=a,cc in Z
and do the same in Z
cc= bb (operation) aa;
where operation is + or - or *
if cc==c no overflow occurs
if cc!=c integer overflow occurs
><snip>
>
>> and this resolve to the root the problem
>
>What problem? I don't see one that can't be solved much more simply by
>behavioural change than by adding a new type to the language and then
>waiting 20 years for it to get implemented across the board.
the problem of overvflow definited above where it should be not be in
the calculation
Richard Heathfield
> On Tue, 02 Jan 2007 10:03:34 +0000, Richard Heathfield wrote:
>>
>>As has been explained ad nauseam, no overflow occurs.
>
> then i do for you the definition for overflow that i see
Your definition of overflow is neither here nor there. It is the Standard's
definition that matters.
> you have 3 unsigned integers type variabiles
> of XXbits of name "a" "b" "c" in the computer
> than you do c= b (operation) a; in the computer
> where operation is + or - or *
> in the computer
>
> than you immagine that have 3 signed integers type variabiles in math
> (do you know the set of natural number N? and the set Z of signed
> integers?)
Yes. Z is irrelevant here. Unsigned arithmetic takes place in a finite
subset of N - a ring of 2^b elements with values 0 to 2^b - 1 (where b is
the number of value bits in the type). All addition and multiplication
operations between any two values of that type give another result that is
within the ring. No overflow occurs.
> bb=b,aa=a,cc in Z
> and do the same in Z
> cc= bb (operation) aa;
> where operation is + or - or *
>
> if cc==c no overflow occurs
> if cc!=c integer overflow occurs
Not so.
>><snip>
>>
>>> and this resolve to the root the problem
>>
>>What problem? I don't see one that can't be solved much more simply by
>>behavioural change than by adding a new type to the language and then
>>waiting 20 years for it to get implemented across the board.
>
> the problem of overvflow definited above where it should be not be in
> the calculation
No overflow occurs in unsigned arithmetic. See 3.1.2.5 of C90, or 6.2.5(9)
of C99.
Simon Biber
> P.J. Plauger a écrit :
>> That may be your experience, but mine has been that 16-bit systems
>> often have >32KB of memory. Anything that interferes with handling
>> all of memory in one go is sure to cause trouble, sooner or later.
>>
>
> Yes. You are right in this point. For 16 bit systems the
> lost of 32K of addressing space is quite a hit. Specially
> if you do have the full 64K.
You may even have more than 64K, for example on MS-DOS you can have 10
of the 64K regions for a total of 640K. An int is -32768 to 32767 and
size_t is 0 to 65535. Pointers must be more than 16 bits of course if
they can refer to allocations inside separate regions. Each of these
allocations may take up nearly the full 64K maximum object size.
#include <stdio.h>
#include <stdlib.h>
#define N 10
int main(void)
{
char *p[N];
int i;
for(i = 0; i < N; i++) p[i] = malloc(60000);
for(i = 0; i < N; i++) printf("%p\n", (void*)p[i]);
return 0;
}
This program attempts to malloc 10 blocks of 60000 bytes, and prints out
the pointers. On a 640K MS-DOS system in 'large' or 'huge' memory model
where pointers are 32 bits in a segment:offset format, it outputs
something like:
10C4:0008
1F6B:0008
2E12:0008
3CB9:0008
4B60:0008
5A07:0008
68AE:0008
7755:0008
85FC:0008
0000:0000
The last allocation failed, since the 10th block is unavailable for
malloc's use; it probably contains the program's code, stack, etc.
However on a 'small' or 'medium' memory model where pointers are just 16
bits, it outputs:
0B24
0000
0000
0000
0000
0000
0000
0000
0000
0000
Indicating that only the first allocation succeeded. Memory outside of
the first 64K is unavailable since 16-bit pointers cannot address it.
--
Simon.
Keith Thompson
> Keith Thompson said:
>> Richard Heathfield <r...@see.sig.invalid> writes:
>>> Keith Thompson said:
>
> <snip>
>>
>>>> Multiplication of two values of type size_t (or of any unsigned
>>>> integer type) can yield a mathematical result that cannot be
>>>> represented as a value of type size_t.
>>>
>>> No, it cannot.
>>
>> Yes, it can, but I probably didn't state it clearly enough.
>>
>> What I meant by "mathematical result" is the result of the
>> multiplication viewed as an operation on the infinite set of
>> mathematical integers.
>
> There are more things in mathematics, Horatio... and I'm sure I need not
> complete the misquote for you. :-)
Good point. I probably should have nailed down what I meant by
"mathematical result" sooner, and/or used a different phrase.
However, note that C99 6.2.5p9 says:
A computation involving unsigned operands can never overflow,
because a result that cannot be represented by the resulting
unsigned integer type is reduced modulo the number that is one
greater than the largest value that can be represented by the
resulting type.
The word "result" here can only refer to what I've been calling the
"mathematical result". You're complaining about my lack of precision,
but the standard's wording here is no more precise than mine. 8-)}
> <snip>
>
>> But the problem, if any, is in the evaluation of X * Y itself.
>
> "If any" being the key words there.
[...]
In the following, an expression enclosed in << and >> is to be
interpreted as being evaluated over the entire infinite set of
integers.
If X and Y are values of type size_t, a call malloc(X * Y) will
attempt to allocate <<(X * Y) % (SIZE_MAX+1)>> bytes. If this is not
equal to <<X * Y>>, then the result is affected by what the standard
doesn't call "overflow" (let's call it "wraparound"). This is the
behavior clearly specified by the standard, but I can't think of any
circumstances in which it's the *desired* behavior. It's almost
certainly a logical error.
If I write malloc(X * Y), it's because I *want* to allocate <<X * Y>>
bytes. The result of the multiplication will be silently reduced
whether I want it to be or not. The only real solution, given the
language as it currently exists, is to be very careful not to let this
kind of logical error occur in the first place (because the language
is of no help in diagnosing it if I'm not sufficiently careful). This
is non-trivial. A more general solution might be to modify the
language so that cases where X * Y wraps around can be detected, but
that's not going to happen any time soon, if ever.
Somebody (jacob, I think) suggested using calloc(X, Y) rather than
malloc(X * Y), assuming that calloc() correctly detects wraparound on
the multiplication. I disagree with this for two reasons. First,
it's already been pointed out that not all implementations of calloc()
get this right. Second, zeroing the allocated memory is likely to be
wasteful.
If this is a concern, you can write a wrapper around malloc() that
takes two arguments (as calloc() does) and checks whether the result
wraps around. Something like this (obviously untested):
void *malloc_wrapper(size_t x, size_t y)
{
if (/* x*y wraps around */) {
return NULL;
}
else {
return malloc(x * y);
}
}
Implementing the wraparound test is left as an exercise.
Keith Thompson
[...]
> No overflow occurs in unsigned arithmetic. See 3.1.2.5 of C90, or 6.2.5(9)
> of C99.
Quibble: You mean 3.1.2.5 of C89 (the original ANSI standard); in C90,
the nearly identical ISO standard, it's 6.1.2.5.
Richard Heathfield
<snip>
> However, note that C99 6.2.5p9 says:
>
> A computation involving unsigned operands can never overflow,
> because a result that cannot be represented by the resulting
> unsigned integer type is reduced modulo the number that is one
> greater than the largest value that can be represented by the
> resulting type.
>
> The word "result" here can only refer to what I've been calling the
> "mathematical result". You're complaining about my lack of precision,
> but the standard's wording here is no more precise than mine. 8-)}
I'll sue them later. Moving on...
> If X and Y are values of type size_t, a call malloc(X * Y) will
> attempt to allocate <<(X * Y) % (SIZE_MAX+1)>> bytes. If this is not
> equal to <<X * Y>>, then the result is affected by what the standard
> doesn't call "overflow" (let's call it "wraparound"). This is the
> behavior clearly specified by the standard, but I can't think of any
> circumstances in which it's the *desired* behavior. It's almost
> certainly a logical error.
The error is the programmer's, not the language's, fault: trying to allocate
memory for a single object of a size greater than the maximum value that a
size_t can store is just silly.
> If I write malloc(X * Y), it's because I *want* to allocate <<X * Y>>
> bytes. The result of the multiplication will be silently reduced
> whether I want it to be or not.
Correct. But this is only a problem if allocating X * Y bytes does not make
sense within the context of the system you are using, in which case you
shouldn't do it in the first place.
<snip>
Richard Heathfield
> Richard Heathfield <r...@see.sig.invalid> writes:
> [...]
>> No overflow occurs in unsigned arithmetic. See 3.1.2.5 of C90, or
>> 6.2.5(9) of C99.
>
> Quibble: You mean 3.1.2.5 of C89 (the original ANSI standard); in C90,
> the nearly identical ISO standard, it's 6.1.2.5.
Clumsy of me. Thank you for the correction.
Kenny McCormack
Richard Heathfield <r...@see.sig.invalid> wrote:
>av said:
>
>> On Tue, 02 Jan 2007 03:32:59 +0000, Richard Heathfield wrote:
>>>No, it cannot. Unsigned integer arithmetic is clearly described as being
>>>performed modulo (2 to the power of the number of value bits in the type);
>>>in other words, the unsigned integers representable by the type form a
>>>ring. Mathematically, the multiplication of two integers in a ring yields
>>>another integer in that ring. So, when you multiply two values of size_t
>>>(which are indeed in such a ring, with 2 to the power of b elements where
>>>b is the number of value bits in a size_t), you get another value in the
>>>ring, so it must be representable in a size_t.
>>
>> are you joking?
>
>No.
The truly scary thing is that after reading RH's drivel for a while, you
begin to realize, that like his mentor Dubya, he *actually* believes the
claptrap he's spewing! I mean, seriously, like Dubya, he's not lying!
And *that*, my friends, is the true definition of scary.
kuy...@wizard.net
It's not claptrap, unlike what Dubya is spewing, and I would therefore
certainly hope he's sufficiently intelligent and mathematically
well-educated to believe it. I'd recommend reading up on the
mathematical concept of rings before you claim otherwise.
CBFalconer
> av said:
>
> <snip>
>
>> it should be good to do some new type
>>
>> struct NewSize_t_{
>> int ovf;
>> size_t n;
>> };
>>
>> typedef NewSize_t_ NewSize_t;
>>
>> with +-*/ defined on NewSize_t
>> and use it like parameter for function that now use size_t
>>
>> if some overflow occur
>
> As has been explained ad nauseam, no overflow occurs.
>
> <snip>
>
>> and this resolve to the root the problem
>
> What problem? I don't see one that can't be solved much more
> simply by behavioural change than by adding a new type to the
> language and then waiting 20 years for it to get implemented
> across the board.
for example:
size_t overflow(size_t x, size_t y) {
size_t xmax, ymax, szmax = -1;
if (xmax && ymax) {
xmax = szmax / ymax; ymax = szmax / xmax;
if ((x > xmax) || (y > ymax) return 0;
return x * y;
}
return 0;
}
assuming 0 is a suitable result for an overflow.
Besides which av is a troll.
--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>
CBFalconer
You're right, and I corrected (over corrected) it in a post a few
minutes ago.
jacob navia
Who cares about rings?
We are speaking about overflow in a well defined
context. Yes, the C standard defines the behavior
for overflow, and overflow then, it is defined
for unsigned integers. This doesn't mean that it
doesn't happen or that this "ring" stuff is meaningful.
Or are you implying that
65521 x 65552 is 65296 and NOT 4295032592
????
With that 'x' I mean multiplication as is
understood by all people in this planet except
C buffs that know more about C standards than about
software engineering and mathematics???
What the standard is doing is merely accepting
overflow after the fact, and "defining" multiplication
by unsigned integers as an operation within a ring of
integers.
This "meaning" to multiplication is maybe OK for justifying
the behavior of C, but it is nothing else than
describing what an overflow DOES in most machines!
The nonsense of heathfield becomes worst given the context where
he is saying that we should go on using this construct:
malloc (n * sizeof *p)
to allocate memory instead of using calloc that should test
for overflow!
The bug I am referring to is when you multiply
65521 * 65552 --> 65 296
Since malloc doesn't see anything wrong it will succeed,
giving you a piece of RAM that it is NOT as long as you
would think it is, but several orders of magnitude SMALLER.
Even when heathfield says a patently insane stuff he is
"the guru" and people here will justify his ramblings.
I am well aware of what the standard says for overflow.
I studied the question when I implemented (as the only C
compiler in existence) an OVERFLOW CHECK feature that
would trap on overflow. And I did it only for signed
integers to remain compatible with the standard.
Nevertheless this behavior of unsigned C arithmetic is
sureley NEVER CORRECT and I have never seen a useful
program that relies on this behavior for something useful.
jacob
Richard Heathfield
<snip>
> Who cares about rings?
Anyone who cares about how unsigned integer arithmetic works in C.
> We are speaking about overflow in a well defined
> context.
Overflow doesn't occur with unsigned integer types in C. We covered that.
> Yes, the C standard defines the behavior
> for overflow, and overflow then, it is defined
> for unsigned integers.
No, overflow *doesn't happen* for unsigned integers.
> This doesn't mean that it
> doesn't happen or that this "ring" stuff is meaningful.
Yes, it does.
> Or are you implying that
>
> 65521 x 65552 is 65296 and NOT 4295032592
It's implementation-defined. The result of multiplying two unsigned types
together depends on their values and the width of the type.
<snip>
> The nonsense of heathfield becomes worst given the context where
> he is saying that we should go on using this construct:
>
> malloc (n * sizeof *p)
>
> to allocate memory instead of using calloc that should test
> for overflow!
What you call nonsense is in fact good sense. Using calloc is almost always
the wrong thing, since calloc writes 0 to every byte in the allocated
block, which is hardly ever the right thing to do. (If all-bits-zero meant
NULL for pointers and 0.0 for floating point values, that would be a
different matter, but they don't so it isn't.) Furthermore, if n is an
unsigned type (as it should be, in my opinion), n * sizeof *p can't
overflow so there is nothing for calloc to test.
> The bug I am referring to is when you multiply
> 65521 * 65552 --> 65 296
If you can meaningfully allocate 65521*65552 bytes in a single aggregate
object, then sizeof has to be able to report the size of such an object,
which means size_t has to be at least 33 bits, which means the "bug" you
refer to doesn't occur. If size_t is no more than 32 bits, it doesn't make
sense to try to allocate an aggregate object > 2^32-1 bytes in size.
> Since malloc doesn't see anything wrong it will succeed,
> giving you a piece of RAM that it is NOT as long as you
> would think it is, but several orders of magnitude SMALLER.
As a matter of fact, it will give you at least the number of bytes you
requested (if it gives you any bytes at all). It is not malloc's fault if
you have misinterpreted unsigned integer arithmetic.
> Even when heathfield says a patently insane stuff he is
> "the guru" and people here will justify his ramblings.
If what I say is insane, it should be easy to disprove. But you've never
managed it yet.
> I am well aware of what the standard says for overflow.
Then you will understand that to use a signed type instead of an unsigned
type as an argument to malloc is to introduce the potential for undefined
behaviour without solving the problem you were setting out to solve, and is
therefore a bad idea.
jacob navia
>
>>The bug I am referring to is when you multiply
>>65521 * 65552 --> 65 296
>
>
> If you can meaningfully allocate 65521*65552 bytes in a single aggregate
> object, then sizeof has to be able to report the size of such an object,
> which means size_t has to be at least 33 bits, which means the "bug" you
> refer to doesn't occur. If size_t is no more than 32 bits, it doesn't make
> sense to try to allocate an aggregate object > 2^32-1 bytes in size.
>
>
C'MON HEATHFIELD
can you STOP IT???????
BUGS NEVER MAKE SENSE!!!
THAT'S WHY THEY ARE BUGS!!!!
Your "reasoning" is
Patient: Doctor doctor, each time I move my leg I have an horrible
pain!!!
Doctor Heathfield: Well, do not move it then!!!
The whole point is precisely that an overflow bug can occur
in your "idiom" and to avoid it there are two solutions:
1) Call calloc(n,size);
2) If you do not want to use calloc because of the wasted
micro-micro seconds in zeroing the memory you write your own.
For example:
With 32 bit ints, and 64 bit long longs ( a common configuration)
you write:
void *mycalloc(size_t n, size_t s)
{
long long m = (long long)s * (long long) n;
if (m>>32)
return 0;
return malloc((size_t)m);
}
The test (m>>32) just tests the upper 32 bits. Obviously
more sophisticated tests are possible.
John Bode
jacob navia wrote:
[snip]
>
> His majesty is always right, no matter how much nonsense
> he says.
>
> > There is no way to obtain a negative size_t.
>
> Of course not. It is an unsigned number. The whole point is that
> when you have
>
> void fn(unsigned arg);
>
> and you write:
>
> fn(-3);
>
> an implicit cast from signed to unsigned is done,
> what in all implementations I know leads to no
> machine code, but just a change in the way the bits
> of the argument are interpreted.
>
> Obviously this is a simplification of a real situation
> when the values are not explicitely gibve like in the
> example above.
>
>
> > Unsigned types don't overflow.
>
> This is nonsense. Why I obtain
>
> 65521 * 65552 --> 65296 ????
>
> You (and heathfield) are just playing with words. That
> overflow is defined for unsigned numbers within C doesn't
> mean that the result is mathematically VALID, or that is
> the expected result.
>
Just because the result isn't what's expected doesn't mean the
operation is mathematically invalid.
> The whole point of my argumentation is that the "idiom"
>
> result = malloc(n * sizeof *p);
>
> is a dangerous construct.
This is like arguing that wearing a seatbelt is always dangerous
because you could potentially become trapped in a burning car after a
wreck. Weakness in one corner case *does not* translate to "dangerous"
in general. The benefit of using the idiom far outweighs the potential
risk.
There are two ways to deal with this problem. One method would be to
perform a basic sanity check against an unsigned wraparound *before*
calling malloc(); it shouldn't be hard to create a wrapper function
that takes an element size and count and returns NULL (and potentially
sets errno) if the requested block is too large:
int *x = sane_malloc(sizeof *x, count);
if (!x)
{
if (errno == EMEMRQST) // or code of your choice
{
// memory request exceeds size_t
}
}
Another method is to hack malloc() to take a signed argument and pray
that every system it's implemented on treats signed integer overflow in
the same useful manner.
I know which method I'd prefer.
> Use
>
> result = calloc(n, sizeof(*p))
>
> and ensure your implementation of calloc doesn't have the bug!
>
>
> > A signed argument to malloc makes absolutely no sense.
>
> It would catch the overflow in some cases above, as I explained in
> my post.
So, basically, you're willing to trade the illusion of safety in one
corner case for cutting your usable address space in half.
>
> You ropinion may be different, but it would be helpful if you
> tried to advance an argument, just saying
> "makes no sense"
>
> makes no sense to anyone but you.
It makes no sense to me, either. You think you're protecting a
programmer from shooting himself in the foot, but in reality you're
just giving him extra bullets. Now he can call malloc() with an
invalid size request *by design*.
Kenny McCormack
Richard Heathfield <r...@see.sig.invalid> wrote:
>jacob navia said:
>
><snip>
>
>> Who cares about rings?
>
>Anyone who cares about how unsigned integer arithmetic works in C.
Found those WMDs yet?
Ben Bacarisse
> What this has brought home to me is that calloc should be included
> in the nmalloc package, so that the same maximum size criterion
> will be applied. I.E:
>
> void *ncalloc(size_t nmemb, size_t size) {
> size_t sz;
> void *p;
>
> sz = nmemb * size;
> if ((sz < nmemb) || (sz < size)) return NULL;
> if (p = nmalloc(sz)) memset(p, 0, sz);
> return p;
> }
>
<snip>
> I am also having qualms about the overflow test.
I think it is wrong about 1/3 of the time. I.e. there are many cases
where the multiplication wraps round but where the reduced result (sz)
is not less than one or other of multipliers.
P J Plauger recently posted a division-based test. I don't know of a
simple test for a multiplication that will overflow that does not use
division. The fact that no one has posted one so far suggests that
there isn't one but I look forward to being wrong about that :-)
--
Ben.
Richard Heathfield
> Richard Heathfield a écrit :
>>
>>>The bug I am referring to is when you multiply
>>>65521 * 65552 --> 65 296
>>
>>
>> If you can meaningfully allocate 65521*65552 bytes in a single aggregate
>> object, then sizeof has to be able to report the size of such an object,
>> which means size_t has to be at least 33 bits, which means the "bug" you
>> refer to doesn't occur. If size_t is no more than 32 bits, it doesn't
>> make sense to try to allocate an aggregate object > 2^32-1 bytes in size.
>
> C'MON HEATHFIELD
> can you STOP IT???????
Stop what? Being right?
> The whole point is precisely that an overflow bug can occur
> in your "idiom"
No, it can't, as I have explained several times. Until you have done the
necessary research that enables you to understand this, there is little
point in my attempting to address your other misconceptions.
Harald van Dijk
> Harald van D?k wrote:
> > CBFalconer wrote:
> >
> ... snip ...
> >>
> >> What this has brought home to me is that calloc should be included
> >> in the nmalloc package, so that the same maximum size criterion
> >> will be applied. I.E:
> >>
> >> void *ncalloc(size_t nmemb, size_t size) {
> >> size_t sz;
> >> void *p;
> >>
> >> sz = nmemb * size;
> >> if ((sz < nmemb) || (sz < size)) return NULL;
> >> if (p = nmalloc(sz)) memset(p, 0, sz);
> >> return p;
> >> }
> >>
> >> nmemb or size being zero. I don't know whether it is worth
> >> worrying about.
> >>
> >> I am also having qualms about the overflow test.
> >
> > There's SIZE_MAX, but even without it, you could convert -1 to size_t.
> >
> >> I am worrying
> >> about something like "p = calloc(7, (SIZE_T_MAX/4 + 1));".
> >>
> >> cross-posted to c.std.c to see if there is any opinion on this.
> >
> > If nmalloc() and ncalloc() are to be used as replacements for or
> > implementations of malloc() and calloc(), keep in mind that whether
> > malloc(0) returns NULL is implementation-defined. I don't believe there
> > is anything preventing the implementation from defining that malloc(0)
> > and calloc(0, 0) are different from calloc(1, 0) and calloc(0, 1) in
> > this regard, but this would need to be documented accurately. I believe
> > it's very easy to avoid this problem by simply replacing your check
> > with:
> > if ((sz < nmemb) && (sz < size))
> > but I may be overlooking something.
>
> I think the test detects a zero field already.
Not consistently. If both nmemb and size are zero, then sz < nmemb is
false, and sz < size is also false, so ncalloc returns nmalloc(0). If
only one of nmemb or size is zero, then ncalloc returns NULL.
av
>Nevertheless this behavior of unsigned C arithmetic is
>sureley NEVER CORRECT and I have never seen a useful
>program that relies on this behavior for something useful.
the "behavior of unsigned C arithmetic" mod 2^n is ok.
but for normal calculation could be well have some flag in the data
position for see if there are errors in overflow for that data
>jacob
kuy...@wizard.net
> kuy...@wizard.net a écrit :
...
> > It's not claptrap, unlike what Dubya is spewing, and I would therefore
> > certainly hope he's sufficiently intelligent and mathematically
> > well-educated to believe it. I'd recommend reading up on the
> > mathematical concept of rings before you claim otherwise.
> >
>
> Who cares about rings?
Rings are the mathematical construct that correspond to the way in
which the C standard defines arithmetic for unsigned types. If you
don't care about C unsigned arithmetic, then you don't need to worry
about rings. You don't have to actually know the term; so long as you
understand what modulo means you've got enough of the concept for
practical purposes. But when you imply that modular arithmetic involves
adjusting the mathematical result, rather than actually BEING the
mathematical result, it implies that there's a deficiency in your
mathematical knowledge that you might want to correct (or not - your
choice). Granted, the mathematical terminology isn't of great practical
importance outside of mathematics. However, when you see someone using
it, I'd recommend delaying any criticism of that terminology until
you've mastered it yourself.
> We are speaking about overflow in a well defined
> context. Yes, the C standard defines the behavior
> for overflow, and overflow then, it is defined
No, it defines the behavior in such a way that overflow never happens,
and says so explicitly.
> for unsigned integers. This doesn't mean that it
> doesn't happen or that this "ring" stuff is meaningful.
>
> Or are you implying that
>
> 65521 x 65552 is 65296 and NOT 4295032592
Correct: that is precisely the case when, as in this case,
multiplication is defined modulo 65536.
> With that 'x' I mean multiplication as is
> understood by all people in this planet except
> C buffs that know more about C standards than about
> software engineering and mathematics???
Don't forget mathematicians, who also have a broader understanding of
the concept of multiplication that you do.
Randy Howard
(in article <ln8xglh...@nuthaus.mib.org>):
> If this is a concern, you can write a wrapper around malloc() that
> takes two arguments (as calloc() does) and checks whether the result
> wraps around.
Yeah, I said that yesterday. It didn't take hold though.
Common sense doesn't count when you are coding to protect those without
common sense. That's the best way I can think of to describe what
Navia has been up to lately.
--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw
christian.bau
> The problem with int is that it throws away half the address space.
> This is a *big* issue with 16-bit address spaces, but of course
> such machines are largely relegated to embedded systems, and small
> ones at that. Nevertheless, I regularly need to deal with objects
> bigger than half the address space even today, perhaps because I
> write so much systems code. So I find the choice of slicing the
> address space in half arbitrary and potentially dangerous.
There are machines where int is 32 bits, and size_t is 64 bit. I am
sure I have used machines with 16 bit ints that could allocate objects
of a few hundred KB successfully in old DOS times.
(Question: What is the largest amount of data anyone here has
successfully allocated using malloc () and used? Just curious).
christian.bau
> In article <45982675$0$18612$8826...@free.teranews.com>,
> Stephen Sprunk <ste...@sprunk.org> wrote:
> ...
> >Most implementors, however, take the position that it's more important
> >to give people the freedom to do unexpected things than to treat them
> >like idiots who need adult supervision. Insulting your customers is not
> >a sustainable business practice.
>
> Seems to work just fine for MS. But then again, they're not "most
> implementors", I suppose. They're insignificant.
In practice, many, perhaps most implementations cannot successfully
allocate anywhere near (size_t) -1 bytes.
First, if for example size_t = 32 bits, then it is likely that the OS
cannot allocate objects greater than 2^32 bytes, and the implementation
of malloc couldn't handle OS objects greater than that. malloc usually
has a few bytes of overhead, so malloc ((size_t) -1) won't succeed
anyway.
If size_t is 64 bits, there is no way malloc ((size_t) -1) could
succeed. And most 32 bit systems have limit at 3GB or 3.5GB. Something
like malloc ((size_t) -100) could only succeed on a system where size_t
is much smaller than the actual address space, for example 32 bits with
64 bit pointers.
Randy Howard
(in article <1167773629.9...@a3g2000cwd.googlegroups.com>):
I don't recall keeping score, or trying to set a record, but I've
written code for memory testing of systems with 64GB of RAM (AMD64 and
EM64t systems) and have malloc'd (successfully) over 8GB in a single
chunk, I'm fairly sure. I don't have one in front of me with that much
memory right now, or I'd do some experiments.
Randy Howard
(in article <1167774543.8...@k21g2000cwa.googlegroups.com>):
> Kenny McCormack wrote:
>> In article <45982675$0$18612$8826...@free.teranews.com>,
>> Stephen Sprunk <ste...@sprunk.org> wrote:
>> ...
>>> Most implementors, however, take the position that it's more important
>>> to give people the freedom to do unexpected things than to treat them
>>> like idiots who need adult supervision. Insulting your customers is not
>>> a sustainable business practice.
>>
>> Seems to work just fine for MS. But then again, they're not "most
>> implementors", I suppose. They're insignificant.
>
> In practice, many, perhaps most implementations cannot successfully
> allocate anywhere near (size_t) -1 bytes.
That may very well be true. However, it doesn't really matter, as
malloc() will return the appropriate response for those cases. :-)
> And most 32 bit systems have limit at 3GB or 3.5GB.
Actually it's usually 2GB, due to splitting of address space between
the kernel and user space. Some go as high as 3GB with special boot
options. However, there are hacks (outside of malloc) that allow for
what Intel calls "Physical Address Extension" (PAE) to allow systems
with Intel 32-bit processors to see memory above 4GB, sort of like the
old extended/expanded memory hacks in the DOS days. Again,
proprietary, and different APIs to use the memory from what standard C
supports.
christian.bau
> What this has brought home to me is that calloc should be included
> in the nmalloc package, so that the same maximum size criterion
> will be applied. I.E:
>
> void *ncalloc(size_t nmemb, size_t size) {
> size_t sz;
> void *p;
>
> sz = nmemb * size;
> if ((sz < nmemb) || (sz < size)) return NULL;
> if (p = nmalloc(sz)) memset(p, 0, sz);
> return p;
> }
Just wanted to say: This will not catch many problems. For example on a
32 bit system, nmemb = 0x10001, size = 0x10001, sz = 0x20001.
jacob navia
lcc-win32 uses this:
void *calloc(size_t n,size_t s)
{
long long siz = (long long)n * (long long)s;
void *result;
if (siz>>32)
return 0;
result = malloc((size_t)siz);
if (result)
memset(result,0,(size_t)siz);
return result;
}
sizeof(long long)=8*8
sizeof(size_t)=4*8
Old Wolf
> If the argument is unsigned, you can't pass a -ve value to it.
"argument" means the value passed in. In this code:
#include <stdlib.h>
int foo() { malloc(-1); }
the argument to malloc is the value -1 (of type int).
The type "size_t" w.r.t. malloc is known as the "formal parameter type"
(often the word 'formal' is dropped for convenience's sake).
Old Wolf
>
> No, you don't get overflow with unsigned types.
Actually you do. However, the behaviour on overflow is well-defined.
I suppose you could quibble about the exact meaning of the word
'overflow', but it is clear what Navia's meaning is and it serves no
purpose to pretend he is saying something other than what he is.
Old Wolf
> On any given implementation, either size_t is big enough to store 65521 *
> 65552 or it isn't. If it is, there is no issue. And if it is not, your
> request is meaningless, since you're asking for an object bigger than the
> system can provide.
Firstly, systems might exist where you can allocate more memory
than SIZE_MAX.
That aside, wouldn't it be more sensible behaviour for malloc to
return NULL or take some other action when you try request an
object bigger than the system can provide, rather than returning
a smaller object than requested? I think this is Navia's point.
Ben Pfaff
> Richard Heathfield wrote:
>>
>> No, you don't get overflow with unsigned types.
>
> Actually you do. However, the behaviour on overflow is well-defined.
C99 6.2.5p9 makes it pretty clear that the Standard takes Richard's
point of view:
A computation involving unsigned operands can never
overflow, [...]
--
"The fact that there is a holy war doesn't mean that one of the sides
doesn't suck - usually both do..."
--Alexander Viro