Implementing my own memcpy
Rajan
I am trying to simulate a memcpy like this
void* mem_cpy(void* dest, void* src, int bytes)
{
dest = malloc(bytes);
}
Now if I want to copy the bytes from src to dest, how do I copy these
bytes.
I am stuck here.
Netocrat
Why can't you use memcpy itself? I'll assume you have a reason that
isn't "it's a homework assignment".
Are you trying to achieve the exact same behaviour as memcpy? If so,
then don't allocate memory - memcpy requires that the memory is already
allocated. The second parameter needs to be qualified with const. Also
the type of the third parameter is size_t not int - this requires that
you include stddef.h. Even if it were declared as int it should be
unsigned.
Here is an implementation:
#include <stddef.h>
void *mem_cpy(void *dest, const void *src, size_t bytes)
{
unsigned char *srcmax = dest + bytes;
while (src < srcmax)
*(unsigned char *)dest++ = *(unsigned char *)src++;
return dest;
}
Ben Pfaff
This does not make sense. If you are going to copy bytes from
src to dest, then replacing dest by something else (such as the
return value of malloc()) is not going to help, because it means
that you won't be able to refer to the destination buffer any
longer.
Have you tried reading a C tutorial?
--
"I ran it on my DeathStation 9000 and demons flew out of my nose." --Kaz
Rajan
With your code fragment, I find that you are using unsigned char* , but
I may have to copy a structure.
We certainly cannot do *dest++ = *src++ ; if they are void pointers.
So how do we deal with this?
Netocrat
I believe that there is no portable, generic way to copy a structure - you
need to write a specific routine that copies each element.
A non-portable way that will probably work would be to call the mem_cpy
function with the addresses of the destination and source structures as
the first and second parameters ie &your_struct_variable1 and
&your_struct_variable2 and with the third parameter as sizeof(struct
your_struct). I'll probably get jumped on for suggesting that though...
Regarding your comment about dest and src being void pointers:
In the code, unsigned char * is used as a cast. It doesn't change the
fact that the original pointers are void *, it just instructs the compiler
to treat them as unsigned char pointers. So you are right that *dest++ =
*src++ is invalid, because you can't dereference a pointer to void. But
typecasting them to unsigned char pointers and then dereferencing them is
guaranteed to always be valid. In this way unsigned char pointers are
unique. Interpret it like this:
1) we are given two pointers which can point to anything
2) we treat the pointers as though they each point to a single byte
(unsigned char can be interpreted as being a single byte)
3) we copy the data from the byte that the source pointer points to into
to the byte that the destination pointer points to
4) we move each pointer on to the next byte
5) we repeat steps 3 and 4 'bytes' times.
Ben Pfaff
> With your code fragment, I find that you are using unsigned char* , but
> I may have to copy a structure.
Any object may be copied as an array of characters.
--
"I'm not here to convince idiots not to be stupid.
They won't listen anyway."
--Dann Corbit
Clark S. Cox III
> On Sat, 25 Jun 2005 08:16:24 -0700, Rajan wrote:
>
>> Hi Netocrat ,
>> With your code fragment, I find that you are using unsigned char* , but I
>> may have to copy a structure.
>> We certainly cannot do *dest++ = *src++ ; if they are void pointers. So
>> how do we deal with this?
>
> I believe that there is no portable, generic way to copy a structure
Of course there is; in fact, there are several:
Assuming a and b are of the same complete type, any of the following
will copy the contents of a into b:
#include <stdlib.h>
#include <string.h>
/*1*/ b = a;
/*2*/ memcpy(&b, &a, sizeof b);
/*3*/ memmove(&b, &a, sizeof b);
/*4*/ const unsigned char *src = (const unsigned char*)&a;
unsigned char *dst = (unsigned char*)&b;
for(size_t i=0; i<sizeof b; ++i)
{
dst[i] = src[i];
}
--
Clark S. Cox, III
clar...@gmail.com
Netocrat
> On 2005-06-25 11:45:13 -0400, Netocrat <neto...@dodo.com.au> said:
>
>> I believe that there is no portable, generic way to copy a structure
>
> Of course there is; in fact, there are several:
>
> Assuming a and b are of the same complete type, any of the following will
> copy the contents of a into b:
I'm glad to be corrected. I know of those approaches but with all the
buzzwords - unbounded this and unspecified that - going around in this
group and without a copy of the standard - or any other authoritative
source - to refer to I'm finding it hard to know what's "conforming" - to
use another buzzword.
Stan Milam
What I think you are trying to do here is duplicate some object in
memory using dynamically allocated memory to store the duplicate. If
that is the case try this:
/**********************************************************************/
/* File Id: dupobj.c. */
/* Author: Stan Milam. */
/* Date Written: 28-Aug-1999. */
/* Description: */
/* Implement a function to duplicate an object in memory. */
/* */
/* (c) Copyright 2005 by Stan Milam. */
/* All rights reserved. */
/* */
/**********************************************************************/
#include <errno.h>
#include <stdlib.h>
#include <string.h>
/**********************************************************************/
/* Name: */
/* dupobj() - Duplicate an lvalue with allocated memory. */
/* */
/* Synopsis: */
/* #include "strtools.h." */
/* void *dupobj( void *obj, size_t objsize ); */
/* */
/* Description: */
/* Use the calloc() and memcpy() functions to allocate memory */
/* large enough to hold an object in memory, then use memcpy() */
/* to copy the contents of the object to the allocated memory. */
/* */
/* Arguments: */
/* void *obj */
/* size_t objsize - The size of the object in memory. */
/* */
/* Return Value: */
/* The address of the duplicated object. NULL when a memory */
/* allocation error occurs, or the *obj address is NULL. When */
/* the *obj address is NULL the global errno variable is set */
/* to EINVAL. */
/* */
/**********************************************************************/
void *
dupobj( void *obj, size_t objsize )
{
void *rv = NULL;
if ( obj == NULL || objsize == 0 )
errno = EINVAL;
else {
if ( ( rv = calloc( 1, objsize ) ) != NULL )
memcpy( rv, obj, objsize );
}
return rv;
}
CBFalconer
Always include adequate quotation from previous articles, so that
your message stands by itself. There is no reason to assume any
previous material is available to the reader.
One of my sigs reads as follows - heed it if you must use the foul
google interface. Better, get yourself a real newsreader.
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
The point about memcpy is that it copies memory areas, measured in
bytes, with no regard for the types involved. I gather you wish
the same thing, but with automatic creation of the destination. In
this case you need a different signature, such as:
void *dupmem(void *src, size_t sz);
The void * type can point at arbitrary things, and a size_t can
specify a size on any machine. But to use void* you have to
convert to other types, thus:
void *dupmem(void *src, size_t sz)
{
unsigned char *sp = src;
unsigned char *dst;
if (dst = malloc(sz)) /* memory is available */
while (sz--) *dst++ = *sp++; /* copy away */
return dst; /* will be NULL for failure */
} /* dupmem, untested */
Note how src is typed into sp, without any casts. Similarly the
reverse typing for the return value of dupmem. The usage will be,
for p some type of pointer:
if (p = dupmem(whatever, howbig)) {
/* success, carry on */
}
else {
/* abject failure, panic */
}
--
"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare
Robert W Hand
wrote:
>On Sat, 25 Jun 2005 07:28:30 -0700, Rajan wrote:
(...)
>#include <stddef.h>
>
>void *mem_cpy(void *dest, const void *src, size_t bytes)
>{
> unsigned char *srcmax = dest + bytes;
>
> while (src < srcmax)
I don't understand this line. You are comparing one array src to
another srcmax. Perhaps I missed something, but I do not believe that
they are looking into the same array. Did you mean to define srcmax
differently?
unsigned char* srcmax = src + bytes;
--
Best wishes,
Bob
Nils Weller
> [...]
> void *
> dupobj( void *obj, size_t objsize )
Why not ``const void *obj''?
> {
> void *rv = NULL;
>
> if ( obj == NULL || objsize == 0 )
> errno = EINVAL;
EINVAL is not a Standard C macro (the UNIX and POSIX standards do define
it, however.) In addition, I would argue that it is the caller's
responsibility to pass valid arguments. Have you noticed that even in
Unix, library-only functions tend not to set EINVAL much, and certainly
not for invalid pointer arguments? This is because such a requirement
would needlessly slow down and bloat correctly written code, i.e. code
that always passes valid arguments. It is usually functions invoking
system calls that set errno to EINVAL because the operating system
kernel *has to* check all arguments for correctness, so as to avoid
accidental or malicious system corruption.
I will also point out that your check is incomplete anyway because
``obj'' may contain an invalid address that isn't NULL; Which, for the
purpose of setting EINVAL in Unix, is an unmapped address.
> else {
> if ( ( rv = calloc( 1, objsize ) ) != NULL )
This will likely execute slower than necessary, since calloc() is
conceptually a malloc() + memset(); A plain malloc() will suffice
because you immediately overwrite the memory anyway.
--
Nils R. Weller, Bremen / Germany
My real email address is ``nils<at>gnulinux<dot>nl''
... but I'm not speaking for the Software Libre Foundation!
Netocrat
Yes - typo, I don't know how that snuck in there - it's obvious from the
naming that it's wrong.
Cheers.
Chris Torek
Netocrat <neto...@dodo.com.au> wrote:
>void *mem_cpy(void *dest, const void *src, size_t bytes)
>{
> unsigned char *srcmax = dest + bytes;
>
> while (src < srcmax)
> *(unsigned char *)dest++ = *(unsigned char *)src++;
> return dest;
>}
This is flawed for three reasons:
- As someone else pointed out, "srcmax" is computed from "dest"
but is used for comparison with "src".
- The "dest++" and "src++" operations are invalid because dest
and src have type "void *", and thus need to increment by
sizeof(void), as it were. (There are some non-C compilers,
such as gcc, that define sizeof(void)==1 just to make this
work anyway. There are some other non-C compilers that fail
to catch the error -- a diagnostic is required in ANSI/ISO
C, either C89 or C99 -- and treat sizeof(void) as 0, and
thus produce an infinite loop!)
- Finally, the return value (dest) would be modified from the
original input value: you would have to retract it by "bytes"
bytes first.
The easiest fix for all of these is to use auxiliary variables.
I prefer to leave the input parameters unmodified, and create
auxiliaries of type "unsigned char *":
void *like_memcpy(void *restrict dst0, const void *restrict src0,
size_t n) {
unsigned char *restrict dst = dst0;
unsigned char *restrict src = src0;
if (n)
do
*dst++ = *src++;
while (--n != 0);
return dst0;
}
(you can also write the loop as "while (n--) *dst++ = *src++" but I
find the above easier to read and think about).
This is strictly conforming C code because it is always allowed to
examine or copy objects' representations one "C byte" (unsigned
char) at a time.
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.
Kevin Bagust
> On Sat, 25 Jun 2005 07:28:30 -0700, Rajan wrote:
>
>>Hi,
>>I am trying to simulate a memcpy like this void* mem_cpy(void* dest, void*
>>src, int bytes) {
>> dest = malloc(bytes);
>>}
>>Now if I want to copy the bytes from src to dest, how do I copy these
>>bytes.
>
> then don't allocate memory - memcpy requires that the memory is already
> allocated. The second parameter needs to be qualified with const. Also
> the type of the third parameter is size_t not int - this requires that
> you include stddef.h. Even if it were declared as int it should be
> unsigned.
>
> Here is an implementation:
>
> #include <stddef.h>
>
> void *mem_cpy(void *dest, const void *src, size_t bytes)
> {
> unsigned char *srcmax = dest + bytes;
>
> while (src < srcmax)
> *(unsigned char *)dest++ = *(unsigned char *)src++;
> return dest;
> }
>
You cannot perform pointer arithmetic on void pointers, you need to
convert them to pointers to another type before you perform the
addition. memcpy should return the value of dest that is passed into the
function. So the code should be something like this:
#include <stddef.h>
void *mem_cpy( void *dest, const void *src, size_t bytes )
{
unsigned char *destPtr = dest;
unsigned char const *srcPtr = src;
unsigned char const *srcEnd = srcPtr + bytes;
while ( srcPtr < srcEnd ) {
*destPtr++ = *srcPtr++;
}
return dest;
}
Kevin.
Netocrat
> In article <pan.2005.06.25....@dodo.com.au> Netocrat
> <neto...@dodo.com.au> wrote:
>>void *mem_cpy(void *dest, const void *src, size_t bytes) {
>> unsigned char *srcmax = dest + bytes;
>>
>> while (src < srcmax)
>> *(unsigned char *)dest++ = *(unsigned char *)src++;
>> return dest;
>>}
>>}
> This is flawed for three reasons:
>
> - As someone else pointed out, "srcmax" is computed from "dest"
> but is used for comparison with "src".
>
> - The "dest++" and "src++" operations are invalid because dest
> and src have type "void *", and thus need to increment by
> sizeof(void), as it were. (There are some non-C compilers, such as
> gcc, that define sizeof(void)==1 just to make this work anyway.
> There are some other non-C compilers that fail to catch the error --
> a diagnostic is required in ANSI/ISO C, either C89 or C99 -- and
> treat sizeof(void) as 0, and thus produce an infinite loop!)
Yes thoughts about what type the increment operator was operating on did
pass through my mind as I was writing it. When it worked (I use gcc) I
assumed that it was because the cast to unsigned char * had informed the
compiler how to properly apply pointer arithmetic - that had I left it
uncast as void * the increment operator would not have worked at all. Now
that I reconsider that assumption I can see that it's flawed because had
the cast been taken into account the result would no longer have been a
modifiable lvalue and the increment could not have been applied.
And indeed checking this shows that gcc still doesn't complain about
incrementing a void pointer.
> - Finally, the return value (dest) would be modified from the
> original input value: you would have to retract it by "bytes" bytes
> first.
Obviously. A silly error.
> The easiest fix for all of these is to use auxiliary variables. I prefer
> to leave the input parameters unmodified, and create auxiliaries of type
> "unsigned char *":
I've got into the habit of trying to minimise variable usage and required
calculations (sometimes orthogonal aims). That's the main reason I didn't
use variables in this case. It's not so big a deal with an optimising
compiler because little details like that all get ironed out in the wash,
but in this forum you can't make any assumptions about whether or not code
will be compiled with optimisations.
Obviously my code won't function without change. Your suggestion works.
The other option is to maintain the cast and use indexing instead of
pointer arithmetic. I don't like that option though because it requires
more calculations to index for a non-optimising compiler.
Whilst decrementing n and testing it for loop end works fine, I prefer to
test against a maximum pointer and save the cost of the decrement
operation. I think it's also more readable.
> void *like_memcpy(void *restrict dst0, const void *restrict src0,
> size_t n) {
> unsigned char *restrict dst = dst0;
> unsigned char *restrict src = src0;
>
> if (n)
> do
> *dst++ = *src++;
> while (--n != 0);
> return dst0;
> }
> (you can also write the loop as "while (n--) *dst++ = *src++" but I find
> the above easier to read and think about).
I prefer the conciseness of the second, but I prefer even more testing
against a maximum pointer.
Netocrat
> Netocrat wrote:
[snip code]
>
> You cannot perform pointer arithmetic on void pointers, you need to
> convert them to pointers to another type before you perform the addition.
> memcpy should return the value of dest that is passed into the function.
> So the code should be something like this:
>
> #include <stddef.h>
>
> void *mem_cpy( void *dest, const void *src, size_t bytes ) {
> unsigned char *destPtr = dest;
> unsigned char const *srcPtr = src;
> unsigned char const *srcEnd = srcPtr + bytes;
>
> while ( srcPtr < srcEnd ) {
> *destPtr++ = *srcPtr++;
> }
> return dest;
> }
Great - that's just how I would have rewritten it to correct the bugs -
apart from naming conventions; and I may not have thought to qualify the
pointers with const but it is appropriate and good practice.
Chris Torek
>> (you can also write the loop as "while (n--) *dst++ = *src++" but I find
>> the above easier to read and think about).
In article <pan.2005.06.25....@dodo.com.au>
My thinking is perhaps colored by too many years of assembly coding
and instruction sets that include "decrement and branch if nonzero":
test r3
bz Laround
Lloop:
mov (r1)+,(r2)+
sobgtr r3,Lloop # cheating (but this is OK)
Laround:
and:
tst d1
bz Laround
addq #-1,d1 # dbcc/dbra uses -1 instead of 0
Lloop:
mov a1@+,a2@+
dbra d1,Lloop
# insert fixup code for 16-bit dbra instructions
...
Laround:
and so on. (The first loop is VAX assembly, and "cheating" is OK
because r1 and/or r2 should never cross from P0/P1 space to S space,
nor vice versa, so the maximum block size never exceeds 2 GB; the
second loop is 680x0 assembly and uses a special trick in the
original 68000, in which a one-instruction dbcc loop never re-fetches
the instructions over the bus, so that the loop runs about twice
as fast as any other version. You *have* to use the dbcc instructions,
no other loop construct does the trick! If you can force-fit any
repeated memory operation into a dbra, even if there is no reason
to decrement, it is worth it, on the 68000. Later CPUs had
instruction caches, making the trick obsolete, though.)
Modern CPUs have "branch on register not zero" but also may have
"branch on r1 not equal to r2". MIPS and SPARCv9 lack the latter,
so the counted loop is still better; but on SPARCv9 at least, you
really want to implement large block-copy operations using the
special cache-conserving streaming load operations, through the
floating point registers. (But that assembly code is far too
large to post off-topically to comp.lang.c :-) )
Netocrat
>>On Sat, 25 Jun 2005 18:31:30 +0000, Chris Torek wrote:
>>> (you can also write the loop as "while (n--) *dst++ = *src++" but I
>>> find the above easier to read and think about).
>
> In article <pan.2005.06.25....@dodo.com.au> Netocrat
> <neto...@dodo.com.au> wrote:
>>I prefer the conciseness of the second, but I prefer even more testing
>>against a maximum pointer.
>
> My thinking is perhaps colored by too many years of assembly coding and
> instruction sets that include "decrement and branch if nonzero":
Interesting where the differences lead us. I did some x86 assembly way
back - not much but enough to get the general idea. As for any other
architectures - I've used them but haven't coded their assembly.
So my thinking is based mainly on my own concepts of an 'abstract machine'
and the operations it would perform with my own idea of the abstract cost
of those operations.
In practice your thinking is likely to lead to code that is efficient on a
given set of architectures.
My approach will be more efficient on a machine in my head. :) Whether
that translates into efficiency on real machines depends on
a) the natural fit of my abstract machine to the real hardware
b) of the ability of the compiler to optimise my semantics for the real
hardware.
There is another dependency - optimising my semantics for the abstract
machine. But that's a step I deem it upon myself to take and prefer not
to rely on the compiler to provide. And that's the point where according
to my abstract concepts, the 'maximum pointer test' is cheaper than the
separate counter.
But it's only valid for my personal abstract machine. Another
programmer might have their own personal abstract machine with a
different cost weighting. And your cost calculations are valid for the
machines you know have that type of instruction.
In the end when we're talking about non-platform-specific C as on this
forum, apart from blazingly obvious inefficiencies, and representations
that can only be implemented by hardware in one particular way, things
like this are a matter of personal preference and style, would you not
agree?
[snip assembly and explanation]
> Modern CPUs have "branch on register not zero" but also may have "branch
> on r1 not equal to r2". MIPS and SPARCv9 lack the latter, so the
> counted loop is still better; but on SPARCv9 at least, you really want
> to implement large block-copy operations using the special
> cache-conserving streaming load operations, through the floating point
> registers. (But that assembly code is far too large to post
> off-topically to comp.lang.c
> :-) )
From vague memory I know x86 has the jump on zero, jump on less than etc
but I have no recollection of whether this can be combined with a
decrement in a single instruction.
We're wandering somewhat off-topic here but these are things to consider
in the context of writing non-specific code and being aware that it must
ultimately run on an implementation which will have strengths and
weaknesses. How much should you abstract those implementation-specific
strengths and weaknesses into generic ones and code for those? Is there
any validity to the concept of a generic strength or weakness for an
abstract machine?
Stan Milam
> In article <J7gve.1882$5w3...@newssvr11.news.prodigy.com>, Stan Milam wrote:
>
>>[...]
>>void *
>>dupobj( void *obj, size_t objsize )
>
>
> Why not ``const void *obj''?
>
>
>>{
>> void *rv = NULL;
>>
>> if ( obj == NULL || objsize == 0 )
>> errno = EINVAL;
>
>
> EINVAL is not a Standard C macro (the UNIX and POSIX standards do define
> it, however.) In addition, I would argue that it is the caller's
> responsibility to pass valid arguments. Have you noticed that even in
> Unix, library-only functions tend not to set EINVAL much, and certainly
> not for invalid pointer arguments? This is because such a requirement
> would needlessly slow down and bloat correctly written code, i.e. code
> that always passes valid arguments. It is usually functions invoking
> system calls that set errno to EINVAL because the operating system
> kernel *has to* check all arguments for correctness, so as to avoid
> accidental or malicious system corruption.
All well and good. However, I choose to use it. I will endevor to
remove it if I post it here to mollify all the useless UBP types.
>
> I will also point out that your check is incomplete anyway because
> ``obj'' may contain an invalid address that isn't NULL; Which, for the
> purpose of setting EINVAL in Unix, is an unmapped address.
>
>
Well, this part I have no way of knowing so I must take it in good
faith. That does not mean I can't be defensive where I can.
>> else {
>> if ( ( rv = calloc( 1, objsize ) ) != NULL )
>
>
> This will likely execute slower than necessary, since calloc() is
> conceptually a malloc() + memset(); A plain malloc() will suffice
> because you immediately overwrite the memory anyway.
>
Again, this is true, but I am comforted that the memory is clean when I
get it. Got it?
Regards,
Stan Milam.
CBFalconer
>
... snip ...
>
> And indeed checking this shows that gcc still doesn't complain
> about incrementing a void pointer.
Showing that you are using gcc incorrectly. Use -W -Wall -ansi
-pedantic. You can replace -ansi with -std=C99 if you wish and
your library can handle it.
--
pete
> without a copy of the standard
You can get by somewhat with N869,
until you acquire a copy of the standard.
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n869/n869.txt.gz
--
pete
pete
> if (n)
> do
> *dst++ = *src++;
> while (--n != 0);
> return dst0;
> }
>
> (you can also write the loop as "while (n--) *dst++ = *src++" but I
> find the above easier to read and think about).
To me,
while (n-- != 0) {}
is very easy to read as a loop that's going to iterate
as many times as what the initial value of n is.
--
pete
Netocrat
> Netocrat wrote:
>>
> ... snip ...
>>
>> And indeed checking this shows that gcc still doesn't complain about
>> incrementing a void pointer.
> Showing that you are using gcc incorrectly. Use -W -Wall -ansi
> -pedantic.
> You can replace -ansi with -std=C99 if you wish and your library can
> handle it.
The word complain was a poor choice, and if you had read it in context
you would have understood that and wouldn't have to pick nits with my
phrasing.
I said that my assumption had been that trying to increment a void
pointer "would not work at all", not that it would generate a "complaint",
and that that assumption is what caused me to falsely believe that the
cast had modified the pointer arithmetic.
So the sentence you quote was added to confirm that my assumption was
wrong: that gcc does still "work" ie compile without error given code that
increments a void pointer; not that it doesn't generate warnings.
Granted, I didn't use the options in this case but that's beside the point
I was trying to make.
I am aware of those options and usually do use them (apart from the -W
option which now that I read about it looks useful, so I'll add it in
future).
Look, I understand the importance of correctness and precision especially
in a group about a standardised programming language, but if you're going
to jump on what you see as an error in someone's phrasing - and even to go
further and make assumptions about their general use of the compiler - at
least give them the benefit of the doubt and try to see if what they're
saying can be interpreted correctly given the overall context.
Netocrat
How widely and fully implemented is the later standard C99 versus the
earlier C90 standard?
Lefty Bigfoot
(in article <pan.2005.06.26....@dodo.com.au>):
Don't write anything that uses C99 extensions unless you don't
care about portability this century.
Stan Milam
Yes! Amen!
Stan.
Lawrence Kirby
> "Rajan" <rsta...@yahoo.com> writes:
>
>> With your code fragment, I find that you are using unsigned char* , but
>> I may have to copy a structure.
>
> Any object may be copied as an array of characters.
But only portably using characters with an unsigned representation.
Lawrence
Richard Harter
wrote:
>
>I prefer the conciseness of the second, but I prefer even more testing
>against a maximum pointer.
>
You shouldn't. IMNSHO one should be equally comfortable with testing
against a minimum as with testing against a maximum, just as in
mathematics one should be comfortable both with proof by induction and
proof by infinite descent, and in programming with both recursion and
iteration. Not being comfortable with both directions limits you.
Richard Harter, c...@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.
Chris Torek
Netocrat <neto...@dodo.com.au> wrote:
>How widely and fully implemented is the later standard C99 versus the
>earlier C90 standard?
"Not very". The situation is improving (inasmuch as C99 is an
"improvement" anyway :-) ), but slowly. A few years ago there were
no compilers that claimed even to attempt C99 support; now I think
there are two or three.
Richard Harter
Indeed. A disadvantage, though, is that n does not correspond to
array indices. An alternate formulation is
for (; --n >= 0;) {}
This can be useful if one wants to work, so to speak, from the high
end to the low end of an array. In a copy routine one wants to be
able to work either way depending on overlaps. Thus if the layout
looks like this
SSSSS.....
DDDDD.....
one wants to work from the high end down whereas if it looks like
DDDDD.....
SSSSS.....
one wants to work from low end up. Thus (in a flat memory model) we
might say something like
if (src <dest) for ( ;--n >= 0 ; ) dst[n] = src[n];
else for (i=0; --n>= 0 ; i++) dst[i] = src[i];
provided that we wanted to use array indexing rather than pointers.
Michael Mair
> On Sun, 26 Jun 2005 01:09:13 GMT, pete <pfi...@mindspring.com> wrote:
>
>
>>Chris Torek wrote:
>>
>>
>>> if (n)
>>> do
>>> *dst++ = *src++;
>>> while (--n != 0);
>>> return dst0;
>>> }
>>>
>>>(you can also write the loop as "while (n--) *dst++ = *src++" but I
>>>find the above easier to read and think about).
>>
>>To me,
>> while (n-- != 0) {}
>>is very easy to read as a loop that's going to iterate
>>as many times as what the initial value of n is.
>
>
> Indeed. A disadvantage, though, is that n does not correspond to
> array indices. An alternate formulation is
>
> for (; --n >= 0;) {}
You just got yourself an infinite loop for n of unsigned integer
type -- which incidentally is the case (look upthread: size_t n).
If you insist on this form, use
for (; --n != (size_t)-1;) {}
instead. Even in situations where the cast is not necessary, I
would leave it there for clarity and as ward against
"optimisations".
[snip]
Cheers
Michael
--
E-Mail: Mine is an /at/ gmx /dot/ de address.
CBFalconer
The following accesses a draft that include further modifications,
but has the disadvantage of being a .pdf file rather than a .txt
file. This hampers quick and easy searches, grepping, quoting,
etc.
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf>
Meanwhile, I think I have a suitably formatted and organized
version of n869.txt mounted as:
<http://cbfalconer.home.att.net/download/n869_txt.bz2>
which means you have to have bzip2 on your system to extract it
properly, but minimizes the download time (it is about 200k).
CBFalconer
> On Sat, 25 Jun 2005 22:12:46 +0000, CBFalconer wrote:
>> Netocrat wrote:
>>>
>> ... snip ...
>>>
>>> And indeed checking this shows that gcc still doesn't complain
>>> about incrementing a void pointer.
>
>> Showing that you are using gcc incorrectly. Use -W -Wall -ansi
>> -pedantic. You can replace -ansi with -std=C99 if you wish and
>> your library can handle it.
>
> The word complain was a poor choice, and if you had read it in
> context you would have understood that and wouldn't have to pick
> nits with my phrasing.
>
> I said that my assumption had been that trying to increment a void
> pointer "would not work at all", not that it would generate a
> "complaint", and that that assumption is what caused me to falsely
> believe that the cast had modified the pointer arithmetic.
The point is not so much what you understand, but what other
readers of your article will understand. My post was intended to
correct the mistaken impression that gcc fails to properly diagnose
misuse of a void *.
--
Chuck F (cbfal...@yahoo.com) (cbfal...@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!
pete
pete
>
> On Sun, 26 Jun 2005 01:09:13 GMT, pete <pfi...@mindspring.com> wrote:
>
> >Chris Torek wrote:
> >
> >> if (n)
> >> do
> >> *dst++ = *src++;
> >> while (--n != 0);
> >> return dst0;
> >> }
> >>
> >> (you can also write the loop as "while (n--) *dst++ = *src++" but I
> >> find the above easier to read and think about).
> >
> >To me,
> > while (n-- != 0) {}
> >is very easy to read as a loop that's going to iterate
> >as many times as what the initial value of n is.
>
> Indeed. A disadvantage, though, is that n does not correspond to
> array indices. An alternate formulation is
>
> for (; --n >= 0;) {}
>
> This can be useful if one wants to work, so to speak, from the high
> end to the low end of an array.
No. Like Michael Mair said, you infinite looped.
(n-- != 0) is exactly perfect for work
from the high end to the low end of an array.
void *mem_cpy(void *s1, const void *s2, size_t n)
{
unsigned char *p1 = s1;
const unsigned char *p2 = s2;
while (n-- != 0) {
p1[n] = p2[n];
}
return s1;
}
--
pete
CBFalconer
> Richard Harter wrote:
>
... snip ...
>>
>> Indeed. A disadvantage, though, is that n does not correspond to
>> array indices. An alternate formulation is
>>
>> for (; --n >= 0;) {}
>
> You just got yourself an infinite loop for n of unsigned integer
> type -- which incidentally is the case (look upthread: size_t n).
> If you insist on this form, use
> for (; --n != (size_t)-1;) {}
> instead. Even in situations where the cast is not necessary, I
> would leave it there for clarity and as ward against
> "optimisations".
For something like that use a while loop, no need for the other for
clauses. However the for can combine initialization, and allows:
for (n = SIZE; n--; /* maybe something else */ ) {}
and is proof against unsignedness, yet n within the loop is a
direct array index. However I would normally use a while loop
anyhow.
Richard Harter
<Michae...@invalid.invalid> wrote:
>Richard Harter wrote:
>You just got yourself an infinite loop for n of unsigned integer
>type -- which incidentally is the case (look upthread: size_t n).
>If you insist on this form, use
> for (; --n != (size_t)-1;) {}
>instead. Even in situations where the cast is not necessary, I
>would leave it there for clarity and as ward against
>"optimisations".
My apologies. I thought it obvious that the form I mentioned required
signed integers. I didn't think it necessary to mention the caveat
but I was in error.
Richard Harter
>Richard Harter wrote:
>>
>> On Sun, 26 Jun 2005 01:09:13 GMT, pete <pfi...@mindspring.com> wrote:
>>
>> >Chris Torek wrote:
>> >
>> >> if (n)
>> >> do
>> >> *dst++ = *src++;
>> >> while (--n != 0);
>> >> return dst0;
>> >> }
>> >>
>> >> (you can also write the loop as "while (n--) *dst++ = *src++" but I
>> >> find the above easier to read and think about).
>> >
>> >To me,
>> > while (n-- != 0) {}
>> >is very easy to read as a loop that's going to iterate
>> >as many times as what the initial value of n is.
>>
>> Indeed. A disadvantage, though, is that n does not correspond to
>> array indices. An alternate formulation is
>>
>> for (; --n >= 0;) {}
>>
>> This can be useful if one wants to work, so to speak, from the high
>> end to the low end of an array.
>
>No. Like Michael Mair said, you infinite looped.
Well, no, I didn't. See reply to Michael.
>
>(n-- != 0) is exactly perfect for work
>from the high end to the low end of an array.
>
>void *mem_cpy(void *s1, const void *s2, size_t n)
>{
> unsigned char *p1 = s1;
> const unsigned char *p2 = s2;
>
> while (n-- != 0) {
> p1[n] = p2[n];
> }
> return s1;
>}
Point taken. That form is safe (and best) for unsigned integers.
However it is unsafe for signed integers; if for any reason n is
negative it blows up.
pete
If you're thinking that n could be negative,
then it's simpler to write the code so that there
are no values of n which cause undefined behavior.
while ( n > 0) {--n;}
--
pete
Netocrat
> On Sun, 26 Jun 2005 05:33:17 +1000, Netocrat <neto...@dodo.com.au> wrote:
>
>>I prefer the conciseness of the second, but I prefer even more testing
>>against a maximum pointer.
>>
> You shouldn't.
You're dictating my personal preferences? Given a choice where all other
things are equal, you are saying that having a preference should not be
allowed?
> IMNSHO one should be equally comfortable with testing
> against a minimum as with testing against a maximum, just as in
> mathematics one should be comfortable both with proof by induction and
> proof by infinite descent, and in programming with both recursion and
> iteration. Not being comfortable with both directions limits you.
Sure - I have no problems with that statement. I understand and can use
all of those concepts where appropriate - apart from proof by infinite
descent with which I am unacquainted. Where I can see no significant
benefit for using one approach over the other though, I disagree that a
preference is limiting. Often there is a link between a preference and a
level of understanding of a particular technique, but that isn't
necessarily true.
Mark F. Haigh
> On Sat, 25 Jun 2005 22:12:46 +0000, CBFalconer wrote:
>
> > Netocrat wrote:
> >>
> > ... snip ...
> >>
> >> And indeed checking this shows that gcc still doesn't complain about
> >> incrementing a void pointer.
> > Showing that you are using gcc incorrectly. Use -W -Wall -ansi
> > -pedantic.
> > You can replace -ansi with -std=C99 if you wish and your library can
> > handle it.
>
> The word complain was a poor choice, and if you had read it in context
> you would have understood that and wouldn't have to pick nits with my
> phrasing.
Quit whining. Here "complain" means "produce at least one diagnostic
message" (C99 5.1.1.3). If you did not receive one for your code, you
are not compiling C; rather, you're compiling some other C-like
language (ie GNU C). C-like languages are not topical here.
Chuck is telling you or anyone listening how to invoke gcc in a
standards-compliant mode. You should listen to him.
>
> I said that my assumption had been that trying to increment a void
> pointer "would not work at all", not that it would generate a "complaint",
> and that that assumption is what caused me to falsely believe that the
> cast had modified the pointer arithmetic.
The difference between the two is not very meaningful. When you ignore
a diagnostic that pertains to syntax or constraint violations, you get
undefined behavior (UB). So it might work when you run it, it might
not, or it might cause earthquakes off the coast of California. What
happens when you run code containing UB is not defined by the C
standard, which is what we discuss here.
>
> So the sentence you quote was added to confirm that my assumption was
> wrong: that gcc does still "work" ie compile without error given code that
> increments a void pointer; not that it doesn't generate warnings.
> Granted, I didn't use the options in this case but that's beside the point
> I was trying to make.
>
> I am aware of those options and usually do use them (apart from the -W
> option which now that I read about it looks useful, so I'll add it in
> future).
>
> Look, I understand the importance of correctness and precision especially
> in a group about a standardised programming language, but if you're going
> to jump on what you see as an error in someone's phrasing - and even to go
> further and make assumptions about their general use of the compiler - at
> least give them the benefit of the doubt and try to see if what they're
> saying can be interpreted correctly given the overall context.
So what's your point? Either speak more precisely or grow a thicker
skin. Consider doing both if you plan on posting here regularly.
Mark F. Haigh
mfh...@sbcglobal.net
Netocrat
> Netocrat wrote:
>> On Sat, 25 Jun 2005 22:12:46 +0000, CBFalconer wrote:
>>
>> > Netocrat wrote:
>> >>
>> > ... snip ...
>> >>
>> >> And indeed checking this shows that gcc still doesn't complain about
>> >> incrementing a void pointer.
>> > Showing that you are using gcc incorrectly. Use -W -Wall -ansi
>> > -pedantic.
>> > You can replace -ansi with -std=C99 if you wish and your library can
>> > handle it.
>>
>> The word complain was a poor choice, and if you had read it in context
>> you would have understood that and wouldn't have to pick nits with my
>> phrasing.
>
> Quit whining.
...
> So what's your point? Either speak more precisely or grow a thicker
> skin.
> Consider doing both if you plan on posting here regularly.
I'll take your advice leave it there then.
Netocrat
<snip>
> The point is not so much what you understand, but what other readers of
> your article will understand. My post was intended to correct the
> mistaken impression that gcc fails to properly diagnose misuse of a void
> *.
Fair enough. As Mark Haigh pointed out my response was a bit of an
over-reaction. I didn't express myself well or correctly in the first
place so my later complaint against your response does seem petulant.
Richard Harter
>
>If you're thinking that n could be negative,
>then it's simpler to write the code so that there
>are no values of n which cause undefined behavior.
>
>while ( n > 0) {--n;}
This won't do; it's best to keep the flow control code out of the loop
body. However your observation is on point. The formula should work
for signed and unsigned (no separate forms for integer types), and
should produce iterate values in the range [n-1...0], and should not
execute the loop body if n<0. For ascending loops the formula
for (i=0; i<n; i++) {}
handles both signed and unsigned. For descending loops a variant of
Chris's formulation may be best, e.e.,
if (n>0) for(i=n; i-- > 0;) {}
Richard Harter
wrote:
>On Sun, 26 Jun 2005 16:19:39 +0000, Richard Harter wrote:
>
>> On Sun, 26 Jun 2005 05:33:17 +1000, Netocrat <neto...@dodo.com.au> wrote:
>>
>>>I prefer the conciseness of the second, but I prefer even more testing
>>>against a maximum pointer.
>>>
>> You shouldn't.
>
>You're dictating my personal preferences? Given a choice where all other
>things are equal, you are saying that having a preference should not be
>allowed?
You're inventing paranoid interpretations?
(IOW: Don't be silly.)
>
>> IMNSHO one should be equally comfortable with testing
>> against a minimum as with testing against a maximum, just as in
>> mathematics one should be comfortable both with proof by induction and
>> proof by infinite descent, and in programming with both recursion and
>> iteration. Not being comfortable with both directions limits you.
>
>Sure - I have no problems with that statement. I understand and can use
>all of those concepts where appropriate - apart from proof by infinite
>descent with which I am unacquainted. Where I can see no significant
>benefit for using one approach over the other though, I disagree that a
>preference is limiting. Often there is a link between a preference and a
>level of understanding of a particular technique, but that isn't
>necessarily true.
We shall disagree, you and I. Preferences are always limiting because
they create habits that lead you to make the preferred choice without
thinking about it. If there truly is no significant difference that's
a win. However if your habit keeps you from asking the question of
which way to do it, then you lose when the choice does matter.
pete
>
> On Mon, 27 Jun 2005 06:33:44 GMT, pete <pfi...@mindspring.com> wrote:
>
> >
> >If you're thinking that n could be negative,
> >then it's simpler to write the code so that there
> >are no values of n which cause undefined behavior.
> >
> >while ( n > 0) {--n;}
>
> This won't do; it's best to keep the flow control code out of the loop
> body.
There is no flow control in the loop body.
Writing a while loop that checks the value of a variable
which was changed in the body of the loop,
is just the most natural thing.
char *str_rev(char *s)
{
char *p, *q, swap;
if (*s != '\0') {
p = s;
q = p + strlen(p + 1);
while (q > p) {
swap = *q;
*q-- = *p;
*p++ = swap;
}
}
return s;
}
--
pete
Richard Harter
>Richard Harter wrote:
>>
>> On Mon, 27 Jun 2005 06:33:44 GMT, pete <pfi...@mindspring.com> wrote:
>>
>> >
>> >If you're thinking that n could be negative,
>> >then it's simpler to write the code so that there
>> >are no values of n which cause undefined behavior.
>> >
>> >while ( n > 0) {--n;}
>>
>> This won't do; it's best to keep the flow control code out of the loop
>> body.
>
>There is no flow control in the loop body.
>Writing a while loop that checks the value of a variable
>which was changed in the body of the loop,
>is just the most natural thing.
Sorry, I wasn't clear. What is wanted here is a formula for which the
iteration variable is referenced within the loop body but not altered,
i.e.,
<expression generating iterate><loop body using unaltered iterate>
Of course one can write loops that change a variable within the loop;
that is the general case. In the general case the flow control code
and the loop body code are mingled together. In the special case we
can cleanly separate them if we choose to do so. AFAIK C lacks any
guaranteed way to keep the loop body from mucking with the flow
control variables; you have to use an idiom.
Netocrat
> On Mon, 27 Jun 2005 19:41:07 +1000, Netocrat <neto...@dodo.com.au> wrote:
>
>>On Sun, 26 Jun 2005 16:19:39 +0000, Richard Harter wrote:
>>
>>> On Sun, 26 Jun 2005 05:33:17 +1000, Netocrat <neto...@dodo.com.au>
>>> wrote:
>>>
>>>>I prefer the conciseness of the second, but I prefer even more testing
>>>>against a maximum pointer.
>>>>
>>> You shouldn't.
>>
>>You're dictating my personal preferences? Given a choice where all other
>>things are equal, you are saying that having a preference should not be
>>allowed?
>
> You're inventing paranoid interpretations?
>
> (IOW: Don't be silly.)
Should I be allowed to prefer my interpretation?
Don't take that seriously - this time it was _intended_ to be silly.
>>> IMNSHO one should be equally comfortable with testing against a minimum
>>> as with testing against a maximum, just as in mathematics one should be
>>> comfortable both with proof by induction and proof by infinite descent,
>>> and in programming with both recursion and iteration. Not being
>>> comfortable with both directions limits you.
>>
>>Sure - I have no problems with that statement. I understand and can use
>>all of those concepts where appropriate - apart from proof by infinite
>>descent with which I am unacquainted. Where I can see no significant
>>benefit for using one approach over the other though, I disagree that a
>>preference is limiting. Often there is a link between a preference and a
>>level of understanding of a particular technique, but that isn't
>>necessarily true.
>
> We shall disagree, you and I. Preferences are always limiting because
> they create habits that lead you to make the preferred choice without
> thinking about it. If there truly is no significant difference that's a
> win. However if your habit keeps you from asking the question of which
> way to do it, then you lose when the choice does matter.
Yes, no, perhaps - I suspect that we are more in agreement than
disagreement on this point anyway.
<Off-topic>
A friend once suggesed that the vast majority of arguments occur between
people who fundamentally agree but are unable to communicate with the
other person in a way that allows them to understand that they are in
agreement. I've always remembered that idea. Often I find that arguments
actually arise over subtle differences in assumptions or definitions, etc
that either don't arise during the overt argument or arise after hours of
frustrating conflict - frustrating because neither can understand the
apparent stupidity of the other in not accepting claims that are based on
assumptions or definitions which they falsely assume the other person to
share.
Anyhow this rant is better suited to a newsgroup about conflict resolution
or pop psychology so I'll leave it there.
</Off-topic>
Netocrat
> Netocrat wrote:
>> On Sun, 26 Jun 2005 00:15:59 +0000, pete wrote:
>>> Netocrat wrote:
>>>
>>>> without a copy of the standard
>>>
>>> You can get by somewhat with N869, until you acquire a copy of the
>>> standard.
>>>
>>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n869/n869.txt.gz
>>
>> How widely and fully implemented is the later standard C99 versus the
>> earlier C90 standard?
>
> The following accesses a draft that include further modifications, but has
> the disadvantage of being a .pdf file rather than a .txt file. This
> hampers quick and easy searches, grepping, quoting, etc.
>
> <http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf>
>
> Meanwhile, I think I have a suitably formatted and organized version of
> n869.txt mounted as:
>
> <http://cbfalconer.home.att.net/download/n869_txt.bz2>
>
> which means you have to have bzip2 on your system to extract it properly,
> but minimizes the download time (it is about 200k).
Does your text version include the modifications of the pdf version?
How is it different from the document in the link pete gave originally -
what are the modifications and when were they introduced?
Also C90 and C89 seem to be interchangeable terms - correct?
Finally I understand that C90/C89 had some modifications made prior to C99
- where are those detailed?
Netocrat
Oh and strictly speaking does ANSI C refer to? The original C89/C90
standard or the updated C99 one or can it be either? Is ANSI still
associated with the later standard?
CBFalconer
> On Sun, 26 Jun 2005 19:06:09 +0000, CBFalconer wrote:
>
>>
>> The following accesses a draft that include further modifications,
>> but has the disadvantage of being a .pdf file rather than a .txt
>> file. This hampers quick and easy searches, grepping, quoting,
>> etc.
>>
>> <http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf>
>>
>> Meanwhile, I think I have a suitably formatted and organized
>> version of n869.txt mounted as:
>>
>> <http://cbfalconer.home.att.net/download/n869_txt.bz2>
>>
>> which means you have to have bzip2 on your system to extract it
>> properly, but minimizes the download time (it is about 200k).
>
> Does your text version include the modifications of the pdf
> version?
No, if you mean the final issue pdf version. It cuts the
indentation down so it doesn't generate over long lines when
quoted, and strips out pagination.
--
"I'm a war president. I make decisions here in the Oval Office
in foreign policy matters with war on my mind." - GWB 2004-2-8
"If I knew then what I know today, I would still have invaded
Iraq. It was the right decision" - G.W. Bush, 2004-08-02
"This notion that the United States is getting ready to attack
Iran is simply ridiculous. And having said that, all options
are on the table." - George W. Bush, Brussels, 2005-02-22
Netocrat
> Netocrat wrote:
>> On Sun, 26 Jun 2005 19:06:09 +0000, CBFalconer wrote:
>>
> ... snip ...
>>>
>>> The following accesses a draft that include further modifications, but
>>> has the disadvantage of being a .pdf file rather than a .txt file.
>>> This hampers quick and easy searches, grepping, quoting, etc.
>>>
>>> <http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf>
>>>
>>> Meanwhile, I think I have a suitably formatted and organized version of
>>> n869.txt mounted as:
>>>
>>> <http://cbfalconer.home.att.net/download/n869_txt.bz2>
>>>
>>> which means you have to have bzip2 on your system to extract it
>>> properly, but minimizes the download time (it is about 200k).
>>
>> Does your text version include the modifications of the pdf version?
>
> No, if you mean the final issue pdf version. It cuts the indentation down
> so it doesn't generate over long lines when quoted, and strips out
> pagination.
Actually I meant the pdf you linked to which is quoted above.
Michael Wojcik
In article <3i5kq9F...@individual.net>, Nils Weller <m...@privacy.net> writes:
>
> In addition, I would argue that it is the caller's
> responsibility to pass valid arguments. Have you noticed that even in
> Unix, library-only functions tend not to set EINVAL much, and certainly
> not for invalid pointer arguments? This is because such a requirement
> would needlessly slow down and bloat correctly written code, i.e. code
> that always passes valid arguments.
So you advocate a design that substitutes premature and very likely
insignificant optimization for safety? Brilliant. It's that sort
of thinking that's given C its sterling reputation for secure
programming.
Here's a better plan: in any function that's not a small, static,
leaf function with only one caller, VALIDATE THE PARAMETERS. It
costs very, very little - if there's a performance impact, the
function call itself must be in the inner loop, so you have other
design issues to examine.
> It is usually functions invoking
> system calls that set errno to EINVAL because the operating system
> kernel *has to* check all arguments for correctness, so as to avoid
> accidental or malicious system corruption.
In typical modern OSes, the kernel "has to check all arguments" as
a side effect of the context switch. EINVAL comes along for free.
> I will also point out that your check is incomplete anyway because
> ``obj'' may contain an invalid address that isn't NULL;
It doesn't verify that the user isn't a moron, either. When C
provides a magic API for validating data against all improper uses,
I'm sure many of us will adopt it. (Some, of course, will continue
to complain that it causes "bloat".)
The point of defensive programming is preventing the problems you
can reasonably prevent. Throwing up your hands because there are
some you cannot prevent is a foolish response.
--
Michael Wojcik michael...@microfocus.com
The way things were, were the way things were, and they stayed that way
because they had always been that way. -- Jon Osborne
CBFalconer
>
... snip ...
>
> Here's a better plan: in any function that's not a small, static,
> leaf function with only one caller, VALIDATE THE PARAMETERS. It
> costs very, very little - if there's a performance impact, the
> function call itself must be in the inner loop, so you have other
> design issues to examine.
In my book, after validation, if you can put a sensible
interpretation on an invalid parameter, do so rather than causing a
crash. My basic example is interpreting a NULL pointer as pointing
to an empty source string. Obviously this doesn't apply to a
destination. My version of strlcpy and strlcat does this.
<http://cbfalconer.home.att.net/download/strlcpy.zip>
Corollary: If a function return has a meaning, don't alter that
meaning for some peculiar set of input values. In the above,
strlcpy/cat always return the space required for a successful
action, irrespective of actual success.
--
"A man who is right every time is not likely to do very much."
-- Francis Crick, co-discover of DNA
"There is nothing more amazing than stupidity in action."
-- Thomas Matthews
Nils Weller
>
> In article <3i5kq9F...@individual.net>, Nils Weller <m...@privacy.net> writes:
>>
>> In addition, I would argue that it is the caller's
>> responsibility to pass valid arguments. Have you noticed that even in
>> Unix, library-only functions tend not to set EINVAL much, and certainly
>> not for invalid pointer arguments? This is because such a requirement
>> would needlessly slow down and bloat correctly written code, i.e. code
>> that always passes valid arguments.
>
> So you advocate a design that substitutes premature and very likely
> insignificant optimization for safety? Brilliant. It's that sort
> of thinking that's given C its sterling reputation for secure
> programming.
I realize that this is a very religious topic involving very strong
feelings (as evidenced by the tone of the paragraph above), and I do not
intend to change anyone's coding style.
It is my opinion that the check under debate does more harm than good
because it does not only needlessly slow down and bloat correctly
written code, but it also makes it harder to detect and handle the bug.
The errno construct is particularly bad to force the problem upon the
callers attention because a caller is unlikely to check whether the
function failed because of an invalid argument; After all, the function
would not have been called had the arguments not been believed to be
valid. The bug is likely to be masked. On the other hand, if the caller
does examine errno, it could just as well have been more careful to
avoid the bug in the first place, making the check within the function
superfluous (it would make more sense to use assert() or similar.)
If you frequently encounter a situation where you erroneously pass a
null pointer to a function, then perhaps it is time to ask yourself why
you could ever lose track of a property as essential as whether or not
your pointer points to a valid object, or perhaps why you cannot write a
few allocator and deallocator functions more carefully rather than
cluttering all other functions with needless argument validations.
> Here's a better plan: in any function that's not a small, static,
> leaf function with only one caller, VALIDATE THE PARAMETERS. It
> costs very, very little - if there's a performance impact, the
> function call itself must be in the inner loop, so you have other
> design issues to examine.
Some validations make sense, others do not. My point of view is that in
cases like the one we are discussing, the validation results in either
an incomplete and suboptimal solution to a dubious and rare problem, or
duplicate and unneeded work. You are free to disagree.
>> It is usually functions invoking
>> system calls that set errno to EINVAL because the operating system
>> kernel *has to* check all arguments for correctness, so as to avoid
>> accidental or malicious system corruption.
>
> In typical modern OSes, the kernel "has to check all arguments" as
> a side effect of the context switch. EINVAL comes along for free.
This statement doesn't make any sense to me. First of all, you probably
mean a ``mode switch'' rather than a ``context switch''. Second, the
arguments are validated when the system call itself accesses (or tries
to access) the data. And this is because a protection boundary is
crossed and so the kernel has to check all arguments for correctness, so
as to avoid accidental or malicious system corruption, which is what I
said above. It is a deliberate and conscious decision to treat the
program as an enemy rather than a trusted partner, and nothing comes for
free.
> The point of defensive programming is preventing the problems you
> can reasonably prevent. Throwing up your hands because there are
> some you cannot prevent is a foolish response.
Yawn.
--
Nils R. Weller, Bremen / Germany
My real email address is ``nils<at>gnulinux<dot>nl''
... but I'm not speaking for the Software Libre Foundation!
Michael Wojcik
In article <3ij1raF...@individual.net>, Nils Weller <m...@privacy.net> writes:
> In article <d9s7o...@news1.newsguy.com>, Michael Wojcik wrote:
> > In article <3i5kq9F...@individual.net>, Nils Weller <m...@privacy.net> writes:
> >>
> >> In addition, I would argue that it is the caller's
> >> responsibility to pass valid arguments. Have you noticed that even in
> >> Unix, library-only functions tend not to set EINVAL much, and certainly
> >> not for invalid pointer arguments? This is because such a requirement
> >> would needlessly slow down and bloat correctly written code, i.e. code
> >> that always passes valid arguments.
> >
> > So you advocate a design that substitutes premature and very likely
> > insignificant optimization for safety? Brilliant. It's that sort
> > of thinking that's given C its sterling reputation for secure
> > programming.
>
> because it does not only needlessly slow down and bloat correctly
> written code,
It does nothing "needlessly". The need is patently obvious from the
miserable security state of a vast number of C programs.
In the vast majority of cases, the increase in execution time and
program size you complain about are negligible.
> but it also makes it harder to detect and handle the bug.
Nonsense. One possible consequence of undefined behavior is that
the program works correctly anyway, or appears to. This is never
an issue when an error is detected by correctly-written error
handling. Error detection is only harder with error handling if
the probability of getting the error handling seriously wrong is
greater than the probability of the undefined behavior being
inobvious. If those odds are against error detection, I suggest
you find a better programmer to write your error-handling code.
> The errno construct is particularly bad ...
I didn't advocate using errno; that was another poster. I think
overloading errno for application error signalling is a poor plan.
> callers attention because a caller is unlikely to check whether the
> function failed because of an invalid argument;
Certainly, if the caller was written by someone incompetent. If
we're going to assume that, though, then your assumption that the
caller provided valid arguments looks a bit shaky, doesn't it?
> After all, the function
> would not have been called had the arguments not been believed to be
> valid.
This is one of the silliest arguments I've heard in some time.
Programs do not "believe" anything. Programmers may believe they
have written correct code; they are often wrong.
> The bug is likely to be masked. On the other hand, if the caller
> does examine errno, it could just as well have been more careful to
> avoid the bug in the first place, making the check within the function
> superfluous (it would make more sense to use assert() or similar.)
I've explained why I believe assert() is useless in other threads;
I won't reiterate that now. Suffice it to say that I find this line
of argument completely unpersuasive, too.
> If you frequently encounter a situation where you erroneously pass a
> null pointer to a function,
I do not reserve error handling for conditions I believe will be
frequent.
> > Here's a better plan: in any function that's not a small, static,
> > leaf function with only one caller, VALIDATE THE PARAMETERS. It
> > costs very, very little - if there's a performance impact, the
> > function call itself must be in the inner loop, so you have other
> > design issues to examine.
>
> Some validations make sense, others do not.
A vapid generalization. Some ideas apply to some situations. So?
> >> It is usually functions invoking
> >> system calls that set errno to EINVAL because the operating system
> >> kernel *has to* check all arguments for correctness, so as to avoid
> >> accidental or malicious system corruption.
> >
> > In typical modern OSes, the kernel "has to check all arguments" as
> > a side effect of the context switch. EINVAL comes along for free.
>
> This statement doesn't make any sense to me. First of all, you probably
> mean a ``mode switch'' rather than a ``context switch''.
I meant just what I wrote. A switch between user mode and kernel
mode is a context switch. This usage is well-established.
> Second, the
> arguments are validated when the system call itself accesses (or tries
> to access) the data. And this is because a protection boundary is
> crossed and so the kernel has to check all arguments for correctness, so
> as to avoid accidental or malicious system corruption, which is what I
> said above.
In the traditional SysV Unix implementation, for example, the kernel
has a different page mapping from each user process, for obvious
reasons; and when a user process makes a system call, the parameters
must be remapped. If any of them point to pages which are not mapped
for that process, the VMM traps the access. See Bach, _The Design of
the UNIX Operating System_, 6.4.2. The detection of invalid pointers
happens automatically by the memory management hardware during the
context switch process.
True, some OSes do make explicit checks (I see the Linux 2.4 kernel
does). Others do not. Your generalization was no more correct than
mine was, it appears.
> > The point of defensive programming is preventing the problems you
> > can reasonably prevent. Throwing up your hands because there are
> > some you cannot prevent is a foolish response.
>
> Yawn.
Those who don't care about security are doomed to lose it. Remind me
never to use any software you've written.
--
Michael Wojcik michael...@microfocus.com
Against all odds, over a noisy telephone line, tapped by the tax authorities
and the secret police, Alice will happily attempt, with someone she doesn't
trust, whom she can't hear clearly, and who is probably someone else, to
fiddle her tax return and to organise a coup d'etat, while at the same time
minimising the cost of the phone call. -- John Gordon
Nils Weller
> [...]
>> The errno construct is particularly bad ...
>
> I didn't advocate using errno; that was another poster. I think
> overloading errno for application error signalling is a poor plan.
And I specifically objected to the errno construct used by said other
poster. And then you objected to my objection ... I think we are getting
dragged into an unnecessary discussion about a style issue, and there is
no enlightenment whatsoever for anyone following this thread because
it's just too pointless (yes, I should have known better than to start
talking about a religious topic.)
> [...]
> Those who don't care about security are doomed to lose it. Remind me
> never to use any software you've written.
I think you need to cool down a bit. Nothing in programming is either
good or bad; some things just suck less than others, and there are
always tradeoffs. I believe that being careful to pass valid arguments
and omitting the checks will pay off in the long run - through reduced
(source) code size and perhaps a little more speed, and in addition,
that the errno construct in particular is likely to mask bugs.
Furthermore, it does not even make sense to say that such argument
validation in case of a bug does actually buy any security because the
definition of ``security'' depends upon the context of the application.
When writing a desktop or server application, an invalid pointer
argument will probably result in a crash, which is good in some ways
because it makes the bug obvious and helps you find and fix it.
On the other hand, if you're writing control software for a spaceship,
or anything else whose survival is ``mission-critical'', then you would
of course prefer sophisticated error detection and handling over a crash
But then you proably wouldn't be using C either, and there would be a
whole range of other potential problems to be handled. The choice of an
argument validation policy and how you can react to errors is not a
choice of the Dark Side vs. the Bright Side, but of whatever makes the
most sense in your circumstances.
Note that I'm not even arguing against argument validation in general,
I just prefer to omit it in cases like the one we are discussing, and I
think that the standard C library, including all of string.h, is giving
a good example to follow for library functions in general - If you pass
an invalid argument, you get undefined behavior.
> It does nothing "needlessly". The need is patently obvious from the
> miserable security state of a vast number of C programs.
So your point is that there are many sloppy programmers who may use the
software, and that you would rather give them an error code than a
crash. This is fine with me, but it doesn't mean that coding with this
purpose in mind results in good software design, nor does it mean that
your choice of policy is better than ``mine'' (again, my comments were
restricted to a very specific case.) While there is a vast number of
broken C programs out there, there is also a vast number of well-working
(though certainly not completely bug-free) C programs out there that
never pass invalid arguments to library functions.
Do HP-UX, AIX and UnixWare with page zero mapped make your software
``more secure'' than, say, Linux or FreeBSD with page zero unmapped,
only because they will permit a program that incorrectly dereferences a
null pointer for reading to continue executing?
> In the vast majority of cases, the increase in execution time and
> program size you complain about are negligible.
Negligible but still unnecessary if the caller is written correctly. A
caller should take full responsibility to invoke the callee correctly.
This will reduce (source) code size and it may help you enforce a design
where the state of your objects is always well-known.
There are certainly times when this rule of thumb should be lifted
because an invalid invocation seems more likely, but low-level library
functions do not belong into this category, and bad null pointer
arguments should probably never be expected (though there are exceptions
where it makes sense to explicitly define, or overload the meaning of a
null pointer - see free() and fflush() for examples.)
>> but it also makes it harder to detect and handle the bug.
>
> Nonsense. One possible consequence of undefined behavior is that
> the program works correctly anyway, or appears to. [...]
The errno construct I objected to, and which triggered this pointless
subthread, requires explicit interaction from the caller. A dereferenced
null pointer will yield a crash on a vast number of implementations.
There are platforms where this is not the case, but perhaps that just
means that you should not be doing software development on them if you
can avoid them?
Don't get me wrong, I'm not saying ``let's trade program robustness for
lower code size and programming efforts!'', but something more along the
lines of ``this simply does not affect robustness in a well-written
application, and even if the bug does occur some time, then it is
questionable whether the `defensive' approach does actually save the
day.''
>> callers attention because a caller is unlikely to check whether the
>> function failed because of an invalid argument;
>
> Certainly, if the caller was written by someone incompetent. If
> we're going to assume that, though, then your assumption that the
> caller provided valid arguments looks a bit shaky, doesn't it?
When was the last time you actually adhered to what you are proposing
here? If you know what you're doing, then you also know with reasonable
certainty whether or not you are invoking a function correctly at any
point in time. Programming is no lottery and you always have to make
essential assumptions about the integrity of your program.
Many Unix functions set a wide variety of errno codes, but it is
impractical and nonsensical to test for all of them.
char buf[128];
int rc;
int fd;
/* ... */
rc = read(buf, sizeof buf - 1, fd);
if (rc == -1) {
/*
* Would you really test for:
* EBADF (bad file descriptor)
* EFAULT (bad address)
* EINVAL (bad STREAM or multiplexer)
* ... and a variety of other obscure
* errors that you *know* cannot occur
* in the particular program context?
*/
} else if (rc == 0) {
/* EOF */
} else {
buf[rc] = 0;
/* Use buf */
}
If you ever find yourself writing something like
if (errno == EFAULT) {
... then you should reassure yourself that the pointer can *never* be
invalid and solve the actual problem, rather than coding around it.
>> After all, the function
>> would not have been called had the arguments not been believed to be
>> valid.
>
> This is one of the silliest arguments I've heard in some time.
> Programs do not "believe" anything. Programmers may believe they
> have written correct code; they are often wrong.
See the example above.
This whole thing really boils down to the question of whether or not
compatibility with buggy code is desirable. Programmers may also be able
to get their implementation to enable them to dereference null pointers,
to emulate misaligned instructions, and to make string constants
writable. That doesn't mean that any development relying on these
features will result in stable and well-designed software.
> the UNIX Operating System_, 6.4.2. The detection of invalid pointers
> happens automatically by the memory management hardware during the
> context switch process.
The kernel is also mapped into the process's address space, yet it
should not be accessed through a userland pointer, which is why address
translation does not suffice to ensure validity.
> True, some OSes do make explicit checks (I see the Linux 2.4 kernel
> does). Others do not. Your generalization was no more correct than
> mine was, it appears.
I don't think I made any generalizations; While the pointer stuff is
indeed for the most part covered by the hardware, all other arguments
also need to be checked explicitly. For example, an integer you pass to
the kernel may or may not be a valid file descriptor, but the kernel
cannot take its validity for granted.
Nils Weller
> rc = read(buf, sizeof buf - 1, fd);
Of course I had to goof this one!
rc = read(fd, buf, sizeof buf - 1);
CBFalconer
> Nils Weller wrote:
>
>> rc = read(buf, sizeof buf - 1, fd);
>
> Of course I had to goof this one!
>
> rc = read(fd, buf, sizeof buf - 1);
I have no idea what it goes with, because your previous article was
much too long to read. :-) However, you have still goofed, because
there is no such standard function as 'read'. Look up fread, which
IS portable.
Nils Weller
> Nils Weller wrote:
>> Nils Weller wrote:
>>
>>> rc = read(buf, sizeof buf - 1, fd);
>>
>> Of course I had to goof this one!
>>
>> rc = read(fd, buf, sizeof buf - 1);
>
> I have no idea what it goes with, because your previous article was
> much too long to read. :-) However, you have still goofed, because
> there is no such standard function as 'read'. Look up fread, which
> IS portable.
And nobody claimed that read() is a standard C function. I explicitly
commented the code as being Unix-specific in the previous, too long
post. Moreover, the macro that triggered this sub-thread has also been
pointed out to be Unix-specific, and there has been some talk about Unix
kernel implementation and compatibility system software.
Perhaps an OT tag was missing, but I think it is clear that we aren't
talking about standard C anymore.
Dave Thompson
<clar...@gmail.com> wrote:
> On 2005-06-25 11:45:13 -0400, Netocrat <neto...@dodo.com.au> said:
<snip>
> > I believe that there is no portable, generic way to copy a structure
>
> Of course there is; in fact, there are several:
>
> Assuming a and b are of the same complete type, any of the following
> will copy the contents of a into b:
>
> #include <stdlib.h>
Not actually needed for anything in this code. (size_t is in string.h)
> #include <string.h>
>
> /*1*/ b = a;
For complete _nonarray_ types.
> /*2*/ memcpy(&b, &a, sizeof b);
> /*3*/ memmove(&b, &a, sizeof b);
> /*4*/ const unsigned char *src = (const unsigned char*)&a;
> unsigned char *dst = (unsigned char*)&b;
> for(size_t i=0; i<sizeof b; ++i)
> {
> dst[i] = src[i];
> }
Rest for all complete types. And if you can determine the (a?) size by
some other means not sizeof, even objects declared-not-defined with
incomplete types.
- David.Thompson1 at worldnet.att.net
Dave Thompson
> >On Sat, 25 Jun 2005 18:31:30 +0000, Chris Torek wrote:
> >> (you can also write the loop as "while (n--) *dst++ = *src++" but I find
> >> the above easier to read and think about).
>
> In article <pan.2005.06.25....@dodo.com.au>
> Netocrat <neto...@dodo.com.au> wrote:
> >I prefer the conciseness of the second, but I prefer even more testing
> >against a maximum pointer.
>
> My thinking is perhaps colored by too many years of assembly coding
> and instruction sets that include "decrement and branch if nonzero":
>
> test r3
> bz Laround
> Lloop:
> mov (r1)+,(r2)+
> sobgtr r3,Lloop # cheating (but this is OK)
> Laround:
<snip>
> and so on. (The first loop is VAX assembly, and "cheating" is OK
> because r1 and/or r2 should never cross from P0/P1 space to S space,
> nor vice versa, so the maximum block size never exceeds 2 GB; <snip>
Not movb? Isn't the default word=long? Or is this some overambitious
assembler that you (have to) tell about value types?
Most (I think all but first two or so) models of PDP-11 also had
sub-1-brback-ne (only) which they managed to publish as SOB before
marketing caught them. PDP-6/10 already had a whole series of SOB*,
but only SOBN or SOBG would do what you wanted here not SOB.
(All 16 dyadic booleans are implemented, but SKIP doesn't; JUMP
doesn't; the fastest jump varies but is never JUMP*; etc., etc.)
ISTR 68k, which you also mentioned (snipped), also had a mildly
offcolor opcode, somewhere else.
- David.Thompson1 at worldnet.att.net
Dave Thompson
wrote:
<snip>
> The void * type can point at arbitrary things, and a size_t can
> specify a size on any machine. But to use void* you have to
> convert to other types, thus:
>
> void *dupmem(void *src, size_t sz)
> {
> unsigned char *sp = src;
> unsigned char *dst;
>
> if (dst = malloc(sz)) /* memory is available */
> while (sz--) *dst++ = *sp++; /* copy away */
> return dst; /* will be NULL for failure */
return dst - sz, unless all your callers will (and must) adjust down
the pointer before using it to access the memory, and free() it.
> } /* dupmem, untested */
>
> Note how src is typed into sp, without any casts. Similarly the
> reverse typing for the return value of dupmem. The usage will be,
> for p some type of pointer:
>
Although it would be more informative, and convenient for some
call(er)s, to declare src and sp as pointer to const void/uchar.
> if (p = dupmem(whatever, howbig)) {
> /* success, carry on */
> }
> else {
> /* abject failure, panic */
> }
- David.Thompson1 at worldnet.att.net
Dave Thompson
wrote:
> Also C90 and C89 seem to be interchangeable terms - correct?
>
Effectively. C89 was the document developed "by" (under) ANSI, then
submitted to "ISO" (already JTC1?) and adopted with technically
identical contents but different numbering scheme and (I believe) some
of the boilerplate about copyright, authority, and such. Thus if you
want to refer to a clause number, as we fairly often do, you need to
specify which; and if you had a lawsuit turning on compliance to one
or the other standard you might have to produce that exact document to
support your case. But as far as what a C implementation is required
or permitted to do, and thus what a program(mer) can rely on or
expect, they are interchangeable.
In contrast C99 was voted first by "ISO" (as I understand it really
SC22), and adopted as-is by ANSI (really NCITS? INCITS?).
> Finally I understand that C90/C89 had some modifications made prior to C99
> - where are those detailed?
See FAQ 11.1 and .2 -- at least in the text version posted and online
at usual places; the webized http://www.eskimo.com/~scs/C-faq/top.html
has been out-of-date the last few times I checked and this is one of
the points that has changed. But:
- the statement about the Rationale was for only the original ANSI
version C89, which is no longer (realistically) available;
- it says Normative Addendum which I'm pretty sure should be
Amendment; C90 plus that amendment is sometimes called C95
- (several!) drafts of an updated Rationale for C99, as well as drafts
of C99 itself (through n869) and C0X (n1124) can be gotten from the WG
site which is now (renamed?) www.open-std.org/JTC1/SC22/WG14 .
(As well as other stuff you might be interested in, for that matter.)
And for your further delectation and enjoyment, you could get the
~1600-page e-book by Derek M Jones discussed in another thread, which
AFAICT-so-far exegizes the standard process, the resulting document,
and the language specified in it, and more.
If you actually want C90 instead of or in addition to C99, ANSI
apparently no longer sells it, but webstore.ansi.org (still) lists DIN
and AS adoptions of 9899:1990, and I'm guessing the latter might be
available to you more conveniently.
- David.Thompson1 at worldnet.att.net
Chris Torek
>On 25 Jun 2005 19:58:19 GMT, Chris Torek <nos...@torek.net> wrote:
>> mov (r1)+,(r2)+
In article <c6ghc1dufoi95bipm...@4ax.com>
Dave Thompson <david.t...@worldnet.att.net> wrote:
>Not movb? Isn't the default word=long? Or is this some overambitious
>assembler that you (have to) tell about value types?
No, just a goof; it should have been "movb".
>ISTR 68k, which you also mentioned (snipped), also had a mildly
>offcolor opcode, somewhere else.
I do not recall any from the 680x0 series, but the 1802 had several.
Each register was 16 bits (I am almost certain, despite the 8-bit
claim on the page referenced below), but the 8-bit opcodes could
address only the high or low half of each register, so there was
a "put low" and "put high" to write to each half, and the corresponding
pair of "get"s. This meant the 1802 had GHI, the "get high"
instruction.
The 1802 also had two special registers named P (program counter)
and X (index). However, neither P nor X were actual registers;
instead, they were register *numbers*, pointing to one of the 16
general-purpose registers. You had to use a "set p" or "set x"
instruction to point the P and X indirection at the appropriate
register. These had three-letter assembler mnemonics; the first
was SEP, and the second was the now-obvious.
(See also <http://shop-pdp.kent.edu/ashtml/as1802.htm>.)
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.
CBFalconer
> CBFalconer <cbfal...@yahoo.com> wrote:
> <snip>
>> The void * type can point at arbitrary things, and a size_t can
>> specify a size on any machine. But to use void* you have to
>> convert to other types, thus:
>>
>> void *dupmem(void *src, size_t sz)
>> {
>> unsigned char *sp = src;
>> unsigned char *dst;
>>
>> if (dst = malloc(sz)) /* memory is available */
>> while (sz--) *dst++ = *sp++; /* copy away */
>> return dst; /* will be NULL for failure */
>
> return dst - sz, unless all your callers will (and must) adjust down
> the pointer before using it to access the memory, and free() it.
That still doesn't fix my goof above. sz ends at 0. Try this:
void *dupmem(void *src, size_t sz)
{
unsigned char *sp = src;
unsigned char *dst, *p;
if (p = dst = malloc(sz)) /* memory is available */
while (sz--) *p++ = *sp++; /* copy away */
return dst; /* will be NULL for failure */
}
BGreene
zero" in many a year.
"Dave Thompson" <david.t...@worldnet.att.net> wrote in message
news:c6ghc1dufoi95bipm...@4ax.com...
Netocrat
[a memcpy function in response to my buggy version]
> void *like_memcpy(void *restrict dst0, const void *restrict src0,
> size_t n) {
> unsigned char *restrict dst = dst0;
> unsigned char *restrict src = src0;
>
> if (n)
> do
> *dst++ = *src++;
> while (--n != 0);
> return dst0;
> }
>
>>On Sat, 25 Jun 2005 18:31:30 +0000, Chris Torek wrote:
>>> (you can also write the loop as "while (n--) *dst++ = *src++" but I
>>> find the above easier to read and think about).
>
> In article <pan.2005.06.25....@dodo.com.au> Netocrat
> <neto...@dodo.com.au> wrote:
>>I prefer the conciseness of the second, but I prefer even more testing
>>against a maximum pointer.
>
> My thinking is perhaps colored by too many years of assembly coding and
> instruction sets that include "decrement and branch if nonzero":
<snip discussion to which I responded in a later post>
I was spurred to actually benchmark the different approaches on my
machine. It's a little over the top, but my belief is that it's not
really possible to predict which approach will be faster - even knowing
the machine's architecture you can't know what the compiler will do. So
to me these sort of things are really a matter of personal preference.
So here is my attempt to back up that intuition at least on my machine.
I used the function quoted above, as well as the quoted proposed
alternative, and my function as fixed by Kevin Bagust:
> void *mem_cpy( void *dest, const void *src, size_t bytes ) {
> unsigned char *destPtr = dest;
> unsigned char const *srcPtr = src;
> unsigned char const *srcEnd = srcPtr + bytes;
>
> while ( srcPtr < srcEnd ) {
> *destPtr++ = *srcPtr++;
> }
> return dest;
> }
I compiled at four of the levels of optimisation available on gcc (none,
-O1, -O2, -O3), and at each level performed two tests - with and without
-march=pentium4 (my machine architecture). I performed the tests at
multiple iterations of 0, 1, 2, 8, 25 and 80 bytes and timed the duration
using clock().
And the results?
At the unoptimised level, both of Chris's alternatives were equal.
In every other case the first of Chris's alternatives far outperformed the
second (by a minimum of 14% and maximum of 21%).
So I modified the 'alternative' expression from
while (n--) *dst++ = *src++;
to
while (n) {
*dst++ = *src++;
n--;
}
This brought the alternative function back close to the performance of the
original. I don't know why the degradation was occurring; presumably
something to do with one or more of the variables being decremented or
incremented one more time than necessary.
In the unoptimised case, my function outperformed Chris's functions by
about 15%. In all of the optimised cases, they were roughly equal -
varying from his performing 3% better than mine to mine performing 2%
better than his.
So even though it's platform-specific I think that this test shows that
choosing between these loop constructions should be based on personal
preference as to readability - a performance benefit can't be assumed for
any particular style - unless you are developing for a particular system
for which you know one style is more performant than the others.
Chris Croughton
<neto...@dodo.com.au> wrote:
> In every other case the first of Chris's alternatives far outperformed the
> second (by a minimum of 14% and maximum of 21%).
>
> So I modified the 'alternative' expression from
> while (n--) *dst++ = *src++;
> to
> while (n) {
> *dst++ = *src++;
> n--;
> }
>
> This brought the alternative function back close to the performance of the
> original. I don't know why the degradation was occurring; presumably
> something to do with one or more of the variables being decremented or
> incremented one more time than necessary.
Some odd optimisation?
Incidentally, if you still have the test code around, could you also try
while (n) {
*dst = *src;
++src;
++dst;
--n;
}
(And is there a difference between n--; and --n; on your system?)
Just to get the results from the same system as used for your original
results. (Incidentally, how did they compare with the system-supplied
memcpy? I believe gcc inlines that to assembler at some optimisation
levels...)
> So even though it's platform-specific I think that this test shows that
> choosing between these loop constructions should be based on personal
> preference as to readability - a performance benefit can't be assumed for
> any particular style - unless you are developing for a particular system
> for which you know one style is more performant than the others.
Indeed. And bear in mind that it may change completely with the next
version of the compiler, or switching to another compiler on the same
platform. I've found that trusting the compiler and library writers to
have picked the best optimisations is right most of the time...
Chris C
Netocrat
> On Sun, 10 Jul 2005 22:27:54 +1000, Netocrat
> <neto...@dodo.com.au> wrote:
>
>> In every other case the first of Chris's alternatives far outperformed
>> the second (by a minimum of 14% and maximum of 21%).
>>
>> So I modified the 'alternative' expression from
>> while (n--) *dst++ = *src++;
>> to
>> while (n) {
>> *dst++ = *src++;
>> n--;
>> }
>> This brought the alternative function back close to the performance of
>> the original. I don't know why the degradation was occurring;
>> presumably something to do with one or more of the variables being
>> decremented or incremented one more time than necessary.
>
> Some odd optimisation?
Anything's possible.
> Incidentally, if you still have the test code around, could you also try
>
> while (n) {
> *dst = *src;
> ++src;
> ++dst;
> --n;
> }
I retested and included this modification that you suggested. Your
modification is always faster than the original while(n--) loop and is
roughly the same across all of the optimisation levels as the modification
that I made (worst performance is 17% slower than my mod at -O1 - an
aberration since for all other cases their separation is a few percent -
and best performance is 5% faster at -O3 -march=pentium4).
> (And is there a difference between n--; and --n; on your system?)
I'm not sure about the general case - but I tested your modification above
with n-- and --n. There is a small variation that differs between the
optimisation levels - neither is consistently faster. The biggest
separation I found was post-decrement being about 3% faster at -O3
-march=pentium4. I repeated this test a few times to check that it wasn't
a one-off error due to system loading and the result was consistently
within the bounds of .05% and 3%. The initial 3% result is probably not
accurate but there's no doubt that in this case the compiler generates
slightly faster code for post-decrement.
> Just to get the results from the same system as used for your original
> results. (Incidentally, how did they compare with the system-supplied
> memcpy? I believe gcc inlines that to assembler at some optimisation
> levels...)
Its execution time doesn't vary between the sizes I originally tested as
much as the other functions' times do. Nor is its performance affected by
optimisation level. With or without optimisations, it is always the
slowest function for sizes of 0..8 bytes. Without optimisations, from
about 16 bytes it starts consistently performing far better - eg at 40
bytes it is 150% faster than any other function. With optimisations it's
"in the mix" - not much better or worse than the others up to roughly 40
bytes and from then on it consistently beats them.
I tested for larger sizes at all optimisation levels:
At 80 bytes the library function was a minimum of 34% faster than any
other function (340% faster when optimisation switches were not used).
At 1024 bytes it was at least 270% faster (1400% faster without
optimisations).
At 10 kilobytes it was at least 400% faster.
At 100 kilobytes it was at least 65% faster. Also optimisations changed
its performance - it was fastest without optimisations and at -O1 it was
twice as slow as without optimisations.
At 1 megabyte things had evened out and it was roughly the same as the
others and in some cases slightly slower. It performed the same at all
optimisation levels.
At 10 and 100 megabytes I only tested for -O3 -march=pentium4 and again it
was roughly the same as the other functions.
>> So even though it's platform-specific I think that this test shows that
>> choosing between these loop constructions should be based on personal
>> preference as to readability - a performance benefit can't be assumed
>> for any particular style - unless you are developing for a particular
>> system for which you know one style is more performant than the others.
>
> Indeed. And bear in mind that it may change completely with the next
> version of the compiler, or switching to another compiler on the same
> platform. I've found that trusting the compiler and library writers to
> have picked the best optimisations is right most of the time...
Agreed - and if you _really_ need specific hard-core optimisations, don't
rely on the compiler except perhaps to use its output as a base - go with
assembly. That way the results aren't dependent on things beyond your
control like compiler code-generation.