stdlib memmove() and/or reverse memcpy()?

[Rick C. Hodgin]

Is memmove() a (the??) standard C method for moving overlapping memory regions?

Is there a stdlib function that is better? Something like a reverse memcpy() which copies from the end of the buffer to the start, rather than the start to the end?

Best regards,
Rick C. Hodgin

[Richard Heathfield]

Rick C. Hodgin wrote:

> Is memmove() a (the??) standard C method for moving overlapping memory
> regions?

Yes. (The.)

>
> Is there a stdlib function that is better? Something like a reverse
> memcpy() which copies from the end of the buffer to the start, rather than
> the start to the end?

memmove is generally very heavily optimised, so don't feel bad about using
it. Unless C has changed drastically while I've been feeding the
metaphorical cat, there is no other function in the standard C library that
does precisely the same job as memmove.

--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[Malcolm McLean]

On Tuesday, September 2, 2014 8:31:15 PM UTC+1, Rick C. Hodgin wrote:
> Is memmove() a (the??) standard C method for moving overlapping memory
> regions?
>
> Is there a stdlib function that is better? Something like a reverse
> memcpy() which copies from the end of the buffer to the start, rather
> than the start to the end?
>

memmove for possibly overlapping copies. You can work out the algorithm,
you copy from source upwards if destination is less than source, and
from source + N -1 downwards if destination is greater than source.

memcpy saves a test. However often it's just an alias for memmove.
If by reverse you mean reverse byte order, you've got to write that
yourself.

[Geoff]

On Tue, 2 Sep 2014 12:31:04 -0700 (PDT), "Rick C. Hodgin"
<rick.c...@gmail.com> wrote:

>Is memmove() a (the??) standard C method for moving
>overlapping memory regions?

Yes, it is the standard method. The standard requires that memmove
behave _as_if_ the source region were copied to a non-overlapping
region of memory before copying into the destination region.

The only danger in memmove is if the size of the destination region is
smaller than the number of bytes you are trying to move. You are
responsible for ensuring the destination is large enough.

>Is there a stdlib function that is better?
>Something like a reverse memcpy() which copies from
>the end of the buffer to the start, rather than the
>start to the end?
>

No.

The C standard does not define the behavior of memcpy when the regions
to be copied overlap.

[Barry Schwarz]

On Tue, 02 Sep 2014 18:10:01 -0700, Geoff <ge...@invalid.invalid>
wrote:

That would be unspecified behavior. In this case, the standard
specifically states that the behavior is undefined.

--
Remove del for email

[Kaz Kylheku]

I suspect you're right; very sharp. The procedures for copying one memory block
to another, by possible permutations of transfer units of various possible
sizes, are enumerable, and finite in number. For each of these, we can
determine the final bytes that end up in the destination object. Therfore the
lack of a specification of behavior for the overlapping case would in fact mean
that it is unspecified behavior. It would not be conforming to terminate
the program with a diagnostic message, for instance.

[Geoff]

On Tue, 02 Sep 2014 20:48:26 -0700, Barry Schwarz <schw...@dqel.com>
wrote:

>That would be unspecified behavior. In this case, the standard
>specifically states that the behavior is undefined.
>

If that's what the standard meant to say then that's what it should
have said. I used the term the standard used.

7.23.2.1 The memcpy function

Synopsis

1 #include <string.h>

void *memcpy(void * restrict s1, const void * restrict s2, size_t n);

Description

2 The memcpy function copies n characters from the object pointed to
by s2 into the object pointed to by s1. If copying takes place between
objects that overlap, the behavior is undefined.

Returns

3 The memcpy function returns the value of s1.

[Richard Heathfield]

Geoff wrote:

> On Tue, 02 Sep 2014 20:48:26 -0700, Barry Schwarz <schw...@dqel.com>
> wrote:
>
>>That would be unspecified behavior. In this case, the standard
>>specifically states that the behavior is undefined.
>>
>
> If that's what the standard meant to say then that's what it should
> have said. I used the term the standard used.

Geoff, if I understand you correctly (and I think I do), you said that the
Standard didn't define the behaviour, basing your observation on the
standard's wording, "the behaviour is undefined". I fail to see your logic.
It is obvious to me that "the behaviour is undefined" is to be interpreted
as "the behaviour is unspecified". I don't see any justification whatsoever
for interpreting "the behaviour is undefined" as "the behaviour is
undefined". This would be as bad as interpreting "shall" as "shall" or
"normative" as "normative", and how ridiculous is that! Please be more
careful next time.

[Geoff]

On Wed, 03 Sep 2014 07:22:05 +0100, Richard Heathfield
<inv...@see.sig.invalid> wrote:

>Geoff wrote:
>
>> On Tue, 02 Sep 2014 20:48:26 -0700, Barry Schwarz <schw...@dqel.com>
>> wrote:
>>
>>>That would be unspecified behavior. In this case, the standard
>>>specifically states that the behavior is undefined.
>>>
>>
>> If that's what the standard meant to say then that's what it should
>> have said. I used the term the standard used.
>
>Geoff, if I understand you correctly (and I think I do), you said that the
>Standard didn't define the behaviour, basing your observation on the
>standard's wording, "the behaviour is undefined". I fail to see your logic.
>It is obvious to me that "the behaviour is undefined" is to be interpreted
>as "the behaviour is unspecified". I don't see any justification whatsoever
>for interpreting "the behaviour is undefined" as "the behaviour is
>undefined". This would be as bad as interpreting "shall" as "shall" or
>"normative" as "normative", and how ridiculous is that! Please be more
>careful next time.

Thank you Richard, I will.

[James Kuyper]

On 09/02/2014 11:48 PM, Barry Schwarz wrote:
> On Tue, 02 Sep 2014 18:10:01 -0700, Geoff <ge...@invalid.invalid>
> wrote:

...

>> The C standard does not define the behavior of memcpy when the regions
>> to be copied overlap.
>

> That would be unspecified behavior. ...

How do you reach that conclusion?

> ... In this case, the standard

> specifically states that the behavior is undefined.

I agree that "the [C] standard specifically states that the behavior is
undefined.". It follows that "The C standard does not define the
behavior". The reverse argument would not hold up, of course:

"If a ‘‘shall’’ or ‘‘shall not’’ requirement that appears outside of a
constraint or runtimeconstraint is violated, the behavior is undefined.
Undefined behavior is otherwise indicated in this International Standard
by the words ‘‘undefined behavior’’ or by the omission of any explicit
definition of behavior. There is no difference in emphasis among these
three; they all describe ‘‘behavior that is undefined’’". (4p4).

Geoff's wording could be taken as implying that the behavior was
undefined by "omission of any explicit definition of behavior", whereas
it's actually undefined because the standard explicitly says so. But
that implication is far from clear - his wording is consistent with any
of the three reasons. In any event, "there is no difference in emphasis"
between those three.
--
James Kuyper

[Keith Thompson]

Barry Schwarz <schw...@dqel.com> writes:
> On Tue, 02 Sep 2014 18:10:01 -0700, Geoff <ge...@invalid.invalid>
> wrote:

[...]

>>The C standard does not define the behavior of memcpy when the regions
>>to be copied overlap.
>
> That would be unspecified behavior. In this case, the standard
> specifically states that the behavior is undefined.

Yes, the standard specifically states that the behavior is undefined,
but it would be undefined behavior either way.

N1570 4p2:

If a "shall" or "shall not" requirement that appears outside
of a constraint or runtimeconstraint is violated, the behavior
is undefined. Undefined behavior is otherwise indicated in this
International Standard by the words "undefined behavior" or by
the omission of any explicit definition of behavior. There is
no difference in emphasis among these three; they all describe
"behavior that is undefined".

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[Tim Rentsch]

Keith Thompson <ks...@mib.org> writes:

> Barry Schwarz <schw...@dqel.com> writes:
>> On Tue, 02 Sep 2014 18:10:01 -0700, Geoff <ge...@invalid.invalid>
>> wrote:
> [...]
>>>The C standard does not define the behavior of memcpy when the
>>>regions to be copied overlap.
>>
>> That would be unspecified behavior. In this case, the standard
>> specifically states that the behavior is undefined.
>
> Yes, the standard specifically states that the behavior is
> undefined, but it would be undefined behavior either way.
>
> N1570 4p2:
>
> If a "shall" or "shall not" requirement that appears outside
> of a constraint or runtimeconstraint is violated, the behavior
> is undefined. Undefined behavior is otherwise indicated in this
> International Standard by the words "undefined behavior" or by
> the omission of any explicit definition of behavior. There is
> no difference in emphasis among these three; they all describe
> "behavior that is undefined".

If the explicit statement of undefined behavior were not there,
why do you think the behavior would be undefined? The
description of memcpy() does give an explicit definition for the
behavior. That definition is not specific enough for the
behavior to be uniquely determined, but it is a definition, so
concluding that the behavior is unspecified seems reasonable. Or
is there a "shall" requirement that you think is violated?

[Tim Rentsch]

Barry Schwarz <schw...@dqel.com> writes:

> [for memcpy() when there is overlap between the operands,
> would the behavior still be undefined if the explict
> statement of undefined behavior were not present?]

>
> That would be unspecified behavior. In this case, the standard
> specifically states that the behavior is undefined.

I won't disagree, but there is another point to be considered
here. In C90 I believe this conclusion is right - without the
explicit statement of undefined behavior the result would be
unspecified behavior. However, in C99/C2011, the specification
for memcpy() includes 'restrict' qualifiers on the parameters. A
case could be made that the rules for 'restrict' suffice to show
undefined behavior when the operands overlap, and the statement
of explicit undefined behavior is just a carryover from C90 which
did not have 'restrict'. The case is not airtight since what it
means for standard library function prototypes to use 'restrict'
is AFAICT never spelled out in the Standard, but it seems
reasonable to infer that 'restrict' in such cases means the
corresponding function may be or will be defined with the same
'restrict' qualifiers. If that is so then we may deduce that
overlapping operands result in undefined behavior without needing
an explicit statement to that effect.

In any case, good observation on the original point.

[Keith Thompson]

For clarity, I'll quote N1570 7.24.2.1:

The memcpy function copies n characters from the object pointed
to by s2 into the object pointed to by s1. If copying takes
place between objects that overlap, the behavior is undefined.

Backing off a bit from my previous statement, I'd say that if that
last sentence were omitted, the description would be very unclear.
If the objects overlap, it woudn't preclude copying the characters in
random order (yielding meaningless results), but it would, I think,
preclude reading characters outside the bounds of the source object
or writing characters outside the bounds of the target object.

Without that sentence, the standard would have been, in my opinion,
in need of improvement to clarify what's required for overlapping
objects. Fortunately, that was never an issue.

[Tim Rentsch]

It seems clear enough to me. But I agree it allows so much
latitude that for all practical purposes memcpy() would be
useless in such cases.

[glen herrmannsfeldt]

Tim Rentsch <t...@alumni.caltech.edu> wrote:
> Keith Thompson <ks...@mib.org> writes:

(snip)

>> For clarity, I'll quote N1570 7.24.2.1:

>> The memcpy function copies n characters from the object pointed
>> to by s2 into the object pointed to by s1. If copying takes
>> place between objects that overlap, the behavior is undefined.

>> Backing off a bit from my previous statement, I'd say that if that
>> last sentence were omitted, the description would be very unclear.

Yes, but other last sentences could have been used.

>> If the objects overlap, it woudn't preclude copying the characters in
>> random order (yielding meaningless results), but it would, I think,
>> preclude reading characters outside the bounds of the source object
>> or writing characters outside the bounds of the target object.

> It seems clear enough to me. But I agree it allows so much
> latitude that for all practical purposes memcpy() would be
> useless in such cases.

Since C doesn't have array expressions, this doens't normally
come up. PL/I defined array expressions like:

X=X+X(10);

such that elements are assigned one by one, in memory order, and
new values are used immediately.

Fortran, when array expressions were added, defined:

X=X+X(10)

such that the effect is as if the whole right hand side was
evaluated before the assignment was done. That is, the old X(10)
is used for all elements.

Seems to me that either could have left it undefined.

-- glen