cp/mv and extended attributes preservation

[Emmanuel Dreyfus]

Hello

Currently, our cp(1) will never preserve extended attributes, and mv(1)
will only do so across the same filesystem (that is, when it can use
rename(2)). While I found no specification that it should be done, it
seems Linux and Darwin do that.

Any objection to modify cp(1) and mv(1) to preserve extended attributes?

--
Emmanuel Dreyfus
ma...@netbsd.org

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de

[David Young]

On Tue, Jul 26, 2011 at 12:26:30PM +0000, Emmanuel Dreyfus wrote:
> Any objection to modify cp(1) and mv(1) to preserve extended attributes?

How much code will that take?

Dave

--
David Young OJC Technologies
dyo...@ojctech.com Urbana, IL * (217) 344-0444 x24

[Emmanuel Dreyfus]

On Tue, Jul 26, 2011 at 12:26:30PM +0000, Emmanuel Dreyfus wrote:

> Currently, our cp(1) will never preserve extended attributes, and mv(1)
> will only do so across the same filesystem (that is, when it can use
> rename(2)). While I found no specification that it should be done, it
> seems Linux and Darwin do that.
>
> Any objection to modify cp(1) and mv(1) to preserve extended attributes?

Attached is a patch that does it. I am concerned that with multi-file
cp or mv, it adds a system call for each copied file, even if it has
no extended attributes. I suspect the overhead is negligible compared
to time spent in I/O, but if someone sees a better way of doing it...

--
Emmanuel Dreyfus
ma...@netbsd.org

[Joerg Sonnenberger]

On Tue, Jul 26, 2011 at 12:26:30PM +0000, Emmanuel Dreyfus wrote:

> Currently, our cp(1) will never preserve extended attributes, and mv(1)
> will only do so across the same filesystem (that is, when it can use
> rename(2)). While I found no specification that it should be done, it
> seems Linux and Darwin do that.
>
> Any objection to modify cp(1) and mv(1) to preserve extended attributes?

Please do that only if -p is active (for cp(1)).

Joerg

[Simon Burge]

If you do this, can a single copy of copy_file_xattr() be used, and
referenced from the other program?

Cheers,
Simon.

[Emmanuel Dreyfus]

On Wed, Jul 27, 2011 at 10:09:50AM +1000, Simon Burge wrote:
> If you do this, can a single copy of copy_file_xattr() be used, and
> referenced from the other program?

I would be nice to have it in libc for everybody use. We could hvbe:
int extattr_copy_file(int namespace, const char *from, const char *to);,
int extattr_copy_link(int namespace, const char *from, const char *to);
int extattr_copy_fd(int namespace, int from_fd, int to_fd);

OTOH, the FreeBSD derived API is ugly enough, we can decide to
depreciate it and go with the Linux style API, with something like:
int cpxattr(const char *from, const char *to);
int lcpxattr(const char *from, const char *to);
int fcpxattr(int from_fd, int to_fd);

The latter would copy all known namespaces. But perhaps you were just
suggesting that the function should move to a standalone src/bin/mv/utils.c
and have src/bin/cp/utils.c doing @ #include "../mv/utils.c" ?

--
Emmanuel Dreyfus
ma...@netbsd.org

[Emmanuel Dreyfus]

On Tue, Jul 26, 2011 at 10:14:42PM +0200, Joerg Sonnenberger wrote:
> > Any objection to modify cp(1) and mv(1) to preserve extended attributes?
> Please do that only if -p is active (for cp(1)).

While it seems desirable performance-wse, it introduce an odd difference
between cp and mv. And other OSes do preserve EA by default, so I
think it breaks the rule of least astonishment.

--
Emmanuel Dreyfus
ma...@netbsd.org

[Simon Burge]

Emmanuel Dreyfus wrote:

> ... But perhaps you were just

> suggesting that the function should move to a standalone src/bin/mv/utils.c
> and have src/bin/cp/utils.c doing @ #include "../mv/utils.c" ?

This is what I meant (using .PATH in one makefile to pull in whatever.c
from the other directory). That said, I wouldn't be against putting it
in a library. Would libutil be more appropriate though?

Cheers,
Simon.

[Mouse]

> The only problem with putting it in a library is that we will need to
> decide what API we [w]ant to promote: the ugly one from FreeBSD or
> the [one] from Linux.

Or we could invent our own.

All the cool ki^W^Wother OSes are doing it.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mo...@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

[Emmanuel Dreyfus]

On Wed, Jul 27, 2011 at 11:54:37AM +1000, Simon Burge wrote:
> This is what I meant (using .PATH in one makefile to pull in whatever.c
> from the other directory). That said, I wouldn't be against putting it
> in a library. Would libutil be more appropriate though?

The only problem with putting it in a library is that we will need to
decide what API we cant to promote: the ugly one from FreeBSD or the only
from Linux.

--
Emmanuel Dreyfus
ma...@netbsd.org

[Emmanuel Dreyfus]

On Tue, Jul 26, 2011 at 11:30:06PM -0400, Mouse wrote:
> > The only problem with putting it in a library is that we will need to

> > decide what API we [w]ant to promote: the ugly one from FreeBSD or
> > the [one] from Linux.
>
> Or we could invent our own.
> All the cool ki^W^Wother OSes are doing it.

This is because specification did not tell about API prototypes, so
everyone has its own solution. Linux has this (we fully implement it):
getxattr lgetxattr fgetxattr
setxattr lsetxattr fsetxattr
listxattr llistxattr flistxattr
removexattr lremovexattr fremovexattr

Darwin has nearly the same (no l* function but there is an option flag
to follow links or not).

FreeBSD has the thing we imported. IRIX has another API, and so on.

[Joerg Sonnenberger]

On Wed, Jul 27, 2011 at 01:15:44AM +0000, Emmanuel Dreyfus wrote:
> On Tue, Jul 26, 2011 at 10:14:42PM +0200, Joerg Sonnenberger wrote:
> > > Any objection to modify cp(1) and mv(1) to preserve extended attributes?
> > Please do that only if -p is active (for cp(1)).
>
> While it seems desirable performance-wse, it introduce an odd difference
> between cp and mv. And other OSes do preserve EA by default, so I
> think it breaks the rule of least astonishment.

I classify EAs the same way as suid/sgid bits. Also it seems like e.g.
coreutils doesn't copy them by default.

Joerg

[Emmanuel Dreyfus]

On Wed, Jul 27, 2011 at 07:21:00AM +0200, Joerg Sonnenberger wrote:
> I classify EAs the same way as suid/sgid bits. Also it seems like e.g.
> coreutils doesn't copy them by default.

How do one tell Linux cp(1) to copy them? I see no flag for that in the
man page.

[Emmanuel Dreyfus]

On Wed, Jul 27, 2011 at 08:13:14AM +0000, Emmanuel Dreyfus wrote:
> On Wed, Jul 27, 2011 at 07:21:00AM +0200, Joerg Sonnenberger wrote:

> > I classify EAs the same way as suid/sgid bits. Also it seems like e.g.
> > coreutils doesn't copy them by default.
>

> How do one tell Linux cp(1) to copy them? I see no flag for that in the
> man page.

Reading the source, they have a --preserve=list where list is comma-separated
list of mode,ownership,timestamp,xattr. -p means --preserve with everything
set.

On MacOS X, -p copies extended attributes, and -X disable copying extended
attributes.

So we can go for -p to preserver extended attributes in cp(1). mv(1)
should probably do it by default, otherwise we have a kinky difference
between moving within a filesystem, and accress filesystems.

[Emmanuel Dreyfus]

On Tue, Jul 26, 2011 at 11:30:06PM -0400, Mouse wrote:

> > The only problem with putting it in a library is that we will need to

> > decide what API we [w]ant to promote: the ugly one from FreeBSD or
> > the [one] from Linux.
>
> Or we could invent our own.
> All the cool ki^W^Wother OSes are doing it.

Here is a patch that implements extended attribute copying routines
in libc, with both FreeBSD and Linux flavors:
extattr_copy_file extattr_copy_fd extattr_copy_link
cpxattr fcpxattr lcpxattr

cp -p and mv to a different filesystem use them (well, just the fcpxattr,
actually).

If nobody has concern about it, I will commit it shortly.

--
Emmanuel Dreyfus
ma...@netbsd.org