std::filesystem::copy_options::overwrite_existing throws

[Ralf Goertz]

Hi,

I just noticed that std::filesystem::copy_options::overwrite_existing
doesn't have an effect using gcc 9.3.0 under windows but works fine
under linux. That is, the program

#include <filesystem>

int main() {
std::filesystem::copy_file("foo","bar",std::filesystem::copy_options::overwrite_existing);
}

throws with

terminate called after throwing an instance of 'std::filesystem::__cxx11::filesystem_error'
what(): filesystem error: cannot copy file: File exists [foo] [bar]

provided that the regular files "foo" and "bar" exist. Under linux this
only happens if I omit the copy option. Since this is such a trivial
thing and I don't find any reports on gcc's bugzilla I wonder if this
behaviour is standard conforming. But then there is also nothing on
cpprefererence.com on that (AFAICT). Can this really be a bug?

[James Kuyper]

The standard only allows an implementation to "Report a file already
exists error as specified in 30.10.7 if:
(4.1.1) — !is_regular_file(from), or
(4.1.2) — exists(to) and !is_regular_file(to), or
(4.1.3) — exists(to) and equivalent(from, to), or
(4.1.4) — exists(to) and
(options & (copy_options::skip_existing |
copy_options::overwrite_existing |
copy_options::update_existing)) == copy_options::none"

Do any of those possibilities apply? If not, it's a bug.

[Paavo Helde]

FWIW, compiling this program with my Cygwin gcc 9.3.0 in Windows works
as expected (copies the file).

Note that in Windows the files may be easily locked by other programs
(including non-obvious ones like virus scanners) so that they cannot be
deleted (though in this case the error message should probably be
something like "Access denied").

In Unix world file deletion means just a refcount drop in inode which
has a much better chance to succeed. To cope with that deficiency there
is even a special Windows feature (MoveEx) to schedule files for
deletion on next restart.

[Ralf Goertz]

Am Thu, 30 Apr 2020 22:58:54 +0300
schrieb Paavo Helde <ees...@osa.pri.ee>:

> 30.04.2020 16:46 Ralf Goertz kirjutas:
> > Hi,
> >
> > I just noticed that
> > std::filesystem::copy_options::overwrite_existing doesn't have an
> > effect using gcc 9.3.0 under windows but works fine under linux.
> > That is, the program
> >
> > #include <filesystem>
> >
> > int main() {
> > std::filesystem::copy_file("foo","bar",std::filesystem::copy_options::overwrite_existing);
> > }
> >
> > throws with
> >
> > terminate called after throwing an instance of
> > 'std::filesystem::__cxx11::filesystem_error' what(): filesystem
> > error: cannot copy file: File exists [foo] [bar]
> >
> > provided that the regular files "foo" and "bar" exist. Under linux
> > this only happens if I omit the copy option. Since this is such a
> > trivial thing and I don't find any reports on gcc's bugzilla I
> > wonder if this behaviour is standard conforming. But then there is
> > also nothing on cpprefererence.com on that (AFAICT). Can this
> > really be a bug?
>
> FWIW, compiling this program with my Cygwin gcc 9.3.0 in Windows
> works as expected (copies the file).

Interesting. I used msys2 64bit. So this is more likely to be an msys2
bug then?

> Note that in Windows the files may be easily locked by other programs
> (including non-obvious ones like virus scanners) so that they cannot
> be deleted (though in this case the error message should probably be
> something like "Access denied").

Yeah I know that, but this is not the case here. I used

$ touch foo
$ fs_copy
$ fs_copy

with fs_copy being the program above and its second execution threw.
(Also in a powershell btw.) Then after deleting "foo" and "bar" I used

$ touch foo
$ cp foo bar
$ cp foo bar

which worked fine.

[Alf P. Steinbach]

Is it /consistently/ reproducible?

During the last year or so Microsoft has introduced a bug-nest somewhere
in the high level file system interfaces, so that some operations such
as obtaining directory listings, involve inordinately long pauses --
usually one long pause (which is not disk spin-up) at the start.

They have a history of introducing such pauses for Odin knows what
reason, e.g. for listing network connections via `netstat`, or listing
open file handles via `openfiles`, or for that matter, for moving, via
Explorer, a directory from one place to another place on the same drive,
but via a logical drive (which shouldn't matter, it should be instant
anyway). Historically it seems to me, but I /could/ be wrong, that these
artifical slow-downs have to do with some manager or group of managers
at Microsoft saving himself/themselves work and face by sabotaging the
system instead of being responsible for a fix of something.

- Alf

[Ralf Goertz]

Yes. I noticed it about a month ago after porting my linux program to
windows. The program is in heavy use by my colleagues and they always
have to delete the files before starting it. It was only yesterday that
I started to investigate it further.

> During the last year or so Microsoft has introduced a bug-nest
> somewhere in the high level file system interfaces, so that some
> operations such as obtaining directory listings, involve inordinately
> long pauses -- usually one long pause (which is not disk spin-up) at
> the start.
>
> They have a history of introducing such pauses for Odin knows what
> reason, e.g. for listing network connections via `netstat`, or
> listing open file handles via `openfiles`, or for that matter, for
> moving, via Explorer, a directory from one place to another place on
> the same drive, but via a logical drive (which shouldn't matter, it
> should be instant anyway). Historically it seems to me, but I /could/
> be wrong, that these artifical slow-downs have to do with some
> manager or group of managers at Microsoft saving himself/themselves
> work and face by sabotaging the system instead of being responsible
> for a fix of something.
>
> - Alf

I wouldn't know but I doubt it. As I said there is no problem copying
and overwriting files by other means.