os.Rename() & .Remove() of opened file

221 views
Skip to first unread message

Liam Breck

unread,
May 19, 2019, 3:17:22 PM5/19/19
to golang-dev
This works on Linux and MacOS, but is disabled on Windows:

fd, err := os.OpenFile(path, ...)
...
err = os.Rename(path, path2) // sharing violation
...
err = os.Remove(path2) // sharing violation


It seems like an oversight; there's no comment in the code about it.  If you've been bitten by this inconsistency, please comment in

Benny Siegert

unread,
May 19, 2019, 3:41:13 PM5/19/19
to Liam Breck, golang-dev
It's not an oversight, it's how the OS works.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAKvHMgTiim-fJ4CAf9-om9S43v3u537TKtPKL4vWvPMW-d3KHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Liam Breck

unread,
May 19, 2019, 3:51:24 PM5/19/19
to Benny Siegert, golang-dev
Benny, please read the issue.

Guy Brandwine

unread,
May 19, 2019, 3:58:47 PM5/19/19
to Liam Breck, Benny Siegert, golang-dev
Windows API (called createFile/ReadFile/LockFIle) is locking the file while the file is in process.
Nix on the other hand, allows a file to be accessed and renamed/removed at the time, keeping a version of the old file until all processes release th file (which allows among other replacing files for version updates without restarting your system as windows must.

In other words - this is not a go behavior - but OS as Benny stated. 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAKvHMgTUFL_xQ8shEkanVWj5biyyvLK-tEQv1fM5G9wOJGiDfg%40mail.gmail.com.

Liam Breck

unread,
May 19, 2019, 5:03:51 PM5/19/19
to Guy Brandwine, Benny Siegert, golang-dev
You can configure either behavior with an argument to CreateFile() which is documented in the issue. Thanks.

Guy Brandwine

unread,
May 19, 2019, 8:43:25 PM5/19/19
to golang-dev
When writing a cross platform language, selecting if your code behavior should behave "native like" (as the system act on default) or "exactly the same os-agnostic" is a hard choice - 
but this is not an option in this case

I assume that a  code that writes a file and then delete it is a very common use for piping data (not sure if this is your use case, but a very common use)
So, lets assume my program is writing a file called "my.data", and then calls another program and does not wait for it to end (or the other program is already running in the background) , this program lets say uploads this file which takes 10 seconds (lets call this other program "uploader") .
When my program calls .Remove on Linux:
1. if anyone would "ls my.data" , he would not get results
2. uploader would continue to upload the data and when it closes the handle (assuming on other app still has a handle to this file) - the file would be "really gone" including space on disk. 
in fact, during these 10 seconds there may be 10 (or more) uploaders uploading 10 diff files all called "my.data" at the point uploader started, and deleted since.

When my program calls .Remove on Windows(FILE_SHARE_DELETE is on):
1. if anyone would "dir my.data", he would not get results
2. uploader would receive an error telling it the file is removed (probably EOF).

So - using FILE_SHARE_DELETE would not gain consistency os-wide, and stop using default os behavior on the other hand.
lose-lose 

so, no resolution?
There may be a resolution, paiing an _open_osfhandle() of the CreateFile to and _fdopen and thus using a file descriptor instead of a file handle.
But this is not a simple "add FILE_SHARE_DELETE flag when opening a file in go ", but a somewhat "larger" change (I had to implement one for interacting with a PTP file sharing service at the time PTP was a hit, this was not a 5 min task :) ) . 

Hope that's a better explanation.

On Monday, 20 May 2019 00:03:51 UTC+3, Liam wrote:
You can configure either behavior with an argument to CreateFile() which is documented in the issue. Thanks.

On Sun, May 19, 2019, 12:58 PM Guy Brandwine <guyb...@gmail.com> wrote:
Windows API (called createFile/ReadFile/LockFIle) is locking the file while the file is in process.
Nix on the other hand, allows a file to be accessed and renamed/removed at the time, keeping a version of the old file until all processes release th file (which allows among other replacing files for version updates without restarting your system as windows must.

In other words - this is not a go behavior - but OS as Benny stated. 

On Sun, 19 May 2019 at 22:51, Liam Breck <networ...@gmail.com> wrote:
Benny, please read the issue.

On Sun, May 19, 2019, 12:41 PM Benny Siegert <bsie...@gmail.com> wrote:
It's not an oversight, it's how the OS works.

Liam Breck <networ...@gmail.com> schrieb am So., 19. Mai 2019, 15:17:
This works on Linux and MacOS, but is disabled on Windows:

fd, err := os.OpenFile(path, ...)
...
err = os.Rename(path, path2) // sharing violation
...
err = os.Remove(path2) // sharing violation


It seems like an oversight; there's no comment in the code about it.  If you've been bitten by this inconsistency, please comment in

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages