Check whether the file is opened by another process or not in pure go

[Jérôme LAFORGE]

Hello,
Is it possible to check whether one specific file is opened by another process or not in pure go (as fuser) ?
On Linux, my goal is to monitor a specific directory with go-fsnotify/fsnotify, and process new files into this directory but only when the copy is done (avoid to process it during they are created/written).

Thx in adv
Jérôme

[Ian Lance Taylor]

My understanding is that fuser works by looking through the /proc file
system. A Go program can do the same thing.

Note that any such program will have race conditions. New processes
can start up, and open files for writing, while you are doing your
processing.

Ian

[Konstantin Khomoutov]

On Thu, 23 Jul 2015 09:47:18 -0700
Ian Lance Taylor <ia...@golang.org> wrote:

> > Is it possible to check whether one specific file is opened by
> > another process or not in pure go (as fuser) ?
> > On Linux, my goal is to monitor a specific directory with
> > go-fsnotify/fsnotify, and process new files into this directory but
> > only when the copy is done (avoid to process it during they are
> > created/written).
>
> My understanding is that fuser works by looking through the /proc file
> system. A Go program can do the same thing.
>
> Note that any such program will have race conditions. New processes
> can start up, and open files for writing, while you are doing your
> processing.

Jérôme, inotify on Linux supports the type of event meaning
"written then closed" [1], that is, if you monitor a directory for
events of this type on its files, these events will only fire when some
file has been opened for writing (or appending, which doesn't matter)
and then closed.

This is a good approximation of what you're after.
Sure it's inherently racy, but no more than scanning /proc (and is way
simpler).

If `go-fsnotify` has no knobs for this, you could supposedly hack it to
have them or resort to spawning the `inotifywait` program
(`inotify-tools` package in Debian) and reading its stdout.

1. http://man7.org/linux/man-pages/man7/inotify.7.html
2. http://linux.die.net/man/1/inotifywait

[Michael Schaller]

Inotify is definitely the solution for this problem. In general code that handles inotify events needs to be extremely fault tolerant because there are a lot of race conditions. Additionally there are a few more things to watch out for:

First of all you need to watch the right inotify events. You can use pyinotify by running 'python -m pyinotify -v /my-dir-to-watch' to get an idea which events you need to handle and how to distinguish between different cases.

Secondly, you might want to consider several variants how files can be created:

1) Normal file creation, write, close.

2) Overwrite an existing file, write, close.

3) Temporary file creation, write, close and then move to the final location (same filesystem).

4) Temporary file creation, write, close and then move to the final location (different filesystem).

5) Hardlink creation.

6) Softlink creation. Link target exists.

7) Softlink creation. Link target doesn't exist (yet).

Thirdly, whatever you watch via inotify is not watched via its name/path but via its inode. If that inode changes because for an instance a whole directory gets moved/replaced then inotify will not be able to detect these changes unless the parent directory is also watched via inotify.

Lastly you might also have to watch out for permission/ownership changes. These can be an additional source of race conditions.

[Jérôme LAFORGE]

Thx you all for the tips.