Testing file system races

[Florian Weimer]

On

commit a076f497577605e4cf0e20c147711e03dee7b2c3
Author: database64128 <free1...@hotmail.com>
Date: Fri Aug 22 01:03:42 2025 +0800

os: fix Root.MkdirAll to handle race of directory creation

No tests were added, because in order to reproduce, the directory would
have to be created precisely between the rootOpenDir and mkdirat calls,
which is impossible to do in a test.

Fixes #75114

you mentioned that testing this is impossible. In glibc, we have had
good success creating such impossible tests with custom FUSE file
systems. We use a service thread in the same process to respond to
kernel events, so it's easy enough the communicate with the test. We
don't have a mkdirhier function in glibc, but mkstemp faces a similar
testing challenge. Our FUSE test just pretends that all the files that
mkstemp tries to create already exist.

Thanks,
Florian

[Florian Weimer]

* Ian Chen:

> On Mon, 2025-08-25 at 11:28 +0200, Florian Weimer wrote:
>> you mentioned that testing this is impossible.  In glibc, we have had
>> good success creating such impossible tests with custom FUSE file
>> systems.  We use a service thread in the same process to respond to
>> kernel events, so it's easy enough the communicate with the test.  We
>> don't have a mkdirhier function in glibc, but mkstemp faces a similar
>> testing challenge.  Our FUSE test just pretends that all the files
>> that
>> mkstemp tries to create already exist.
>

> Thanks for the suggestion. I'm not familiar with FUSE, and there seems
> to be no FUSE usage anywhere in the Go standard library. I wonder
> about the feasibility of implementing such a FUSE test in Go.

You just have to open a device not and read and write binary blobs. For
non-root testing, you need to create user and mount namespaces, which I
understand is a bit of a hassle with Go. There could be reentrancy
issues if the Go run-time performs operations on FUSE-backed descriptors
in such a way that prevent the FUSE thread from being scheduled again.

Thanks,
Florian

[Ian Chen]

On Mon, 2025-08-25 at 11:28 +0200, Florian Weimer wrote:

> you mentioned that testing this is impossible.  In glibc, we have had
> good success creating such impossible tests with custom FUSE file
> systems.  We use a service thread in the same process to respond to
> kernel events, so it's easy enough the communicate with the test.  We
> don't have a mkdirhier function in glibc, but mkstemp faces a similar
> testing challenge.  Our FUSE test just pretends that all the files
> that
> mkstemp tries to create already exist.

Thanks for the suggestion. I'm not familiar with FUSE, and there seems
to be no FUSE usage anywhere in the Go standard library. I wonder
about the feasibility of implementing such a FUSE test in Go.

Ian

[Russ Cox]

The general point about testing against a synthetic file system implementation is a very good one. For Go, I think it would work better to do the indirect inside the MkdirAll implementation rather than involve FUSE. That is, MkdirAll could be implemented internally as a call to mkdirAll(fs) where fs is an implementation of

type mkdirAllFS interface {

    ... whatever mkdirAll needs ...

}

and then normally it would be passed the real os implementation of that interface, but for testing conditions like the one in this bug, a custom implementation would be passed.

Using FUSE has the benefit of moving the redirect out of the code-under-test but also adds more complexity and introduces the significant non-portability of only working on some operating systems. Neither a Go interface nor FUSE is a substitute for testing against "real file systems", since in both cases the simulation may not be accurate in various respects. 

I do see that for the case of glibc, which runs on fewer operating systems than Go does, the portability problem is not as a big a concern, and the scales may tip more toward the FUSE approach.

Best,

Russ