unix.Select with fd gotten from named pipe on macos behaves differently compared to linux

[TheDiveO]

Hi, I need to detect on the producer side (writing end) of a named pipe when the consumer (reading end) has disconnect/closed. This detection needs to work "quickly" even if the producer doesn't produce anything; thus, SIGPIPE wouldn't help.

On Linux, when using unix.Select() on the fd of the producer's writing end of the named pipe, the fd becomes readable only upon the consumer disconnecting.

On macos, unix.Select indicates that the writing end fd becomes readable as soon as the producer writes(!) data. But it never reports the fd becoming readable upon the consumer disconnecting.

I'm opening both named pipe ends as follows (in different processes):

os.OpenFile(fifoname, os.O_WRONLY, os.ModeNamedPipe)

os.OpenFile(fifoname, os.O_RDONLY, os.ModeNamedPipe)

• is there something to know of in Go's runtime/stdlib (and especially the poller) that might interfere with my usage of unix.Select on an ordinary *os.File?
• how can I detect the consumer disconnecting on macos?

[Kurtis Rader]

Do not make us guess what your code looks like and which packages you are using. I'm guessing you are using https://pkg.go.dev/golang.org/x/sys/unix#Select but I shouldn't have to do so. You should be able to show us a minimal reproducible example of code that illustrates a problem of this nature.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3f8df371-4fda-44e2-8acb-b0743fb6b27en%40googlegroups.com.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[Kurtis Rader]

It is also more likely, but not certain, that the bug is in your code rather than the macOS kernel or the https://pkg.go.dev/golang.org/x/sys/unix#Select Go code. Which is why it is important to actually show us the code you wrote. I've been programming for a living since 1979 and have written a lot of buggy code. I long ago learned to assume the mistake was on my end rather than the code I was relying on. That assumption is sometimes wrong but so infrequent as to be a rounding error compared to how often the mistake is mine.

On Fri, Dec 15, 2023 at 7:13 AM 'TheDiveO' via golang-nuts <golan...@googlegroups.com> wrote:

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3f8df371-4fda-44e2-8acb-b0743fb6b27en%40googlegroups.com.

[TheDiveO]

• implementation using unix.Select (btw, I explicitly mentioned unix.Select in my OP): https://github.com/siemens/cshargextcap/blob/2f45f96748e835f0fef4cf429ca27f92a6c60a33/pipe/checker_notwin.go
• unit test producing this behavior, differing between Linux and macos: https://github.com/siemens/cshargextcap/blob/2f45f96748e835f0fef4cf429ca27f92a6c60a33/pipe/checker_notwin_test.go

That should suffice as a minimal example. "go test -v ./pipe -ginkgo.v" gives details as the test progresses.

[Kurtis Rader]

On Sat, Dec 16, 2023 at 7:54 AM 'TheDiveO' via golang-nuts <golan...@googlegroups.com> wrote:

• implementation using unix.Select (btw, I explicitly mentioned unix.Select in my OP): https://github.com/siemens/cshargextcap/blob/2f45f96748e835f0fef4cf429ca27f92a6c60a33/pipe/checker_notwin.go

That you mentioned "unix.Select" in your first message  does not change the fact that we were left to assume you meant golang.org/x/sys/unix rather than some other "unix" package. Sure, that assumption was likely correct but, again, don't require people to make such assumptions. 

• unit test producing this behavior, differing between Linux and macos: https://github.com/siemens/cshargextcap/blob/2f45f96748e835f0fef4cf429ca27f92a6c60a33/pipe/checker_notwin_test.go

That should suffice as a minimal example. "go test -v ./pipe -ginkgo.v" gives details as the test progresses.

I copied your code at https://github.com/siemens/cshargextcap/blob/2f45f96748e835f0fef4cf429ca27f92a6c60a33/pipe/checker_notwin.go, changed the package to "main", added a "func main()" and tested it. It works identically on Linux and macOS. I did this test by running "mknod p p" in one terminal and having my copy of your code open that file for reading in a second terminal. I then executed "sleep 1 > p" in the first terminal. I got the same behavior on Linux and macOS. So I assume your unit test is flawed.

I only glanced at your unit test but the sleeps and goroutines without any explicit synchronization (e.g., using a sync.WaitGroup) look to me like a source of non-deterministic behavior.

Your expectation of the behavior of your "WaitTillBreak()" function might also be incorrect. A file descriptor open for reading on a FIFO becomes readable when there is data available to be read or the write side of the FIFO is closed. And it is not at all clear why you named it "WaitTillBreak"; even with the comment "checks a fifo/pipe to see when it breaks." A fifo/pipe doesn't "break". It either has at least one open fd for writing or not; in which case the last writer has closed its end of the fifo/pipe. It's not "broken". It simply has entered an EOF condition for the reader since no more data will be sent. Are you using the named pipe solely as a semaphore rather than to send data? That is an atypical use for a named pipe.

[Kurtis Rader]

I should probably clarify that my copy of TheDiveO code simply added the following function and replaced the use of github.com/sirupsen/logrus with println.

func main() {
        f, err := os.OpenFile("p", os.O_RDONLY, 0)
        if err != nil {
                println("open p", err.Error())
                os.Exit(1)
        }
        println("fifo opened")
        WaitTillBreak(f)
}

[Kurtis Rader]

On Fri, Dec 15, 2023 at 7:13 AM 'TheDiveO' via golang-nuts <golan...@googlegroups.com> wrote:

I'm opening both named pipe ends as follows (in different processes):

os.OpenFile(fifoname, os.O_WRONLY, os.ModeNamedPipe)

os.OpenFile(fifoname, os.O_RDONLY, os.ModeNamedPipe)

Passing  os.ModeNamedPipe to os.OpenFile doesn't make any sense unless the open is creating the named pipe by also including os.O_CREATE and it's not clear doing so is even valid (I haven't tested whether it works). You can't force a file to behave like a named pipe by passing that value to the os.OpenFile function and that flag isn't needed to open an existing named pipe. The os.ModeNamedPipe constant is meant to be used when testing the file mode returned by os.Stat. You would normally use the unix.Mkfifo function to create a named pipe.

[TheDiveO]

Please note that the unit test I linked to tests on the "writing end" of the pipe. In fact, I wrote in the OP right in my first sentence:

> Hi, I need to detect on the producer side (writing end) of a named pipe when the consumer (reading end) has disconnect/closed. 

I'm afraid, but you are not reproducing the situation I was asking for and provided a unit test for.

[TheDiveO]

It actually does work (again, there's a unit test as well as working code for five years now), but good to know that it is actually useless, so this is something to remove as part of the usual maintenance chores.

[TheDiveO]

The issue is also reproducible using Python3 on macOS, whereas it works as expected once again on Linux; just for reference here's the corresponding SO question. To a large extend, this now excludes potential Go and/or Ginkgo/Gomega-related effects, yet is consistent with my original observation and question here.

On Sunday, December 17, 2023 at 6:02:17 AM UTC+1 Kurtis Rader wrote:

On Sat, Dec 16, 2023 at 7:54 AM 'TheDiveO' via golang-nuts <golan...@googlegroups.com> wrote:

[...] 

I only glanced at your unit test but the sleeps and goroutines without any explicit synchronization (e.g., using a sync.WaitGroup) look to me like a source of non-deterministic behavior.

[...] 

[Steven Hartland]

poll and select for this behavior was broken in FreeBSD up until 195423 was committed as detailed by this bug report.

Given the MacOS and FreeBSD have related history I wouldn't be surprised if this is a bug in the OS implementation with poll on MacOS.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a7fe0e6e-d6b0-4881-a02c-61516684caf9n%40googlegroups.com.

[Kurtis Rader]

On Thu, Dec 21, 2023 at 5:40 PM Steven Hartland <stevenm...@gmail.com> wrote:

poll and select for this behavior was broken in FreeBSD up until 195423 was committed as detailed by this bug report.

Given the MacOS and FreeBSD have related history I wouldn't be surprised if this is a bug in the OS implementation with poll on MacOS.

Thanks for the link to that bug report. It was opened when I was a senior support engineer at IBM doing, primarily, Linux and DYNIX/ptx kernel crash analysis and dealing with similar problems. The latter OS was because I had been doing the same job when IBM purchased Sequent Computer Systems. I don't recall any customer reporting this "bug" for either OS in 2006, but since DYNIX/ptx was derived from AT&T UNIX System V (aka SVR5) it is likely it exhibited the same behavior (i.e., "bug") as macOS does today.

Whether that behavior is "broken" is debatable since the behavior is likely the documented, or at least expected behavior when the syscall was introduced. If you read that bug report you'll notice comments that the old behavior was consistent with the relevant standards. Which is not to say that behavior is optimal. Changes since 2006 by FreeBSD and Linux clearly represent improvements to the semantics of polling/selecting a FIFO file descriptor. But someone needs to open an issue with Apple and convince them to adopt the same improvement. This isn't something that the Go ecosystem can workaround. Too, for many use cases switching from a named fifo to something like a unix domain socket will provide the desired behavior on all the supported platforms.