Does I/O in Goroutine blocks the underlying thread?

[Haoyang Fan]

Hi everyone,

I was always under the impression that Go solely uses async I/O under the hood so that when invoking a seemingly blocking call like os.File.Read the underlying thread won't be blocked. Go scheduler will save the context of current goroutine and schedule other goroutines to run on that thread. This understanding seems to be aligned with most material I can find on the internet.

However, recently when I was reading the slides (https://go.dev/talks/2012/waza.slide#32), on slide 32 I notice it says "When a goroutine blocks, that thread blocks but no other goroutine blocks". This is contradictory and make me wonder does Go really perform I/O in an asynchronous manner (e.g. like select/poll/epoll in Linux) under the hood?

Can somebody please clarify?

[robert engels]

It does. I think the wording is wrong - the Go routine blocks - but the carrier thread (that was running the Go routine) runs another Go routine (if needed and ready to run).

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e1c49031-821c-42af-9449-2c7586c3676fn%40googlegroups.com.

[Ian Lance Taylor]

On Fri, May 31, 2024 at 12:09 PM Haoyang Fan <haoyan...@gmail.com> wrote:
>
> I was always under the impression that Go solely uses async I/O under the hood so that when invoking a seemingly blocking call like os.File.Read the underlying thread won't be blocked. Go scheduler will save the context of current goroutine and schedule other goroutines to run on that thread. This understanding seems to be aligned with most material I can find on the internet.

That is how it works for most operations. That said, since you
specifically mentioned os.File.Read, if the os.File is a disk file,
then on most Unix systems the goroutine and the underlying thread will
indeed block for the duration of the I/O. That is because most Unix
systems have no mechanism for non-blocking I/O for disk files, so the
I/O does block the underlying thread. The underlying thread will not
block for I/O on a network connection or a pipe. As a practical
matter this is only relevant when using a networked file system, as
local file systems are fast.

> However, recently when I was reading the slides (https://go.dev/talks/2012/waza.slide#32), on slide 32 I notice it says "When a goroutine blocks, that thread blocks but no other goroutine blocks". This is contradictory and make me wonder does Go really perform I/O in an asynchronous manner (e.g. like select/poll/epoll in Linux) under the hood?
>
> Can somebody please clarify?

That talk is from 2012. The network poller and scheduler have been
rewritten since then.

Ian

[Haoyang Fan]

Let's say if I'm writing a program in Golang that is similar to the "top" command, which rapidly reads the files under /proc directory (e.g. every 1 second) and I'm launching goroutines to read different parts of that directory. Since /proc is a virtual filesystem not actually stored on the disk, can I assume in this case there won't be any thread blocked?

[Ian Lance Taylor]

On Mon, Jun 3, 2024 at 1:09 AM Haoyang Fan <haoyan...@gmail.com> wrote:
>
> Let's say if I'm writing a program in Golang that is similar to the "top" command, which rapidly reads the files under /proc directory (e.g. every 1 second) and I'm launching goroutines to read different parts of that directory. Since /proc is a virtual filesystem not actually stored on the disk, can I assume in this case there won't be any thread blocked?

Technically the threads will block briefly as they read from the /proc
file system. However, I would expect that the time they spend blocked
would be very short, imperceptible to a 1 second loop.

Ian

> On Friday, May 31, 2024 at 12:34:38 PM UTC-7 Ian Lance Taylor wrote:
>>
>> On Fri, May 31, 2024 at 12:09 PM Haoyang Fan <haoyan...@gmail.com> wrote:
>> >
>> > I was always under the impression that Go solely uses async I/O under the hood so that when invoking a seemingly blocking call like os.File.Read the underlying thread won't be blocked. Go scheduler will save the context of current goroutine and schedule other goroutines to run on that thread. This understanding seems to be aligned with most material I can find on the internet.
>>
>> That is how it works for most operations. That said, since you
>> specifically mentioned os.File.Read, if the os.File is a disk file,
>> then on most Unix systems the goroutine and the underlying thread will
>> indeed block for the duration of the I/O. That is because most Unix
>> systems have no mechanism for non-blocking I/O for disk files, so the
>> I/O does block the underlying thread. The underlying thread will not
>> block for I/O on a network connection or a pipe. As a practical
>> matter this is only relevant when using a networked file system, as
>> local file systems are fast.
>>
>> > However, recently when I was reading the slides (https://go.dev/talks/2012/waza.slide#32), on slide 32 I notice it says "When a goroutine blocks, that thread blocks but no other goroutine blocks". This is contradictory and make me wonder does Go really perform I/O in an asynchronous manner (e.g. like select/poll/epoll in Linux) under the hood?
>> >
>> > Can somebody please clarify?
>>
>> That talk is from 2012. The network poller and scheduler have been
>> rewritten since then.
>>
>> Ian
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4e467662-c516-4956-aa76-06f3b244c656n%40googlegroups.com.