Goroutine never yields
Trevor Fancher
567b924e9041+ tip
trevor$ env | grep GO
GOBIN=/Users/trevor/dev/go/bin
GOARCH=amd64
GOROOT=/Users/trevor/dev/go
GOOS=darwin
trevor$ cat good.go
package main
import "fmt"
import "runtime"
func f() {
for {
runtime.Gosched()
}
}
func main() {
runtime.GOMAXPROCS(2)
go f()
for i := 0; i < 25; i++ {
fmt.Print(i, " ")
}
fmt.Print("\n")
}
trevor$ 6g good.go && 6l -o good good.6 && ./good
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
trevor$ cat bug.go
package main
import "fmt"
import "runtime"
func f() {
for {
//runtime.Gosched()
}
}
func main() {
runtime.GOMAXPROCS(2)
go f()
for i := 0; i < 25; i++ {
fmt.Print(i, " ")
}
fmt.Print("\n")
}
trevor$ 6g bug.go && 6l -o bug bug.6 && ./bug
^C
Notes:
- The problem is bug.go hangs (sometimes a few i's get printed out,
but usually not).
- I have a MacBook with a Core 2 Duo.
- The difference between good.go and bug.go is the
"runtime.Gosched()" line is commented out in bug.go.
- I expect both versions to eventually quit with the same output.
Further, I would expect the same if I set GOMAXPROCS to 1.
Trevor
Andrew Gerrand
> - The problem is bug.go hangs (sometimes a few i's get printed out, but
> usually not).
> - I have a MacBook with a Core 2 Duo.
> - The difference between good.go and bug.go is the "runtime.Gosched()" line
> is commented out in bug.go.
Creating a goroutine does not guarantee it will be bound to a separate thread.
The goroutine in the second example never blocks or sends on a
channel. It is a contrived and somewhat nonsensical program, and thus
the runtime does not schedule the main thread as you might expect.
> - I expect both versions to eventually quit with the same output. Further,
> I would expect the same if I set GOMAXPROCS to 1.
Why would you expect that? Goroutines are not garbage collected, and
your function f() never returns in either program. I would expect both
programs to spin indefinitely.
Andrew
Devon H. O'Dell
> On Tue, Jan 19, 2010 at 12:25 PM, Trevor Fancher <tre...@fancher.org> wrote:
>> - The problem is bug.go hangs (sometimes a few i's get printed out, but
>> usually not).
>> - I have a MacBook with a Core 2 Duo.
>> - The difference between good.go and bug.go is the "runtime.Gosched()" line
>> is commented out in bug.go.
>
> Creating a goroutine does not guarantee it will be bound to a separate thread.
>
> The goroutine in the second example never blocks or sends on a
> channel. It is a contrived and somewhat nonsensical program, and thus
> the runtime does not schedule the main thread as you might expect.
Still a bug. Russ pushed a change to have goroutines scheduled earlier
with MAXOSPROCS so that a goroutine that is always runnable will
execute in a separate OS thread. This goroutine should not yield; it
should run indefinitely. However, the main goroutine should get time
in the first goroutine on the first OS thread, and it should
eventually return (thus causing the spinning goroutine to exit).
I haven't checked to see whether that revision is actually tip, but
I'm guessing it is.
--dhoe
andrey mirtchovski
runtime a chance to run the scheduler. something as simple as
allocating some memory. put the following instead of the commented
Gosched() in runtime:
_ = make([]byte, 10) //runtime.Gosched()
Russ Cox
Today, neither exits with GOMAXPROCS == 1 because there is
no preemptive scheduling, so once f starts running, it keeps running.
When GOMAXPROCS > 1, bad.go still fails to exit because the
main goroutine eventually tries to garbage collect, which waits for
f to get rescheduled so that memory will not be changing underfoot.
But f never obliges. This is the same bug that came up last week:
http://groups.google.com/group/golang-nuts/t/7a9535c4136d3e2
I've created an issue to track this (#543) but it's going to be a while.
Typical programs don't need to worry, but ones that use infinite
compute loops for whatever reason will need to explicitly schedule
once in a while.
Russ
Trevor Fancher
> I agree: both programs should exit, no matter what GOMAXPROCS is set to.
Ok, glad my expected behaivor is /the/ expected behavor.
> I've created an issue to track this (#543) but it's going to be a while.
Understandable.
> Typical programs don't need to worry, but ones that use infinite
> compute loops for whatever reason will need to explicitly schedule
> once in a while.
For the record, I'd like to note my bug code was based on real world code.
> Russ
Trevor
Miguel Pignatelli
I have some goroutines (hundreds of thousands) writing to stdout with
fmt.Printf and I have found that (a few times) the output of different
goroutines appears mixed in the final output (i.e. before a goroutine
ends writing its output, the output of another goroutine appears). Is
this possible? is the printing to stdout atomic? If not, which is the
best solution?
Thanks in advance,
M;
Russ Cox
a single fmt.Printf can do multiple writes.
you could use a wrapper to make sure each printf turns into a single write:
func printf(format string, args ...interface{}) {
var buf bytes.Buffer
fmt.Fprintf(&buf, format, args)
os.Stdout.Write(buf.Bytes())
}
that's more likely to work, though if you're writing to a unix pipe
it's possible for the kernel to interlace writes itself depending on
the details of the pipe filling. if you must have truly synchronous
output, the next step would be to put a lock inside printf, so that
the operating system only sees one writer at a time:
var printfLock sync.Mutex
func printf(format string, args ...interface{}) {
printfLock.Lock()
defer printfLock.Unlock()
var buf bytes.Buffer
fmt.Fprintf(&buf, format, args)
os.Stdout.Write(buf.Bytes())
}
russ
andrey mirtchovski
a printing goroutine on the side which shares a channel with all the
printers. have them all send messages on that channel and print those
sequentially. create a chan with a large-ish buffer if you don't want
them all to wait.
----8<-----------
package main
import "fmt"
func worker(c chan string, id int) {
for {
c <- fmt.Sprint("some reasonably long string from thread: ", id)
}
}
func main() {
c := make(chan string, 100)
for i := 0; i < 10; i++ {
go worker(c, i)
}
for {
s := <-c
fmt.Println(s)
}
}
Rob 'Commander' Pike
On Apr 13, 2010, at 10:04 AM, Russ Cox wrote:
>> I have some goroutines (hundreds of thousands) writing to stdout with
>> fmt.Printf and I have found that (a few times) the output of
>> different
>> goroutines appears mixed in the final output (i.e. before a
>> goroutine ends
>> writing its output, the output of another goroutine appears). Is this
>> possible? is the printing to stdout atomic? If not, which is the best
>> solution?
>
> a single fmt.Printf can do multiple writes.
Actually it doesn't do multiple Writes, but it may do multiple
writes. Printf accumulates all its data into a byte buffer and then
does a single Write to os.Stdout.
> you could use a wrapper to make sure each printf turns into a single
> write:
>
> func printf(format string, args ...interface{}) {
> var buf bytes.Buffer
> fmt.Fprintf(&buf, format, args)
> os.Stdout.Write(buf.Bytes())
> }
That would just add overhead; Printf already does this.
>
> that's more likely to work, though if you're writing to a unix pipe
> it's possible for the kernel to interlace writes itself depending on
> the details of the pipe filling.
I suspect this is the real problem: a single Write to os.Stdout might
be broken into pieces by the kernel, so pieces of Writes from
different goroutines might mingle.
> if you must have truly synchronous
> output, the next step would be to put a lock inside printf, so that
> the operating system only sees one writer at a time:
>
> var printfLock sync.Mutex
>
> func printf(format string, args ...interface{}) {
> printfLock.Lock()
> defer printfLock.Unlock()
> var buf bytes.Buffer
> fmt.Fprintf(&buf, format, args)
> os.Stdout.Write(buf.Bytes())
> }
That should fix it, but you don't need the byte buffer. Just put a
lock on printf.
-rob