goroutine to run when everybody else is idle..
492 views
Skip to first unread message
Sunil S Nandihalli
May 3, 2011, 2:23:55 AM5/3/11
to golan...@googlegroups.com
Hello everybody,
I was just wondering if there is a way to have a go-routine start off when everybody else is idle .. and do some computation and make way as soon as somebody else is ready .. and start again when everybody else is idle... this is just a thought ..
Thanks,
Sunil.
⚛
May 3, 2011, 4:14:19 AM5/3/11
to golang-nuts
It is doable in Go if you program it by hand. But I don't think this
is what you are asking. You are asking whether there exists an easy
way (just 1 line of Go code) of how to run the "idle goroutine". The
answer is: no (as far as I know).
On May 3, 8:23 am, Sunil S Nandihalli <sunil.nandiha...@gmail.com>
wrote:
is what you are asking. You are asking whether there exists an easy
way (just 1 line of Go code) of how to run the "idle goroutine". The
answer is: no (as far as I know).
On May 3, 8:23 am, Sunil S Nandihalli <sunil.nandiha...@gmail.com>
wrote:
Dmitriy Vyukov
May 3, 2011, 4:22:22 AM5/3/11
to golang-nuts
On Tue, May 3, 2011 at 12:14 PM, ⚛ <0xe2.0x...@gmail.com> wrote:
> It is doable in Go if you program it by hand. But I don't think this
> is what you are asking. You are asking whether there exists an easy
> way (just 1 line of Go code) of how to run the "idle goroutine". The
> answer is: no (as far as I know).
> It is doable in Go if you program it by hand. But I don't think this
> is what you are asking. You are asking whether there exists an easy
> way (just 1 line of Go code) of how to run the "idle goroutine". The
> answer is: no (as far as I know).
That looks basically like goroutine priorities, because then one will
ask for "idle idle goroutines"...
Sunil S Nandihalli
May 3, 2011, 4:36:00 AM5/3/11
to ⚛, golang-nuts
I realize there is no one line go code .. can you probably define a pattern of code which would make this happen...
Sunil S Nandihalli
May 3, 2011, 6:06:52 AM5/3/11
to Dmitriy Vyukov, golang-nuts
Yes I am looking for priorities..for goroutines..
Jan Mercl
May 3, 2011, 6:31:03 AM5/3/11
to golan...@googlegroups.com, Dmitriy Vyukov
On Tuesday, May 3, 2011 12:06:52 PM UTC+2, Sunil Nandihalli wrote:
Yes I am looking for priorities..for goroutines..
I guess (not tested) one can roughly get - in some limited/specific cases - close to this using the default case of a select statement:
In the default clause there will by a channel send to the "idle" goroutine, assuming it will be a short one.
Russ Cox
May 3, 2011, 6:31:45 AM5/3/11
to Sunil S Nandihalli, golan...@googlegroups.com
> I was just wondering if there is a way to have a go-routine start off when
> everybody else is idle .. and do some computation and make way as soon as
> somebody else is ready .. and start again when everybody else is idle...
> everybody else is idle .. and do some computation and make way as soon as
> somebody else is ready .. and start again when everybody else is idle...
No.
⚛
May 3, 2011, 9:32:22 AM5/3/11
to golang-nuts
It would involve counting how many goroutines are capable of running
at a particular moment. The idle goroutine starts running if the
counter drops to 0. If the idle goroutine is running, it would need to
periodically check whether the counter is still 0 and voluntarily
suspend itself as soon as it determines the counter is non-zero.
There is no simple answer how to count the number of goroutines that
are running. If a goroutine is sending through a channel, it would
first need to make sure that the send is non-blocking. If the send
blocks the sender, it decrements the counter and goes to sleep. The
receiver would have to notify the sleeping senders, so that they can
retry the send procedure. The receiver needs to decrement the
counter if a channel receive would block, and then needs to go sleep.
A sender would have to notify the sleeping receivers, so that they can
retry the receive procedure. The counter is incremented when a
sender or receiver starts running.
In addition, the counter is modified when a goroutine is waiting
because of an OS call. Same in respect to CGO calls. Handling the
counter in these cases means to *predict* whether the OS/CGO call will
block, the prediction needs to be made before the call is executed. It
effectively amounts to at least one of the following: (1) to control
the access to OS calls and CGO functions (some kind of encapsulation,
and some kind of access policy), and (2) building a sufficiently
accurate emulator/tracker of the called OS and CGO functions. The
problem is that each OS/CGO call is unique (well, they fall into
categories, so in fact those categories are unique), and each category
requires a separate implementation in the encapsulation layer or in
the emulator/tracker. Not to mention, the interactions of multiple OS/
CGO calls can be very complex (ok, they can be made simpler by
intentionally disallowing certain types of interactions, but such
simplifications have their consequences: lower performance in the
better case, and a loss of functionality in the worse case).
... so, this is basically what I meant when I wrote that "it is doable
"idle goroutine" or not.
On May 3, 10:36 am, Sunil S Nandihalli <sunil.nandiha...@gmail.com>
wrote:
at a particular moment. The idle goroutine starts running if the
counter drops to 0. If the idle goroutine is running, it would need to
periodically check whether the counter is still 0 and voluntarily
suspend itself as soon as it determines the counter is non-zero.
There is no simple answer how to count the number of goroutines that
are running. If a goroutine is sending through a channel, it would
first need to make sure that the send is non-blocking. If the send
blocks the sender, it decrements the counter and goes to sleep. The
receiver would have to notify the sleeping senders, so that they can
retry the send procedure. The receiver needs to decrement the
counter if a channel receive would block, and then needs to go sleep.
A sender would have to notify the sleeping receivers, so that they can
retry the receive procedure. The counter is incremented when a
sender or receiver starts running.
In addition, the counter is modified when a goroutine is waiting
because of an OS call. Same in respect to CGO calls. Handling the
counter in these cases means to *predict* whether the OS/CGO call will
block, the prediction needs to be made before the call is executed. It
effectively amounts to at least one of the following: (1) to control
the access to OS calls and CGO functions (some kind of encapsulation,
and some kind of access policy), and (2) building a sufficiently
accurate emulator/tracker of the called OS and CGO functions. The
problem is that each OS/CGO call is unique (well, they fall into
categories, so in fact those categories are unique), and each category
requires a separate implementation in the encapsulation layer or in
the emulator/tracker. Not to mention, the interactions of multiple OS/
CGO calls can be very complex (ok, they can be made simpler by
intentionally disallowing certain types of interactions, but such
simplifications have their consequences: lower performance in the
better case, and a loss of functionality in the worse case).
... so, this is basically what I meant when I wrote that "it is doable
in Go if you program it by hand".
A question is whether the problem you are solving actually needs the
"idle goroutine" or not.
On May 3, 10:36 am, Sunil S Nandihalli <sunil.nandiha...@gmail.com>
wrote:
> I realize there is no one line go code .. can you probably define a pattern
> of code which would make this happen...
>
> of code which would make this happen...
>
Michael Jones
May 3, 2011, 9:43:20 AM5/3/11
to ⚛, golang-nuts
Need it really be so hard? If the idle routine(s) have way to yield (explicitly or by "tail going" an unrolled loop) AND if there are two queues, the normal queue and the idle queue AND if the scheduler never starts an idle queue entry unless there are no normal queue entries to run, then that's that, or at least it seems that way. Why need it be more complicated?
Michael
--
Michael T. Jones
Chief Technology Advocate, Google Inc.
1600 Amphitheatre Parkway, Mountain View, California 94043
Email: m...@google.com Mobile: 650-335-5765 Fax: 650-649-1938
Organizing the world's information to make it universally accessible and useful
Mikael Tillenius
May 3, 2011, 9:59:30 AM5/3/11
to golang-nuts
time you might be able to get a similar effect by setting a low
priority on the OS thread, e.g. via sched_setscheduler() on Linux.
I think CGO plays funny games with the threads so I don't think you
can use that, you have to use the syscall directly.
It would definitely be a ugly hack;-)
/Mikael
Russ Cox
May 3, 2011, 10:52:09 AM5/3/11
to Michael Jones, ⚛, golang-nuts
On Tue, May 3, 2011 at 09:43, Michael Jones <m...@google.com> wrote:
> Need it really be so hard? If the idle routine(s) have way to yield
> (explicitly or by "tail going" an unrolled loop) AND if there are two
> queues, the normal queue and the idle queue AND if the scheduler never
> starts an idle queue entry unless there are no normal queue entries to run,
> then that's that, or at least it seems that way. Why need it be more
> complicated?
> Need it really be so hard? If the idle routine(s) have way to yield
> (explicitly or by "tail going" an unrolled loop) AND if there are two
> queues, the normal queue and the idle queue AND if the scheduler never
> starts an idle queue entry unless there are no normal queue entries to run,
> then that's that, or at least it seems that way. Why need it be more
> complicated?
It's just not a problem that Go is trying to solve.
That's what operating system schedulers are for.
Russ
Michael Jones
May 3, 2011, 11:23:40 AM5/3/11
to Russ Cox, ⚛, golang-nuts
Indeed it is--at least when one knows what goroutine-as-OS-priority-manageable-entity to do a "nice" on. We can agree then that the reason is philosophical rather than technical, and that implementation would be straightforward rather than cumbersome. I was not arguing for it, just surprised by all the complicated implementations being proposed. (I should admit that my proposal was slightly more complex than it seems, as "go x" would need a target run-queue or priority system and there is no happy syntactic place for that: go (slow) name(args) is awful. You would also have possible deadlock from starvation with perverse uses of such a scheduler.)
Rob 'Commander' Pike
May 3, 2011, 1:05:05 PM5/3/11
to Michael Jones, Russ Cox, ⚛, golang-nuts
If you run a goroutine when the program is idle, the program is no
longer idle, the goroutine can't run, and the world slips into a
wormhole.
longer idle, the goroutine can't run, and the world slips into a
wormhole.
-rob
Andrew Gerrand
May 3, 2011, 1:14:50 PM5/3/11
to Sunil S Nandihalli, golan...@googlegroups.com
It's a simple matter to have a goroutine that monitors the state of
various processes within your program (running in other goroutines),
and runs some kind of background process when the other processes have
concluded or are at rest.
various processes within your program (running in other goroutines),
and runs some kind of background process when the other processes have
concluded or are at rest.
The definition of what is a "process" and what is "idle" is
application-specific, though, and that's why there's no provision for
this in the language itself.
Andrew
Reply all
Reply to author
Forward
0 new messages