Mimicking Goroutines in other languages

[Bienlein]

Hello,

I lately sat down to find a way to have thread multiplexing + channels in Java. The intention was not to demonstrate how to steal ideas and bring them to other languages... I like the idea of Goroutines and I will have to stick to Java for a while I believe. So I looked for some other way to get the same thing accomplished. That's why. I wrote down my approach in this blog (it is basically about leveraging HawtDispatch).

Now my question is what you think about this approach, because I have some gut feeling that I missed out on some things. The proposed solutions does not support selects on channels as Go; that I'm aware of. But I wonder whether that's all I missed as my understanding of CSP is (still) somewhat limited ...

-- Bienlein

[Dave Cheney]

Select is kind of important, otherwise all you have is a thread safe ring buffer. 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Bienlein]

Am Donnerstag, 20. Juni 2013 14:19:35 UTC+2 schrieb Dave Cheney:

Select is kind of important, otherwise all you have is a thread safe ring buffer. 

Okay, I fear I don't really understand what selects on channels are needed for: have only 1 goroutine but several channels in order to:

1. distribute load onto several channels
2. distribute logic over several channels

Is 1. or 2. correct? Or both?

Thanks, Bienlein

[andrey mirtchovski]

alef and limbo are touted as precursors to Go, both support select.
the Plan 9 "libthread" library, available on unix from plan9port, has
'alt' which does a reasonable job:

http://plan9.bell-labs.com/magic/man2html/2/thread

this is a reasonably definitive reference to all four languages
mentioned and their relationship with CSP:

http://swtch.com/~rsc/thread/

[andrey mirtchovski]

sorry if unclear, alt and libthread are C, for those keeping count.

[Robert Melton]

On Thu, Jun 20, 2013 at 8:17 AM, Bienlein <jet...@web.de> wrote:

Hello,

I lately sat down to find a way to have thread multiplexing + channels in Java. The intention was not to demonstrate how to steal ideas and bring them to other languages... I like the idea of Goroutines and I will have to stick to Java for a while I believe. So I looked for some other way to get the same thing accomplished. That's why. I wrote down my approach in this blog (it is basically about leveraging HawtDispatch).

It seems the natural compliment to something like HawtDispatch would be a fast messaging / select library like ZMQ that allows for communication between multiplexed workers. 

-- 

Robert Melton

[atomly]

http://akka.io/

:: atomly ::

[ ato...@atomly.com : www.atomly.com  : http://blog.atomly.com/ ...
[ atomiq records : new york city : +1.347.692.8661 ...
[ e-mail atomly-new...@atomly.com for atomly info and updates ...

On Thu, Jun 20, 2013 at 8:17 AM, Bienlein <jet...@web.de> wrote:

-- Bienlein

--

[Bienlein]

Am Donnerstag, 20. Juni 2013 14:19:35 UTC+2 schrieb Dave Cheney:

Select is kind of important, otherwise all you have is a thread safe ring buffer. 

Okay, so I would implement a select by creating a subclass of the BlockingQueue where the add method is overriden so that the element is added to the queue, then index/name of the queue is added to some internal queue of the select mechanism. Last some semaphore is signaled so that some thread is released which picks the queue index/name from the internal queue and then serves the referenced queue by taking its first element and calling the handler for that queue with the taken element.When done the semaphore is closed again.

Wonder that is efficient/feasible ...

[atomly]

Have you looked at Akka at all?  It even actually uses HawtDispatch for its "faster" dispatcher, according to the HawtDispatch website.

It does everything you want and more, trust me.

:: atomly ::

[ ato...@atomly.com : www.atomly.com  : http://blog.atomly.com/ ...
[ atomiq records : new york city : +1.347.692.8661 ...
[ e-mail atomly-new...@atomly.com for atomly info and updates ...

--

[Bienlein]

Am Donnerstag, 20. Juni 2013 15:13:52 UTC+2 schrieb atomly:

Have you looked at Akka at all?  It even actually uses HawtDispatch for its "faster" dispatcher, according to the HawtDispatch website.

It does everything you want and more, trust me.

:: atomly ::

Man, how I hate this. All those "great ideas" I have and every time someone else had them long time before me already ...

[atomly]

That's true, but try to see the silver lining-- Akka has so many cool, advanced features that, just by reading up on it, you'll probably come up with at least one new great idea for a project.

:: atomly ::

[ ato...@atomly.com : www.atomly.com  : http://blog.atomly.com/ ...
[ atomiq records : new york city : +1.347.692.8661 ...
[ e-mail atomly-new...@atomly.com for atomly info and updates ...

--

[Andrew Francis]

Hi Bienlen

On Thursday, June 20, 2013 8:17:36 AM UTC-4, Bienlein wrote:

Hello,

 

Now my question is what you think about this approach, because I have some gut feeling that I missed out on some things. The proposed solutions does not support selects on channels as Go; that I'm aware of. But I wonder whether that's all I missed as my understanding of CSP is (still) somewhat limited ...

I am not well versed in CSP's formalism. However I think in CSP,  select is represented by the (none-deterministic) CHOICE (OR) operation. 

What I believe is interesting to an implementer is if you look at Russ Cox's Plan 9 channel implementation, or for that matter read Rob Pike's "The Implementation of Newsqueak," you see that select is *intrinsic* to channel operations. Intrinsic in the sense that a channel send and a channel receive are implemented as a select (alt) with one operator. In short, one ought to implement the select algorithm from the get-go. Select/Alt isn't an add-on. I haven't looked at Go's code lately, but it may deviate from this model in a few ways for efficiency.

My experiences: I discovered Go because I play with Stackless Python. Stackless Python has a concurrency model that was influenced by Denis Ritchie's "Descent into Limbo." However Stackless Python did not implement select. Using PyPy, a Python based toolchain, it was a snap for me to write a prototype using Select. The select algorithm (as it is in the Plan9 Channel code) is all that and a bag of chips. Much to my surprise, select can be extended to support join patterns. 

Cheers,

Andrew

[Øyvind Teig]

Great idea, especially in the view of what Per Brich Hansen once wrote about Java's faulty implementation of the monitor concept. 

Have a look at "Communicating Sequential Processes for JavaTM (JCSP)" at http://www.cs.kent.ac.uk/projects/ofa/jcsp/. This was developed, starting about 15 years ago, by the team at University of Kent at Canterbury in the UK, and I believe it has been stable for quite some years. It may help you with some ideas for your project. They also had some part of it formally proven with (I assume) a model in CSPm.

Øyvind 

[Øyvind Teig]

Per Brinch Hansen, sorry

[John Nagle]

Unless you need some incredibly huge number of semi-concurrent
activities, one thread per concurrent activity works fine. The
use case for a goroutine-like mechanism is when you have a very
large number of things waiting, like persistent HTTP connections.
Google has that problem, hence Go, but most applications don't.

Bounded buffers aren't hard to implement. Here's a
classic implementation, from 1972:

http://www.fourmilab.ch/documents/univac/fang/hsource/scheduler.asm.html

It's cleaner than most of its successors.

The innovation in Go is "select" which waits on any of a set of bounded
buffers. That's hard to do efficiently. Take a look a the library
code for the general N-channel case, which is painful and slow.

There's a common special case which can be implemented efficiently.
If all the channels read in an N-channel select are read only in that
select, and all the channels written in that select are written only in
that select, there's a big potential simplification. Instead of having
one P/V lock for each channel at the relevant end, there can be one
lock for all those channel ends. When the selecting thread unblocks,
at least one of the channels will have work (read data or a write slot)
available.

That's an optimization Go could do by static analysis. Does it?

John Nagle

[Ian Lance Taylor]

On Sat, Jun 22, 2013 at 1:41 PM, John Nagle <na...@animats.com> wrote:
>
> There's a common special case which can be implemented efficiently.
> If all the channels read in an N-channel select are read only in that
> select, and all the channels written in that select are written only in
> that select, there's a big potential simplification. Instead of having
> one P/V lock for each channel at the relevant end, there can be one
> lock for all those channel ends. When the selecting thread unblocks,
> at least one of the channels will have work (read data or a write slot)
> available.
>
> That's an optimization Go could do by static analysis. Does it?

Not at present. It would be nice, though.

Ian

[Bienlein]

Hi Andrew,

interesting read!

What I believe is interesting to an implementer is if you look at Russ Cox's Plan 9 channel implementation, or for that matter read Rob Pike's "The Implementation of Newsqueak," you see that select is *intrinsic* to channel operations. Intrinsic in the sense that a channel send and a channel receive are implemented as a select (alt) with one operator. In short, one ought to implement the select algorithm from the get-go. Select/Alt isn't an add-on. I haven't looked at Go's code lately, but it may deviate from this model in a few ways for efficiency.

Firtst I thought "Rob Pike would write a paper about Newspeak?". Newspeak is some kind of Smalltalk-descendent language. Then I realized the language is named Newsqueak! What a difference a single character may make ;-).

My experiences: I discovered Go because I play with Stackless Python. Stackless Python has a concurrency model that was influenced by Denis Ritchie's "Descent into Limbo." However Stackless Python did not implement select. Using PyPy, a Python based toolchain, it was a snap for me to write a prototype using Select. The select algorithm (as it is in the Plan9 Channel code) is all that and a bag of chips. Much to my surprise, select can be extended to support join patterns. 

I didn't knew about Limbo and Stackless Python. Some interesting hint for me.