Re: [go-nuts] Go threading model in C++

[Andreas Krennmair]

If you're interested in a coroutine system for C (I guess it will also work in C++) not unlike goroutines, libtask might be for you: http://swtch.com/libtask/

Cheers,
Andreas

On 15.07.2012, at 14:47, wojtek...@gmail.com wrote:

> Hi!
> I'm writing application, that aims at high performance data processing (2d, 3d and sound data).
> Because of that I want to write it in C++ (I want to get the fastest solution available with the finest control of compilation optimalizations, but of course not so "low level" as assembler). Additional I would love to have threads so light as threads in go.
>
> So the question is - is there any library that provides the functionality and speed of go threads in C++ ? (Go is written in C, so I suppose something like that migh exist).
>
> Thank you :)

[Dmitry Maluka]

On 07/15/2012 04:46 PM, wojtek...@gmail.com wrote:
> Additional libtask does not multiplex the coroutines onto threads. All
> coroutines run on single thread, while goroutines are multiplexed.
> This aspect is very important too. :(

You might look at libthread:
http://swtch.com/plan9port/man/man3/thread.html. It supports multiple OS
threads but doesn't multiplex coroutines automatically, so you would
need to explicitly distinguish between OS threads and cooperatively
scheduled coroutines.

But - it doesn't support Windows either.

[Ian Lance Taylor]

On Jul 15, 2012 10:11 AM, <wojtek...@gmail.com> wrote:
>
> 1) Gorutines are multiplexed automatically onto system threads?

Yes.

> 1.2) Are Gorutines somehow connected to Libtask/Libthread? (I see Libtask have something called channels, so the only difference is support of Windows and multiplexing between os threads?)

No.  The goroutines implementation is independent.

> 3) If Go is written in C is it using some libraries for its gorutines and channels or are they implemented in Go's source code?

They are implemented in the Go runtime sources, not in a separate library.

> 3.2) If they are implemented in the source code, would it be possible to use them "in some easy way" in custom C++ application?

No.  A key part is intercepting all system calls.

> 4) If Gorutines are further extension of libtask, why Google have not implemented them as separate library for C++?

System call issues.

Ian

[Ian Lance Taylor]

On Jul 15, 2012 10:44 AM, <wojtek...@gmail.com> wrote:
>
> Could you please provide further details to point 3.2? 
> Maybe can I use this "intercepting all system calls" logic in my custom application together with gorutines and channels? (I understend this could be a little bit stupid question, but I don't know deeply the architecture of go) 

The entire Go runtime library is written in Go and a relatively small amount of carefully written C and assembler.  Whenever a goroutine makes a system call that may block, the goroutine is removed from the scheduler for the duration of the system call, permitting other goroutines to run.

That would be harder to do in C, short of replacing or extensively intercepting the C system library.

> Where can I find documentation (if such exists) with more precise informations? 

src/pkg/{runtime,syscall}

Ian

[Dmitry Maluka]

On 07/15/2012 05:11 PM, wojtek...@gmail.com wrote:
> *2*) If I use libthread together with libtask I have to manually say
> what task are running on what threads? (or is it impossible to
> manually swtch tasks over threads?)

I believe there is no need to use libtask if you are already using
libthread. Read libthread manual for details. If you want networking
facilities of libtask, you can use dial which is in plan9port too:
http://swtch.com/plan9port/man/man3/dial.html.

> *3.2*) If they are implemented in the source code, would it be

> possible to use them "in some easy way" in custom C++ application?

You could read through libthread source and try to implement something
like that in your app.

Also quick googling for "CSP C libraries" reveals this:
http://www.cs.kent.ac.uk/projects/ofa/c++csp/. I know nothing about that
project, and I hate C++ (not C), but if you're ok with that horrible
language, maybe this link will be helpful to you.

[Jim Whitehead II]

On Sun, Jul 15, 2012 at 1:47 PM, <wojtek...@gmail.com> wrote:
> Hi!
> I'm writing application, that aims at high performance data processing (2d,
> 3d and sound data).
> Because of that I want to write it in C++ (I want to get the fastest
> solution available with the finest control of compilation optimalizations,
> but of course not so "low level" as assembler). Additional I would love to
> have threads so light as threads in go.
>
> So the question is - is there any library that provides the functionality
> and speed of go threads in C++ ? (Go is written in C, so I suppose something
> like that migh exist).
>
> Thank you :)

You could look at something like C++CSP2, which offers support for
hybrid and native threading models. It doesn't have the split stacks
like Go does, but you can tweak the stack size as necessary to keep it
small enough to be performant. You should find the style of
programming to be quite similar to Go, albeit slightly more verbose.

http://www.cs.kent.ac.uk/projects/ofa/c++csp/

- Jim

[Mathieu Lonjaret]

On Sun, Jul 15, 2012 at 4:11 PM, <wojtek...@gmail.com> wrote:

> 1) Gorutines are multiplexed automatically onto system threads?

> 1.2) Are Gorutines somehow connected to Libtask/Libthread? (I see Libtask
> have something called channels, so the only difference is support of Windows
> and multiplexing between os threads?)

The "connection", that I believe was not directly pointed out and that
maybe you missed, is that libthread comes from Plan 9, which both rob
and rsc worked on.

[Paulo Pinto]

You could have a look at Cilk Plus from Intel, which is being proposed
for C++17, and is already part of a gcc 4.7 branch,
http://software.intel.com/en-us/articles/intel-cilk-plus/

Just with libraries you have the Thread Building Blocks tasks and task
groups.
http://threadingbuildingblocks.org/documentation.php

On Windows you have the asynchronous agents and tasks from the
Concurrency
Runtime, http://msdn.microsoft.com/en-us/library/dd504870

If only looking for OS level APIs, then there is also the possibility
to use fibers
on Windows, but you have to handle the scheduling yourself.
http://developer.amd.com/documentation/articles/pages/1031200677.aspx

Most UNIX systems also offer special APIs to handle M:N threading
which
helps replicating go routines.

--
Paulo

[Uriel]

On Sun, Jul 15, 2012 at 7:09 PM, <wojtek...@gmail.com> wrote:
> I'll take a look at C++ CSP. During searching in web I come across library
> called Intel TBB (it seems quite popular). I've found an article that was
> comparing TBB tasks and Go's gorutines (link to article below) and according
> to this article, TBB's tasks are far more better than go's.
> Have you tried TBB or do you know how it compares to C++ CSP / libthread /
> libtask or if it has some downsides compared to go?
>
> article comparing go and tbb:
> http://www.google.pl/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0CFUQFjAC&url=http%3A%2F%2Fwww.ualr.edu%2Fpxtang%2Fpapers%2FACMSE12-comparing.pdf&ei=Z_ACUJCaGorRtAby8onJBg&usg=AFQjCNFVTuj3_jG_Gdw4_OKGJGK7h_Q5HA&sig2=TOynfjcMa6XHKifWnuK_aw

This article seems to be, at best, seriously misinformed. Just from
the introduction, this is plain wrong: "Go channels are the only
synchronization mechanism available to goroutines."

Haven't looked at the rest of the article, but I would take it and its
conclusions with a big rock of salt.

Uriel

[Øyvind Teig]

A thick "Go Very Missing Manual" with design rationale from the designers is missing.

The fact that some of the documentation is in the source code, some in a YouTube video, some in some interview somewhere and some at some blog will not make misunderstandings like in the referenced paper any less frequent.

In the long run it could be a problem for the acceptance of the language. Histories tend to be hard to kill or make right. And hard to admit and change.

- Øyvind

[Patrick Mylund Nielsen]

http://golang.org/pkg/

...

sync     Package sync provides basic synchronization primitives such as mutual exclusion locks.

...

http://golang.org/ref/mem

Synchronization

   - Locks

"The sync package implements two lock data types, sync.Mutex and sync.RWMutex...."

It's just sloppy work, and to an unusual degree for an academic paper.

Read the spec. Check out the APIs in the standard library. Read the questions and answers in the FAQ. All of this on http://golang.org. You don't have to read the source code or watch any YouTube videos to understand Go.

Both the specification and the standard library are very well documented. I don't buy that there's a lot of confusion about how Go works, just shorter attention spans in this, the Twitter age, than when people were learning C.

> A thick "Go Very Missing Manual" with design rationale from the designers is missing.

What do you want from this that's not already on the website? Anecdotes?

[Øyvind Teig]

You are right in that documentation is present. And I forgot to mention the primary source: golang.org.

As much as I like anecdotes about these things, that was not what I had in mind.

Stroustrup in his his C++ book has a lot of design rationale. A design description is not complete before one both knows why a certain trait was chosen, why something else was not chosen, and how compromises were made. And why something related was not included at all. This could add up to almost become anecdotes. But remember, the human brain remembers stories better than facts!

But ok then I can remove the "very", but I still mean that the "Go Missing Manual" as a comprehensive all included in one place is missing.

- Øyvibd

[kortschak]

There are quite a few papers from Rob and others that describe some of the thinking that has gone into why design decisions were made, but finding out exactly why some decisions have been made is not all that easy. I'd agree that it is very easy to find out the reason why things behave the way they do from the docs and source, but not the next level about why that behaviour was chosen.

I've occasionally found myself wondering why a particular choice was made, most of the time asking this will get an answer, but sometime not.

Hopefully at some time in the future there were be more documentation about this, but in the mean time I'm happy that the core devs are spending time on what they are, making the language work.

[Øyvind Teig]

Agree that they spend their time making the language work.

But I, for my part, can get more time doing the next thing by thoroughly explaining the previous thing. Better, it some times takes me a small step back to fix that little detail that I overlooked with only code around me.

Just bringing the white boards over to a document is often rewarding!

(They probably do all this, but I can't wait..)

- Øyvind

[Florian Weimer]

* wojtek danilo:

> Could you please provide further details to point 3.2?
> Maybe can I use this "intercepting all system calls" logic in my custom
> application together with gorutines and channels? (I understend this could
> be a little bit stupid question, but I don't know deeply the architecture
> of go)

> Where can I find documentation (if such exists) with more precise
> informations?

System call interceptin for threading has a long history. There was
the FSU Threads implementation in the 90s which provided a pthreads
implementation in user space, and before NPTL came along, various
folks experimented with n:m threading. GNU libc and ld.so should
still support this today, with the exception of cheap context switches
for TLS data.

[maciej.g...@gmail.com]

I'm working on exactly this.

The code is here: https://github.com/maciekgajewski/coroutines

And some human-readable text here: http://maciekgajewskiprogramming.blogspot.nl/2013/10/go-style-channels-and-coroutines-in-c.html

It's still work in progress, but it's already usable!

M

[Bienlein]

There is also libdispatch: http://en.wikipedia.org/wiki/Grand_Central_Dispatch

Apple: https://github.com/nickhutchinson/libdispatch
Linux: https://libdispatch.macosforge.org/

[Alexei Sholik]

I've heard that it is not recommended to run blocking operations on libdispatch queues. So, apparently, not really lightweight processes there.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Best regards
Alexei Sholik

[Andy Balholm]

On Sunday, October 6, 2013 9:38:28 AM UTC-7, alco wrote:

I've heard that it is not recommended to run blocking operations on libdispatch queues. So, apparently, not really lightweight processes there.

In some ways GCD is more like a multicore version of the node.js concurrency model.

[Bienlein]

Am Sonntag, 6. Oktober 2013 18:38:28 UTC+2 schrieb alco:

I've heard that it is not recommended to run blocking operations on libdispatch queues. So, apparently, not really lightweight processes there.

I tried this with HawtDispatch and you are right. Blocking tasks added to HawtDispatch queues prevent non-blocking tasks from running. HawtDispatch also really says in its focumentation that it is made with non-blocking io in mind.

As a work-around you can do things in libdispatch using an actor-style approach: If an actor has an empty queue it sits and waits for input and the scheduler does not assign any thread to an actor with an empty queue. But an actor never blocks. If it has nothing to do, it waits till some item is added to its work queue. And here waiting does not mean blocking. The actor just remains put until it receives some work.

[Alexei Sholik]

libdispatch simply delegates the responsibility of scheduling blocking calls to the programmer. Apart from queues, it has dispatch sources[1] that provide a low level API for subscribing to various signal-based or descriptor-based events. In a sense, it becomes an evented system like the one you have in Node.js, but with more than one thread. Definitely nothing in common with Go.

[1]: https://developer.apple.com/library/ios/DOCUMENTATION/General/Conceptual/ConcurrencyProgrammingGuide/GCDWorkQueues/GCDWorkQueues.html#//apple_ref/doc/uid/TP40008091-CH103-SW1

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.