Channels over pipes
Scott Lawrence
believe pipes are generally faster than IP-based sockets (supported by
netchan) (let me know if this is wrong). Is there any way to create a
channel over an arbitrary file descriptor?
--
Scott Lawrence
Archos
greater speed to lesser:
Shared memory (single thread) > Named pipes > Unix domain socket > TCP
socket
And there is a great difference of speed between pipes and sockets.
Russ Cox
> Respect to speed on IPC, here there are several of them sorted of
> greater speed to lesser:
>
> Shared memory (single thread) > Named pipes > Unix domain socket > TCP
> socket
I'd be careful about making such sweeping generalizations.
Much better to measure and see what you find.
It might also vary from operating system to operating system.
> And there is a great difference of speed between pipes and sockets.
There's a great difference between shared memory and not.
The others are not nearly so clear cut. There's no inherent
reason one would perform better than another.
Russ
Rob 'Commander' Pike
netchan over pipes would be nice, and easy. when i get my computing
world reassembled (don't ask) some netchan work will be high on my list.
-rob
Scott Lawrence
> There's a great difference between shared memory and not.
> The others are not nearly so clear cut. There's no inherent
> reason one would perform better than another.
Really? As I understand it, netchan will only work with TCP sockets -
unless the OS does some incredibly hacky stuff for the loopback
interface to avoid redundant chat, that should be way slower than
pipes. (I guess I'll have to look into this...)
> (don't ask)
I don't have to. I'll just go back to hating certain partitioning programs. ;-)
--
Scott Lawrence
Russ Cox
> On 8/26/10, Russ Cox <r...@golang.org> wrote:
>> There's a great difference between shared memory and not.
>> The others are not nearly so clear cut. There's no inherent
>> reason one would perform better than another.
>
> Really? As I understand it, netchan will only work with TCP sockets -
> unless the OS does some incredibly hacky stuff for the loopback
> interface to avoid redundant chat, that should be way slower than
> pipes. (I guess I'll have to look into this...)
Let us know what you find. Be sure to try multiple operating systems.
Russ
Scott Lawrence
(64-bit qemu/kvm) - I'll do OS X (64-bit) and 32-bit linux (VM)
shortly. In brief, pipes are at least twice as fast as local
inet-domain sockets, and often more than 5 times as fast (10,
occasionally). (I haven't tried with unix sockets yet - I suspect they
fall in between, probably closer to the pipes side.)
X86-64 LINUX
--------------------
For 4KB messages bounced back and forth 1000 times (measuring
latency), pipes clock in at 15ms, and sockets at 25ms. Up the
iterations to 10000, and for pipes its about 100ms, and sockets,
250ms.
40-KB messages, same scheme, the differences are more pronounced. Note
that the times aren't just multiplied by 10. 1000 iterations, 28
(pipes) and 180 (sockets) ms (yes, I'm averaging over multiple runs).
10000 iterations, 280ms and about a second.
Now for large messages, sent back and forth 10 times (latency is no
longer a factor).
4MB: 30ms and 100ms.
40MB: 293ms, 580ms.
qemu-system-x86_64 (w/ kvm) FreeBSD
------------------------------------------------------------
message-size iterations : pipems socketms
4KB 1000 : 13 50
4KB 10000 : 131 500
40KB 1000 : 100 700
40KB 10000 : 300 1600
4MB 10 : 80 180
40MB 10 : 250 2200
Conclusion: we need channels over pipes ;-)
The program I used: http://gist.github.com/555626 - let me know if you
receive drastically different results (or a segfault - especially a
segfault). I can't figure out how to close the socket correctly (I've
never liked sockets), so you have to change the port number in between
runs. Feel free to fork and fix that. Note that I send everything in
4096-byte chunks - you may be able to improve performance by tweaking
that. Let us know.
I'll post more thorough data (more operating systems and some pretty
pictures) sometime.
On 8/26/10, Russ Cox <r...@golang.org> wrote:
> On Thu, Aug 26, 2010 at 22:28, Scott Lawrence <byt...@gmail.com> wrote:
>> On 8/26/10, Russ Cox <r...@golang.org> wrote:
>>> There's a great difference between shared memory and not.
>>> The others are not nearly so clear cut. There's no inherent
>>> reason one would perform better than another.
>>
>> Really? As I understand it, netchan will only work with TCP sockets -
>> unless the OS does some incredibly hacky stuff for the loopback
>> interface to avoid redundant chat, that should be way slower than
>> pipes. (I guess I'll have to look into this...)
>
> Let us know what you find. Be sure to try multiple operating systems.
>
> Russ
>
--
Scott Lawrence
Archos
communication (bidirectional) which, I'm supposed, is better to use in
channels.
A pipe is a one-way mechanism. And if we want a two-way communication,
we'll need two pipes, and a lot of caution.
Eleanor McHugh
On the other hand this is a well-known pattern that's been being used in Unix software for decades so whilst caution is certainly necessary in getting the implementation right, it shouldn't be beyond the wit of many people here.
Now Channels over shared memory... well that would really be a challenge ;)
Ellie
Eleanor McHugh
Games With Brains
http://feyeleanor.tel
----
raise ArgumentError unless @reality.responds_to? :reason
David Leimbach
On Aug 29, 4:14 am, Eleanor McHugh <elea...@games-with-brains.com>
wrote:
>
> >> Conclusion: we need channels over pipes ;-)
>
> > I only wanted to remind one thing, and is that sockets can do two way
> > communication (bidirectional) which, I'm supposed, is better to use in
> > channels.
>
> > A pipe is a one-way mechanism. And if we want a two-way communication,
> > we'll need two pipes, and a lot of caution.
>
> On the other hand this is a well-known pattern that's been being used in Unix software for decades so whilst caution is certainly necessary in getting the implementation right, it shouldn't be beyond the wit of many people here.
>
> Now Channels over shared memory... well that would really be a challenge ;)
something reasonable? The difference between OS threads and processes
on some systems is pretty small. At least it's small enough that I
wouldn't want to use complicated shared memory coding between forked
processes unless there was a really good reason to do so.
>
> Ellie
>
> Eleanor McHugh
> Games With Brainshttp://feyeleanor.tel
Eleanor McHugh
> On Aug 29, 4:14 am, Eleanor McHugh <elea...@games-with-brains.com>
> wrote:
>> On 29 Aug 2010, at 00:34, Archos wrote:
>>
>>>> Conclusion: we need channels over pipes ;-)
>>
>>> I only wanted to remind one thing, and is that sockets can do two way
>>> communication (bidirectional) which, I'm supposed, is better to use in
>>> channels.
>>
>>> A pipe is a one-way mechanism. And if we want a two-way communication,
>>> we'll need two pipes, and a lot of caution.
>>
>> On the other hand this is a well-known pattern that's been being used in Unix software for decades so whilst caution is certainly necessary in getting the implementation right, it shouldn't be beyond the wit of many people here.
>>
>> Now Channels over shared memory... well that would really be a challenge ;)
>
> What's the advantage of that over using goroutines + GOMAXPROC set to
> something reasonable? The difference between OS threads and processes
> on some systems is pretty small. At least it's small enough that I
> wouldn't want to use complicated shared memory coding between forked
> processes unless there was a really good reason to do so.
Who would? But if speed of data blitting between processes is critical then shared memory may well be the right tool for the job despite its many potential pitfalls.
David Leimbach
On Aug 29, 5:46 pm, Eleanor McHugh <elea...@games-with-brains.com>
> On 29 Aug 2010, at 21:38, David Leimbach wrote:
>
>
>
>
>
> > On Aug 29, 4:14 am, Eleanor McHugh <elea...@games-with-brains.com>
> > wrote:
> >> On 29 Aug 2010, at 00:34, Archos wrote:
>
> >>>> Conclusion: we need channels over pipes ;-)
>
> >>> I only wanted to remind one thing, and is that sockets can do two way
> >>> communication (bidirectional) which, I'm supposed, is better to use in
> >>> channels.
>
> >>> A pipe is a one-way mechanism. And if we want a two-way communication,
> >>> we'll need two pipes, and a lot of caution.
>
> >> On the other hand this is a well-known pattern that's been being used in Unix software for decades so whilst caution is certainly necessary in getting the implementation right, it shouldn't be beyond the wit of many people here.
>
> >> Now Channels over shared memory... well that would really be a challenge ;)
>
> > What's the advantage of that over using goroutines + GOMAXPROC set to
> > something reasonable? The difference between OS threads and processes
> > on some systems is pretty small. At least it's small enough that I
> > wouldn't want to use complicated shared memory coding between forked
> > processes unless there was a really good reason to do so.
>
> Who would? But if speed of data blitting between processes is critical then shared memory may well be the right tool for the job despite its many potential pitfalls.
>
I would not reach for a garbage collected programming language to
begin with :-).
Then again, you never know.
Dave
Albert Strasheim
A few comments...
On Aug 29, 12:25 am, Scott Lawrence <byt...@gmail.com> wrote:
> The program I used:http://gist.github.com/555626- let me know if you
> segfault). I can't figure out how to close the socket correctly (I've
> never liked sockets), so you have to change the port number in between
> runs. Feel free to fork and fix that. Note that I send everything in
> 4096-byte chunks - you may be able to improve performance by tweaking
> that. Let us know.
See
http://home.gna.org/pysfst/tests/pipe-limit.html
Also, Linux has some interesting syscalls related to pipes, namely
splice, vmsplice and tee. Maybe these could be useful.
And some more benchmarks:
http://blogs.gnome.org/abustany/2010/05/20/ipc-performance-the-return-of-the-report/
Regards,
Albert
David/Jones
situation and the GC).
This judgement also depends on how much of the performance of the
application depends on communication, rather than the GC.
> What's the advantage of that over using goroutines + GOMAXPROC set to
> something reasonable?
sure if channels and shared memory are actually a good approach there,
because they restrict communication to a Go2Go communication.
Allowing IPC between Go and C and Ruby and so on would be a very
interesting concept, if you're asking me.
I think that it would have a serious effect on performance, though.
Just my two cents,
David
Carl
> I suppose if I was in that situation, where I needed such performance,
> I would not reach for a garbage collected programming language to
> begin with :-).
Go Language is to solve exactly this problem, to develope a state of
the art garbage collector that runs in its own core and can do its
thing with minimal overall impact on performance.
Also I think most, if not all garbage collected languages, run in a VM
of sorts, on an Interpreter or with JIT of somekind.
Go Language does not do that, its runs native mode, one of the design
goals is to run in optimized native mode. Thats very fast. Off course
absolute speed will depend on the quality of optimization, which
correlates to the quality of the compiler, which correlates to the
depth of the skill set and the amount of work invested.
The question that comes to my mind in this discussion is: What about
context switching? Every time you have to switch to another process
there is a lot that happens on the hardware and operating system
level, how can that be avoided? I suppose it depends in no small part
on the OS design.
Cheers.
David Roundy
> Also I think most, if not all garbage collected languages, run in a VM
> of sorts, on an Interpreter or with JIT of somekind.
Actually, quite a few languages are garbage collected and compiled to
native code. Haskell, Ocaml and the whole ML family come to mind
(since I've been immersed in functional languages for a while), but I
believe D also falls in this category. You may be right about most
recent languages running in a VM, interpreter or JIT, though.
--
David Roundy
Ian Lance Taylor
> The question that comes to my mind in this discussion is: What about
> context switching? Every time you have to switch to another process
> there is a lot that happens on the hardware and operating system
> level, how can that be avoided? I suppose it depends in no small part
> on the OS design.
Having a garbage collector which runs on its own core but which shares
memory with other goroutines is not a full-on context switch. It only
requires changing registers, not the page maps. This kind of context
switch is not free, but the cost is more on the order of a couple of
function calls, plus the memory cache hit. No operating system calls
are involved.
Ian
Cory Mainwaring
> If I understand it correctly then one of the major design goals of the
> Go Language is to solve exactly this problem, to develope a state of
> the art garbage collector that runs in its own core and can do its
> thing with minimal overall impact on performance.
There is a niche to be filled: a language that has the good,
avoids the bad, and is suitable to modern computing
infrastructure:
- comprehensible
- statically typed
- light on the page
- fast to work in
- scales well
- doesn't require tools, but supports them well
So no, making a garbage collector was not a design goal of the
language. Designing a garbage collector helps some of the design
goals, but it's not in and of itself one. So, the garbage collector is
lower on the list than the compilers and bugs, and it also explains
why there's been such simple garbage collection thus far.
David Leimbach
space (I guess).
The majority of my background is HPC where people still write code in
C, Fortran etc. Cache efficiency was an extremely important part of
making code run well with certain benchmarks, and code like Kazushige
Goto's implementation of BLAS, which was highly cache optimized for
different processor types were used to "win" on the top500
supercomputer list, was common to find.
You're not going to see go code, but you might see Goto's code :-).
For garbage collection to be viable in the really high end HPC space,
you have to be able to make reasonable predictions about when it would
occur, almost to the point where I think you'd be better off
triggering it yourself rather than guessing, at which point you're
back to wanting manual memory allocation.
The same problems exist in real time systems with garbage collection,
except the penalties are worse when you miss a deadline in some real
time systems vs just not winning a benchmark competition.
So as good as GCs look on paper, I remain very skeptical that we'll
see them in HPC code for some time.
Dave
Skip Tavakkolian
a net.Conn or perhaps a (new) Dialer interface would do it. Am I on
the right track?
-Skip
David Roundy
> And it's those cache hits that will keep Go out of the extreme HPC
> space (I guess).
It depends on the HPC, but most often large-memory problems involve...
large arrays. In such a case, there would be very little cache
penalty, since the GC wouldn't need to scan any of the large arrays
(since go is type safe, it can tell there aren't any pointers hiding
in there). Also, most reasonably-optimized go code won't do any
allocations in the core of the code, and thus won't require any gc.
The real problem with GC in HPC seems to me the question of
unpredictable lifetimes. Many (most) expensive algorithms do no
allocation (e.g. all of LAPACK), so it's not a problem. But there are
lots of cheap O(N) operations where I would like the convenience of
automatically-tracked temporaries, and there having GC seems likely to
fail miserably (as in, use too much memory).
That said, go is clearly not targeting the HPC market, and I doubt
it'll appeal much either with or without the overhead of garbage
collection.
David Roundy