Inter-process shared memory: shmget, shmat, shmdt

[Paul Lalonde]

Hi all,
I have an app that needs to share a frame buffer (an image.RGBA's
Pix data) across multiple processes. Mmap is implemented in syscall,
but I can't use MAP_ANONYMOUS as some of the processes can't be forked
from my go code - the clients connect in an ad-hoc manner. So I'd
like to use the shm subsystem, though those syscalls are
unimplemented.
I've got a patch for linux_amd64 that works for my need, by
extending the syscall generating scripts to add the syscalls, defines,
etc., as per the others, and forming a new mmapper with a couple of
trampolines that use shmget/shmat to implement the mmap func, and
shmdt for munmap. Add Ftok() and you're pretty much done.
I know shared memory isn't a favorite of the Go authors, but is
there interest in accepting such a patch to the main tree? I have a
deadline looming, but I should be able to do the linux_x86 version in
short order if it's of interest; other platforms are harder for me.
Paul

[Adam Langley]

On Fri, Jan 13, 2012 at 10:44 PM, Paul Lalonde <paul.a....@gmail.com> wrote:
>   I have an app that needs to share a frame buffer (an image.RGBA's
> Pix data) across multiple processes.  Mmap is implemented in syscall,
> but I can't use MAP_ANONYMOUS as some of the processes can't be forked
> from my go code - the clients connect in an ad-hoc manner.  So I'd
> like to use the shm subsystem, though those syscalls are
> unimplemented.

Does POSIX shared memory not work for you? There's two shared memory
systems on modern UNIXes: the SysV shared memory (shmget and friends)
and POSIX shared memory (shm_open and friends).

POSIX shared memory is actually just creating a mmaping files in
/dev/shm. I think talking to the X server is the only reason to need
SysV shared memory these days.

POSIX shared memory can also be passed, capability-style, by using
file descriptors.

Cheers

AGL

[Ian Lance Taylor]

Paul Lalonde <paul.a....@gmail.com> writes:

Sounds good to me.

Ian

[Paul Lalonde]

My shmget-foo is from long ago.  I'll try it with shm_open - I don't see why that wouldn't work for my case, and will relieve code-churn on the devs as they sprint to Go 1 :-)

Paul

[Paul Lalonde]

I've got my implementation working with shm_open, however it's still far from a good situation:

On Darwin shm_open is a syscall, on Linux it's set up by opening a file in /dev/shm; that leaks platform-specific code into my app.  

A solution could involve adding an shm_open/shm_unlink implementation to syscall. 

Alternately, does "go build" understand platform specific code in the same way as the makefiles understood <file>_$(GOARCH)?

Then I could maintain a go-installable package implementing these shm_open/shm_unlink for various platforms.

Thanks,

Paul

On Sat, Jan 14, 2012 at 8:59 AM, Adam Langley <a...@golang.org> wrote:

[Adam Langley]

On Sun, Jan 15, 2012 at 4:39 PM, Paul Lalonde <paul.a....@gmail.com> wrote:
> On Darwin shm_open is a syscall, on Linux it's set up by opening a file in
> /dev/shm; that leaks platform-specific code into my app.
> A solution could involve adding an shm_open/shm_unlink implementation to
> syscall.

Ah, does darwin not have an equivalent of tmpfs?

> Alternately, does "go build" understand platform specific code in the same
> way as the makefiles understood <file>_$(GOARCH)?
> Then I could maintain a go-installable package implementing these
> shm_open/shm_unlink for various platforms.

Yes. For example, put:

// +build linux

Near the top of the file. (See src/pkg/exp/terminal/util.go)

Cheers

AGL

[Russ Cox]

On Sun, Jan 15, 2012 at 19:56, Adam Langley <a...@golang.org> wrote:
> Yes. For example, put:
>
> // +build linux

Also, if the file name ends in _linux.go then you get
this for free.

[Paul Lalonde]

Sweet.  Works a beaut.

Paul