Memory share between 2 executables

[Léa Galet]

Hi,

I have 2 executables in go (called exec1 and exec 2)

exec1 is the main application, loading exec2 as a grpc plugin

Exec 1 is the one making the calls to exec 2, so there's a possibility of downward communication

i wanted my exec2 to be able to update some values of an item passed from exec1 so i gave the item as a pointer (wanted to use the inline update property of pointers)

unfortunatly, in exec2 it's a copy of the pointer and not the pointer directly (i printed the pointer adresses to be sure)

i don't seem to find a definitive answer anywhere so

Is there a method to have exec2 use the same pointer as exec1?

[Robert Engels]

You can’t typically do this from user code. You should used shared memory or a memory mapped file, or socket communication. 

On Jul 25, 2025, at 7:20 PM, Léa Galet <leaesper...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/d7d3b535-e670-48e8-af1e-366dfb1d0b56n%40googlegroups.com.

[Jon Perryman]

Caveat, I'm not familiar with gRPC nor your implementation.

In addition to Robert's suggestions, have you considered a single executable using GoRoutines and channels. Maybe channels can be shared between 2 executables but I can't speak to that. You could run a test to see if it GO allows it. If so, my expectation would be that it's thread safe.

Maybe pipes can be passed to another executable and be thread safe?

Robert recommended shared memory or memory mapped file which aren't thread safe. You may need to make it thread safe.  

[Sharon Mafgaoker]

The Correct Approach: Return the Updated Value

The standard and correct way to handle this is to embrace the client-server nature of gRPC. The plugin (exec2) should perform its logic and then return the modified data back to the main application (exec1).

The workflow should be:

1. exec1 (Client): Calls a gRPC function on exec2, passing the current state of the item in the request.

2. exec2 (Server): Receives the item (as a copy), performs its updates on that copy.

3. exec2 (Server): Returns the fully updated item in the gRPC response.

4. exec1 (Client): Receives the response and updates its original item with the returned data.

Sharon Mafgaoker – Senior Solutions Architect 

M. 050 995 99 16 | sha...@cloud5.co.il

To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/600BA26A-8FEA-42E6-80D8-96A082B0BB9C%40ix.netcom.com.

[Jon Perryman]

> The standard and correct way to handle this is to embrace the client-server nature of gRPC.

I could easily be wrong but my interpretation is that exec1 does not involve gRPC. It's implied that exec1 and exec2 are in the same process because the op expects the pointer will reference the same storage without using shared memory. This is not unusual for C but GO requires the use of the UNSAFE class which should be avoided.

Without details, we are guessing at what the OP is attempting to achieve. We're assuming that gRPC is the right solution for the problem being solved.

On Sat, Jul 26, 2025 at 4:07 AM Sharon Mafgaoker <sha...@cloud5.co.il> wrote:

The Correct Approach: Return the Updated Value

 The plugin (exec2) should perform its logic and then return the modified data back to the main application (exec1).

The workflow should be:

1. exec1 (Client): Calls a gRPC function on exec2, passing the current state of the item in the request.

2. exec2 (Server): Receives the item (as a copy), performs its updates on that copy.

3. exec2 (Server): Returns the fully updated item in the gRPC response.

4. exec1 (Client): Receives the response and updates its original item with the returned data.

Sharon Mafgaoker – Senior Solutions Architect 

M. 050 995 99 16 | sha...@cloud5.co.il

[Robert Engels]

The OP doesn’t understand how gRPC works. It doesn’t have an in process mode that could share pointers. Everything is serialized and copied. The OP is confused because the generated gRPC code uses pointers in the stubs. 

On Jul 26, 2025, at 2:00 PM, Jon Perryman <jon.pe...@gmail.com> wrote:



[Jason E. Aten]

I'm assuming that Léa is using Hashicorp's gRPC plugin approach for Go 

(https://github.com/hashicorp/go-plugin) or similar, and that executable

really does mean a separate program in a separate process.

As Robert points out, it is possible to share memory if you really want to

between processes.

As Sharon points out, you probably should not. :) 

Rather you should change the signature of your remote procedure call to exec2 to return the

update you want made as a returned value, and let exec1 apply

the change itself.

If you are using Hashicorp's approach, the whole idea there is that your

main program can be fault-tolerant against badly behaved "plugins".

Thus while a traditional (C world) plugin would be in the form of a shared library

(an .so or DLL) loaded into the same process, in the Hashicorp gRPC

approach, your "plugin" is really a separate process. 

This trades off stability for a little less performance. The plugin process can crash,

or restart, or even be upgraded, without taking down the main exec1

program. That's a pretty big benefit, and the main reason folks use

that approach (plus you cannot unload DLLs from Go without crashing

the runtime, so its almost but not quite mandatory). 

The only tradeoff is that going over the network with gRPC is slightly slower

performance wise. Unless you are doing stuff where perf is critical,

and given the question that is unlikely, then the separate process 

approach of Hashicorp/gRPC to "plugins" is much nicer. In fact

the next step if you need perf is to move from TCP to unix domain

sockets (before looking at shared memory). Unix domain sockets

are fast enough for Chrome, for instance, and still give you nice

isolation/fault tolerance.

[Léa Galet]

Many thanks to all the people who commented

So as an update, i'll explain and then close the thread

With my colleagues we dived down deeper in the code of exec1 (not our code obviously, most of the structural changes offered couldn't be done as it would have been refused by the team in charge of exec1)

they use protobuff to communicate with exec2, asssigning a new variable with the protobuff type, hence why even if it looks like the same variable in exec2, it's actually not the same

As we can't update the Notifier interface (breaking change too big for the other team), using a shared memory space would have no effect since it's not the same variable at all

So our solution is easy for us, don't involve changes on exec1's code (that an other team ows) : we now use a storage in our exec2 and we match the data from exec1 with our local storage in exec2

Thanks to all again