Why go plugin addresses do not load with go binary #49225

[Manoj Chauhan]

I did not understand the use of plugin. I can't use a plugin with different go versions even with minor version because plugin.open is failed. The only way which I found to use plugin'functions is through LD_PRELOAD.

I created a plugin using https://github.com/vladimirvivien/go-plugin-example
go build -buildmode=plugin -o eng/eng.so eng/greeter.go

Created a go binary using command:
go build -linkshared test.go

Load plugin with test binary:
export LD_PRELOAD=./eng.so; ./test
plugin addresses will be loaded into firstmoduledata structure of go runtime.

Created another go binary (which I want to use) using command:
go build test2.go

load plugin with test binary:
export LD_PRELOAD=./eng.so; ./test2

in this case, plugin addresses are not loaded into firstmoduledata. This is because addmoduledata() is not called.

I did not understand why addresses are not loaded in this case while if i load plugin using plugin.open then addresses are loaded for same go binary test2

Please suggest how to load plugin addresses into go runtime for this case. Can't I call addmoduledata in this case?

go versions 1.15,1.16,1.17

os - ubuntu 16

[Brian Candler]

On Monday, 1 November 2021 at 03:44:03 UTC chauhan...@gmail.com wrote:

I did not understand the use of plugin. I can't use a plugin with different go versions even with minor version because plugin.open is failed.

That is indeed correct, and is a major limitation of plugins in go.  See:

https://www.reddit.com/r/golang/comments/b6h8qq/is_anyone_actually_using_go_plugins/ejkxd2k/?utm_source=reddit&utm_medium=web2x&context=3

I'm not sure about your issue with LD_PRELOAD, but it may be that you need to set LD_LIBRARY_PATH instead (see "man ld.so")

[Manoj Chauhan]

If eng.so is under /usr/lib then I think it's not required to set LD_LIBRARY_PATH. My question was how to load plugin address in go runtime using LD_PRELOAD. I am using LD_PRELOAD because plug.open is failed. Is there any other solution?

[Brian Candler]

> My question was how to load plugin address in go runtime using LD_PRELOAD. I am using LD_PRELOAD because plug.open is failed. Is there any other solution?

When I tried it, I didn't need to set any environment variables (and certainly no LD_XXX variables).  Follow this tutorial:

https://medium.com/learning-the-go-programming-language/writing-modular-go-programs-with-plugins-ec46381ee1a9

It works fine for me:

ubuntu@builder:~/go-plugin-example$ cat /etc/lsb-release

DISTRIB_ID=Ubuntu

DISTRIB_RELEASE=18.04

DISTRIB_CODENAME=bionic

DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
ubuntu@builder:~/go-plugin-example$ ls

LICENSE  README.md  chi  eng  greeter.go  swe

ubuntu@builder:~/go-plugin-example$ go version

go version go1.16.6 linux/amd64

ubuntu@builder:~/go-plugin-example$ go build -buildmode=plugin -o eng/eng.so eng/greeter.go

ubuntu@builder:~/go-plugin-example$ go run greeter.go english

Hello Universe
ubuntu@builder:~/go-plugin-example$

If you're still using Ubuntu 16, note that it went obsolete nearly a year ago (unless you're paying for extended maintenance).

[Manoj Chauhan]

Hi Brian,

greeter.go calls plugin.open and it will be work only for one sub version of go. If plugin and binary are made in different sub versions of go1.15 then plugin.open will not work.  

[David Finkel]

On Mon, Nov 1, 2021 at 9:44 AM Manoj Chauhan <chauhan...@gmail.com> wrote:

Hi Brian,

greeter.go calls plugin.open and it will be work only for one sub version of go. If plugin and binary are made in different sub versions of go1.15 then plugin.open will not work.  

It sounds like you want to ignore a major warning on the use of plugins.

It's my understanding that go plugins' limitations on requiring that both the plugin and the main binary to be built with identical toolchains (same go version) comes from the lack of a stable internal ABI within Go binaries across versions. (also the possibility of runtime changes across versions).

Using LD_PRELOAD does not seem advisable as a way to bypass that limitation. (unfortunately, I don't have a better option other than to more tightly constrain your build system so both the original binary and the plugin are built with the same toolchain)

On Monday, November 1, 2021 at 6:55:02 PM UTC+5:30 Brian Candler wrote:

> My question was how to load plugin address in go runtime using LD_PRELOAD. I am using LD_PRELOAD because plug.open is failed. Is there any other solution?

When I tried it, I didn't need to set any environment variables (and certainly no LD_XXX variables).  Follow this tutorial:

https://medium.com/learning-the-go-programming-language/writing-modular-go-programs-with-plugins-ec46381ee1a9

It works fine for me:

ubuntu@builder:~/go-plugin-example$ cat /etc/lsb-release

DISTRIB_ID=Ubuntu

DISTRIB_RELEASE=18.04

DISTRIB_CODENAME=bionic

DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
ubuntu@builder:~/go-plugin-example$ ls

LICENSE  README.md  chi  eng  greeter.go  swe

ubuntu@builder:~/go-plugin-example$ go version

go version go1.16.6 linux/amd64

ubuntu@builder:~/go-plugin-example$ go build -buildmode=plugin -o eng/eng.so eng/greeter.go

ubuntu@builder:~/go-plugin-example$ go run greeter.go english

Hello Universe
ubuntu@builder:~/go-plugin-example$

If you're still using Ubuntu 16, note that it went obsolete nearly a year ago (unless you're paying for extended maintenance).

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/42e3d250-cb69-43b6-a297-7a56fcbe688en%40googlegroups.com.

[Brian Candler]

On Monday, 1 November 2021 at 13:44:28 UTC chauhan...@gmail.com wrote:

greeter.go calls plugin.open and it will be work only for one sub version of go. If plugin and binary are made in different sub versions of go1.15 then plugin.open will not work.  

That is correct, that is how it is designed.  Quoting from the link I posted before:

• The plugin compiler version must exactly match the program's compiler version. If the program was compiled with 1.11.4, it won't work to compile the plugin with 1.11.5. When distributing a program binary, you must communicate what the compiler version you used is.

• Any packages outside of the standard library that are used by both the plugin and the program must have their versions match exactly. This means that when distributing a program binary, you must communicate the exact versions of all dependencies
  

and:

"It seems that the intended use-case for plugins is in-tree plugins; when you have code you don't want to always include in the main binary, but you have no problem with it living in the same git repo and getting compiled at the same time. It does not seem to be a good way to let users/customers add in their own code; there are headaches if you try to use them that way; they have to sufficiently emulate your environment when compiling the plugin"

If you want a more generic way to do "plugin" type functionality, which is less tightly coupled, then I suggest you look at talking gRPC over a socket.  There is a go-plugin library for this from Hashicorp, which they use in Vault etc.

[Manoj Chauhan]

Thanks.