As I understand it, ed25519 is using Curve25519 in EdDSA, which is a signing scheme. So using "ed25519" for encryption does not make any sense.
NaCl also uses Curve25519, ultimately using ECDH (again, as I understand it) to establish a secret key for Salsa20. So it is pretty fundamentally different than ed25519.
Note that
golang.org/x/crypto/nacl *is* a pure Go package, not a libsodium wrapper. And yes, it's very likely what you want, if you want to use Curve25519 for encryption. Unless you want to roll your own cryptography, in which case, here be dragons. But the `crypto/ecdh` package (available since Go 1.20) would probably the primitive to look at.