Deploy a Go application that depends on private modules

847 views
Skip to first unread message

George Moschovitis

unread,
Feb 24, 2019, 12:41:32 PM2/24/19
to google-appengine-go
I have a small Go application that depends on a couple of private modules. For development I use `replace (...)` in go.mod.
Alternatively the modules could be fetched from a private git repo.

How could I deploy such an application to Google App Engine (go111 runtime)? Is this even possible?

-g.

George Moschovitis

unread,
Feb 24, 2019, 2:00:59 PM2/24/19
to google-appengine-go
Btw, I noticed that I can run:

go mod vendor
rm go.mod go.sum

copy the directory within $GOPATH

and deploy successfully (i.e. the vendor dir is used).

Is there any less 'hackish' way to do it?

-g.

George Moschovitis

unread,
Feb 24, 2019, 4:09:41 PM2/24/19
to google-appengine-go
One solution would be to have the App Engine build system ignore go.mod (and use the vendor directory instead) if $GO111MODULE is set to off in app.yaml:

runtime: go111
env_variables: 
  GO111MODULE: off

-g.

dir Ls

unread,
Feb 26, 2019, 10:21:49 PM2/26/19
to google-appengine-go
I assumed this will work. In my case I have two services each in a directory within the app root directory. I have the following directory structure.

app-root/
app-root/go/
app-root/service1/
app-root/service2/

And in the go.mod of serice1 and service2 I have been using the require option to refer to the local path to resolve the common code. It seems like this strategy won't work when the services are deployed? If so, how to share code between services?

George Moschovitis

unread,
Feb 27, 2019, 2:28:36 AM2/27/19
to dir Ls, google-appengine-go
Nope it's not working. Perhaps someone from Google can chime in?
This is an important use-case.

-g.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/e82c01aa-fad9-4c2b-92e0-0c9512d51f29%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

George Moschovitis

unread,
Feb 27, 2019, 11:39:51 AM2/27/19
to google-appengine-go
rm go.mod go.sum

Slight improvement, no need to rm the files, you can add them in .gcloudignore

Still need to copy the dir within $GOPATH :(

dir Ls

unread,
Mar 7, 2019, 11:02:32 AM3/7/19
to google-appengine-go
Have you tried setting the environment variable GOFLAGS=-mod=vendor . It works locally, don't know about the deployment.

George Moschovitis

unread,
Mar 8, 2019, 1:48:10 AM3/8/19
to google-appengine-go
I am not sure where to set this flag for deployment. In app.yaml? Before gcloud app deploy?.

-g.

dir Ls

unread,
Mar 8, 2019, 2:09:25 AM3/8/19
to google-appengine-go
I set it on the terminal before launching the app dev server. If you turn on the server with --dev_appserver_log_level=debug you will see the go build command and the environment variables that it uses which includes the GOFLAGS.

George Moschovitis

unread,
Mar 8, 2019, 7:59:32 AM3/8/19
to google-appengine-go
Doesn't work. I tried it both in app.yaml

runtime: go111
env_variables:
GOFLAGS:-mod=vendor

and before gcloud app deploy:

GOFLAGS=-mod=vendor gcloud app deploy...

but it didn't work :(

-g.

dir Ls

unread,
Mar 13, 2019, 11:52:50 PM3/13/19
to google-appengine-go
Hello George, could you give this approach


a try?

Kun Li

unread,
Sep 12, 2019, 4:28:42 PM9/12/19
to google-appengine-go
Is there any update on this deployment flow? I am using go112 runtime and also need to handle the go.mod thing every single time. It's really annoying and with the release of the go113, all the dev repo will be based on the module, it just becomes even more annoying.


On Wednesday, February 27, 2019 at 12:28:36 AM UTC-7, George Moschovitis wrote:
Nope it's not working. Perhaps someone from Google can chime in?
This is an important use-case.

-g.

On Wed, Feb 27, 2019 at 5:21 AM dir Ls <my.s...@gmail.com> wrote:
I assumed this will work. In my case I have two services each in a directory within the app root directory. I have the following directory structure.

app-root/
app-root/go/
app-root/service1/
app-root/service2/

And in the go.mod of serice1 and service2 I have been using the require option to refer to the local path to resolve the common code. It seems like this strategy won't work when the services are deployed? If so, how to share code between services?

On Sunday, February 24, 2019 at 9:41:32 AM UTC-8, George Moschovitis wrote:
I have a small Go application that depends on a couple of private modules. For development I use `replace (...)` in go.mod.
Alternatively the modules could be fetched from a private git repo.

How could I deploy such an application to Google App Engine (go111 runtime)? Is this even possible?

-g.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

dir Ls

unread,
Sep 13, 2019, 3:07:26 PM9/13/19
to google-appengine-go
Go 1.12 resolved the issue of trying to fetch the url even though there is a replace pointing to local directory.


With that fix, I was hoping this issue is going to be resolved. Isn't that the case?

Steven Buss

unread,
Sep 13, 2019, 4:45:26 PM9/13/19
to dir Ls, google-appengine-go
Go module replace directives work correctly, but you cannot traverse _up_. eg, `replace foo => ../path/to/foo` won't work. All paths must be in a subdirectory of your app.yaml and go.mod. Note, though, that we *do* follow symlinks.

On Fri, Sep 13, 2019 at 12:07 PM dir Ls <my.st...@gmail.com> wrote:
Go 1.12 resolved the issue of trying to fetch the url even though there is a replace pointing to local directory.


With that fix, I was hoping this issue is going to be resolved. Isn't that the case?

On Thursday, September 12, 2019 at 1:28:42 PM UTC-7, Kun Li wrote:
Is there any update on this deployment flow? I am using go112 runtime and also need to handle the go.mod thing every single time. It's really annoying and with the release of the go113, all the dev repo will be based on the module, it just becomes even more annoying.

On Wednesday, February 27, 2019 at 12:28:36 AM UTC-7, George Moschovitis wrote:
Nope it's not working. Perhaps someone from Google can chime in?
This is an important use-case.

-g.

On Wed, Feb 27, 2019 at 5:21 AM dir Ls <my.s...@gmail.com> wrote:
I assumed this will work. In my case I have two services each in a directory within the app root directory. I have the following directory structure.

app-root/
app-root/go/
app-root/service1/
app-root/service2/

And in the go.mod of serice1 and service2 I have been using the require option to refer to the local path to resolve the common code. It seems like this strategy won't work when the services are deployed? If so, how to share code between services?

On Sunday, February 24, 2019 at 9:41:32 AM UTC-8, George Moschovitis wrote:
I have a small Go application that depends on a couple of private modules. For development I use `replace (...)` in go.mod.
Alternatively the modules could be fetched from a private git repo.

How could I deploy such an application to Google App Engine (go111 runtime)? Is this even possible?

-g.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/c46e7b65-8436-4177-9b8d-659f453e3e37%40googlegroups.com.

Kun Li

unread,
Sep 16, 2019, 1:58:34 PM9/16/19
to Steven Buss, dir Ls, google-appengine-go
The symlinks hint solves the problem. Thanks!

It would be great if this is documented.

----------------------------------------------------------------------------------
Yours Sincerely
Kun
> To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/CAN_t6Fu%3DSv-e_JYuBB7nAm_9mbhcz%2BkvM48nHyxNmrqHiW5M8A%40mail.gmail.com.

Chaoming Li

unread,
Nov 1, 2019, 2:36:07 PM11/1/19
to google-appengine-go
Is it possible to copy the shared local packages from its folder to vendor folder in cloudbuild.yaml?


On Saturday, September 14, 2019 at 6:45:26 AM UTC+10, Steven Buss wrote:
Go module replace directives work correctly, but you cannot traverse _up_. eg, `replace foo => ../path/to/foo` won't work. All paths must be in a subdirectory of your app.yaml and go.mod. Note, though, that we *do* follow symlinks.

On Fri, Sep 13, 2019 at 12:07 PM dir Ls <my.s...@gmail.com> wrote:
Go 1.12 resolved the issue of trying to fetch the url even though there is a replace pointing to local directory.


With that fix, I was hoping this issue is going to be resolved. Isn't that the case?

On Thursday, September 12, 2019 at 1:28:42 PM UTC-7, Kun Li wrote:
Is there any update on this deployment flow? I am using go112 runtime and also need to handle the go.mod thing every single time. It's really annoying and with the release of the go113, all the dev repo will be based on the module, it just becomes even more annoying.

On Wednesday, February 27, 2019 at 12:28:36 AM UTC-7, George Moschovitis wrote:
Nope it's not working. Perhaps someone from Google can chime in?
This is an important use-case.

-g.

On Wed, Feb 27, 2019 at 5:21 AM dir Ls <my.s...@gmail.com> wrote:
I assumed this will work. In my case I have two services each in a directory within the app root directory. I have the following directory structure.

app-root/
app-root/go/
app-root/service1/
app-root/service2/

And in the go.mod of serice1 and service2 I have been using the require option to refer to the local path to resolve the common code. It seems like this strategy won't work when the services are deployed? If so, how to share code between services?

On Sunday, February 24, 2019 at 9:41:32 AM UTC-8, George Moschovitis wrote:
I have a small Go application that depends on a couple of private modules. For development I use `replace (...)` in go.mod.
Alternatively the modules could be fetched from a private git repo.

How could I deploy such an application to Google App Engine (go111 runtime)? Is this even possible?

-g.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

James Percy

unread,
Jan 6, 2020, 1:42:58 PM1/6/20
to google-appengine-go
hmmm, this still seems over-complicated to deploy an app which builds and runs fine locally? 

Really hoping Google finds a better way of doing this. The symlink and replace option is an extra bit of friction each developer will have to discover. Another hurdle preventing them from getting their code out there (for me one of the most value props of appengine)


On Saturday, 14 September 2019 06:45:26 UTC+10, Steven Buss wrote:
Go module replace directives work correctly, but you cannot traverse _up_. eg, `replace foo => ../path/to/foo` won't work. All paths must be in a subdirectory of your app.yaml and go.mod. Note, though, that we *do* follow symlinks.

On Fri, Sep 13, 2019 at 12:07 PM dir Ls <my.s...@gmail.com> wrote:
Go 1.12 resolved the issue of trying to fetch the url even though there is a replace pointing to local directory.


With that fix, I was hoping this issue is going to be resolved. Isn't that the case?

On Thursday, September 12, 2019 at 1:28:42 PM UTC-7, Kun Li wrote:
Is there any update on this deployment flow? I am using go112 runtime and also need to handle the go.mod thing every single time. It's really annoying and with the release of the go113, all the dev repo will be based on the module, it just becomes even more annoying.

On Wednesday, February 27, 2019 at 12:28:36 AM UTC-7, George Moschovitis wrote:
Nope it's not working. Perhaps someone from Google can chime in?
This is an important use-case.

-g.

On Wed, Feb 27, 2019 at 5:21 AM dir Ls <my.s...@gmail.com> wrote:
I assumed this will work. In my case I have two services each in a directory within the app root directory. I have the following directory structure.

app-root/
app-root/go/
app-root/service1/
app-root/service2/

And in the go.mod of serice1 and service2 I have been using the require option to refer to the local path to resolve the common code. It seems like this strategy won't work when the services are deployed? If so, how to share code between services?

On Sunday, February 24, 2019 at 9:41:32 AM UTC-8, George Moschovitis wrote:
I have a small Go application that depends on a couple of private modules. For development I use `replace (...)` in go.mod.
Alternatively the modules could be fetched from a private git repo.

How could I deploy such an application to Google App Engine (go111 runtime)? Is this even possible?

-g.

-- 
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.
>>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/e82c01aa-fad9-4c2b-92e0-0c9512d51f29%40googlegroups.com.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/c46e7b65-8436-4177-9b8d-659f453e3e37%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages