gcloud app deploy fails (works with goapp deploy and dev_appserver.py)

[Dave Jack]

I have my dependencies vendored, but it doesn't seem to work with gcloud app deploy:

2017/05/03 14:39:57 go-app-builder: Failed parsing input: package "vidston.com/vendor/google.golang.org/appengine/log" cannot import internal package "google.golang.org/appengine/internal/log"

dev_appserver.py . and $GO_APPENGINE_STANDALONE_SDK/goapp deploy from the same directory / source tree works fine.

Any advice about what I'm doing wrong?

[Steven Buss]

Could you try this again without vendoring that package, please?

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsubscribe...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Jack]

I created a project to demonstrate the issue: https://github.com/davars/app-engine-deploy-reproducer .

It failed earlier than I thought it would.  I thought I'd have to create a vendored dependency with an internal package.  But it seems like vendoring is ignored by the gcloud app builder. (eg, mypkgs/handlers/handlers.go:8: can't find import: "unstableapi").

[Steven Buss]

I think vendoring with gcloud (via go-app-stager) might be slightly broken at the moment. Don't quote me on that, as I unfortunately haven't had time to investigate fully.

Vendoring should work as expected with goapp from the legacy API. It's on my TODOs to fix this soon (TM). To double check: does this work with goapp but not with gcloud app deploy?

[Dave Jack]

That's correct.  This change (https://github.com/davars/app-engine-deploy-reproducer/commit/ad4c942a028687f2b7f9f19b141dcc5bd0443b90) breaks deployment with gcloud, but not with goapp.  I checked the entire GOPATH into the repo to create a hermetic-ish environment (I didn't attempt to bundle the gcloud & appengine SDKs too).  It should be fairly easy to reproduce just by running export GOPATH="$(git rev-parse --show-toplevel)/gopath" from within the repo and attempting to deploy with both tools.

[Karan Misra]

Hi Steven,

Did you ever get around to this?

Regards,

Karan

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

[Chaim-Leib Halbert]

I'm also interested to see the resolution of this issue, since I am encountering the same thing. My repository is structured very similarly:

.

├── app

│   ├── app.yaml

│   └── main.go

└── gopath

    └── src

        └── server

            ├── assets

            ├── config

            ├── server.go

            └── vendor

                ├── cloud.google.com

                ├── github.com

                ├── golang.org
                └── google.golang.org

When I deploy, I get this:

> gcloud app deploy --project myproj --version 2017-10-30t17-21-445061130f --no-promote app.yaml

...

╔════════════════════════════════════════════════════════════╗

╠═ Uploading 5 files to Google Cloud Storage                ═╣

╚════════════════════════════════════════════════════════════╝

File upload done.

Updating service [default]...failed.

ERROR: (gcloud.app.deploy) Error Response: [9] Deployment contains files that cannot be compiled: Compile failed:

2017/10/30 17:30:45 go-app-builder: Failed parsing input: package "server/vendor/golang.org/x/text/language" cannot import internal package "golang.org/x/text/internal/tag"

Just like with Karan, I'm able to deploy with goapp but not with gcloud. I'm using Google Cloud SDK 177.0.0, go version 1.8.3 (appengine-1.9.62).

Chaim

[ralph...@gmail.com]

Also running into this issue. Pretty annoying. Would be nice to get support for vendoring. Deployment is pretty much impossible for me right now.

[john.reu...@digiexam.se]

Running into the same issue. 

Google Cloud SDK v185.0.0

App Engine Go Extensions v1.9.62

ERROR: (gcloud.app.deploy) Error Response: [9] Deployment contains files that cannot be compiled: Compile failed:

2018/01/19 06:07:01 go-app-builder: Failed parsing input: package "<my-app>/vendor/golang.org/x/oauth2" cannot import internal package "golang.org/x/oauth2/internal"

Any updates on this?

[st...@mygogoband.com]

I am also running into the same situation.  Errors below.  I have been using package oriented design suggested by William Kennedy of Ardan Labs.  It works great locally and I have used it in several projects.  I also use dep for dependencies and vendoring.

I dont understand why the gcloud deploy tool cant just use the basic golang suite of tools to build the code, find dependencies, etc in a normal way or allow the caller to pass in the cross compiled binary directly.  I really need this fixed or a workaround.

Errors:  

Step 1/3 : FROM gcr.io/google-appengine/golang

latest: Pulling from google-appengine/golang

Digest: sha256:7610fa0b2c90ea822b21052cd3db5297af225056dc99729dd6e524197bb9fe05

Status: Downloaded newer image for gcr.io/google-appengine/golang:latest

 ---> 04f98c801050

Step 2/3 : COPY . /go/src/app

 ---> 906de1462727

Removing intermediate container cb4db2fb60e2

Step 3/3 : RUN go-wrapper install -tags appenginevm

 ---> Running in ec53aa9cf083

+ exec go install -v -tags appenginevm

ingest.go:11:2: use of internal package not allowed

ingest.go:17:2: use of internal package not allowed

ingest.go:18:2: use of internal package not allowed

ingest.go:12:2: use of internal package not allowed

ingest.go:13:2: cannot find package "github.com/gorilla/mux" in any of:

/usr/local/go/src/github.com/gorilla/mux (from $GOROOT)

/go/src/github.com/gorilla/mux (from $GOPATH)

/go/src/app/_gopath/src/github.com/gorilla/mux

The command '/bin/sh -c go-wrapper install -tags appenginevm' returned a non-zero code: 1

ERROR

ERROR: build step 0 "gcr.io/cloud-builders/docker@sha256:51808bfe2ff20dc7a45df21177d68b577d3b2607437b79c38091dc94193d064e" failed: exit status 1

---------------------------------------------------------------------------------------------------------------------------------------------------------

ERROR: (gcloud.app.deploy) Cloud build failed. Check logs at https://console.cloud.google.com/gcr/builds/b845f65c-1efd-46a1-8a82-1fad5e7d1157?project=gogo-prod Failure status: UNKNOWN: Error Response: [2] Build failed; check build logs for details

[st...@mygogoband.com]

I fixed this by writing a script to move the VENDOR directory to some other location temporarily and then doing the gcloud app deploy.  Then it compiles when it is running.  Of course this is full of issues, like out of sync VENDOR and default package directory.  But at least I am back to being able to deploy.

[Chaim-Leib Halbert]

I tried to move the vendor directory, but I got an error about missing packages. Am I doing something different from you?

.

├── README.md

├── app

│   ├── build

│   ├── content

│   ├── development.yaml

│   ├── production.yaml

│   ├── testing.yaml

│   ├── go-templates

│   ├── include.yaml

│   └── main.go

└── gopath

    ├── bin

    └── src

        ├── util

        │   ├── ...

        │   └── vendor

        └── server

            ├── ...

            ├── server.go

            └── vendor

~/Desktop/myproj

$ cd gopath/src

~/Desktop/myproj/gopath/src

$ for d in *; do mv "$d/vendor" ~/Desktop/vendor-temp/"$d"; done

~/Desktop/myproj/gopath/src

$ cd ../../app

~/Desktop/myproj/app

$ version="$(date +'%Y-%m-%dt%H-%M')-$(git rev-parse HEAD | head -c10)"

~/Desktop/myproj/app

$ gcloud app deploy --project development --version "$version" --no-promote development.yaml

ERROR: (gcloud.app.deploy) Staging command [/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/go-app-stager /Users/chalbert/Desktop/myproj/app/development.yaml /Users/chalbert/Desktop/myproj/app /var/folders/xs/8pxy1b055g5cltf84g4_pkfc0000gn/T/tmpas3SKf/tmpxrqlfq] failed with return code [1].

------------------------------------ STDOUT ------------------------------------

------------------------------------ STDERR ------------------------------------

2018/01/30 16:47:33 staging for go1.8

2018/01/30 16:47:33 failed analyzing /Users/chalbert/Desktop/myproj/app: cannot find package "golang.org/x/text/language" in any of:

  ($GOROOT not set)

  /Users/chalbert/Desktop/myproj/gopath/src/golang.org/x/text/language (from $GOPATH)

GOPATH: /Users/chalbert/Desktop/myproj/gopath

--------------------------------------------------------------------------------

[jboa...@sparkfish.com]

Hi Chaim... Did you ever find a resolution to this issue?  I've been struggling with the exact same issue.  I'm starting to think that GAE is simply not ready for dep for reasons that make little sense.  :/

[Chaim-Leib Halbert]

GAE can be used with dep and other solutions that use the vendor folder, but only if you use goapp deploy instead of gcloud app deploy. goapp deploy is an older tool that Google still documents, although you have to click the "goapp" tab on the shell command examples. Google used to enable it by default (adding it to PATH), but lately, it has not been included in a default install. I found out a way to re-enable it, though:

1. Install App Engine Go support: gcloud components install app-engine-go

2. Save the following file as goapp (no extension)

#!/bin/bash

# A wrapper script to let us use goapp, since Google isn't installing it

# for us anymore.

gcloud="$(which gcloud)"

[ -L "$gcloud" ] && gcloud="$(readlink "$gcloud")"

appdir="$(dirname "$(dirname "$gcloud")")"

goapp="${appdir}/platform/google_appengine/goapp"

if ! [ -f "$goapp" ]; then

  echo "goapp is not in the Google Cloud SDK libraries." >&2

  echo "Hint: try running this command:" >&2

  echo "" >&2

  echo "    gcloud components install app-engine-go" >&2

fi

python "$goapp" "$@"

3. cd to the directory of your new file, and then make it executable: chmod +x goapp

4. Move the file to some place on your PATH.

After doing the above, I can successfully deploy using this command:

goapp deploy \

  -application "$project" \

  -version "$version" \

  "${project}.yaml"