Docker cache issue when deploying GO to Managed VM using Cloud Shell and aedeploy

[Roy Palacios]

Hello,

I have a Go project running on ManagedVMs and wanted to try the Cloud Shell.

So, I have my repository in $HOME/default and a symbolic link from my repository to $GOPATH

The problem is that whenever I pull changes from the Cloud Shell and try to deploy with aedeploy, I end up using the Docker Cache and the changes are not reflected.

I guess it is because somehow the build process didnt detect the changes.

Is there anything to fix this? My workaround now is just to do some dummy code change using the Cloud Shell.

Thanks,

Roy

roy_palacios@sanguine-form-95600:~/default/xxxxxxxxxxxxxxx.pe/module/batch$ git pullUnpacking objects: 100% (6/6), done.From https://source.developers.google.com/p/sanguine-form-95600/r/default 8fd0d40..08c487b master -> origin/masterUpdating 8fd0d40..08c487bFast-forward cpe.pe/batch/padronbatch/padronbatch.go | 9 +++++++++ 1 file changed, 9 insertions(+)roy_palacios@sanguine-form-95600:~/default/xxxxxxxxxx.pe/module/batch$ aedeploy gcloud preview app deploy app.yamlYou are about to deploy the following modules: - sanguine-form-95600/batch (from [/tmp/aedeploy433757798/app.yaml]) Deployed URL: [xxxxxxxxxxxxxxxxx]Do you want to continue (Y/n)? yBeginning deployment...Verifying that Managed VMs are enabled and ready.Building and pushing image for module [batch]-------------------------------------------------------------------------------- DOCKER BUILD OUTPUT ---------------------------------------------------------------------------------Step 1 : FROM gcr.io/google_appengine/golang---> 28d7c86afc7aStep 2 : COPY . /go/src/app---> Using cache---> 9e8c6f93f56cStep 3 : RUN go-wrapper install -tags appenginevm---> Using cache---> 6693827d78cdSuccessfully built 6693827d78cd--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Copying files to Google Cloud Storage...Synchronizing files to [gs://staging.sanguine-form-95600.appspot.com/].Updating module [batch]...done.Deployed module [batch] to [xxxxxxxxxx]

[Roy Palacios]

The text below messed up. But there is this line:

Using cache---> 9e8c6f93f56c

And no build output

[Roy Palacios]

Ok, looks like changing files doesn't work. aedeploy on Cloud Shell is still using the cache.

But, provisioning a new Cloud Shell does the trick ("Restart Cloud Shell" option).

So, my guess is that temp files are messing up in Cloud Shell.

Thanks,

Roy

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/esBdozrIgW8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/b9700622-2402-45ad-8f8a-067978ad389f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Andrew Jessup]

Hi Roy,

You can try using the container builder API to run your image builds in App Engine (rather than relying on the local docker installation). To do this, run:

$ gcloud config set app/use_cloud_build true

Then try a deployment again using aedeploy.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.

To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/2e8f6844-3f7b-45ba-973d-9ab89e1a8ada%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Jessup | Product Manager, Google Cloud Platform | jes...@google.com 

[Roy Palacios]

Hi Andrew,

Thanks for the response. It looks like it is working perfectly :)

I see it documented here as well: https://cloud.google.com/appengine/docs/managed-vms/go/testing-and-deploying-your-app

However, it's not really clear what the advantage is. Both methods run remotely (the first one is a transient VM).

Thanks,

Roy

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/esBdozrIgW8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.

To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAEF6f2vo-QPCOqwcyu_D_nBZRBshtTXXjNmERCQT9%3D0QB%3DotjA%40mail.gmail.com.

[Andrew Jessup]

Thanks, we'll clear up the documentation to make it clearer. In the meantime, here's a quick explanation. The difference between the two modes is the way builds are performed. 

In the current (default) way, a transient GCE VM (called the builder VM) is provisioned in your project, which runs the Docker build. The gcloud tool uses the Docker wire protocol to connect to the GCE VM to perform the build. This usually works well for most customers, but has some disadvantages. The most notable are:

• GCE VMs can take a few seconds (~30) to be provisioned, which adds that much delay to the overall deployment time
  
  
• There are some edge cases where GCE VM deployments fail (for example, the one you just encountered, or when certain GCE quotas are exceeded) which then causes the failure of the App Engine deployment.
  
  
• As a customer, you will see transient builder VMs in your project (which can be annoying to have to parse through)

The container builder service addresses these concerns, by performing builds via a dedicated service. This avoids both the 30s VM provisioning delay and the issues associated with transient VMs. It also allows us to more easily improve and optimize the build experience over time. Overall deployments on Managed VMs should thus be faster and more reliable.

We are working to make Container Builder the default underlying mechanism for Managed VM deployments. We have a few teething issues to sort out (mainly relating to how the service is activated) before we do that, but I'd encourage all interested customers to try it out using the gcloud configuration setting described above. Please also make sure you're using the latest release (96.0.0 or later) of the gcloud tool as that includes some important updates.

And if you're trying out, please give us feedback! If you'd prefer you can reach me directly - jessup AT google DOT com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CA%2BhTChuFajKeiFxcQNF66u_Y48eoVO0OA5-YnGJMgq%2BbW_5c%2BA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Roy Palacios]

Thanks Andrew,

That's indeed a more efficient method.

Now I'm using Cloud Shell + Container Builder and looks good, will reach you if any issue.

Thanks,

Roy

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAEF6f2u7qMc8wT%2B%2BZKQMOwdNDOhm0UiD7FRBQEaUXe%2B%3DJsv5dw%40mail.gmail.com.