Deployments of small modules very slow

[Cameron Tangney]

Is anybody else finding that deployments are extraodinarily slow today?

We have a small python module (called 'users') that we're trying to deploy (reported size is less than 1mb) and it's consistently taking over 30 minutes to deploy...

$ gcloud preview app deploy ./app.yaml --version 7 --project <project_name>

You are about to deploy the following modules:

 - blah blah blah

Do you want to continue (Y/n)?  

Beginning deployment...

Updating module [users]...\

It's been sitting there for over 30 minutes. The last time (for version 6 this morning) it took about 40 minutes before it succeeded. Google support doesn't seem to notice anything strange about appengine or my project. Previous versions of this module have deployed immediately (within seconds).

Anybody else experiencing things like this?

[Chirag Kakani]

Yes I am facing an issue while deploying.

It starts cloning the static file but nothing really happens afterwards.

[Cameron Tangney]

FWIW... my deployment does eventually succeed, but only after a frustratingly long time...

time gcloud preview app deploy ./app.yaml --version 7 --project <project>

Beginning deployment...

Updating module [users]...done.

Deployed module [users] to [<url>]

real 25m45.778s

user 0m0.387s

sys 0m0.102s

The size of the app is 589KB in the versions section of the admin console, so pretty unclear why this is taking so long.

[Nick (Cloud Platform Support)]

Hey Folks,

This might be related to your network performance. Please provide some more technical details showing the network latency from your box to the deployment server. You can determine the server being communicated with by turning on debug output for your deployment command. 

Sincerely,

Nick

[Cameron Tangney]

Thanks for the idea Nick. In the verbose output I'm seeing lines like this:

DEBUG: Sending request to
https://appengine.google.com/api/appversion/clonefiles?app_id=<project>&module=users&version=6
headers={'X-appcfg-api-version': '1', 'content-length': '2854',
'Content-Type': 'application/octet-stream'}
body=static/js/components

...

Is that the deployment server you're talking about?
How should I go about determining the details re:network latency to this endpoint?

[Cameron Tangney]

of all strange rabbit-holes, this seems to have fixed my issue

http://askubuntu.com/questions/574569/apt-get-stuck-at-0-connecting-to-us-archive-ubuntu-com

noticed that when installing iperf to try and do what Nick suggested, i got stuck trying to get updates, which led me to this solution, which seems to have resolved my appengine issues.

[Nick (Cloud Platform Support)]

Hey Cameron,

I'm glad that I somehow put you on the right path, although I have to say this is one of the strangest resolutions to an issue I've ever seen, at first glance. Looking deeper, it appears to make sense - the apt-get problem and this both seem to derive from a mis-configuration of the DNS system on your box. gai.conf helps resolve system calls to getaddrinfo, which is used to resolve hostnames to IP addresses. 

Best of luck in your further development!