Deployment failing with HTTP Error 400: Bad Request

[Alan Keister]

I am trying to deploy changes to an existing project. This worked yesterday. My command line is:

appcfg.py update .  --noauth_local_webserver --verbose

I get this "Bad request" error today back from app engine. I have reauthenticated and then blew away my local copy of the project and re-downloaded it but neither have fixed this problem. What gives?

12:21 PM Starting deployment.
2016-04-15 12:21:10,849 INFO appcfg.py:1693 Send: /api/appversion/deploy, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 
2016-04-15 12:21:11,468 INFO appcfg.py:2559 HTTP Error (HTTP Error 400: Bad Request Unexpected HTTP status 400) 
12:21 PM Rolling back the update.
2016-04-15 12:21:11,469 INFO appcfg.py:1693 Send: /api/appversion/rollback, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 
Error 400: --- begin server output ---

Client Error (400)
The request is invalid for an unspecified reason.
--- end server output ---

[Nick (Cloud Platform Support)]

Hey Alan,

Usually a specific issue like this should not be posted to Groups but to a relevant tag on Stack Overflow. However, since this is a process of learning for the community and you are certainly not the first to start a support thread in this forum (which is meant for more general discussion), I'll be happy to assist you here.

Is this still affecting you? Could you clarify a bit the sequence of events around "rollback"?

Regards,

Nick
Cloud Platform Community Support

[Alan Keister]

Hi Nick,

I did crosspost this to stackoverflow.  Yes, I am still having the issue.  It started this morning.  The verbose log is below.  To try to get around this, I deleted my local copy of the project, downloaded it again and reauthenticated.  Still, no luck.

$ appcfg.py update .  --noauth_local_webserver --verbose

02:05 PM Application: cynomix3; version: 1

02:05 PM Host: appengine.google.com

2016-04-15 14:05:44,093 INFO sdk_update_checker.py:229 Checking for updates to the SDK. 

2016-04-15 14:05:44,658 INFO sdk_update_checker.py:257 The SDK is up to date. 

2016-04-15 14:05:44,728 INFO appcfg.py:2378 Reading app configuration. 

02:05 PM Starting update of app: cynomix3, version: 1

02:05 PM Getting current resource limits.

2016-04-15 14:05:44,729 INFO appcfg.py:1693 Send: /api/appversion/getresourcelimits, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 

02:05 PM Scanning files on local disk.

2016-04-15 14:05:45,404 INFO appcfg.py:2501 Processing file 'api_keys.py' 

2016-04-15 14:05:45,406 INFO appcfg.py:2501 Processing file 'app.yaml' 

2016-04-15 14:05:45,408 INFO appcfg.py:2501 Processing file 'bottle.py' 

2016-04-15 14:05:45,411 INFO appcfg.py:2501 Processing file 'index.yaml' 

...

2016-04-15 14:05:45,536 INFO appcfg.py:1693 Send: /api/appversion/create, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 

02:05 PM Cloning 12 static files.

2016-04-15 14:05:46,425 INFO appcfg.py:1693 Send: /api/appversion/cloneblobs, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 

02:05 PM Cloning 23 application files.

2016-04-15 14:05:47,880 INFO appcfg.py:1693 Send: /api/appversion/clonefiles, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 

02:05 PM Compilation starting.

2016-04-15 14:05:48,793 INFO appcfg.py:1693 Send: /api/appversion/precompile, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 

02:05 PM Compilation completed.

02:05 PM Starting deployment.

2016-04-15 14:05:49,405 INFO appcfg.py:1693 Send: /api/appversion/deploy, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 

2016-04-15 14:05:50,083 INFO appcfg.py:2559 HTTP Error (HTTP Error 400: Bad Request Unexpected HTTP status 400) 

02:05 PM Rolling back the update.

2016-04-15 14:05:50,084 INFO appcfg.py:1693 Send: /api/appversion/rollback, params={'version': '1', 'app_id': 'cynomix3', 'module': 'default'} 

[Nick (Cloud Platform Support)]

Hey Alan,

Could you attempt to use the gcloud command to deploy (gcloud preview app deploy app.yaml)? This might have better luck although it's not assured. Just a possibility (figuring out why would have to follow, but that's our job).

Other than this, there's few clues on what next to try. Perhaps you could use the --noisy flag if running appcfg again?  

[Alan Keister]

--noisy didn't show anything new

I confirmed I could deploy my test app with gcloud. Sign.  This one still fails.I don't see anything useful in the output.  

Updating module [default].../DEBUG: Received operation: [apps/cynomix3/operations/eed72f1e-bc69-4858-84fa-08255e2ebf8c]

DEBUG: Operation [apps/cynomix3/operations/eed72f1e-bc69-4858-84fa-08255e2ebf8c] not complete. Waiting 5s.

Updating module [default]...\DEBUG: Operation [apps/cynomix3/operations/eed72f1e-bc69-4858-84fa-08255e2ebf8c] complete. Result: {

    "metadata": {

        "target": "apps/cynomix3/modules/default/versions/20160415t160100", 

        "insertTime": "2016-04-15T20:01:04.755Z", 

        "method": "google.appengine.v1beta4.Versions.CreateVersion", 

        "user": "email@domain", 

        "operationType": "create_version", 

        "@type": "type.googleapis.com/google.appengine.v1beta4.OperationMetadata"

    }, 

    "done": true, 

    "name": "apps/cynomix3/operations/eed72f1e-bc69-4858-84fa-08255e2ebf8c", 

    "error": {

        "message": "An internal error occurred.", 

        "code": 13

    }

}

Updating module [default]...failed.

DEBUG: (gcloud.preview.app.deploy) Error Response: [13] An internal error occurred.

Traceback (most recent call last):

  File "/home/alan/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 654, in Execute

    result = args.cmd_func(cli=self, args=args)

  File "/home/alan/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 1401, in Run

    resources = command_instance.Run(args)

  File "/home/alan/google-cloud-sdk/lib/surface/preview/app/deploy.py", line 536, in Run

    images.get(module))

  File "/home/alan/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 91, in DeployModule

    return operations.WaitForOperation(self.client.apps_operations, operation)

  File "/home/alan/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/api/operations.py", line 69, in WaitForOperation

    encoding.MessageToPyValue(completed_operation.error)))

OperationError: Error Response: [13] An internal error occurred.

ERROR: (gcloud.preview.app.deploy) Error Response: [13] An internal error occurred.

[Nick (Cloud Platform Support)]

Hey Alan,

Glad to hear it's working with gcloud. That seems to point to an issue not with the app payload itself, but the SDK. Could you try to reinstall your version of appcfg, a component not of gcloud but of the App Engine SDK?

Cheers,

Nick
Cloud Platform Community Support

[Nick (Cloud Platform Support)]

If that isn't working, you could try both 

gcloud config set app/use_appengine_api true

and 

gcloud config set app/use_appengine_api false

and see if either of those manages to work. Let me know the results!

Best wishes,

Nick
Cloud Platform Community Support

[Alan Keister]

Only the test app worked with gcloud. The other app I have been having trouble with still has the issue

[Alan Keister]

I assume I should run those commands before "gcloud preview app deploy app.yaml --verbosity debug".  If that is correct, neither worked.

[Nick (Cloud Platform Support)]

Thanks for checking those things. Here's a last test before it appears that the priority will pass to us in investigating our side of things: can you attempt to transfer the affected app's code to another environment, or reinstall the SDK, and deploy? If this still fails, it can be determined that the issue either lies in the app payload itself, or something on our side.

Cheers!

Nick
Cloud Platform Community Support 

[Alan Keister]

Thanks, I have. I have tried this in another system as well with both gcloud and appcfg.py. I was able to download my project but had the same error deploying. I paid for support and opened a ticket, Case 09108089.

[Nick (Cloud Platform Support)]

Hey Alan,

Thanks for letting me know. If there's anything else we can do here to help, feel free to let me know. Otherwise I'll leave this to the support agents.

Best of luck!

Nick
Cloud Platform Community Support