Error Response: [3] Docker image appengine.gcr.io/gcloud/project_id.version.time was either not found, or you do not have access to it.

[Nao Ito]

Hi,

I have served one website with GCE(GAE flexible = vm).

But suddenly the deployment command got to result in failure.

I could deploy successfully with the same command about four months ago.

I googled the error code and message, but there is no related post at all.

Does anyone face the same error? or come up with some hints to solve the problem?

Thank you in advance.

# Details

## Development Environment: Mac OS 10.11.6

## Development Language: Go (GAE/Go)

## Command: aedeploy gcloud preview app deploy app.yaml --promote

## Debug message(using option: --verbose debug):

Updating module [default]...\DEBUG: Operation [apps/project_id/operations/ab08cc84-9104-43db-9430-805fce9936a7] complete. Result: {

    "metadata": {

        "target": "apps/project_id/modules/default/versions/20160821t144637",

        "method": "google.appengine.v1beta4.Versions.CreateVersion",

        "operationType": "create_version",

    },

    "done": true,

    "name": "apps/project_id/operations/ab08cc84-9104-43db-9430-805fce9936a7",

    "error": {

        "message": "Docker image appengine.gcr.io/gcloud/project_id.version.time was either not found, or you do not have access to it.",

        "code": 3

    }

}

Updating module [default]...failed.

[Nao Ito]

It's just fixed!

The reason was very simple. My gcloud command version was too old. (0.7.x ?)

So I installed latest Google Cloud SDK(122.0.0).

Then the problem was fixed.

Bottle neck of the problem is Container Registry, I think.

Old gcloud command couldn't treat Container Registry, so the docker image also couldn't be pushed on the registry.

# As long as I saw the debug log of deploy command, the processing of pushing docker image on Container Registry seemed to be success.

Maybe its error handling in aedeploy command should be fixed. It's a trap. :(

Thanks

[Nicholas (Google Cloud Support)]

Many features have change significantly since the version you mention using previously.  For one, the gcloud preview app command group has been removed in favor of gcloud app as of 116.0.0 (June 29).  In addition, the container registry has been used by default for Flex application deployment as of 0.9.49 (February 25, 2015) though I could not find release notes for 0.7.x (perhaps you meant 0.9.7).

Regarding the aedeploy command, do you mean that the wrapper effectively swallows errors or logs when using it?  If so, can you provide an example of a use of gcloud whose output was truncated when wrapped with aedeploy?

[Geoffroy Lesage]

Hi there,

I seem to be getting the same issue right now... just updated from Google Cloud SDK 124.0.0 to 125.0.0 (latest for Mac) but am getting the same issue.

A few hours ago I was able to deploy correctly... Any advice appreciated. thanks!

Debug log:

DEBUG: Received operation: [apps/starchup-1472916020/operations/bf68bfd9-8368-4c9e-a2ee-042142cea4fa]

Updating service [default].../DEBUG: Operation [apps/starchup-1472916020/operations/bf68bfd9-8368-4c9e-a2ee-042142cea4fa] complete. Result: {

    "metadata": {

        "user": "o...@email.com", 

        "target": "apps/starchup-1472916020/services/default/versions/20160912t004322", 

        "@type": "type.googleapis.com/google.appengine.v1beta5.OperationMetadataV1Beta5", 

        "insertTime": "2016-09-12T05:45:03.559Z", 

        "method": "google.appengine.v1beta5.Versions.CreateVersion"

    }, 

    "done": true, 

    "name": "apps/starchup-1472916020/operations/bf68bfd9-8368-4c9e-a2ee-042142cea4fa", 

    "error": {

        "message": "Docker image us.gcr.io/starchup-1472916020/appengine/default.20160912t004322:latest was either not found, or you do not have access to it.", 

        "code": 3

    }

}

Updating service [default]...failed.                                                                                                     

DEBUG: (gcloud.app.deploy) Error Response: [3] Docker image us.gcr.io/starchup-1472916020/appengine/default.20160912t004322:latest was either not found, or you do not have access to it.

Traceback (most recent call last):

  File "/Users/glesage/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 720, in Execute

    resources = args.calliope_command.Run(cli=self, args=args)

  File "/Users/glesage/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 1455, in Run

    resources = command_instance.Run(args)

  File "/Users/glesage/google-cloud-sdk/lib/surface/app/deploy.py", line 53, in Run

    return deploy_util.RunDeploy(self, args)

  File "/Users/glesage/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 300, in RunDeploy

    all_services)

  File "/Users/glesage/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 167, in Deploy

    service, manifest, image, endpoints_info)

  File "/Users/glesage/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 118, in DeployService

    return operations.WaitForOperation(self.client.apps_operations, operation)

  File "/Users/glesage/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/api/operations.py", line 70, in WaitForOperation

    encoding.MessageToPyValue(completed_operation.error)))

OperationError: Error Response: [3] Docker image us.gcr.io/starchup-1472916020/appengine/default.20160912t004322:latest was either not found, or you do not have access to it.

ERROR: (gcloud.app.deploy) Error Response: [3] Docker image us.gcr.io/starchup-1472916020/appengine/default.20160912t004322:latest was either not found, or you do not have access to it.

[Nicholas (Google Cloud Support)]

I have not yet been able to reproduce the issues you are reporting.  The container not found by the gcloud app deploy command, is it listing in your Container Registry under Container Engine?  Do you have full visibility over this registry?  Can you pull it using the command provided by the Developers Console?

On Monday, August 22, 2016 at 2:11:18 AM UTC-4, Nao Ito wrote:

[Geoffroy Lesage]

Hi Nicholas,

I am able to see it on the container Registry however I seem unable to pull it...

I should mention that I deleted all versions except the latest under App Engine --> Versions.. so there is only 1 version left

[Geoffroy Lesage]

Nevermind I am able to pull it once I navigate inside the correct directory. Is there anything I can do with that image to help with this issue?

[Nicholas (Google Cloud Support)]

When getting the pull command from the Container Registry of the Developers Console, can you then successfully deploy with gcloud app deploy using the provided container URL with the --image-url option?  Apologies if it seems like I'm asking you to repeat yourself but wish to clarify if your success with the pull command is limited to said command or if you can also now deploy with gcloud app deploy after navigating to the directory.

[Geoffroy Lesage]

Hi Nicholas, thank you for your time!

I am indeed able to pull the image using the gcloud command. However, if I paste the exact same url as the --image-url option it throws the same error

`gcloud app deploy app.yaml --image-url "us.gcr.io/..."`

`ERROR: (gcloud.app.deploy) Error Response: [3] Docker image us.gcr.io/starchup-1472916020/appengine/default.20160922t203639:latest was either not found, or you do not have access to it.`

[Nicholas (Google Cloud Support)]

As you've confirmed that the container exists, based on the error message, one can only conclude that the issue lies with access control.  How is your dev machine (the one from which you are using gcloud) authenticated (service account, other credentials, etc.)?  At the moment, I cannot reproduce the behavior you're experiencing.

On Monday, August 22, 2016 at 2:11:18 AM UTC-4, Nao Ito wrote:

[Geoffroy Lesage]

Hey Nicholas, the local machine is OSX, setup using authentication with the email associated to the account... I've been able to deploy until now as well and nothing has changed.

Would I be able to reset permissions possible? I tried gcloud init again and setting up with our email, but after attempting to deploy I get the same error

[Nicholas (Google Cloud Support)]

Is any other developer for your project experiencing the same issue?  Can you create a new project, run gcloud init with selected that new project ID and successfully deploy the same code to that project?  If both of the above scenarios work as expected, I would strongly suggest filing a defect report on the public issue tracker.

On Monday, August 22, 2016 at 2:11:18 AM UTC-4, Nao Ito wrote:

[Geoffroy Lesage]

I'm not sure. I just created a brand new project and was able to use all this correctly.

Feel free to archive this thread if you'd like.

Thank you!

[Nicholas (Google Cloud Support)]

Thanks for getting back to me.  I'm glad to hear you were successful in the end by creating a new project, though not a most favorable solution.  I'm afraid I don't have any new findings on the issue.  Should this exception return, I would suggest filing a new defect report on the Google Cloud Platform public issue tracker.

On Monday, August 22, 2016 at 2:11:18 AM UTC-4, Nao Ito wrote: