Status 405 trying to push repository - Do I need to create my private Docker repo?

rishi.in...@gmail.com

unread,

Nov 9, 2016, 5:03:07 PM11/9/16

to Deis Users List

I am getting closer .... trying to understand why I would be getting this. Do I need to run a local Docker registry?

Deploying my first app:

# deis create my-deis-tomcatapp --no-remote

# deis pull brunswickheads/nginx-1.8-s390x -a my-deis-tomcatapp (I am pulling this image off DockerHub)

Creating build... Error: Unknown Error (400): {"detail":"Status 405 trying to push repository my-deis-tomcatapp: \"<html>\\r\

<head><title>405 Not Allowed</title></head>\\r\

<body bgcolor=\\\"white\\\">\\r\

<center><h1>405 Not Allowed</h1></center>\\r\

<hr><center>nginx/1.11.0</center>\\r\

</body>\\r\

</html>\\r\

\""}

Controller logs indicate:

/======================================================/

# kubectl logs -f --namespace=deis deis-controller-121201537-iazx6:

INFO [my-deis-tomcatapp]: build my-deis-tomcatapp-95883fc created

INFO [my-deis-tomcatapp]: admin deployed brunswickheads/nginx-1.8-s390x

INFO Pulling Docker image brunswickheads/nginx-1.8-s390x:latest

INFO Tagging Docker image brunswickheads/nginx-1.8-s390x:latest as localhost:5555/my-deis-tomcatapp:v6

INFO Pushing Docker image localhost:5555/my-deis-tomcatapp:v6

INFO [my-deis-tomcatapp]: Status 405 trying to push repository my-deis-tomcatapp: "<html>\r\n<head><title>405 Not Allowed</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>405 Not Allowed</h1></center>\r\n<hr><center>nginx/1.11.0</center>\r\n</body>\r\n</html>\r\n"

ERROR:root:Status 405 trying to push repository my-deis-tomcatapp: "<html>\r\n<head><title>405 Not Allowed</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>405 Not Allowed</h1></center>\r\n<hr><center>nginx/1.11.0</center>\r\n</body>\r\n</html>\r\n"

Traceback (most recent call last):

File "/app/api/models/release.py", line 89, in new

release.publish()

File "/app/api/models/release.py", line 136, in publish

publish_release(source_image, self.image, deis_registry, self.get_registry_auth())

File "/app/registry/dockerclient.py", line 195, in publish_release

return DockerClient().publish_release(source, target, deis_registry, creds)

File "/app/registry/dockerclient.py", line 114, in publish_release

self.push("{}/{}".format(self.registry, name), tag)

File "/usr/local/lib/python3.5/dist-packages/backoff.py", line 286, in retry

ret = target(*args, **kwargs)

File "/app/registry/dockerclient.py", line 131, in push

log_output(stream, 'push', repo, tag)

File "/app/registry/dockerclient.py", line 174, in log_output

stream_error(chunk, operation, repo, tag)

File "/app/registry/dockerclient.py", line 191, in stream_error

raise RegistryException(message)

registry.dockerclient.RegistryException: Status 405 trying to push repository my-deis-tomcatapp: "<html>\r\n<head><title>405 Not Allowed</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>405 Not Allowed</h1></center>\r\n<hr><center>nginx/1.11.0</center>\r\n</body>\r\n</html>\r\n"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/app/api/models/build.py", line 63, in create

source_version=self.version

File "/app/api/models/release.py", line 96, in new

raise DeisException(str(e)) from e

api.exceptions.DeisException: Status 405 trying to push repository my-deis-tomcatapp: "<html>\r\n<head><title>405 Not Allowed</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>405 Not Allowed</h1></center>\r\n<hr><center>nginx/1.11.0</center>\r\n</body>\r\n</html>\r\n"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/usr/local/lib/python3.5/dist-packages/rest_framework/views.py", line 471, in dispatch

response = handler(request, *args, **kwargs)

File "/app/api/views.py", line 181, in create

return super(AppResourceViewSet, self).create(request, **kwargs)

File "/usr/local/lib/python3.5/dist-packages/rest_framework/mixins.py", line 21, in create

self.perform_create(serializer)

File "/app/api/viewsets.py", line 21, in perform_create

self.post_save(obj)

File "/app/api/views.py", line 258, in post_save

self.release = build.create(self.request.user)

File "/app/api/models/build.py", line 79, in create

raise DeisException(str(e)) from e

api.exceptions.DeisException: Status 405 trying to push repository my-deis-tomcatapp: "<html>\r\n<head><title>405 Not Allowed</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>405 Not Allowed</h1></center>\r\n<hr><center>nginx/1.11.0</center>\r\n</body>\r\n</html>\r\n"

172.17.0.9 "POST /v2/apps/my-deis-tomcatapp/builds/ HTTP/1.1" 400 280 "Deis Client dev-0391c6c"

/======================================================/

Matthew Fisher

unread,

Nov 9, 2016, 5:37:48 PM11/9/16

to rishi.in...@gmail.com, Deis Users List

Yes. That error seems to indicate your cluster is trying to push the image directly to DockerHub and isn't pointing at a local registry. The helm charts install a local docker registry. See https://github.com/deis/charts/blob/master/workflow-dev/tpl/deis-registry-deployment.yaml

--
You received this message because you are subscribed to the Google Groups "Deis Users List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to deis-users+unsubscribe@googlegroups.com.
To post to this group, send email to deis-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/deis-users/b3a8c96b-b471-4005-a852-fabf87e62285%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

rishi.in...@gmail.com

unread,

Nov 14, 2016, 10:13:21 AM11/14/16

to Deis Users List, rishi.in...@gmail.com

Hi Matt,

Any way I could disable registry-proxy and let app use deis-registry directly?

It seems like something is messing up registry-proxy. I can push image to deis-registry directly (after bypassing registry-proxy).

Docker log indicate "error parsing HTTP 404 response body: invalid character '<' ".

registry-proxy error.log says: (this may be normal though until a blob entry gets created)

/=======================================/

2016/11/14 14:07:28 [error] 12#0: *4 "/usr/html/v2/my-deis-nginx/blobs/uploads/index.html" is not found (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "POST /v2/my-deis-nginx/blobs/uploads/ HTTP/1.1", host: "localhost:5555"

/=======================================/

registry-proxy access log looks suspect with "_" entries:

/=====================================/

172.17.0.1 - - [14/Nov/2016:14:07:28 +0000] "\x16\x03\x01\x00\x89\x01\x00\x00\x85\x03\x03\xDE{1\xB1'$\xB0\xA5\x85\x90\xE3\xD4" 400 173 "-" "-"

172.17.0.1 - - [14/Nov/2016:14:07:28 +0000] "GET /v2/ HTTP/1.1" 404 169 "-" "docker/1.11.2 go/go1.6.2 git-commit/b9f10c9 kernel/4.4.0-38-generic os/linux arch/s390x UpstreamClient(Docker-Client/1.11.2 \x5C(linux\x5C))"

172.17.0.1 - - [14/Nov/2016:14:07:28 +0000] "POST /v2/my-deis-nginx/blobs/uploads/ HTTP/1.1" 404 169 "-" "docker/1.11.2 go/go1.6.2 git-commit/b9f10c9 kernel/4.4.0-38-generic os/linux arch/s390x UpstreamClient(Docker-Client/1.11.2 \x5C(linux\x5C))"

/=====================================/

To unsubscribe from this group and stop receiving emails from it, send an email to deis-users+...@googlegroups.com.

bacongobbler

unread,

Nov 14, 2016, 12:13:46 PM11/14/16

to rishi.in...@gmail.com, Deis Users List

You'll likely face the same problem with or without the registry-proxy, but you can use https://github.com/deis/registry/issues/64#issuecomment-254916941 as a guide.

Matt

To unsubscribe from this group and stop receiving emails from it, send an email to deis-users+unsubscribe@googlegroups.com.

To post to this group, send email to deis-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/deis-users/f2c0eda4-7d17-45c5-a312-98345c9491b0%40googlegroups.com.

rishi.in...@gmail.com

unread,

Nov 14, 2016, 3:27:25 PM11/14/16

to Deis Users List, rishi.in...@gmail.com

Thanks Matt - that worked! So clearly some issues in navigating through proxy.

However, would you know why my apps health checks are failing? Image gets pushed but deployment craps out due to:

15m 2s 187 {kubelet 127.0.0.1} spec.containers{my-deis-nginx-cmd} Warning Unhealthy Readiness probe failed: dial tcp 172.17.0.15:443: getsockopt: connection refused

Do I need special health check settings on my app pods?

Matthew Fisher

unread,

Nov 14, 2016, 3:56:55 PM11/14/16

to rishi.in...@gmail.com, Deis Users List

I know that the nginx container exposes two ports (80 and 443) by default which doesn't play nicely in the one-port world of Workflow. That being said you can customize the healthchecks using `deis help healthchecks:set` and it should be good to go. Even a dumb `deis healthchecks:set readiness exec exit 0` should do the trick.

Matt

To unsubscribe from this group and stop receiving emails from it, send an email to deis-users+unsubscribe@googlegroups.com.

To post to this group, send email to deis-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/deis-users/043caed6-a47c-4d4c-9f3b-ebba28495c24%40googlegroups.com.

Reply all

Reply to author

Forward