How to get a local docker image into kubernetes

[Wiley Sanders]

I have a local docker image "serverstatus" that I want to deploy on my shiny new kubernetes cluster:

# docker images

REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE

serverstatus                           latest              6d4365c6b9f3        7 hours ago         267.9 MB

docker.io/registry                     2                   182810e6ba8c        2 days ago          37.6 MB

gcr.io/google_containers/pause-amd64   3.0                 99e59f495ffa        7 months ago        746.9 kB

gcr.io/google-samples/node-hello       1.0                 4c7ea8709739        8 months ago        643.7 MB

How can I "get this image into kubernetes" for lack of a better term, so I can deploy it with "kubectl create"?

As you can see, I have the docker.io/registry image as well. I tried to follow instructions at 

http://imhotep.io/k8s/docker/2016/08/04/docker-registry.html - it seems a bit out of date - I can get a registry pod running: 

# ku describe svc registry

Name: registry

Namespace: default

Labels: app=registry

Selector: app=registry

Type: LoadBalancer

IP: 172.30.246.214

Port: client 5000/TCP

NodePort: client 31667/TCP

Endpoints: 172.30.36.2:5000

So it looks like I have a registry started but the best I can do is:

[root@ops-k7s301 /]# docker push 172.30.36.2:5000/serverstatus

The push refers to a repository [172.30.36.2:5000/serverstatus]

unable to ping registry endpoint https://172.30.36.2:5000/v0/

v2 ping attempt failed with error: Get https://172.30.36.2:5000/v2/: http: server gave HTTP response to HTTPS client

 v1 ping attempt failed with error: Get https://172.30.36.2:5000/v1/_ping: http: server gave HTTP response to HTTPS client

What am I missing here? 

Is uploading to docker.io the only practical way to share images between docker and kubernetes?

thx -w

[Maurizio Vitale]

You can have your private repository. It is a bit tricky to setup, mainly due to the docker daemon considering any non-local access insecure, but very doable.

There're quite a few way to get images into a cluster, here the ones I can think about:

- (w/o registry): docker save, scp the image to all worker nodes, docker import in each and now you can run pods. Not terribly nice, but is good to know if you need your modified images before you have the registry configured (or during cluster bootstrap)

- upload to a public registry, only ok if a) your cluster can reach it and b) you're ok w/ your images in the wild

- a registry outside the cluster. Configure it as in any docker documentation. Some people prefer this solution, I think a registry is perfect for k8s, especially for scale w/ bittorrent distribution of images

- a registry in cluster configured as in the docs (e.g. w/ a proxy on each node --these days w. Daemonsets, docs should be  modified -- this makes docker think all access are local). I don't like this and wouldn't work for me because it relies on socat for external access and  that's not available in coreos. With other OS it looks like it should work.

- a registry w/ a service on a NodePort. This is what I use myself:

#!/bin/bash

rm -f registry.pem registry-key.pem kube-registry.tar ca.pem ca-key.pem ca.crt

cp ../../CLUSTER_ASSETS/ssl/ca* .

../../scripts/init-ssl . registry kube-registry

# I don't think this is needed, it is probably my intermedate experiments.

sudo cp ca.pem /usr/local/share/ca-certificates/Kube_Root_CA.crt

sudo update-ca-certificates --verbose

# This is probably not needed

mkdir -pv ~/.docker

cp -v ca.pem ~/.docker/ca.pem

cp -v registry.pem ~/.docker/cert.pem

cp -v registry-key.pem ~/.docker/key.pem

chmod 0444 ~/.docker/ca.pem

chmod 0444 ~/.docker/cert.pem

chmod 0400 ~/.docker/key.pem

DOCKER_TLS_DIR=/etc/docker/certs.d/w01:32000

# This is for access from outside the cluster

mkdir -pv $DOCKER_TLS_DIR

sudo cp -v ca.pem $DOCKER_TLS_DIR/ca.crt

sudo cp -v registry.pem $DOCKER_TLS_DIR/client.cert

sudo cp -v registry-key.pem $DOCKER_TLS_DIR/client.key

sudo chmod 0444 $DOCKER_TLS_DIR/ca.crt

sudo chmod 0444 $DOCKER_TLS_DIR/client.cert

sudo chmod 0400 $DOCKER_TLS_DIR/client.key

# This is for access inside the cluster.

for m in w01 w02 w03 m01; do

#for m in w01 m01; do

    ssh $m sudo mkdir -pv /etc/docker/certs.d/w01:32000

    cat ca.pem | ssh $m sudo tee /etc/docker/certs.d/w01:32000/ca.crt

    cat registry.pem | ssh $m sudo tee /etc/docker/certs.d/w01:32000/client.cert

    cat registry-key.pem | ssh $m sudo tee /etc/docker/certs.d/w01:32000/client.key

done

kubectl --namespace=kube-system delete secret registry-tls-secret

kubectl --namespace=kube-system create secret generic registry-tls-secret --from-file=registry.pem=registry.pem --from-file=registry-key.pem=registry-key.pem --from-file=ca.pem=ca.pem

kubectl delete --namespace=kube-system svc kube-registry

kubectl create -f registry-svc.yaml

kubectl delete -f registry-rc.yaml

kubectl create -f registry-rc.yaml

The critical part that took me the longest to figure out is where to put the certificates so that docker would use them when talking with the registry. You'll have to adapt a lot of the above, but it should give you an idea.

I'm still working on ingress, because I don't like having to tag and talk to a specific node.

init_ssl is a script I've got from somewhere, you can probably google for it. Nothing special, creates a certificate, a key and sign it w/ the secret key of a certificate authority known to the registry (see the ca.pem we pass around).

For running the registry I use:

apiVersion: v1

kind: ReplicationController

metadata:

  name: kube-registry-v0

  namespace: kube-system

  labels:

    k8s-app: kube-registry

    version: v0

#    kubernetes.io/cluster-service: "true"

spec:

  replicas: 1

  selector:

    k8s-app: kube-registry

    version: v0

  template:

    metadata:

      labels:

        k8s-app: kube-registry

        version: v0

#        kubernetes.io/cluster-service: "true"

    spec:

      containers:

      - name: registry

        image: registry:2

        resources:

          # keep request = limit to keep this container in guaranteed class

          limits:

            cpu: 100m

            memory: 100Mi

          requests:

            cpu: 100m

            memory: 100Mi

        env:

        - name: REGISTRY_LOG_LEVEL

          value: debug

        - name: REGISTRY_HTTP_ADDR

          value: :5000

        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY

          value: /var/lib/registry

        - name: REGISTRY_HTTP_TLS_CERTIFICATE

          value: /certs/registry.pem

        - name: REGISTRY_HTTP_TLS_KEY

          value: /certs/registry-key.pem

        - name: REGISTRY_HTTP_TLS_CLIENTCAS

          value: ' - /certs/ca.pem'

        volumeMounts:

        - name: image-store

          mountPath: /var/lib/registry

        - name: cert-dir

          mountPath: /certs

        ports:

        - containerPort: 5000

          name: registry

          protocol: TCP

      volumes:

      - name: image-store

        emptyDir: {}

      - name: cert-dir

        secret:

          secretName: registry-tls-secret

and the service is:

apiVersion: v1

kind: Service

metadata:

  name: kube-registry

  namespace: kube-system

  labels:

    k8s-app: kube-registry

    kubernetes.io/cluster-service: "true"

    kubernetes.io/name: "KubeRegistry"

spec:

  type: NodePort

  selector:

    k8s-app: kube-registry

  ports:

  - name: registry

    nodePort: 32000

    port: 5000

    protocol: TCP

Hope this helps. It was very painful for me.

  Maurizio

On Thu, Dec 29, 2016 at 8:26 PM, Wiley Sanders <wsan...@turnitin.com> wrote:

I have a local docker image "serverstatus" that I want to deploy on my shiny new kubernetes cluster:

# docker images

REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE

serverstatus                           latest              6d4365c6b9f3        7 hours ago         267.9 MB

docker.io/registry                     2                   182810e6ba8c        2 days ago          37.6 MB

gcr.io/google_containers/pause-amd64   3.0                 99e59f495ffa        7 months ago        746.9 kB

gcr.io/google-samples/node-hello       1.0                 4c7ea8709739        8 months ago        643.7 MB

How can I "get this image into kubernetes" for lack of a better term, so I can deploy it with "kubectl create"?

As you can see, I have the docker.io/registry image as well. I tried to follow instructions at 

http://imhotep.io/k8s/docker/2016/08/04/docker-registry.html - it seems a bit out of date - I can get a registry pod running: 

# ku describe svc registry

Name: registry

Namespace: default

Labels: app=registry

Selector: app=registry

Type: LoadBalancer

IP: 172.30.246.214

Port: client 5000/TCP

NodePort: client 31667/TCP

Endpoints: 172.30.36.2:5000

but the best I can do is 

[root@ops-k7s301 /]# docker push 172.30.36.2:5000/serverstatus

The push refers to a repository [172.30.36.2:5000/serverstatus]

unable to ping registry endpoint https://172.30.36.2:5000/v0/

v2 ping attempt failed with error: Get https://172.30.36.2:5000/v2/: http: server gave HTTP response to HTTPS client

 v1 ping attempt failed with error: Get https://172.30.36.2:5000/v1/_ping: http: server gave HTTP response to HTTPS client

What am I missing here? 

Is uploading to docker.io the only practical way to share images between docker and kubernetes?

thx -w

--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.