Calling docker stack deploy on a docker host from within a Jenkins container

[Ethan Wilansky]

I've posted this to stackoverflow, but after a number of views, I haven't gotten an answer or even a suggestion. I'm hoping by posting the same question here, this community might have a suggestion. For reference, the stackoverflow post is: https://stackoverflow.com/questions/52920540/calling-docker-stack-deploy-on-a-docker-host-from-within-a-jenkins-container. Here's the post:

On my OS X host, I'm using Docker CE (18.06.1-ce-mac73 (26764)) with Kubernetes enabled and using Kubernetes orchestration. From this host, I can run a stack deploy to deploy a container to Kubernetes using this simple docker-compose file (kube-compose.yml):

version: '3.3'

services:

  web:

    image: dockerdemos/lab-web

    volumes:

      - "./web/static:/static"

    ports:

      - "9999:80"

and this command-line run from the directory containing the compose file:

docker stack deploy --compose-file ./kube-compose.yml simple_test

However, when I attempt to run the same command from my Jenkins container, Jenkins returns:

this node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again

I do not want the docker client in the Jenkins container to be initialized for a swarm since I'm not using Docker swarm on the host.

The Jenkins container is defined in a docker-compose to include a volume mount to the docker host socket endpoint:

version: '3.3'

services:

  jenkins:

    # contains embedded docker client & blueocean plugin

    image: jenkinsci/blueocean:latest

    user: root

    ports:

      - "8080:8080"

      - "50000:50000"

    volumes:

      - ./jenkins_home:/var/jenkins_home

      # run Docker from the host system when the container calls it.

      - /var/run/docker.sock:/var/run/docker.sock

      # root of simple project

      - .:/home/project

    container_name: jenkins

I have also followed this guide to proxy requests to the docker host with socat: https://github.com/docker/for-mac/issues/770 and here: Docker-compose: deploying service in multiple hosts.

Finally, I'm using the following Jenkins definition (Jenkinsfile) to call stack to deploy on my host. Jenkins has the Jenkins docker plug-in installed:

node {

    checkout scm

    stage ('Deploy To Kube') {

        docker.withServer('tcp://docker.for.mac.localhost:1234') {

            sh 'docker stack deploy app --compose-file /home/project/kube-compose.yml'

        }

    }      

}

I've also tried changing the withServer signature to:

docker.withServer('unix:///var/run/docker.sock')

and I get the same error response. I am, however, able to telnet to the docker host from the Jenkins container so I know it's reachable. Also, as I mentioned earlier, I know the message is saying to run swarm init, but I am not deploying to swarm.

I checked the version of the docker client in the Jenkins container and it is the same version (Linux variant, however) as I'm using on my host:

Docker version 18.06.1-ce, build d72f525745

Here's the code I've described: https://github.com/ewilansky/localstackdeploy.git

Please let me know if it's possible to do what I'm hoping to do from the Jenkins container. The purpose for all of this is to provide a simple, portable demonstration of a pipeline and deploying to Kubernetes is the last step. I understand that this is not the approach that would be taken anywhere outside of a local development environment.