[JIRA] (JENKINS-49567) Automatically support docker networks in pipelines with multiple containers

4 views
Skip to first unread message

andrew@furrypaws.ca (JIRA)

unread,
Feb 14, 2018, 7:38:03 PM2/14/18
to jenkinsc...@googlegroups.com
Andrew Berry created an issue
 
Jenkins / New Feature JENKINS-49567
Automatically support docker networks in pipelines with multiple containers
Issue Type: New Feature New Feature
Assignee: Unassigned
Components: docker-workflow-plugin
Created: 2018-02-15 00:37
Priority: Minor Minor
Reporter: Andrew Berry

As is, it's not possible to create a Jenkinsfile with docker containers that need to communicate with each other (and not just a basic connection from container A to container B). For example, when running Behat tests for Drupal:

  • The webapp container (with PHP etc) çontains Behat, which needs to be able to link to the Selenium container.
  • The Selenium container needs to be able to link back to the webapp container to load pages.

This leads to a chicken-and-the-egg problem, since you can't know the ID of a container before it starts. I imagine there's a way to hack it with container names, but that seems collision prone.

Under other tools like Circle CI, a docker network is transparently created and all containers ports from their Dockerfiles are automatically exposed to each other. For example, even though there are two containers, from both perspectives all ports are available on localhost.

On top of this, the recommended --link parameter has been deprecated by docker, so moving to bridge networks will keep the plugin working with future Docker releases.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

ryan.desmond@radiantsolutions.com (JIRA)

unread,
Mar 24, 2019, 3:24:03 PM3/24/19
to jenkinsc...@googlegroups.com
Ryan Desmond commented on New Feature JENKINS-49567
 
Re: Automatically support docker networks in pipelines with multiple containers

This would be great, with the deprecation of the –link option in docker.

An alternative that would be a more purposeful:

 

docker.withNetwork{ n ->
   docker.image('sidecar').withRun("–network ${n.id}") {
      docker.image('main').inside("–network ${n.id}") {
         // do something
      }
   }
}

 

 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

ryan.desmond@radiantsolutions.com (JIRA)

unread,
Mar 24, 2019, 4:12:02 PM3/24/19
to jenkinsc...@googlegroups.com
Ryan Desmond edited a comment on New Feature JENKINS-49567
This would be great, particularly with the deprecation of the {{–link}} option in docker.

An alternative that would be a more purposeful with a sidecar pattern would be to have a function that surrounds any docker calls and switches each from the default network to a local one :
 {code:java}
docker.withNetwork {
     docker.image('sidecar').withRun() { c->
      docker.image('main').inside() {
         // do something with host "${c.id}"
      }
   }
}
{ code }
 

Currently, I have a function that works around this :

 
{code: java}
 def withDockerNetwork(Closure inner) {
    try {
        network_id = UUID.randomUUID().toString()
        sh " docker network create ${network_id}"
        inner . withNetwork call(network_id)
    } finally {
        sh "docker network rm ${network_id}"
    }
}
{code}
{code:java}
withDockerNetwork{ n ->
    docker.image('sidecar').withRun(" – -- network ${n.id}") { c->
      docker.image('main').inside(" – -- network ${n.id}") {
         // do something with host "${c.id}"
      }
    }
}

{code}
 

 

ryan.desmond@radiantsolutions.com (JIRA)

unread,
Mar 24, 2019, 8:05:03 PM3/24/19
to jenkinsc...@googlegroups.com
Ryan Desmond edited a comment on New Feature JENKINS-49567
This would be great, particularly with the deprecation of the {{ – -- link}} option in docker.


An alternative that would be a more purposeful with a sidecar pattern would be to have a function that surrounds any docker calls and switches each from the default network to a local one:
{code:java}
docker.withNetwork {
   docker.image('sidecar').withRun() { c->
      docker.image('main').inside() {
         // do something with host "${c.id}"
      }
   }
}
{code}
 

Currently, I have a function that works around this:

 
{code:java}
def withDockerNetwork(Closure inner) {
    try {
        network_id = UUID.randomUUID().toString()
        sh "docker network create ${network_id}"
        inner.call(network_id)

    } finally {
        sh "docker network rm ${network_id}"
    }
}
{code}
{code:java}
withDockerNetwork{ n ->
  docker.image('sidecar').withRun("--network ${n.id}") { c->
      docker.image('main').inside("--network ${n.id}") {

         // do something with host "${c.id}"
      }
  }
}
{code}
 

 

ryan.desmond@radiantsolutions.com (JIRA)

unread,
Mar 24, 2019, 8:09:05 PM3/24/19
to jenkinsc...@googlegroups.com
Ryan Desmond edited a comment on New Feature JENKINS-49567
This would be great, particularly with the deprecation of the {{--link}} option in docker.


An alternative that would be a more purposeful with a sidecar pattern would be to have a function that surrounds any docker calls and switches each from the default network to a local one:
{code:java}
docker.withNetwork {
   docker.image('sidecar').withRun() { c->
      docker.image('main').inside() {
         // do something with host "${c.id}"
      }
   }
}
{code}
 

Currently, I have a function that works around this:

 
{code:java}
def withDockerNetwork(Closure inner) {
    try {
        network_id = UUID.randomUUID().toString()
        sh "docker network create ${network_id}"
        inner.call(network_id)
    } finally {
        sh "docker network rm ${network_id}"
    }
}
{code}
{code:java}
withDockerNetwork{ n ->
  docker.image('sidecar').withRun("--network ${n .id }") { c->
      docker.image('main').inside("--network ${n .id }") {

         // do something with host "${c.id}"
      }
  }
}
{code}
 

 

ryan.desmond@radiantsolutions.com (JIRA)

unread,
Mar 25, 2019, 12:42:03 PM3/25/19
to jenkinsc...@googlegroups.com
Ryan Desmond edited a comment on New Feature JENKINS-49567
This would be great, particularly with the deprecation of the {{--link}} option in docker.

An alternative that would be a more purposeful with a sidecar pattern would be to have a function that surrounds any docker calls and switches each from the default network to a local one:
{code:java}
docker.withNetwork {
   docker.image('sidecar').withRun() { c->
      docker.image('main').inside() {
         // do something with host "${c. id name }"

      }
   }
}
{code}
 

Currently, I have a function that works around this:

 
{code:java}
def withDockerNetwork(Closure inner) {
    try {
        network_id         networkId = UUID.randomUUID().toString()
        sh "docker network create ${ network_id networkId }"
        inner.call( network_id networkId )
    } finally {
        sh "docker network rm ${ network_id networkId }"

    }
}
{code}
{code:java}
withDockerNetwork{ n ->
  docker.image('sidecar').withRun("--network ${n} --name sidecar ") { c->
      docker.image('main').inside("--network ${n}") {
         // do something with host " ${c.id} sidecar "
      }
  }
}
{code}
 

 

ryan.desmond@radiantsolutions.com (JIRA)

unread,
Apr 17, 2019, 12:40:02 PM4/17/19
to jenkinsc...@googlegroups.com
Ryan Desmond edited a comment on New Feature JENKINS-49567
This would be great, particularly with the deprecation of the {{--link}} option in docker.

An alternative that would be a more purposeful with a sidecar pattern would be to have a function that surrounds any docker calls and switches each from the default network to a local one:
{code:java}
docker.withNetwork {
   docker.image('sidecar').withRun() { c->
      docker.image('main').inside() {
         // do something with host "${c.name}"

      }
   }
}
{code}
 

Currently, I have a function that works around this:

 
{code:java}
def withDockerNetwork(Closure inner) {
    try {
        networkId = UUID.randomUUID().toString()
        sh "docker network create ${networkId}"
        inner.call(networkId)
    } finally {
        sh "docker network rm ${networkId}"

    }
}
{code}
{code:java}
withDockerNetwork{ n ->
  docker.image('sidecar').withRun("--network ${n} --name sidecar") { c->
      docker.image('main').inside("--network ${n}") {
         // do something with host "sidecar"
      }
  }
}
{code}
  (note that the names have to be globally unique, so they have to be generated)


 

y.vivekanand@gmail.com (JIRA)

unread,
Nov 19, 2019, 2:03:06 AM11/19/19
to jenkinsc...@googlegroups.com

I was looking for the exact same thing, but then we could use the docker.withRun to start a container(pause container) and then join all the other containers to the pause container network.

This gives us the benefit of not having to remember to delete the network and is similar to how AWS fargate or Kubernetes does the networking to provide inter container communciation.
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

y.vivekanand@gmail.com (JIRA)

unread,
Nov 20, 2019, 2:24:04 AM11/20/19
to jenkinsc...@googlegroups.com
Vivek Anand updated an issue
 
Change By: Vivek Anand
Comment:
I was looking for the exact same thing, but then we could use the docker.withRun to start a container(pause container) and then join all the other containers to the pause container network.

This gives us the benefit of not having to remember to delete the network and is similar to how AWS fargate or Kubernetes does the networking to provide inter container communciation.
Reply all
Reply to author
Forward
0 new messages