[JIRA] (JENKINS-36776) Support Windows Server Containers

18 views
Skip to first unread message

cdockens@vertafore.com (JIRA)

unread,
Jul 18, 2016, 6:43:03 PM7/18/16
to jenkinsc...@googlegroups.com
Cody D created an issue
 
Jenkins / New Feature JENKINS-36776
Support Windows Server Containers
Issue Type: New Feature New Feature
Assignee: Jesse Glick
Attachments: windows-container-pipeline.groovy
Components: docker-workflow-plugin
Created: 2016/Jul/18 10:42 PM
Environment: Jenkins 2.7.1
CloudBees Docker Pipeline 1.6
Labels: docker pipeline
Priority: Minor Minor
Reporter: Cody D

I have Jenkins 2.7.1 running on Centos 7.2, and I'm trying to use the docker pipeline commands to run instances of containers on Windows Server 2016 TP5 - https://msdn.microsoft.com/en-us/virtualization/windowscontainers/docker/configure_docker_daemon

I've attached a sample pipeline of what I'm trying to do.

The docker commands seem to hang with:

Still waiting to schedule task
Waiting for next available executor

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

jglick@cloudbees.com (JIRA)

unread,
Jul 25, 2016, 1:57:01 PM7/25/16
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
Change By: Jesse Glick
Labels: docker pipeline windows

jglick@cloudbees.com (JIRA)

unread,
Jul 25, 2016, 1:59:01 PM7/25/16
to jenkinsc...@googlegroups.com
Jesse Glick commented on New Feature JENKINS-36776
 
Re: Support Windows Server Containers

Related to, but I think distinct from, JENKINS-34454.

I have a Windows environment to test Jenkins & its plugins in generally, but due to virtualization limitations I probably cannot simulate the environment you are using.

jglick@cloudbees.com (JIRA)

unread,
Jul 25, 2016, 2:00:01 PM7/25/16
to jenkinsc...@googlegroups.com

Anyway the message you report is not specific to this plugin; it means that there is no match for the label docker&&windows.

cdockens@vertafore.com (JIRA)

unread,
Jul 25, 2016, 3:03:01 PM7/25/16
to jenkinsc...@googlegroups.com
Cody D commented on New Feature JENKINS-36776

The label 'docker&&windows' is to select a Windows Server 2016 agent, which is working correctly.

If my pipeline is as simple as "bat 'echo hello'", that works fine so the label is not a problem.

Instead of running bat, I try to use the docker pipeline commands to spin up an instance of image 'windowsservercore' on that host, and that is the part that is failing with "waiting for available executor".

jglick@cloudbees.com (JIRA)

unread,
Jul 26, 2016, 1:15:02 PM7/26/16
to jenkinsc...@googlegroups.com

Well that might simply be a duplicate of JENKINS-31225, for which I am about to release the fix.

cdockens@vertafore.com (JIRA)

unread,
Oct 7, 2016, 5:20:01 PM10/7/16
to jenkinsc...@googlegroups.com
Cody D commented on New Feature JENKINS-36776

OK, so I download the docker-workflow-plugin source and started hacking around and I've found a few things that get most of the plugin functionality to work on windows server 2016/windows server containers.

1. src\main\resources\org\jenkinsci\plugins\docker\workflow\Docker.groovy
This groovy script has several lines that are calling the CPS 'sh' step. There should be a way to determine the platform and pass the docker command to 'sh' on linux, or 'bat' on windows. For testing purposes I swapped all sh with bat.

2. src\main\java\org\jenkinsci\plugins\docker\workflow\client\DockerClient.java
The whoAmI method calls the 'id' utility on linux to determine the GID/UID of the current user. The docker client appends '-u [user]' when starting the container. I don't know if windows has the same behavior of using a SID in the container that matches a user on the windows host, so I commented out this code, returned null, as well as remove -u option from run for testing purposes.

3. src\main\java\org\jenkinsci\plugins\docker\workflow\WithContainerStep.java
There's a hardcoded 'cat' command that the container starts with. I changed this to 'cmd' for windows.

With these three changes, I was able to successfully pull microsoft/windowsservercore, run the container, then have it stopped and removed. I'm sure build, tag, push would have worked as well, but I didn't get around to testing those features.

Unfortunately, if I try to run 'bat' in docker.image().inside{}, the process hangs indefinitely. The WithContainerStep Decorator decorate() is getting called, and the return statement is executed 'return super.launch(starter);', but then trying to debug the durable task, groovy cps, and others was difficult for me.

cdockens@vertafore.com (JIRA)

unread,
Oct 7, 2016, 5:30:04 PM10/7/16
to jenkinsc...@googlegroups.com
Cody D updated an issue
Change By: Cody D
Environment: Jenkins 2.7. 1 4
CloudBees Docker Pipeline docker-workflow-plugin 1. 6 9

cdockens@vertafore.com (JIRA)

unread,
Oct 7, 2016, 5:37:01 PM10/7/16
to jenkinsc...@googlegroups.com
Cody D updated an issue
I have Jenkins 2 The Docker . 7.1 running on Centos 7.2, groovy and I'm trying to use the docker pipeline WithContainerStep.java files have linux-specific commands that will fail to run instances of containers on Windows Server 2016 TP5 - https://msdn.microsoft.com/en-us/virtualization/windowscontainers/ with docker /configure_docker_daemon installed.

I Docker.groovy invokes CPS ' ve attached a sample pipeline of what I sh ' m trying step in several cases, and tries to do start the container with the 'cat' command .

The
docker commands seem WithContainerStep whoAmI is using the 'id' utility to hang with: determine the user owning the workspace.

{quote}Still waiting to schedule task
Waiting for next available executor{quote}

cdockens@vertafore.com (JIRA)

unread,
Oct 7, 2016, 5:38:01 PM10/7/16
to jenkinsc...@googlegroups.com
Cody D updated an issue
Change By: Cody D
Attachment: windows-container-pipeline.groovy

cdockens@vertafore.com (JIRA)

unread,
Oct 7, 2016, 5:40:03 PM10/7/16
to jenkinsc...@googlegroups.com
Cody D updated an issue
Change By: Cody D
Attachment: docker_pipeline_example.txt

jenkinsio@aidans.org (JIRA)

unread,
Oct 23, 2016, 5:53:02 PM10/23/16
to jenkinsc...@googlegroups.com

Thanks for your research Cody D - that looks super useful. I'm disappointed that `bat` inside the container didn't work, otherwise I'd duplicate your hacking efforts and love to start using this functionality today.

I might take a crack at trying to figure out what's blocking it, but my Java-fu is pretty weak.

todd@deepelement.com (JIRA)

unread,
Mar 29, 2017, 8:40:01 AM3/29/17
to jenkinsc...@googlegroups.com

Has anyone tried installing cygwin (via chocolatey) in the Dockerfile and attempting to set the default shell as the *nix sh?

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

henryborchers@yahoo.com (JIRA)

unread,
Feb 28, 2018, 12:29:02 PM2/28/18
to jenkinsc...@googlegroups.com

It's been a while since any comments were posted here. Is anybody working on this? Docker on Windows would make my life so much easier.

yacoub.hossain@autodesk.com (JIRA)

unread,
Mar 13, 2018, 2:59:02 PM3/13/18
to jenkinsc...@googlegroups.com

i just got this error "Caused: java.io.IOException: Cannot run program "nohup"..." whilst trying to run linux container on windows host. Docker containers can be run on either windows or linux, why does the plugin assume otherwise?

raphael@hoeser.info (JIRA)

unread,
Aug 17, 2018, 4:32:03 AM8/17/18
to jenkinsc...@googlegroups.com

The nohup isn't a big deal, because you can work around it by installing a sh environment on windows (e.g. git with git-scm and add C:\Program Files\Git\usr\bin to the PATH variable).

This isn't a nice workaround, but the real problem is the way Jenkins calls docker!

Jenkins passes a "z" flag in the volume description, which isn't supported by docker for windows and throws an error.

Fixing this, I tried to write a bat and a sh script, which remove the "z" flag and naming them "docker.bat" and "docker" (I renamed docker.exe to docker-client.exe). Running from sh and cmd I was able to run all docker commands, but Jenkins complained about not finding the file.

 

Caused: java.io.IOException: Cannot run program "docker": CreateProcess error=2, The system cannot find the file specified

 

In general I think, that the plugin shouldn't provide the "z" flag outside of an SELinnux environment and that would solve the problem.

This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

lglijo@gmail.com (JIRA)

unread,
Sep 1, 2018, 9:35:03 PM9/1/18
to jenkinsc...@googlegroups.com
Lijo George started work on New Feature JENKINS-36776
 
Change By: Lijo George
Status: Open In Progress
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

lglijo@gmail.com (JIRA)

unread,
Sep 1, 2018, 9:46:02 PM9/1/18
to jenkinsc...@googlegroups.com

henryborchers@yahoo.com (JIRA)

unread,
Sep 18, 2018, 1:47:03 PM9/18/18
to jenkinsc...@googlegroups.com

I'm super excited to see a pull request that's has passed all the required check. I'm wondering what's holding it back from being merged.

Using Windows containers as Jenkins Agents would be very very useful to me for testing in isolation and caching of dependencies and tools.

raphael@hoeser.info (JIRA)

unread,
Sep 28, 2018, 5:23:22 AM9/28/18
to jenkinsc...@googlegroups.com
Raphael Höser stopped work on New Feature JENKINS-36776
 
Change By: Raphael Höser
Status: In Progress Open

raphael@hoeser.info (JIRA)

unread,
Sep 28, 2018, 5:23:23 AM9/28/18
to jenkinsc...@googlegroups.com

Jesse Glick commented under that pull request saying, that the Plugin shoud better be left untouched to prevent "a critical regression".

Even though I do not share his opinion on leaving the plugin untouched, I get his point - especially because the plugin has no active maintainer - and appreciate, that he even looked into this.

So the current state of this problem shouldn't be "In Progress" anymore, as the development won't go on in the foreseeable future.

jglick@cloudbees.com (JIRA)

unread,
Sep 28, 2018, 2:14:10 PM9/28/18
to jenkinsc...@googlegroups.com

Depends on whether existing maintainers (Nicolas De Loof? Andrew Bayer?) are comfortable writing/testing/supporting major new features or refactorings like this; or whether someone else volunteers to take over maintenance. For the time being, withDockerContainer remains in fairly widespread use via the agent docker (or agent dockerfile) syntax in Declarative Pipeline, though I have proposed an alternative agent type based on the dockerNode step in the docker plugin, which seems less problematic. (It, too, is completely untested on Windows containers—pending INFRA-1400.)

nicolas.deloof@gmail.com (JIRA)

unread,
Sep 29, 2018, 4:55:03 AM9/29/18
to jenkinsc...@googlegroups.com

I just got a PR merged on this plugin, I don't think that makes me a maintainer, does it ?

I have no experience with windows container, and no personal interest for them, so won't be a great support for such a feature - whenever I fully understand the interest for others. 

 

gegles@gmail.com (JIRA)

unread,
Feb 20, 2019, 5:23:03 PM2/20/19
to jenkinsc...@googlegroups.com

Any progress update on this?

 

It would be so nice to be able to leverage Windows Container support for our jenkins build!

gegles@gmail.com (JIRA)

unread,
Jul 9, 2019, 5:57:03 PM7/9/19
to jenkinsc...@googlegroups.com

raphael@hoeser.info (JIRA)

unread,
Jul 10, 2019, 11:43:04 AM7/10/19
to jenkinsc...@googlegroups.com

Guillaume Egles I don't know whether you really need direct docker support, but we moved onto using Kubernetes in our setup - you just need a windows based jnlp container which you insert as "jnlp" into your pod and then you can have windows based kubernetes pods.

That way you need one Linux Kube Master Node in your cluster and after that you can mix and match windows and linux containers in seperate pods as you need for your builds. 

Note for high scalings: Avoid running Linux Containers on Windows nodes, since this is possible with current builds, but the windows process scheduler doesn't handle high core counts well (up to 16 works totally fine).

isanych@gmail.com (JIRA)

unread,
Jul 10, 2019, 12:08:02 PM7/10/19
to jenkinsc...@googlegroups.com

 It is much easier to use docker client for running jobs and deal with corresponding issues (like interrupting job would not stop running container) than to setup Kubernetes cluster  just for that.

francois.rivard@gmail.com (JIRA)

unread,
Sep 3, 2019, 3:52:04 PM9/3/19
to jenkinsc...@googlegroups.com

> the windows process scheduler doesn't handle high core counts well

Please see this : https://techcommunity.microsoft.com/t5/Windows-Kernel-Internals/One-Windows-Kernel/ba-p/267142

It supports 896 cores / 1792 logical processors. I don't think your computer has that many cores.

josephp90@gmail.com (JIRA)

unread,
Oct 10, 2019, 4:19:16 AM10/10/19
to jenkinsc...@googlegroups.com
Joseph Petersen updated an issue
Change By: Joseph Petersen
Component/s: pipeline-model-definition-plugin
Component/s: docker-workflow-plugin
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

josephp90@gmail.com (JIRA)

unread,
Oct 10, 2019, 4:20:05 AM10/10/19
to jenkinsc...@googlegroups.com
Joseph Petersen updated an issue
Change By: Joseph Petersen
Component/s: docker-workflow-plugin
Component/s: pipeline-model-definition-plugin

josephp90@gmail.com (JIRA)

unread,
Oct 10, 2019, 4:21:32 AM10/10/19
to jenkinsc...@googlegroups.com

josephp90@gmail.com (JIRA)

unread,
Mar 7, 2020, 1:07:40 AM3/7/20
to jenkinsc...@googlegroups.com
Joseph Petersen assigned an issue to Joseph Petersen
Change By: Joseph Petersen
Assignee: Joseph Petersen (old)
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages