[JIRA] (JENKINS-49759) Can't create temp file in Kubernetes Plugin Pipeline

2 views
Skip to first unread message

michael@macfadden.org (JIRA)

unread,
Feb 26, 2018, 10:56:02 PM2/26/18
to jenkinsc...@googlegroups.com
Michael MacFadden created an issue
 
Jenkins / Bug JENKINS-49759
Can't create temp file in Kubernetes Plugin Pipeline
Issue Type: Bug Bug
Assignee: Carlos Sanchez
Attachments: failure.log
Components: kubernetes-plugin
Created: 2018-02-27 03:55
Environment: Kuberetes 1.8
Jenkins 2.107
Kubernetes Plugin: 1.2.1
Java Version: 1.8.0_151
Jenkins Master running in Docker Container in k8s
Labels: pipeline slave kubernetes-plugin
Priority: Major Major
Reporter: Michael MacFadden

We are having an issue where we can not create a temp file in a build pipeline within a PodTemplate. Oddly inserting a random 'sh' command before the creation of the temp file solved the problem. Here is the code that fails:

 

 import hudson.FilePath;
import jenkins.model.Jenkins;

def label = "mynode"
podTemplate(
  label: label,
  nodeSelector: "kops.k8s.io/instancegroup=jenkins-workers",
  containers: [containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.9.3', command: 'cat', ttyEnabled: true)]) {

  node(label) {
    container("kubectl") {
      def workspace = new FilePath(Jenkins.getInstance().getComputer(env.NODE_NAME).getChannel(), env.WORKSPACE);
      workspace.createTempFile("test", ".txt")
      sh "ls -al"
    }
  }
}

 

The full log is attached, but essentially we get the following exception:

Caused: java.io.IOException: Failed to create a temp file on /home/jenkins/workspace/folder/test-job
at hudson.FilePath.createTempFile(FilePath.java:1330)

Oddly, if we add a "sh" command before the creation of the temp file the job succeeds. The following code works:

 import hudson.FilePath;
import jenkins.model.Jenkins;

def label = "mynode"
podTemplate(
  label: label,
  nodeSelector: "kops.k8s.io/instancegroup=jenkins-workers",
  containers: [containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.9.3', command: 'cat', ttyEnabled: true)]) {

  node(label) {
    container("kubectl") {
      sh "ls -al"
      def workspace = new FilePath(Jenkins.getInstance().getComputer(env.NODE_NAME).getChannel(), env.WORKSPACE);
      workspace.createTempFile("test", ".txt")
      sh "ls -al"
    }
  }
}

It does not seem to matter what the "sh" command before the temp file creation is. We have used "pwd", "echo $HOME", etc. and they all have the same effect of making the pipeline work. Of course both sh "ls -al" commands were put in there as debugging steps, and it just so happened that the first one accidentally made the pipeline work.

 

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

michael@macfadden.org (JIRA)

unread,
Feb 26, 2018, 10:57:03 PM2/26/18
to jenkinsc...@googlegroups.com
Michael MacFadden updated an issue
Change By: Michael MacFadden
We are having an issue where we can not create a temp file in a build pipeline within a PodTemplate. Oddly inserting a random 'sh' command before the creation of the temp file solved the problem. Here is the code that fails:

 
{code:java}

import hudson.FilePath;
import jenkins.model.Jenkins;

def label = "mynode"
podTemplate(
  label: label,
  nodeSelector: "kops.k8s.io/instancegroup=jenkins-workers",
  containers: [containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.9.3', command: 'cat', ttyEnabled: true)]) {

  node(label) {
    container("kubectl") {
      def workspace = new FilePath(Jenkins.getInstance().getComputer(env.NODE_NAME).getChannel(), env.WORKSPACE);
      workspace.createTempFile("test", ".txt")
      sh "ls -al"
    }
  }
}
{code}

 

The full log is attached, but essentially we get the following exception:
{code:java}

Caused: java.io.IOException: Failed to create a temp file on /home/jenkins/workspace/folder/test-job
at hudson.FilePath.createTempFile(FilePath.java:1330)
{code}

Oddly, if we add a "sh" command before the creation of the temp file the job succeeds. The following code works:
{code:java}

import hudson.FilePath;
import jenkins.model.Jenkins;

def label = "mynode"
podTemplate(
  label: label,
  nodeSelector: "kops.k8s.io/instancegroup=jenkins-workers",
  containers: [containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.9.3', command: 'cat', ttyEnabled: true)]) {

  node(label) {
    container("kubectl") {
      sh "ls -al" // This added code somehow makes this pipeline succeed.
      def workspace = new FilePath(Jenkins.getInstance().getComputer(env.NODE_NAME).getChannel(), env.WORKSPACE);
      workspace.createTempFile("test", ".txt")
      sh "ls -al"
    }
  }
}
{code}

It does not seem to matter what the "sh" command before the temp file creation is. We have used "pwd", "echo $HOME", etc. and they all have the same effect of making the pipeline work. Of course both sh "ls -al" commands were put in there as debugging steps, and it just so happened that the first one accidentally made the pipeline work.

 

jenkins-ci@carlossanchez.eu (JIRA)

unread,
Feb 27, 2018, 2:54:02 AM2/27/18
to jenkinsc...@googlegroups.com
Carlos Sanchez commented on Bug JENKINS-49759
 
Re: Can't create temp file in Kubernetes Plugin Pipeline

I'm actually quite surprised it works at all.
From my understanding createTempFile runs on the agent container, and because the filesystem is shared you could see the file in all the containers. So it would work when outside the container step.

Inside container you can execute commands, but that's all you can do because it uses kubectl exec

michael@macfadden.org (JIRA)

unread,
Feb 27, 2018, 9:58:01 AM2/27/18
to jenkinsc...@googlegroups.com

Carlos Sanchez Thanks for your comments, and thanks for looking at this. You'll have to forgive my analysis here since I am fairly new at looking through the Jenkins Code Base and understanding how it works under the hood.

The behavior is the same regardless of if you put it inside the "container" block or not. It works with the added "sh" command and does not work without it. So it really doesn't' matter if you put it in the container, or directly in the node. Like so:

podTemplate(
  label: label,
  nodeSelector: "kops.k8s.io/instancegroup=jenkins-workers",
  containers: [containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.9.3', command: 'cat', ttyEnabled: true
)]) {

  node(label) {
    sh "ls -al" // Works with this, does not work without it.
    def workspace = new FilePath(Jenkins.getInstance().getComputer(env.NODE_NAME).getChannel(), env.WORKSPACE);
    workspace.createTempFile("test", ".txt")
    sh "ls -al"
  }
}
 
                                                            

I believe that pipeline steps are interpreted on the master and the sent to the slaves . So when you call createTempFile, the master is interpreting that request and ultimately asks the salve to create the file. This salve happens to be the JNLP slave running in a container itself. The point being "createTempFile" is not running "inside" the container, no matter where it appears in the pipeline code block. It's actually the master that interprets it and then asks the slave to create the file. In the case of the createTempFile, it's not going to actually ask the container to do it, but rather the JNLP Slave container running of the JNLP protocol directly to that container (not using kubectl exec).

You are correct, then that the filesystem is shared between the JNLP Slave and the kubectl container, so the temp file should then be available inside the kubectl container.

So all that said, I think the issue is still valid.

 

michael@macfadden.org (JIRA)

unread,
Feb 27, 2018, 9:59:03 AM2/27/18
to jenkinsc...@googlegroups.com
Michael MacFadden edited a comment on Bug JENKINS-49759
[~csanchez] Thanks for your comments, and thanks for looking at this. You'll have to forgive my analysis here since I am fairly new at looking through the Jenkins Code Base and understanding how it works under the hood.


The behavior is the same regardless of if you put it inside the "container" block or not. It works with the added "sh" command and does not work without it. So it really doesn't' matter if you put it in the container, or directly in the node. Like so:
{code:java}

podTemplate(
  label: label,
  nodeSelector: "kops.k8s.io/instancegroup=jenkins-workers",
  containers: [containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.9.3', command: 'cat', ttyEnabled: true)]) {

  node(label) {
    sh "ls -al" // Works with this, does not work without it.
    def workspace = new FilePath(Jenkins.getInstance().getComputer(env.NODE_NAME).getChannel(), env.WORKSPACE);
    workspace.createTempFile("test", ".txt")
    sh "ls -al"

    container("kubectl") {
      // some commands inside the container.
  }
}
}
{code}

I believe that pipeline steps are interpreted on the master and the sent to the slaves . So when you call createTempFile, the master is interpreting that request and ultimately asks the salve to create the file. This salve happens to be the JNLP slave running in a container itself. The point being "createTempFile" is not running "inside" the container, no matter where it appears in the pipeline code block. It's actually the master that interprets it and then asks the slave to create the file. In the case of the createTempFile, it's not going to actually ask the container to do it, but rather the JNLP Slave container running of the JNLP protocol directly to that container (not using kubectl exec).

You are correct, then that the filesystem is shared between the JNLP Slave and the kubectl container, so the temp file should then be available inside the kubectl container.

So all that said, I think the issue is still valid.

 

michael@macfadden.org (JIRA)

unread,
Feb 27, 2018, 10:00:03 AM2/27/18
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Jul 16, 2019, 3:43:44 PM7/16/19
to jenkinsc...@googlegroups.com
Jesse Glick assigned an issue to Unassigned
 
Change By: Jesse Glick
Assignee: Carlos Sanchez
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
Reply all
Reply to author
Forward
0 new messages