[JIRA] [core] (JENKINS-12699) Temp upload files from file param not removed after transferred to slave

chris@orr.me.uk (JIRA)

unread,

Jun 30, 2015, 8:44:01 AM6/30/15

to jenkinsc...@googlegroups.com

Christopher Orr reopened an issue

Reopening as per

~~JENKINS-29121~~ .

Jenkins /

JENKINS-12699

Temp upload files from file param not removed after transferred to slave

Change By:	Christopher Orr
Resolution:	Incomplete
Status:	Resolved Reopened

Add Comment

This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)

chris@orr.me.uk (JIRA)

unread,

Jun 30, 2015, 8:46:02 AM6/30/15

to jenkinsc...@googlegroups.com

Christopher Orr commented on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

I'm not seeing this behaviour with large file parameters, whether the build is executed on master or a slave.

But I'm not using buildWithParameters, but rather the build format documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Remote+access+API#RemoteaccessAPI-Submittingjobs

i.e.

 
                                                    curl -v -X POST http://jenkins.local:8080/job/upload-test/build \
  --form f0=@/data/fw123.bin \
  --form json='{"parameter": [{"name":"firmware.bin", "file":"f0"}]}'

When the job is running, I see "Copying file to firmware.bin" in the console output.
But checking /tmp and the java.io.tmpdir directory on the slave while this is happening shows no temporary files being created.

Elgin Peteza: Can you provide more detail on exactly how you're starting a job?

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 12:48:05 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza commented on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job via a remote PC and POSTing to Jenkins using python's requests module.
i.e.

                                                        Unable to find source-code formatter for language: python. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml
                                                    
import requests
with open("file.zip", "rb") as z:
    file_to_send = {"jenkins_file.zip": z}
    requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)

An upload_*.tmp file is created in /tmp directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that jenkins_file.zip is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's java.io.tmpdir directory.

The instructions for configuring my job's parameters are documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

[~Christopher Orr]
I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the http://jenkins.local:8080/job/my_job/build_number/parameters page. Although, I noticed that this time, it automatically deleted the upload_*.tmp that was generated.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 12:48:05 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job via a remote PC and POSTing to Jenkins using python's requests module.
i.e.

{code:python}

import requests
with open("file.zip", "rb") as z:
file_to_send = {"jenkins_file.zip": z}
requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)

{code}

An {{upload_*.tmp}} file is created in {{/tmp}} directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that {{jenkins_file.zip}} is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's {{java.io.tmpdir}} directory.

The instructions for configuring my job's parameters are documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

[ ~ Christopher Orr]

I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the {{http://jenkins.local:8080/job/my_job/build_number/parameters}} page. Although, I noticed that this time, it automatically deleted the {{upload_*.tmp}} that was generated.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 12:49:02 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job via a remote PC and POSTing to Jenkins using python's requests module.
i.e.
{code :python

}

import requests
with open("file.zip", "rb") as z:
file_to_send = {"jenkins_file.zip": z}
requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)
{code}
An {{upload_*.tmp}} file is created in {{/tmp}} directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that {{jenkins_file.zip}} is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's {{java.io.tmpdir}} directory.

The instructions for configuring my job's parameters are documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

[Christopher Orr]

I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the {{http://jenkins.local:8080/job/my_job/build_number/parameters}} page. Although, I noticed that this time, it automatically deleted the {{upload_*.tmp}} that was generated.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 12:51:01 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job via a remote PC and POSTing to Jenkins using python's requests module.
i.e.
{code}

import requests
with open("file.zip", "rb") as z:
file_to_send = {"jenkins_file.zip": z}
requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)
{code}
An {{upload_*.tmp}} file is created in {{/tmp}} directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that {{jenkins_file.zip}} is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's {{java.io.tmpdir}} directory.

The instructions for configuring my job's parameters are documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

[ Christopher Orr ] :
I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the {{http://jenkins.local:8080/job/my_job/build_number/parameters}} page. Although, I noticed that this time, it automatically deleted the {{upload_*.tmp}} that was generated.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 12:52:02 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job via a remote PC and POSTing to Jenkins using python's requests module.
i.e.
{code}
import requests
with open("file.zip", "rb") as z:
file_to_send = {"jenkins_file.zip": z}
requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)
{code}
An {{upload_*.tmp}} file is created in {{/tmp}} directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that {{jenkins_file.zip}} is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's {{java.io.tmpdir}} directory.

The instructions for configuring my job's parameters are documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

Christopher Orr [~orrc] :

I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the {{http://jenkins.local:8080/job/my_job/build_number/parameters}} page. Although, I noticed that this time, it automatically deleted the {{upload_*.tmp}} that was generated.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 4:07:02 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job via a remote PC and by POSTing to Jenkins using python's requests module via a remote PC .

i.e.
{code}
import requests
with open("file.zip", "rb") as z:
file_to_send = {"jenkins_file.zip": z}
requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)
{code}
An {{upload_*.tmp}} file is created in {{/tmp}} directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that {{jenkins_file.zip}} is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's {{java.io.tmpdir}} directory.

The instructions for configuring my job's parameters are documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

[~orrc]:
I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the {{http://jenkins.local:8080/job/my_job/build_number/parameters}} page. Although, I noticed that this time, it automatically deleted the {{upload_*.tmp}} that was generated.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 4:10:02 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job by POSTing to Jenkins using python's requests module via a remote PC.

i.e.
{code}
import requests
with open("file.zip", "rb") as z:
file_to_send = {"jenkins_file.zip": z}
requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)
{code}
An {{upload_*.tmp}} file is created in {{/tmp}} directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that {{jenkins_file.zip}} is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's {{java.io.tmpdir}} directory.

The instructions for configuring my job's parameters are documented here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

[~orrc]:
I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the {{http://jenkins.local:8080/job/my_job/build_number/parameters}} page. Although, I noticed

that this time, it automatically deleted the {{upload_*.tmp}} that was generated.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 1, 2015, 4:15:03 AM7/1/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Basically, I execute a job by POSTing to Jenkins using python's requests module via a remote PC.
i.e.
{code}
import requests
with open("file.zip", "rb") as z:
file_to_send = {"jenkins_file.zip": z}
requests.post("http://jenkins.local:8080/job/my_job/buildWithParameters", files=file_to_send)
{code}
An {{upload_*.tmp}} file is created in {{/tmp}} directory of the master after executing the script. The master then builds the job on one of its nodes but the tmp file is not deleted even after the build is completed.

I observed that {{jenkins_file.zip}} is created in the node's workspace once the job starts building (as expected) but no file is copied/created in the node's {{java.io.tmpdir}} directory.

The instructions guide I used for configuring my job's parameters are documented can be found here:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build

[~orrc]:
I tried using your curl command with my job but it doesn't seem to set the parameters correctly. The parameters do not reflect on the {{http://jenkins.local:8080/job/my_job/build_number/parameters}} page. Although, I noticed this time, it automatically deleted the {{upload_*.tmp}} that was generated.

Add Comment

dbeck@cloudbees.com (JIRA)

unread,

Jul 1, 2015, 8:23:01 AM7/1/15

to jenkinsc...@googlegroups.com

Daniel Beck commented on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Christopher Orr Jenkins itself (on job/foo/api) documents that parameterized builds should be started via /buildWithParameters. The POST to /build with JSON is just replicating what the browser does when you submit the form and IMO therefore more fragile/subject to change.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 2, 2015, 3:09:02 AM7/2/15

to jenkinsc...@googlegroups.com

Elgin Peteza commented on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Update:
I encountered another problem with the upload_*.tmp files.

I executed many jobs such that several builds will be queued on one node (and several upload_.tmp files created in /tmp directory of the master), and then I restarted Jenkins in the middle. Those builds before restart were built successfully (upload_.tmp files still not deleted). The remaining builds after restart, surprisingly, failed with the following error in the console:
{{FATAL: null
java.lang.NullPointerException
at hudson.model.FileParameterValue$2.setUp(FileParameterValue.java:143)
at hudson.model.Build$BuildExecution.doRun(Build.java:156)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1744)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)}}
All upload_*.tmp files were still in /tmp directory when the error occurred.
From previous successful builds, the console showed:
Copying file to jenkins_file.zip

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 2, 2015, 3:10:01 AM7/2/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Update:
I encountered another problem with the {{upload_ \ *.tmp}} files.

I executed many jobs such that several builds will be queued on one node (and several {{upload_ \ *.tmp}} files created in {{/tmp}} directory of the master), and then I restarted Jenkins in the middle. Those builds before restart were built successfully ({{upload_ \ *.tmp}} files still not deleted). The remaining builds after restart, surprisingly, failed with the following error in the console:

{{FATAL: null
java.lang.NullPointerException
at hudson.model.FileParameterValue$2.setUp(FileParameterValue.java:143)
at hudson.model.Build$BuildExecution.doRun(Build.java:156)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1744)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)}}

All {{upload_ \ *.tmp}} files were still in {{/tmp}} directory when the error occurred.

From previous successful builds, the console showed:
{{Copying file to jenkins_file.zip}}

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 2, 2015, 3:11:01 AM7/2/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Update:
I encountered another problem with the {{upload_\*.tmp}} files.

I executed many jobs such that several builds will be queued on one node (and several {{upload_\*.tmp}} files created in {{/tmp}} directory of the master), and then I restarted Jenkins in the middle. Those builds before restart were built successfully ({{upload_\*.tmp}} files still not deleted). The remaining builds after restart, surprisingly, failed with the following error in the console:

{ { code:java}

FATAL: null
java.lang.NullPointerException
at hudson.model.FileParameterValue$2.setUp(FileParameterValue.java:143)
at hudson.model.Build$BuildExecution.doRun(Build.java:156)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1744)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)

{code } }

All {{upload_\*.tmp}} files were still in {{/tmp}} directory when the error occurred.
From previous successful builds, the console showed:
{{Copying file to jenkins_file.zip}}

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 2, 2015, 3:12:01 AM7/2/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Update:
I encountered another problem with the {{upload_\*.tmp}} files.

I executed the job many jobs times such that several builds will be queued on one node (and several {{upload_\*.tmp}} files created in {{/tmp}} directory of the master), and then I restarted Jenkins in the middle. Those builds before restart were built successfully ({{upload_\*.tmp}} files still not deleted). The remaining builds after restart, surprisingly, failed with the following error in the console:

{code:java}
FATAL: null
java.lang.NullPointerException
at hudson.model.FileParameterValue$2.setUp(FileParameterValue.java:143)
at hudson.model.Build$BuildExecution.doRun(Build.java:156)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1744)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)
{code}

All {{upload_\*.tmp}} files were still in {{/tmp}} directory when the error occurred.
From previous successful builds, the console showed:
{{Copying file to jenkins_file.zip}}

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 2, 2015, 3:13:01 AM7/2/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Update:
I encountered another problem with the {{upload_\*.tmp}} files.

I executed the job many times such that several builds will be queued on one node (and several {{upload_\*.tmp}} files created in {{/tmp}} directory of the master), and then I restarted Jenkins in the middle. Those builds before restart were built successfully ({{upload_\*.tmp}} files still not deleted). The remaining builds after restart, surprisingly, failed with the following error in the console:

{code:java}
FATAL: null
java.lang.NullPointerException
at hudson.model.FileParameterValue$2.setUp(FileParameterValue.java:143)
at hudson.model.Build$BuildExecution.doRun(Build.java:156)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1744)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)
{code}

All {{upload_\*.tmp}} files were still in {{/tmp}} directory when the error occurred.

From previous successful builds, the console showed:

{ { code:java}
Copying file to jenkins_file.zip
{code } }

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 2, 2015, 3:22:01 AM7/2/15

to jenkinsc...@googlegroups.com

All {{upload_\*.tmp}} files were still in {{/tmp}} directory after the error occurred.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 3, 2015, 1:48:02 AM7/3/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Update:
I encountered another problem with the {{upload_\*.tmp}} files.

I executed the job many times such that several builds will be queued on one node (and several {{upload_\*.tmp}} files created in {{/tmp}} directory of the master), and then I restarted Jenkins in the middle. Those builds before restart were built successfully ({{upload_\*.tmp}} files still not deleted). The remaining builds after restart, surprisingly, failed with the following error in the console:

{code:java}
FATAL: null
java.lang.NullPointerException
at hudson.model.FileParameterValue$2.setUp(FileParameterValue.java:143)
at hudson.model.Build$BuildExecution.doRun(Build.java:156)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1744)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)
{code}

From previous successful builds, the console showed:

{code:java}
Copying file to jenkins_file.zip
{code}

All {{upload_\*.tmp}} files were still in {{/tmp}} directory after the error occurred.

Edited:
Maybe this is an entirely different bug. Based on my tests, only file parameters greater than 10kB generate {{upload_\*.tmp}} files in {{/tmp}} directory. The error described above also occurred even for file parameters smaller than 10kB, which didn't create an {{upload_\*.tmp}} in the first place.

Add Comment

ecpeteza@yahoo.com (JIRA)

unread,

Jul 3, 2015, 1:48:02 AM7/3/15

to jenkinsc...@googlegroups.com

Elgin Peteza edited a comment on

JENKINS-12699

Re: Temp upload files from file param not removed after transferred to slave

Update:
I encountered another problem with the {{upload_\*.tmp}} files.

I executed the job many times such that several builds will be queued on one node (and several {{upload_\*.tmp}} files created in {{/tmp}} directory of the master), and then I restarted Jenkins in the middle. Those builds before restart were built successfully ({{upload_\*.tmp}} files still not deleted). The remaining builds after restart, surprisingly, failed with the following error in the console:

{code:java}
FATAL: null
java.lang.NullPointerException
at hudson.model.FileParameterValue$2.setUp(FileParameterValue.java:143)
at hudson.model.Build$BuildExecution.doRun(Build.java:156)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1744)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)
{code}

From previous successful builds, the console showed:

{code:java}
Copying file to jenkins_file.zip
{code}

All {{upload_\*.tmp}} files were still in {{/tmp}} directory after the error occurred.

Edited

Added :

Maybe this is an entirely different bug. Based on my tests, only file parameters greater than 10kB generate {{upload_\*.tmp}} files in {{/tmp}} directory. The error described above also occurred even for file parameters smaller than 10kB, which didn't create an {{upload_\*.tmp}} in the first place.