Build failing because Jenkins is unable to delete a tmp file

2,178 views
Skip to first unread message

Stefano Perticoni

unread,
Aug 31, 2012, 8:15:05 AM8/31/12
to jenkins...@googlegroups.com
Dear Jenkins users,

we are running Jenkins 1.472

and we are experiencing this strange error from some time on a linux build node:

Error first line:
30-Aug-2012 11:50:56 ---> Debug: ("mafPluginOutFATAL: Unable to delete script file /tmp/hudson7088876974817491254.sh

Full log is available at the end of the email.

It seems that Jenkins is not able to delete a file even if Jenkins login user can do that: in fact by ssh the Jenkins user can both delete the file and run a successful build.

Any idea?



Thank you very much for your help.

Regards,
Stefano

_______________________________________________

Ing. Stefano Perticoni
Super Computing Solutions s.r.l.


Via Parini 1, 40033 Casalecchio di Reno BO, Italy


mailto: s.perticoni@scsitaly.com
_______________________________________________




30-Aug-2012 11:50:56 ---> Debug: ("mafPluginOutFATAL: Unable to delete script file /tmp/hudson7088876974817491254.sh
hudson.util.IOException2: remote file operation failed: /tmp/hudson7088876974817491254.sh at hudson.remoting.Channel@10a1001:SCSSVN01
	at hudson.FilePath.act(FilePath.java:838)
	at hudson.FilePath.act(FilePath.java:824)
	at hudson.FilePath.delete(FilePath.java:1129)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
	at hudson.model.Build$BuildExecution.build(Build.java:199)
	at hudson.model.Build$BuildExecution.doRun(Build.java:160)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1488)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:236)
Caused by: hudson.remoting.ChannelClosedException: channel is already closed
	at hudson.remoting.Channel.send(Channel.java:492)
	at hudson.remoting.Request.call(Request.java:129)
	at hudson.remoting.Channel.call(Channel.java:663)
	at hudson.FilePath.act(FilePath.java:831)
	... 13 more
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
	at java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.io.ObjectInputStream.readObject(Unknown Source)
	at hudson.remoting.Command.readFrom(Command.java:90)
	at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.call(Request.java:174)
	at hudson.remoting.Channel.call(Channel.java:663)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
	at $Proxy35.join(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:861)
	at hudson.Launcher$ProcStarter.join(Launcher.java:345)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
	at hudson.model.Build$BuildExecution.build(Build.java:199)
	at hudson.model.Build$BuildExecution.doRun(Build.java:160)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1488)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:236)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.abort(Request.java:299)
	at hudson.remoting.Channel.terminate(Channel.java:719)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
	at java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.io.ObjectInputStream.readObject(Unknown Source)
	at hudson.remoting.Command.readFrom(Command.java:90)
	at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

JanneG

unread,
Aug 31, 2012, 8:56:59 AM8/31/12
to jenkins...@googlegroups.com
We're experiencing the same problem, seems like no one can give a hint on what is causing it to randomly fail once a week.
We're running Jenkins 1.467.

BR,

/Janne


Den fredagen den 31:e augusti 2012 kl. 14:15:05 UTC+2 skrev Stefano Perticoni:
Dear Jenkins users,

we are running Jenkins 1.472

and we are experiencing this strange error from some time on a linux build node:

Error first line:
30-Aug-2012 11:50:56 ---> Debug: ("mafPluginOutFATAL: Unable to delete script file /tmp/hudson7088876974817491254.sh

Full log is available at the end of the email.

It seems that Jenkins is not able to delete a file even if Jenkins login user can do that: in fact by ssh the Jenkins user can both delete the file and run a successful build.

Any idea?



Thank you very much for your help.

Regards,
Stefano

_______________________________________________

Ing. Stefano Perticoni
Super Computing Solutions s.r.l.


Via Parini 1, 40033 Casalecchio di Reno BO, Italy


mailto: s.per...@scsitaly.com
_______________________________________________

Mark Waite

unread,
Aug 31, 2012, 9:31:34 AM8/31/12
to jenkins...@googlegroups.com
That stack trace surprises me because there seems to be a space in the file name which is being deleted.  I've not seen that myself in any of my Jenkins configurations.  Is something which you have configured which might have caused Jenkins to insert a space in the file name?  (Just guessing, since I don't see any obvious reasons for the issue).

Mark Waite

From: JanneG <janne...@gmail.com>
To: jenkins...@googlegroups.com
Sent: Friday, August 31, 2012 6:56 AM
Subject: Re: Build failing because Jenkins is unable to delete a tmp file

We're experiencing the same problem, seems like no one can give a hint on what is causing it to randomly fail once a week.
We're running Jenkins 1.467.

BR,

/Janne

Den fredagen den 31:e augusti 2012 kl. 14:15:05 UTC+2 skrev Stefano Perticoni:
Dear Jenkins users,

we are running Jenkins 1.472

and we are experiencing this strange error from some time on a linux build node:

Error first line:
30-Aug-2012 11:50:56 ---> Debug: ("mafPluginOutFATAL: Unable to delete script file /tmp/ hudson7088876974817491254.sh

Full log is available at the end of the email.

It seems that Jenkins is not able to delete a file even if Jenkins login user can do that: in fact by ssh the Jenkins user can both delete the file and run a successful
 build.

Any idea?



Thank you very much for your help.

Regards,
Stefano

______________________________ _________________

Ing. Stefano Perticoni
Super Computing Solutions s.r.l.


Via Parini 1, 40033 Casalecchio di Reno BO, Italy


mailto: s.per...@scsitaly. com
______________________________ _________________




30-Aug-2012 11:50:56 ---> Debug: ("mafPluginOutFATAL: Unable to delete script file /tmp/ hudson7088876974817491254.sh

hudson.util.IOException2: remote file operation failed: /tmp/ hudson7088876974817491254.sh at hudson.remoting.Channel@ 10a1001:SCSSVN01
	at hudson.FilePath.act(FilePath. java:838)
	at hudson.FilePath.act(FilePath. java:824)
	at hudson.FilePath.delete( FilePath.java:1129)
	at hudson.tasks. CommandInterpreter.perform( CommandInterpreter.java:92)
	at hudson.tasks. CommandInterpreter.perform( CommandInterpreter.java:58)
	at hudson.tasks.BuildStepMonitor$ 1.perform(BuildStepMonitor. java:19)
	at hudson.model.AbstractBuild$ AbstractBuildExecution. perform(AbstractBuild.java: 717)
	at hudson.model.Build$ BuildExecution.build(Build. java:199)
	at hudson.model.Build$ BuildExecution.doRun(Build. java:160)
	at hudson.model.AbstractBuild$ AbstractBuildExecution.run( AbstractBuild.java:499)
	at hudson.model.Run.execute(Run. java:1488)
	at hudson.model.FreeStyleBuild. run(FreeStyleBuild.java:46)
	at hudson.model. ResourceController.execute( ResourceController.java:88)
	at hudson.model.Executor.run( Executor.java:236)
	... 13 more
Caused by: java.io
.IOException: Unexpected termination of the channel
	at hudson.remoting. SynchronousCommandTransport$ ReaderThread.run( SynchronousCommandTransport. java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$ BlockDataInputStream.peekByte( Unknown Source)
	at java.io.ObjectInputStream. readObject0(Unknown Source)
	at java.io.ObjectInputStream. readObject(Unknown Source)
	at hudson.remoting.Command. readFrom(Command.java:90)
	at hudson.remoting. ClassicCommandTransport.read( ClassicCommandTransport.java: 59)
	at hudson.remoting. SynchronousCommandTransport$ ReaderThread.run( SynchronousCommandTransport. java:48)
FATAL: hudson.remoting. RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting. RequestAbortedException: hudson.remoting. RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.call( Request.java:174)
	at hudson.remoting.Channel.call( Channel.java:663)
	at hudson.remoting. RemoteInvocationHandler. invoke( RemoteInvocationHandler.java: 158)
	at $Proxy35.join(Unknown Source)
	at hudson.Launcher$ RemoteLauncher$ProcImpl.join( Launcher.java:861)
	at hudson.Launcher$ProcStarter. join(Launcher.java:345)
	at hudson.tasks. CommandInterpreter.perform( CommandInterpreter.java:82)
	at hudson.tasks. CommandInterpreter.perform( CommandInterpreter.java:58)
	at hudson.tasks.BuildStepMonitor$ 1.perform(BuildStepMonitor. java:19)
	at hudson.model.AbstractBuild$ AbstractBuildExecution. perform(AbstractBuild.java: 717)
	at hudson.model.Build$ BuildExecution.build(Build. java:199)
	at hudson.model.Build$ BuildExecution.doRun(Build. java:160)
	at hudson.model.AbstractBuild$ AbstractBuildExecution.run( AbstractBuild.java:499)
	at hudson.model.Run.execute(Run. java:1488)
	at hudson.model.FreeStyleBuild. run(FreeStyleBuild.java:46)
	at hudson.model. ResourceController.execute( ResourceController.java:88)
	at hudson.model.Executor.run( Executor.java:236)
Caused by: hudson.remoting. RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.abort( Request.java:299)
	at hudson.remoting.Channel. terminate(Channel.java:719)
	at hudson.remoting. SynchronousCommandTransport$ ReaderThread.run( SynchronousCommandTransport. java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting. SynchronousCommandTransport$ ReaderThread.run( SynchronousCommandTransport. java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$ BlockDataInputStream.peekByte( Unknown Source)
	at java.io.ObjectInputStream. readObject0(Unknown Source)
	at java.io.ObjectInputStream. readObject(Unknown Source)
	at hudson.remoting.Command. readFrom(Command.java:90)
	at hudson.remoting. ClassicCommandTransport.read( ClassicCommandTransport.java: 59)
	at hudson.remoting. SynchronousCommandTransport$ ReaderThread.run( SynchronousCommandTransport. java:48)



Richard J

unread,
Aug 31, 2012, 12:30:35 PM8/31/12
to jenkins...@googlegroups.com
I see the same problem when running a "Windows batch file" type build step on Windows slaves.
Maybe 1 out of 100 runs.
 
There was talk about this a year ago, but it was claimed to be fixed.  (I see it in ver 1.466.1)

Tom Huybrechts

unread,
Sep 3, 2012, 4:59:47 AM9/3/12
to jenkins...@googlegroups.com
The build is not failing because Jenkins can't delete the file. It's failing (and the file can't be deleted) because the slave's network connection to Jenkins was broken.

stefano perticoni

unread,
Sep 3, 2012, 4:59:58 AM9/3/12
to jenkins...@googlegroups.com
Hi,

We did some investigation and the error message ie the strange stack trace with the space under Jenkins 1472 is related to the integration of the profiling tool gperftool in our codebase:


By disabling it the dashboard runs succesfully.

We just upgraded Jenkins to the latest build 1479 to see if we can get around this problem.


Regards,
Stefano

stefano perticoni

unread,
Sep 4, 2012, 9:18:28 AM9/4/12
to jenkins...@googlegroups.com
Ok we traced the issue: it was not a Jenkins one but a gperftool multi thread profiling issue.


We're switching to Valgrind (gperftool seems quite dead anyway...)

Thank you very much for your help,

Regards,
Stefano
Reply all
Reply to author
Forward
0 new messages