Master node is disabled because of "Disk space is too low..."

[Akiko Takano]

Hi, all.
Now I saw the problem on Jenkins master node.
Though disk space of master node is enough but Jenkins says “Disk
space is too low. Only ...left 0.xx GB...”.

So Jenkins tells the status of master node is "offline", that all the
jobs on master node are not executed.

After rebooting host os, the situation is the same. (Jenkins still
says, "Disk space is too low....")
I'm quite confuse this.

Before writing this email, I've asked the same on Jenkins IRC.
I'm deeply sorry for the multi-post.

Any help will be appreciated.

[Khai Do]

I had the same problem.  Jenkins was actually looking at the temp space.  Check that you have enough temp space on your host.

[David Karlsen]

The limits may also be set individually for each node.

2011/7/1 Khai Do <do....@gmail.com>

--
--
David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen

[Akiko Takano]

Thanks for messages, Khai and David.

To tell the truth, I met this problem, the master node stopped due to
low disk space, three times.
At first and second, the problem was solved just after rebooting os.

Then, the latest case was quite different.

I found some opened files which were deleted by jenkins, with using
lsof cfommand. After rebooted os, I could recover enough disk space,
the value is about 10GB.
But jenkins still says “Disk space is too low. Only ...left
0.7xxGB...” and master node is recognized as offline.

I think the temp space of "master" node can't be configured on
Jenkins's configuration screen but assigned following the property
such as java.io.tmpdir. So the tmpdir for my jenkins is /tmp and this
is the same partition of "/" .

I tried to reboot some times, but the problem was not fixed.
Since my os is running as VM, finally I gave up to fix this and
restored and recovered from my VM's backup image.

I wonder this depends on the version ofJVM, jenkins, or OS distribution.

2011/7/1 David Karlsen <davidk...@gmail.com>:

[Khai Do]

The disk space configuration can be set if the master is also a node.  if the master is a node then jenkins will detect disk space requirement.   the setting can be found at http://<jenkins_host>/jenkins/computer/configure

Do you have the master turned off as being a node.  if you have then it won't be listed on jenkins as a slave (or node). 

-khai

[Akiko Takano]

Thanks again for your pointing, Khai.
I knew the disk space requirement and the limit is 1GB.

The problem is why my "master" node is recognized as offline after
recovering enough disk space, rebooting os, and restarting jenkins's
process...

Of course, I think such a strange thing is not likely to happen...

I'll report again if the same problem happens.

2011/7/4 Khai Do <do....@gmail.com>:

[Akiko Takano]

Hello.

I am sorry for bothering you many times.

Now I think my problem is related to this issue.

https://issues.jenkins-ci.org/browse/JENKINS-8881

My all jenkins server who acts as master is CentOS 5.5, which is the
same to this issue.

I'm going to comment my situation to above issue.

2011/7/4 Akiko Takano <akiko...@gmail.com>:

[Khai Do]

sorry, i cannot help you further.  I am running my jenkins master on CentOS as well and it seems to be working just fine.  I have Jenkins ver 1.409 installed so it's probably completely different than what are running.

[Akiko Takano]

No problem and thank you so much for taking the time to answer me.
I'm keeping to review my os environment, check the log and filesystem.

2011/7/6 Khai Do <do....@gmail.com>: