cron problem and /tmp out of space

[ra...@joesbar.cc.vt.edu]

Hello folks,
I am having a problem with an RS/6000 43P, running AIX 4.1.5.
I was trying to set up a cron job for a general userid (no privs),
and after one test I could not read mail due to /tmp being full.
I looked in /tmp and found a big file owned by the user with a
name indicating a cron process probably created it. So, I rm'ed
the file, but the filesystem stayed 100% used.
I went to SMIT and added 4M of space. I checked with df after
adding the space and it was there OK. I then went back to the
non-root user and tried to do a crontab -e only to get a /tmp full
condition again. Checked with df and the extra 4 meg was still
there, but showed as being used (it hadn't before).
I could find no other files taking up space in /tmp. Does
anyone have any ideas what the problem might be? I'm thinking
an fsck might be in order. Maybe just a reboot, but I'd really
like to understand this first, and it's a production machine that
I don't have unlimited access to so I have to go cautiously.

Suggestions, comments, ideas?

--
*****************************************************************************
* Bill Ranck +1-540-231-3951 ra...@vt.edu *
* Virginia Polytechnic Institute & State University, Computing Center *
*****************************************************************************

[Christian Bauernfeind]

In article <6rc6oj$f83$2...@solaris.cc.vt.edu>,
ra...@joesbar.cc.vt.edu () writes:

[rm /tmp/crout* doesn't free up space]

When you rm a file that another process still has open, it will
continue to take up space until that process dies or closed it.
Probably your runaway cron job is still active and generating
output. Find it and kill it to reclaim the space.

HTH

Christian
--
Christian Bauernfeind
Not speaking for Siemens
Not even working for IBM
e-mail: v2ba...@fishkill.ibm.com

[John Crothers]

# fuser -u /dev/hd3

This will return a list of process IDs for those processes
with files open under /tmp.

The file that you deleted: was the time of last access recent?
If you delete a file which a process has open for writing,
the disk space used by the file will not be returned to the filesystem's free
list.
In this situation df shows no decrease in the level of filesystem utilisation.

# du -sk /tmp

This does a straight count of the number of blocks used
by the files in /tmp whereas df retrieves it's information
using the filesystem block allocation tables.

If du indicates that the amount of disk space used in /tmp is
significantly less than the actual size of the filesystem, yet df
indicates 100% utilisation, then it may be that a deleted file is
still being written to by a process. Killing that process, assuming
that you can identify it, should free up the space.

Re-booting the box is one way to kill the process but that's obviously
a bit drastic. Hopefully fuser will help to you kill just the process concerned.

For future reference I can recommend the lsof utility which lists
all files opened by the processes running on a UNIX system:

ftp://coast.cs.purdue.edu/pub/tools/unix/lsof

John Crothers
State St Bank & Trust