ERROR: No space left on device /dev/stdout
maximb
It is my first post to this group, so please be patient if I do not
follow an accepted format. Also, if this group is not the right place
to ask such a questions, please say so and kindly suggest an
alternative.
I have a legacy software application running on Red Hat 6.1. Yes, it
is pretty old. Known, that the filesystem in use is the ext2fs.
Sometimes, after a few days of running without a problem, the system
throws the following error messages as a response to an attempt to
save or display the curently running configuration:
awk: write failure (No space left on device)
awk: close failed on file /dev/stdout (No space left on device)
echo: error writing to the standard output: No space left on device
Somewhere I had read an opinion, that such kind of errors can be
related to size or avalability of free memory in /tmp directory. Is
that correct ?
Thanks in advance.
maximb
André Gillibert
Indeed, if the program writes to /tmp and there's not enough space on the tmp file system, then, this is expected.
$ df
Gives information about free spaces of mounted file systems.
If a tmpfs is mounted on /tmp and /dev/stdout is redirected to a file on /tmp, then, this may be due to an out-of-RAM condition. If /tmp is on a physical drive, make sure it has enough free space.
In both cases, make sure that no program "leak" files in /tmp.
Is size of /tmp (as shown by du(1)) slowly growing until it's full?
--
André Gillibert
Bill Marcum
> Sometimes, after a few days of running without a problem, the system
> throws the following error messages as a response to an attempt to
> save or display the curently running configuration:
>
> awk: write failure (No space left on device)
> awk: close failed on file /dev/stdout (No space left on device)
> echo: error writing to the standard output: No space left on device
>
> Somewhere I had read an opinion, that such kind of errors can be
> related to size or avalability of free memory in /tmp directory. Is
> that correct ?
>
Have you tried "df" to make sure that you haven't actually run out of space?
You might also do "ls -l /dev/stdout" and make sure the /proc filesystem
is mounted. On my system /dev/stdout is a symbolic link to /proc/self/fd/1.
I'm not sure if it was the same in Redhat 6.1.
maximb
Hi.
Thanks for your suggestions.
The legacy application I had mentioned in previous post replaces the
standard Linux shell with its own, so its seems like I have to
'enhance' the application's shel with some Linux CLI commands, which
would allow me to debug the problem - file leaks in /tmp, steady
growth of consumed space in the filesystem, etc.
But, after your explanation I realized, that in worst case I have to
track _every_ redirection of /dev/stdout.
Thanks for your time.
maximb
Josef Moellers
Actulally ... no, you don't have to trace every symptom, just cure the
problem!
If /dev/stdout is a plain file, then you'll soon enough run out of space
(BTDT (*) ;-). It *has* to be something else, a symbolic link into /proc
or a special file.
(*) This error may be a consequence of some script cleaning up by
removing output files and one of these output files was /dev/stdout.
Josef
--
These are my personal views and not those of Fujitsu Technology Solutions!
Josef M�llers (Pinguinpfleger bei FTS)
If failure had no penalty success would not be a prize (T. Pratchett)
Company Details: http://de.ts.fujitsu.com/imprint.html
maximb
wrote:
> If failure had no penalty success would not be a prize (T. Pratchett)
>
> - Show quoted text -
Hi, Josef.
So far I did not found any syslink() call in the application's code.
Also, there are no calls to shell command "ln -a" which are related to
stdout file descriptor.
Then, my conclusion is that the stdout is indeed a plain file.
What I do see in the code is the huge amounts of CLI commands output
redirections to the standard output.
Can you please give an example of a script designed to clean up output
files ?
Josef Moellers
>> If failure had no penalty success would not be a prize (T. Pratchett)
>> Company Details:http://de.ts.fujitsu.com/imprint.html- Hide quoted text -
>>
>> - Show quoted text -
>
> Hi, Josef.
>
> So far I did not found any syslink() call in the application's code.
> Also, there are no calls to shell command "ln -a" which are related to
> stdout file descriptor.
> Then, my conclusion is that the stdout is indeed a plain file.
OK, so the first step would be to remove that file and create a symlink
as described by Bill:
rm /dev/stdout
ln -s /proc/self/fd/1 /dev/stdout
Maybe it's a good idea to also check /dev/stdin (link to .../0) and
/dev/stderr (link to .../2).
> What I do see in the code is the huge amounts of CLI commands output
> redirections to the standard output.
>
> Can you please give an example of a script designed to clean up output
> files ?
That totally depends upon your code.
Some programs/scripts generate temporary intermediate output and, if the
program finishes successfully, these intermediate files need to be removed.
My assumption (as I've made the very same mistake) was, that some script
or some program has removed /dev/stdout.
E.g. a program is designed as to *not* create a destination file if no
output is generated. Now it has been called with "-" as the destination
file name, which usually means that the output is to be sent to stdout
rather than some file. So, the program, rather than just use "stdout" as
the FILE pointer or 1 as the fd, internally specifies "/dev/stdout" as
the output filename and, e.g. if no output is generated at all, decides
to close the output file and remove the destination file (which has been
created when the file was opened). Voila ... /dev/stdout is gone:
main(int argc, char *argv[])
{
char *dstname;
FILE *dst;
int nlines; /* number of output lines created */
dstname = argv[1];
if (strcmp(argv[1], "-") == 0)
dstname = "/dev/stdout";
dst = fopen(dstname, "w");
if (dst == NULL)
{
perror(dstname);
exit(1);
}
/* so far, so good ... */
nlines = process_data(dst);
fclose(dst); /* No problem here */
if (nlines == 0)
unlink(dstname); /* No need to keep file, it's empty! */
}
Obviously, this can also be done in a shell script:
if [ "X$1" -eq "X-" ]; then dstname=/dev/stdout; else dstname=$1; fi
process_input > $dstname
if [ ! -s $dstname ]; then rm -f $dstname; fi
I'd say that the script-version is more likely.
Note that this would only work if this program (once completed ;-) is
run by root! Non-privileged users cannot remove entries from the /dev
directory.
HTH,
Josef
--
These are my personal views and not those of Fujitsu Technology Solutions!
Josef M�llers (Pinguinpfleger bei FTS)
maximb
> >> If failure had no penalty success would not be a prize (T. Pratchett)
> If failure had no penalty success would not be a prize (T. Pratchett)
> Company Details:http://de.ts.fujitsu.com/imprint.html- Hide quoted text -
>
> - Show quoted text -
Hi, Josef.
By establishing a symbolic link betwen to target /proc/self/fd/1 I
will, as far as I understand, affect all the application code which
explicitly refer the name /dev/stdout or plain stdout. That's
correct ?
Whether a script code of form:
prog arg 1> /dev/null
will be affected as a result of created stdout alias?
Trevor Hemsley
<maxim.b...@gmail.com> wrote:
> What I do see in the code is the huge amounts of CLI commands output
> redirections to the standard output.
Any chance that this might have generated ~4GB output before it goes wrong and
you do not have large file support on such an old release?
--
Trevor Hemsley, Brighton, UK
Trevor dot Hemsley at ntlworld dot com
Josef Moellers
>> If failure had no penalty success would not be a prize (T. Pratchett)
>> Company Details:http://de.ts.fujitsu.com/imprint.html- Hide quoted text -
>>
>> - Show quoted text -
>
> Hi, Josef.
>
> By establishing a symbolic link betwen to target /proc/self/fd/1 I
> will, as far as I understand, affect all the application code which
> explicitly refer the name /dev/stdout or plain stdout. That's
> correct ?
No, just those who explicitly refer to "/dev/stdout" (nitpicking: and
obviously to all those whe explicitly refer to "stdout" when the current
working directory is "/dev") as a filename.
> Whether a script code of form:
>
> prog arg 1> /dev/null
>
> will be affected as a result of created stdout alias?
No. The shell will already have the fd open for fd 1. Redirecting fd 1
will close fd 1 (and *not* remove whatever fd 1 was open to) and reopen
another file, /dev/null" in this case.
You'll explicitly use "/dev/stdout" when you must have a filename which
refers to ... well ... stdout, e.g. if your code is such that you want
to explicitly [f]open() a file and that file happens to be whatever fd 1
refers to.
Josef
--
These are my personal views and not those of Fujitsu Technology Solutions!
Josef M�llers (Pinguinpfleger bei FTS)
Josef Moellers
> On Mon, 2 Nov 2009 09:59:43 UTC in comp.os.linux.development.system, maximb
> <maxim.b...@gmail.com> wrote:
>
>> What I do see in the code is the huge amounts of CLI commands output
>> redirections to the standard output.
>
> Any chance that this might have generated ~4GB output before it goes wrong and
> you do not have large file support on such an old release?
>
That would not generate ENOSPC but rather EFBIG:
man 2 write:
ERRORS
EFBIG An attempt was made to write a file that exceeds the
implementation-defined maximum file size or the process's file size
limit, or to write at a position past the maximum allowed offset.
:
ENOSPC The device containing the file referred to by fd has no room for
the data.
Josef
--
These are my personal views and not those of Fujitsu Technology Solutions!
Josef Mï¿œllers (Pinguinpfleger bei FTS)
Trevor Hemsley
Moellers <josef.m...@ts.fujitsu.com> wrote:
> That would not generate ENOSPC but rather EFBIG:
The OP is running RH 6.1. I have no idea if that error code even existed back
then - if I recall correctly the definition of modern hardware in those days was
a Pentium 90 and a massive hard disk was 2GB :)
André Gillibert
$ ls -l /proc/$pid/fd/1
Should give the file name.
Or:
$ lsof
Josef Moellers
> maximb <maxim.b...@gmail.com> wrote:
>> So far I did not found any syslink() call in the application's code.
>> Also, there are no calls to shell command "ln -a" which are related to
>> stdout file descriptor.
>> Then, my conclusion is that the stdout is indeed a plain file.
>> What I do see in the code is the huge amounts of CLI commands output
>> redirections to the standard output.
>>
>
>
> $ ls -l /proc/$pid/fd/1
>
> Should give the file name.
>
> Or:
> $ lsof
Beware: the file name (i.e. the directory entry) may not exist any more.
There is nothing that prevents
fd = open(path, O_RDONLY);
unlink(path);
read(fd, buffer, nbytes);
close(fd);
AFAIR tmpfile() does just that. It has the advantage that if the program
crashes, there are no files left as collateral damage.