Re: 3.1.6 system() fails when at the ulimit of open files

[Aharon Robbins]

Hi. I just tried this with gawk 3.1.7 and 3.1.6 on my Linux system
and I cannot reproduce it.

$ mkdir z
$ ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 32621
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 32621
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
$ yes|head -2000|gawk-3.1.6 '{print "hi" > ("z/"z++)}END{system("echo hiho;date")}'
$ ls z | wc -l
2000

So, the usual questions:

1. What operating system?
2. What compiler?
3. Did you try recompiling from source if you used a pre-supplied binary?

Thanks,

Arnold

> Date: Wed, 14 Oct 2009 08:45:35 -0500
> Subject: 3.1.6 system() fails when at the ulimit of open files
> From: Chris Pearson <ker...@gmail.com>
> To: bug-...@gnu.org
>
> With an open file limit of 1024:
>
> mkdir z
>
> yes|head -200|gawk '{print "hi" > ("z/"z++)}END{system("echo
> hiho;date")}' # the system() call works
> yes|head -2000|gawk '{print "hi" > ("z/"z++)}END{system("echo
> hiho;date")}' # the system() call doesnt work, even though all 2000
> files are created.
>
> So, I guess whatever decides to make room to open a new file getting
> called for system(), which is inconsistant with how open file juggling
> is handled by gawk in other cases, and probably easily fixed. Or at
> the very least should print an error if not.

[Dave B]

On Wednesday 14 October 2009, Aharon Robbins wrote:

> Hi. I just tried this with gawk 3.1.7 and 3.1.6 on my Linux system
> and I cannot reproduce it.

>[snip]

> $ yes|head -2000|gawk-3.1.6 '{print "hi" > ("z/"z++)}END{system("echo
> hiho;date")}'
> $ ls z | wc -l
> 2000

Where's the output of the system() call there?

--
D.

[Aharon Robbins]

OK, ok, ok. This is what I get for trying to debug late at night.

It's a real bug. Patch is below. Thanks to Chris for reporting it.

Arnold

Wed Oct 14 23:25:47 2009 Arnold D. Robbins <arn...@skeeve.com>

* io.c (redirect): When opening a file, set the close-on-exec flag.
Thanks to Chris Pearson <ker...@gmail.com> for the bug report.

Index: io.c
===================================================================
RCS file: /d/mongo/cvsrep/gawk-stable/io.c,v
retrieving revision 1.23
diff -u -r1.23 io.c
--- io.c 9 Jul 2009 19:54:38 -0000 1.23
+++ io.c 14 Oct 2009 21:24:12 -0000
@@ -740,6 +740,7 @@
else
#endif
rp->fp = fdopen(fd, (const char *) mode);
+ os_close_on_exec(fileno(rp->fp), str, "file", "");
rp->mode = (const char *) mode;
/* don't leak file descriptors */
if (rp->fp == NULL)

[Andreas Schwab]

Aharon Robbins <arn...@skeeve.com> writes:

> Index: io.c
> ===================================================================
> RCS file: /d/mongo/cvsrep/gawk-stable/io.c,v
> retrieving revision 1.23
> diff -u -r1.23 io.c
> --- io.c 9 Jul 2009 19:54:38 -0000 1.23
> +++ io.c 14 Oct 2009 21:24:12 -0000
> @@ -740,6 +740,7 @@
> else
> #endif
> rp->fp = fdopen(fd, (const char *) mode);
> + os_close_on_exec(fileno(rp->fp), str, "file", "");
> rp->mode = (const char *) mode;
> /* don't leak file descriptors */
> if (rp->fp == NULL)

If this is true then you crash above.

Andreas.

--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."

[Chris Pearson]

[Chris Pearson]

This also applies to opening or reopening system()-like targets of |