[lug:16195] "lsof -u user | wc -l" shows more entries then "ulimit -n"

3,189 views
Skip to first unread message

wenlong

unread,
Apr 25, 2010, 8:02:05 PM4/25/10
to Linux Users Group
Hi All,

Today, I'm playing with the "ulimit " command. My login has a ulimit
of 1024 for open file descriptor:

$ ulimit -n
1204

But when using lsof to list down all files (files, socket, etc) opened
by my login, I got:

$ lsof -u user | wc -l
3977

Can someone tell me why can my login open more files than granted by
ulimit? I'm running:

Debian stable (lenny)
Gnome desktop environment + compiz

Thanks.

-- wenlong (boon leong)

--
You received this message because you are subscribed to the Linux Users Group.
To post a message, send email to linuxus...@googlegroups.com
To unsubscribe, send email to linuxusersgro...@googlegroups.com
For more options, visit our group at http://groups.google.com/group/linuxusersgroup

Daniel Eggleston

unread,
Apr 26, 2010, 10:13:03 AM4/26/10
to linuxus...@googlegroups.com


The ulimit -n is a per-process limitation (i.e. any process cannot have more
than 1024 files open). Here is a brief example, showing a process that can
open 1024 files, but not 1025 (bearing in mind that there are already 3 files
open: stdin, stdout, and stderr):

eggled@pokeserver ~ $ ulimit -n
1024
eggled@pokeserver ~ $ lsof | grep eggled | wc -l
4863
eggled@pokeserver ~ $ cat openLots.pl
#!/usr/bin/perl

## There are already 3 file descriptors
## Plus an offset to make the index equal to the file count.
my @fileList = (-1,0,1,2);
for (;;) {
push(@fileList,">/tmp/lotsaFilesTemp.$#fileList");
open($fileList[$#fileList],$fileList[$#fileList]) or (print "\n\n" and die "\n\nCould not open $#fileList: $!");
print "$#fileList ";
}
eggled@pokeserver ~ $ ./openLots.pl
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
---- snip ----
1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021
1022 1023 1024

Could not open 1025: Too many open files at ./openLots.pl line 8.
eggled@pokeserver ~ $


The 1025th file failed to open, with the error message "Too many files open".

wenlong

unread,
Apr 27, 2010, 1:41:31 AM4/27/10
to Linux Users Group
Hi Daniel,

Thanks for your explanation.

-- wenlong (boon leong)



On Apr 26, 10:13 pm, Daniel Eggleston <egg...@gmail.com> wrote:
> On Sun, Apr 25, 2010 at 05:02:05PM -0700, wenlong wrote:
> > Hi All,
>
> > Today, I'm playing with the "ulimit " command.  My login has a ulimit
> > of 1024 for open file descriptor:
>
> > $ ulimit -n
> > 1204
>
> > But when using lsof to list down all files (files, socket, etc) opened
> > by my login, I got:
>
> > $ lsof -u user | wc -l
> > 3977
>
> > Can someone tell me why can my login open more files than granted by
> > ulimit?  I'm running:
>
> > Debian stable (lenny)
> > Gnome desktop environment + compiz
>
> > Thanks.
>
> > -- wenlong (boon leong)
>
> > --
> > You received this message because you are subscribed to the Linux Users Group.
> > To post a message, send email to linuxus...@googlegroups.com
> > To unsubscribe, send email to linuxusersgro...@googlegroups.com
> > For more options, visit our group athttp://groups.google.com/group/linuxusersgroup
>  application_pgp-signature_part
> < 1KViewDownload

Scott Hamer

unread,
Apr 26, 2010, 7:15:58 PM4/26/10
to linuxus...@googlegroups.com
Ubuntu 9.1:

shamer@shamer-Vbox:~$ ulimit
unlimited
shamer@shamer-Vbox:~$ ulimit -n
1024
shamer@shamer-Vbox:~$ lsof -u shamer | wc
6105 54930 605988
shamer@shamer-Vbox:~$ ./openLots.pl
blah blah ~snip~
Could not open 1023: Too many open files at ./openLots.pl line 8.

Gentoo:

shamer@hp ~ $ ulimit
unlimited
shamer@hp ~ $ ulimit -n
1024
shamer@hp ~ $ lsof -u shamer | wc
2320 20899 228824
shamer@hp ~ $ ./openLots.pl
Could not open 1025: Too many open files at ./openLots.pl line 8.




Daniel Eggleston

unread,
Apr 27, 2010, 10:04:00 AM4/27/10
to linuxus...@googlegroups.com
Interesting point, Ubuntu must have open a couple of file descriptors
within your process. Maybe for dbus or something?

Was your Ubuntu test run on a VT or in a terminal emulator? (Or
something else, like an SSH session)?

My initial test was on Gentoo, over SSH. Ubuntu 10.04 over SSH has the
same behavior. I also tried an Xterm running in fluxbox on Ubuntu (it's
not feasible to run Gnome remotely). All showed the 1025.

I'm curious, though - could you try it again via ssh or even just
running 'sudo su - shamer', which would give you a full login instead of
a Gnome subshell?

On Mon, Apr 26, 2010 at 06:15:58PM -0500, Scott Hamer wrote:
> Ubuntu 9.1:
> shamer@shamer-Vbox:~$ ulimit
> unlimited
> shamer@shamer-Vbox:~$ ulimit -n
> 1024
> shamer@shamer-Vbox:~$ lsof -u shamer | wc
> 6105 54930 605988
> shamer@shamer-Vbox:~$ ./openLots.pl
> blah blah ~snip~
> Could not open 1023: Too many open files at ./openLots.pl line 8.
>
> Gentoo:
> shamer@hp ~ $ ulimit
> unlimited
> shamer@hp ~ $ ulimit -n
> 1024
> shamer@hp ~ $ lsof -u shamer | wc
> 2320 20899 228824
> shamer@hp ~ $ ./openLots.pl
> Could not open 1025: Too many open files at ./openLots.pl line 8.

> On Mon, Apr 26, 2010 at 09:13, Daniel Eggleston <[1]egg...@gmail.com>


> wrote:
>
> On Sun, Apr 25, 2010 at 05:02:05PM -0700, wenlong wrote:
> > Hi All,
> >

> > Today, I'm playing with the "ulimit " command. *My login has a ulimit


> > of 1024 for open file descriptor:
> >
> > $ ulimit -n
> > 1204
> >
> > But when using lsof to list down all files (files, socket, etc) opened
> > by my login, I got:
> >
> > $ lsof -u user | wc -l
> > 3977
> >
> > Can someone tell me why can my login open more files than granted by

> > ulimit? *I'm running:


> >
> > Debian stable (lenny)
> > Gnome desktop environment + compiz
> >
> > Thanks.
> >
> > -- wenlong (boon leong)
> >
> > --
> > You received this message because you are subscribed to the Linux
> Users Group.

> > To post a message, send email to [2]linuxus...@googlegroups.com


> > To unsubscribe, send email to

> [3]linuxusersgro...@googlegroups.com


> > For more options, visit our group at

> [4]http://groups.google.com/group/linuxusersgroup


>
> The ulimit -n is a per-process limitation (i.e. any process cannot have
> more

> than 1024 files open). *Here is a brief example, showing a process that


> can
> open 1024 files, but not 1025 (bearing in mind that there are already 3
> files
> open: stdin, stdout, and stderr):
>
> eggled@pokeserver ~ $ ulimit -n
> 1024
> eggled@pokeserver ~ $ lsof | grep eggled | wc -l
> 4863
> eggled@pokeserver ~ $ cat openLots.pl
> #!/usr/bin/perl
>
> ## There are already 3 file descriptors
> ## Plus an offset to make the index equal to the file count.
> my @fileList = (-1,0,1,2);
> for (;;) {

> * * * *push(@fileList,">/tmp/lotsaFilesTemp.$#fileList");
> * * * *open($fileList[$#fileList],$fileList[$#fileList]) or (print


> "\n\n" and die "\n\nCould not open $#fileList: $!");

> * * * *print "$#fileList ";


> }
> eggled@pokeserver ~ $ ./openLots.pl
> 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
> 30 31
> ---- snip ----
> 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019
> 1020 1021
> 1022 1023 1024
>
> Could not open 1025: Too many open files at ./openLots.pl line 8.
> eggled@pokeserver ~ $
>
> The 1025th file failed to open, with the error message "Too many files
> open".
>
> --
> You received this message because you are subscribed to the Linux Users
> Group.
> To post a message, send email to linuxus...@googlegroups.com
> To unsubscribe, send email to linuxusersgro...@googlegroups.com
> For more options, visit our group at

> [5]http://groups.google.com/group/linuxusersgroup
>
> References
>
> Visible links
> 1. mailto:egg...@gmail.com
> 2. mailto:linuxus...@googlegroups.com
> 3. mailto:linuxusersgro...@googlegroups.com
> 4. http://groups.google.com/group/linuxusersgroup
> 5. http://groups.google.com/group/linuxusersgroup

Scott Hamer

unread,
Apr 27, 2010, 4:42:56 PM4/27/10
to linuxus...@googlegroups.com
Ubuntu 9.1 is running in Virtual box, on Windows 7 host (yeah i know, but it's at work).  When I ran the perl script as sudo i did get to the 1025 limit, unlike running as normal user.   I'm relatively new to Ubuntu, but have been running Gentoo for 4 years and have not seen this behavior.  I'm only using Ubuntu as it's quick and easy to get into a Virtual Machine with 0 compile time.   It could possibly be an issue with 9.1, or the fact I'm running Konsole instead of X-term, KDE has ruined me ;)

shamer@shamer-Vbox:~$ sudo ./openLots.pl
Could not open 1025: Too many open files at ./openLots.pl line 8.

VS

shamer@shamer-Vbox:~$ ./openLots.pl
For more options, visit our group at http://groups.google.com/group/linuxusersgroup

Daniel Eggleston

unread,
Apr 27, 2010, 8:44:02 PM4/27/10
to linuxus...@googlegroups.com
Well, on Ubuntu 10.04, in gnome-terminal (running under gnome) I see the '1023' cutoff, and here's why:

user@user-desktop:~$ ls -l /dev/fd/
total 0
lrwx------ 1 user user 64 2010-04-27 19:06 0 -> /dev/pts/0
lrwx------ 1 user user 64 2010-04-27 19:06 1 -> /dev/pts/0
lr-x------ 1 user user 64 2010-04-27 19:06 12 -> pipe:[20372603]
l-wx------ 1 user user 64 2010-04-27 19:06 13 -> pipe:[20372603]
lrwx------ 1 user user 64 2010-04-27 19:06 2 -> /dev/pts/0
lr-x------ 1 user user 64 2010-04-27 19:06 3 -> /proc/23620/fd

0 1 and 2 are the aforementioned stdin stdout and stderr.  3 is the 'ls' process fd, and 12/13 are the pipes that limit the number of file descriptors.  strace on gnome-terminal gives the appearance that these are for polling, and are probably how gnome-terminal can say "You have processes running in this terminal. Are you *sure* you want to quit?"
--

          Daniel
Reply all
Reply to author
Forward
0 new messages