Too many open files

[Prabhpal S. Mavi]

Greetings Friends,

have anyone has come across this warning / error? This occurs when i ssh
to my FreeBSD 9.0 System. any help would be greatly appreciated.

Warning:
/usr/share/games/fortune/freebsd-tips.dat: Too many open files in system
[mavi@titan ~]$ su
su: pam_start: system error

Thanks / Regards
Prabhpal

_______________________________________________
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stabl...@freebsd.org"

[C. P. Ghost]

On Sun, Mar 25, 2012 at 6:46 PM, Prabhpal S. Mavi
<prab...@digital-infotech.net> wrote:
> Greetings Friends,
>
> have anyone has come across this warning / error? This occurs when i ssh
> to my FreeBSD 9.0 System. any help would be greatly appreciated.
>
> Warning:
> /usr/share/games/fortune/freebsd-tips.dat: Too many open files in system
> [mavi@titan ~]$ su
> su: pam_start: system error
>
> Thanks / Regards
> Prabhpal

What does this command say on your system?

% sysctl kern.maxfiles kern.maxfilesperproc kern.openfiles

You may have exceeded the maximum number of open files
in the system. Maybe some ill-conceived program that doesn't
close non-needed connections, files, etc is at fault? It's easy
to open more and more files, and to gradually fill the open
files descriptor table in the kernel this way.

-cpghost.

--
Cordula's Web. http://www.cordula.ws/

[Erich Dollansky]

Hi,

do you run KDE?

Erich

[Shane Ambler]

On 26/03/2012 02:19, C. P. Ghost wrote:
> On Sun, Mar 25, 2012 at 6:46 PM, Prabhpal S. Mavi
> <prab...@digital-infotech.net> wrote:
>> Greetings Friends,
>>
>> have anyone has come across this warning / error? This occurs when i ssh
>> to my FreeBSD 9.0 System. any help would be greatly appreciated.
>>
>> Warning:
>> /usr/share/games/fortune/freebsd-tips.dat: Too many open files in system
>> [mavi@titan ~]$ su
>> su: pam_start: system error
>>
>> Thanks / Regards
>> Prabhpal
>
> What does this command say on your system?
>
> % sysctl kern.maxfiles kern.maxfilesperproc kern.openfiles
>
> You may have exceeded the maximum number of open files
> in the system. Maybe some ill-conceived program that doesn't
> close non-needed connections, files, etc is at fault? It's easy
> to open more and more files, and to gradually fill the open
> files descriptor table in the kernel this way.
>
> -cpghost.
>

From knowing that you have too many files open you can increase the
maxfile numbers - but if you want to know what uses them try this -

lsof -n | awk '{print $2 "\t" $1}' | sort | uniq -c | sort

lsof outputs open file info, awk then gives us the PID and proc name
which gets sorted and uniq gives a count of each which we sort to have
the largest file count at the bottom of the list. What you end up with
is a list of two numbers and a name - count of files open followed by
the PID and proc name that has them open.

The catch is that it also includes network connections (I know how to
list only network but not sure how to exclude them)

ps ax | grep PID

will show you the full program name if it has been shortened.

lsof -p PID

will show all the open files for PID

Not sure if this is the best way but it works for me.

[Prabhpal S. Mavi]

Hello Shane,

thanks for your valuable response, this is brilliant. this is what i was
exactly looking for. very good command indeed.

Grateful for your kind assistance

Thanks / Regards

[Kurt Jaeger]

Hi!

> > From knowing that you have too many files open you can increase the
> > maxfile numbers - but if you want to know what uses them try this -
> >
> > lsof -n | awk '{print $2 "\t" $1}' | sort | uniq -c | sort

On my system, it shows interesting numbers for firefox-instances:

3895 4150 firefox-b
4160 72958 firefox-b
4240 3594 firefox-b
4320 4232 firefox-b
4431 89391 firefox-b

This seems to come from threads, where lsof shows the same
descriptor as open for multiple times, even if it comes from the
same process memory.

For example for pid 4232:

lsof -n | grep icon-theme.cache | grep 4232 | wc -l

firefox-b 4232 pi txt VREG 0,102 10784 924293 /usr/local/share/icons/hicolor/icon-theme.cache

This happens on 8.1-REL-p5 amd64. So the method is not fail-safe
for the real number of open files.

--
p...@opsec.eu +49 171 3101372 8 years to go !

[Matthew Seaman]

On 26/03/2012 09:56, Kurt Jaeger wrote:
>>> From knowing that you have too many files open you can increase the
>>> > > maxfile numbers - but if you want to know what uses them try this -
>>> > >
>>> > > lsof -n | awk '{print $2 "\t" $1}' | sort | uniq -c | sort
> On my system, it shows interesting numbers for firefox-instances:
>
> 3895 4150 firefox-b
> 4160 72958 firefox-b
> 4240 3594 firefox-b
> 4320 4232 firefox-b
> 4431 89391 firefox-b
>
> This seems to come from threads, where lsof shows the same
> descriptor as open for multiple times, even if it comes from the
> same process memory.
>
> For example for pid 4232:
>
> lsof -n | grep icon-theme.cache | grep 4232 | wc -l
>
> firefox-b 4232 pi txt VREG 0,102 10784 924293 /usr/local/share/icons/hicolor/icon-theme.cache
>
> This happens on 8.1-REL-p5 amd64. So the method is not fail-safe
> for the real number of open files.
>

Does 'procstat -fa' give better results for you?

It seems to be one of those little hidden secrets that FreeBSD comes
with a bunch of native applications that provide pretty much equivalent
functionality to lsof(1). See: fstat(1), procstat(1), sockstat(1).

Which is odd, given that since these sort of applications have to read
and interpret kernel memory -- an action for which there isn't a nice
well defined ABI -- the application has to be kept rigorously in synch
with the kernel it is used against. Something that is intrinsically
easier to do when kernel and application are compiled at the same time
and from the same source tree.

Cheers,

Matthew

--
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey

[Kurt Jaeger]

Hi!

> >>> > > lsof -n | awk '{print $2 "\t" $1}' | sort | uniq -c | sort

> > On my system, it shows interesting numbers for firefox-instances:

[...]

> Does 'procstat -fa' give better results for you?

Yes, much better.

[Jilles Tjoelker]

On Mon, Mar 26, 2012 at 10:15:59AM +0100, Matthew Seaman wrote:
> Does 'procstat -fa' give better results for you?

> It seems to be one of those little hidden secrets that FreeBSD comes
> with a bunch of native applications that provide pretty much equivalent
> functionality to lsof(1). See: fstat(1), procstat(1), sockstat(1).

> Which is odd, given that since these sort of applications have to read
> and interpret kernel memory -- an action for which there isn't a nice
> well defined ABI -- the application has to be kept rigorously in synch
> with the kernel it is used against. Something that is intrinsically
> easier to do when kernel and application are compiled at the same time
> and from the same source tree.

procstat (in all versions that have it) and fstat (in FreeBSD 9.0 and
newer) use a well-defined sysctl-based API to access the information.
This API was extended in FreeBSD 9.0 and a library libprocstat provides
a convenient interface.

Reading from kernel memory not only couples the application tightly to
the kernel implementation, but also can also be considered a security
issue because there is a lot of sensitive information in kernel memory;
it cannot be permitted in a jail.

--
Jilles Tjoelker

[Jim Bryant]

1:23:14pm argus(50): lsof -n | awk '{print $2 "\t" $1}' | sort | uniq
-c | sort
lsof: Command not found.
1:35:12pm argus(51): man lsof | cat
No manual entry for lsof
1:35:38pm argus(52): find /usr/src -iname \*lsof\*
1:36:42pm argus(53): find /usr/src -type f -iname \*lsof\*
1:36:50pm argus(54):

huh??? 9-stable here. is this a -port?

Aha!

1:39:07pm argus(1): cd /usr/ports
1:39:11pm argus(2): make search name=lsof
Port: lsof-4.86A,6
Path: /usr/ports/sysutils/lsof
Info: Lists information about open files (similar to fstat(1))
Maint: l...@lerctr.org
B-deps:
R-deps:
WWW: http://people.freebsd.org/~abe/

Port: p5-Unix-Lsof-0.0.5_1
Path: /usr/ports/sysutils/p5-Unix-Lsof
Info: Unix::Lsof -- a wrapper to the Unix lsof utility
Maint: gj...@gjvc.com
B-deps: p5-IPC-Run3-0.044 perl-5.12.4_3
R-deps: p5-IPC-Run3-0.044 perl-5.12.4_3
WWW: http://search.cpan.org/dist/Unix-Lsof/

Next time, please state that it is a port and not a builtin.

Thanks
jim