What is the NFS max thread limit

[Eric Browning]

I'm currently still debugging slow NFS performance on mac clients. Running
9 stable with a mix of 200 Mac OS X 10.8, 10.7 and 10.6 clients.

Periodically I get this error on the mac side:
automountd[1766]: set_and_fake_mapent_mntlevel: subdir=/Contents error:
Contents not found in map=-fstab

I'm wondering with 200 clients with five nfsiod threads (~1000 nfsiod
threads total) am I oversubscribing the nfs server? I can't seem to
configure any more than 256 threads (nfs_server_flags="-t -n 256"). When I
configure above 256 I get an error about it resetting to 4:

Starting nfsd.
nfsd: nfsd count 512; reset to 4

Tomorrow when all the mac clients have restarted I have changed their
nfsiod threads down to 1 each to see if this is indeed the issue.

Thanks in advance,
--
Eric Browning
Systems Administrator
801-984-7623

Skaggs Catholic Center
Juan Diego Catholic High School
Saint John the Baptist Middle
Saint John the Baptist Elementary
_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-...@freebsd.org"

[Rick Macklem]

Eric Browning wrote:
> I'm currently still debugging slow NFS performance on mac clients.
> Running
> 9 stable with a mix of 200 Mac OS X 10.8, 10.7 and 10.6 clients.
>
> Periodically I get this error on the mac side:
> automountd[1766]: set_and_fake_mapent_mntlevel: subdir=/Contents
> error:
> Contents not found in map=-fstab
>
> I'm wondering with 200 clients with five nfsiod threads (~1000 nfsiod
> threads total) am I oversubscribing the nfs server? I can't seem to
> configure any more than 256 threads (nfs_server_flags="-t -n 256").
> When I
> configure above 256 I get an error about it resetting to 4:
>
> Starting nfsd.
> nfsd: nfsd count 512; reset to 4
>
> Tomorrow when all the mac clients have restarted I have changed their
> nfsiod threads down to 1 each to see if this is indeed the issue.
>

Well, normally not all clients would be actively reading/writing files
concurrently, so I wouldn't expect your load to need more than 256 nfsd
threads. I also recall that you had a very large Access RPC load and
Access RPCs are not done by nfsiod threads. (nfsiod threads only do
readaheads and write behinds for buffer cache blocks of files.)

However, if you do want to try more than 256 nfsd threads, you will
need to edit nfsd.c and increase MAXNFSDCNT and then rebuild the nfsd
daemon from these modified sources.

I have no idea what causes the Mac OS X error, but you could try
posting on one of the Apple mailing lists like darwin...@lists.apple.com.

rick
ps: The nfsd.c in head/current/10.0 also has a "--maxthreads" option
that I don't think is limited to MAXNFSDCNT from a glance at the code.

[Eric Browning]

Here is the nfsstat -c output for one mac client for one day from one of my
busy workers. Are these numbers unreasonable? There are only 14 retries
which doesn't seem high so I think the network is ok. Any thoughts on
client side optimizations to reduce RPC load?

Client Info:

RPC Counts:

Getattr Setattr Lookup Readlink Read
Write

72725986 7586 56972 63 17082
22317

Create Remove Rename Link Symlink
Mkdir

4583 2041 3108 0 5
45

Rmdir Readdir RdirPlus Access Mknod
Fsstat

140 0 1106 49382068 0
26197

Fsinfo PathConf Commit

1 1 3650

RPC Info:

TimedOut Invalid X Replies Retries Requests

0 0 0 14 122252951

Cache Info:

Attr Hits Misses Lkup Hits Misses BioR Hits
Misses

761115632 3278207 110222766 53251 132864002
16282

BioW Hits Misses BioRLHits Misses BioD Hits
Misses

185211 22316 16004 63 12257339
5883

DirE Hits Misses

2990343 23