[Lustre-discuss] NFS problem after upgrade to 1.8.3
Tina Friedrich
we re-export the file system via NFS for a couple of things. All the
re-exporters are Red Hat 5.5 servers running kernel 2.6.18-194.17.1.el5
(patchless clients).
We upgraded our Lustre system from 1.6.7 to 1.8.3.ddn3.3 last week. That
seems to have introduced a problem. Since this upgrade, some of our NFS
clients can't mount the file system any more. They get a permission
denied error. Others can mount the same file system just fine. The
difference appears to be that the clients that can't mount it try to use
mountd version 1.
Some investigation exposed the following behaviour:
[root@pc030 ~]# mount -o udp -o mountvers=1
cs04r-sc-com99-01:/mnt/play01/tmp/tina/mounttest/ /tmp/mnttest/
[root@pc030 ~]# ls -l /tmp/mnttest/
total 0
?--------- ? ? ? ? ? realdir
I.e. when I use mountvers=1 I get this weird 'broken link' type entry (I
get the 'permission denied' error if I try to mount anything below
'realdir' in this example).
Whereas when using mountvers=3:
[root@pc030 ~]# mount -o udp -o mountvers=3
cs04r-sc-com99-01:/mnt/play01/tmp/tina/mounttest/ /tmp/mnttest/
[root@pc030 ~]# ls -l /tmp/mnttest/
total 4
drwxrwxr-x 3 root root 4096 Nov 12 10:04 realdir
all is as it should be.
This is only when NFS exporting our Lustre file system; when exporting a
local file system instead all versions of mountd work fine.
Anyone seen this before? Any ideas? Unfortunately the systems using the
mountd version 1 are an embedded box so I'm not sure the way they mount
can be changed.
--
Tina Friedrich, Computer Systems Administrator, Diamond Light Source Ltd
Diamond House, Harwell Science and Innovation Campus - 01235 77 8442
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
_______________________________________________
Lustre-discuss mailing list
Lustre-...@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
Bernd Schubert
On Friday, November 12, 2010, Tina Friedrich wrote:
> Hello List,
>
> we re-export the file system via NFS for a couple of things. All the
> re-exporters are Red Hat 5.5 servers running kernel 2.6.18-194.17.1.el5
> (patchless clients).
that is your problem. You MUST use a patched version, or least a kernel with
8kB stack size. RHEL5 has 4kB by default, which is not sufficient and
therefore in early 1.8 versions a patch landed that disallowed NFS exports.
Cheers,
Bernd
--
Bernd Schubert
DataDirect Networks
Tina Friedrich
It's just the version 1 & 2 were it doesn't work. But thanks, I'll try
the patched kernel.
Tina
On 12/11/10 12:46, Bernd Schubert wrote:
> Hello Tina,
>
> On Friday, November 12, 2010, Tina Friedrich wrote:
>> Hello List,
>>
>> we re-export the file system via NFS for a couple of things. All the
>> re-exporters are Red Hat 5.5 servers running kernel 2.6.18-194.17.1.el5
>> (patchless clients).
>
> that is your problem. You MUST use a patched version, or least a kernel with
> 8kB stack size. RHEL5 has 4kB by default, which is not sufficient and
> therefore in early 1.8 versions a patch landed that disallowed NFS exports.
>
>
> Cheers,
> Bernd
>
>
>
--
Tina Friedrich, Computer Systems Administrator, Diamond Light Source Ltd
Diamond House, Harwell Science and Innovation Campus - 01235 77 8442
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
Tina Friedrich
nope, running with / exporting from a server with the patched kernel
running does not change this behaviour at all. mountvers=3 works, 1 and
2 don't.
Tina
On 12/11/10 13:28, Tina Friedrich wrote:
> It does seem to allow NFS exports using RPC version 3 just fine though.
> It's just the version 1& 2 were it doesn't work. But thanks, I'll try
Bernd Schubert
On 11/12/2010 03:44 PM, Tina Friedrich wrote:
> Hello again,
>
> nope, running with / exporting from a server with the patched kernel
> running does not change this behaviour at all. mountvers=3 works, 1 and
> 2 don't.
I can reproduce it, so NFSv2 support got broken. Which issue has higher
priority, tar or NFSv2?
Cheers,
Bernd
Tina Friedrich
again for the next couple of months anyway and it appears to be no
problem to make the stupid embedded things use NFSv3.
So tar, I would say.
Tina
Alexey Lyashkov
if i correctly remember, lustre inode identifier can't correctly encoded in NFS fid v1, due NFS limits.
NFS have too short structure to store FS id, generated from lustre client superblock.
that is reason to have incorrect conversion from NFS fid to lustre inode identifier and nfsd can't find corresponded lustre inode to get attributes.
you can try to specify fsid option in mount command line, but not sure that is helps.
On Nov 12, 2010, at 14:26, Tina Friedrich wrote:
> Hello List,
>
> we re-export the file system via NFS for a couple of things. All the
> re-exporters are Red Hat 5.5 servers running kernel 2.6.18-194.17.1.el5
> (patchless clients).
>
> We upgraded our Lustre system from 1.6.7 to 1.8.3.ddn3.3 last week. That
> seems to have introduced a problem. Since this upgrade, some of our NFS
> clients can't mount the file system any more. They get a permission
> denied error. Others can mount the same file system just fine. The
> difference appears to be that the clients that can't mount it try to use
> mountd version 1.
> '
--------------------------------------
Alexey Lyashkov
alexey....@clusterstor.com
Alexey Lyashkov
that is problem not related to stack size.
mountd should encode {sb->sb_dev, inode->i_no} in own structure, but for mount v1 sb_dev too short to store full sb_dev (it is 16bit, but nfs fid v1 has 8 for it) bit but lustre client can generate that.
in that case nfsd got invalid NFS fid to client and broken to find correct inode to get attributes and return -ENOENT. in that case ls -l will show "?" instead of real inode attributes.