[Lustre-discuss] lfs setstripe

[Kaizaad Bilimorya]

Hello,

I would like to adjust the stripe count for our lustre filesystem. Would
it be better to:

1) Kill all jobs, unmount the lustre filesystem from all clients, and then
adjust the default stripe count for the lustre filesystem on the MDS
using lctl.

or

2) Use "find" and the "lfs setstripe" command to traverse and set the
stripe count for all directories in a currently mounted lustre
filesystem.

Besides the traversal cost of the filesystem, are there other
disadvantages, performance reasons, or other reasons not to use option 2?

thanks
-k
_______________________________________________
Lustre-discuss mailing list
Lustre-...@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

[Brock Palen]

I don't think you need to do this.
If i understand right, you can set the stripe size of the mount, and
everything inside that directory inherits it, unless they them self's
were explicitly set.

Also files that already are created will keep the stripe settings
they were created with. You would need to copy them, and move over
the old one to change to the new stripe settings. Check the lustre
manual they have something about this.

You can use 'getstripe' to see what a file/directory use for their
settings.

Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
bro...@umich.edu
(734)936-1985

[Kaizaad Bilimorya]

On Thu, 17 Apr 2008, Brock Palen wrote:
> I don't think you need to do this.
> If i understand right, you can set the stripe size of the mount, and
> everything inside that directory inherits it, unless they them self's were
> explicitly set.

Hi Brock, thanks for the reply.

I have set the stripe count on the lustre mount using "lfs setstripe" but
the problem is that any sub directories that already existed under this
mount will have the default filesystem stripe count and not the new one I
set, so any new files created under these existent sub directories will
inherit their parent directory stripe count and not the newly set one
from the lustre mount.

eg:

/lustremount -> "lfs setstripe /lustremount 0 -1 2"
/lustremount/existing_dir -> has filesystem default stripe count (1 in
this case)
/lustremount/new_dir -> gets stripe count of parent (2 in this case)
/lustremount/existing_dir/newfile -> has filesystem default stripe count
of 1

So that is why I have to do either option 1 (change default) or 2
(traverse and set explicitly for all existing dirs) that I specified, but I
would like to know if there are any performance or other reasons not to do
option 2.

thanks
-k

> Also files that already are created will keep the stripe settings they were
> created with. You would need to copy them, and move over the old one to
> change to the new stripe settings. Check the lustre manual they have
> something about this.
>

[Brock Palen]

On Apr 17, 2008, at 10:48 PM, Kaizaad Bilimorya wrote:
>
> On Thu, 17 Apr 2008, Brock Palen wrote:
>> I don't think you need to do this.
>> If i understand right, you can set the stripe size of the mount,
>> and everything inside that directory inherits it, unless they them
>> self's were explicitly set.
>
> Hi Brock, thanks for the reply.
>
> I have set the stripe count on the lustre mount using "lfs
> setstripe" but the problem is that any sub directories that already
> existed under this mount will have the default filesystem stripe
> count and not the new one I set, so any new files created under
> these existent sub directories will inherit their parent directory
> stripe count and not the newly set one from the lustre mount.

Ahh I see, I really don't know, I think try walking the system and
changing all the old directories. I have not had to do this my self.

[Andreas Dilger]

On Apr 18, 2008 11:02 -0400, Brock Palen wrote:
> On Apr 17, 2008, at 10:48 PM, Kaizaad Bilimorya wrote:
> >
> > On Thu, 17 Apr 2008, Brock Palen wrote:
> >> I don't think you need to do this.
> >> If i understand right, you can set the stripe size of the mount,
> >> and everything inside that directory inherits it, unless they them
> >> self's were explicitly set.
> >
> > Hi Brock, thanks for the reply.
> >
> > I have set the stripe count on the lustre mount using "lfs
> > setstripe" but the problem is that any sub directories that already
> > existed under this mount will have the default filesystem stripe
> > count and not the new one I set, so any new files created under
> > these existent sub directories will inherit their parent directory
> > stripe count and not the newly set one from the lustre mount.

Note that this is a new feature in 1.6. I'm not sure if it was in the
1.6.4 release or only 1.6.5, but setting the striping on the root directory
will change the filesystem-wide default striping parameters, immediately
affecting all directories that do not have their own striping set.

This was considered a better interface for setting the default striping
than having to rewrite the MDS configuration file using a different
mechanism.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.