[Lustre-discuss] Resizing OSTs?

[Roy Dragseth]

We have replaced all our hardrives with ones twice the size (because of some
problems with the drive type we had).

It would be nice to make use of the extra volume and I'm wondering if we can
just take down the OST, extend the logical partition on the array, run
resize2fs on the filesystem and be good to go? Will the MDT just pick up the
new OST size automatically?

We do not plan to resize the MDT as it still has less than 5% fill-degree.

Regards,
Roy.

--

The Computer Center, University of Tromsø, N-9037 TROMSØ Norway.
phone:+47 77 64 41 07, fax:+47 77 64 41 00
Roy Dragseth, Team Leader, High Performance Computing
Direct call: +47 77 64 62 56. email: roy.dr...@uit.no
_______________________________________________
Lustre-discuss mailing list
Lustre-...@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

[Brian J. Murrell]

On Tue, 2010-11-09 at 14:06 +0100, Roy Dragseth wrote:
>
> It would be nice to make use of the extra volume and I'm wondering if we can
> just take down the OST, extend the logical partition on the array, run
> resize2fs on the filesystem and be good to go? Will the MDT just pick up the
> new OST size automatically?

There is another ongoing thread that you should be paying attention to
under the subject "questions about an OST content". I think it more
than answers these questions but for the resizing.

As for resizing, as far I understand, it is something that should work,
but we don't test it -- _at_all_. If you have enough OSTs that you need
to do this with, it might be worth your while investing in backing one
up and resizing it to see if it works. Of course, you can use your
backup to verify the operation. If you do this, please report your
findings here.

But also, if you have the space to back one up (per the above) you could
simply use the information in the other thread I mentioned to go through
your OSTs rebuilding them one by one on the larger disks, utilizing all
of the space when you do the initial formatting of them.

b.

[Wojciech Turek]

I tried to resize OST a couple years ago. Before I touched production OST I set up a sandbox lustre and tried to resize operation on it and it worked. However when it came to resize the production OST rsize2fs failed (segfaulted) at the end of the process. This was several e2fsprogs releases ago so a lot might have changed since then. Also the production filesystem was 70% used whereas the test lustre was almost empty. Anyway if you are going to try it I recommend to get the latest e2fsprogs and also increasing stack limit to unlimited which may help avoid those pesky segfaults.

Best regards,

Wojciech

[Roy Dragseth]

Thanks to both of you for quick replies with useful info.

I think we will try resize2fs on one ost on the scratch filesystem first and see
how it goes.

The change is still weeks away, we need to disable the ost and move all
important stuff to other osts. I will report back any experience we gain.

Again, thanks.

r.

[Andreas Dilger]

On 2010-11-09, at 06:16, Brian J. Murrell wrote:
> On Tue, 2010-11-09 at 14:06 +0100, Roy Dragseth wrote:
>>
>> It would be nice to make use of the extra volume and I'm wondering if we can
>> just take down the OST, extend the logical partition on the array, run
>> resize2fs on the filesystem and be good to go? Will the MDT just pick up the new OST size automatically?
>

> As for resizing, as far I understand, it is something that should work,
> but we don't test it -- _at_all_. If you have enough OSTs that you need
> to do this with, it might be worth your while investing in backing one
> up and resizing it to see if it works. Of course, you can use your
> backup to verify the operation. If you do this, please report your
> findings here.

Speaking with my non-Oracle hat on - I have done offline resizing of OSTs on top of LVM many times w/o problems (subject to other OST size limitations of course). As suggested elsewhere, using the latest Lustre e2fsprogs is important. Also, as Brian mentions, having a backup is really a good idea. If you only resize a single OST at a time, it would not need a huge amount of space for the backup.

> But also, if you have the space to back one up (per the above) you could
> simply use the information in the other thread I mentioned to go through
> your OSTs rebuilding them one by one on the larger disks, utilizing all
> of the space when you do the initial formatting of them.

Though this would be slower, I've done this on occasion as well if I want to change the configuration of the filesystem (e.g. creating fewer inodes, or chaning other format-time options). It is possible to use lfs_migrate script to empty out the OST first, if this is possible/safe in your environment, so that the OST can be take offline without impacting the rest of the filesystem.

Also, in bug 14489 att 15794 there is an untested patch to pass ioctls down from the lustre mountpoint (e.g. /mnt/mdt/lustre-mdt0000) to the underlying ldiskfs filesystem, so that

Cheers, Andreas

[Andreas Dilger]

On 2010-11-14, at 04:28, Andreas Dilger wrote:
> Speaking with my non-Oracle hat on - I have done offline resizing of OSTs on top of LVM many times w/o problems (subject to other OST size limitations of course). As suggested elsewhere, using the latest Lustre e2fsprogs is important. Also, as Brian mentions, having a backup is really a good idea. If you only resize a single OST at a time, it would not need a huge amount of space for the backup.
>
>> But also, if you have the space to back one up (per the above) you could
>> simply use the information in the other thread I mentioned to go through
>> your OSTs rebuilding them one by one on the larger disks, utilizing all
>> of the space when you do the initial formatting of them.
>
> Though this would be slower, I've done this on occasion as well if I want to change the configuration of the filesystem (e.g. creating fewer inodes, or chaning other format-time options). It is possible to use lfs_migrate script to empty out the OST first, if this is possible/safe in your environment, so that the OST can be take offline without impacting the rest of the filesystem.
>
> Also, in bug 14489 att 15794 there is an untested patch to pass ioctls down from the lustre mountpoint (e.g. /mnt/mdt/lustre-mdt0000) to the underlying ldiskfs filesystem, so that

... it is possible to run resize2fs on the mountpoint of a Lustre OST to increase the size of the underlying filesystem on-the-fly.

Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.

[Alfonso Pardo]

Hello,

I am trying to compile the lustre 1.8.4 modules for debian lenny with
kernel 2.6.26. I have compiled (and installed and reboot) the image
linux kernel with the lustre patch. But When I try to compile the lustre
kernel modules with "m-a a-i lustre" I got the next error:

/usr/share/modass/packages/generic.sh: line 74: debian/rules: Permission
denied

Any suggeest?

thanks!!!

--
Alfonso Pardo Díaz

Unidad de Sistemas y Explotacion (USE)
CETA-CIEMAT
Calle Sola nº 1, Trujillo (CACERES)
Tel. 927 65 93 17
www.ceta-ciemat.es

----------------------------
Confidencialidad:
Este mensaje y sus ficheros adjuntos se dirige exclusivamente a su destinatario y puede contener informaci�n privilegiada o confidencial. Si no es vd. el destinatario indicado, queda notificado de que la utilizaci�n, divulgaci�n y/o copia sin autorizaci�n est� prohibida en virtud de la legislaci�n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente respondiendo al mensaje y proceda a su destrucci�n.

Disclaimer:
This message and its attached files is intended exclusively for its recipients and may contain confidential information. If you received this e-mail in error you are hereby notified that any dissemination, copy or disclosure of this communication is strictly prohibited and may be unlawful. In this case, please notify us by a reply and delete this email and its contents immediately.
----------------------------

[Brian J. Murrell]

On Mon, 2010-11-15 at 11:28 +0100, Alfonso Pardo wrote:
> Hello,

Hi,

> /usr/share/modass/packages/generic.sh: line 74: debian/rules: Permission
> denied

chmod +x debian/rules?

b.