[Lustre-community] MDS RAM Calculation

Skip to first unread message

Indivar Nair

Feb 9, 2012, 8:50:08 AM2/9/12
to lustre-c...@lists.lustre.org
Hi ...,

We are having trouble with the MDS in our setup. It runs out of memory when we do large searches on the storage.

The Setup:

We have a Lustre Setup with 2 MDS Servers, replicated using DRBD and 4 OSS Nodes.
The total storage capacity is aroung 18TB.
We are using Lustre

We have -
- 30 Lustre Clients (CentOS 6)
- 4 Samba Gateway Servers
    - around 120 - 130 Users connect through the Samba Gateway

Both the MDS Servers have -

- 2 x 4 Core Intel CPUs
- 12Gb RAM

The DRBD replication happens over an Infinband Link.

The Issue:

We have around 5.5Million files in the storage. As such everything works fine during normal operations.
But there are times when we need to search the whole storage, like for taking backup of recently changed files, and this is when the MDS crashes giving OOM errors. Any such operation where a single client side process tries to search the whole storage, causes this OOM problem.

1. Is there any setting that could prevent this?
    Since the same files are not accessed frequently, we don't require extensive caching.
    Is there anyway we can optimize the RAM utilization accordingly?

    If not -
2. Overtime we see the number of files growing from 5.5Million to 7.5Million, but I would like to size the RAM for 10Million files. Just to be on the safe side.
    How do I go about calculating the exact RAM requirement?

Do tell me if you need any further information on this.

Thanks and Regards,

Indivar Nair

Reply all
Reply to author
0 new messages