DRBD without LVM?

[Wee Sritippho]

Hi,

Each of my servers only have 2 HDDs. 1 HDD used for OS, so only 1 HDD 
will be available for either DRBD or LVM, is this correct?

Is it possible to use only DRBD?

Thank you

[Phil Regnauld]

Wee Sritippho (weeaix) writes:
>
>
> Hi,
>
> Each of my servers only have 2 HDDs. 1 HDD used for OS, so only 1 HDD
> will be available for either DRBD or LVM, is this correct?

Not exactly. It depends on how you installed your OS. Typically
you'd put both disks in the same Volume Group (i.e.: in LVM) and
then you'd have one or more logical volumes for your OS, and the
rest of the VG for ganeti.

DRBD is placed above the LV for each pair of mirrorred disks.

> Is it possible to use only DRBD?

Ganeti is designed to use LVM for plain + drbd instances, so the
short answer is no. But if we get more details about your setup,
we may be able to give you some advice on (re)configuring things.

[candlerb]

On Monday, 12 September 2016 09:15:22 UTC+1, Wee Sritippho wrote:

Is it possible to use only DRBD?

That question doesn't make much sense as-is.

In ganeti, "plain" instances use LVM logical volumes for their storage; "drbd" instances use LVM logical volumes, but with drbd devices sitting on top of LVM. This is what gives ganeti the ability to migrate individual VMs - it can flip over the replication direction for each instance individually.

So any ganeti setup which is configured to use DRBD, is also capable of plain LVM storage.

[Wee Sritippho]

I though Ganeti would use a whole physical for DRBD, which (from reading both answers) is not the case.

I drew this diagram to express my current understanding:

So Ganeti can use the same volume group as the system and create logical volumes for each instances. Users can choose whether they will use DRBD for that logical volume or not. From above diagram, vm2 will be marked as "DRBD enabled" and could be migrated to another host if it's current host failed, while vm1 and vm3 can still use normal logical volume and won't migrate anywhere in case of host failure.

Please correct me if I still misunderstand anything.

Thank you very much.

[Phil Regnauld]

>
>
> <https://lh3.googleusercontent.com/-ON52qtRd_74/V9fW4QbM5UI/AAAAAAAAC8w/o1H4S-u-ccY89j4EEpiGkxk-lAxeDB2GwCLcB/s1600/ganeti-storage-layout2.png>

Very nice illustration - mostly correct, except drbd would be *above*
the LV for vm2.

> So Ganeti can use the same volume group as the system and create logical
> volumes for each instances. Users can choose whether they will use DRBD for
> that logical volume or not. From above diagram, vm2 will be marked as "DRBD
> enabled" and could be migrated to another host if it's current host failed,
> while vm1 and vm3 can still use normal logical volume and won't migrate
> anywhere in case of host failure.

Absolutely.

And yes you can share the VG between the OS and the instances, you'll
need to

a) make sure you tell the cluster what the name of the VG is if it's
not "ganeti", at init time

b) tell ganeti to ignore the LVs

gnt-cluster modify --reserved-lvs='xxx/swap,xxx/root,...'

[candlerb]

On Tuesday, 13 September 2016 11:59:37 UTC+1, Wee Sritippho wrote:

I though Ganeti would use a whole physical for DRBD, which (from reading both answers) is not the case.

I drew this diagram to express my current understanding:

Excellent diagram :-) And of course, on the second host there will be a lv:drbd1 as well.

Actually for drbd instances, ganeti creates two LVs: one for the actual instance data, and one for the DRBD metadata (which tracks the synchronization state of that replicated pair). The latter is always a small fixed size, 128MB I think.

The command "gnt-node volumes" will list all the LVs on all the nodes, and the instances they are associated with.

 

So Ganeti can use the same volume group as the system and create logical volumes for each instances. Users can choose whether they will use DRBD for that logical volume or not. From above diagram, vm2 will be marked as "DRBD enabled" and could be migrated to another host if it's current host failed, while vm1 and vm3 can still use normal logical volume and won't migrate anywhere in case of host failure.

Spot on. When you activate vm2, a drbd device will be created dynamically at both sides, with the master at the side where the instance is running and the slave at the other side. When you live-migrate the VM, the roles are reversed.

Minor point: if you do decide to keep your system LVs in the same volume group as ganeti's LVs, then "gnt-cluster verify" will warn about these foreign LVs, unknown in its database. You can silence these warnings using:

gnt-cluster modify --reserved-lvs=centos/root,centos/swap

(assuming your lvs are called "root" and "swap" respectively, and both in the volume group "centos")

Another minor point for the diagram: normally the non-LV partition would be /boot, not /root.

Regards,

Brian.

[Wee Sritippho]

For the sake of anyone who might found this post via search engines, here are the fixed diagram and its corresponding Inkscape svg file:

Thanks to Phil Regnauld and candlerb.