shared storage and other doubts

[Santiago Garcia Mantinan]

Hi!

I'm starting with ganeti and I had read that it was planned that 2.4
would support shared storage, however I read on recent posts that it
is not supported, I'd like to know what the current (2.4.1) status is
(I cannot find any info on this on 2.4.1 so I suppose that it is not
there yet) and if it is not supported what are current plans for its
support?

Another doubt I have is what is the secondary IP used for, I have read
two different explanations, one says that it is used just for drbd and
another says that it is also used for migrations, if it is just used
for drbd wouldn't set it up as my plans are to use shared storage when
available, but if it would be used for other stuff as well... I would
set it up.

Now that I'm thinking about this... I cannot find how would you add
the secondary IP if you have created the cluster without it.

Also If you want to have the master netdev on a management network,
not the "public" one that the virtual machines will use (br0 on my
case), I've read that there might be problems if the master netdev is
not a bridge, can anybody confirm this?

If I have to run a bridge on this management network it is not a
problem but I'd rather not do it, I already have a lot of bridges for
the different machine netwoks.

Thanks in advance. Regards...
--
Manty/BestiaTester -> http://manty.net

[Iustin Pop]

On Fri, Mar 25, 2011 at 10:10:15AM +0100, Santiago Garcia Mantinan wrote:
> Hi!
>
> I'm starting with ganeti and I had read that it was planned that 2.4
> would support shared storage, however I read on recent posts that it
> is not supported, I'd like to know what the current (2.4.1) status is
> (I cannot find any info on this on 2.4.1 so I suppose that it is not
> there yet) and if it is not supported what are current plans for its
> support?

It didn't get in 2.4.0, and we don't add such huge features in
incremental releases (2.4.x).

Part/most of the code has been integrated into what will become 2.5, but
it's not yet tested well.

> Another doubt I have is what is the secondary IP used for, I have read
> two different explanations, one says that it is used just for drbd and
> another says that it is also used for migrations, if it is just used
> for drbd wouldn't set it up as my plans are to use shared storage when
> available, but if it would be used for other stuff as well... I would
> set it up.

Where did you read these explanations? It's used for both DRBD and
migrations.

> Now that I'm thinking about this... I cannot find how would you add
> the secondary IP if you have created the cluster without it.

You can't right now.

> Also If you want to have the master netdev on a management network,
> not the "public" one that the virtual machines will use (br0 on my
> case), I've read that there might be problems if the master netdev is
> not a bridge, can anybody confirm this?

master_netdev can be on any interface, it's not bridge specific. Where
did you read about problems.

> If I have to run a bridge on this management network it is not a
> problem but I'd rather not do it, I already have a lot of bridges for
> the different machine netwoks.

That shouldn't be the case…

regards,
iustin

[Damien Churchill]

On 25 March 2011 10:00, Iustin Pop <ius...@google.com> wrote:
> On Fri, Mar 25, 2011 at 10:10:15AM +0100, Santiago Garcia Mantinan wrote:
>> Hi!
>>
>> I'm starting with ganeti and I had read that it was planned that 2.4
>> would support shared storage, however I read on recent posts that it
>> is not supported, I'd like to know what the current (2.4.1) status is
>> (I cannot find any info on this on 2.4.1 so I suppose that it is not
>> there yet) and if it is not supported what are current plans for its
>> support?
>
> It didn't get in 2.4.0, and we don't add such huge features in
> incremental releases (2.4.x).
>
> Part/most of the code has been integrated into what will become 2.5, but
> it's not yet tested well.
>

Is there support for shared block devices in git master at the moment?
I saw the shared file support but nothing relating to block devices.

[Iustin Pop]

Both the shared file and shard block devices code has been comitted.
However, I haven't actually worked with it _yet_, so Apollon should know
how well it works.

Just create a instance with disk adoption (--disk 0:adopt=/dev/disks/…).

regards,
iustin

[Damien Churchill]

Ah excellent I will have to give that a try. So for use within Ceph I
would have to set up the rbd manually before creating the instance,
and then point it at that?

[Santiago Garcia Mantinan]

> It didn't get in 2.4.0, and we don't add such huge features in
> incremental releases (2.4.x).
>
> Part/most of the code has been integrated into what will become 2.5, but
> it's not yet tested well.

:-(

Any advise on how to proceed now to achieve later a migration to shared storage?

> Where did you read these explanations? It's used for both DRBD and
> migrations.

I don't remember about this, I suppose this was on a list or something
like that, I could't find a clear official doc on network topology.

> master_netdev can be on any interface, it's not bridge specific. Where
> did you read about problems.

Here on a reply from Serge van Ginderachter
http://groups.google.com/group/ganeti/browse_thread/thread/7528ffc36dadcdb

Thanks for the quick reply.

[Iustin Pop]

On Fri, Mar 25, 2011 at 01:07:44PM +0100, Santiago Garcia Mantinan wrote:
> > It didn't get in 2.4.0, and we don't add such huge features in
> > incremental releases (2.4.x).
> >
> > Part/most of the code has been integrated into what will become 2.5, but
> > it's not yet tested well.
>
> :-(
>
> Any advise on how to proceed now to achieve later a migration to shared storage?

Not sure (depends on how much flexibility you have for downtime). Use
DRBD for now, and migrate later via import/export?

> > Where did you read these explanations? It's used for both DRBD and
> > migrations.
>
> I don't remember about this, I suppose this was on a list or something
> like that, I could't find a clear official doc on network topology.

Good point. We need to fix that. There's a bit of text here:

http://docs.ganeti.org/ganeti/current/html/install.html#separate-replication-network

But it's not enough.

> > master_netdev can be on any interface, it's not bridge specific. Where
> > did you read about problems.
>
> Here on a reply from Serge van Ginderachter
> http://groups.google.com/group/ganeti/browse_thread/thread/7528ffc36dadcdb

Ack. Yeah, we need to document this better indeed.

I've filled http://code.google.com/p/ganeti/issues/detail?id=151 to
track this.

thanks for the feedback!
iustin

[Serge van Ginderachter]

On 25 March 2011 16:50, Iustin Pop <ius...@google.com> wrote:

> I don't remember about this, I suppose this was on a list or something
> like that, I could't find a clear official doc on network topology.

Good point. We need to fix that. There's a bit of text here:

http://docs.ganeti.org/ganeti/current/html/install.html#separate-replication-network

But it's not enough.

One other thing that might be interesting to elaborate, is when looking at performance optimisation.

Let me explain my situation.

I have a cluster, with 2 nics per node. One nic is the public facing side, the other nic is used for DRBD.

I set this up with a routed setup on the "public network" and a bridged setup on the internal network.

(This is a setup at hetzner.de, so public really is public ip's, whilst private is a fully separate private GB network, used for setting up all ganeti communication and DRBD.)

Public nic's are plain eth0, routed setup, no bridges (specific for Hetzner).

Internal nic's are bridged. Some vm's also have a private nic on that internal network.

Now, when looking at how to optimize settings, one thread (a couple of weeks ago) mentioned some kernel level tunings to perform on the network card.

https://groups.google.com/forum/#!searchin/ganeti/performan,ce$20sysctl/ganeti/JKejnQkAhgI/overview

This didn't work out for me, as those nic's were bridged. I don't remember the details, but I couldn't make those sysctl tunings because it was on a bridge.

On a sidenote, I think it was a design mistake to use the bridge as the master netdev. I probably should have used the specific network card.

(maybe I should change the paster netdev (and puyt the cluster ip's on eth1 instead of br1, not sure if that is supported.)

Anyway, tuning on a network card is one thing, tuning a bridge is another. I guess it would be helpfull to put some extra documentation about this also :)

--

Met vriendelijke groet,

Serge van Ginderachter