How Do I gnt-network? (was Create VM with Static IP via snf-image?)

[Daniel Howard]

On Mon, Oct 17, 2016 at 6:32 AM, Nikos Skalkotos <skal...@grnet.gr> wrote:

Hello,

If anybody still cares to know how this is done, it is answered here:
https://github.com/grnet/snf-image/issues/90#issuecomment-251619181

​Yes yes thank you!!

I deferred messing with gnt-network when I set up my cluster because ... I already have a network that works and I didn't want to break anything, and I can not find any idiot-level documentation. The documentation speaks of NIC links and VLANs and I was loath to run commands that might break the networking on my Ganeti nodes.

​Our network is flat: 10.10/16 and server hardware intermixes freely with VMs​. We have some structure that maps to, say, rack location, and a few ranges assigned to the DHCP pools.

It looks like gnt-network would be willing to allocate non-conflicting IP addresses from its "network" ... though, I would prefer it allocated from a range. (In practice, I could just allocate IPs in the DNS server and ganeti would pull the IP from there?)

http://docs.ganeti.org/ganeti/2.12/html/man-gnt-network.html

So, assuming the above .. I could tell Ganeti something like this: (?)

gnt-network add --network 10.10.0.0/16 --gateway 10.10.n.n mtv

gnt-network connect mtv

And then ... I might could say, give gnt-test0 an IP address and run this command:

sudo gnt-instance add -s 12G -B memory=2G,vcpus=1 -o snf-image+default -O img_format=diskdump,img_id=ubuntu1604 -t plain gnt-test0

And my gnt-test0 instance would get created with a correctly-configured static IP?

... or does Ganeti need its own subnet which needs to get routed and does that have to match the nodes' physical subnet ... ?

And what are reserved IPs?

... and where do I pull-request the tutorial I am now obliged to write? :)

Thanks!!

-danny

--

http://dannyman.toldme.com

[Phil Regnauld]

Daniel Howard (dannyman) writes:
>
> And what are reserved IPs?
>
> ... and where do I pull-request the tutorial I am now obliged to write? :)

While we're there, I'm a taker on a good intro/tutorial on "routed"
mode in Ganeti - heck, I'll write it if someone explains it to me
in plain English (French, Danish or Spanish will do as well :)

[candlerb]

[Warning: I have not deployed gnt-network in anger]

On Tuesday, 18 October 2016 22:41:01 UTC+1, Daniel Howard wrote:

​Our network is flat: 10.10/16 and server hardware intermixes freely with VMs​. We have some structure that maps to, say, rack location, and a few ranges assigned to the DHCP pools.

It looks like gnt-network would be willing to allocate non-conflicting IP addresses from its "network" ... though, I would prefer it allocated from a range.

Yes, this is a pain. I raised a ticket for this:

https://code.google.com/p/ganeti/issues/detail?id=688

It is marked "PatchesWelcome" if you'd like to contribute :-)

 

 

(In practice, I could just allocate IPs in the DNS server and ganeti would pull the IP from there?)

Not sure about that, but I am pretty sure you can still allocate manually, i.e. --net ... ip=10.10.123.123 instead of --net ... ip=pool

I think it would be another feature request to set "ip=dns".  You could script it though, e.g. --net "...,ip=$(dig +short my.instance.name a)"

 

 

http://docs.ganeti.org/ganeti/2.12/html/man-gnt-network.html

So, assuming the above .. I could tell Ganeti something like this: (?)

gnt-network add --network 10.10.0.0/16 --gateway 10.10.n.n mtv

gnt-network connect mtv

Somewhere in there you'll also have to define which bridge it connects to. I believe this is done at the node-group level; this allows you to have a split cluster where different groups of nodes connect to the same network in different ways.

Then when you create an instance with --net network=mtv, it will know which bridge to attach it to as well.

 

And then ... I might could say, give gnt-test0 an IP address and run this command:

sudo gnt-instance add -s 12G -B memory=2G,vcpus=1 -o snf-image+default -O img_format=diskdump,img_id=ubuntu1604 -t plain gnt-test0

And my gnt-test0 instance would get created with a correctly-configured static IP?

You didn't specify a --net argument there, so you'd have to configure your cluster defaults so that new instances use your "mtv" network. Something like (untested)

gnt-cluster modify -N network=mtv

Not sure if it would be:

gnt-cluster modify -N network=mtv,ip=pool

or if that's implied at instance creation time.

 

... or does Ganeti need its own subnet which needs to get routed and does that have to match the nodes' physical subnet ... ?

It's just a label for a block of IPs which Ganeti allocates out of. Your existing 10.10 network can be a ganeti "network".

If assign a VM an IP address from a block which is not routed anywhere, well, it may not be very useful :-)

 

And what are reserved IPs?

Ones which you don't want ganeti to pick automatically from the range - like the upstream router, or any physical servers which are sitting on the same network.

Given a /16 network, if you only want ganeti only to allocate from a /24 range, you can achieve this by adding 65,280 separate reserved addresses to ganeti. But that is a major pain, hence the ticket above.

I look forward to seeing the tutorial :-)

HTH,

Brian.

[Dimitris Aragiorgis]

Hi all,

* candlerb <b.ca...@pobox.com> [2016-10-19 04:20:48 -0700]:

> [Warning: I have not deployed gnt-network in anger]
>
> On Tuesday, 18 October 2016 22:41:01 UTC+1, Daniel Howard wrote:

> > I deferred messing with gnt-network when I set up my cluster because ...
> > I already have a network that works and I didn't want to break anything,
> > and I can not find any idiot-level documentation. The documentation
> > speaks of NIC links and VLANs and I was loath to run commands that might
> > break the networking on my Ganeti nodes.
> >

Well, gnt-network was implemented at first place because nothing
network related (IP, subnet, gateway, nameservers) was exposed to
external scripts during instance creation or modification. So network
customization on-demand and per instance was practically impossible and
everything should be done beforehand.

gnt-network does not touch the underlying infra. It just exports info to
ifup scripts and hooks. It does not run any command on its own. So it
should not break things. Is it more like a per Ganeti cluster
orchestrator that does IP reservations (just like Ganeti checks for MAC
and UUID uniqueness), abstracts the underlying L2 configuration, groups
things together, etc.

How it interacts with OS definitions, Nikos explained it perfectly here [1].
The environment that gets exported to ifup scripts is here [2] (and for some
reason this is not stated somewhere in the official docs..).

> > ​Our network is flat: 10.10/16 and server hardware intermixes freely with
> > VMs​. We have some structure that maps to, say, rack location, and a few
> > ranges assigned to the DHCP pools.
> >
> > It looks like gnt-network would be willing to allocate non-conflicting IP
> > addresses from its "network" ... though, I would prefer it allocated from a
> > range.
> >
>
> Yes, this is a pain. I raised a ticket for this:
> https://code.google.com/p/ganeti/issues/detail?id=688
>
> It is marked "PatchesWelcome" if you'd like to contribute :-)
>
>

Very welcome indeed! Please do not forget that there is also a concrete
design doc [3] about gnt-network refactoring. Among other things it also
includes the following:

Logic must be added to support IP ranges.

gnt-network modify --add-reserved-ips 192.0.2.20-192.0.2.50 net1

Still, for now, one could use something like:

python -c "import ipaddr; net = ipaddr.IPNetwork('10.0.0.0/16'); print ','.join([str(net[i]) for i in range(1,10)])"

>
> > (In practice, I could just allocate IPs in the DNS server and ganeti would
> > pull the IP from there?)
> >
> >

True. The allocation can be done beforehand or one can use the custom
ifup scripts (i.e. /etc/ganeti/kvm-vif-bridge) or even instance related
hooks to update DNS entries dynamically. Here is an example on how
snf-network does it [4].

>
> Not sure about that, but I am pretty sure you can still allocate manually,
> i.e. --net ... ip=10.10.123.123 instead of --net ... ip=pool
>

True. And it will check if it is already reserved by someone else.

> I think it would be another feature request to set "ip=dns". You could
> script it though, e.g. --net "...,ip=$(dig +short my.instance.name a)"
>
>
>

Still this would require the DNS entry to preexist which is not always
the case.

> > http://docs.ganeti.org/ganeti/2.12/html/man-gnt-network.html
> >
> > So, assuming the above .. I could tell Ganeti something like this: (?)
> > gnt-network add --network 10.10.0.0/16 --gateway 10.10.n.n mtv
> > gnt-network connect mtv
> >
> >
> Somewhere in there you'll also have to define which bridge it connects to.
> I believe this is done at the node-group level; this allows you to have a
> split cluster where different groups of nodes connect to the same network
> in different ways.
>

True. This is done with the --nic-parameters option in the gnt-network
connect command.

> Then when you create an instance with --net network=mtv, it will know which
> bridge to attach it to as well.
>
>
>
> > And then ... I might could say, give gnt-test0 an IP address and run this
> > command:
> >
> > sudo gnt-instance add -s 12G -B memory=2G,vcpus=1 -o snf-image+default -O
> > img_format=diskdump,img_id=ubuntu1604 -t plain gnt-test0
> >
> > And my gnt-test0 instance would get created with a correctly-configured
> > static IP?
> >
>
> You didn't specify a --net argument there, so you'd have to configure your
> cluster defaults so that new instances use your "mtv" network. Something
> like (untested)
>
> gnt-cluster modify -N network=mtv
>
> Not sure if it would be:
>
> gnt-cluster modify -N network=mtv,ip=pool
>
> or if that's implied at instance creation time.
>
>

Preferably at instance creation time. Please note that ip=pool is not
valid when one does not use a network (e.g. one could want just L2
connectivity on a bridge). The design doc mentioned above does include a
proposal how gnt-network should treat L2 only networks and how to separate
L3 info and IP pools from them.

>
> >
> >
> > ... or does Ganeti need its own subnet which needs to get routed and does
> > that have to match the nodes' physical subnet ... ?
> >
> >
> It's just a label for a block of IPs which Ganeti allocates out of. Your
> existing 10.10 network can be a ganeti "network".
>
> If assign a VM an IP address from a block which is not routed anywhere,
> well, it may not be very useful :-)
>
>
>
> > And what are reserved IPs?
> >
> >
> Ones which you don't want ganeti to pick automatically from the range -
> like the upstream router, or any physical servers which are sitting on the
> same network.
>
> Given a /16 network, if you only want ganeti only to allocate from a /24
> range, you can achieve this by adding 65,280 separate reserved addresses to
> ganeti. But that is a major pain, hence the ticket above.
>

Major pain at least I would say.. The design doc tries to cover such a
case with something like:

gnt-network modify --pool add:10.0.0.7-10.0.0.20 net1

> I look forward to seeing the tutorial :-)
>

There is also a hands on section in the admin guide [5] that might
help.

To sum up I think the desired functionality is described in the existing
design doc that unfortunately no one had the time to implement so far.
So, needless to say, patches are more than welcome :)

Cheers,
dimara

[1] https://github.com/grnet/snf-image/issues/90#issuecomment-251619181
[2] https://www.synnefo.org/docs/snf-network/latest/index.html#exported-environment
[3] http://docs.ganeti.org/ganeti/current/html/design-network2.html
[4] https://www.synnefo.org/docs/snf-network/latest/index.html#snf-network-dnshook
[5] http://docs.ganeti.org/ganeti/master/html/admin.html#hands-on-with-gnt-network

> HTH,
>
> Brian.

[George K.]

Hey Phil,

 

There's an explanation on what routed mode is here: https://wiki.xen.org/wiki/Vif-route

How to use it though is not as straightforward as it should and one might also need snf-network + snf-nfdhcpd packages to take full advantage of it.

As we said in GanetiCon2016, we plan to improve documentation and maybe rename these packages so they're "closer" to the rest of the ganeti ecosystem.

When ? soon-ish :)

Regards,