Instance Communication Network

[Ian Chilton]

Hi,

Please could someone explain what the "instance communication network" is (or point me to some documentation)?

I see it with a blank value in my 'gnt-cluster info' command but i'm struggling to find any info about it.

The man page says:

The --instance-communication-network enables instance communication by specifying the name of the Ganeti network that should be used for instance communication. If the supplied network does not exist, Ganeti will create a new network with the supplied name with the default parameters for instance communication. If the supplied network exists, Ganeti will check its parameters and warn about unusual configurations, but it will still use that network for instance communication.

...which doesn't explain much! :)

My nodes have a separate, non-routed VLAN using RFC1918 addressing for replication and intra-node communication (or that was the intention!!) so I created my cluster with: --secondary-ip 10.205.1.1.

So I have:

- br0: Primary network interface

- br1: Local only, non-routed network for intra-node comms/replication etc

Then a whole bunch of bridges for VMs which have no IP on the nodes:

- br-external

- br-internal

- etc

Thanks,

Ian

[candlerb]

The "instance communication network" is *only* for the ganeti instance metadata service, which is a half-baked feature and I think hardly anyone uses it, although I saw one person trying to make an OS installation using it here.

It's a way for a running instance to make a HTTP request to ganeti, to find out information about itself, over a private point-to-point network.

The documentation, such as it is, can be found at:

http://docs.ganeti.org/ganeti/2.16/html/design-os.html#communication-mechanism

And the rest comes from reverse-engineering:

https://groups.google.com/forum/#!topic/ganeti/WOGH3uru2GQ

https://groups.google.com/forum/#!topic/ganeti/LZAovP_5Fsk

My nodes have a separate, non-routed VLAN using RFC1918 addressing for replication and intra-node communication (or that was the intention!!) so I created my cluster with: --secondary-ip 10.205.1.1.

That's something completely different - you have a secondary network for communication between ganeti nodes (for DRBD), and that's a standard thing to do.

 

So I have:

- br0: Primary network interface

- br1: Local only, non-routed network for intra-node comms/replication etc

Then a whole bunch of bridges for VMs which have no IP on the nodes:

- br-external

- br-internal

- etc

All that is fine.  Unless you are developing new ways to auto-install OS images, you don't need the instance communication network to be setup.  Even if you *are* writing OS installation scripts, you don't have to use it (e.g. snf-image doesn't. I believe it makes a temporary floppy image and attaches it to the guest).

[Ian Chilton]

That makes more sense, thank you!

I'm using instance-deboostrap with a bunch of hook scripts for os installation!

Ian