Enabling the metadata daemon

[candlerb]

The NEWS file says that as of 2.12.0:

"The metadata daemon is now optionally available, as part of the

  partial implementation of the OS-installs design. It allows pass

  information to OS install scripts or to instances.

  It is also possible to run Ganeti without the daemon, if desired."

But how do I go about enabling it?

With ganeti 2.15.2 under ubuntu 16.04, I see no metadata daemon running, and I see no mention of the metadata service in the gnt-cluster or gnt-instance manpages.

It looks like a compile-time option, but I see

ENABLE_METAD = True

in  /usr/share/ganeti/2.15/ganeti/_constants.py

Thanks... Brian.

[candlerb]

Ah, so I found this:

http://docs.ganeti.org/ganeti/2.15/html/design-os.html#communication-mechanism

"The communication mechanism will be enabled automatically during an installation procedure that requires a virtualized environment, but, for backwards compatibility, it will be disabled when the instance is running normally, unless explicitly requested. Specifically, a new parameter --communication=yes|no (short version: -C) will be added to gnt-instance add and gnt-instance modify. This parameter will determine whether the communication mechanism is enabled for a particular instance. The value of this parameter will be saved as part of the instance’s configuration."

But this by itself is not enough:

# gnt-instance modify -c yes lxd1

Failure: prerequisites not met for this operation:

error type: wrong_input, error details:

ip=pool, but no network found

The design doc talks about a DHCP server, but I don't know how to configure its pool.  I have not created any gnt-network yet; perhaps one is needed by metad?

<...digs further...>  I found that gnt-cluster has an option "--instance-communication-network", which should create the network if it doesn't exist. Here we go:

# gnt-cluster modify --instance-communication-network cloud-init

Sat Jul  8 17:57:31 2017 Changing instance communication network to 'cloud-init', only new instances will be affected.

Submitted jobs 1737

Waiting for job 1737 ...

And:

# gnt-instance modify -c yes lxd1

Sat Jul  8 17:58:22 2017  - INFO: Chose IP 169.254.0.1 from network cloud-init

Modified instance lxd1

 - nic.1 -> add:mac=52:54:00:01:0a:c7,ip=169.254.0.1,mode=routed,link=communication_rt,network=cloud-init

Please don't forget that most parameters take effect only at the next (re)start of the instance initiated by ganeti; restarting from within the instance will not be enough.

# gnt-instance reboot lxd1

After a "dhclient eth1" inside the instance, I can do:

root@lxd1:~# curl http://169.254.169.254/

Resource not found

That's something. So now I need to know what the actual metadata URLs are.  Try the EC2 one:

root@lxd1:~# curl http://169.254.169.254/2009-04-04/user-data

Resource not found

Back to the source code, and after some grepping through haskell:

root@lxd1:~# curl http://169.254.169.254/ganeti/latest/meta_data.json

root@lxd1:~# curl http://169.254.169.254/ganeti/latest/os/parameters.json

{}

root@lxd1:~#

OK, that's something. Unfortunately without the EC2-compatible resources, I don't think cloud-init is going to work with these without some modifications to the images themselves.

So close...!

Regards, Brian.

[candlerb]

Minor note: I see that ganeti-metad is listening on all interfaces.

tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      25642/ganeti-metad

Maybe it would be better if it bound to 169.254.169.254, which is the gnt.com.0 address? Not that it really matters I don't suppose.

[Scott]

Brian,

I'm looking into cloud-init support in ganeti, thank you for your notes.

Where did you end up on this? Were you able to get the metadata daemon
to work with standard cloud-init images or did you end up modifying the
images themselves?

Tips are welcome from anyone else on getting standard cloud-init images
to work with ganeti.

Scott

[candlerb]

No, sorry.  I gave up with this, and just stuck with snf-image and OS parameters.

[candlerb]

Aside: I also haven't checked if this has changed in 2.16.  "Ganeti OS installation redesign" is marked "partially completed" in the 2.16 design notes, and describes the metadata service:

http://docs.ganeti.org/ganeti/2.16/html/design-2.16.html

http://docs.ganeti.org/ganeti/2.16/html/design-os.html#metadata-service