Debian Wheezy Xen xm to xl toolstack migration on 2.14.1

[Artur]

Hello,

I'm programming Debian Wheezy to Jessie upgrade. My current setup is a 2
nodes 2.14.1 ganeti cluster (compiled from source).
On Debian Wheezy Xen version is 4.1, it includes xl toolstack. It is not
activated by default.
Before swithing to Jessie I would try to change the toolstack from the
default xm to xl.

I currently identified /etc/default/xen file with the TOOLSTACK variable
that have to be set to "xl" value.
The reboot of each Ganeti node will be necessary.
This is on the system setup side.
Is there any configuration settings to modify while switching from xm to
xl (xen config files and so on) ?

In Ganeti cluster setup, I've seen the xen-pvm/xen-hvm -> xen_cmd
parameters that must be changed as well.
Is there any other parameters I'll have to change while migrating ?

What will be the right sequence of tasks to do to achieve this migration ?

--

Best regards,
Artur.

[Hrvoje Ribicic]

Hi,

As there is no easy and recommended way to achieve a live xm->xl instance migration right now, you have to reboot all the nodes and all the instances eventually.

My suggestion is to:

1. Change the xen_cmd which Ganeti uses to xl, making Ganeti pretty useless because it will not be able to issue any more instance-affecting or -inspecting commands.

2. Change the toolstack on a node, reboot it, make sure the instances are back up through xl list.

3. Do the same with the other node.

4. Check if all is well with gnt-instance list and gnt-cluster verify.

You will not be able to migrate instances between the xm and xl stacks to ensure they remain online, so make sure to plan for at least some instance downtime.

Cheers,

Riba

Hrvoje Ribicic
Ganeti Engineering

Google Germany GmbH
Dienerstr. 12, 80331, München

Geschäftsführer: Graham Law, Christine Elizabeth Flores

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

[Artur]

Hello,

Thank you for your help.

I planned to shutdown all the instances before the upgrade (before the step 1) then proceed with steps 1,2,3,4 then start the instances again.

I suppose getting back to xm toolstack should work well if there is any error after the upgrade from xm to xl ?

-- 

Best regards,
Artur.

[Hrvoje Ribicic]

Hi,

Shutting down everything is probably the safest route.

As Ganeti will produce the configuration files anew, and disks will be untouched during this process, you should be able to revert to xm.

Cheers,

Riba

[Artur]

OK, that's fine.

One last question. Xen configuration files have been tuned as described in the ganeti documentation. I don't remember exactly but it was probably with version 2.6 of Ganeti at that time.
Xen documentation mentions that any embedded python code in configuration files should be removed but I don't remember of any.
Do you have any idea of this kind of Python code placed in the configuration files on a Ganeti cluster or its instances ?

-- 

Best regards,
Artur.

[Hrvoje Ribicic]

On Thu, Jul 16, 2015 at 5:10 PM, Artur <gan...@netdirect.fr> wrote:

OK, that's fine.

One last question. Xen configuration files have been tuned as described in the ganeti documentation. I don't remember exactly but it was probably with version 2.6 of Ganeti at that time.

I've recently updated the documentation related to that, you can check:

http://docs.ganeti.org/ganeti/2.14/html/install.html#xen-settings

The short version is that with xl, you do not need migration adjustments, and any CPU or memory restrictions need only the autoballoon flag - the rest you take care of with kernel arguments.

 

Xen documentation mentions that any embedded python code in configuration files should be removed but I don't remember of any.
Do you have any idea of this kind of Python code placed in the configuration files on a Ganeti cluster or its instances ?

To the best of my knowledge, we do not generate Python code in the config files. Yuck.

We run xl instances ourselves so this problem would have surfaced laready.

[Artur]

Nice ! The pointer to Xen wiki explain very well the different settings. I have done it already for xm toolstack in the past and the corresponding settings are explained for xl as well.
Thanks a lot, once again. :)

-- 

Best regards,
Artur.

[Artur]

Hello,

I proceed this way to migrate from xm to xl toolstack and everything seems to be ok :

* Shutdown of all running instances in the cluster (gnt-instance shutdown ...)
* Modification of the xen-cmd in the cluster setup :
 gnt-cluster modify -H xen-pvm:xen_cmd=xl
 gnt-cluster modify -H xen-hvm:xen_cmd=xl
* TOOLSTACK default command change in /etc/default/xen :
 TOOLSTACK=xl
* All nodes reboot
* Quick ganeti check after reboot :
 gnt-cluster verify
 gnt-instance list
* Restart of instances (gnt-instance startup ...)
* Additional optimization step in /etc/rc.local :
 xl sched-credit -d 0 -w 512

Remark : Changing TOOLSTACK value seems to have immediate effect on ganeti commands even without reboot. So don't be surprised to see "errors" in 'gnt-instance list' command for example.
BTW : My current ganeti version is 1.15.0 (build from source).

Next step : Ganeti nodes upgrade from Debian Wheezy to Jessie.

I hope this helps. :)

-- 

Best regards,
Artur.