Using virt-install with ignition and kernel arguments

1,094 views
Skip to first unread message

Jeffrey Forman

unread,
Dec 9, 2016, 1:56:02 PM12/9/16
to CoreOS User
Long time listener, first time caller.

tl;dr: I'm trying to convert my CoreOS cloud-config-template setup to using Ignition, but am having some issues passing in the Ignition file path as a kernel parameter.

More: I currently use virt-install on top of a ubuntu/libvirt/kvm based VM host, and follow all the practices in the libvirt coreos doc [1] to spin up new CoreOS guest VMs. But to move to Ignition, I have to specify the Ignition config file [2], see "bare metal" via a kernel parameter. According to virt-install [3], I can do this with --extra-args flag, BUT this requires using --location, which is a local (or remote) path to a kernel/initrd to launch the install [4]. That latter part, about the kernel/initrd path is something I havent been able to figure out as it relates to CoreOS.

So my questions are:
* Is this (kernel/initrd) something CoreOS support as an install mechanism?
* Have other folks been able to get virt-install + CoreOS + ignition to launch CoreOS installs working?
* Is there another/better way to do this? (I'd rather not go down the PXE route and have to configure all that) 

Thanks,
Jeff

Alex Crawford

unread,
Dec 9, 2016, 2:20:45 PM12/9/16
to Jeffrey Forman, CoreOS User
On 12/09, Jeffrey Forman wrote:
> Long time listener, first time caller.
>
> tl;dr: I'm trying to convert my CoreOS cloud-config-template setup to using
> Ignition, but am having some issues passing in the Ignition file path as a
> kernel parameter.
>
> More: I currently use virt-install on top of a ubuntu/libvirt/kvm based VM
> host, and follow all the practices in the libvirt coreos doc [1] to spin up
> new CoreOS guest VMs. But to move to Ignition, I have to specify the
> Ignition config file [2], see "bare metal" via a kernel parameter.
> According to virt-install [3], I can do this with --extra-args flag, BUT
> this requires using --location, which is a local (or remote) path to a
> kernel/initrd to launch the install [4]. That latter part, about the
> kernel/initrd path is something I havent been able to figure out as it
> relates to CoreOS.
>
> So my questions are:
> * Is this (kernel/initrd) something CoreOS support as an install mechanism?
> * Have other folks been able to get virt-install + CoreOS + ignition to
> launch CoreOS installs working?
> * Is there another/better way to do this? (I'd rather not go down the PXE
> route and have to configure all that)

Thanks for calling in.

Ignition supports the QEMU firmware configuration device [1] for passing
through configs. The following can be added to the invocation of QEMU to
pass "example.ign" through to Ignition:

-fw_cfg name=opt/com.coreos/config,file=example.ign

It looks like libvirt supports passing through arbitrary commands to
QEMU [2], so you might be able to use this (I haven't tested this
specifically).

I'm curious to know how well this works (if at all).

-Alex

[1]: https://raw.githubusercontent.com/qemu/qemu/master/docs/specs/fw_cfg.txt
[2]: https://libvirt.org/drvqemu.html#qemucommand
signature.asc

Alex Crawford

unread,
Dec 9, 2016, 2:54:57 PM12/9/16
to Jeffrey Forman, CoreOS User
On 12/09, Alex Crawford wrote:
> It looks like libvirt supports passing through arbitrary commands to
> QEMU [2], so you might be able to use this (I haven't tested this
> specifically).

It looks like we've already tried this [1] but ran into some trouble.
I'll need to dig into this to figure out where the problem lies.

-Alex

[1]: https://github.com/coreos/bugs/issues/1693
signature.asc

Forman, Jeffrey

unread,
Dec 9, 2016, 2:57:20 PM12/9/16
to Alex Crawford, CoreOS User
Ack, yep. I just happened to be looking at that same issue at exactly the same time. I've now subscribed, and will be happy to test when you have some ideas.

Eamonn O'Toole

unread,
Jan 24, 2017, 8:12:20 AM1/24/17
to CoreOS User, li...@jeffreyforman.net
Hi Alex,

I am a first-time poster.  I came across this post and the bug https://github.com/coreos/bugs/issues/1693 when searching for information on passing ignition configuration into CoreOS using libvirt.  I am using Ubuntu 16.04.  My problem is this: as far as I can tell, the correct XML is being emitted by "coreloko" and also by some mods that I've made to the terraform libvirt provider (https://github.com/eamonnotoole/terraform-provider-libvirt/tree/ignition).  However the provided ignition file is not loaded into CoreOS.  And I don't understand why.

What platforms have you tested CoreOS libvirt ignition on?  I've also tried RedHat EL7, but on my installation the -fw_cfg option to qemu-kvm is not recognised (I think that the version of qemu-kvm installed might not support -fw_cfg, it is running 1.5.3).

I've attached the /etc/libvirt/qemu file for an example domain generated by my terraform libvirt mods.

Thanks.

-Eamonn.
domain-0.xml

paul...@coreos.com

unread,
Jan 31, 2017, 11:46:31 AM1/31/17
to CoreOS User, li...@jeffreyforman.net
Hi Eamonn,

To give this a bit more visibility, I've added to the coreos/bugs issue tracker.

Feel free to add more detail and subscribe.

Cheers,

paul...@coreos.com

unread,
Feb 1, 2017, 11:51:40 AM2/1/17
to CoreOS User, li...@jeffreyforman.net
Hi Eamonn,

You got some dev responses on the GitHub issue:

Cheers,

cr...@craigtracey.com

unread,
Mar 21, 2017, 5:30:33 PM3/21/17
to CoreOS User, li...@jeffreyforman.net
Hi Eamonn,

I just wanted to update on this issue.  I, too, am using terraform (with libvirt) to stand up coreos, and ran into the same issue.  My host OS was Xenial running QEMU 2.5.  It was not until I upgraded to QEMU 2.6 that I was able to use the fw_cfg/ignition functionality I was looking for (despite it being a feature of 2.4+).

Hope this helps!
Craig  

Gin

unread,
Jul 10, 2017, 3:14:20 PM7/10/17
to CoreOS User
Hi,

I am having similar issues when trying to load ignition config to CoreOS.
with the following software versions:
QEMU 2.6.2
virsh --version 1.3.1

Excerpt from libvirt xml:
<qemu:commandline>
<qemu:arg value='-fw_cfg'/>
<qemu:arg value='name=opt/com.coreos/config,file=/tmp/config.ign'/>
</qemu:commandline>

Virsh define works fine. Virsh start throws the following:
error: Failed to start domain an-master-1
error: internal error: process exited while connecting to monitor: warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2017-07-10T19:11:17.421602Z qemu-system-x86_64: -fw_cfg name=opt/com.coreos/config,file=/tmp/config.ign: can't load /tmp/config.ign

Any ideas what might have gone wrong?

Thanks,
Gin

Gin

unread,
Jul 10, 2017, 5:35:26 PM7/10/17
to CoreOS User
To add, config.ign is owned by ubuntu with 777 access permissions.

Lewis Deng

unread,
Jul 31, 2017, 1:35:29 AM7/31/17
to CoreOS User
do you solve this problem?

Gintautas Sulskus

unread,
Jul 31, 2017, 2:59:02 AM7/31/17
to Lewis Deng, CoreOS User
No, I ended up using coreos' cloud-config.

Gintas

--
You received this message because you are subscribed to a topic in the Google Groups "CoreOS User" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/coreos-user/g4ujTncIW8M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to coreos-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages