How to change ignition data post disk install

[Andrew Webber]

With cloud config I could change the user_data and reboot the machine to make changes with the following command

sudo vim /var/lib/coreos-install/user_data

What is the equivalent after performing an iPXE matchbox install?

kind regards,

Andrew

[Alex Crawford]

You can just change the Ignition config and reboot. Since you are PXE
booting, the machine will pick up the new config and reprovision itself.

-Alex

[Andrew Webber]

My understanding is that this assumes I am running in RAM and have not installed to disk. I would not PXE boot a machine that has already been provisioned.

One of the many advantages of the cloud init process that a misconfigured machines would be restored on reboot because cloud init runs every boot. Ignition, as far as I know, does not (only if a special kernel flag it set).

This helped when a malicious program or strange admin performed some changes on the machine that made the machine unhealthy. Many times cloud init would reset those changes in for example a write_files section. This felt like a nice process to combine with a coreos update restart (clean partition with new update and a reset of all files and services specified in the cloud config).

I guess this is not possible because one cannot specific in ignition sections that sections should only run on first boot (e.g. wipe file system).

I now make changes using a daemonset in K8, logically achieving the same goals as fleet however I have no semi-tampering protection that cloud init gave me

https://github.com/kubernetes/contrib/blob/master/startup-script/startup-script.yml

kind regards,

Andrew

[Alex Crawford]

On 07/12, 'Andrew Webber' via CoreOS User wrote:
> My understanding is that this assumes I am running in RAM and have not
> installed to disk. I would not PXE boot a machine that has already been
> provisioned.

Ah, yes. I misunderstood and thought you were PXE booting the system
everytime.

> One of the many advantages of the cloud init process that a misconfigured
> machines would be restored on reboot because cloud init runs every boot.
> Ignition, as far as I know, does not (only if a special kernel flag it set).
> This helped when a malicious program or strange admin performed some
> changes on the machine that made the machine unhealthy. Many times cloud
> init would reset those changes in for example a write_files section. This
> felt like a nice process to combine with a coreos update restart (clean
> partition with new update and a reset of all files and services specified
> in the cloud config).

This is outside the scope of Ignition. It's designed to be a
provisioning tool only. If you are okay with re-provisioning your
machine every boot (without wiping data), it would be possible to set up
your machines to PXE boot and use a physical disk for their data. This
would ensure that your configuration is reapplied every boot without
throwing away docker containers, cache, etc..

> I guess this is not possible because one cannot specific in ignition
> sections that sections should only run on first boot (e.g. wipe file
> system).

Every section is evaluated when Ignition is run, but that doesn't
necessarily mean that Ignition will make a change. The "filesystems"
section is a good example of this [1]. Ignition just makes sure that the
end state is what was requested. If the filesystems are already
configured, Ignition won't touch them or their data.

-Alex

[1]: https://coreos.com/ignition/docs/latest/filesystems.html

[Andrew Webber]

Many thanks, I will evaluate your recommendations.

If I PXE boot and mount local storage presumably I run CoreOS in RAM and do not get automatic updates/reboots (from my understanding). But I think this is outside of the scope of my question.

[Alex Crawford]

On 07/12, 'Andrew Webber' via CoreOS User wrote:
> If I PXE boot and mount local storage presumably I run CoreOS in RAM and do
> not get automatic updates/reboots (from my understanding).

That is correct. If you go this route, you'll need to keep the PXE image
up-to-date and occasionally reboot.

-Alex