Having real trouble setting Static IP on VMWare ESXI based install?

[Robert Bridgeman]

Hi All!

I am trying to setup CoreOS with a static IP, which is configured via the Cloud-config file..

Here is the file as it stands. Its a bit messy due to *alot* of trial and error!

#cloud-config


hostname: ConfigCoreOSTest
users:
      - name: rob
        passwd: "PasswdHash"
        groups:
          - "sudo"
          - "docker"


coreos:
 units:
     - name: systemd-networkd.service
       command: stop
     - name: down-interfaces.service
       command: start
       content: |
         [Service]
         Type=oneshot
         ExecStart=/usr/bin/ip link set e* down
         ExecStart=/usr/bin/ip addr flush dev e*
     - name: systemd-networkd.service
       command: start
     - name: 10-static.network
       content: |
         [Match:]
         Name=e*
       
         [Network]
         DHCP=no
         DNS=8.8.8.8
         DNS=4.4.4.4
         Address=192.168.1.227
         Netmask=255.255.255.0
         Gateway=192.168.1.1
       
write_files:
 - path: "/etc/resolv.conf"
   permissions: "0644"
   content: |
     nameserver=8.8.8.8
     nameserver=4.4.4.4
- path: "/usr/lib/systemd/network/zz-default.network"
  permissions: "0777"
  content: |
     [Network]
     DHCP=no
     
     [DHCP]
     UseMTU=false
     useDomains=false

As this is VMware based, the network device seems to randomize its name, hence the e*.

When i run this config file on a new install, i get two IP addresses on my 1 network device, one being DHCP (Set as primary address) and the other being my static IP. 

Any ideas why im still getting a H+DHCP address?

Thanks!

Rob

[Alex Crawford]

On 08/24, Robert Bridgeman wrote:
> As this is VMware based, the network device seems to randomize its name,
> hence the e*.
>
> When i run this config file on a new install, i get two IP addresses on my
> 1 network device, one being DHCP (Set as primary address) and the other
> being my static IP.
> Any ideas why im still getting a H+DHCP address?

coreos-cloudinit runs too late in the boot process to correctly
configure the network. Stopping and starting networkd will not drop
existing IP addresses, so the DHCP address is preserved. On second boot,
it should work properly, since that network config has already been
written. There isn't a good way to fix this with coreos-cloudinit...

...which is why we are throwing it away and going with Ignition [1]
instead. Ignition runs before the system pivot (before disks are
mounted, systemd is started, and the network is configured) so it has
absolute control over your system. Configuring the network properly is
trival [2] with Ignition. Since VMware is supported, it is just a matter
of replacing the cloud-config with an Ignition config (written to the
'coreos.config.data' guest variable).

While developing your Ignition config, you can use the online validator
[3] to double check that it is valid.

Hope that helps.

-Alex

[1]: https://coreos.com/ignition/docs/latest/
[2]: https://coreos.com/ignition/docs/latest/network-configuration.html
[3]: https://coreos.com/validate/

[Robert Bridgeman]

Hi Alex,

Thanks for the reply.

I am looking to setup Ignition, and i need to be able to disable the automatic updates for CoreOS.

I cant seem to find how to do this via Ignition.

I havent personally worked with Json before, so the formatting is new to me.

This is what i have so far.

{

  "ignition": { "version": "2.0.0" },

  "storage": {

    "files": [{

      "filesystem": "root",

      "path": "/etc/hostname",

      "mode": 420,

      "contents": { "source": "data:,CoreOSIgnitionTest" }

    }]

  },

      "networkd": {

        "units": [{

          "name": "00-eth0.network",

          "contents": "[Match]\nName=eth0\n\n[Network]\nAddress=192.168.1.227\nDNS=81.91.102.36\nGateway=192.168.1.1\nDHCP=no"

      }]

    },

    "passwd": {

        "users": [{

            "name": "rob",

            "passwordHash": "$1$W13/jd12$PEdDlpp5yJJX5.akl2gBx1"

        }]

    }

}

I can find how to do this via the cloud-config with ease (below), but i cant see how to do this with ignition.

#cloud-config
coreos:
  units:
    - name: update-engine.service
      command: stop
    - name: locksmithd.service
      command: stop

What can i do to get Ignition to stop the automatic updates?

Am i also correct with the Json above, that this will stop DHCP assigning a new IP, and force the machine to 1.227?

Thanks

[Alex Crawford]

On 08/25, Robert Bridgeman wrote:
> What can i do to get Ignition to stop the automatic updates?

First off, why do you want to disable automatic updates? If you do
really need them off, update_engine and locksmith are the two programs
responsible for the reboots. Both of the service units are enabled by
default, which is why they automatically run. You can just mask them to
prevent them from running.

> Am i also correct with the Json above, that this will stop DHCP assigning a
> new IP, and force the machine to 1.227?

At a glance, that looks correct.

-Alex

[Robert Bridgeman]

Hi Alex,

We need to disable updates because the software we have created to sit on tp of this isnt compatible with the newest version. (Something causes some files to not run on start, and others not to save correctly) and rolling back has already been proven to resolve these issues.

Is the process for compiling the ISO the same as it was for cloud-config? (Guessing it is)

cd /Users/robertbridgeman/config-drive/openstack/latest 
cp user_data /tmp/new-drive/openstack/latest/user_data 
hdiutil makehybrid -iso -joliet -default-volume-name config-2 -o configdrive.iso /tmp/new-drive

I tried to do this, and it seemed to ignore my config. So either im creating the ISO wrong, or ive got an issue with my code which the validator isnt picking up.

Here is the updated config. Is the masking there correct? 

{  
   "ignition":{  
      "version":"2.0.0"

   },
   "storage":{  
      "files":[  
         {  
            "filesystem":"root",
            "path":"/etc/hostname",
            "mode":420,
            "contents":{  
               "source":"data:,CoreOSIgnitionTest"
            }
         },

         {  
            "filesystem":"root",
            "path":"/usr/lib/systemd/network/zz-default.network",
            "contents":{  
               "source":"data:,[Network]\nDHCP=false\n\n[DHCP]\nUseMTU=false\nuseDomains=false"

            }
         }
      ]
   },
   "networkd":{  
      "units":[  
         {  
            "name":"00-eth0.network",
            "contents":"[Match]\nName=eth0\n\n[Network]\nAddress=192.168.1.227\nDNS=81.91.102.36\nGateway=192.168.1.1\nDHCP=no"
         }
      ]
   },
   "passwd":{  
      "users":[  
         {  
            "name":"rob",

            "passwordHash":"PasswordHash"
         }
      ]
   },
   "systemd":{  
      "units":[  
         {  
            "name":"update_engine.service",
            "mask":true
         },
         {  
            "name":"locksmith.service",
            "mask":true
         }
      ]
   }
}

Your help is greatly appreciated.

Regards,

Rob

[Alex Crawford]

On 08/26, Robert Bridgeman wrote:
> We need to disable updates because the software we have created to sit on
> tp of this isnt compatible with the newest version. (Something causes some
> files to not run on start, and others not to save correctly) and rolling
> back has already been proven to resolve these issues.

Ah, bummer. Hopefully you can get that resolved soon.

> Is the process for compiling the ISO the same as it was for cloud-config?
> (Guessing it is)
>

> I tried to do this, and it seemed to ignore my config. So either im
> creating the ISO wrong, or ive got an issue with my code which the
> validator isnt picking up.

Ignition does not read from the config-drive. I assumed you were using
the VMware Guest Variable interface that coreos-cloudinit and Ignition
support. You'll need to pass your config in the "coreos.config.data"
guest variable to the hypervisor (this can be done by editing the .vmx
or via the VIX API).

The masking is correct, but you have a network unit in the files
section. It's not a huge deal, but it's better suited in the
networkd.units section.

-Alex

[Robert Bridgeman]

Hi Alex!

That would explain why its not picking it up! (facepalm)

Starting to feel a little lost, not being able to find where i would change the config to put this code in...

Do you have a "Idiots guide" to accessing this, and adding the Json?

Im currently on the trial version of VMware, would that cause an issue?

Regards,

[Alex Crawford]

On 08/30, Robert Bridgeman wrote:
> Starting to feel a little lost, not being able to find where i would change
> the config to put this code in...
>
> Do you have a "Idiots guide" to accessing this, and adding the Json?
> Im currently on the trial version of VMware, would that cause an issue?

This guide [1] should help you in the right direction. When adding the
JSON config, you'll either need to escape the double quotes (with a
`/22` I believe) or base64-encode the config and specify the encoding in
the guest variables.

I don't have an ESXi install in front of me but you should be able to
use the VMware service console to set the guest variables before booting
the machine for the first time.

I hope that helps.

-Alex

[1]: https://github.com/coreos/docs/blob/master/os/booting-on-vmware.md#vmware-guestinfo-interface