Need to add metadata properties to Glance images in OpenStack CPI

[aa...@hubernet.net]

Currently the CPI is filtering the list of properties that can be set in the metadata to only a small list:

https://github.com/cloudfoundry/bosh/blob/master/bosh_openstack_cpi/lib/cloud/openstack/cloud.rb#L137

vanilla_options = ['name', 'version', 'os_type', 'os_distro', 'architecture', 'auto_disk_config']

The Glance API will accept any name/value pair and add it to the metadata for a stemcell so there is really no need to limit this.  In our new OpenStack environments we're experimenting with multiple hypervisor times and need to set a value named "hypervisor_type" to ensure that we go to the correct compute nodes for provisioning.

So would the preference be to just add additional values that we know we'll need, or just stop filtering the values and let us add what's needed?

Just FYI, the values would actually be pulled from the stemcell manifest file inside the tarball.  I've tested out a code modification and by adding "hypervisor_type" to the list above it will pull the property and push it all the way through to the Glance API correctly.

Aaron Huber

Intel Corporation

[Dmitriy Kalinin]

Since this is an API that BOSH exposes to stemcell builders (us
included) I think it's best to have an explicit list of properties
that are accepted. Who knows what kinds of properties are introduced
later that might conflict with OpenStack glance API.

Btw semi-related to this topic:
https://github.com/cloudfoundry/bosh/blob/master/stemcell_builder/stages/stemcell_openstack/apply.sh#L26

> To unsubscribe from this group and stop receiving emails from it, send an
> email to bosh-dev+u...@cloudfoundry.org.

[aa...@hubernet.net]

Some of the more common properties are listed at http://docs.openstack.org/cli-reference/content/chapter_cli-glance-property.html and as you can see the list is pretty long.  In addition, according to the Glance API, any property can be set with no limit (http://docs.openstack.org/developer/glance/glanceapi.html, see "x-image-meta-property-*" section) so there is no possible way to conflict with the API as these are by definition free-form custom properties.

I'm happy to submit a pull request to add just the ones that we need but since this might become a more common thing I wanted to check if you'd rather just open it up and add any key/value pair stored in the manifest so people are free to add custom values without needing to change the code in the future.  For BOSH and CF this isn't a huge problem as you can always add custom properties after uploading the stemcell but for MicroBOSH there is no time between uploading the stemcell and trying to boot the VM, so in our case if we don't have the properties already set then the VM doesn't boot.  If other folks implement custom nova scheduler rules based on metadata properties then they will have the same issue.

Let me know which you'd prefer and I'll submit a pull request on Monday.

Aaron

[aa...@hubernet.net]

Here are the properties that we need for our multi-hypervisor testing:

hw_vif_model
hypervisor_type
vmware_adaptertype
vmware_disktype
vmware_linked_clone
vmware_ostype

Some of these are on the previous link I posted and the others are documented at http://docs.openstack.org/havana/config-reference/content/vmware.html, so these are all "common" properties (not specific to Intel).  So to close on this, would you prefer I submit a pull request that just adds these properties to the list, or one that opens it up to any custom property as allowed by the Glance API?

Aaron Huber

Intel Corporation

[Dmitriy Kalinin]

I wonder if it's best to then namespace them if there is a large
variety of these attributes. e.g.

manifest = {
"name" => stemcell_name,
"version" => version,
"bosh_protocol" => bosh_protocol,
"sha1" => stemcell_checksum,

"cloud_properties" => {
"name" => stemcell_name,
"version" => version,
"infrastructure" => stemcell_infrastructure,
"architecture" => "x86_64",
"root_device_name" => "/dev/sda1",

"additional_glance_properties" => {
"vmware_adaptertype" =>
"vmware_ostype" =>
"vmware_image_version" =>
...
}
}
}

additional_glance_properties will be then merged with existing white
listed attributes. How does that sound?

Just out curiosity are you creating light stemcells or full stemcells
with these attributes?

[aa...@hubernet.net]

That would take a bit more code than I'd want to be responsible for, but that would work - as long as all the properties end up getting passed as part of image_params[:properties] it doesn't matter how they get there.  For now I'll submit a pull request just add the properties I listed above to the existing whitelist and you can refactor it to be more sophisticated than that as desired.

FYI, we're not actually creating custom stemcells.  The OpenStack ones you guys provide work just fine on VMware once I un-tar them, add the extra properties to the stemcell.MF file, and tar them up again (well, they also need to be converted to raw but we do that to our OpenStack stemcells anyway to support boot from volume).

Aaron Huber

Intel Corporation

[aa...@hubernet.net]

Pull request submitted:

https://github.com/cloudfoundry/bosh/pull/689

Aaron Huber

Intel Corporation