I am automating tasks on my vm infrastructure.
I want to deploy OVF files on the ESX. And do further operations on
them( on/off/status/suspend etc.).
How is it possible?
Any ideas?
Thanks
Nikunj
I've just come from holiday and seen your message. I'm at home right
now and I don't have any ESXs to test against. Tomorrow at the office
I'll try to write some code snippet that might guide you in your task.
I've never worked with OVF files before but I'll see what I can do.
Regards,
Sebastian.
According to the vSphere API documentation this is the process for
importing a vm or virtual appliance from an OVF:
"""
(via the OvfManager managed object)
Import
For the import scenario, the typical sequence of events is as follows:
The client calls parseDescriptor to obtain information about the OVF
descriptor. This typically includes information (such as a list of
networks) that must be mapped to VI infrastructure entities.
The OVF descriptor is validated against the OVF Specification, and any
errors or warnings are returned as part of the ParseResult. For
example, the parser might encounter a section marked required that it
does not understand, or the XML descriptor might be malformed.
The client decides on network mappings, datastore, properties etc. It
then calls createImportSpec to obtain the parameters needed to call
ResourcePool.importVApp.
If any warnings are present, the client will review these and decide
whether to proceed or not. If errors are present, the ImportSpec will
be missing, so the client is forced to give up or fix the problems and
then try again.
The client now calls ResourcePool.importVApp, passing the ImportSpec
as a parameter. This will create the virtual machines and VirtualApp
objects in VI and return locations to which the files of the entity
can be uploaded. It also returns a lease that controls the duration of
the lock taken on the newly created inventory objects. When all files
have been uploaded, the client must release this lease.
"""
And this is the ImportVApp documentation:
"""
ImportVApp
Creates a new entity in this resource pool. The import process
consists of two steps:
Create the VMs and/or vApps that make up the entity.
Upload virtual disk contents.
In step 1, the client must wait for the server to create all inventory
objects. It does that by monitoring the state property on the
HttpNfcLease object returned from this call. When the server is done
creating objects, the lease will change to the ready state, and step 2
begins. If an error occurs while the server is creating inventory
objects, the lease will change to the error state, and the import
process is aborted.
In step 2, the client uploads disk contents using the URLs provided in
the info property of the lease. The client must call
HttpNfcLeaseProgress on the lease periodically to keep the lease alive
and report progress to the server. Failure to do so will cause the
lease to time out, and the import process will be aborted.
When the client is done uploading disks, it completes the lease by
calling HttpNfcLeaseComplete. The client can also abort the import
process by calling HttpNfcLeaseAbort.
If the import process fails, is aborted, or times out, all created
inventory objects are removed, including all virtual disks.
This operation only works if the folder's childType includes
VirtualMachine.
Depending on the properties of the virtual machine bring imported,
additional privileges may be required. See CreateVM_Task for a
description of these privileges.
"""
The script I'll show later does all this up to the point where I get
the HttpNfcLease, I keep querying its state while it is on
"initializing" (meaning that the server is still creating the new vm
directory and file structure) and then suddenly the server crashes.
I did the same thing manually with the vSphere Client tool, and it
works all right. However got soap requests, and responses traces from
both, pysphere and the vSphere Client but I haven't found anything
wrong there.
Nkunj, can you run the following code using a test environment (DON'T
USE PRODUCTION as you might crash the server) an confirm I you crash
it too?
Of course you have to modify the server connection settings and the
following parameters with your values:
RESOURCE_POOL = "/Resources"
OVF_FILE = r"C:\FreeSCO\FreeSCO-043.ovf"
HOST = "localhost.localdomain"
DATASTORE = "datastore1"
NETWORK_NAME = "VM Network"
VAPP_NAME = "My New VM2"
Here's the code:
from pysphere import VIServer, VIProperty
from pysphere.resources import VimService_services as VI
def get_descriptor(ovf_path):
fh = open(ovf_path, "r")
ovf_descriptor = fh.read()
fh.close()
return ovf_descriptor
def parse_descriptor(ovf_descriptor):
ovf_manager = s._do_service_content.OvfManager
request = VI.ParseDescriptorRequestMsg()
_this =request.new__this(ovf_manager)
_this.set_attribute_type(ovf_manager.get_attribute_type())
request.set_element__this(_this)
request.set_element_ovfDescriptor(ovf_descriptor)
pdp = request.new_pdp()
pdp.set_element_locale("")
pdp.set_element_deploymentOption("")
request.set_element_pdp(pdp)
return s._proxy.ParseDescriptor(request)._returnval
def validate_host(host_name, ovf_descriptor):
hosts = s.get_hosts()
host = hosts.get(host_name)
if not host:
raise ValueError("invalid ESX host name")
ovf_manager = s._do_service_content.OvfManager
request = VI.ValidateHostRequestMsg()
_this =request.new__this(ovf_manager)
_this.set_attribute_type(ovf_manager.get_attribute_type())
request.set_element__this(_this)
request.set_element_ovfDescriptor(ovf_descriptor)
h = request.new_host(host)
h.set_attribute_type(host.get_attribute_type())
request.set_element_host(h)
vhp = request.new_vhp()
vhp.set_element_locale("")
vhp.set_element_deploymentOption("")
request.set_element_vhp(vhp)
return s._proxy.ValidateHost(request)._returnval
def find_datastore_by_name(datastore_name):
ret = None
for dc in s._get_datacenters().values():
dc_properties = VIProperty(s, dc)
for ds in dc_properties.datastore:
if ds.name == datastore_name:
ret = ds._obj
break
if ret:
break
if not ret:
raise ValueError("Couldn't find datastore '%s'" %
(datastore_name))
return ret
def find_network_by_name(network_name):
ret = None
for dc in s._get_datacenters().values():
dc_properties = VIProperty(s, dc)
for nw in dc_properties.network:
if nw.name == network_name:
ret = nw._obj
break
if ret:
break
if not ret:
raise ValueError("Couldn't find network '%s'" %
(network_name))
return ret
def find_vmfolder_by_name(folder_name):
ret = None
for dc in s._get_datacenters().values():
dc_properties = VIProperty(s, dc)
if dc_properties.vmFolder.name == folder_name:
return dc_properties.vmFolder._obj
raise ValueError("Couldn't find network '%s'" % (network_name))
def create_import_spec(resource_pool_mor,
datastore,
ovf_descriptor,
name,
host=None,
network=None,
ip_allocation_policy="fixedPolicy",
ip_protocol="IPv4",
disk_provisioning="flat"
):
#get the host MOR
if host:
hosts = s.get_hosts()
host = hosts.get(host)
if not host:
raise ValueError("invalid ESX host name")
#get the network MOR:
if network:
network_mor = find_network_by_name(network)
#get the datastore MOR
datastore_mor = find_datastore_by_name(datastore)
ovf_manager = s._do_service_content.OvfManager
request = VI.CreateImportSpecRequestMsg()
_this =request.new__this(ovf_manager)
_this.set_attribute_type(ovf_manager.get_attribute_type())
request.set_element__this(_this)
request.set_element_ovfDescriptor(ovf_descriptor)
rp = request.new_resourcePool(resource_pool_mor)
rp.set_attribute_type(resource_pool_mor.get_attribute_type())
request.set_element_resourcePool(rp)
ds = request.new_datastore(datastore_mor)
ds.set_attribute_type(datastore_mor.get_attribute_type())
request.set_element_datastore(ds)
cisp = request.new_cisp()
cisp.set_element_entityName(name)
cisp.set_element_locale("")
cisp.set_element_deploymentOption("")
if host:
h = cisp.new_hostSystem(host)
h.set_attribute_type(host.get_attribute_type())
cisp.set_element_hostSystem(h)
if network:
network_mapping = cisp.new_networkMapping()
network_mapping.set_element_name(network)
n_mor = network_mapping.new_network(network_mor)
n_mor.set_attribute_type(network_mor.get_attribute_type())
network_mapping.set_element_network(n_mor)
network_mapping2 = cisp.new_networkMapping()
network_mapping2.set_element_name("Internal")
n_mor = network_mapping.new_network(network_mor)
n_mor.set_attribute_type(network_mor.get_attribute_type())
network_mapping2.set_element_network(n_mor)
cisp.set_element_networkMapping([network_mapping,
network_mapping2])
if ip_allocation_policy:
cisp.set_element_ipAllocationPolicy(ip_allocation_policy)
if ip_protocol:
cisp.set_element_ipProtocol(ip_protocol)
if disk_provisioning:
cisp.set_element_diskProvisioning(disk_provisioning)
request.set_element_cisp(cisp)
return s._proxy.CreateImportSpec(request)._returnval
def import_vapp(resource_pool, import_spec, host=None, folder=None):
#get the host MOR
if host:
hosts = s.get_hosts()
host = hosts.get(host)
if not host:
raise ValueError("invalid ESX host name")
#get the vm folder MOR
if folder:
folder = find_vmfolder_by_name(folder)
request = VI.ImportVAppRequestMsg()
_this =request.new__this(resource_pool)
_this.set_attribute_type(resource_pool.get_attribute_type())
request.set_element__this(_this)
request.set_element_spec(import_spec.ImportSpec)
if host:
h = request.new_host(host)
h.set_attribute_type(host.get_attribute_type())
request.set_element_host(h)
if folder:
f = request.new_folder(folder)
f.set_attribute_type(folder.get_attribute_type())
request.set_element_folder(f)
return s._proxy.ImportVApp(request)._returnval
if __name__ == "__main__":
#you can get the resource pools running s._get_resource_pools()
RESOURCE_POOL = "/Resources"
OVF_FILE = r"C:\FreeSCO\FreeSCO-043.ovf"
#you can get the host names running s.get_hosts()
HOST = "localhost.localdomain"
DATASTORE = "datastore1"
NETWORK_NAME = "VM Network"
VAPP_NAME = "My New VM2"
s = VIServer()
s.connect("host", "username", "password")
try:
resource_pool = s._get_resource_pools()[RESOURCE_POOL]
ovf = get_descriptor(OVF_FILE)
descriptor_info = parse_descriptor(ovf)
support_info = validate_host(HOST, ovf)
import_spec = create_import_spec(resource_pool,
DATASTORE,
ovf,
VAPP_NAME,
host=HOST,
network=NETWORK_NAME,
)
if hasattr(import_spec, "Warning"):
print "Warning:", import_spec.Warning[0].LocalizedMessage
http_nfc_lease = import_vapp(resource_pool, import_spec,
host=HOST)
lease = VIProperty(s, http_nfc_lease)
while lease.state == 'initializing':
print lease.state
lease._flush_cache()
if lease.state != 'ready':
print "something went wrong"
exit()
for url in lease.info.deviceURL:
print url.key
#Once we reach here, we should use the provided url to upload
the .vmdk file(s)
#But the ESX has already crashed by this time
finally:
s.disconnect()
Besides trying to make this work I'll see if the bug is exploitable,
maybe I can get to execute arbitrary code in the server :P
Regards,
Seba
Thanks a lot for your reply.
I tried running the script with my set of parameters.
I am getting following error.
{{{
C:\Documents and Settings\badjan\Desktop>python pysphere_test_ovf.py
Traceback (most recent call last):
File "pysphere_test_ovf.py", line 238, in <module>
network=NETWORK_NAME,
File "pysphere_test_ovf.py", line 176, in create_import_spec
return s._proxy.CreateImportSpec(request)._returnval
File "C:\Python27\lib\site-packages\pysphere\resources
\VimService_services.py", line 1790, in Crea
teImportSpec
response =
self.binding.Receive(CreateImportSpecResponseMsg.typecode)
File "C:\Python27\lib\site-packages\pysphere\ZSI\client.py", line
544, in Receive
return _Binding.Receive(self, replytype, **kw)
File "C:\Python27\lib\site-packages\pysphere\ZSI\client.py", line
463, in Receive
raise FaultException(msg)
pysphere.ZSI.FaultException:
<pysphere.resources.VimService_services_types.InvalidHostStateFault_Dec_Holder
object at 0x0293DAD0>
}}}
I am giving NETWORK_NAME = "VM Network" and this is the name available
on the esx host and the ovf too has the same network name.
Any ideas?
Regards,
Nikunj
InvalidState | Thrown if the operation failed due to the current state of the system. |
I checked once more by putting the exception handling line into the
script.
{{{
C:\Documents and Settings\badjan\Desktop>python pysphere_test_ovf.py
InvalidHostStateFault:
}}}
This is what I get.
Further,
I am trying to find the number of NIC cards present at a particular
ESX. I had a look at the source code but couldnt get anymethod for it.
Also, some methods require "mor" objects. What are those?
Regards,
Nikunj
On Dec 14, 5:37 pm, Seba <argo...@gmail.com> wrote:
> Hi!
> This is the exception you are getting (which is returned by the server):http://pubs.vmware.com/vsphere-50/index.jsp?topic=/com.vmware.wssdk.a...
> It's a subclass of this one:http://pubs.vmware.com/vsphere-50/topic/com.vmware.wssdk.apiref.doc_5...
>
> The createImportSpec method described herehttp://pubs.vmware.com/vsphere-50/index.jsp?topic=/com.vmware.wssdk.a...
>
> may throw an InvalidState fault:
> InvalidState<http://pubs.vmware.com/vsphere-50/topic/com.vmware.wssdk.apiref.doc_5...>Thrown
> if the operation failed due to the current state of the system.
>
> Perhaps the host is maintenance mode or some required objects are locked by
> another operation.
>
> You can catch the exception in the try block at the end of my script to get
> more details about what caused the error:
>
> .....
> except VI.ZSI.FaultException, e:
> print "%s: %s" % (e.fault.detail[0].typecode.pname, e.fault.args[1])
> finally:
> s.disconnect()
>
> Regards,
>
> Seba.
>
> 2011/12/14 Nikunj <nikunjbadja...@gmail.com>
>
>
>
> > Hi Seba,
>
> > Thanks a lot for your reply.
> > I tried running the script with my set of parameters.
> > I am getting following error.
>
> > {{{
> > C:\Documents and Settings\badjan\Desktop>python pysphere_test_ovf.py
> > Traceback (most recent call last):
> > File "pysphere_test_ovf.py", line 238, in <module>
> > network=NETWORK_NAME,
> > File "pysphere_test_ovf.py", line 176, in create_import_spec
> > return s._proxy.CreateImportSpec(request)._returnval
> > File "C:\Python27\lib\site-packages\pysphere\resources
> > \VimService_services.py", line 1790, in Crea
> > teImportSpec
> > response =
> > self.binding.Receive(CreateImportSpecResponseMsg.typecode)
> > File "C:\Python27\lib\site-packages\pysphere\ZSI\client.py", line
> > 544, in Receive
> > return _Binding.Receive(self, replytype, **kw)
> > File "C:\Python27\lib\site-packages\pysphere\ZSI\client.py", line
> > 463, in Receive
> > raise FaultException(msg)
> > pysphere.ZSI.FaultException:
>
> > <pysphere.resources.VimService_services_types.InvalidHostStateFault_Dec_Holder
> ...
>
> read more »- Hide quoted text -
>
> - Show quoted text -
Thanks a lot Seba!
That worked for me. :)
I myself tried few other examples to make myself more comfortable. Slower method but have a look.
In a particular hostsystem:
1. Finding no. of networks a VM is using .
{{{
>>> for h, mor in server.get_hosts().items():
... if h ==’esx.domain.com’:
... prop = VIProperty(server, mor)
... for vmname in prop.vm:
... print vmname.name
... for net in vmname.network:
... print " ", net.name
}}}
2. Number of ethernet cards and virtual disks a VM has.
{{{
>>> for h, mor in server.get_hosts().items():
... if h == ’esx.domain.com':
... prop = VIProperty(server, mor)
... for vmname in prop.vm:
... print vmname.name
... print “ ethernet cards: “ , vmname.summary.config.numEthernetCards, “Virtual disks: “, vmname.summary.config.numVirtualDisks
}}}
I now understood what a MOR means and how to get different properties of Hostsystem, VM, Datastore etc.
As I said earlier, we are trying to deploy OVF file. Previously we were using powershell because of greater bonding with powercli. But now we are planning to remove powershell and make it 100% python.
We are in dire need for completed Advanced Usage guide of pysphere. May be now at some places I can too contribute in the documentation ;)
So, since pysphere is for interacting with vsphere web services sdk. It can do virtually anything what the sdk offers ??
When we compare pysphere with vsphere powercli sdk, how good it stands ??
Some more queries coming up! ;)
Thanks
Nikunj
> When we compare pysphere with vsphere powercli sdk, how good it stands ??
I answered a similar question here: http://communities.vmware.com/message/1876247#1876247
Thanks!
Sebastian
I've figured it out and successfully deployed a vm!!! I changed the PUT to POST and it worked! I don't understand why that worked. Do you think we should always be using POST instead of PUT?
Also the response code was not 200 but everything worked. I'm not sure what we got back because I didn't print it out.
How do I access the lease.info.status to keep the connection alive?
I've got a couple more issues to work out but I won't trouble you with them.
Thanks for your help on this issue!
Nick
On Fri, Jan 20, 2012 at 11:22 AM, Nick Wiesmueller <nick.wie...@singlewire.com> wrote:
Bah!
I'm getting a connection refused on this line
resp = opener.open(request)
File "C:\Python27\lib\urllib2.py", line 394, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 412, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 372, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 1207, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "C:\Python27\lib\urllib2.py", line 1174, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 10054] An existing connection was forcibly closed by the remote host>
Still, it's progress, but I don't know how to troubleshoot the problem.
The vmware logs are very difficult to decipher. I did see this message at the time of attempted deploy:
2012-01-20T10:17:07.887-06:00 [03904 warning 'Default'] [VpxdVmprovUtil::GetDiskDatastoreFromUrl] Unable to lookup ds for disk
I don't know that this has anything to do with the problem though.
Nick
I looked at the vmcenter logs and I didn't see anything useful.
elf.localt.http_nfc_lease = self.import_vapp(self.localt.resource_pool, self.localt.import_spec, host=self.localt.hostname, folder=folder) 09:29:54 File "./src/informacast/linux_util/vm_util.py", line 978, in import_vapp 09:29:54 request.set_element_spec(import_spec.ImportSpec) 09:29:54 File "/root/jenkins/workspace/DeployONELinuxVM/lib/pysphere-0.1.6-py2.7.egg/pysphere/ZSI/generate/pyclass.py", line 167, in get 09:29:54 return getattr(self, what().aname) 09:29:54 AttributeError: 'DynamicData_Holder' object has no attribute '_importSpec'
Hello Seba,I was wondering if you had any ideas on this (see below also posted to the group)?If I do a dir on an importspec object from a working system (esxi 5.1) I see this:11:26:52 informacast.linux_util.vm_util.Vm_util.deploy_ovf Dir of import spec is ['DynamicProperty', 'DynamicType', 'Error', 'FileItem', 'ImportSpec', 'Warning', '__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__metaclass__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_dynamicProperty', '_dynamicType', '_fileItem', '_importSpec', '_warning', 'get_element_dynamicProperty', 'get_element_dynamicType', 'get_element_error', 'get_element_fileItem', 'get_element_importSpec', 'get_element_warning', 'new_dynamicProperty', 'new_error', 'new_fileItem', 'new_importSpec', 'new_warning', 'set_element_dynamicProperty', 'set_element_dynamicType', 'set_element_error', 'set_element_fileItem', 'set_element_importSpec', 'set_element_warning', 'typecode']on a non-working system (esxi 5.5)11:25:07 informacast.linux_util.vm_util.Vm_util.deploy_ovf Dir of import spec is ['DynamicProperty', 'DynamicType', 'Error', 'FileItem', 'ImportSpec', 'Warning', '__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__metaclass__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_dynamicProperty', '_dynamicType', '_error', 'get_element_dynamicProperty', 'get_element_dynamicType', 'get_element_error', 'get_element_fileItem', 'get_element_importSpec', 'get_element_warning', 'new_dynamicProperty', 'new_error', 'new_fileItem', 'new_importSpec', 'new_warning', 'set_element_dynamicProperty', 'set_element_dynamicType', 'set_element_error', 'set_element_fileItem', 'set_element_importSpec', 'set_element_warning', 'typecode']I don't understand why attempting to access importspec.ImportSpec actually accesses _importSpec internally or why this property would not be present.Any ideas you have would be greatly appreciated.Thanks,Nick
---------- Forwarded message ----------
From: Nick Wiesmueller <nick.wie...@singlewire.com>
Date: Wed, Feb 5, 2014 at 9:57 AM
Subject: Re: Deploying OVF files on ESX
cisp.set_element_deploymentOption("")
if host:
h = cisp.new_hostSystem(host)
h.set_attribute_type(host.get_attribute_type())
cisp.set_element_hostSystem(h)
if network:
network_mapping = cisp.new_networkMapping()
network_mapping.set_element_name(network)
n_mor = network_mapping.new_network(network_mor)
n_mor.set_attribute_type(network_mor.get_attribute_type())
network_mapping.set_element_network(n_mor)
network_mapping2 = cisp.new_networkMapping()
network_mapping2.set_element_name("Internal")
n_mor = network_mapping.new_network(network_mor)
n_mor.set_attribute_type(network_mor.get_attribute_type())
network_mapping2.set_element_network(n_mor)
cisp.set_element_networkMapping([network_mapping,
network_mapping2])
if ip_allocation_policy:
cisp.set_element_ipAllocationPolicy(ip_allocation_policy)
if ip_protocol:
cisp.set_element_ipProtocol(ip_protocol)
if disk_provisioning:
cisp.set_element_diskProvisioning(disk_provisioning)
request.set_element_cisp(cisp)
return s._proxy.CreateImportSpec(request)._returnval
def import_vapp(resource_pool, import_spec, host=None, folder=None):
#get the host MOR
if host:
hosts = s.get_hosts()
host = hosts.get(host)
if not host:
raise ValueError("invalid ESX host name")
#get the vm folder MOR
if folder:
folder = find_vmfolder_by_name(folder)
request = VI.ImportVAppRequestMsg()
_this =request.new__this(resource_pool)
_this.set_attribute_type(resource_pool.get_attribute_type())
request.set_element__this(_this)
request.set_element_spec(import_spec.ImportSpec)
if host:
h = request.new_host(host)
h.set_attribute_type(host.get_attribute_type())
request.set_element_host(h)
if folder:
f = request.new_folder(folder)
f.set_attribute_type(folder.get_attribute_type())
request.set_element_folder(f)
return s._proxy.ImportVApp(request)._returnval
if __name__ == "__main__":#you can get the resource pools running s._get_resource_pools()
RESOURCE_POOL = "/Resources"
OVF_FILE = r"C:\FreeSCO\FreeSCO-043.ovf"
#you can get the host names running s.get_hosts()
HOST = "localhost.localdomain"
DATASTORE = "datastore1"
NETWORK_NAME = "VM Network"
VAPP_NAME = "My New VM2"
s = VIServer()
s.connect("host", "username", "password")
try:
resource_pool = s._get_resource_pools()[RESOURCE_POOL]
ovf = get_descriptor(OVF_FILE)
descriptor_info = parse_descriptor(ovf)
support_info = validate_host(HOST, ovf)
import_spec = create_import_spec(resource_pool,
DATASTORE,
ovf,
VAPP_NAME,
host=HOST,
network=NETWORK_NAME,
)
if hasattr(import_spec, "Warning"):
print "Warning:", import_spec.Warning[0].LocalizedMessage
http_nfc_lease = import_vapp(resource_pool, import_spec,
host=HOST)
lease = VIProperty(s, http_nfc_lease)
while lease.state == 'initializing':
print lease.state
lease._flush_cache()
if lease.state != 'ready':
print "something went wrong"
exit()
for url in lease.info.deviceURL: