A running VMI does not have a guest agent connected.

[Rohith Vemula]

Hello everyone,

I have tried kubevirt for the first time and migrated a ubuntu virtual machine to kubernetes cluster on cloud, by uploading the vmdk using virtctl to DataVolume, and started the VM.

When I query for the guest operating system running in the VMI using "virtctl guestosinfo <VMI_name>", I get an error "VMI does not have guest agent connected" (unable to login to the VM using ssh or vnc).

I have tried uploading vmdk in both vmdk and raw formats to DataVolume and also tried uploading "*-flat.vmdk" file in both formats. None of them worked.

I have also verified that the disk data is present in data volume by logging in to the virt-launcher pod corresponding to the VMI and checking the volume mount point path. 

Any suggestions would be much appreciated.

Regards

Rohith.

[Rohith Vemula]

Updating details here..

I have also tried creating a VM from the template provided in Lab1 in kubevirt web site. But, still got the same issue (VMI does not have a guest agent connected) while trying to get guest OS info. Also, pinging IP fails and SSH/console into VM fails.

Any help would be appreciated.

Regards
Rohith.

[Stu Gott]

Chris,

Is this something you're aware of?

--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/1336470f-96f0-4ff7-9a54-ed6aa39f1494n%40googlegroups.com.

[Rohith Vemula]

Chris, thanks for the info.

I will check out the link and try to upload VMDK to DataVolume directly from vcenter.

I have a few questions here:

1. Is the concept of guest OS in the actual virtual machine (on-premise vmware VM), same as the guest OS referred to, in the kubevirt VM? The intention here is to know if the guest OS in kubevirt VM is an optional component, and if it is possible to have a VM running in kubevirt and SSH into kubevirt VM possible, without a guest agent.

2. I have used the artifacts mentioned in the Lab1 tutorial as well (deployed test-vm template), but couldn't find guest OS details and was unable to SSH into the machine even after exposing port 22 using a LoadBalancer service. As you mentioned above, the Lab1 tutorial makes use of open source/free providers. So, could this be an issue in my cluster environment? Is there some prerequisite that has not been documented?

Regards

Rohith.

On Tue, Nov 23, 2021 at 10:43 PM Chris Callegari <ccal...@redhat.com> wrote:

VMware is tough.  I don't know of any licenses available for us to use to deploy and tinker.  Our labs just use open source / free providers.  We don't document packing virtual machines with guest tools.

CDI can import VMware virtual disks using these instructions

https://github.com/kubevirt/containerized-data-importer/blob/main/doc/datavolumes.md#vddk-data-volume

--

Thanks,

/Chris Callegari

Pronouns: he, him, and his

Software Engineer, Solutions Engineering

Red Hat

irc: ccallegar

[Rohith Vemula]

Thanks Chris. I was able to successfully login into the VM (created using Lab1 tutorial) created in kubernetes cluster. SSH worked from a different network (not from my home network, although I was able to ssh into a different linux machine from my home network).

>>> Extremely likely nested virtualization settings are missing, preventing Kubevirt from starting virtual machines on the guest.

I was under the impression that kubevirt creates a pod (virt-launcher pod) for every virtual machine instance that comes up in the cluster. But, you mentioned that nested virtualization settings are needed in order for kubevirt to start an actual VM on the guest. 

Is there an actual VM created on the nodepool node in the cluster corresponding to each virtual machine instance? If so, how are these VMs managed on the node (hypervisor)?

Regards

Rohith.

On Wed, Nov 24, 2021 at 10:25 PM Chris Callegari <ccal...@redhat.com> wrote:

comments in-line

On Tue, Nov 23, 2021 at 9:22 PM Rohith Vemula <vemularoh...@gmail.com> wrote:

Chris, thanks for the info.

I will check out the link and try to upload VMDK to DataVolume directly from vcenter.

I have a few questions here:

1. Is the concept of guest OS in the actual virtual machine (on-premise vmware VM), same as the guest OS referred to, in the kubevirt VM? The intention here is to know if the guest OS in kubevirt VM is an optional component, and if it is possible to have a VM running in kubevirt and SSH into kubevirt VM possible, without a guest agent.

Guest agent should not matter unless your guest is Windows and devices are virt-io or whatever VMware calls their hardware accelerated devices.

I believe what you are referring to here is known as nested virtualization running a vm within vm on the VMware platform.  Many users do this especially when they run KubeVirt virtual machines in public clouds.  We document the nested virtualization setting only for Linux guests on Linux hosts. 

VMware has a very detailed procedure to enable nested virtualization.  Of course that procedure is different depending on your vSphere and ESXi versions.

2. I have used the artifacts mentioned in the Lab1 tutorial as well (deployed test-vm template), but couldn't find guest OS details and was unable to SSH into the machine even after exposing port 22 using a LoadBalancer service. As you mentioned above, the Lab1 tutorial makes use of open source/free providers. So, could this be an issue in my cluster environment? Is there some prerequisite that has not been documented?

Extremely likely nested virtualization settings are missing preventing Kubevirt from starting virtual machines on the guest.

[Mazzystr]

I'm not familiar with the nodepool node terminology.  Is that VMware terminology?

KubeVirt does create a pod for every vm.

There is no link to the VMware vSphere environment therefore hosts and guests in the VMware vSphere environment do not pertain to hosts and guests in the Kubernetes and KubeVirt environments. They are entirely separate layers HENCE the requirement for nested virtualization for that environment.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAC0dyiN-4ExzpGN9Psy_1SBOCx1C2w_87pU0GJAczQk%2B-vYgvg%40mail.gmail.com.

[Dan Kenigsberg]

On Mon, Nov 15, 2021 at 8:40 AM Rohith Vemula <vemularoh...@gmail.com> wrote:

Hello everyone,

I have tried kubevirt for the first time and migrated a ubuntu virtual machine to kubernetes cluster on cloud, by uploading the vmdk using virtctl to DataVolume, and started the VM.

When I query for the guest operating system running in the VMI using "virtctl guestosinfo <VMI_name>", I get an error "VMI does not have guest agent connected" (unable to login to the VM using ssh or vnc).

Is the guest actually running ? Can you see it with `virtctl vnc`?

Which operating system is running there? You would need to install the relevant guest agent to report guestosinfo. Have you done this already?

I have tried uploading vmdk in both vmdk and raw formats to DataVolume and also tried uploading "*-flat.vmdk" file in both formats. None of them worked.

I have also verified that the disk data is present in data volume by logging in to the virt-launcher pod corresponding to the VMI and checking the volume mount point path. 

Any suggestions would be much appreciated.

By the way, as a Kubevirt user, have you already replied to our survey at https://forms.gle/vX18ymac4bsCGRpS6 ?

Regards

Rohith.

--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/c8836321-bff7-4498-99fc-6754ac2d68b2n%40googlegroups.com.

[Roman Mohr]

On Tue, Dec 7, 2021 at 2:32 PM Dan Kenigsberg <dan...@redhat.com> wrote:

On Mon, Nov 15, 2021 at 8:40 AM Rohith Vemula <vemularoh...@gmail.com> wrote:

Hello everyone,

I have tried kubevirt for the first time and migrated a ubuntu virtual machine to kubernetes cluster on cloud, by uploading the vmdk using virtctl to DataVolume, and started the VM.

When I query for the guest operating system running in the VMI using "virtctl guestosinfo <VMI_name>", I get an error "VMI does not have guest agent connected" (unable to login to the VM using ssh or vnc).

Is the guest actually running ? Can you see it with `virtctl vnc`?

Which operating system is running there? You would need to install the relevant guest agent to report guestosinfo. Have you done this already?

In addition to what Dan asked for, could you just share the status section of your VMI? That would help diagnosing the issue. Also what error do you get returned if you try to run `virtctl vnc <vm>`?

Best regards,

Roman

 

I have tried uploading vmdk in both vmdk and raw formats to DataVolume and also tried uploading "*-flat.vmdk" file in both formats. None of them worked.

I have also verified that the disk data is present in data volume by logging in to the virt-launcher pod corresponding to the VMI and checking the volume mount point path. 

Any suggestions would be much appreciated.

By the way, as a Kubevirt user, have you already replied to our survey at https://forms.gle/vX18ymac4bsCGRpS6 ?

Regards

Rohith.

--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/c8836321-bff7-4498-99fc-6754ac2d68b2n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAHOEP56%3DTeEQOy7CKD3LYVPdoshK1eVfmcQCJovHT%2BtJmTEsPA%40mail.gmail.com.

[Rohith Vemula]

Thanks Chris. So, the infrastructure that comes up on the cluster actually depends on, and would be similar to the source infra.

I have tried creating a virtual machine using kubevirt by importing complete disk data into a data volume (backed by PVC). I have used 'dd if=/dev/sda of=disk.img' command to copy the entire disk data into an img format file. I imported this .img file into a data volume and tried creating a VM from that data volume.

After doing this, the VM is in running state (with IP allocated). I am able to see that there is an actual VM created on the kubernetes node corresponding to the virtual machine instance (by logging into any pod on that node and using virsh command). But, I am unable to SSH into the VM.

Is there some location where I can check the logs for VM creation or SSH?

Does importing an img file (obtained using 'dd' command) into data volume, work as a correct disk image for kubevirt?

I checked this link (https://github.com/kubevirt/containerized-data-importer/blob/main/doc/datavolumes.md), but it does not mention if the way in which I have obtained the .img file is correct.

Regards

Rohith.

On Mon, Nov 29, 2021 at 10:46 PM Chris Callegari <ccal...@redhat.com> wrote:

On Thu, Nov 25, 2021 at 3:44 AM Rohith Vemula <vemularoh...@gmail.com> wrote:

Thanks Chris. I was able to successfully login into the VM (created using Lab1 tutorial) created in kubernetes cluster. SSH worked from a different network (not from my home network, although I was able to ssh into a different linux machine from my home network).

Great!

I also found that my vpn setting interferes with all my local network connections.  Sometimes I find I can't access anything and start down the path of deep debugging then remember I still have vpn activated.

 

>>> Extremely likely nested virtualization settings are missing, preventing Kubevirt from starting virtual machines on the guest.

I was under the impression that kubevirt creates a pod (virt-launcher pod) for every virtual machine instance that comes up in the cluster. But, you mentioned that nested virtualization settings are needed in order for kubevirt to start an actual VM on the guest. 

Is there an actual VM created on the nodepool node in the cluster corresponding to each virtual machine instance? If so, how are these VMs managed on the node (hypervisor)?

KubeVirt does not manage any aspect of the Kubernetes cluster (control-plane or workers).  VMware is considered an infrastructure provider.  There are other mechanisms such as metal3 that handle scaling the kubernetes cluster itself. 

If the kubernetes worker is a virtual machine that is when the nested virtualization setting needs to be set such as a AWS EC2 of VMware virtual machine.  The kubeVirt vm would then be a vm within a vm.

Hopefully this explains things better.  :)

[Rohith Vemula]

Hi Dan and Roman,

I have tried uploading the vmdk data directly to a data volume.

But as mentioned here (https://github.com/kubevirt/containerized-data-importer/blob/main/doc/datavolumes.md#vddk-data-volume), VMDK upload has to be done by creating a VDDK standalone container which can be used to upload the data in the proper format. So, the steps that I have followed in the above trial were wrong.

>>> Which operating system is running there? You would need to install the relevant guest agent to report guestosinfo. Have you done this already?

    I have not installed the guest agent. I was using ubuntu OS machine.

Regards

Rohith.

[Rohith Vemula]

Hi Roman,

>>> Status section of the VMI

     Guest OS Info is empty in this section. This is as expected because there is no guest agent (VM inside a VM) running on the source infra (and hence there wouldn't be any such thing in the target).

>>> Also what error do you get returned if you try to run `virtctl vnc <vm>`?

     Can't access VMI onpremvm: Can't connect to websocket (504): websocket: bad handshake

Regards

Rohith.