KVM Testbed setup bring up

130 views
Skip to first unread message

prasath s

unread,
Nov 3, 2022, 8:43:02 AM11/3/22
to sonicproject
Hi Team,

We are in the process of bringing up KVM testbed setup to run the test suites and  When we tried to deploy the VM using below command we are facing an error libvirt module is not importable  . Our testing environmental set up described below, Could you suggest a solution to bring up a KVM testbed setup.

Our Testing environmental Setup:

1. Ubuntu 18.04 installed on a physical machine
    Intel processor
    Virtualization enabled in bios level
    32 GB RAM

2. Python3 installed
    sudo apt install python python-pip openssh-server
    sudo pip install j2cli==0.3.10

3. Run the host setup script to install required packages and initialize the management  bridge network
    git clone https://github.com/sonic-net/sonic-mgmt
    cd sonic-mgmt/ansible
    sudo ./setup-management-network.sh

4. Docker(sonic-mgmt) installed. ubuntu 20.04 on the Inside the docker

5. vEOS (kvm based)image copied to ~/veos-vm/images

6. Run the setup-container.sh in the root directory of the sonic-mgmt repository
    cd sonic-mgmt
    ./setup-container.sh -n sonic-mgmt -d /data
    start the docker --> docker start sonic-mgmt
    enter to the docker --> docker exec -it <container name> bash
    sonic-mgmt directory mounted at /data/sonic-mgmt

7. Modified /data/sonic-mgmt/ansible/veos_vtb to use the user name
    STR-ACS-VSERV-01:
       ansible_host: 172.17.0.1
       ansible_user: sonic
       vm_host_user: foo

8. Modified /data/sonic-mgmt/ansible/ansible.cfg to uncomment the two lines
    become_user='root'
    become_ask_pass=False

9. Modified /data/sonic-mgmt/ansible/group_vars/vm_host/creds.yml

        ansible_user: sonic
        ansible_password: admin123
        ansible_become_password: admin123

        # Use the following username/password variables to login to vm hosts
        # instead of the default variables (defined above).
        vm_host_user: foo
        vm_host_password: foo123
        vm_host_become_password: foo123

10. Created a dummy password.txt file under /data/sonic-mgmt/ansible

11. On the host, run sudo visudo and added the following line at the end
    sonic ALL=(ALL) NOPASSWD:ALL
    foo ALL=(ALL) NOPASSWD:ALL

12.sonic@bf124e552bad:/data/sonic-mgmt/ansible$ ./testbed-cli.sh -m veos_vtb -n 4 -k vsonic start-vms server_1 password.txt 

Error Log:

sonic@bf124e552bad:/data/sonic-mgmt/ansible$ ./testbed-cli.sh -m veos_vtb -n 4 -k vsonic start-vms server_1 password.txt
Starting VMs on server 'server_1'
 [WARNING] Ansible is being run in a world writable directory (/data/sonic-mgmt/ansible), ignoring it as an ansible.cfg source. For more information see http
s://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir
/usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python co
re team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature

PLAY [servers:&vm_host] *************************************************************************************************************************************

TASK [vm_set : Set the default variable package_installation] ***********************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : get host distribution] ***********************************************************************************************************************
[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host STR-ACS-VSERV-01 should use /usr/bin/python3, but is using /usr/bin/python for backward
compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See
https://docs.ansible.com/ansible/2.8/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in version 2.12.
Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ok: [STR-ACS-VSERV-01]

TASK [vm_set : get host distribution version] ***************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : get host kernel version] *********************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Check if kernel upgrade needed] **************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : debug] ***************************************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Upgrade kernel package] **********************************************************************************************************************
[DEPRECATION WARNING]: Invoking "apt" only once while using a loop via squash_actions is deprecated. Instead of using a loop to supply multiple items and
specifying `pkg: "{{ item }}"`, please use `pkg: ['linux-image-generic', 'linux-image-extra-virtual']` and remove the loop. This feature will be removed in
version 2.11. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
skipping: [STR-ACS-VSERV-01] => (item=[])
TASK [vm_set : Prompt for rebooting] ************************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Install necessary packages] ******************************************************************************************************************
[DEPRECATION WARNING]: Invoking "apt" only once while using a loop via squash_actions is deprecated. Instead of using a loop to supply multiple items and
specifying `pkg: "{{ item }}"`, please use `pkg: ['ifupdown', 'qemu', 'openvswitch-switch', 'net-tools', 'bridge-utils', 'util-linux', 'iproute2', 'vlan',
'apt-transport-https', 'ca-certificates', 'curl', 'software-properties-common', 'libvirt-clients']` and remove the loop. This feature will be removed in
version 2.11. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ok: [STR-ACS-VSERV-01] => (item=[u'ifupdown', u'qemu', u'openvswitch-switch', u'net-tools', u'bridge-utils', u'util-linux', u'iproute2', u'vlan', u'apt-transport-https', u'ca-certificates', u'curl', u'software-properties-common', u'libvirt-clients'])

TASK [vm_set : Install necessary packages] ******************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Install necessary packages] ******************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Get default pip_executable] ******************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Get default pip_executable] ******************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : remove old python packages] ******************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : include_tasks] *******************************************************************************************************************************
included: /data/sonic-mgmt/ansible/roles/vm_set/tasks/docker.yml for STR-ACS-VSERV-01

TASK [vm_set : Add docker official GPG key] *****************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Check docker repository] *********************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Report docker repository exists] *************************************************************************************************************
ok: [STR-ACS-VSERV-01] => {
    "msg": "Docker repository already exists"
}

TASK [vm_set : Report docker repository not exists] *********************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Add docker repository for 16.04] *************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Add docker repository for 17.04] *************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Add docker repository for 18.04] *************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Add docker repository for 20.04] *************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Run the "apt-get update" as a separate and retryable step] ***********************************************************************************
changed: [STR-ACS-VSERV-01]

TASK [vm_set : Install docker-ce] ***************************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : remove old python packages] ******************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Install python packages] *********************************************************************************************************************
changed: [STR-ACS-VSERV-01]

TASK [vm_set : Ensure sonic in docker,sudo group] ***********************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Ensure sonic in libvirt group] ***************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Install br_netfilter kernel module] **********************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Set sysctl bridge parameters for testbed] ****************************************************************************************************
ok: [STR-ACS-VSERV-01] => (item=net.bridge.bridge-nf-call-arptables)
ok: [STR-ACS-VSERV-01] => (item=net.bridge.bridge-nf-call-ip6tables)
ok: [STR-ACS-VSERV-01] => (item=net.bridge.bridge-nf-call-iptables)

TASK [vm_set : Set sysctl RCVBUF max parameter for testbed] *************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Set sysctl RCVBUF default parameter for testbed] *********************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : Setup external front port] *******************************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Setup internal management network] ***********************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : getent] **************************************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : set_fact] ************************************************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : set_fact] ************************************************************************************************************************************
[WARNING]: conditional statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: not '{{ root_path }}'.startswith('/')

ok: [STR-ACS-VSERV-01]

TASK [vm_set : debug] ***************************************************************************************************************************************
ok: [STR-ACS-VSERV-01] => {
    "msg": "home_path = /home/sonic root_path = /home/sonic/veos-vm"
}

TASK [vm_set : Require veos or SONiC VMs by default] ********************************************************************************************************
ok: [STR-ACS-VSERV-01]

TASK [vm_set : veos or SONiC VMs not needed when setting up Kubernetes master] ******************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : VMs not needed in case of Keysight API Server] ***********************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : VMs not needed in case of Keysight IxANVL] ***************************************************************************************************
skipping: [STR-ACS-VSERV-01]

TASK [vm_set : Retrieve a list of the defined VMs] **********************************************************************************************************
fatal: [STR-ACS-VSERV-01]: FAILED! => {"changed": false, "msg": "The `libvirt` module is not importable. Check the requirements."}


PLAY RECAP **************************************************************************************************************************************************
STR-ACS-VSERV-01           : ok=26   changed=2    unreachable=0    failed=1    skipped=17   rescued=0    ignored=0

sonic@bf124e552bad:/data/sonic-mgmt/ansible$

krishna bansal

unread,
May 21, 2024, 3:00:09 AMMay 21
to sonicproject
Hi Team,

I am using ubuntu 20.04 krishna@krishna:~/veos-vm/images$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.6 LTS Release: 20.04 Codename: focal

I am trying to create a T0 toplogy: https://github.com/sonic-net/sonic-mgmt/blob/master/docs/testbed/README.testbed.VsSetup.md#setup-vms-on-the-server using this link.

Facing issue while running the Setup VMs on the server.

below errors:






TASK [vm_set : Define vm VM0100] *********************************************************************************************
Tuesday 21 May 2024  06:32:28 +0000 (0:00:14.681)       0:01:20.155 ***********
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'libvirtError' object has no attribute 'message'
<172.17.0.1> Failed to connect to the host via ssh: libvirt: QEMU Driver error : unsupported configuration: Emulator '/usr/bin/qemu-system-x86_64' does not support virt type 'kvm' <stdin>:18: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses The full traceback is: Traceback (most recent call last): File "/tmp/ansible_virt_payload_8g451wqh/__main__.py", line 542, in core domain = v.define(xml) File "/tmp/ansible_virt_payload_8g451wqh/__main__.py", line 455, in define return self.conn.define_from_xml(xml) File "/tmp/ansible_virt_payload_8g451wqh/__main__.py", line 284, in define_from_xml return self.conn.defineXML(xml) File "/usr/lib/python3/dist-packages/libvirt.py", line 4047, in defineXML if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self) libvirt.libvirtError: unsupported configuration: Emulator '/usr/bin/qemu-system-x86_64' does not support virt type 'kvm'``` During handling of the above exception, another exception occurred: ```Traceback (most recent call last): File "/tmp/ansible_virt_payload_8g451wqh/__main__.py", line 593, in main rc, result = core(module) File "/tmp/ansible_virt_payload_8g451wqh/__main__.py", line 551, in core module.fail_json(msg='libvirtError: %s' % e.message) AttributeError: 'libvirtError' object has no attribute 'message'```   
[WARNING]: 'xml' is given - ignoring 'name'

fatal: [STR-ACS-VSERV-01]: FAILED! => {"changed": false, "msg": "'libvirtError' object has no attribute 'message'"}

PLAY RECAP *******************************************************************************************************************
STR-ACS-VSERV-01           : ok=61   changed=4    unreachable=0    failed=1    skipped=45   rescued=0    ignored=0

Tuesday 21 May 2024  06:32:30 +0000 (0:00:02.599)       0:01:22.755 ***********
===============================================================================
vm_set : Copy veos disk image for VM0100 ----------------------------------------------------------------------------- 14.68s
vm_set : Run the "apt-get update" as a separate and retryable step --------------------------------------------------- 13.06s
vm_set : Install python packages -------------------------------------------------------------------------------------- 6.70s
vm_set : remove old python packages ----------------------------------------------------------------------------------- 3.03s
vm_set : Create VMs network ------------------------------------------------------------------------------------------- 2.97s
vm_set : Install necessary packages ----------------------------------------------------------------------------------- 2.83s
vm_set : Add docker official GPG key ---------------------------------------------------------------------------------- 2.80s
vm_set : Define vm VM0100 --------------------------------------------------------------------------------------------- 2.60s
vm_set : Check hdd image ---------------------------------------------------------------------------------------------- 2.52s
vm_set : remove old python packages ----------------------------------------------------------------------------------- 2.19s
vm_set : Install necessary packages ----------------------------------------------------------------------------------- 1.98s
vm_set : Install docker-ce -------------------------------------------------------------------------------------------- 1.89s
vm_set : Install necessary packages ----------------------------------------------------------------------------------- 1.88s
vm_set : Set sysctl bridge parameters for testbed --------------------------------------------------------------------- 1.85s
vm_set : get host distribution ---------------------------------------------------------------------------------------- 1.62s
vm_set : Copy vm_resumer.py to the /home/krishna/veos-vm -------------------------------------------------------------- 1.57s
vm_set : Install cleanup script --------------------------------------------------------------------------------------- 1.45s
vm_set : Retrieve a list of the defined VMs --------------------------------------------------------------------------- 1.17s
vm_set : Create directory for vm images and vm disks ------------------------------------------------------------------ 1.00s
vm_set : Check docker repository -------------------------------------------------------------------------------------- 0.93s

Kindly let me know how i can able to fix this.

Regards,
Krishna
Reply all
Reply to author
Forward
0 new messages