Spytest2.0 - community-build

[Perumal Venkatesh]

Hi all,

Is the community-build support removed from Spytest. Even though the option is specified in the intro doc, it is not supported while running the tests.

spytest: error: unrecognized arguments: --community-build

Also hitting other issues like :

2021-01-20 04:58:59,743 T0006: ERROR [D2-SD2] Failed to execute the command sudo python /etc/spytest/remote/spytest-helper.py --wait-for-ports 0

2021-01-20 04:58:59,793 T0004: INFO  ========= Report(Fail): Exception observed - Traceback (most recent call last):

  File "/home/vxr/sonic-test/sonic-mgmt/spytest/utilities/parallel.py", line 44, in _thread_func

    retvals[index] = func(*args, **kwargs)

  File "/home/vxr/sonic-test/sonic-mgmt/spytest/spytest/net.py", line 1032, in connect_to_device

    self.do_post_reboot(devname, phase=0, max_ready_wait=max_ready_wait)

  File "/home/vxr/sonic-test/sonic-mgmt/spytest/spytest/net.py", line 807, in do_post_reboot

    sys_status = self.wa.wait_system_status(devname, max_time=max_ready_wait)

  File "/home/vxr/sonic-test/sonic-mgmt/spytest/spytest/framework.py", line 1213, in wait_system_status

    return self._wait_for_ports(dut)

  File "/home/vxr/sonic-test/sonic-mgmt/spytest/spytest/framework.py", line 1196, in _wait_for_ports

    return self.net._apply_remote(dut, "wait-for-ports", [max_time])

  File "/home/vxr/sonic-test/sonic-mgmt/spytest/spytest/net.py", line 3771, in _apply_remote

    raise ValueError(msg)

ValueError: Failed to execute the command sudo python /etc/spytest/remote/spytest-helper.py --wait-for-ports 0

All these issues are hit after moving to Spytest2.0 framework. 

Regards

Venkatesh

[Sasthri Kristipati]

I will update the document in the next commit.

+SPyTest supports multiple SONiC releases and the differences in the

+ feature/behavior across the releases are handled using flags

+ known as device-features defined in **[ROOT]/spytest/features.py**.

+ These features can be independently turned on/off using command line arguments

+ **--device-feature-enable** and **--device-feature-disable**.

+ They can also be selected using feature groups with command line argument **--device-feature-group**.

+ The current supported feature groups are:

+

+    * broadcom

+    * upstream

+    * master

+    * 201911

+    * 202012

+

+The features in each feature group can be found from **[ROOT]/spytest/features.py**..

+

 #### Running test script(s)

     [ROOT]/bin/spytest --testbed testbed_file.yaml  \

+        --device-feature-group master \

         [ROOT]/tests/sanity/test_sanity_l2.py \

         [ROOT]/tests/sanity/test_sanity_l3.py \

         --logs-path <folder-to-create-logs>

[Perumal Venkatesh]

Thanks Sasthri, that helped but immediately hit the next issue. At the very start of the script, spytest infra used to check for various things and then do a config reload. After a config reload, it would wait for sometime before checking the interface status. In the new infra, it doesn’t do that and then complains that the interface has not come up, goes on to do fast-reboot.

Old run snippet : Notice the extra options for spytest-helper and also the time stamps before it checks the interfaces

2021-01-21 01:51:13,357 T0024: INFO  [D1-SD1] Reloading Monit configuration ...

2021-01-21 01:51:13,357 T0024: INFO  [D1-SD1] Reinitializing monit daemon

2021-01-21 01:51:13,357 T0024: INFO  [D1-SD1] Remote CMD: 'systemctl restart bgp'

2021-01-21 01:51:13,357 T0024: INFO  [D1-SD1] Remote CMD: 'sleep 10'

2021-01-21 01:51:13,357 T0024: INFO  [D1-SD1] Remote CMD: 'vtysh -c write file'

2021-01-21 01:51:13,357 T0024: INFO  [D1-SD1] Note: this version of vtysh never writes vtysh.conf

2021-01-21 01:51:13,358 T0024: INFO  [D1-SD1] Building Configuration...

2021-01-21 01:51:13,358 T0024: INFO  [D1-SD1] Configuration saved to /etc/frr/zebra.conf

2021-01-21 01:51:13,358 T0024: INFO  [D1-SD1] Configuration saved to /etc/frr/bgpd.conf

2021-01-21 01:51:13,358 T0024: INFO  [D1-SD1] Configuration saved to /etc/frr/staticd.conf

2021-01-21 01:51:13,358 T0024: INFO  [D1-SD1] Remote CMD: 'cp -f /etc/sonic/frr/frr.conf /etc/spytest/base_frr.conf'

2021-01-21 01:51:13,360 T0025: INFO  [D2-SD2] SCMD: sudo python /etc/spytest/remote/spytest-helper.py --wait-for-ports  --port-init-wait 240 --poll-for-ports yes --community-build

2021-01-21 01:51:13,368 T0024: INFO  [D1-SD1] SCMD: sudo python /etc/spytest/remote/spytest-helper.py --wait-for-ports  --port-init-wait 240 --poll-for-ports yes --community-build

2021-01-21 01:51:41,028 T0025: INFO  [D2-SD2] Remote CMD: 'show platform summary'

2021-01-21 01:51:41,030 T0025: INFO  [D2-SD2] Remote CMD: 'show interfaces status Ethernet35'

Spytest2.0 infra run:

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Reloading Monit configuration ...

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Reinitializing monit daemon

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Remote CMD: 'systemctl restart bgp'

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Remote CMD: 'sleep 10'

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Remote CMD: 'vtysh -c write file'

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Note: this version of vtysh never writes vtysh.conf

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Building Configuration...

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Configuration saved to /etc/frr/zebra.conf

2021-01-20 21:15:08,912 T0018: INFO  [D2-SD2] Configuration saved to /etc/frr/bgpd.conf

2021-01-20 21:15:08,913 T0018: INFO  [D2-SD2] Configuration saved to /etc/frr/staticd.conf

2021-01-20 21:15:08,913 T0018: INFO  [D2-SD2] Remote CMD: 'cp -f /etc/sonic/frr/frr.conf /etc/spytest/init_frr.conf'

2021-01-20 21:15:09,508 T0017: INFO  [D1-SD1] SCMD: sudo python /etc/spytest/remote/spytest-helper.py --config-profile na

2021-01-20 21:15:09,516 T0018: INFO  [D2-SD2] SCMD: sudo python /etc/spytest/remote/spytest-helper.py --config-profile na

2021-01-20 21:15:11,913 T0017: INFO  [D1-SD1]

2021-01-20 21:15:11,913 T0017: INFO  [D1-SD1] Skipping the profile config as it is not required for 'NA'.

2021-01-20 21:15:11,913 T0017: INFO  [D1-SD1] DONE

2021-01-20 21:15:11,914 T0017: INFO  [D1-SD1] save configuration as TA configuration - 2

2021-01-20 21:15:11,921 T0018: INFO  [D2-SD2]

2021-01-20 21:15:11,921 T0018: INFO  [D2-SD2] Skipping the profile config as it is not required for 'NA'.

2021-01-20 21:15:11,921 T0018: INFO  [D2-SD2] DONE

2021-01-20 21:15:11,922 T0018: INFO  [D2-SD2] save configuration as TA configuration - 2

2021-01-20 21:15:12,518 T0017: INFO  [D1-SD1] SCMD: sudo python /etc/spytest/remote/spytest-helper.py --save-base-config

2021-01-20 21:15:12,526 T0018: INFO  [D2-SD2] SCMD: sudo python /etc/spytest/remote/spytest-helper.py --save-base-config

2021-01-20 21:15:14,923 T0017: INFO  [D1-SD1]

2021-01-20 21:15:14,923 T0017: INFO  [D1-SD1] Remote CMD: 'cp /etc/spytest/init_config_db.json /etc/spytest/base_config_db.json'

2021-01-20 21:15:14,923 T0017: INFO  [D1-SD1] Remote CMD: 'cp /etc/spytest/init_frr.conf /etc/spytest/base_frr.conf'

2021-01-20 21:15:14,924 T0017: INFO  [D1-SD1] Remote CMD: 'rm -f /etc/spytest/base_copp.json'

2021-01-20 21:15:14,924 T0017: INFO  [D1-SD1] Remote CMD: 'rm -f /etc/spytest/base_minigraph.xml'

2021-01-20 21:15:14,924 T0017: INFO  [D1-SD1] DONE

2021-01-20 21:15:14,930 T0018: INFO  [D2-SD2]

2021-01-20 21:15:14,930 T0018: INFO  [D2-SD2] Remote CMD: 'cp /etc/spytest/init_config_db.json /etc/spytest/base_config_db.json'

2021-01-20 21:15:14,930 T0018: INFO  [D2-SD2] Remote CMD: 'cp /etc/spytest/init_frr.conf /etc/spytest/base_frr.conf'

2021-01-20 21:15:14,931 T0018: INFO  [D2-SD2] Remote CMD: 'rm -f /etc/spytest/base_copp.json'

2021-01-20 21:15:14,931 T0018: INFO  [D2-SD2] Remote CMD: 'rm -f /etc/spytest/base_minigraph.xml'

2021-01-20 21:15:14,931 T0018: INFO  [D2-SD2] DONE

2021-01-20 21:15:14,932 T0000: INFO  building port list and save base config

2021-01-20 21:15:15,134 T0019: INFO  [D1-SD1] FCMD: show interfaces status

2021-01-20 21:15:15,135 T0020: INFO  [D2-SD2] FCMD: show interfaces status

2021-01-20 21:15:16,996 T0019: INFO  [D1-SD1]   Interface    Lanes    Speed    MTU    FEC    Alias    Vlan    Oper    Admin    Type    Asym PFC

2021-01-20 21:15:16,997 T0019: INFO  [D1-SD1] -----------  -------  -------  -----  -----  -------  ------  ------  -------  ------  ----------

Regards

Venkatesh

This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.

--
You received this message because you are subscribed to the Google Groups "sonic-test-workgroup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonic-test-workg...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonic-test-workgroup/57558ad9-02a4-40e1-ab58-c4e8f484a389n%40googlegroups.com.

[Perumal Venkatesh]