Unable to collect failure test case robot framework logs/reports via jenkins plans to the tester board setup.

[Srinivasan Shanmugam]

Dear Jenkins Experts,

As am newbie to Robot framework script and Jenkins, as my earlier expertise was in Embedded Linux development, my head is struggling one of the below issues in robot framework and jenkins as described below:

The setup is as below:

I have jenkins agent running in Orange Pi prime board (called as tester board), where I trigger the Jenkins plans via this Orange Pi prime board(tester board)(where the test code is downloaded in this tester board from git and copied to my DUT via SSH commands mentioned below to my final DUT and this is where the tests ie., robot framework scripts are being executed in the DUT) and the robot framemork logs/reports are collected back to this tester board from DUT. 

********** Commands executed from Tester board to DUT via Jenkins Plan *********************************************************************************************************************************************************

#!/bin/bash

ifconfig

echo "Debugging for knowing the path"

pwd

mkdir /home/root/results

whoami

ls -l

board_ip=$(cat /home/root/target_ip)

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb -r qa/ root@${board_ip}:/home/root

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'pip3 install robotframework'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'mkdir results'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'ls'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'python3 -m robot --outputdir results ~/qa/robot_tests/bios_settings_verification.robot'

#!/bin/bash

board_ip=$(cat /home/root/target_ip)

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb -r root@${board_ip}:/home/root/results/ .

sync

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'rm -r /home/root/results'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'rm -r /home/root/qa'

***********************************************************************************************************************************************************************************************************************************************

Logs:

15:14:05 [EnvInject] - Loading node environment variables.

15:14:05 Building remotely on PenguinTeam_OrangePI-Prime_2 (PenguinTeam-x86) in workspace /home/root/workspace/PenguinTeam/BIOS_settings_verification

15:14:05  > git rev-parse --is-inside-work-tree # timeout=10

15:14:05 Fetching changes from the remote Git repository

15:14:05  > git config remote.origin.url https://bitbucket.bln.native-instruments.de/scm/emb/qa.git # timeout=10

15:14:05 Fetching upstream changes from https://bitbucket.bln.native-instruments.de/scm/emb/qa.git

15:14:05  > git --version # timeout=10

15:14:05 using GIT_ASKPASS to set credentials UsernamePassword

15:14:05  > git fetch --tags --progress https://bitbucket.bln.native-instruments.de/scm/emb/qa.git +refs/heads/*:refs/remotes/origin/*

15:14:05  > git rev-parse refs/remotes/origin/master^{commit} # timeout=10

15:14:05  > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10

15:14:06 Checking out Revision 594cc81aa291f1e9f0730d8476d129592e792f28 (refs/remotes/origin/master)

15:14:06  > git config core.sparsecheckout # timeout=10

15:14:06  > git checkout -f 594cc81aa291f1e9f0730d8476d129592e792f28

15:14:06 Commit message: "Updated Thomas review comments"

15:14:06  > git rev-list --no-walk 594cc81aa291f1e9f0730d8476d129592e792f28 # timeout=10

15:14:06 [BIOS_settings_verification] $ /bin/bash /tmp/jenkins8191990223406863240.sh

15:14:06 docker0   Link encap:Ethernet  HWaddr 02:42:0B:8D:B2:4B  

15:14:06           inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0

15:14:06           UP BROADCAST MULTICAST  MTU:1500  Metric:1

15:14:06           RX packets:0 errors:0 dropped:0 overruns:0 frame:0

15:14:06           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

15:14:06           collisions:0 txqueuelen:0 

15:14:06           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

15:14:06 

15:14:06 eth0      Link encap:Ethernet  HWaddr 02:01:E4:1C:3E:33  

15:14:06           inet addr:10.1.9.51  Bcast:10.1.11.255  Mask:255.255.252.0

15:14:06           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

15:14:06           RX packets:4793744 errors:0 dropped:860 overruns:0 frame:0

15:14:06           TX packets:818823 errors:0 dropped:0 overruns:0 carrier:0

15:14:06           collisions:0 txqueuelen:1000 

15:14:06           RX bytes:1046531973 (998.0 MiB)  TX bytes:544566075 (519.3 MiB)

15:14:06           Interrupt:25 

15:14:06 

15:14:06 lo        Link encap:Local Loopback  

15:14:06           inet addr:127.0.0.1  Mask:255.0.0.0

15:14:06           UP LOOPBACK RUNNING  MTU:65536  Metric:1

15:14:06           RX packets:149 errors:0 dropped:0 overruns:0 frame:0

15:14:06           TX packets:149 errors:0 dropped:0 overruns:0 carrier:0

15:14:06           collisions:0 txqueuelen:1000 

15:14:06           RX bytes:16640 (16.2 KiB)  TX bytes:16640 (16.2 KiB)

15:14:06 

15:14:06 Debugging for knowing the path

15:14:06 /home/root/workspace/PenguinTeam/BIOS_settings_verification

15:14:06 mkdir: can't create directory '/home/root/results': File exists

15:14:06 root

15:14:06 total 12

15:14:06 drwxr-xr-x    9 root     root          4096 Nov  9 08:36 qa

15:14:06 drwxr-xr-x    2 root     root          4096 Nov 16 14:14 qa@tmp

15:14:06 drwxr-xr-x    2 root     root          4096 Nov 16 12:13 results

15:14:06 Warning: Permanently added '10.1.9.37' (RSA) to the list of known hosts.

15:14:07 Warning: Permanently added '10.1.9.37' (RSA) to the list of known hosts.

15:14:09 Requirement already satisfied: robotframework in /usr/lib/python3.5/site-packages

15:14:09 You are using pip version 9.0.2, however version 18.1 is available.

15:14:09 You should consider upgrading via the 'pip install --upgrade pip' command.

15:14:09 Warning: Permanently added '10.1.9.37' (RSA) to the list of known hosts.

15:14:09 mkdir: cannot create directory 'results': File exists

15:14:10 Warning: Permanently added '10.1.9.37' (RSA) to the list of known hosts.

15:14:10 qa

15:14:10 results

15:14:10 speedtest

15:14:10 Warning: Permanently added '10.1.9.37' (RSA) to the list of known hosts.

15:14:10 ==============================================================================

15:14:10 Bios Settings Verification :: This is the Maschine Native OS BIOS settings ...

15:14:10 ==============================================================================

15:14:10 Verify Backup Map file BIOS settings                                  | FAIL |

15:14:11 'False' should be true.

15:14:11 ------------------------------------------------------------------------------

15:14:11 Verify Default Map file BIOS settings                                 | PASS |

15:14:12 ------------------------------------------------------------------------------

15:14:12 Bios Settings Verification :: This is the Maschine Native OS BIOS ... | FAIL |

15:14:12 2 critical tests, 1 passed, 1 failed

15:14:12 2 tests total, 1 passed, 1 failed

15:14:12 ==============================================================================

15:14:12 Output:  /home/root/results/output.xml

15:14:12 Log:     /home/root/results/log.html

15:14:12 Report:  /home/root/results/report.html

15:14:12 Build step 'Execute shell' marked build as failure

15:14:12 Robot results publisher started...

15:14:12 -Parsing output xml:

15:14:12 Done!

15:14:12 -Copying log files to build dir:

15:14:12 Done!

15:14:12 -Assigning results to build:

15:14:13 Done!

15:14:13 -Checking thresholds:

15:14:13 Done!

15:14:13 Done publishing Robot results.

15:14:13 [BFA] Scanning build for known causes...

15:14:13 [BFA] No failure causes found

15:14:13 [BFA] Done. 0s

15:14:13 Finished: FAILURE

The issue is as below:

In my Jenkins plans, for the success case scenarios, I see the logs are getting generated in the DUT and collected to the tester board via SSH in Jenkins, but for failure case scenarios (ie., even though I can see the job failing in jenkins, but w.r.t the reports/logs, it still shows the earlier successfully passed reports in the Jenkins UI browser instead of showing the number of failed test cases reports in the Jenkins UI), And eventhough the failure case scenarios logs are successfully generated with the number of test cases failed in the DUT, these logs are not getting copied back to the tester board via SSH and not being updated in the Jenkins UI or browser. But for the success case scenarios the below commands works fine and logs getting in the Jenkins UI or browser

#!/bin/bash

board_ip=$(cat /home/root/target_ip)

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb -r root@${board_ip}:/home/root/results/ .

sync

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'rm -r /home/root/results'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'rm -r /home/root/qa'

the logs are in the (DUT) -  ssh ro...@10.1.9.37) ---> /home/root/results/log.html --> these failure  logs are not getting scp to the tester board (

ssh ro...@10.1.9.51) to the path "/home/root/workspace/PenguinTeam/BIOS_settings_verification/results"

The robot script is executed on DUT and results are copied back to the tester board.

I hope once I collect successfully the failure logs in the tester board , the failure reports might get successfully updated in the Jenkins UI in the browser.

Could you please help me in resolving the issue in Jenkins, finally why am unable to collect the logs for the failure case scenarios to the tester baord from Jenkins.

I have hereby attached the robot script, screenshots of Jenkins Plans in the UI int he browser and the failure case reports/logs thats not getting copied to the tester board, please let me know if you require any other info w.r.t the issue for providing me the inputs

If this is not the right place, please redirect me to the correct mail ids where I can post this issues, as am stuck with this issue for more than 2 days

Many Thanks in advance,

[srinivasan]

Thanks a lot for your quick replies Rudd,

1.We have Jenkins agent running on Orange pi, yeah our setup is configured in such a way that the DUT is connected to the Orange pi where Jenkins server is running on in Orange pi.

2.It is not possible to connect the DUT directly (as we don't have Jenkins server running on it) to the Jenkins without the orange pi, as this is how we have defined the test setup for triggering the plans like flashing to DUT and all etc., with orange pi board via specific Jenkins plan.

3.Yeah there is only one log.html for all these tests -- as all these tests are part of One jenkins plan ie., BIOS_settings_verification

4.By my problem is failure case robot framework reports are generated in DUT and not getting SCP'ed back to the tester board, whereas this works in case of success test log reports and are getting SCP'ed to tester board and getting updated in the Jenkins UI in the browser, but fails for failure log reports, not getting updated in Jenkins UI  when opened in the browser and not getting SCP'ed back to the tester board, as it always  still shows the old success log reports instead of showing the failure case reports in Jenkins UI in the browser even though when the job is failed as expected.

5. I feel some Jenkins configuration is missing for logging the failure reports firstly on the Jenkins UI  when opened in the browser. Could you please let me know if you have any idea like is there any configuration that is missing in Jenkins for logging failure case log reports running on orange pi?

Could you please let me know if you have any clues as am wandering from past 3 days still no clues yet.

 

Many Thanks in advance

On Mon, Nov 19, 2018 at 4:37 PM Ruud Prijs <ruud....@gmail.com> wrote:

Hi

I really wondering why you use a orange pi, is it not possible to connect the dut directly to jenkins?

I also don't understand your issue, there is just one log file for all tests, if you run test by test you should

give the log file a unique name for every run. in that case you can better use xml and merge those output

files at the end of the run. and use in Jenkins the robot framework report plugin.

Op za 17 nov. 2018 om 11:14 schreef srinivasan <sriniva...@gmail.com>:

Dear Jenkins /Robot framework community,

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at https://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

[srinivasan]

You're Awesome Ruud, and hereafter if you dont mind can I send all jenkins and robot frameworks issues to you?

1. Yeah this really worked, I have just changed the line to "python3 -m robot --outputdir results ~/qa/robot_tests/bios_settings_verification.robot || true" as you suggested, it started working.. now am able to get the failure logs getting updated on the UI and as well as the logs are getting redirected to the tester board"

#!/bin/bash

ifconfig

whoami

ls -l

mkdir results

board_ip=$(cat /home/root/target_ip)

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb -r qa/ root@${board_ip}:/home/root

#ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'pip3 install robotframework'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'mkdir results'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'ls'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'python3 -m robot --outputdir results ~/qa/robot_tests/bios_settings_verification.robot || true'

#ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'robot --outputdir results qa/robot_tests/bios_settings_verification.robot'

2. But one quick question, I didn't understand from your yesterday's input that is " move this second script to a post action in jenkins" -- could you please elaborate this point as I see lot of options being listed in the drop down of the "Add post build Actions" as attached in the second snapshot which option to be choosen?, and is that what you meant the below scripts as the second script to be moved to the post build actions? and if this what you meant, could you please help me how can I move this to post-build actions?

#!/bin/bash

board_ip=$(cat /home/root/target_ip)

#echo "COPYING THE RESULTS BACK TO THE TESTER BOARD!!!!!!!!!!!!!!!!!!!!!!!"

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb -r root@${board_ip}:/home/root/results/ .

sync

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'rm -r /home/root/results'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'rm -r /home/root/qa'

Many Thanks you have really made my day.. finally.. after struggling from past 3 days....

On Mon, Nov 19, 2018 at 4:55 PM Ruud Prijs <ruud....@gmail.com> wrote:

just some thinking:  can you try? =>     'python3 -m robot --outputdir results ~/qa/robot_tests/bios_settings_verification.robot || true'  => result will always be true
i think the second shell is not executed at all instead of || true you can also move this second script to a post action in jenkins

Ruud

Op ma 19 nov. 2018 om 16:29 schreef srinivasan <sriniva...@gmail.com>:

[srinivasan]

In the above point number 1.

ie., regarding the fix "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'python3 -m robot --outputdir results ~/qa/robot_tests/bios_settings_verification.robot || true'"

I need to press "F5" for getting updated in the jenkins UI is that ok and is it as expected?

[srinivasan]

Dear Ruud,

When I try to use the --nostatusrc as below I see the below error:

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'python3 -m robot --variable SIGNAL_LEVEL_THRESHOLD:-70 --variable SSID:"NI WiFi" --variable PW:"abcddfefgh1234!" --outputdir results qa/robot_tests/wifi_testing.robot --nostatusrc'

[ ERROR ] Parsing '--nostatusrc' failed: Data source does not exist.

Could you please help me, am I passing "--nostatusrc" is the right place?

On Tue, Nov 20, 2018 at 10:41 AM Ruud Prijs <ruud....@gmail.com> wrote:

:) it was just a very small issue, Just keep sending your questions to this group,  I bet there are also some good developers in this group who can help you as well and others can read the solutions. its a good place to learn about the robotframework and post your questions.

 

Did you also read the post of tatu? you can also use the option --nostatusrc instead of || true

see below:

Ugh

There is also a command line opinion, more details on the documentation: http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#return-codes

-Tatu

greetings Ruud

Op di 20 nov. 2018 om 10:21 schreef srinivasan <sriniva...@gmail.com>:

[srinivasan]

I have one more jenkins Plan/Job  performing "Wi-Fi" testing (validating Negative Scenario) on different target with the same SoC and  OS linux (might be the build version might be different not sure whether if this is really causing the issue), as below when I run it manually on the target, the test case should fail as expected, it fails and am getting the results as expected, just because am passing the invalid SSID. 

But the same thing is getting passed sometimes or failed with incorrect failure results in jenkins with " || true" or without " || true" even at the end of python3 -m robot --variable SIGNAL_LEVEL_THRESHOLD:-70 --variable SSID:"nkopaqwe" --variable PW:"T.f.o.s.1996!" --outputdir results qa/robot_tests/wifi_testing.robot

Could you please let me know what could be wrong?

In Jenkins Execute Shell box:

#!/bin/bash

ifconfig

whoami

ls -l

mkdir results

board_ip=$(cat /home/root/target_ip)

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb -r qa/ root@${board_ip}:/home/root

#ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'pip3 install robotframework'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'mkdir results'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'ls'

#ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'python3 -m robot --outputdir results qa/robot_tests/wifi_testing.robot'

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ~/.ssh/id_rsa_dtfb root@${board_ip} 'python3 -m robot --variable SIGNAL_LEVEL_THRESHOLD:-70 --variable SSID:"nkopaqwe" --variable PW:"T.f.o.s.1996!" --outputdir results qa/robot_tests/wifi_testing.robot || true'

Jenkins(Console output):

15:08:52 Started by user Srini Shanmugam

15:08:52 [EnvInject] - Loading node environment variables.

15:08:52 Building remotely on PenguinTeam_OrangePI-Prime_2 (PenguinTeam-x86) in workspace /home/root/workspace/PenguinTeam/Wi-Fi_testing

15:08:53  > git rev-parse --is-inside-work-tree # timeout=10

15:08:53 Fetching changes from the remote Git repository

15:08:53  > git config remote.origin.url https://bitbucket.bln.native-instruments.de/scm/emb/qa.git # timeout=10

15:08:53 Fetching upstream changes from https://bitbucket.bln.native-instruments.de/scm/emb/qa.git

15:08:53  > git --version # timeout=10

15:08:53 using GIT_ASKPASS to set credentials UsernamePassword

15:08:53  > git fetch --tags --progress https://bitbucket.bln.native-instruments.de/scm/emb/qa.git +refs/heads/*:refs/remotes/origin/*

15:08:53 Seen branch in repository origin/feature/Thilo_Sandbox

15:08:53 Seen branch in repository origin/feature/Wi-Fi_tests

15:08:53 Seen branch in repository origin/feature/bios_settings_verification

15:08:53 Seen branch in repository origin/feature/cyclictest-cpu-isolation

15:08:53 Seen branch in repository origin/feature/cyclictest-duration

15:08:53 Seen branch in repository origin/feature/sd_tests

15:08:53 Seen branch in repository origin/feature/wifi_tests

15:08:53 Seen branch in repository origin/master

15:08:53 Seen branch in repository origin/ref/py2topy3_code

15:08:53 Seen 9 remote branches

15:08:53  > git show-ref --tags -d # timeout=10

15:08:53 Checking out Revision 335a7a89731487b5aa7177b019257bbc67317057 (refs/remotes/origin/feature/wifi_tests)

15:08:53  > git config core.sparsecheckout # timeout=10

15:08:53  > git checkout -f 335a7a89731487b5aa7177b019257bbc67317057

15:08:53 Commit message: "Updated the comments from Thilo"

15:08:53  > git rev-list --no-walk 335a7a89731487b5aa7177b019257bbc67317057 # timeout=10

15:08:53 [Wi-Fi_testing] $ /bin/bash /tmp/jenkins9201255351948952183.sh

15:08:53 docker0   Link encap:Ethernet  HWaddr 02:42:0B:8D:B2:4B  

15:08:53           inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0

15:08:53           UP BROADCAST MULTICAST  MTU:1500  Metric:1

15:08:53           RX packets:0 errors:0 dropped:0 overruns:0 frame:0

15:08:53           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

15:08:53           collisions:0 txqueuelen:0 

15:08:53           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

15:08:53 

15:08:53 eth0      Link encap:Ethernet  HWaddr 02:01:E4:1C:3E:33  

15:08:53           inet addr:10.1.9.51  Bcast:10.1.11.255  Mask:255.255.252.0

15:08:53           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

15:08:53           RX packets:6744343 errors:0 dropped:960 overruns:0 frame:0

15:08:53           TX packets:1230759 errors:0 dropped:0 overruns:0 carrier:0

15:08:53           collisions:0 txqueuelen:1000 

15:08:53           RX bytes:1410933656 (1.3 GiB)  TX bytes:809158061 (771.6 MiB)

15:08:53           Interrupt:25 

15:08:53 

15:08:53 lo        Link encap:Local Loopback  

15:08:53           inet addr:127.0.0.1  Mask:255.0.0.0

15:08:53           UP LOOPBACK RUNNING  MTU:65536  Metric:1

15:08:53           RX packets:164 errors:0 dropped:0 overruns:0 frame:0

15:08:53           TX packets:164 errors:0 dropped:0 overruns:0 carrier:0

15:08:53           collisions:0 txqueuelen:1000 

15:08:53           RX bytes:18152 (17.7 KiB)  TX bytes:18152 (17.7 KiB)

15:08:53 

15:08:53 root

15:08:53 total 12

15:08:53 drwxr-xr-x    9 root     root          4096 Nov  9 13:30 qa

15:08:53 drwxr-xr-x    2 root     root          4096 Nov 20 14:08 qa@tmp

15:08:53 drwxr-xr-x    2 root     root          4096 Nov 13 09:27 results

15:08:53 mkdir: can't create directory 'results': File exists

15:08:53 Warning: Permanently added '10.1.11.107' (RSA) to the list of known hosts.

15:08:55 Warning: Permanently added '10.1.11.107' (RSA) to the list of known hosts.

15:08:55 mkdir: cannot create directory 'results': File exists

15:08:55 Warning: Permanently added '10.1.11.107' (RSA) to the list of known hosts.

15:08:55 qa

15:08:55 results

15:08:55 Warning: Permanently added '10.1.11.107' (RSA) to the list of known hosts.

15:08:56 ==============================================================================

15:08:56 Wifi Testing :: This is the Maschine Native OS Build Wi-Fi Test.              

15:08:56 ==============================================================================

15:08:56 Initialize Wi-Fi Module                                               | PASS |

15:08:56 ------------------------------------------------------------------------------

15:08:56 Enable Wi-Fi Module                                                   | PASS |

15:09:01 ------------------------------------------------------------------------------

15:09:01 Connect Wi-Fi Module to SSID                                          | PASS |

15:09:08 ------------------------------------------------------------------------------

15:09:08 Check for Wi-Fi Connectivity                                          | FAIL |

15:09:08 'False' should be true.

15:09:08 ------------------------------------------------------------------------------

15:09:08 Log Wi-Fi Module IP                                                   | PASS |

15:09:08 ------------------------------------------------------------------------------

15:09:09 Get Gateway IP and Check Whether Wi-Fi is Pingable                    | FAIL |

15:09:09 AssertionError

15:09:09 ------------------------------------------------------------------------------

15:09:09 Check for Wi-Fi Signal Strength with Threshold                        | FAIL |

15:09:09 ValueError: invalid literal for int() with base 10: ''

15:09:09 ------------------------------------------------------------------------------

15:09:09 Wifi Testing :: This is the Maschine Native OS Build Wi-Fi Test.      | FAIL |

15:09:09 7 critical tests, 4 passed, 3 failed

15:09:09 7 tests total, 4 passed, 3 failed

15:09:09 ==============================================================================

15:09:09 Output:  /home/root/results/output.xml

15:09:09 Log:     /home/root/results/log.html

15:09:09 Report:  /home/root/results/report.html

15:09:09 [Wi-Fi_testing] $ /bin/bash /tmp/jenkins5528543601986170281.sh

15:09:09 Warning: Permanently added '10.1.11.107' (RSA) to the list of known hosts.

15:09:09 Warning: Permanently added '10.1.11.107' (RSA) to the list of known hosts.

15:09:09 Warning: Permanently added '10.1.11.107' (RSA) to the list of known hosts.

15:09:10 Robot results publisher started...

15:09:10 -Parsing output xml:

15:09:10 Done!

15:09:10 -Copying log files to build dir:

15:09:10 Done!

15:09:10 -Assigning results to build:

15:09:10 Done!

15:09:10 -Checking thresholds:

15:09:10 Done!

15:09:10 Done publishing Robot results.

15:09:10 Build step 'Publish Robot Framework test results' changed build result to FAILURE

15:09:10 [BFA] Scanning build for known causes...

15:09:10 [BFA] No failure causes found

15:09:10 [BFA] Done. 0s

15:09:10 Finished: FAILURE

I tried Manually on the target (the failure results are correct as expected):

root:~# python3 -m robot --variable SIGNAL_LEVEL_THRESHOLD:-70 --variable SSID:"nkopaqwe" --variable PW:"T.f.o.s.1996!" --outputdir results qa/robot_tests/wifi_testing.robot || true

==============================================================================

Wifi Testing :: This is the Maschine Native OS Build Wi-Fi Test.              

==============================================================================

Initialize Wi-Fi Module                                               | PASS |

------------------------------------------------------------------------------

Enable Wi-Fi Module                                                   | PASS |

------------------------------------------------------------------------------

Disconnect from current Network                                       | PASS |

------------------------------------------------------------------------------

Connect Wi-Fi Module to SSID                                          | FAIL |

'False' should be true.

------------------------------------------------------------------------------

Check for Wi-Fi Connectivity                                          | FAIL |

'False' should be true.

------------------------------------------------------------------------------

Log Wi-Fi Module IP                                                   | PASS |

------------------------------------------------------------------------------

Get Gateway IP and Check Whether Wi-Fi is Pingable                    | FAIL |

AssertionError

------------------------------------------------------------------------------

Check for Wi-Fi Signal Strength with Threshold                        | FAIL |

ValueError: invalid literal for int() with base 10: ''

------------------------------------------------------------------------------

Wifi Testing :: This is the Maschine Native OS Build Wi-Fi Test.      | FAIL |

8 critical tests, 4 passed, 4 failed

8 tests total, 4 passed, 4 failed

==============================================================================

Output:  /home/root/results/output.xml

Log:     /home/root/results/log.html

Report:  /home/root/results/report.html

root:~# ifconfig 

enp0s21f0u4 Link encap:Ethernet  HWaddr 00:0E:C6:A6:AE:E9  

          inet addr:10.1.9.49  Bcast:10.1.11.255  Mask:255.255.252.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:43731 errors:0 dropped:2151 overruns:0 frame:0

          TX packets:27227 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:13622436 (12.9 MiB)  TX bytes:12332900 (11.7 MiB)

enp2s0    Link encap:Ethernet  HWaddr 00:13:95:30:CB:45  

          inet addr:10.1.11.107  Bcast:10.1.11.255  Mask:255.255.252.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:39262 errors:0 dropped:0 overruns:0 frame:0

          TX packets:17794 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:14033698 (13.3 MiB)  TX bytes:2285045 (2.1 MiB)

          Memory:81300000-8131ffff 

lo        Link encap:Local Loopback  

          inet addr:127.0.0.1  Mask:255.0.0.0

          UP LOOPBACK RUNNING  MTU:65536  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

wlp1s0    Link encap:Ethernet  HWaddr 76:95:35:89:35:69  

          UP BROADCAST MULTICAST  MTU:1500  Metric:1

          RX packets:51854 errors:0 dropped:0 overruns:0 frame:0

          TX packets:1101 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:7365820 (7.0 MiB)  TX bytes:221255 (216.0 KiB)

root:~# 

Many Thanks in advance,

On Tue, Nov 20, 2018 at 2:26 PM Ruud Prijs <ruud....@gmail.com> wrote:

just put it before --variable

Ruud

Op di 20 nov. 2018 om 14:13 schreef srinivasan <sriniva...@gmail.com>:

[srinivasan]

Sorry it was my mistake I have not included in my local branch in Jenkins correctly.. Sorry once again, please ignore my inline query...