Distributed testing in robot framework

[himanshu mehrotra]

Hi,

Can anybody please tell me how i can achieve distributed testing in robot framework. I found pabot library using which i can do parallel execution on same different.

Can anybody please let me if i can achieve this using pabot. I found below on pabot git hib page.
https://github.com/mkorpela/pabot/blob/master/TRICKS.md

It says below.

Cookbook example: How to use pabot to distribute testing to multiple remote machines
Basic idea:

1. Run remote library servers on the remote machines
2.Use pabotlib with a resource file to share the IP addresses of the remote machines
3.On a suite setup and teardown use pabot.PabotLib.Acquire Value Set, Get Value From Set and pabot.PabotLib.Release Value Set to set a specific suite to be executed with a specific remote machine (for example set the host ip to a global variable and use it in the Remote library import)

Any idea how can i run remote library server on remote machines

[Tatu Aalto]

Ugh

The answer to your question depends on the details of your infrastructure. Are we example talking about single OS, like Ubuntu 18.04 or is there a mixture of different OS. Or do you perhaps use containers or something running in the cloud? Do you have Puppet or other deployment tool in use? 

The bottom line is that there is almost infinite number of tools available and there is not a single correct answer to your question. 

The best answer is: If your company is already doing something similar, then I would look closely to that direction.

-Tatu
Send from my mobile

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at https://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

[yiyang zhang]

Hi, below is my way:

1. Prepare 3 test systems.

2. Set up CI, such as jenkins, with master-slave mode.

3. Then setup 5 jobs in jenkins

1) job 1:  run smoke test

2) job 2~4: If smoke passed, will start these 3 jobs in the same time to run the left tests in the 3 test systems filter by tags, pabot will help according to your test design.

3) job 5: use `rebot` to merge all the test reports & logs and then send the email.

below is some uesful jenkins plugins

Copy Artifact Plugin

Email Extension

Folders

JobFanIn

Robot Framework plugin

Workspace Cleanup

[Ruud Prijs]

Hi,

if your interested how i am doing this:

Its similar as yiyang zhang but instead of pabot (step 2) i use Jenkins to handle the distributed testing (a Jenkins pipeline that runs jobs in parallel).

Kind regards Ruud

Op vrijdag 15 maart 2019 04:15:29 UTC+1 schreef himanshu mehrotra:

[himanshu mehrotra]

Hi,

I dont want to use any ci tool for this purpose. And my requirement is to perform distributed testing of my robot test cases across different windows system.

Can anyone of you let me know any library or else one way i can think is using python code i will connect to windows vm machines and make bat file on different VMs and run those from my server machine.

Any other thought?

Thanks,

Himanshu

[Hélio Guilherme]

I have used Jenkins CI running standalone (it is just a .jar file), on my laptop. I had on VM with Windows 7, connected to Jenkins as a slave. Other real PCs and VMs from my colleagues would also connect to my Jenkins. This setup is very practical.

[Tatu Aalto]

Ugh

We also use Jenkins, but our test do not actually run tests  in Jenkins or in Jenkins slaves. Jenkins slaves are used but slaves are used to:

1) Reserve a VM from a VM cluster.

2) Push automation package to the reserved VM,

3) Run commands in the VM. In practice the command to start test execution but any valid command which can be executed trough windows cmd goes.

4) Get files. In practice we use it to get the test run results.

5) Tell to the VM cluster that VM is free and can be deleted.

There is common tooling in the Jenkins slaves, which relies in separate repositories and are build by Jenkins. Yes, test automation has a builds, like any other artifacts of the product. This tooling is responsible of building the test automation package from several sources, like Python installation, our Python dependencies, repository where we keep test and libraries and so on. Tooling is also responsible for communication with the VM cluster and communication with the VM. But it unifies the running of the test quite nicely. All I need to define are:

1) List of VM which I need to run my tests. Can be single Windows VM or can be range of VM's. Anything starting from Windows XP to latest version of Windows 10 is goes.

2) Definition of the tests, we have an external files called set, which defines a list of suites which we want to run.

3) Maximum life time of the VM. If running test last longer, VM cluster will delete VM by force. 

This allows the separation of the test and the test infrastructure. I do not have to care where the VM actually is, I just tell what kind of VM I need, the tooling will get me a VM. It also allows me spawn many VM with a single job/configuration, Jenkins matrix job is good for this type of things. There are some OS specific things, which sometimes are needed to take in account in tests, example some Windows API may have changed between Windows 8 and Windows 10, but that is quite rare thing. 

As a bottom line, CI is a good tool for this type of things and as you have noticed there are many ways to do it. And my way is not any better then what other have explained, it is just better suited for my team. But many CI have nice features, like dash boards, integration to instant messaging system or email (to notify failures or other stuff), queue system in build, trends (like how long the execution time takes and more) and many features which are useful. For me the best feature in Jenkins is the Jenkins DSL, it allows us to create Jenkins jobs based on a configuration in a repository. My team has so many jobs, that keeping then up to date would be impossible without Jenkins DSL. So I hope that you do not overlook the CI option and try to reinvent the wheel.

-Tatu

[Anne kootstra]

What you propose to code in Python is a simplified CI. It is also something you shouldn't want to do during your test as it shouldn't have to deal with platform setup. So, for me the library route is really out of the question. 

It would be understandable to think that it's too complex for what you need today. However, a good CI approach will be extendable (because this is only the first step) and provide a scale-able platform for you to innovate your test approach on. If CI is also not your area of expertise, then definitely pick one of the developed open source or commercial platforms. Make use of the experience of others put in those platforms. Not having to deal with reports, user administration, security and user interfaces is really a time saver vs the small investment in learning to install and setup the tool.

Given your presumed setup (pre configured and running VM boxes) I'd go with Jenkins. Significant amount of tutorials and a large user base will get you quickly started, probably less then a day if you're committed to it.

Regards,

Anne

Op vrijdag 15 maart 2019 18:19:23 UTC+1 schreef himanshu mehrotra:

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-users+unsub...@googlegroups.com.

[Dmitriy Chernoshey]

Docker swarm, perhaps?

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.

To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at https://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

--

Dmitriy Chernoshey

Senior Automation Engineer

dmitriy.c...@rallyhealth.com

R⩓LLY by Rally Health, Inc.