Pabot not threading on start

[OliverNZ]

Hi,

I'm using pabot to execute our somewhat large regression. What I did note is that pabot will take quite a while to start. As in minutes. This is probably where it loads in the actual test and gets to grips with what it needs to do. What I can see is one CPU churning at 100% but all other CPUs being at idle. Once the test does start things get to be executed accross cores but at maybe 10-20%. 

I wonder if there is a way to multi thread that 1st startup of the test or to at least already get the automation going on a script as they become available so the time is better used. Since our regression will still grow I'm a bit apprehensious about that startup bit. 

I did read the code somewhat but lost track at some point before I understood what was going on. Someone any observations about this or comments?

Cheers Oliver

[Mikko Korpela]

Hi Oliver,

I'm almost certain that the CPU churning is done while Robot Framework parses your data directory. The 1st process must do this once for the whole dataset to be able to distribute executable suites to worker processes.

As Robot Framework parser is one threaded (actually only way in Python to make it quicker would be to use processes because of GIL) it will only use one core to do this.

You can try to debug it further if you want to with cProfiler.

Here is an example script that profiles pybot (you need to replace the relevant parts with pabot related stuff):

https://gist.github.com/mkorpela/3907371

Here is pabot starter script:

#!/usr/bin/python

from pabot.pabot import main

import sys

main(sys.argv[1:])

Hope this helps!

Cheers,

Mikko Korpela

[OliverNZ]

Thanks Mikko,

I might go there and have a look. Otherwise thanks for a great parallel processor. Works very well for us :-)

Cheers

Oliver