test result file mechanism

[Falk]

Hi,

https://bitten.edgewall.org/wiki/Documentation/commands.html#python-unittest shows this example:

<python:unittest file="build/test-results.xml"/>

Where is that xml file located, on server or bitten slave side?

If on bitten slave side, what is the base directory?

CU, Falk

[Simon Cross]

It's on the slave initially, and then uploaded to the master (that's
what the `<python:unittest ...>` does).

I believe the base directory is the working folder.

[Falk]

I actually don't have a unittest but my (Squish) tests will result in a test-results.xml file with the same format. Anyway, so is my plan. Will it work that the slave transfers that result xml file to the server as well? In the end I want to see such Unit Tests history graph over the revisions as you can see here at https://bitten.edgewall.org/build/trunk 

[Simon Cross]

Yes, the slave should send the results to the server and you should
see a report. :)

[Falk]

...
 <step id="Test1">
   <sh:exec file="C:/bitten/branches/trunk/myproject/bitten_test1.bat" args="${os} ${revision} ${repository.branch} ${build}"/>
   <sh:exec file="echo" args="Test1 complete"/>
   <python:unittest file="test-result.xml"/>
 </step>
...

What could be wrong? I configured this step in my recipe and put a dummy file test-result.xml into the folder C:/bitten/branches/trunk/myproject but a graph did not appear.

The content of that dummy file test-result is:

<report category="test">
   <test name="my_module"
             file="my_module.py"
             percentage="75"
             lines="4"
             line_hits="2 0 - 1 1">
    </test>
</report>

[Simon Cross]

It's a bit hard to say exactly what is wrong from the detail given,
but if you run your slave worker with `--keep` you should be able to
look at the build folders and check that the appropriate file was
generated.

The slave worker logs should also show the build file being uploaded.

I haven't used Bitten in a long time so my knowledge of it is very rusty.

[Falk]

 <step id="Test1">
   <sh:exec file="C:/bitten/branches/${repository.branch}/myproject/bitten_test1.bat" args="${os} ${revision} ${repository.branch} ${build}"/>

   <sh:exec file="echo" args="Test1 complete"/>

   <report category="test" file="C:/bitten/branches/${repository.branch}/myproject/squishresults_report.xml"/>
 </step>

I found out the tag <report> was missing. And using absolute pathes ensures the file is always found.

Well, I noticed now all of the Bitten platforms try to report that xml file. How can I say in the recipe I just want to report the xml file for certain Bitten platforms (e.g. for Windows yes but not for Linux)?

[Falk]

And I wondered, in case of more than 1 Bitten platform, which xml report of the several Bitten platforms is shown in the diagram? (x axis = revision, y axis = number of test errors / total count)

[Simon Cross]

Maybe try and find out?

I'm not sure if there is an easy way to turn reports on or off. I
think if the report file doesn't exist the step is skipped, which
might help. Otherwise you might be able to make something work the
sh:pipe and passing the platform as an argument to the script
(although I don't have a concrete plan).

[Falk]

I found out the last target platform rules and finally sets the result displayed in the graph. Since I have several target platforms for just building the software and an additional target platform for tests only (which waits for the result of the build platforms), it's OK for me, because it's the last target platform called. I just wondered what the basic idea of the graph is when there are more than 1 bitten target platforms.
Thank you for your attention and help! CU, Falk

[Simon Cross]

Yay! Glad everything worked out.