[Selenium-users] Posting Selenium test results?

[Bret Pettichord]

A question from one of my team mates:

>For anyone currently using Selenium as part of their CI build:
>Has anyone created a postresults servlet as described on the bottom of
this page http://selenium.thoughtworks.com/testrunner.html for writing
Selenium test reports to a file?
>If so can you share what you came up with? I can't seem to find much on
this on the internet community.

_____________________
Bret Pettichord
www.pettichord.com

[Darren Cotterill]

I have Selenium integrated with Cruise / ANT on our current project.

The code is fairly generic - we have the build on a different box to
the app server, so we couldn't just get the servlet to write a file.

I don't have chance at the minute, but I'll try and post some code and
scripts tomorrow.

I got part way to doing JUnit style reporting, but hit a few
roadblocks. I might post the code I have for this so far and my
current problems to see if anyone can help. I think this would be the
best approach for the results.

Cheers,
Darren.


On Wed, 23 Feb 2005 19:16:27 -0600, Jason R Huggins
<JRHu...@thoughtworks.com> wrote:
>
> I'm *way* over-due in checking in my code to do this into the Selenium
> repository. The version we have for this comes from the MyThoughtWorks
> project. I suspect that I'm the only one using Selenium integrated with
> CI... but I do hope that will change in the future!
>
> When Selenium is run with the URL "command line" argument of "?auto=true"...
> when the test suite is done, the results will be posted to /postResults.
> Selenium sends two things to the postResults servlet:
> 1) A "formData" object containing the following:
> a) a hidden form containing all the tests and test suite marked up with
> the green/red markup that comes from running the tests
> b) the "meta-data" of the test run, like number of tests passed, failed,
> etc.
> 2) The name of the user-agent (Firefox, IE, etc.)
>
> The postResults servlet splits this data into two files:
> 1) One master HTML file containing all the test results.
> 2) A text file with name/value pairs for the test suite meta-data.
>
> Until I checkin this code to the Selenium repository, here's the code that
> we run now....
> ------------------------------------------------------------------------------------------------
> def postResults(self, formData, userAgent):
> browserName = self.getBrowserName(userAgent)
>
> suiteTable = ''
> if formData.has_key('suite'):
> suiteTable = formData.get('suite', '')
> del formData['suite']
>
> testTables = []
>
> testTableNum = 1
> while formData.has_key('testTable.%s' % testTableNum):
> testTable = formData['testTable.%s' % testTableNum]
> testTables.append(testTable)
> del formData['testTable.%s' % testTableNum]
> testTableNum += 1
>
>
> # Unescape the HTML tables
> suiteTable = urllib.unquote(suiteTable)
> testTables = map(urllib.unquote, testTables)
>
> self.writeResultsToFiles(browserName, formData, suiteTable, testTables)
>
> return 'Results have been successfully posted to the server'
>
> # Writes the data out to 2 files. The formData is written to
> # result-browserName.txt as a list of key: value, one per line. The
> # suiteTable and testTables are written to output-browserName.html.
> def writeResultsToFiles(self, browserName, formData, suiteTable,
> testTables):
> ploneRootDir = os.environ.get('PLONE_ROOT', 'c:/rcenter/Plone')
> outputDir = os.path.join(ploneRootDir,
> 'Data/Products/TWSiteCustomizations/skins/TWSiteCustomizations_dtml/test_results')
>
> resultFile = 'result-%s.txt' % browserName
>
> f = open(os.path.join(outputDir, resultFile), 'w')
> for key in formData.keys():
> print >> f, '%s: %s' % (key, formData[key])
> f.close()
>
> outputFile = 'output-%s.html' % browserName
>
> f = open(os.path.join(outputDir, outputFile), 'w')
> print >> f, suiteTable
>
> for testTable in testTables:
> print >> f, '<br/><br/>'
> print >> f, testTable
>
> f.close()
>
>
> def getBrowserName(self,userAgent):
> if userAgent.find('MSIE') > -1:
> return 'IE'
> elif userAgent.find('Firefox') > -1:
> return 'Firefox'
> elif userAgent.find('Gecko') > -1:
> return 'Mozilla'
> else:
> return 'Unknown'
>
> ------------------------------------------------------------------------------------------------
>
> There are two higher level bits needed to integrate this into a CI build:
> 1) A script that launches the browser with the selenium URL and the argument
> of "?auto=true"
> 2) A script that parses the results*.txt file for errors and triggers some
> kind of notification process as a result.
>
> We have all these bits in the MyThoughtWorks code base, and it should be
> straight-forward to make this generically useful. If someone is interested
> in porting all the code to the Selenium project, I'd love the help!
>
> Jason R. Huggins
> Senior Developer - ThoughtWorks, Inc.
> W: 312.373.8556 Cell: 312.543.4566
> jrhu...@thoughtworks.com
> selenium.thoughtworks.com
>
> selenium-us...@lists.public.thoughtworks.org wrote on 02/23/2005
> 08:41:27 PM:

> > _______________________________________________
> > Selenium-users mailing list
> > Seleniu...@lists.public.thoughtworks.org
> > http://lists.public.thoughtworks.org/mailman/listinfo/selenium-users
>
> _______________________________________________
> Selenium-users mailing list
> Seleniu...@lists.public.thoughtworks.org
> http://lists.public.thoughtworks.org/mailman/listinfo/selenium-users
>
>
>

[br...@skife.org]

Bret Pettichord writes:

> A question from one of my team mates:
>
> >For anyone currently using Selenium as part of their CI build:
> >Has anyone created a postresults servlet as described on the bottom of
> this page http://selenium.thoughtworks.com/testrunner.html for writing
> Selenium test reports to a file?

Attached is the servlet I use to handle /postResults; I'm not sure who wrote
the original version. Note that this variant requires the resultsFileName
to be specified via an extra "resultsUrl" request-parameter; we invoke
Selenium as:

http://server/selenium/TestRunner.html?auto=true&resultsUrl=/postResults%3Fr
esultsFileName%3D/path/to/seleniumResults.html

NOTE: Support for "resultsUrl" is new (it wasn't in selenium-0.2) so for
simplicity, you may prefer to just hardcode a resultsFileName in the
servlet.

--
cheers, Mike

package au.com.someclient.selenium;

import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.io.Writer;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**

* Writes the results of a Selenium test run to a file.
*/
public class SeleniumResultsServlet extends HttpServlet {

protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {

String report = null;
try {
report = generateReport(unpackRequest(request));
}
catch (Exception e) {
reportError(e, HttpServletResponse.SC_BAD_REQUEST, response);
return;
}

String resultsFileName = request.getParameter("resultsFileName");
if (resultsFileName == null) {
getServletContext().log("WARNING: no resultsFileName specified");
}
else {
getServletContext().log("writing results to " + resultsFileName);
try {
Writer resultsFileWriter = new FileWriter(resultsFileName);
resultsFileWriter.write(report);
resultsFileWriter.close();
}
catch (IOException e) {
reportError(e, HttpServletResponse.SC_INTERNAL_SERVER_ERROR, response);
return;
}
}

response.getWriter().write(report);
}

private String getMandatoryParameter(HttpServletRequest request, String name) {
String paramValue = request.getParameter(name);
if (paramValue == null) {
throw new IllegalArgumentException("missing parameter: <" + name + ">");
}
return paramValue;
}

private SeleniumResults unpackRequest(HttpServletRequest request) {
SeleniumResults results = new SeleniumResults();
results.status = getMandatoryParameter(request,"result").toUpperCase();
results.totalTime = getMandatoryParameter(request,"totalTime");
results.numTestPasses = Integer.parseInt(getMandatoryParameter(request,"numTestPasses"));
results.numTestFailures = Integer.parseInt(getMandatoryParameter(request,"numTestFailures"));
results.numCommandPasses = Integer.parseInt(getMandatoryParameter(request,"numCommandPasses"));
results.numCommandFailures = Integer.parseInt(getMandatoryParameter(request,"numCommandFailures"));
results.numCommandErrors = Integer.parseInt(getMandatoryParameter(request,"numCommandErrors"));
results.suite = getMandatoryParameter(request,"suite");
results.testList = new ArrayList();
for (int i = 1;; i++) {
String testTable = request.getParameter("testTable." + i);
if (testTable == null) {
break;
}
results.testList.add(testTable);
}
return results;
}

private String generateReport(SeleniumResults results) {
StringWriter reportWriter = new StringWriter();
writeReport(results, new PrintWriter(reportWriter));
return reportWriter.toString();
}

private void writeReport(SeleniumResults results, PrintWriter writer) {
writer.println("<HTML>");
writer.println("<BODY>");
writer.println("<H2>Selenium Test Results</H2>");

writer.println("Result: " + results.status + "<br/>");
writer.println("Total time: " + results.totalTime + "<br/>");
writer.println("Tests passed: " + results.numTestPasses + "<br/>");
writer.println("Tests failed: " + results.numTestFailures + "<br/>");
writer.println("Commands passed: " + results.numCommandPasses + "<br/>");
writer.println("Commands failed: " + results.numCommandFailures + "<br/>");
writer.println("Command errors: " + results.numCommandErrors + "<br/>");

writer.println("<HR/>");
writer.println(results.suite);
writer.println("<HR/>");

for (Iterator i = results.testList.iterator(); i.hasNext();) {
String test = (String) i.next();
writer.println(test);
writer.println("<HR/>");
}

writer.println("</BODY>");
writer.println("</HTML>");
}

private void reportError(Throwable t, int errorCode, HttpServletResponse response) throws IOException {
response.setContentType("text/plain");
response.setStatus(errorCode);
PrintWriter out = response.getWriter();
t.printStackTrace(out);
}

private static final class SeleniumResults {
String status;
int numTestPasses;
int numTestFailures;
int numCommandPasses;
int numCommandFailures;
int numCommandErrors;
String totalTime;
String suite;
List testList;
}

}

[Mike Williams]

Mike (in the disguise of <br...@skife.org>) writes:

> Attached is the servlet I use to handle /postResults...

Urk! Sorry, my web-mail agent is all screwed up. The message above was
from me (not Brian McAllister).

--
cheers, Mike

[Voisey, Darren]

We currently have selenium running with Cruise Control on our .Net web
project. The tests are run within IE using a bit of COM automation
inside a Nant script. The test results posts to a .Net aspx page which
writes out the test results to an XML file for inclusion within cruise
output. One advantage of 'hosting' the browser using COM is that you can
spot when the tests finish [or indeed hang] and close the browser.

The only change we had to make to selenium was to make it post to an
aspx page within the tests virtual directory. It would be nice if this
could be provided on the url.

An example of building up some xml from the posted data in C# is below:

StringBuilder sbXML = new StringBuilder("<postresults>");
foreach (object key in Request.Form.Keys)
{
sbXML.AppendFormat("<{0}>{1}</{0}>",key,
Request.Form[Convert.ToString(key)]);
}
sbXML.Append("</postresults>");

Regards,

Darren


________________________________

There are two higher level bits needed to integrate this into a CI
build:
1) A script that launches the browser with the selenium URL and the
argument of "?auto=true"
2) A script that parses the results*.txt file for errors and triggers
some kind of notification process as a result.

We have all these bits in the MyThoughtWorks code base, and it should be
straight-forward to make this generically useful. If someone is
interested in porting all the code to the Selenium project, I'd love the
help!

Jason R. Huggins
Senior Developer - ThoughtWorks, Inc.
W: 312.373.8556 Cell: 312.543.4566
jrhu...@thoughtworks.com
selenium.thoughtworks.com

selenium-us...@lists.public.thoughtworks.org wrote on 02/23/2005
08:41:27 PM:

> A question from one of my team mates:
>
> >For anyone currently using Selenium as part of their CI build:
> >Has anyone created a postresults servlet as described on the bottom
of
> this page http://selenium.thoughtworks.com/testrunner.html for writing

> Selenium test reports to a file?