SimpleTest is a nice testing framework for testing PHP-based applications at the unit and functional level. However, I (and others) have had trouble getting JUnit-compatible XML output for use in continuous integration servers like Jenkins.
SimpleTest provides a built-in XML-based output, but it seems to be mostly for internal communication by SimpleTest itself. It's not JUnit compatible.
To convert SimpleTest output into JUnit-compatible XML, I've seen at least a few posts :
These involve using an XSLT to transform SimpleTest's "native" XML output into JUnit. However, this seems rather kludgy, and I had limited success getting the transform to work properly. I particularly encountered problems with entities not being encoded properly.
I later found a "JUnitReporter" class that extended SimpleReporter, but I couldn't get that to work too well with SimpleTest 1.1. Some of SimpleTest's reporting may have changed from version 1.0 to 1.1, and that may explain some issues.
Anyway, I have found something that works reasonably well. In the SimpleTest source tree, there is a contributed JUnitXMLReporter which seems to work fairly well.
This JUnitXMLReporter class file is located in the extensions directory, but it is not present in the SimpleTest 1.1 download.
After placing the JUnitXMLReporter class file in the extensions directory, I was able to get test results that looked like JUnit-compatible XML. However, Simpletest was still outputing a "footer" like this :
Test cases run: 2/3, Passes: 6, Failures: 0, Exceptions: 0
I imagine there is a more elegant way to override the footer, but I first tried to hack the reporter.php file (paintFooter) to remove the trailing footer. But I was still getting the "testSuite.php" suite name output. Additionally, I was getting some PHP warnings at the top of the output.
So I reverted my hacks to reporter.php and decided to "tidy up" the output the stock JUnitXMLReporter was giving me. I run my Jenkins tests in a short bash script which looks like this :
php /home/me/tests/testSuite.php > testResultsTmp.xml
tidyXml.php is a simple script which looks like this :
$xml = file_get_contents ('testResultsTmp.xml');
$xml = tidy_repair_string($xml, array(
'output-xml' => true,
'input-xml' => true
Nothing's ever easy, so I first had to install the PHP Tidy extension with "sudo aptitude install php5-tidy". But now at least I'm able to launch my tests, get JUnit-compatible XML, and manage test results with Jenkins.
Not the most elegant, but it works for me. Any suggestions to improve? Please note in the comments below.