Running the tests, the easy way with autorun

Careful : this documentation refers to un-released code. It's only available in CVS.

You started with unit testing not so long ago and this is what you came up with... A file test_of_all_modules.php containing : 

<?php
require_once('simpletest/unit_tester.php');
require_once('simpletest/reporter.php');

$group = new TestSuite();
$group->addFile('test_of_module_ship.php');
$group->addFile('test_of_module_pay.php');

$group->run(new HtmlReporter());
?>

But now you have a new way of shipping your socks overseas. Easily you just need to add a new file test_of_module_ship_overseas.php to your test runner... Quickly enough, you end up with a lot of file : 

<?php
require_once('simpletest/unit_tester.php');
require_once('simpletest/reporter.php');

$group = new TestSuite();
$group->addFile('test_of_module_ship.php');
$group->addFile('test_of_module_ship_overseas.php');
$group->addFile('test_of_module_ship_france.php');
$group->addFile('test_of_module_ship_uk.php');
$group->addFile('test_of_module_ship_us.php');
...
$group->addFile('test_of_module_pay.php');

$group->run(new HtmlReporter());
?>

As you list grows, running the test suite becomes too time- consuming : the tests are actually slowing you down while writing code.

Introducing 'autorun'

What we're looking for now is a way to run one test at a time and still be able to run every one of them easily... What if we could just write - in a console for example :

home: /$ php test_of_module_ship_us.php
and still be able to do :
home: /$ php test_of_all_modules.php
It's possible now : you just need to add one include at the top of each test case. Your test_of_module_ship_us.php will look like : 
<?php
require_once('simpletest/autorun.php');

class TestUsingParcelForce extends UnitTestCase {
  function testParcelForceIsFine() {
    $this->assertTrue(true);
  }
}

class TestUsingMyOwnAirplane extends UnitTestCase {
  function testUsingMyOwnAirplaneOnlyWorksForSmallGifts() {
    $this->assertFalse(false);
  }
}
?>

Setting the prefered runner

PHP is mainly for web stuff : I'm guessing your first test platform is a web browser, Firefox, Opera or Safari usually. And SimpleTest comes with a nice HTML interface : it's the one we want to use when running one file. We can tell our test_of_module_ship_us.php to use it by default : 

<?php
require_once('simpletest/autorun.php');
SimpleTest :: prefer(new HtmlReporter());

class TestUsingParcelForce extends UnitTestCase {
  function testParcelForceIsFine() {
    $this->assertTrue(true);
  }
}

class TestUsingMyOwnAirplane extends UnitTestCase {
  function testUsingMyOwnAirplaneOnlyWorksForSmallGifts() {
    $this->assertFalse(false);
  }
}
?>

We get a nice green bar :

TestSuite

2/2 test cases complete: 2 passes, 0 fails and 0 exceptions.

Autorun for test suites

So we've managed to get the time spent running the test under control during the coding period. We now want to make sure all the tests still pass, all the time. Or at least after every commit to the SVN (or CVS) trunk.

Such process is called continuous integration : Martin Fowler writes really well about this practice where members of a team integrate their work frequently [...]. Each integration is verified by an automated buid (including test) to detect integration errors as quickly as possible. So I can just give you the link : Continuous Integration.

Now our post-commit hook in SVN (or Phing task if you prefer) doesn't need the HTML reporter. We need something useful at the command line : TextReporter to the rescue. Our test_of_all_modules.php becomes : 

<?php
require_once('simpletest/autorun.php');

SimpleTest :: prefer(new TextReporter());

class AllTests extends TestSuite {
  function AllTests() {
    parent::TestSuite();
    $this->addFile('test_of_module_ship.php');
    $this->addFile('test_of_module_ship_overseas.php');
    $this->addFile('test_of_module_ship_france.php');
    $this->addFile('test_of_module_ship_uk.php');
    $this->addFile('test_of_module_ship_us.php');
    ...
    $this->addFile('test_of_module_pay.php');
  }
}
?>

The results show up nicely in the console :

home: /$ php test_of_all_modules.php
grouptest
OK
Test cases run: 10/10, Passes: 37, Failures: 0, Exceptions: 0

Introducing 'autorun'.
Setting the prefered runner.
Autorun for test suites.
The JUnit FAQ has plenty of useful testing advice.
You will need the SimpleTest testing framework for these examples.