Testilence Library Reference

Author: Roman Neuhauser
Contact: neuhauser@sigpipe.cz
Copyright: This document has been placed in the public domain.
Id:reference.rest 498 2008-06-03 05:45:17Z roman
HeadURL:$HeadURL: https://svn.sigpipe.cz/r/trunk/testilence/docs/reference.rest $

Contents

1   Overview

The library code is organized into six circuits (with some overlaps): testing, organization, execution, evaluation, reporting, and presentation.

The testing circuit contains these important classes: Tence_TestCase, Tence_Assertion (there's more but those are private parts).

Organization comprises Tence_TestSuite, Tence_TestCase classes, and the Tence_TestCollection interface.

Execution includes the Tence_RunnableTest interface, and the Tence_TestCaseClass class.

The evaluation, reporting, and presentation circuits are still too raw for documentation.

Tence_TestCase is an abstract base class you're welcome to use as the parent of your test cases. However, any class that implements Tence_RunnableTest (and optionally Tence_TestCollection) will do. Tence_TestCase provides a handful of assertion methods and provides a very concise interface for exception testing.

Tence_TestCaseClass helps keep the number of non-assertion methods in Tence_TestCase as low as possible, a sort of proxy for Tence_TestCase, identifies test methods in the test case instance.

Tence_Assertion transports details of test method execution from the case into outside world. Instances of this class are virtually invisible to normal Testilence users, the class is mentioned here to prevent confusion later as it appears a lot in the Tence_TestCase's synopsis.

1.1   Test Results

Possible test results are:

  • success
  • failure
  • exception
  • defect

A "test" consists of calls to setUp(), the test method, and tearDown(). All three are considered during evaluation of the test, the formula is:

  1. if setUp() or tearDown() threw, the test was defective, else
  2. if the test method threw, and it was an expected exception (see willThrow()), the test was successful, else
  3. if the test method threw, but a different exception class was expected, the test failed, else
  4. if the test method threw and no exception was expected, the result is an exception, else
  5. if an exception was expected but not thrown, the test failed, else
  6. if the test method returned a Tence_Assertion, that object defines whether the test was a success or a failure, else
  7. the test was defective

2   Public Components

2.1   Tence_RunnableTest

Interface through which the tests are run.

void Tence_RunnableTest::run(Tence_Reporter $reporter)

2.2   Tence_TestCollection

Interface for access to components of a composite test.

Tence_RunnableTest[] Tence_TestCollection::getTests()

2.3   Tence_TestMethod

implements:Tence_RunnableTest
Tence_TestMethod::__construct(Tence_TestCaseClass $tcc, ReflectionMethod $rm)
$rm must contain a method of the class wrapped in $tcc.
void Tence_TestMethod::run(Tence_Reporter $reporter)
Constructs a new instance using $tcc and calls the method passed in $rm. Result of the invocation is passed to $reporter.

2.4   Tence_TestCase

implements:Tence_RunnableTest, Tence_TestCollection

Abstract base for test case classes.

Intent of this class is to provide comfortable environment for robust tests. To this end, Tence_TestCase provides a set of assertion methods. A test case class inherits Tence_TestCase. It can define any number of test methods, and optionally either or both of setUp() and tearDown(). It should not define any constructor or destructor methods: the library is free to create and destroy suite and case instances without actually running them.

2.4.1   Constructor and Destructor

public final function __construct() { ... }
public final function __destruct()  { ... }

Instances of Tence_TestCase's subclasses must be default constructible, and carry no state outside of the setUp()-tearDown() interval. Tence_TestCase has final constructor and destructor to enforce this policy.

2.4.2   Test Methods

public Tence_Assertion Tence_TestCase::testXXX()

Any public nullary method whose name begins with "test" is considered a test method, and must return an instance of Tence_Assertion. Failing to do so results in the test method being marked as defective. The reasons behind this requirement are better tests and better code.

  1. Multi-assertion test methods reduce test coverage, since the first failed assertion will abort the test method, skipping latter assertions in that method.
  2. Single-assertion test methods provide the author with a strong incentive to think about the PUT, because each test method should name a behavior. Thinking of meaningful names for all the tests is sometimes painful, but the reward is a better understanding of the code.
  3. Single-assertion test methods provide clearer picture to the reader.

Important part of robustness in tests is isolation: any test that depends on things it doesn't control is fragile. This library guarantees that each test method will be called on a separate test case instance.

2.4.3   setUp() and tearDown()

public void Tence_TestCase::setUp()
public void Tence_TestCase::tearDown()

setUp() and/or tearDown() can be used for setup and cleanup tasks common to all test methods of the test case. setUp() is called before every test method, tearDown() afterwards, both on the same instance as the test method.

If setUp() throws, tearDown() is run immediately. The test method is skipped. If either of these two methods throws, the test method is marked as defective. Exceptions thrown from setUp() mute all exceptions thrown from tearDown().

2.4.4   Assertion Methods

Note

None of the assertFoo() methods performs any type conversions on the tested values. This eliminates a few false negatives: for example, 0 == false, but their string representations differ drammaticaly ("0" vs. ""); the sooner this kind of things is caught the better.

assertTrue(1) will fail, as will assertEquals(1, "1")!

Tence_Assertion Tence_TestCase::assertTrue($bool, $msg = '')
Checks that $bool is true.
Tence_Assertion Tence_TestCase::assertFalse($bool, $msg = '')
Checks that $bool is false.
Tence_Assertion Tence_TestCase::assertEquals($exp, $act, $msg = '')

Checks that $act equals $exp.

Instances of the same class are compared by value (using the == operator).

Tence_Assertion Tence_TestCase::assertSame($exp, $act, $msg = '')

Checks that $act equals $exp.

Instances of the same class are compared by identity (using the === operator).

Tence_Assertion Tence_TestCase::assertNull($any, $msg = '')
Checks that $any is null.
Tence_Assertion Tence_TestCase::assertNotNull($any, $msg = '')
Checks that $any is not null.
Tence_Assertion Tence_TestCase::assertIsA($o, $class, $msg = '')
Checks that $o is an instance of $class.
Tence_Assertion Tence_TestCase::assertMatches($re, $val, $msg = '')
Checks that $val matches the Perl-compatible regular expression $re.
void Tence_TestCase::willThrow($type)

Alias: void Tence_TestCase::setExpectedException($type)

Iff the code following the call of this method throws an exception that is instanceof $type, the test method will "return" successful assertion. All other outcomes cause a failure.

Actual return value of this test method (if any) will be ignored.

void Tence_TestCase::run(Tence_Reporter $reporter)
Calls every public method whose name begins with test. Result of each invocation is passed to $reporter.
Tence_RunnableTest[] Tence_TestCase::getTests()
Returns an array of Tence_TestMethod instances, one for each test method.

2.4.5   Utility methods

Tence_util_TempFile Tence_TestCase::mkstemp($dir, Tence_util_FilenameGenerator $gen = null)
Returns a Tence_util_TempFile instance.
Tence_util_TempDir Tence_TestCase::mkdtemp($dir, Tence_util_FilenameGenerator $gen = null)
Returns a Tence_util_TempDir instance.

2.5   Tence_util_TempFile

This class provides safe temporary files. The file is created in the constructor with:

fopen(..., 'x+')

which should translate to open(..., O_EXCL|O_CREAT), and removed in the destructor.

Tence_util_TempFile::__construct($dir, Tence_util_UniqueFilenameGenerator $generator = null)

$dir (falls back to getenv("TMPDIR") and "/tmp" if null) is used as the directory component of the path of the file being created.

$generator defaults to a fresh instance of Tence_util_UniqueFilenameGenerator.

Throws RuntimeException if $dir is wrong, or if mkdir() fails. Throws Tence_util_GeneratorExhausted if a unique filename cannot be found within the configured number of attempts.

resource Tence_util_TempFile::fd()
Returns descriptor for the underlying file, open for reading and writing.
string Tence_util_TempFile::path()
Returns path to the underlying file.
Tence_util_TempFile::__destruct()
Unlinks the underlying file and closes the associated descriptor.

2.6   Tence_util_TempDir

This class provides temporary directories. The directory is created in the constructor, and recursively removed in the destructor.

Tence_util_TempDir::__construct($dir, Tence_util_UniqueFilenameGenerator $generator = null)

$dir (falls back to getenv("TMPDIR") and "/tmp" if null) is used as the directory component of the path of the directory being created.

$generator defaults to a fresh instance of Tence_util_UniqueFilenameGenerator.

Throws RuntimeException if $dir is wrong, or if fopen() fails. Throws Tence_util_GeneratorExhausted if a unique filename cannot be found within the configured number of attempts.

string Tence_util_TempDir::path()
Returns path to the underlying directory.
Tence_util_TempDir::__destruct()
Removes the underlying directory recursively.

2.7   Tence_TestSuite

Tence_TestSuite & Tence_TestSuite::add(Tence_RunnableTest $test)
Adds $test to the suite. Returns $this.
void Tence_TestSuite::run(Tence_Reporter $reporter)
Calls run($reporter) on every $test it contains.
Tence_RunnableTest[] Tence_TestSuite::getTests()
Returns an array of all $tests added so far.