Testilence

Unit testing library for PHP 5

Author: Roman Neuhauser
Contact: neuhauser@sigpipe.cz
Copyright: This document has been placed in the public domain.
Id:README.rest 407 2007-02-13 23:51:36Z roman
HeadURL:$HeadURL: https://svn.sigpipe.cz/r/trunk/testilence/README.rest $

Contents

1   Introduction

Testilence is an open-source [1] unit testing library for PHP [2] 5. It is published under the MIT license [3] which means you are free and welcome to use it in any way. Although it is similar to other JUnit [4]-inspired unit testing libraries, Testilence is written with emphasis on what's useful, not what's usual, and combines original ideas with the best features found in different unit-testing toolkits into a coherent set.

The testing part roughly follows the canon [5], comprising an abstract test case class with a number of assertFoo() methods, and a test suite class which allows arbitrary nesting of cases and suites, thanks to the Composite organization. Testilence differs in that you only get to have one assertion per test method (this is a Good Thing).

Testilence features highly isolated tests with deterministic execution [6] and guaranteed cleanup [7].

Testilence is an essential ingredient in BEER [8].

1.1   Development Status

Testilence is in early stages of development, output of the command line tool lacks lots of essential information, and the reporting and presentation circuits [9] of the library need much work. Test isolation is still not perfect: environment, installed functions and classes, error handler, etc. still carry over from previous tests; certain forms of misuse will kill the command line tool with a fatal error.

That said, Testilence is already quite usable, and has gone through lots or realworld use.

2   Prerequisities

Basic guarrantee: Testilence is developed and tested on FreeBSD 6 with PHP 5.1.[3-6], and 5.2.[0-1], the Makefile is tested (no automation yet) with the system make (should work on any BSD) and GNU make, and on RHEL 4 with PHP 5.1.4 and 5.2.0, and GNU make.

2.1   Buildtime requirements

The build and installation mechanism requires a subset of the POSIX shell environment, plus make.

Testilence might switch to Scons [10] in the future.

Subversion checkouts require Docutils [11] to build HTML documentation.

2.2   Library runtime requirements

The Testilence library has the following runtime requirements:

  • PHP >= 5.1.3 (with PCRE, Reflection, SPL)

    Testilence works with 5.1.3 or higher. Expect a bump to 5.2.0 (for better __toString() behavior). Expect addition of any of these PHP modules: operator [12], pcntl [13], posix [14], runkit [15], xdebug [16].

  • BSD, Linux, or UNIX

    This is a conservative statement.

    Some features might be missing on non-POSIX systems. Testilence is written with preference for portability, and should work on Microsoft Windows as is. Windows is, however, a second-tier operating system, problems are bound to appear from time to time.

2.3   Command line interface runtime requirements

Runtime requirements of the Testilence command line interface are the same as the library requirements, plus:

There's no batch file for Microsoft Windows. You should be able to run tence with:

path/to/php path/to/tence <arguments>

3   Downloads

3.1   Repository

The latest source get be checked out from the Subversion [18] repository [19], which is also available for anonymous browsing using the viewvc [20] interface [21].

3.2   Official tarballs

Testilence source releases are distributed as bzip2ed tarballs created with either Tim Kientzle's bsdtar [22] or GNU tar, Julian Seward's bzip2 [23] in all cases. Your operating system should include compatible extraction tools.

All Testilence releases can be found in the download directory [24].

4   Build and Installation

4.1   From a package

The most comfortable way to install Testilence is using your operating system's packaging system. Ask your OS vendor for a package of the latest Testilence release.

The source distribution includes a FreeBSD port skeleton.

4.2   From the source

4.2.1   Wash and Go instructions

% url=http://www.testilence.org/dist/testilence-x.y.x.tar.bz2
% fetch -o - $url | tar -xjf -
% cd testilence-x.y.z
% make check install

4.2.2   Detailed description

The build and installation process can be configured using a number of make variables.

TENCE_PREFIX

Install location prefix. All installation paths are based on this one unless overriden by more specific variables. Substituted into built files.

Default: /usr/local

TENCE_DESTDIR

Install location prefix. All installation paths are prefixed with this one. This variable is used only during the install step, and is not substituted into any files.

Default: '' (empty string)

TENCE_BINDIR

Installation directory for executable programs.

Default: ${TENCE_PREFIX}/bin

TENCE_DOCSDIR

Installation directory for reST and HTML documentation.

Default: ${TENCE_PREFIX}/share/doc/testilence

TENCE_MAN1DIR

Installation directory for section 1 manual pages.

Default: ${TENCE_PREFIX}/man/man1

TENCE_SITEPKGDIR

Installation directory for the Testilence and tence libraries.

Default: ${TENCE_PREFIX}/share/pear

TENCE_TESTDIR

Installation directory for Testilence's test suite.

Default: ${TENCE_PREFIX}/share/pear/tests/Testilence

TENCE_PHP

Path to the PHP CLI. This value has two uses: it is substituted into tence, and used in the check targets to run the tests.

Default: /usr/bin/env php

Use the help target to display builtin help.

% make help
these targets display help for individual topics:

help-targets:  help for targets
help-vars:     help for variables
%

Functionality not described above or in the builtin help should be considered implementation details. If the documented features don't support all your configuration needs, please do send a feature request.

4.2.2.1   Running the test suite

Testilence's own test suite can be exercised with or without installed Testilence by running make check or make check-all in the source directory. The former target excludes tests that use filesystem or other external resources, because a bug in Testilence might prevent proper cleanup or (although unlikely) delete your files. The tests will be run using src/tence/tence (will be built if necessary).

The test suite is described in more detail in Test Suite.

5   Test suite

The suite is organized around Testilence's classes, meaning there is one or more cases per class under test. These are in turn organized into suites, possibly multiple levels of them. Any part of the resulting tree can be run.

The top level suite is called tencetest_AllTests. This suite contains tencetest_SafeTests and tencetest_DangerousTests.

Source code for the complete suite can be found in the tests [25] subdirectory in the source distribution.

6   Reference

6.1   Overview

Testilence consists of a library and a command line test runner. The library is perfectly usable without the tool.

6.2   Command line interface

tence, the command line frontend, is documented in the accompanying man page.

Source code for the command line tool can be found in the src/tence [26] subdirectory in the source distribution.

6.3   Library

Testilence Library Reference [27] describes public interfaces and behaviors.

Source code for the library can be found in the src/Testilence [28] subdirectory in the source distribution.

7   Examples

Testilence Usage Examples [29] illustrate usage of Tence_TestCase [30] and Tence_TestSuite [31] in writing tests, and tence for their execution.

8   Releases

8.1   Release types

Testilence releases fall into one of the following four categories:

  • snapshots
  • bugfix releases
  • new functionality
  • backward-incompatible change

8.2   Release versioning

Each release is tagged with a version number string with the following structure (ABNF [32]):

version-string = compat-cnt "." newfun-cnt "." bugfix-cnt [rel-candidate]
compat-cnt     = counter
newfun-cnt     = counter
bugfix-cnt     = counter
rel-candidate  = "." snapshot counter
; "snap"
snapshot       = %x73.6e.61.70
; counter is a positive integer (includes 0)
counter        = 1 * DIGIT

For each release of any type the appropriate counter is incremented by at least one and counters to the right of it are reset to 0 (the release-candidate part is removed altogether), while counters to the left of it are left at their current values.

Version string of the first nonsnapshot Testilence release is 0.0.0, while "0.0.0.snap135" is the version string of a pre-0.0.0 snapshot. There may be gaps between.

This handling is consistent with the way FreeBSD's pkg_version -t [33] interprets version strings.

[1]http://opensource.org/docs/definition.php
[2]http://www.php.net/
[3]http://opensource.org/licenses/mit-license.php
[4]http://www.junit.org/
[5]http://www.xprogramming.com/testfram.htm
[6]docs/reference.rest#test-methods
[7]docs/reference.rest#setup-and-teardown
[8]http://beer.sigpipe.cz/
[9]docs/reference.rest#overview
[10]http://www.scons.org/
[11]http://docutils.sourceforge.net/
[12]http://pecl.php.net/package/operator
[13]http://php.net/manual/en/ref.pcntl.php
[14]http://php.net/manual/en/ref.posix.php
[15]http://pecl.php.net/package/runkit
[16]http://www.xdebug.org/
[17]http://pear.php.net/package/Console_Getopt
[18]http://subversion.tigris.org/
[19]https://svn.sigpipe.cz/r/
[20]http://www.viewvc.org/
[21]http://svn.sigpipe.cz/
[22]http://people.freebsd.org/~kientzle/libarchive/
[23]http://www.bzip.org/
[24]http://www.testilence.org/dist/
[25]tests/
[26]src/tence/
[27]docs/reference.rest
[28]src/Testilence/
[29]docs/examples.rest
[30]docs/reference.rest#tence-testcase
[31]docs/reference.rest#tence-testsuite
[32]http://tools.ietf.org/html/rfc4234
[33]http://www.freebsd.org/cgi/man.cgi?query=pkg_version