Team learning session: surviving legacy code by J.B. Rainsberger

This is the description, and experience report, of an exercise I picked up many years ago and then used in a team I lead. I'll describe this in the present tense as I imagine applying the exercise in similar contexts. Credit to J.B. Rainsberger for coming up with this diabolical codebase and make us work on it!

Context

A team has been formed and has been working on a single product, for example a TypeScript monolith comprising a frontend and APIs.

The team is now tasked with taking over a set of services and frontends written in PHP (or some other programming language and ecosystem). It turns out this set of services is the flagship product of the organization, and it needs to support innovative business change. Yet, normal operations and incremental improvements had been mostly outsourced as part of the technical strategy for the last few years. 

The focus of innovation was elsewhere. Testing coverage is redundant, confusing or missing, depending on the area you are working on. Very few people in house know the traps of these codebases: we are now in the realm of legacy code.

Learning goal

The original exercise by J.B. Rainsberger is oriented to learning patterns to take over legacy code: inherited codebases that exhibit a lot of business value. If they don't have business value, then why working on it?

Specifically, the patterns regard:

• characterizing and understanding code
• testing it, at various level of scope from whole application to small units
• refactoring, with a safety net in place to support those changes without introducing regressions.

There isn't an emphasis on shipping new features, and there is no product owner.

To the original goals, we add a separate one here with its own new difficulty level: learn a new programming language that most of the team has not worked professionally in before, or has not picked up for a long time. With a new programming language, there's also knowledge of a new set of tools for running, testing, or linting the code. For PHP alone, in this example, the toolset ranges from Composer to PHPUnit, PHPStan, or PHP-CS-Fixer.

The learning at all these levels translate in being ready to use these patterns and tools on real projects, making it much easier and safer to deliver business value on those.

Activity

The original problem statement gives this guidance:

• Refactor this code to understand it
• [Decide] when to refactor and when to rewrite, and how to do that safely.
• lResolve the central conflict of legacy code: I need tests to refactor safely, but I need to refactor to write tests effectively.

I've been running these workshops back in the days of the Legacy Code Retreats, which is where the name came from. The setting diverges from a whole-day session, where iterations are handle by various pairs. In this case, the repository is forked at the beginning of the activity and several sessions can take place, for example with one or more hour per week reserved. The participants work together as an ensemble, but they could split into multiple rooms working independently if there are too many.

11? 12? What?!

A loose plan to follow can be:

1. run the application. Does everyone understand the problem domain and what problem the code solves?
2. introduce golden master testing for characterization. This step involves figuring out seeding and reproducibility, and how to achieve the isolation of automated tests from the outside world and its changing conditions.
3. introduce further testing at lower levels. This step involves introducing tooling for running code easily, testing it, or performing static analysis; manipulating the folders and file structure safely; and refactoring the code to isolate the units under test.

I act as a language expert here, but not with an agenda in mind. What to learn is decided by what moves the participants want to take, and what they are missing to be able to do so. It also pays off to prompt the team to research rather than trusting anyone's memory or word: tests and experiments are the source of truth.

The fact that this is a completely new, fictional, and ugly codebase should help with the feeling of safety when raising lack of understanding. The code is supposed to be hard to understand and fragile. Once that is established we can then work on our own learning, directed to improve our situation. It's a very different framing that delivering features on real legacy code with a deadline in mind.

Resources

The original code repository used in legacy code retreats.

The canonical explanation of a golden master approach to characterization testing.

Specifically for PHP, it helps to understand how seeding random number generation works to achieve reproducibility.

Retrospective

Add to a board two prompts to allow reflections to appear concurrently. For example:

• what did we learn today? Anything from a language construct that did not map easily to something I already knew, to a tool's use cases.
• any ideas for next time? There are lots of potential directions of exploration, and we want to crowdsource the gaps the participants are starting to see so we can fill them. This helps getting into context quickly when we start a new hourly session at another date.

Eris 0.11.0 is out

Eris 0.11.0 has been freshly released, and I'll be listing here various contributions that the project has received that are included in this new version and in the previous one, 0.10.0, which didn't have an associated blog post.

For a full list and links to the relevant pull requests and commits, see the ChangeLog.

0.10

• The Eris\Facade class was introduced to allow usage outside of a PHPUnit context.
• Official PHPUnit 7 support was introduced.
• Fixed a corner case in suchThat()

There are some small backward compatibility breaks with respect to 0.9; they regard unused features (or at least I thought) including Generator::contains().

0.11

• Official PHP 7.2 support
• Annotations support for configuring behavior that is usually configured through methods: @eris-method, @eris-shrink, @eris-ratio, @eris-repeat, @eris-duration

Some acknowledgements

Most of this work comes from contributions, not from me. I'd like to say a word of thanks to the people that have taken the time to use Eris in some of their projects but also to feed back a fix, an extension, or a substantial improvement.

Eris 0.9.0 is out

In 2016 I moved to another country and as a result of this change I didn't had much time to develop Eris further. Thankfully Eris 0.8 was already pretty much stable, and in this last period I could pick up development again.

What's new?

The ChangeLog for 0.9 contains one big new feature, multiple shrinking. While minimization of failing test cases is usually performed with a single linear search, multiple shrinking features a series of different options for shrinking a value.
For example, the integer 1234 was usually shrunk to 1233, 1232, 1231 and so on. With multiple shrinking, there are a series of options to explore that make the search logarithmic, such as 617, 925, 1080, 1157, 1195, 1214, 1224, 1129, and 1231. If the simplest failing values is below 617 for example, at least (1234-617) runs of the test will be skipped by this optimization, just in the first step.
This feature is the equivalent of QuickCheck's (and other property-based testing libraries') Rose Trees, but implemented here with an object-oriented approach that makes use of `GeneratedValueSingle` and `GeneratedValueOptions` as part of a Composite pattern.

This release also features support for the latest versions of basic dependencies:

• PHPUnit 6.x is now supported
• PHP 7.1 is officially supported (I expect there were mostly no issues in previous releases, but not the test suite fully passes.)

Several small bugs were fixed as part of feedback from projects using Eris:

• the pos() and neg() generators should not shrink to 0.
• float generation should never divide by 0.
• shrinking of dates fell into a case of wrong operator precedence.
• reproducible PHPUnit commands were not escaped correctly in presence of namespaced classes.

A few backward compatibility fixes were necessary to make room for new features:

• minimumEvaluationRatio is now a method to be called, not a private field.
• GeneratedValue is now an interface and not a class. This is supposed to be an internal value: project code should never depend on it and it should build custom generators with map() and other composite generators rather than implementing the Generator interface, which is much more complex.
• the Listener::endPropertyVerification() method now takes the additional parameters $iterations and the optional $exception. When creating listeners should always subclass EmptyListener in order not to have to modify the not implemented methods, which will be inherited.

What's next?

My Trello board says:

• still decoupling from PHPUnit, for usage in scripts, mainly as a programmable source of randomness.
• more advanced Generators for finite state machines and in general a more stateful approach, for testing stateful systems.
• faster feedback for developers, like having the option to run fewer test cases in a development environment but the full set in Continuous Integration.

I'm considering opening up the Trello board for public read-only visibility, as there's nothing sensible in there, but potential value in transparency and feedback from the random people encountering the project for the first time.

As always, if you feel there is a glaring feature missing in Eris, feel free to request it on the Github's project issues.

Eris 0.8.0 is out

In the period before my move to Cambridge I got some time to work on Eris, and to use it to test the guts of Onebip's infrastructure. Lots of new features are now incorporated in the 0.8.0 version, along with a modernization of PHP standards compliance carried out by @localheinz.

What's new?

Here's the most important news, a selection from the ChangeLog:

• The bind Generator allows to use the random output of a Generator to build another Generator.
• Optionally logging generations with `hook(Listener\log($filename))`.
• disableShrinking() option.
• limitTo() accepts a DateInterval to stop tests at a predefined maximum time.
• Configurability of randomness: choice between rand, mt_rand, and a pure PHP Mersenne Twister.
• The suchThat Generator accepts PHPUnit constraints like `when()`.

Some bugs and annoyances were fixed:

• No warnings on PHP 7 anymore.
• Fixed bug of size not being fully explored due to slow growth.
• Switched to PSR-2 coding standards and PSR-4 autoloading.

And there were some backward compatibility breaks (we are in 0.x after all):

• The frequency generator only accepts variadics args, not an array anymore.
• Removed strictlyPos and strictlyNeg Generators as duplicated of pos and neg ones.                                     
• Removed andAlso, theCondition, andTheCondition, implies, imply aliases which expand the surface area of the API for no good reason. Added and for multiple preconditions.

Eris is now quite extensible with custom Generators for new types of data; custom Listeners to know what's going on; and even different sources of randomness to tune repeatability and performance.
I believe what's very important about this release is the release of technical documentation. This is not a list of APIs generated by parsing the code, but is a full manual of Eris features, which will be kept up-to-date religiously in the repository itself and rebuilt automatically at each commit.

What's next?

My Trello board says:

• decoupling from PHPUnit: it should be possible to run Eris also with PHPSpec (already possible but not as robustly as it can be) or in scripts.
• Multiple possibilities for shrinking, borrowing from test.check rose trees. This feature may speed up the shrinking process and make it totally deterministic.
• A few more advanced Generators: for example testing Finite State Machines.

If you are using Eris and wanna give feedback, feel free to open a Github issue to discuss. 

PHPUnit_Selenium 2.0.0 is out

Here is the text of the change I have just merged to make a new major version of PHPUnit_Selenium a reality:

As signaled in #351, there are incompatibilities between the current version of PHPUnit_Selenium and PHPUnit 5.
It is a losing proposition to still support Selenium 1 API (SeleniumTestCase), as it redefines methods that have even changed signatures. It has not been maintained for years.
So to support PHPUnit 5.x there will be a new major version of this project, 2.x. The old 1.x branch will remain available but not updated anymore.
2.x will contain:

• Selenium2TestCase

and work with PHPUnit 4.x or 5.x, with correspondent PHP versions.
1.x will contain:

• SeleniumTestCase
• Selenium2TestCase

but will only work with PHPUnit 4.x, with correspondent PHP versions. In general, it will not be updated anymore. 

Supported PHP versions vary from 5.3 (!) to 5.6, according to the PHPUnit's version requirement.
Installation is available through Composer, as before.

Eris 0.4.0 is out

Eris is a PHPUnit extension for property-based testing, that is to say testing based on generating inputs to a system and check its state and output respect a set of properties. The project is a porting of QuickCheck and Eris is the name of the Greek goddess of chaos, since its aim is to break the System Under Test with unexpected inputs and actions.

I am planning a talk at the PHP User Group Milano and a longer blog post to introduce the general public to how property-based testing works. I held the same talk for a few friends at the phpDay 2015 Unconference.

Meanwhile version 0.4.0 is out, with the following ChangeLog:

• Showing generated input with ERIS_ORIGINAL_INPUT=1.
• names and date (DateTime) new Generator.
• tuple Generator supports variadic arguments.
• Shrinking respects when() clauses.
• Dates and sorting examples.

As for all semantic-versioned projects, the 0.x series should be considered alpha and no API backward compatibility is guaranteed on update.

Image credits

PHPUnit Essentials review

PHPUnit Essentials by Zdenek Machek is a modern and complete book about PHPUnit usage. I've been sent an electronic copy by Packt Publishing and am now reviewing it here.

The first thing that struck me about the book was the breadth of subjects: you start from mocks and command line options, to get even to Selenium usage. You have to know your tools and given PHPUnit being a standard, this is all knowledge that will accompany you for several years.

Every book on PHPUnit must be compared with the wonderful manual, to see what it adds to the picture with respect to the documentation. PHPUnit Essentials, in this respect, looks also at 3rd party libraries such as mocking libraries or "competitors" such as PHPSpec to enlarge the picture to the whole open source PHP landscape. This is something the documentations of single projects cannot do, and where a bit of opinionated advice can be taken.

There is a bit of what may seem outdated information in the book such as how to perform a PEAR-based installation, but it's identified as such (PEAR being deprecated and dismissed by the end of the year.) Another seemingly outdated tool is Selenium IDE, but once upgraded with a formatter for Selenium2TestCase like explained in this book it becomes usable again. This kind of advice demonstrates the real world experience of the author and makes you trust the content.

On the whole by reading this book you go in as a naive tester and you come out with lots of skills on using PHPUnit in different scenarios; so I would recommended it to programmers wanting to dive into testing PHP applications. Probably it's not worth a read for the medium-to-advanced users, for which most of the content is already known from PHPUnit manual or personal experience. After all the book's named Essentials, so it delivers all that you expect from the title in a convenient single package.

The PHPUnit Refcard is out

My PHPUnit Refcard is now available for download. Hope you enjoy it.

Do you occasionally forget the syntax for creating Mocks and Stubs in PHPUnit? Or wonder how a command line option work? Or if there is an assertion that may be perfect for the test you're writing? Then this Refcard can help you.

The Refcard is a PDF cheatsheet of 6 pages, which interleaves code samples with explaining test and sample outputs. Getting to know PHPUnit to use it at its full potential will be a little easier now. :)

PHPUnit MockBuilder in master branch

A quick note: my MockBuilder class has been integrated by Sebastian Bergmann in the master branch of phpunit. The name MockBuilder reflects the nature of this addition, a Builder pattern for the mock objects, more than the original MockSpecification does.
He said it should be included in PHPUnit 3.5, the next minor version. I'll soon add some documentation by forking the phpunit-documentation repository.

Refactoring PHPUnit's getMock()

Not an actual refactoring, but at least the introduction of a layer of indirection, a Parameter object, called PHPUnit_Framework_MockSpecification. I have already written the patch in a branch of my github repository. They are actually two independent patches, since PHPUnit core and the mocking component are in two separate repositories:
http://github.com/giorgiosironi/phpunit/commit/c7d62874ff9c1ed6f520e98cab2568c9bb933ec6
http://github.com/giorgiosironi/phpunit-mock-objects/blob/mock-specification/PHPUnit/Framework/MockSpecification.php
http://github.com/giorgiosironi/phpunit-mock-objects/blob/mock-specification/Tests/MockSpecificationTest.php
All functionalities were Test-Driven Developed.

Use cases
The current API of getMock(), the Facade for the mocking library, actually prescribes 7 parameters. Most of them are optional, like in use case (a):

$this->getMock('MyClass');

But if you want to specify an uncommon parameter, you have to include the previous ones, and hunt around for their default values, praying that you will get them right and insert the boolean or empty values in the correct order, like in (b):

return $this->getMock('MyClass', array(), array(), '', false);

In some cases (c):

$this->getMock('MyClass', array(), array(), '', true, true, false);

A Specification objectBuilder pattern, which I intend to propose as a feature request after getting some feedback from the community, will aid some of these use cases. For example a) remains the same: there is no need to complicate the API here.

$this->getMock('MyClass');

For case b):

$this->getMockSpecification('MyClass')
     ->disableOriginalConstructor()
     ->getMock();

For case c):

$this->getMockSpecification('MyClass')
     ->disableAutoload()
     ->getMock();

State of development
I have currently implemented support for 6/7 of the getMock parameters in the MockSpecification object (only the autoload-related parameter is missing). This solution is an instance of the Builder pattern (I need a new name for MockSpecification, which started out as a parameter object but then acquired a getMock() method for a faster access to the created object).
Once the mock is created, it behaves exactly like an ordinary mock: MockSpecification calls getMock() internally.
This would be a totally backward compatible change, since it only adds a new way to creating a mock.

What I want from you
Any feedback, from glitches in the code to better names for the API methods and the class itself. I guess PHPUnit_Framework_Mock_MockBuilder can be the right name. Once tidied up the code, I'll open a ticket for a feature request on PHPUnit's trac asking to assess it and merge in the master repository.

Acceptance Test-Driven Development

I am halfway through reading Growing object-oriented software, guided by tests, a book that teaches Test-Driven Development in a Java environment. A review will come soon, since the process described in this work is really language-agnostic and interesting also for php developers.
However, the book's authors introduce a very productive practice, which consists in a double cycle of TDD:

• a longer cycle, where you write acceptance (aka end-to-end) tests, deriving them from the user stories or formal requirements, and make them pass;
• a shorter cycle contained in the first, which happens in the phase when an acceptance test is red: you write unit tests and make them pass until the related acceptance test does not fail anymore.

This approach is an implementation of Acceptance Test-Driven Development, and in particular makes you write several unit tests for every acceptance test (read for every feature) you want to add. Acceptance testing gives immediate feedback on the application's external qualities: simplicity of use and setup, consistency of the interface. At the same time, unit testing gives feedback on the internal qualities: decoupling, cohesion, encapsulation.
When I started employing the double cycle, getting in the zone suddenly became less difficult. The advantages of the TDD process were for the first time applied to the whole process, from the requirements formalization to the end of a feature's development:

• test-first paradigm. By the end of the development phase, regression tests will be already in place, and the production code will be forced to be testable.
• The definition of "done" is very clear (the acceptance test passes), and you are more likely to write only the mandatory code to get a green bar at the higher level.
• measuring progress is easy: the number of acceptance tests that are satisfied (weighted by points). You can even write a set of acceptance tests for the whole iteration in advance and keep them in a separate suite, moving them in the main suite when they start to pass.

To be a bit more specific, the php technologies I use for the two cycles of development are Zend_Test for the acceptance tests suite and plain old PHPUnit test cases for the unit tests one.
Zend_Test is an extension to PHPUnit that lets me define a contract for the http interface of a Zend Framework application, assert redirects, check parts of the html response via css selectors, and even falsify request headers to simulate ajax requests. The unit tests usually have no dependencies on a particular infrastructure, so PHPUnit itself is a powerful enough tool to write them with.
By the way, triting an automated acceptance test suite is more difficult than writing unit tests, as there is more infrastructure that gets in the way and a large amount of details that may render the tests brittle. Fortunately Zend_Test takes care of almost all the infrastructure (aside from the database, which I reset in the setUp phase of test cases), and acceptance tests code can and should be refactored to avoid duplication of the implementation details. For instance, Css selectors used to assert on parts of the html response can be stored in constants with an expressive name, and the request creation can be wrapped in helper methods that are composed in actual test ones. Also custom made assertions are helpful in keeping the noise to a minimum.

I hope this technique will be useful for all test-infected developers. It certainly enhanced my productivity and will to get a green bar. :)

Clever Mock Objects with PHPUnit

PHPUnit is the standard testing framework for php applications, and it has native support for mocking. It can produce various Stubs and Mocks by automatically extending a class or implementing an interface. Expectations can be set on these mocks so that assertions will be made on parameters and fake return values will be produced when particular methods are called (for a full explanation of mocks' utility see my free ebook.)
One thing I think PHPUnit lacks is support for multiple expectations. Let's fix ideas with a practicalfanciful example.

Imagine that you have a Telescope class (or interface) which defines an observeColor() method. When you call this method with the name of a planet or star, say Earth, Jupiter or Sun, it returns Blue, Grey, Yellow depending on what its lenses find out. I imagine these variables as plain strings to keep the discussion simple, but using ValueObjects do not change much the context.
The system under test is a class which composes Telescope as a collaborator. Given a list of objects in space, it does some calculations which involve their colors. Orienting a Telescope everytime you run the test suite is not practical, and the planets may not even be visible some times, thus you are forced to mock out Telescope (of course this situation is totally made up, but dependency management is a fundamental part of object-oriented design.)
To accomplish its responsibilities, the SUT needs to call observeColor() with a small set of different parameters and to obtain different results, depending on the particular test case. This is in fact a very common situation in testing.
As far as I know PHPUnit does not have a out-of-the-box support for defining multiple expectations on a mocked method without generating a conflict: if you set up expectations and constraint on method parameters, they must be satisfied in every single call to the mocked method. Similarly, if you set up a fake return value for the method, it will be hardcoded as it cannot depend on the input parameters.
I will present now a solution involving a callback anonymous function which works for a one-parameter mocked method. Only the mock setup and verification part is included, as the hypothetical SUT is not important in this explanation.
Note that anonymous functions are available only on php 5.3, so for previous versions of php you should create a small class with only one method, and pass array($object, 'methodName') as the callback.

<?php

class MultipleExpectationsTest extends PHPUnit_Framework_TestCase
{
    /**
     * This does not work. Multiple with() and will() calls
     * set up conflicting expectations and behaviors.
     */
    public function testMultipleExpectationsCanBePutOnTheSameMethodNatively()
    {
        $mock = $this->getMock('Telescope');
        $mock->expects($this->any())
             ->method('observeColor')
             ->with('Earth')
             ->will($this->returnValue('Blue'));
        $mock->expects($this->any())
             ->method('observeColor')
             ->with('Mars')
             ->will($this->returnValue('Red'));
        $this->assertEquals('Blue', $mock->observeColor('Earth'));
        $this->assertEquals('Red', $mock->observeColor('Mars'));
    }

    /**
     * We tune the expectation by passing it to a custom method.
     */
    public function testMultipleExpectationsCanBePutOnTheSameMethodViaACallback()
    {
        $inputs = array('Earth', 'Mars');
        $outputs = array('Blue', 'Red');
        $mock = $this->getMock('Telescope');
        $expectation = $mock->expects($this->exactly(2))
                            ->method('observeColor');
        $this->setMultipleMatching($expectation, $inputs, $outputs);
        $this->assertEquals('Blue', $mock->observeColor('Earth'));
        $this->assertEquals('Red', $mock->observeColor('Mars'));
    }

    /**
     * A callback is built and linked to the mocked method.
     */
    public function setMultipleMatching($expectation,
                                        array $inputs,
                                        array $outputs)
    {
        $testCase = $this;
        $callback = function() use ($inputs, $outputs, $testCase) {
            $args = func_get_args();
            $testCase->assertContains($args[0], $inputs);
            $index = array_search($args[0], $inputs);
            return $outputs[$index];
        };
        $expectation->will($this->returnCallback($callback));
    }
}

interface Telescope
{
    public function observeColor($name);
}

Update: in recent versions PHPUnit includes a $this->returnValueMap() option for will() that performs this job. Apparently it was inspired by this post.

Practical Php Testing exercises

As you probably know there is a Creative Commons licensed ebook available on this blog, Practical Php Testing.
Tomaž Muraus wrote to me yesterday about solving the TDD exercises contained in the various chapters:

I had some time during the past few days so I read your book and solved the exercises found in the book.
I'm pretty sure not all of my solutions are the best and some could probably be improved / changed so I decided to create a Github repository with my solutions (http://github.com/Kami/practical-php-testing-exercise-solutions) so others can fork it and improve / refactor my solutions.

I hope you can find these examples useful, but try not to read a solution before actually trying to solve an exercise. I have not proof read this code but it seems that its quality is good. There is always room for improvement, especially in the stubs section of the tests, and in refactoring of the production code.

PHPUnit and Phing cohabitation

During the publication of Practical Php Testing some readers asked me to include information on how to make PHPUnit and Phing work together. It was not possible due to time constraints to include an appendix on this topic, so I will talk about it here.

First, some background:

• PHPUnit is the leading testing harness in the php world: it consists in a small powerful framework for defining test cases, making assertion and mocking classes.
• Phing is an Ant clone written in php, that should become the standard solution for automating php applications targets such as deployment, running different test suites at the same time and generating documentation. Why using Phing instead of Ant? Because it interfaces well with php applications.
  

Integrating these two tools means giving Phing access to a PHPUnit test suite and letting the phing build files, which manage configuration, contain also information on how to run the test suite. In the build.xml file of an application you should find different targets like generate-documentation, test-all, compile-all (if php were a compiled language), and so on.

There are two ways for accessing PHPUnit test suites via phing: exec and phpunit tasks.
At the time of this writing, the phpunit task bundled in stable releases of Phing lacks functionalities, primarily the ability to define a bootstrap file to execute before the test suite is run. I can't live without --boostrap and I look forward to a release of Phing that lets me specify this file in the configuration.
This release will be Phing 2.4.0 (at least a Release Candidate 3 version, while on December 2009 it is in RC2). There are two things that are being fixed and that would annoy the average developer a lot:

• There was a bug in the last RC release affecting the bootstrap parameter, and the inclusion took place too late in the process, producing fatal errors when the suite for instance relies on autoloading. This bug is fixed in the repositories and will be gone in the next RC release. I downloaded the simple patch and applied it manually to try out the bootstrap functionality and it works very well. (http://phing.info/trac/ticket/378)
• The summary formatter is not a summary: it uses the wrong hook method, producing a report for every single test case and resulting in an output hundreds of lines long. I opened a ticket to tackle this issue. (http://phing.info/trac/ticket/401)

What we will be able to do

<target name="test">
    <phpunit bootstrap="tests/bootstrap.php">
        <formatter type="summary" usefile="false" />
        <batchtest>
            <fileset dir="tests">
                <include name="**/*Test.php"/>
            </fileset>
        </batchtest>
    </phpunit> 
</target> 

When you push the big test button on your desktop (from the cli type phing test), this xml configuration will hopefully produce a report while your test suite runs.
The problems with this approach are it does not work yet due to the bugs I have listed earlier, and that it eats quite a bit of memory, forcing me to increase the limit to 128 Megabyte for a suite composed of 144 unit tests.

What we do now
Until a stable version of Phing 2.4 is released, we should rely on exec commands, which directly call the phpunit binary executable (not so binary: it is in fact a php script):

   <target name="test">
        <exec command="phpunit --bootstrap tests/bootstrap.php 

--configuration tests/phpunit.xml --colors"

dir="${srcRoot}" passthru="true" />
        <exec command="phpunit --bootstrap=example/application/bootstrap.php  

--configuration example/application/tests/phpunit.xml --colors"

dir="${srcRoot}" passthru="true" />
    </target>

$srcRoot is a property that specifies the working directory to run the phpunit command in. passthru makes the task echo the output of the command.
This approach is sometimes more flexible than using the specialized phpunit task. More flexible in the sense that you don't have to expect that phing includes in its tasks options for configuring new phpunit features, because you can use them just as they are available from the command line. On the other hand, it may be difficult to perform different actions (like lighting up a red semaphore in your office) basing on the last build state (red or green).

So I'm relying on exec tasks for now. By the way, the result is pretty and colors are even conserved, but I have to expect the end of the exec command to see any output (no dots slowly piling up on the screen).
If you enjoy using Phing and PHPUnit, please provide feedback and contribute to the projects, especially in the case of Phing. It is a project that deserves more attention from the community due to its integration tasks.
UPDATE: Phing 2.4.0 was released on January 17, 2010.

Practical Php Testing is here

Download now

Practical Php Testing, my ebook on testing php applications, is finally here as promised, in the first days of December.

How many times in the last month have you seen a broken screen in the browser? How many times did you have to debug in the browser, by looking at the output, inserting debug statements and breaking redirects? How many times did you perform manual testing, by loading a staging version of your application and tried out different workflows in the browser?

If the answer to these questions is more than very few, it's likely that
you should give automated testing a chance.

This book is aimed to php developers and features the articles from the Practical php testing series, while the other half of it is composed by new content:

• bonus chapter on TDD theory;
• a case study on testing a php function;
• working code samples, some of whom were originally kept on pastebin.com;
• sets of TDD exercises at the end of each chapter;
• glossary that substitutes external links to wiki and other posts, to not interrupt your reading with terms lookup.

The book comes for free and is licensed under Creative Commons. This phrase means you are free to copy it and give it to anyone. If you find my work useful and you want to be supportive, you can make a donation with the link on the right menu or with the one provided in the book.

Mocking and template methods

As you probably know, stubbing or mocking is a practice used in unit testing where a class methods are substituted via subclassing with test-friendly versions of themselves. The difference between stubbing and mocking resides in the place where the assertions are made, but it is not the main topic of this post.
The need for small and cohesive interfaces is particularly perceived while mocking a class. We typically want to test in isolation a unit and write mocks for its collaborators without going mad.
Let's see an example of a class I may want to mock:

class NakedEntity
{
    public function getMethods()
    {
        return $this->_class->getMethods();
    }

    public function getMethod($name)
    {
        $methods = $this->_class->getMethods();
        return $methods[$name];
    }
    
    public function hasMethod($name)
    {
        $methods = $this->_class->getMethods();
        return isset($methods[$name]);
    }

    // other methods, constructor...
}

As I said earlier, mocking is effective if there is a small interface to mock. Note that every class defines an implicit interface: the set of its public methods. Sometimes the interface comprehends several methods that give access to the same data or behavior, and that have to be present to avoid abstraction inversion. In this particular case, if I defined only getMethods() to conserve a small and cohesive interface, every class that depends on NakedEntity would have to implement the other two missing methods.
Mocking all three methods of NakedEntity in phpunit means writing this:

$mock = $this->getMock('NakedEntity');
$mock->expects($this->any())
     ->method('getMethods')
     ->will($this->returnValue(array('doSomething' => ..., 'foo' => ...)));
$mock->expects($this->any())
     ->method('getMethod')
     ->will($this->returnValue(...));
$mock->expects($this->any())
     ->method('hasMethod')
     ->will($this->returnValue(true));

Compare this to the creation of a real NakedEntity. I should definitely create a real object to save test code, but the unit tests will then not be executed in isolation and I will have to define a mocked NakedClass object (the $this->_class property) and break the Law of Demeter.
Moreover, the mocking capabilities of phpunit are limited and for example we cannot define different return values based on the parameters (a real subclass is needed in that case) without a callback. I could mock only the methods effectively used from the SUT, but I don't really know which of them are really called (since they are more or less equivalent) and I want to refactor the SUT without changing the tests.
So I found a 2-step alternative solution.

Step 1: convenience methods become template methods
I started with refactoring the NakedEntity class:

class NakedEntity
{
    public function getMethods()
    {
        return $this->_class->getMethods();
    }

    public function getMethod($name)
    {
        $methods = $this->getMethods();
        return $methods[$name];
    }
    
    public function hasMethod($name)
    {
        $methods = $this->getMethods();
        return isset($methods[$name]);
    }

    // other methods, constructor...
}

The users of getMethods() are now template methods, and the base method (primitive operation in design patterns jargon) can be subclassed to provide alternative behavior. The subclass can be implemented as a real reusable class, which will include a setMethods() utility method (no pun intended), or via mocking.

Step 2: mock the base method
Now only getMethods() need to be substituted:

$mock = $this->getMock('NakedEntity', array('getMethods'));
$mock->expects($this->any())
     ->method('getMethods')
     ->will($this->returnValue(array('doSomething' => $myMethod, 'foo' => ...)));

$this->assertEquals($myMethod, $mock->getMethod('doSomething')); 

This approach works well because the contract of NakedEntity is already cohesive and the different methods provide different ways to do the same thing. The template methods contain nearly no logic and they are exercised in unit tests which are not their own: it is a very small trade-off because it is highly improbable that they will break and cause another class unit tests to fail without reasons. The template methods in this case are only glue code.
Don't use this testing pattern as an excuse to write many public methods: you should indeed break up a class in different units if its contract grows too much. You can implement a Decorator pattern if convenience template methods are implementing business logic on a public method, or it may be the case that your class is doing too much and the Api is too complicated. Another viable solution if you have an explicit interface instead of a concrete class is creating a reusable Fake implementation which will contain the convenience methods as well.
In conclusion, if you have a contract with many cohesive and dumb methods, which relies on a central one to provide data, you can create template methods and reuse them in other unit tests, via subclassing of the primitive operations.

Php dependencies, require_once() and namespaces

Before the introduction of autoloading, every php class had to be explicitly included before being available, referencing its physical file. Autoloading has decimated this approach but there is also an advantage in using require_once(), the principal construct for inclusion of php class sourcefiles. This pro is pointing out dependencies to other classes and interfaces instead of sweeping them under the carpet.

Back in 2002 and php 4, you could not do things like this:

<?php
$db = new Zend_Db();

When the class Zend_Db is requested, for example to instantiate an object, its name is passed to an autoload function or method and its file included to give the class a chance to be defined before using it. If the autoloading process does not find a class, a fatal error is raised so it's not a little problem you can ignore by tuning the error_reporting level.
However, if you are not using autoloading the following code will be familiar:

<?php
require_once 'Zend/Db.php';
$db = new Zend_Db();

The require_once() call performs a manual inclusion of the Zend_Db class definition file, making Zend_Db available in all scopes. require_once() arguments are tipically relative paths from the list of directories defined in the include_path. Note that require_once() will not include the same physical file more than one time and that it will throw a fatal error too if it does not find the file. Since it is a language construct and not a function, parentheses are not needed.
In a typical project which does not rely on autoloading (such as Zend Framework and PHPUnit), every class file contains a list of require_once() calls at the top:

<?php
/* [license...] */
/** Zend_Filter */
require_once 'Zend/Filter.php';

/** Zend_Validate_Interface */
require_once 'Zend/Validate/Interface.php';

/**
 * Zend_Form_Element
 * [docblock annotations...]
 */
class Zend_Form_Element implements Zend_Validate_Interface
{
    ...

This can be annoying: every time you reference a class you have to write its name, slightly modifying it to form the file name, and put a require_once() call at the top of the file you're working on. I'm sure there are automated tools for this process that scan the file and write require_once()s by themselves, but why adding more lines of code? Isn't maintaining less code better?
Of course it's better, unless the value the code adds to the application is worth the time for writing and maintain it. Otherwise there will never be new features in applications since no one would want to write new code.
The value provided here is declaration of dependencies. From the first part of Zend_Form_Element class file, I instantly learned that it has a dependency on Zend_Validate_Interface (obviously because this class implements it) and on Zend_Filter.
For example, these dependencies lists have been used for automatic generation of packages: you pick for example Zend_Form and a script packs all its dependencies in a zip you can download and decompress in your library folder. It would be great if it has no dependencies, but for a component to be useful a little coupling is mandatory.
During development, is always useful for current maintainers and for new programmers to learn the dependencies a class has. There is no need for static analysis tools in Zend Framework: simply open the file and see what is being included.
The problem is this methodology does not feature transitivity: Zend_Filter can be dependent on other classes and interfaces without them being listed in Zend/Form/Element.php; the inclusion of their files is placed in Zend/Filter.php and it's not a strange choice. Though, to test or develop a class you probably only need to know its direct dependencies (that's the usefulness of encapsulation) as they are classes which it communicates with by method calls and composition. In unit tests, for example, I'll mock out Zend_Filter and I won't care what it depends on.

Another problem is require_once() ties the dependencies to physical files. Unlike Java import statements, which references only a fully qualified class name such as Zend_Filter, we are including Zend/Filter.php; it is unlikely that this location will change in the future if the include_path is set correctly, but why saying something we do not intend? I really care only to state there is a dependency.
Php 5.3 can help us.
In an hypothetical version of Zend Framework with namespaces, the code of Zend\Form\Element will be as simple as:

<?php
namespace Zend\Form;
use Zend\Validate\ValidateInterface;
use Zend\Filter;

This form of collateral dependencies declaration was created to avoid name clashing. Its advantages are clear:

• it does not say anything on the file containing the referred class or interface;
• it is not needed for classes contained in the same namespace (like it was a Java package). Classes often depend on siblings and with require_once() statements we would be cluttering the list of external, important dependencies with the internal and obvious ones.
• it also shorts the names available for the collaborators: you can refer to Filter and ValidateInterface in the source file.
• it also declares what namespace/package the class belongs to, without recurring to docblock annotations.

I hope you now do not find so boring to write and encounter use statements (or require_once() calls), backed by autoloading. Their purpose is not only to help the php interpreter, which would not work otherwise. You can take advantage of them also to declare dependencies to other developers and automated tools: the next person that will start reading your code from scratch will be grateful.

Unit testing view helpers

The architecture of the Zend Framework, one of the most popular php frameworks, is very extensible and presents a lot of hooks for subclassing and new implementations. The problem which arises from time to time is the testability of the components produced: view helpers are an example of problematic testing and here I will write about solutions for unit testing them.

View helpers are a key point of the Mvc implementation in Zend Framework, along with the Zend_Controller component. The responsibility of view helpers is to keep programming logic out of the view scripts, which are rendered by the view object. We are talking about the presentational layer: every big of logic kept in a view helper can be reused in other scripts.
View helpers are instantiated on the fly by the view object (a Zend_View instance, or another Zend_View_Interface implementation) and kept there. This object will then include the scripts in a method, providing access to its scope to call fake methods with the name of view helpers (a __call() implementation). Since the scope of the object provides the $this handler, view scripts reference it to call view helpers:

<?php
echo $this->doctype();
?>
<html>
...

This script takes advantage of the Doctype view helper (a Zend_View_Helper_Doctype instance) to produce an html doctype declaration.
You can also write your own view helpers: according to the manual, they should provide an empty constructor to allow instantiation by the view object and a method which name corresponds to the class base name. For instance doctype() is the strategy method of Zend_View_Helper_Doctype.
The view helper class should implement Zend_View_Helper_Interface, which has the only injection point in this architecture, the setView() method.

The empty constructor is the problem in view helper management, since it gets in the way of simple test code when you write view helpers that make use of other view helpers as collaborators. For instance, I wrote yesterday a IconLoader helper which use the standard HeadStyle one to add css rules to the page.
The injection point which I was talking about, the setView() method, is called after instantiation. Once my IconLoader helper is instantiated, the view object injects itself in the helper using this method, providing a reference to other helpers.
In this design, the view acts as a Service Locator, and we have no idea which helpers could be called by another one: every helper class could depend on everything else.
Unit testing prescribes to test the helper class in isolation, substituting the collaborators with mocks or stubs. We have to put in test doubles as collaborators, otherwise we are testing more than one helper at the time and if the test fails we cannot tell if the problem is in the SUT (IconLoader) or in the referenced helpers (HeadStyle, ...). Note that collaborators can reference more collaborators, and soon IconLoader class can depend on the entire framework.

My very-simple-testing solution would be require helpers to specify their collaborators in the constructor or via setters, implementing Dependency Injection. We cannot change the standard Zend Framework architecture, though, but it's not a framework's fault. This solution would have required to build a small automatic dependency injection system, which is not in the scope of Zend Framework 1.x (but will be in 2.x as far as I know).

With the current architecture, we can make different choices to simplify testing (remember an helper's constructor must not have parameters, and we must test our components in isolation):

• Use the real view object and the real view helpers as collaborators. This is integration testing, and failures on the collaborators or view object or what else they refer to will make my main test fail too. Moreover, at every test you should bootstrap all the Zend Framework's Mvc system, so it's not a viable solution.
  
• Providing setters for collaborators. The view object cannot call these setters for us, so in the bootstrap we should require the view helper from the already set up view object and call the setters with the mandatory collaborators. In my example, I would have created a My_Helper_IconLoader::setHeadStyle() method. This is the simplest solution for testability, since we have only to call setters in the test for IconLoader view helper and passing in mocks; however, it requires to instantiate all view helpers which need collaborators at every page request, so it's a bit heavy but mandatory if the collaborators are not view helpers but other complex service classes. Starting to mock collaborators is right, though.
  
• Mocking the view object/Service Locator with phpunit. This can be done for one view helper, by setting an expectation on a (mocked too) view object __call() method. But when using multiple helpers as collaborator, phpunit cannot distinguish between calls with different parameters and decide which collaborator helper to return. It's an hack which would not work very well.
  
• Provide a Fake view object. This is my choice: write once a fake view class, which implements Zend_View_Interface but instead of creating view helpers only returns the ones set at construction time. The definition of a fake object is a class with a running implementation, but different from the production one, often a simplified implementation used for testing.
  

With my fake view object, the testing phase becomes very simple. In my setUp() method I have:

    $this->_headStyleMock = $this->getMock('Zend_View_Helper_HeadStyle', array('appendStyle'));
    $view = new View();
    $view->setHelper('headStyle', $this->_headStyleMock);
    $this->_helper = new IconLoader();
    $this->_helper->setView($view);

and after this injection, I can set expectations on $this->_headStyleMock and exercise IconLoader in the test methods.
You can find the View fake class at:
http://nakedphp.svn.sourceforge.net/viewvc/nakedphp/trunk/tests/NakedPhp/Stubs/View.php?revision=52&view=markup
along with some tests on it, which could help you grasp its usage:
http://nakedphp.svn.sourceforge.net/viewvc/nakedphp/trunk/tests/NakedPhp/Stubs/ViewTest.php?revision=52&view=markup
Obviously this fake class was Test-Driven Developed.

Feel free to ask any questions. I care about (unit) testability and using extensively frameworks can often make difficult the TDDer life.

Practical testing in php part 9: command line options

This is the ninth and last part of the php testing series. If you liked it, you should subscribe to the feed to check out similar articles in the future.

Optimizing a test suite by adding test methods and test cases can be useful to improve the quality of your application code. Yet, every optimization starts with a profiling phase, that tells you where there is a need for test cases and where there is already a good coverage.
Code coverage is defined as the ratio of lines of code exercised by the unit tests to the overall number of lines; the same ratio can be calculated using code blocks as the unit of measure.
Phpunit provides code coverage reports generation via command line switches: one of them is --coverage-html $directory which places a human-readable html report in the $directory specified. There are other formats available, such as Xml, created for the purpose of interpreting a report with a third party application.
Please note that phpunit code coverage features require the xdebug extension to work.

Another useful switch is the --configuration $file one. $file should be an xml configuration file that tells phpunit what files have to be considered as containing test cases. This is very handy to compose a suite and can substitute the famous and hard to mantain AllTests.php files.
Here is a simple example for a configuration file:

<phpunit>
    <testsuite name="Ossigeno Test Suite">
        <directory suffix="Test.php">tests/</directory>
        <directory suffix="Test.php">application/modules</directory>
    </testsuite>
</phpunit>

Running phpunit with this switch, instead of specifying a particular file, will force the runner to consider all php files which name ends in "Test.php" in the directories tests/ and application/modules/. While running a single test case gives as output a line of dots, running all these tests in sequence will result in multiple lines and in a list of all failed tests (although you can require the list of skipped and incomplete tests by using the --verbose switch) in the overall list generated according to the configuration.

Along with the --configuration option, I strongly suggest to use the --bootstrap $phpScript directive. Your test cases probably need a global bootstrap phase for autoloading and setting up the include_path or other options. In some old versions of phpunit, you had to include a require_once() call at the start of each test script to make sure it was executed before the test. Now you can simply tell phpunit to run a file of your choice before starting with the test phase.

Running an entire suite is a good practice to discover if your changes or refactorings have broken some functionalities. However, it's an overkill if you have to do it very often, like in a short feedback cycle for TDD: supposing you have more than one test case for your SUT, it can be useful to select all those tests and leaving out the rest of the suite.
This is the case when the @group annotation is handy. You can mark with the @group $name annotation the docblock of test case classes, and also add multiple lines if you feel the contained tests can be useful in more than one scenario. Then the --group $group command line switch excludes test cases which do not belong to $group from being run.

So we can finally give an example of running a test suite:

phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml

for instance we can restrict the selected tests to the NakedPhp_Form package ones:

phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml --group=NakedPhp_Form

or requiring a code coverage report to see where we need to add test code:

phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml --coverage-html directory/

I hope these tips will be useful to you for utilizing phpunit at its best. It is a very well-crafted tool that you can take advantage of for TDD purposes, and also for functional and integration tests. Although the name suggests unit testing as a goal, you should certainly include in your test suite some functional tests, which exercise a feature provided of more than one object, and integration tests, which covers the wiring of your object and verify that your application works on an end-to-end basis.

Bonus tip: using --no-globals-backup and --no-static-backup can speed up your tests execution by avoiding unuseful isolation of tests. If your application has no global state they will work correctly anyway.
If you liked this testing series, you should subscribe to the feed to be informed of new articles on software development and php.

Practical testing in php part 8: mocks

This is the eighth part of the php testing series. You may want to check out the previous parts or subscribe to the feed for being updated on new articles.

In the last part of this series, we have listed the various types of Test Doubles along with the ones that phpunit can easily generate: Stubs and Mocks. The latter are utilized in a different kind of testing than the one presented so far: behavior verification.

The behavior verification testing style differs from the state verification one in the subjects of the assertion methods. While state verification specifies explicit assertion methods to be called upon a test result, behavior verification is focused on checking the actions the system under test undertakes. These actions comprehend which methods it calls, and how many times it does so; but also the parameters it passes to these methods and their order.
The standard interaction with collaborators in object-oriented systems consists of method calls. This kind of testing prescribes to place assertions directly in the overridden methods of Test Doubles, or at the end of every test, to verify that the SUT behavior conforms to specifical rules. These Test Doubles, which can run assertions on their methods parameters, are called Mock Objects (or simply Mocks). The contraposition here is with Stubs, which extend the capabilities of a state based testing but do not make any assumption on method calls or parameters.
Note that the assertions on parameters are placed inside the generated methods, while assertions on method calls are executed by phpunit after the test has run. This means that in a pure behavior verification test you won't find any assert*() calls, which perform state verification.

Now we are going to rewrite the unit test of the previous part taking advantage of phpunit mocks generation, but with a mixed approach which contains also explicit  assertions. The test was about verifying that the GeolocationService class made use of a GoogleMaps collaborator to find out the latitude and longitude of an User object, and the key characteristic was insulation of the test from the GoogleMaps real implementation with a Test Double. You can find the Stub example here.

class GeolocationServiceWithMocksTest extends PHPUnit_Framework_TestCase
{
    public function testProvidesLatitudeAndLongitudeForAnUser()
    {
        $coordinates = array('latitude' => '42N', 'longitude' => '12E');
        $googleMapsMock = $this->getMock('GoogleMaps', array('getLatitudeAndLongitude'));
        $googleMapsMock->expects($this->once())
                       ->method('getLatitudeAndLongitude')
                       ->with('Rome')
                       ->will($this->returnValue($coordinates));
        $service = new GeolocationService($googleMapsMock);
        $user = new User;
        $user->location = 'Rome';
        $service->locate($user);
        $this->assertEquals('42N', $user->latitude);
        $this->assertEquals('12E', $user->longitude);
    }
}

The test is actually very similar to the Stub one, but there are some differences:

• the expect() method of the mock returns an expectation object with a fluent interface we can work with. However, this time a matcher is passed which specifies how many times the mocked method should be called. In the Stub example, the matcher used is $this->any(), that does not run any assertions on the number of calls at the end of the test. Other available matchers are $this->never() and $this->exactly($number). The power of the matchers used in xUnit frameworks is they augment the test's code readability, making it similar to plain English.
• On the expectation object, along with will() and method(), we are also calling with() to specify the parameter we want to check as passed to getLatitudeAndLongitude(). If we wanted to check more parameters as exact values, we would pass an array to with() containing the actual list. However, we can make also weak assertions by using constraints objects, like $this->attributeEqualTo($name, $value) or $this->isInstanceOf($className), or maybe $this->anything() if no assertion has to be made on a particular parameter.
• There is no formal definition that says Mocks can't return canned results, as this is often mandatory for the code flow and to complete the test successfully. Though, if you TDD the system under test using mocks without predefined results, it's likely that you will produce a class with a different programming style which works with those tests, and uses mocks very effectively.
  
• Whenever you write a with() call or a matcher in expects(), be aware you are building a Mock and not a Stub.

You can find the complete, running test case here on pastebin. I tried to include complete examples in this series to show the practical side of testing instead of tips which are great in theory, but fail to apply in a real situation.

After this example of behavior verification, which makes use of the most advanced phpunit features, we are ready to explore the code coverage features in the next part.

You may want to subscribe to the feed to be updated when new articles in this series are available.