Showing posts with label php testing series. Show all posts
Showing posts with label php testing series. Show all posts

Wednesday, January 20, 2010

Practical Php Patterns: Composite

This post is part of the Practical Php Pattern series.

One of the most important structural patterns is the Composite one: its goal is managing a hierarchy of objects where both leaf objects and composition of other objects conform to a common interface. This hierarchy is usually constituted by part-whole relationships (often composition in the strict sense or aggregation), and it can be viewed as a tree.
The power of the Composite pattern resides in a Client which refers only to a Component interface, the common abstraction between all kinds of tree elements, and it is  oblivious to changes in the underlying structure. The Client is not even aware of the hierarchical structure existence and it is passed a reference to the tree head.
Meanwhile, the Composite responsibility is to build on its Component children's operations, and propagate their calls towards the bottom of the hierarchical graph until they reach Leafs.

Participants:
  • Client: sends message to the head Component.
  • Component: declares the interface that various parts of the graph should respect.
  • Leaf: concrete class that has no children.
  • Composite: concrete class that composes other Components (no pun intended).
The code sample shows a Composite pattern in action to duplicate a small subset of Javascript's Document Object Model.
<?php
/**
 * Component interface.
 * The Client depends only on this abstraction, whatever graph is built using
 * the specializations.
 */
interface HtmlElement
{
    /**
     * @return string   representation
     */
    public function __toString();
}

/**
 * Leaf sample implementation.
 * Represents an <h1> element.
 */
class H1 implements HtmlElement
{
    private $_text;

    public function __construct($text)
    {
        $this->_text = $text;
    }

    public function __toString()
    {
        return "<h1>{$this->_text}</h1>";
    }
}

/**
 * Leaf sample implementation.
 * Represents a <p> element.
 */
class P implements HtmlElement
{
    private $_text;

    public function __construct($text)
    {
        $this->_text = $text;
    }

    public function __toString()
    {
        return "<p>{$this->_text}</p>";
    }
}

/**
 * A Composite implementation, which accepts as children generic Components.
 * These children may be H1, P or even other Divs.
 */
class Div implements HtmlElement
{
    private $_children = array();

    public function addChild(HtmlElement $element)
    {
        $this->_children[] = $element;
    }

    public function __toString()
    {
        $html = "<div>\n";
        foreach ($this->_children as $child) {
            $childRepresentation = (string) $child;
            $childRepresentation = str_replace("\n", "\n    ", $childRepresentation);
            $html .= '    ' . $childRepresentation . "\n";
        }
        $html .= "</div>";
        return $html;
    }
}

// Client code
$div = new Div();
$div->addChild(new H1('Title'));
$div->addChild(new P('Lorem ipsum...'));
$sub = new Div();
$sub->addChild(new P('Dolor sit amet...'));
$div->addChild($sub);
echo $div, "\n";
Some notes on the implementation:
  • evaluate if maintaining parent references in Component specializations would be a waste of time. This addition prevents sharing of objects as children of different Composites and makes the mutual reference management more complex.
  • often also the remove() operation is not even required in php implementations, because of the transient nature of object graphs. It is rare to modify a tree unless if its objects are entities; service trees such as Chains of responsibility are built via configuration at the startup.
  • the only operation which is really mandatory, besides the ones present in the Component interface, is a way to specify the children, via an add() method or the constructor.
  • a Builder can encapsulate the mechanics of the construction process of the object tree.

Monday, December 28, 2009

Practical Php Testing exercises

As you probably know there is a Creative Commons licensed ebook available on this blog, Practical Php Testing.
Tomaž Muraus wrote to me yesterday about solving the TDD exercises contained in the various chapters:
I had some time during the past few days so I read your book and solved the exercises found in the book.
I'm pretty sure not all of my solutions are the best and some could probably be improved / changed so I decided to create a Github repository with my solutions (http://github.com/Kami/practical-php-testing-exercise-solutions) so others can fork it and improve / refactor my solutions.
I hope you can find these examples useful, but try not to read a solution before actually trying to solve an exercise. I have not proof read this code but it seems that its quality is good. There is always room for improvement, especially in the stubs section of the tests, and in refactoring of the production code.

Wednesday, December 02, 2009

Practical Php Testing is here


Practical Php Testing, my ebook on testing php applications, is finally here as promised, in the first days of December.
How many times in the last month have you seen a broken screen in the browser? How many times did you have to debug in the browser, by looking at the output, inserting debug statements and breaking redirects? How many times did you perform manual testing, by loading a staging version of your application and tried out different workflows in the browser?
If the answer to these questions is more than very few, it's likely that
you should give automated testing a chance.
This book is aimed to php developers and features the articles from the Practical php testing series, while the other half of it is composed by new content:
  • bonus chapter on TDD theory;
  • a case study on testing a php function;
  • working code samples, some of whom were originally kept on pastebin.com;
  • sets of TDD exercises at the end of each chapter;
  • glossary that substitutes external links to wiki and other posts, to not interrupt your reading with terms lookup.
The book comes for free and is licensed under Creative Commons. This phrase means you are free to copy it and give it to anyone. If you find my work useful and you want to be supportive, you can make a donation with the link on the right menu or with the one provided in the book.

Friday, October 02, 2009

Practical testing in php part 9: command line options

This is the ninth and last part of the php testing series. If you liked it, you should subscribe to the feed to check out similar articles in the future.

Optimizing a test suite by adding test methods and test cases can be useful to improve the quality of your application code. Yet, every optimization starts with a profiling phase, that tells you where there is a need for test cases and where there is already a good coverage.
Code coverage is defined as the ratio of lines of code exercised by the unit tests to the overall number of lines; the same ratio can be calculated using code blocks as the unit of measure.
Phpunit provides code coverage reports generation via command line switches: one of them is --coverage-html $directory which places a human-readable html report in the $directory specified. There are other formats available, such as Xml, created for the purpose of interpreting a report with a third party application.
Please note that phpunit code coverage features require the xdebug extension to work.

Another useful switch is the --configuration $file one. $file should be an xml configuration file that tells phpunit what files have to be considered as containing test cases. This is very handy to compose a suite and can substitute the famous and hard to mantain AllTests.php files.
Here is a simple example for a configuration file:
<phpunit>
    <testsuite name="Ossigeno Test Suite">
        <directory suffix="Test.php">tests/</directory>
        <directory suffix="Test.php">application/modules</directory>
    </testsuite>
</phpunit>
Running phpunit with this switch, instead of specifying a particular file, will force the runner to consider all php files which name ends in "Test.php" in the directories tests/ and application/modules/. While running a single test case gives as output a line of dots, running all these tests in sequence will result in multiple lines and in a list of all failed tests (although you can require the list of skipped and incomplete tests by using the --verbose switch) in the overall list generated according to the configuration.

Along with the --configuration option, I strongly suggest to use the --bootstrap $phpScript directive. Your test cases probably need a global bootstrap phase for autoloading and setting up the include_path or other options. In some old versions of phpunit, you had to include a require_once() call at the start of each test script to make sure it was executed before the test. Now you can simply tell phpunit to run a file of your choice before starting with the test phase.

Running an entire suite is a good practice to discover if your changes or refactorings have broken some functionalities. However, it's an overkill if you have to do it very often, like in a short feedback cycle for TDD: supposing you have more than one test case for your SUT, it can be useful to select all those tests and leaving out the rest of the suite.
This is the case when the @group annotation is handy. You can mark with the @group $name annotation the docblock of test case classes, and also add multiple lines if you feel the contained tests can be useful in more than one scenario. Then the --group $group command line switch excludes test cases which do not belong to $group from being run.

So we can finally give an example of running a test suite:
phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml
for instance we can restrict the selected tests to the NakedPhp_Form package ones:
phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml --group=NakedPhp_Form
or requiring a code coverage report to see where we need to add test code:
phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml --coverage-html directory/

I hope these tips will be useful to you for utilizing phpunit at its best. It is a very well-crafted tool that you can take advantage of for TDD purposes, and also for functional and integration tests. Although the name suggests unit testing as a goal, you should certainly include in your test suite some functional tests, which exercise a feature provided of more than one object, and integration tests, which covers the wiring of your object and verify that your application works on an end-to-end basis.

Bonus tip: using --no-globals-backup and --no-static-backup can speed up your tests execution by avoiding unuseful isolation of tests. If your application has no global state they will work correctly anyway.
If you liked this testing series, you should subscribe to the feed to be informed of new articles on software development and php.

Thursday, October 01, 2009

Practical testing in php part 8: mocks

This is the eighth part of the php testing series. You may want to check out the previous parts or subscribe to the feed for being updated on new articles.

In the last part of this series, we have listed the various types of Test Doubles along with the ones that phpunit can easily generate: Stubs and Mocks. The latter are utilized in a different kind of testing than the one presented so far: behavior verification.

The behavior verification testing style differs from the state verification one in the subjects of the assertion methods. While state verification specifies explicit assertion methods to be called upon a test result, behavior verification is focused on checking the actions the system under test undertakes. These actions comprehend which methods it calls, and how many times it does so; but also the parameters it passes to these methods and their order.
The standard interaction with collaborators in object-oriented systems consists of method calls. This kind of testing prescribes to place assertions directly in the overridden methods of Test Doubles, or at the end of every test, to verify that the SUT behavior conforms to specifical rules. These Test Doubles, which can run assertions on their methods parameters, are called Mock Objects (or simply Mocks). The contraposition here is with Stubs, which extend the capabilities of a state based testing but do not make any assumption on method calls or parameters.
Note that the assertions on parameters are placed inside the generated methods, while assertions on method calls are executed by phpunit after the test has run. This means that in a pure behavior verification test you won't find any assert*() calls, which perform state verification.

Now we are going to rewrite the unit test of the previous part taking advantage of phpunit mocks generation, but with a mixed approach which contains also explicit  assertions. The test was about verifying that the GeolocationService class made use of a GoogleMaps collaborator to find out the latitude and longitude of an User object, and the key characteristic was insulation of the test from the GoogleMaps real implementation with a Test Double. You can find the Stub example here.
class GeolocationServiceWithMocksTest extends PHPUnit_Framework_TestCase
{
    public function testProvidesLatitudeAndLongitudeForAnUser()
    {
        $coordinates = array('latitude' => '42N', 'longitude' => '12E');
        $googleMapsMock = $this->getMock('GoogleMaps', array('getLatitudeAndLongitude'));
        $googleMapsMock->expects($this->once())
                       ->method('getLatitudeAndLongitude')
                       ->with('Rome')
                       ->will($this->returnValue($coordinates));
        $service = new GeolocationService($googleMapsMock);
        $user = new User;
        $user->location = 'Rome';
        $service->locate($user);
        $this->assertEquals('42N', $user->latitude);
        $this->assertEquals('12E', $user->longitude);
    }
}
The test is actually very similar to the Stub one, but there are some differences:
  • the expect() method of the mock returns an expectation object with a fluent interface we can work with. However, this time a matcher is passed which specifies how many times the mocked method should be called. In the Stub example, the matcher used is $this->any(), that does not run any assertions on the number of calls at the end of the test. Other available matchers are $this->never() and $this->exactly($number). The power of the matchers used in xUnit frameworks is they augment the test's code readability, making it similar to plain English.
  • On the expectation object, along with will() and method(), we are also calling with() to specify the parameter we want to check as passed to getLatitudeAndLongitude(). If we wanted to check more parameters as exact values, we would pass an array to with() containing the actual list. However, we can make also weak assertions by using constraints objects, like $this->attributeEqualTo($name, $value) or $this->isInstanceOf($className), or maybe $this->anything() if no assertion has to be made on a particular parameter.
  • There is no formal definition that says Mocks can't return canned results, as this is often mandatory for the code flow and to complete the test successfully. Though, if you TDD the system under test using mocks without predefined results, it's likely that you will produce a class with a different programming style which works with those tests, and uses mocks very effectively.
  • Whenever you write a with() call or a matcher in expects(), be aware you are building a Mock and not a Stub.
You can find the complete, running test case here on pastebin. I tried to include complete examples in this series to show the practical side of testing instead of tips which are great in theory, but fail to apply in a real situation.

After this example of behavior verification, which makes use of the most advanced phpunit features, we are ready to explore the code coverage features in the next part.

You may want to subscribe to the feed to be updated when new articles in this series are available.

Wednesday, September 30, 2009

Practical testing in php part 7: stubs

This is the seventh part in the php testing series. You may want to checkout the other parts or to subscribe to the feed to be updated when new posts are available.

The name Unit testing explains the most special and powerful characteristic of this type of testing: the granularity of verifications is taken to the class size, which is the smallest cohesive unit we can find in an object-oriented application.
Despite these good intentions, tests often cross the borders of the single class because the system under test is constituted by more than one object: the main one which methods are called on (to verify the results returned with assertions) and its collaborators.
This is expected as no object is an island: there are very few classes which work alone. While we can skip the middle man on the upper side by calling methods directly on the system under test, there's no trivial way to wire the internal method calls to predefined result.
However, it's very important to find a way for insulating a class from its collaborators: while a small percentage of functional tests (which exercise an object graph instead of a single unit) are acceptable and required in a test suite, the main concern of a test-infected developer should be to have the most fine-grained test suite of the country:
  • tests are very coupled to the production code, in particular to the classes which they use directly. It's a good idea to limit this coupling to the unit under test as much as possible;
  • the possible interactions with a single object are a few, but integrating collaborators raises the number of situations that can happen very quickly;
  • the collaborators could be very slow (relying on a database or querying a web service) or not even worth testing because it is not our responsibility;
  • finally, if a collaborator breaks, not only its tests will fail but also the tests of the classes which use it. The defect could then be hard to locate.
In short, every collaborator introduced in the test arrangement is a step towards integration testing rather than unit testing. However, don't take this advice as a prohibition: many classes (like Entities/newables, but also infrastructure ones like an ArrayIterator) are too much of an hassle to substitute with the techniques described later in this post. You should definitely instantiate User and Group in tests of service classes which acts on them.

The verb substitute is appropriate: the only way to keep out collaborators from a unit test is to replace them with other objects. These objects are commonly known as Test Doubles and they are subclasses or alternative implementations respectively of the class or interface required by the SUT. This property allows them to be considered instanceof the original collaborator class/interface and to be kept as a field reference or to be passed as a method parameter, which are the only ways I can think of to recognize a collaborator.
Dependency Injection plays a key role in many unit tests: in the case of a field reference on a class, there is the need to replace this reference with the Test Double one, to hijack the method calls on the collaborator itself. Since field references are usually private, it is difficult to access them (requiring reflection) and violating a class encapsulation to simplify testing does not seem a good idea.
Thus, the natural way to provide Test Doubles for collaborators is Dependency Injection, being it implemented via the constructor or via setters. Instead of instantiating the production class, simply produce an object of its custom-made subclass and use it during the construction of the SUT. My SUTs usually have optional constructor parameters to allow passing in a null when the collaborator is not needed at all.

While entering the Test Doubles world, the average programmer hears many terms which describes substitutes of an object's collaborators. In crescent order of complexity, they are:
  • Dummies: object which do not provide any interaction, but are placeholders used to satisfy compile-time dependencies and to avoid breaking the null checks: no method is called on them but they are likely to be required parameters of a SUT method or part of its constructor signature. To avoid the creation of dummies I prefer to keep all constructor parameters optional, trusting that the Factory which creates the object passes regular collaborators instead of null values.
  • Stubs: objects where some methods have been overridden to return canned results. They may have more than one precalculated result available, depending on the method parameters combination, but these variables are known values once you reach the act part of the test method.
  • Mock objects: also known as Test Spies, mock objects are used in a testing style different from what we have worked on since the first part of this series (behavior based testing). They will be the main argument of the next episode.
  • Fakes: objects which have a working implementation, but much simpler than the real collaborator one. An in-memory ArrayObject which substitutes a database result Iterator is an example of a Fake.
You generally don't need to write a Dummy object since there is no interaction with it: the real collaborator can be used instead if its constructor is thin. A Fake is a running implementation so if an already existing class cannot work, there's usually no other choice than write a real subclass and reusing it in all the tests that require the collaborator.
For Stubs and Mocks the situation is different: there are plenty of frameworks for nearly every language which provide help in generating them, which take care of evaluating the subclass code and instantiating an object. Phpunit incorporates a small mocking framework, accessible through the method getMock() on the test case class. Remember that while the method is named getMock(), both Stubs and Mocks can be created via this Api. In this part we'll focus again on state verification and we'll use a Stub to improve the granularity of a test.

We are going to give a meaningful example of unit testing using a Stub. In this example, a GeolocationService takes a User object and fills its latitude and longitude fields using the location specified. GeolocationService requires an instance of a fictional GoogleMaps object to work, and since we all love Dependency Injection it is passed in its constructor.
Note that GoogleMaps can also be an interface or a base abstract class: there is no technical difference. Moreover, if it was a less important collaborator it can even be passed as a locate() parameter.
This is the test case:
class GeolocationServiceTest extends PHPUnit_Framework_TestCase
{
    public function testProvidesLatitudeAndLongitudeForAnUser()
    {
        $coordinates = array('latitude' => '42N', 'longitude' => '12E');
        $googleMapsMock = $this->getMock('GoogleMaps', array('getLatitudeAndLongitude'));
        $googleMapsMock->expects($this->any())
                       ->method('getLatitudeAndLongitude')
                       ->will($this->returnValue($coordinates));
        $service = new GeolocationService($googleMapsMock);
        $user = new User;
        $user->location = 'Rome';
        $service->locate($user);
        $this->assertEquals('42N', $user->latitude);
        $this->assertEquals('12E', $user->longitude);
    }
}
and here is the full, running example.
Note that I have created a Stub only for the external service and not for the User class. The former is external, slow, and unpredictable, but the latter is simple, with little or none internal behavior, and there's a small chance it will break. Nothing behaves like a String more than a String, as Misko says. The test method now focuses on exercising the locate() method and not also the GoogleMaps class.

Finally, let's take a look at the getMock() Api:
object getMock($originalClassName, [array $methods, [array $arguments, 
[string $mockClassName, [boolean $callOriginalConstructor, 
[boolean $callOriginalClone, [boolean $callAutoload]]]]]])
$originalClassName is the name of the class you want to create a Stub/Mock for. $methods is a numeric indexed array containing the names of the methods you want to subclass; if left empty, every method will be substituted. $arguments are arguments to pass to the constructor of the original class (almost never used), while $mockClassName is a custom name for the subclass created.
The last three arguments are boolean values used to determine if leaving the original constructor or clone method, or to allow autoloading of $originalClassName. They default to true. Often you want to set $callOriginalConstructor to false if its signature requires other collaborators to be passed in. All arguments but the first one are optional.
The mock produced, along with the original class methods, also has the expects() one available. For now, simply calling it with the argument $this->any() will do the job. This method returns a PHPUnit_Framework_MockObject_Builder_InvocationMocker instance; in short, an internal object that you can call method() and will() on to decide the method name to replace and its predefined behavior. The simplest possible behavior is $this->returnValue(...), but also $this->returnArgument($argumentNumber) is available, along with $this->returnCallback($callbackName); refer to the phpunit documentation for supplemental informations.

I hope this introduction to Stub objects has helped you grasping the essence of Unit testing in php. Feel free to ask clarifications in the comments. In the next part of this series, we will explore the possibilities of Mock objects and behavior based testing.

You may want to subscribe to the feed to remain updated on new posts in this series.

Tuesday, September 29, 2009

Practical testing in php part 6: refactoring and patterns


This is the sixth part of the php testing series. You may want to check out the other parts or to subscribe to the feed to remain updated on new posts.

In the xUnit world, tests are code. While there are testing tools which treats tests as data, phpunit and companions recognize classes and objects: this means that they are first class citizens and there should be no distinction in importance between production and test code.

Why it is important to refactor production code? To improve maintainability and ensure that changes which break the system appear less often. The same can be said for the tests: a suite that embrace change and is maintainable will make the developers actually use it, from the start to the long run. While the focus is usually on production code refactoring, today we will talk about test refactoring and the patterns where you should head to.
The worst thing that can happen is having an outdated test suite which fails because it is not maintained with production code: it will quickly lose credibility and thus it will be run sparingly, and then forgotten.
One of the best methodologies to improve production code maintainability is to test it: the more easy to test is a class, the more decoupled and maintainable it becomes. You will often find yourself refactoring a production class to simplify testing, for instance making Demeter happy, resulting in the application to have a simpler design.
Following our duality of production and test code, sometimes test methods and test cases grow and present a lot of repetition. What can be done to avoid these problems and maintain an agile (with the lowercase a) test suite is to refactor test code towards some patterns, some of them you already started to grasp in the previous parts of this series. Test code has often a low complexity compared to production code: it runs in an isolated environment, with nearly no state to maintain, with very decoupled classes (the test cases) and the help of a framework. Thus, it's tempting to use a lot of copy&paste to write tests, but knowing a bunch of patterns can flatten even tests little complexity and help you avoid duplication. As all patterns, they have been catalogued and given a standard name.
  • Standard Fixture and Fresh Fixture reuse the code which builds fixtures for the tests (and not the fixture itself). This pattern can be implemented with phpunit setUp() method.
  • Shared Fixture reuse the same object graph for a set of tests: obviously it should have no state or a way to reach a particular state for testing purposes. This pattern can be implemented with phpunit setUpBeforeClass() method.
  • Four Phase Test is the classical motif of a test method: arrange, act, assert and the optional teardown.
  • Test Runner and Test Suite Object are pattern which phpunit implements for you. You can then specifiy metadata to alter the building of a test suite or execution options, or specifical annotations which the runner supports.
  • State Verification is the simplest way of using phpunit and it's what we have done until now, writing assertions on explicit results of the system under test. Behavior Verification is based on making assertiong on indirect results, like method calls on collaborators and will be treated in the next parts of this series; Mock and Stub are patterns used in Behavior Verification, and phpunit provides support for their dynamic creation.
  • Table Truncation Teardown and Transaction RollBack TearDown are standard patterns for testing components which interact with a database.
  • Literal, Derived and Generated Value are patterns to provide fake data to the system under test. They all have their place in unit testing, depending on the unit purpose.
If you are interested in learning more about patterns you should check out the book xUnit Test Patterns: Refactoring Test Code and its website, which is a very complete guide to probably every single testing construct that has been explored in the xUnit frameworks. On the website you can find description and usage examples of all the patterns described here and of other specific ones.
Moreover, remember that test code is still code and the basic refactorings like Extract Method, Extract Superclass, Introduce Explaining Variable etc. are valid also in the testing land. Simply refactor some boilerplate code in private methods of a test case can save you the boring job of updating duplicated blocks.

As a side note, remember that when refactoring production code you have the safety net of the test suite, that will tell you when you have just broke something. No one tests the tests, however, and so you may want to temporarily break the behavior under test before refactoring a method or a test case. Simply altering the return statements of production methods can make the test fail so you can control that it continue to fail after the refactoring. When writing the original test, the TDD methodology crafts the method even before the production code exists and this is one of the main reason why the test is solid; a test is valid if it's able to find problems in the production code: that is, failing when it should fail.

I hope this series is becoming interesting as now you have learned your tests have the same importance of the production code. They can even be more important: if I have to choose between throwing away the production code and its documentation, and losing a good test suite, I will definitely throw away the first. It can be rewritten using the tests, while writing a complete test suite of an application without any tests is an harder task.
In the next parts, we'll enter the world of Test Doubles and of behavior verification, taking advantage of Mocks, Stubs, Fakes and similar pattern.

You may want to subscribe to the feed to remain updated about new articles in this series.

Monday, September 28, 2009

Practical testing in php part 5: annotations

This is the fifth parth of the php testing series. You may want to check out the other parts or to subscribe to the feed to be informed of new articles.

Now that we have learned much about writing tests (with or without fixtures) and using assertions, we can improve our tests further by exploiting phpunit features. This awesome testing tool provides support for several annotations which can add behavior to your test case class without making you write boilerplate code. Annotations are a standard way to add metadata to code entities, such as classes or methods, and are composed by a @ tag followed by zero, one or more arguments. While the parsing implementation is left to the tool which will use them, their aspect is consistent: phpDocumentor also collects @param and @return annotations to describe an Api.
Remember that annotations must be placed in docblock comments as in the php engine there is no native support for them: phpunit extracts them from the comment block using reflection.

While writing an Api or also a simple class, the corner cases and incorrect inputs have to be taken into consideration. The standard way to manage errors and bad situations in an oop application is to use exceptions. But how to test that a method raises an exception when needed? Of course the normal behavior is tested with real data that returns a real result. For the exceptional behavior, we can start with this test method:
    public function testMethodRaiseException()
    {
        try {
            $iterator = new ArrayIterator(42);
            $this->fail();
        } catch (InvalidArgumentException $e) {
        }
    }
The purpose of this code is to raise an exception by passing invalid data to the constructor of ArrayIterator, which requires an array. If the exception is raised accordingly, it bubbles up to the end of the try block and it is catched correctly, making the test pass. If the exception is not thrown, the call to fail() declares the test failed.
However, this paradigm will be repeated very often everytime you need to test an exception and so it can be abstracted away. Also, this code does not convey the intent of testing an exception since it is cluttered with details like an empty catch block and calls to fail().
Phpunit already abstracts away this code providing an annotation, @expectedException, which has to be put in the method docblock:
    /**
     * @expectedException InvalidArgumentException
     */
    public function testMethodRaiseExceptionAgain()
    {
        $iterator = new ArrayIterator(42);
    }
This code is much more clear than the constructs we used earlier. The only code present in the method is the one required to throw the exception, while the intent is described in the method name and in its annotations.

Another common repetition is testing a method with different kind of inputs, while executing always the same code. This is commonly resolved with a loop:
    public function testBooleanEvaluationInALoop()
    {
        $values = array(1, '1', 'on', true);
        foreach ($values as $value) {
            $actual = (bool) $value;
            $this->assertTrue($actual);
        }
    }
But phpunit can do the loop for you, taking advantage of the @dataProvider annotation:
    public static function trueValues()
    {
        return array(
            array(1),
            array('1'),
            array('on'),
            array(true)
        );
    }
    /**
     * @dataProvider trueValues
     */
    public function testBooleanEvaluation($value)
    {
        $actual = (bool) $value;
        $this->assertTrue($actual);
    }
This annotation should be followed by the name of a static method in the test case which returns an array of data sets to be passed to the test method. Phpunit will iterate over this array and using each one of its elements (which will be an array containing the arguments) to run the test method, telling you which data set was in use in case of a test failure. Of course you can put anything in the data sets: input for the SUT or expected result, or both.
The code becomes a bit longer, but the expressivity of defining the concept of different data sets in a standard way are worth considering.

The last common situation we will look at today is test dependency. Again, we are talking about dependency beneath the same test case since interdependencies between unit tests are a small of high coupling and should be raise suspects about your classes design.
It happens often that some test methods are more specific than the first you wrote and they will obviously fail if the formers do. The classic example is the add()/remove() tests on a container: to make sure remove() works you have to use add() for the arrange part of the test method. Phpunit solve this common problem of logic and temporal precedence (I won't present a workaround like in the other cases since it was not possible to solve this issue before phpunit 3.4 introduced @depends):
    public function testArrayAdditionWorks()
    {
        $array = array();
        $array[0] = 'foo';
        $this->assertTrue(isset($array[0]));
        return $array;
    }

    /**
     * @depends testArrayAdditionWorks
     */
    public function testArrayRemovalWorks($fixture)
    {
        unset($fixture[0]);
        $this->assertFalse(isset($fixture[0]));
    }
Not only testArrayAdditionWorks() is executed before testArrayRemovalWorks(), but since it returns something, this result is passed as an argument to the dependent method. If the former test fails, however, the dependent ones are marked as skipped as they will fail anyway by definition. They will clutter the output too, while it is clear that the functionality that needs repairment is the array addition.

I hope this standard phpunit annotations can help you enjoy writing tests for your php classes, leaving you the exciting work and taking off the boring one. In the next parts, we'll look at refactoring for test code before taking a journey with stubs and mocks.

I have uploaded on pastebin these code examples in a running phpunit test case. You may also want to subscribe to the feed to be informed of new posts in this series.

Saturday, September 26, 2009

Practical testing in php part 4: fixtures

This is the fourth part in the php testing series. You may want to subscribe to the feed to be informed when new posts are available.

In the previous parts, we have explored how to install phpunit and how to write tests which exercise our production code. Also we have learned to use the assertion methods to check the actual results: now we are ready to improve the test code from a refactoring point of view, and to take advantage of phpunit features.
While writing more and more test methods, you can notice that you follow a common pattern, commonly known as arrange-act-assert; this is the main motif of state based testing.
The first operation in a test is usually to set up the system under test, being it an object or a complex object graph; then the methods of the object are called during the act part and some assertions (hopefully not more than one) are done on the results returned from these calls. In some cases, when you have allocated external resources like a fake database, a final cleaning up phase is needed.
What you will actually discover is that often part of the arrange phase and the final cleanup code are shared between test methods: for example in case you are testing a single class, the instantiation of an object is a simple operation you can extract from the test methods. To support this extraction, phpunit (and all xUnit frameworks) provide the setUp() and tearDown() template methods.
These methods are executed respectively before and after every test method: default implementations are provided in PHPUnit_Framework_TestCase with an empty body. You can override this empty methods when useful, to have arrange/cleanup code to be shared between tests in the same test case and prepare a known state before every run. This known state is called a fixture.
Your test case class can go from this:
<?php
class ArrayIteratorTest extends PHPUnit_Framework_TestCase
{
    public function testSomething()
    {
        $iterator = new ArrayIterator(array('a', 'b', 'c'));
        // act, assert...
    }

    public function testOtherFeature()
    {
        $iterator = new ArrayIterator(array('a', 'b', 'c'));
        // act, assert...
    }
}
to this:
<?php
class ArrayIteratorwithFixtureTest extends PHPUnit_Framework_TestCase
{
    private $_iterator;

    public function setUp()
    {
        $this->_iterator = new ArrayIterator(array('a', 'b', 'c'));
    }

    public function testSomething()
    {
        // act on $this->_iterator, assert...
    }

    public function testOtherFeature()
    {
        // act on $this->_iterator, assert...
    }
}
Observe that, since an object of this class will be created to run the test, you can conserve every variable you want as a private member, and then have a reference to it available in the test method. setUp() usage provides a cleaner and Dry solution, and saves many lines of code when many test methods are needed.

Here is some know-how on using fixtures:
  • usually the tearDown() method should not be provided since the fixture is an object graph and will be garbage-collected after all the tests are executed, or overwritten by the next setUp() call. Thus, the empty body provided by default is often enough.
  • the fixture methods are executed for every test, so the test methods have the same state as a starting point. When more than one fixture is requested, the common practice is to break down the test case, preparing more than one test case class for the system under test; these classes represents different scenarios and together constitutes the overall test suite for this system.
  • sharing a fixture between test cases can be a smell for a bad design, since they are not insulated enough and classes know too much of each other. This cannot be done with setUp() methods however, but there are suite-level setup available in phpunit if you must share a fixture. However, keep in mind that you probably can refactor your classes to improve the maintainability of the application and of its test suite.
  • setUpBeforeClass() and tearDownAfterClass() are two hooks (static methods) which are executed before a test case methods are considered and after the overall process is finished. They are the equivalent of setUp() and tearDown(), but at the test case level instead of the test method one.
  • finally, assertPreConditions() and assertPostConditions() are two methods executed before and after the a test method. They differ from setUp() and tearDown() since they are executed only if the test did not already fail and they can halt the execution with a failing assertion. setUp() and tearDown() should never throw exceptions and they are executed anyway, unconcerned by the current test outcome.
This is all you must know on test fixtures to start experimenting with them. I hope your test code will be much more well written after introducing setUp().
In the next part, we'll explore the annotations that can influence phpunit test runner, like @depends and @dataProvider.

You may want to subscribe to the feed to be informed when new articles in this testing series will be available.

Friday, September 25, 2009

Practical testing in php part 3: assertions

This is the third part in the php testing series. You may want to checkout the previous parts or to subscribe to the feed to be notified of new posts.

Assertions are declarations that must hold true for a test to be declared successful: a test pass when it does not execute assertions or the one called are all verified correctly. Assertions are the final goal of a test, the place where you confront the expected and precalculated values of your variables with the ones that come from the system under test.
Assertions are implemented with methods and you have to make sure they are actually executed: thus, an if() construct inside a test is considered an anti-pattern as test methods should follow only one possible execution path where they find the assertions defined by the programmer.
There is also a assert() construct in php, used for enable checks on variables in production code. The assertions used in tests are a little different as they are real code (and not code passed in a string) and they do not clutter the production code but constitute a valuable part of test cases.
In phpunit there are some convenience methods which help to write expressive code and do different kind of assertions. These methods are available with a public signature on the test case class which is extended by default.

The first assertion which fails causes an exception to be raised and captured by phpunit runner. This means that if you are using an exception per test you are safe, but if you are writing test methods which contain multiple assertions, beware that the first failure will prevent the subsequent assertions from being executed. Only the assert*() calls which strictly depends on the previous ones to make sense should be placed in the same method as them.
Here is a list of the most common assertions available in phpunit: since the documentation is very good on this features I'm not going to go into the details. Most important and widely used methods are evidenced in bold.
  • assertTrue($argument) takes a boolean as a mandatory parameter and make the test fail if $argument is not true. You must pass to it a result from a method which returns booleans, such as a comparison operator result. assertFalse($argument) presents the inverse of this method behavior, failing if $argument is different from false.
  • assertEquals($expected, $actual) takes two arguments and confront them with the == operator, declaring the test failed if they do not match. The canned result should be put in the $expected argument, while the result obtained from the system under test in the $actual one: they will be shown in this order if the test fails, along with a comparison of the arguments dumps when applicable. assertNotEquals() is this method's opposite.
  • assertSame($expected, $actual) does the identical job of assertEquals(), but comparing the arguments with the === operator, which checks also the equality of variable types along with their values.
  • assertContains($needle, $haystack) searches $needle in $haystack, which can be an array or an Iterator implementation. assertNotContains() can also be very handy.
  • assertArrayHasKey($key, $array) evals if $key is in $array. It is used for both numeric and associative ones.
  • assertContainsOnly($type, $haystack) fails if $haystack contains element whose type differs from $type. $type is one of the possible result from gettype().
  • assertType($type, $variable) fails if $variable is not a $type. $type is specified as in assertContainsOnly(), or with PHPUnit types constants.
  • assertNotNull($variable) fails if $variable is the special value null.
  • assertLessThan(), assertGreaterThan(), assertGreatherThanOrEqual(), assertLessThanOrEquals() perform verifications on numbers and their names are probably self explanatory. They all take two arguments.
  • assertStringsStartsWith($prefix, $string) and assertStringsEndsWith($suffix, $string) are also self explanatory and section a string for you, avoiding the need for substr() magic in a test.
Remember that you can still make up nearly any assertion by calling a verification method and pass the result to assertTrue(). Moreover, nearly everyone of this methods support a supplemental string parameter named $message, which will be shown in the case of a failing test caused by the assertion; if you're making up a complex method for a custom assertion you may want to provide $message to assertTrue() to provide information in case the production code regress. Obviously, the custom assertion methods should be tested too.
If you want to see some code, I put up some examples on the assertion methods usage on pastebin.

I think you will start soon to use the more expressive assertions for what you are testing for: test methods should be short and easily understandable, and assertion methods which abstract away the verification burden are very beneficial. In the next parts, we'll dig into ways to reuse test code and in the annotations which phpunit recognizes to drive our test execution, such as @dataProvider and @depends.

You may want to subscribe to the feed to be informed when new posts in this series are published.

Thursday, September 24, 2009

Practical testing in php part 2: write clever tests

This is the second part of the testing series about php. You may want to subscribe to the feed to check out previous parts and not miss the next ones.

In the previous part, we have discovered the syntax and the infrastructure needed to run a test with phpunit. Now we are going to show a practical example using a test case/production code couple of classes.
What we are going to test is the Spl class ArrayIterator; for the readers that do not know this class, it is a simple Iterator implementation which abstracts away a foreach on the elements of an array.
Of course it would be very useful to write the tests before the production code class, but this is not the time to talk about TDD and its advantages: let's simply write a few tests to ensure the implementation works as we desire. This is also a common way to study the components of an object-oriented system: read and understand its unit test and write more of them to verify our expectations about the production classes behavior are fulfilled.
Let's start with the simplest test case: an empty array.
class ArrayIteratorTest extends PHPUnit_Framework_TestCase
{
    public function testEmptyArrayIsNotIteratedOver()
    {
        $iterator = new ArrayIterator(array());
        foreach ($iterator as $element) {
            $this->fail();
        }
    }
}
The test case class is named ArrayIteratorTest, following the convention of using a 1:1 mapping from production classes to test ones. The test method simply creates a new instance of the system under test, setting up the situation to have it iterate over the empty array. If the execution path enter the foreach, the test fails, as the call to fail() is equivalent to assertTrue(false).
The next step is to cover other possible situations:
public function testIteratesOverOneElementArrays()
    {
        $iterator = new ArrayIterator(array('foo'));
        $i = 0;
        foreach ($iterator as $element) {
            $this->assertEquals('foo', $element);
            $i++;
        }
        $this->assertEquals(1, $i);
    }
This test ensures that one-element numeric arrays are iterated correctly. The first assertion states that every element which is passed as the foreach argument is the element in the array, while the second that the foreach is executed only one time. You have probably guessed that assertEquals() confronts its two arguments with the == operator and fails if the result is false.
When it is not too computational expensive, we should strive to have the few possible assertions per method; so we can separate the test method testIteratesOverOneElementArrays() in two distinct ones:
public function testIteratesOverOneElementArraysUsingValues()
    {
        $iterator = new ArrayIterator(array('foo'));
        foreach ($iterator as $element) {
            $this->assertEquals('foo', $element);
        }
    }
    
    public function testIteratesOneTimeOverOneElementArrays()
    {
        $iterator = new ArrayIterator(array('foo'));
        $i = 0;
        foreach ($iterator as $element) {
            $i++;
        }
        $this->assertEquals(1, $i);
    }
Now the two test methods are nearly independent and can fail independently to provide information on two different broken behaviors: not using the array values and iterating more than one time on an element. This is a very simple case, but try to think of this example of a methodology to identify responsibilities of a production class: the test names should describe what features the class provides at a good level of specification (and they are really used for this purpose in Agile documentation). This is what we are doing by adopting descriptive test names and using a single assertion per test where it is possible: broken up the role of the class in tiny pieces which together give the full picture of the unit requirements.
We can go further and test also the use of ArrayIterator on associative arrays:
public function testIteratesOverAssociativeArrays()
    {
        $iterator = new ArrayIterator(array('foo' => 'bar'));
        $i = 0;
        foreach ($iterator as $key => $element) {
            $i++;
            $this->assertEquals('foo', $key);
            $this->assertEquals('bar', $element);
        }
        $this->assertEquals(1, $i);
    }
As an exercise you can try to refine this method two three independent ones, for instance creating the first of them with a name such as testIteratesOverAssociativeArraysUsingArrayKeysAsForeachKeys(). Don't worry about long method names as long as they are long to strengthen the specification, but only when the code can be refactored to smaller test methods. Even then, finding descriptive test names is the most difficult part of the process.
We can go on and add other test methods, and Spl has many.

Whenever a bug is found which you can impute to the class under test, you should add a test method which exposes the bug, and thus fails; then you can fix the class to make the test pass. This methodology helps to not reintroduce the bug in subsequent changes to the class, as a regression test is in place. It also defines more and more the behavior of a class by adding a method at the time.
The TDD methodology not only forces to add test methods to expose bug, but also to define new features. Implementing a user story is done by first writing a fail test which exposes the problem (the feature is not present at the time in the class) and then by implementing it.

I hope you're liking this journey in testing and you're considering to test extensively your code if you currently are not using phpunit or similar tools. In the next part, we will make a panoramic the assertion methods which phpunit provides to simplify the tester work. Remember that, in software unit testing, developer and tester coincide, or at least are at one next to the other, in the case of pair programming.

I have posted on pastebin the ArrayIteratorTest class if you want to play with it by yourself.
You may want to subscribe to the feed to not miss subsequent posts in this series.

Wednesday, September 23, 2009

Practical testing in php part 1: PHPUnit usage

What is unit testing and why a php programmer should adopt it? It may seem simple, but testing is the only way to ensure your work is completed and you will not called in the middle of the night by a client whose website is going nuts. The need for quality is particularly present in the php environment, where it is very simple to deploy an interpreted script, but it is also very simple to break something: a missing semicolon in a common file can halt the entire application.
Unit testing is the process of writing tests which exercise the basic functionality of the smallest cohesive unit in php: a class. You can also write tests for procedures and functions, but unit testing works at its best with cohesive and decouple classes. Thus, object-oriented programming is a requirement; this process is contrapposed to functional and integration testing, which build medium and large graphs of objects when run. Unit testing istances one, or very few, units at the time, and this implies that unit tests are tipically easy to run in every environment and do not burden the system with heavy operations.

When the time comes for unit and functional testing, there's only one leader in the php world: PHPUnit. PHPUnit is the xUnit instance for the average and top-class php programmer; born as a port of JUnit, has quickly filled the gap with the original Java tool thanks to the potential of a dynamic language such as php. It even surpassed the features and the scope of JUnit providing a simple mocking framework (whose utility will be discovered during this article series).

The most common and simplest way to install PHPUnit is as Pear package. On your development box, you have to be available the php binary and the pear executable.
sudo pear channel-discover pear.phpunit.de
sudo pear install phpunit/PHPUnit
Use a root shell (or a administrator on if you develop on other systems) instead of sudo if you prefer. These commands tell pear to add the channel of phpunit developers as it was a package repository, and to install the PHPUnit package from the phpunit channel. The grabbed release is the latest stable; at the time of this writing, the 3.4 version.
If the installation is successful, you now have a phpunit executable available from the command line. This is where you will run tests; if you use an IDE, probably there is a menu for running tests that will show you the command line output (and you should also install phpunit from the IDE interface to make it discover the new tool).

Before exploring the endless possibilities of testing, let's write our first one: the simplest test that could possibly work. I saved this php code in MyTest.php:
class MyTest extends PHPUnit_Framework_TestCase
{
public function testComparesNumbers()
{
$this->assertTrue(1 == 1);
}
}
What is a test? And a test case? A test case is constituted by a method by a class which extends PHPUnit_Framework_TestCase, which as its name tells is an abstract test case class provided by the PHPUnit framework. When developing a object-oriented application, you may want to start with one test case per every class you want to test (and if you're going the TDD way every class will be tested), thus there will be a 1:1 correspondence between classes and test cases. For the moment, we don't want to go too fast and we simply write a class that tests php common functionality.
Every test is a method which by convention starts with the keyword 'test'. Also for convention, the method name should tell what operation the system under test is capable, in this test .
Every method will be run independently in an isolated environment, and will make some assertion on what should happen. assertTrue() is one of the many assert*() method inherited from the abstract test case, which declares the test failed if an argument different from true is passed to it. The test as it is written now should pass. In fact, we can simply run it and find out:
[giorgio@Marty:~]$ phpunit MyTest.php
PHPUnit 3.4.0 by Sebastian Bergmann.

.

Time: 1 second

OK (1 test, 1 assertion)
Instant feedback is one of the pillars of TDD and of unit testing in general: the code in tests should instance your classes and exercising their functionality to ensure they don't blow up and respect the specifications. With the phpunit script, it's very simple and fast to run a test case or a group of them after you have made a change to your class and make sure you haven't break an existing feature.
The result of phpunit run is easily interpretable: a dot (.) for every test method which passed (without failed assertions), and a statistic of the number of tests and assertions encountered.
Let's try to make it fail, changing 1==1 to 1==0:
[giorgio@Marty:~]$ phpunit MyTest.php
PHPUnit 3.4.0 by Sebastian Bergmann.

F

Time: 0 seconds

There was 1 failure:

1) MyTest::testComparesNumbers
Failed asserting that  is true.

/media/sdb1/giorgio/MyTest.php:6

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
For every failed test, you get an F instead of the point in the summary. Other letters can be encountered, for instance the E if the test caused no assertion to fail but it raised an exception or a php error. The only passed tests are the one which present a dot: you should strive for a long list of dots that fill more than one row.
You also get a description of the assertion which has failed and in what test case, method and line it resides. Since you can run many test cases as a single command, this is very useful to localize defects and regression.
This time the test has failed because it is bad written: zero is not equal to one and php is right in giving out false. But assertTrue() does not know this and in the next part we'll write some tests which works upon userland code and it is in fact useful to detect if production classes are still satisfying all their requirements.

You may want to subscribe to the feed to not miss the subsequent parts of this php testing series. Feel free to ask clarifications in the comments or to raise testing topics you particularly care for.

Featured post

A map metaphor for architectural diagrams

It is a (two-dimension) representation of a pipe. The map is not the territory , but in software engineering terms they are models of it....

Popular posts