Practical testing in php part 3: assertions

This is the third part in the php testing series. You may want to checkout the previous parts or to subscribe to the feed to be notified of new posts.

Assertions are declarations that must hold true for a test to be declared successful: a test pass when it does not execute assertions or the one called are all verified correctly. Assertions are the final goal of a test, the place where you confront the expected and precalculated values of your variables with the ones that come from the system under test.
Assertions are implemented with methods and you have to make sure they are actually executed: thus, an if() construct inside a test is considered an anti-pattern as test methods should follow only one possible execution path where they find the assertions defined by the programmer.
There is also a assert() construct in php, used for enable checks on variables in production code. The assertions used in tests are a little different as they are real code (and not code passed in a string) and they do not clutter the production code but constitute a valuable part of test cases.
In phpunit there are some convenience methods which help to write expressive code and do different kind of assertions. These methods are available with a public signature on the test case class which is extended by default.

The first assertion which fails causes an exception to be raised and captured by phpunit runner. This means that if you are using an exception per test you are safe, but if you are writing test methods which contain multiple assertions, beware that the first failure will prevent the subsequent assertions from being executed. Only the assert*() calls which strictly depends on the previous ones to make sense should be placed in the same method as them.
Here is a list of the most common assertions available in phpunit: since the documentation is very good on this features I'm not going to go into the details. Most important and widely used methods are evidenced in bold.

• assertTrue($argument) takes a boolean as a mandatory parameter and make the test fail if $argument is not true. You must pass to it a result from a method which returns booleans, such as a comparison operator result. assertFalse($argument) presents the inverse of this method behavior, failing if $argument is different from false.
  
• assertEquals($expected, $actual) takes two arguments and confront them with the == operator, declaring the test failed if they do not match. The canned result should be put in the $expected argument, while the result obtained from the system under test in the $actual one: they will be shown in this order if the test fails, along with a comparison of the arguments dumps when applicable. assertNotEquals() is this method's opposite.
• assertSame($expected, $actual) does the identical job of assertEquals(), but comparing the arguments with the === operator, which checks also the equality of variable types along with their values.
  
• assertContains($needle, $haystack) searches $needle in $haystack, which can be an array or an Iterator implementation. assertNotContains() can also be very handy.
  
• assertArrayHasKey($key, $array) evals if $key is in $array. It is used for both numeric and associative ones.
• assertContainsOnly($type, $haystack) fails if $haystack contains element whose type differs from $type. $type is one of the possible result from gettype().
• assertType($type, $variable) fails if $variable is not a $type. $type is specified as in assertContainsOnly(), or with PHPUnit types constants.
• assertNotNull($variable) fails if $variable is the special value null.
• assertLessThan(), assertGreaterThan(), assertGreatherThanOrEqual(), assertLessThanOrEquals() perform verifications on numbers and their names are probably self explanatory. They all take two arguments.
• assertStringsStartsWith($prefix, $string) and assertStringsEndsWith($suffix, $string) are also self explanatory and section a string for you, avoiding the need for substr() magic in a test.
  

Remember that you can still make up nearly any assertion by calling a verification method and pass the result to assertTrue(). Moreover, nearly everyone of this methods support a supplemental string parameter named $message, which will be shown in the case of a failing test caused by the assertion; if you're making up a complex method for a custom assertion you may want to provide $message to assertTrue() to provide information in case the production code regress. Obviously, the custom assertion methods should be tested too.
If you want to see some code, I put up some examples on the assertion methods usage on pastebin.

I think you will start soon to use the more expressive assertions for what you are testing for: test methods should be short and easily understandable, and assertion methods which abstract away the verification burden are very beneficial. In the next parts, we'll dig into ways to reuse test code and in the annotations which phpunit recognizes to drive our test execution, such as @dataProvider and @depends.

You may want to subscribe to the feed to be informed when new posts in this series are published.

Practical testing in php part 2: write clever tests

This is the second part of the testing series about php. You may want to subscribe to the feed to check out previous parts and not miss the next ones.

In the previous part, we have discovered the syntax and the infrastructure needed to run a test with phpunit. Now we are going to show a practical example using a test case/production code couple of classes.
What we are going to test is the Spl class ArrayIterator; for the readers that do not know this class, it is a simple Iterator implementation which abstracts away a foreach on the elements of an array.
Of course it would be very useful to write the tests before the production code class, but this is not the time to talk about TDD and its advantages: let's simply write a few tests to ensure the implementation works as we desire. This is also a common way to study the components of an object-oriented system: read and understand its unit test and write more of them to verify our expectations about the production classes behavior are fulfilled.
Let's start with the simplest test case: an empty array.

class ArrayIteratorTest extends PHPUnit_Framework_TestCase
{
    public function testEmptyArrayIsNotIteratedOver()
    {
        $iterator = new ArrayIterator(array());
        foreach ($iterator as $element) {
            $this->fail();
        }
    }
}

The test case class is named ArrayIteratorTest, following the convention of using a 1:1 mapping from production classes to test ones. The test method simply creates a new instance of the system under test, setting up the situation to have it iterate over the empty array. If the execution path enter the foreach, the test fails, as the call to fail() is equivalent to assertTrue(false).
The next step is to cover other possible situations:

public function testIteratesOverOneElementArrays()
    {
        $iterator = new ArrayIterator(array('foo'));
        $i = 0;
        foreach ($iterator as $element) {
            $this->assertEquals('foo', $element);
            $i++;
        }
        $this->assertEquals(1, $i);
    }

This test ensures that one-element numeric arrays are iterated correctly. The first assertion states that every element which is passed as the foreach argument is the element in the array, while the second that the foreach is executed only one time. You have probably guessed that assertEquals() confronts its two arguments with the == operator and fails if the result is false.
When it is not too computational expensive, we should strive to have the few possible assertions per method; so we can separate the test method testIteratesOverOneElementArrays() in two distinct ones:

public function testIteratesOverOneElementArraysUsingValues()
    {
        $iterator = new ArrayIterator(array('foo'));
        foreach ($iterator as $element) {
            $this->assertEquals('foo', $element);
        }
    }
    
    public function testIteratesOneTimeOverOneElementArrays()
    {
        $iterator = new ArrayIterator(array('foo'));
        $i = 0;
        foreach ($iterator as $element) {
            $i++;
        }
        $this->assertEquals(1, $i);
    }

Now the two test methods are nearly independent and can fail independently to provide information on two different broken behaviors: not using the array values and iterating more than one time on an element. This is a very simple case, but try to think of this example of a methodology to identify responsibilities of a production class: the test names should describe what features the class provides at a good level of specification (and they are really used for this purpose in Agile documentation). This is what we are doing by adopting descriptive test names and using a single assertion per test where it is possible: broken up the role of the class in tiny pieces which together give the full picture of the unit requirements.
We can go further and test also the use of ArrayIterator on associative arrays:

public function testIteratesOverAssociativeArrays()
    {
        $iterator = new ArrayIterator(array('foo' => 'bar'));
        $i = 0;
        foreach ($iterator as $key => $element) {
            $i++;
            $this->assertEquals('foo', $key);
            $this->assertEquals('bar', $element);
        }
        $this->assertEquals(1, $i);
    }

As an exercise you can try to refine this method two three independent ones, for instance creating the first of them with a name such as testIteratesOverAssociativeArraysUsingArrayKeysAsForeachKeys(). Don't worry about long method names as long as they are long to strengthen the specification, but only when the code can be refactored to smaller test methods. Even then, finding descriptive test names is the most difficult part of the process.
We can go on and add other test methods, and Spl has many.

Whenever a bug is found which you can impute to the class under test, you should add a test method which exposes the bug, and thus fails; then you can fix the class to make the test pass. This methodology helps to not reintroduce the bug in subsequent changes to the class, as a regression test is in place. It also defines more and more the behavior of a class by adding a method at the time.
The TDD methodology not only forces to add test methods to expose bug, but also to define new features. Implementing a user story is done by first writing a fail test which exposes the problem (the feature is not present at the time in the class) and then by implementing it.

I hope you're liking this journey in testing and you're considering to test extensively your code if you currently are not using phpunit or similar tools. In the next part, we will make a panoramic the assertion methods which phpunit provides to simplify the tester work. Remember that, in software unit testing, developer and tester coincide, or at least are at one next to the other, in the case of pair programming.

I have posted on pastebin the ArrayIteratorTest class if you want to play with it by yourself.
You may want to subscribe to the feed to not miss subsequent posts in this series.

Practical testing in php part 1: PHPUnit usage

What is unit testing and why a php programmer should adopt it? It may seem simple, but testing is the only way to ensure your work is completed and you will not called in the middle of the night by a client whose website is going nuts. The need for quality is particularly present in the php environment, where it is very simple to deploy an interpreted script, but it is also very simple to break something: a missing semicolon in a common file can halt the entire application.
Unit testing is the process of writing tests which exercise the basic functionality of the smallest cohesive unit in php: a class. You can also write tests for procedures and functions, but unit testing works at its best with cohesive and decouple classes. Thus, object-oriented programming is a requirement; this process is contrapposed to functional and integration testing, which build medium and large graphs of objects when run. Unit testing istances one, or very few, units at the time, and this implies that unit tests are tipically easy to run in every environment and do not burden the system with heavy operations.

When the time comes for unit and functional testing, there's only one leader in the php world: PHPUnit. PHPUnit is the xUnit instance for the average and top-class php programmer; born as a port of JUnit, has quickly filled the gap with the original Java tool thanks to the potential of a dynamic language such as php. It even surpassed the features and the scope of JUnit providing a simple mocking framework (whose utility will be discovered during this article series).

The most common and simplest way to install PHPUnit is as Pear package. On your development box, you have to be available the php binary and the pear executable.

sudo pear channel-discover pear.phpunit.de
sudo pear install phpunit/PHPUnit

Use a root shell (or a administrator on if you develop on other systems) instead of sudo if you prefer. These commands tell pear to add the channel of phpunit developers as it was a package repository, and to install the PHPUnit package from the phpunit channel. The grabbed release is the latest stable; at the time of this writing, the 3.4 version.
If the installation is successful, you now have a phpunit executable available from the command line. This is where you will run tests; if you use an IDE, probably there is a menu for running tests that will show you the command line output (and you should also install phpunit from the IDE interface to make it discover the new tool).

Before exploring the endless possibilities of testing, let's write our first one: the simplest test that could possibly work. I saved this php code in MyTest.php:

class MyTest extends PHPUnit_Framework_TestCase
{
public function testComparesNumbers()
{
$this->assertTrue(1 == 1);
}
}

What is a test? And a test case? A test case is constituted by a method by a class which extends PHPUnit_Framework_TestCase, which as its name tells is an abstract test case class provided by the PHPUnit framework. When developing a object-oriented application, you may want to start with one test case per every class you want to test (and if you're going the TDD way every class will be tested), thus there will be a 1:1 correspondence between classes and test cases. For the moment, we don't want to go too fast and we simply write a class that tests php common functionality.
Every test is a method which by convention starts with the keyword 'test'. Also for convention, the method name should tell what operation the system under test is capable, in this test .
Every method will be run independently in an isolated environment, and will make some assertion on what should happen. assertTrue() is one of the many assert*() method inherited from the abstract test case, which declares the test failed if an argument different from true is passed to it. The test as it is written now should pass. In fact, we can simply run it and find out:

[giorgio@Marty:~]$ phpunit MyTest.php
PHPUnit 3.4.0 by Sebastian Bergmann.

.

Time: 1 second

OK (1 test, 1 assertion)

Instant feedback is one of the pillars of TDD and of unit testing in general: the code in tests should instance your classes and exercising their functionality to ensure they don't blow up and respect the specifications. With the phpunit script, it's very simple and fast to run a test case or a group of them after you have made a change to your class and make sure you haven't break an existing feature.
The result of phpunit run is easily interpretable: a dot (.) for every test method which passed (without failed assertions), and a statistic of the number of tests and assertions encountered.
Let's try to make it fail, changing 1==1 to 1==0:

[giorgio@Marty:~]$ phpunit MyTest.php
PHPUnit 3.4.0 by Sebastian Bergmann.

F

Time: 0 seconds

There was 1 failure:

1) MyTest::testComparesNumbers
Failed asserting that  is true.

/media/sdb1/giorgio/MyTest.php:6

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

For every failed test, you get an F instead of the point in the summary. Other letters can be encountered, for instance the E if the test caused no assertion to fail but it raised an exception or a php error. The only passed tests are the one which present a dot: you should strive for a long list of dots that fill more than one row.
You also get a description of the assertion which has failed and in what test case, method and line it resides. Since you can run many test cases as a single command, this is very useful to localize defects and regression.
This time the test has failed because it is bad written: zero is not equal to one and php is right in giving out false. But assertTrue() does not know this and in the next part we'll write some tests which works upon userland code and it is in fact useful to detect if production classes are still satisfying all their requirements.

You may want to subscribe to the feed to not miss the subsequent parts of this php testing series. Feel free to ask clarifications in the comments or to raise testing topics you particularly care for.

The power of tracking and logging

Tracking resources - time, money, code changes and whatever else - is a powerful way to improve your understanding of them. Logging is even more powerful as it's automatic tracking done by your tools for you.

The practice of tracking time for the various activities one intraprends is one of the pillar of time management. Analyzing what is sucking up precious man-months gives you a clue to eliminate the tasks that really aren't worth the effort, instead of finding them by chance or by some preconcepts you may have. Tracking your spending is the first advice a financial counselor will tell you if you experience money problems.
The concept of time tracking is also present in programming: the only sure-fire way to optimize an algorithm or an application is to profile it as the first step, and then making changes to the code which constitutes the bottleneck. While an algorithm theorical analysis produces a result expressed in O(f(n)), profiling it on real data allows the programmer even to confront different O(n log n) sorting algorithms.

At a different perspective level, tracking is present in the modern methodologies of development with tools as source control systems. Every code check-in is tracked and remains forever in the history of the codebase: no line of code is ever lost in shared directories or email folders. This type of tracking information is better named as logging.
It is very cheap for a software system, if built correctly, to conserve every chunk of data that passes on its bridge. Subversion and other vcs do exactly the process of logging any single commit, and the logs reveal useful when viewed as changesets or while generating a changelog for a new release. Project management tools like trac, built upon subversion, log every change to the tickets which report bugs and feature requests, along with the edits of the wiki pages. It is a small job paragonated to the extent of tracking Wikipedia does.
If you're talking on irc or other istant messenger, your client is probably writing logs of the conversations to disk. Every enterprise java application logs exceptions to a file or to database, too.
Having a large amount of organized data is a source of valuable information, as this logging capabilities allow to:

• posting a conversation between developers on the wiki for further reference;
• giving commit access to new developers knowing their commits can be rolled back;
• listing the commits which affects a particular bug, which trac does;
• generating a changelog file by looking at the list of commits in a particular period on a branch;
  
• updating a working copy or a deployment of a php application transmitting only the modified files;
• generating a list of the locally modified files in a few seconds to see what is being committed (svn status).
  

There are infinite possibilities for the usage of logged data. It is often said that every item of a project which cannot be automatically generated (like builds) should be put under version control.
Logging was once expensive, when it was done by hand on dusted registers: the data was patiently annotated during the day and it wasn't going to be useful anywhere else. Now that the information era is arrived, take advantage of the logging capabilities of your tools and never write the same thing twice when a machine can do it for you.

The image is a photo of a rinascimental ledger, used for accounting. Money transaction have a long logging tradition for fiscal purposes.

Setting SMART goals for developers

The journey of a developer is constituted of hard work and self-improvement. Maybe you are going to work on a big freelance project, to solve programming challenges or expand your knowledge in an area where you lack expertise; whatever you want to be productive for, setting SMART goals can help you stay focused and reach big achievements.

According to Wiktionary, the definition of goal is a result that one is attempting to achieve: setting a goal is the first step in deciding what you want to do. But to find out how you're gonna achieve a goal and concentrate on it, you must refine and work also on its definition.

Verba volant, scripta manent
The most important part of a goal is where it's kept: while verbal and mental goals are volatile and can change over time, a written goal is carved in stone until you consciously decide to modify it. The characteristic of a SMART goal are best exploited when they are consistently written and reviewed. You're gonna write a Unix clone? Write it. You're gonna launch a new search engine? Force yourself to write it down.
To help your productivity, you should definitely do what works for you; however, keep in mind that this system has helped a lot of people to get out of lazyness and to maintain a good attitude while working on their passions.

The five SMART keywords
SMART is an acronym for five terms a goal must implement to be really useful. The SMART criteria is used in many management fields, for instance to question a project's objectives. Writing user stories instead of a list of features is an implementation of SMART goals.
These keywords are:

• Specific: you must be specific in your goal as vague ones cannot give you the needed focus. Learning Java is not a specific goal, while Write a small application in Java is a little better. The logic of being specific is that writing a specific, narrow goal will make you think of a plan and to respect the other SMART characteristics; if a goal is too big you should break it into smaller sub-goals that are enough specific, doing a sort of idea refactoring.
• Measurable: the achievement should be scientificly and quantitatively measurable. While it seems obvious, this goal characteristic is really important: otherwise you'll never know if the goal has been achieved and should be deleted form you list. Again as an example, Learning Java is not measurable, while Writing a clone of the ls command in Java is very measurable.
• Attainable: while a goal can be out of your comfort zone, you must consider the time and the work you will put in this experience. Earn $100,000 in one week is often not an attainable goal for the majority of us.
• Relevant: a goal has to be challenging. Otherwise why set it? If a goal is too simple or short, it is more a task on yur todo list instead of a life-changing experience. In our example, if you're already experienced with Java you should strive for writing a more complex application instead of the simple ls, while this can be a good programming problem for a person that has never tried an object-oriented language.
  
• Time-bound: you must set a deadline for measure your achievements. As Keith Ferrazzi says, a goal is a dream with a deadline: if you do not set a specific date when you want to have accomplished this goal, it will remain a dream. A deadline is what forces you to work and stimulate your productivity, even if it's self-imposed.

In sum, our goal has gone from Learning Java to Write wget in Java before Sep 30, 2009, written for instance in a GOALS.txt file in my home directory, which I really have. I hope seeing this version of the goal helps you to think of what you need to do in order to accomplish it: learn the basic Java syntax and approach to programming, find out what libraries for networking and filesystem management are avaiable and how to set up a working development environment. Note that these are all googleable tasks, while Learning Java will point you to base level courses when you learn to create Dog and Cat classes which extend Animal.
TDD is another example of goal setting: writing a test is in fact setting a goal which is very Measurable, since the red/green bar will tell you if the test has passed and a computer is deterministic (if you do not write brittle tests). There are other problems to solve, though: productivity is halted if you do not implement enough user stories per day; or you might fall into testing only getters and setters, which is not a Relevant goal.

Feel free to raise questions: productivity is a difficult and subjective field, in which everyone of us can learn from each other.

Coaches talk about schemes, game plans and tactic, but the purpose of football is only to score a goal more than the opponent team. Set your goals to know what you're heading towards.

Zend Framework Api: what $options is?

I wanted to share an insight that repetitevely using and studying the Zend Framework API has given me. I know finding a lack in documentation can be annoying but ZF is usually very coherent in its API and I find out it is predictable also in this case, which is a good trait for a very large set of classes such as the one from Zend.

The tipical constructor of a Zend Framework class (or factory method) has a signature such as:

Zend_Form::__construct([mixed $options = null]);

and this always left me wondering what would $options contain, since I cannot found a documentation for this constructor. I depended on finding examples in the reference manual for what I want to do with my objects.
What I have learned is a convention widely used in Zend Framework main components and in the incubator: the options are passed to the setters after the mandatory construction process have taken place. This means that if we build a submit button in this way:

$button = new Zend_Form_Element_Submit('submit', array('ignore' => true, 'value' => 'Click me'));

the result is equivalent to:

$button = new Zend_Form_Element_Submit('submit');
$button->setIgnore();
$button->setValue('Click me');

So you can simply refer to the documentation of the particular setter you want to call in the constructor and provide a key in the array with the initial lowercase (to respect the Pear/Zend coding standard): setElementsBelongTo() will require a elementsBelongTo key.

It would be interesting to know if there is an official guideline to provide constructors like the ones in Zend_Form component or if do some classes follow a different convention: feel free to share what you know in the comments.

What's new in PHPUnit 3.4

PHPUnit is the leader testing tool in the php world: the equivalent of JUnit for Java and it's what comes to mind when you're thinking of testing a php application.
Although the Api for PHPUnit 3 is stable, some useful improvements and functionalities have been developed for the 3.4 branch, which has seen its first official release two days ago.

If you have a Pear-based installation, it's very easy to get PHPUnit 3.4. Simply run:
# pear upgrade phpunit/PHPUnit
as root or preceded by sudo.
For some important features, you can refer to the official manual, that is currently in the process of being updated.

Here's a list of some novelties that I have experimented this morning on NakedPhp:

• test methods dependencies: mark with @depends a test method, followed by the name of the test method which it relies upon. For instance, a test method for deletion of an object in a container depends upon the addition test; this way if the first fails the dependent tests will be automatically skipped and you will locate the bug instantly. Moreover, you can pass fixture from the testAdd() method to testDelete(), effectively reusing it. Keep in mind, however, that this feature works on test methods and not between different test cases.
• annotations @runTestsInSeparateProcesses and @runInSeparateProcess allow tests running in parallel. They should be used on the test case class and on the test method respectively. However, I've had trouble adding them because some kind of recursive dependency is raised when exporting global variables and xdebug stop it mercilessly.
• setUpBeforeClass() and tearDownBeforeClass() a la junit have been introduced. Heavy fixture set up now can be done one time per test case.
• getMockForAbstractClass() will fill in the abstract methods for you.
• assertStringStartsWith() and assertStringEndsWith() have been added for rapid assertions.
• PHPUnit_Extensions_TicketListener_Trac can open and close Trac tickets basing on a test result: I think the @ticket annotation is stricly related to this component. I have not yet tried it since I do not have a Trac instance available (and I think this process can be slow), but it seems a good step forward to expose brittle tests. Especially if used in continuos integration.
• Mock Objects can now be passed to with() without issues. The cloning problems for with() can be though to resolve since objects are cloned when passed to with() to allow assertions being run on them after the test method has returned: obviously asserting that an object is identical to an expected one would never work, and asserting simple equality is often worse since very big objects can be passed around and duplicated.
  
• You can mock namespaced classes (in php 5.3) without going to a complicated process to autoload them first and then specify all the parameters to getMock(). Good!
• @covers annotation support has been extended to setUp() and tearDown(), and the relative code coverage calculation has been improved.
  

These are the most important features that I have found in the ChangeLog and I have already tried to experiment a few of them them personally in my projects. Feel free to add your discoveries in the comments.