How to avoid phase-of-the-Moon bugs

Wikipedia documents some definitions of particularly dangerous bugs, which are very hard to fix and when they manifest constitute a real problem in development.
Pay attention to these examples:

• Heisenbug: bug that changes its behavior when someone is trying to reproduce it. The very attempt to study it changes the conditions (or requires irreproducible conditions) so that the Heisenbug does not manifest anymore. The naming of this bug is based on Heisenberg's uncertainty principle.
• Phase of the Moon bug: bug that arises from a dependency on external conditions, usually time. In the linked original definition, a piece of software has a marginal dependency on the Moon phase. Surely software developers should not feverishly expect monthly events to determine if their application works (unless it's some werewolves-oriented project.)

These are two categories that are really dangerous and can bring development to an halt by forcing long debugging session to find the cause of the failure, which is "non-deterministic" (in the former case) or hidden (in the latter). They certainly look scary but I chose them as examples because these particular bugs can be avoided by employing some engineering practices, such as a good test suite.

The test suite for a project should be mainly composed by unit tests. While acceptance end-to-end tests constitute an important part of it, because they validate the application's fulfillment of its requirements, unit tests are usually more powerful even if they do not drive the design. Their potential is to quickly locate defects, by testing the contract of individual classes: the first step in exposing a bug, and particularly an Heisenbug, consists in locating it, and being able to reproduce it reliably every time that is needed, to check that the bug has been fixed. A well-written test suite provides an automatic way to check, at the push of a button, that all previously fixed bugs have not reappeared, so that there are no regressions.
The other advantage of unit tests is in the way they promote isolation. External dependencies are usually mocked out in the testing environment, so that boundary conditions can be reproduced at will. If you suspect that a piece of software may fail during a combination of unusual date and time, you only have to add a test case where you provide the set of conditions that you are scared about.
The isolation of components is not limited to external capricious dependencies, such as time and database state. Mutable global state can also be avoided just by maintaining a unit test suite, primarily because it makes the tests brittle and difficult to write since they may fail depending on their execution order. A unit test should be able to coherently fail or pass both when the single test method is run as when the whole suite is. If  you're facing global state clings in the testing environment, which may lead to Heisenbugs and similar issues in production, listening to the tests will tell you to change your design to accomodate a simpler testing procedure, and a overall better architecture.
I agree completely to the Google Testing blog's motto:

Debugging sucks. Testing rocks.

Last holiday week roundup

This has been a productive week, given that I had finished my exams early in February and I was free from university commitments. Since you follow this blog I thought you would enjoy some focused links to my writings outside of here. I do not want to waste your weekend time so I will be short.

php|architect
I wrote two original articles on phparch.com this week:
Month of PHP Security 2010
Contributing to Zend Framework

DZone
Three blog posts have been republished by DZone, with my consent, as you can see from my personal page. The original pages for these articles are:
Practical Php Patterns: Mediator
Practical Php Patterns: Memento
Why I'm leaving Subversion for Git
Well, people have gone from You really don't know English to We republish nearly everything you write. If you encounter some snobbish trolls that belittle you, don't give up on your projects. It's the norm.

I hope you enjoy this material if you have not check it out yet.

Clever Mock Objects with PHPUnit

PHPUnit is the standard testing framework for php applications, and it has native support for mocking. It can produce various Stubs and Mocks by automatically extending a class or implementing an interface. Expectations can be set on these mocks so that assertions will be made on parameters and fake return values will be produced when particular methods are called (for a full explanation of mocks' utility see my free ebook.)
One thing I think PHPUnit lacks is support for multiple expectations. Let's fix ideas with a practicalfanciful example.

Imagine that you have a Telescope class (or interface) which defines an observeColor() method. When you call this method with the name of a planet or star, say Earth, Jupiter or Sun, it returns Blue, Grey, Yellow depending on what its lenses find out. I imagine these variables as plain strings to keep the discussion simple, but using ValueObjects do not change much the context.
The system under test is a class which composes Telescope as a collaborator. Given a list of objects in space, it does some calculations which involve their colors. Orienting a Telescope everytime you run the test suite is not practical, and the planets may not even be visible some times, thus you are forced to mock out Telescope (of course this situation is totally made up, but dependency management is a fundamental part of object-oriented design.)
To accomplish its responsibilities, the SUT needs to call observeColor() with a small set of different parameters and to obtain different results, depending on the particular test case. This is in fact a very common situation in testing.
As far as I know PHPUnit does not have a out-of-the-box support for defining multiple expectations on a mocked method without generating a conflict: if you set up expectations and constraint on method parameters, they must be satisfied in every single call to the mocked method. Similarly, if you set up a fake return value for the method, it will be hardcoded as it cannot depend on the input parameters.
I will present now a solution involving a callback anonymous function which works for a one-parameter mocked method. Only the mock setup and verification part is included, as the hypothetical SUT is not important in this explanation.
Note that anonymous functions are available only on php 5.3, so for previous versions of php you should create a small class with only one method, and pass array($object, 'methodName') as the callback.

<?php

class MultipleExpectationsTest extends PHPUnit_Framework_TestCase
{
    /**
     * This does not work. Multiple with() and will() calls
     * set up conflicting expectations and behaviors.
     */
    public function testMultipleExpectationsCanBePutOnTheSameMethodNatively()
    {
        $mock = $this->getMock('Telescope');
        $mock->expects($this->any())
             ->method('observeColor')
             ->with('Earth')
             ->will($this->returnValue('Blue'));
        $mock->expects($this->any())
             ->method('observeColor')
             ->with('Mars')
             ->will($this->returnValue('Red'));
        $this->assertEquals('Blue', $mock->observeColor('Earth'));
        $this->assertEquals('Red', $mock->observeColor('Mars'));
    }

    /**
     * We tune the expectation by passing it to a custom method.
     */
    public function testMultipleExpectationsCanBePutOnTheSameMethodViaACallback()
    {
        $inputs = array('Earth', 'Mars');
        $outputs = array('Blue', 'Red');
        $mock = $this->getMock('Telescope');
        $expectation = $mock->expects($this->exactly(2))
                            ->method('observeColor');
        $this->setMultipleMatching($expectation, $inputs, $outputs);
        $this->assertEquals('Blue', $mock->observeColor('Earth'));
        $this->assertEquals('Red', $mock->observeColor('Mars'));
    }

    /**
     * A callback is built and linked to the mocked method.
     */
    public function setMultipleMatching($expectation,
                                        array $inputs,
                                        array $outputs)
    {
        $testCase = $this;
        $callback = function() use ($inputs, $outputs, $testCase) {
            $args = func_get_args();
            $testCase->assertContains($args[0], $inputs);
            $index = array_search($args[0], $inputs);
            return $outputs[$index];
        };
        $expectation->will($this->returnCallback($callback));
    }
}

interface Telescope
{
    public function observeColor($name);
}

Update: in recent versions PHPUnit includes a $this->returnValueMap() option for will() that performs this job. Apparently it was inspired by this post.

Why I'm leaving Subversion for Git

I always believed in Subversion's potential and that it would be a wide improvement over the nightmare that was CVS, but I found out that, as Linus says, There is no way to do CVS right.
Subversion is fairly good and it's probably the better centralized version control system on the market, and it's open source. But after my first day of Git real-world usage, I now cannot negate that the distributed model of Git will be superior at last.

For example, in Git, you are not forced to have a remote repository: you can start with your local repository, and commit to it. No need to set up server software if you don't want to, as your machine is already your own server.
After working locally for some time and having done very fast commits as you usually did much more slowly with your remote Subversion server, you can push or pull changesets to other repositories. I was worried about the inherent uncertainty in this process (which branch? Which repository?), but Git tracks the origin of your repository, precisely the common ancestor in the remote branch, which your master branch was forked from. When you start by cloning a remote repository (as it is almost always the case), it will push to that origin. But you can also push to nearly everything, even if there is not a common ancestor.
The converse is true: you can also pull to update your repository from an external one, that may be your point of convergence. Obviously, you can also pull from everywhere, in a local branch if you are not sure to incorporate the changes.

The commit access notion vanishes in Git. You and I typically do not have commit access to major projects such as Zend Framework or Doctrine anyway (ok, I have to Doctrine, but when I come out with a small patch I just put it the bug tracker for a review), and we have only our working copy, having to upload patches to the bug tracker. Multiple versions of those patches. That quickly become not in sync with the current revision of the codebase.
Collaboration between working copies is difficult, as an official branch is needed. Moreover, it's rare that you know a branch is needed ahead of time. If for every feature you started a remote branch, even with Subversion (which has cheap copy-on-write branches) the development would be very slow. So many times you end up working on the trunk and struggling because you cannot break it, and sending patches by email. These problems are described in Linus's talk at Google about Git - which is amusing. When I watched his talk using Subversion really started bugging me; I knew that here was an alternative, and it consisted of mature software since the talk was old, from 2007.
What's the solution then? In Git everyone has his own repository, so you can just pull changes from other people and pushing changes to them. There is no commit access, only a group the lead developer trusts and he will pull from. This group may pull and review from other people, in a hierarchical work flow where a chain of trust is established. For an open source project, Git is worth its weight in gold (which actually I don't know, but it will be surely expressed in hexadecimal.)
You can break your repository if you want, committing all the day without problems, and being able to revert to every commit's revision as if Git were a time machine, and very cheaply (in Subversion you are forced to hit the network to revert to revisions older than the last commit.) When you have fixed your copy after ten, twenty or an hundred commits and all tests are passing, you may finally push. Awesome.

And this was only my first day in Git!

Practical Php Patterns: Memento

This post is part of the Practical Php Pattern series.

The pattern of today is the Memento one, whose intent is storing the state of an Originator object without breaking its encapsulation, which typically consists in a set of private fields.

For the sake of preserving encapsulation, the ConcreteMemento should be created only by its correspondent Originator. Only Originator has access to its private members, and only Originator can restore its state given that a ConcreteMemento has been handed back to it.
Another caveat is that only Originator sees the full contract of ConcreteMemento, while the Caretaker objects, which pass the Memento around, work with a very narrow interface, mostly present only for type safety and possibly without any methods on it. Any access to the ConcreteMemento data out of the Originator will, again, break its encapsulation and the best way to avoid tampering is defining Caretaker's method signatures to accept a Memento instead of ConcreteMemento: for instance, you can unit test them with a Memento mock.
ConcreteMemento itself is a dumb object, usually implemented as an immutable ValueObject, with little or no behavior. Thus, there is no need for a factory that takes care of its creation and it can be instantiated directly both in Originator and in test suites (no behavior to stub out).
Php's dynamic typing is very useful since avoids the casting of Memento to ConcreteMemento when is passed back to the Originator. What in a statically typed language should be seriously dealt with, in php is a minor inconvenience.
Anyway, why not storing the whole object whose state we want to preserve instead of a dumb, little Memento? I see two diffused situations where this pattern is applicable:

• the object whose state has to be maintained is heavy, and serializing it would bring together many other collaborators.
• moreover, in php scripts it may be linked to external services or resources that do not survive the http request.

The code sample approaches the latter case, tackling the problem by recreating a Service and re-establishing its state with a serializable Memento. The Service may contain anything from database connections to stream wrappers - only the Memento should be stored in $_SESSION.

<?php
/**
 * This interface is primarily here for type safety,
 * specifically to avoid that someone passes to Caretaker's
 * methods strings, instead of Memento instances.
 */
interface Memento
{
}

/**
 * The ConcreteMemento should be used only in the Originator
 * (and in testing).
 */
class ConcreteMemento implements Memento
{
    protected $_url;
    protected $_currentLine;

    public function __construct($url, $currentLine)
    {
        $this->_url = $url;
        $this->_currentLine = $currentLine;
    }

    public function getUrl()
    {
        return $this->_url;
    }

    public function getCurrentLine()
    {
        return $this->_currentLine;
    }
}

/**
 * The Originator. Creates a ConcreteMemento and returns it
 * as a Memento, taking back the Memento and checking it
 * to rebuild its internal state.
 * This class responsibility is to take an url and
 * returning one line from it at the time. Its work is
 * often broken into multiple [Ajax] requests, depending
 * on the parts the user wants to see.
 */
class Service
{
    protected $_url;
    protected $_currentLine = 0;

    public function __construct($url = null)
    {
        $this->_url = $url;
    }

    public function init()
    {
        $this->_content = file($this->_url);
    }

    /**
     * @return ConcreteMemento
     */
    public function getState()
    {
        return new ConcreteMemento($this->_url, $this->_currentLine);
    }

    public function setState(ConcreteMemento $state)
    {
        if (!($state instanceof Memento)) {
            throw new Exception('Memento object not recognized.');
        }
        $this->_url= $state->getUrl();
        $this->_currentLine = $state->getCurrentLine();
        $this->init();
    }

    public function getLine()
    {
        $line = $this->_content[$this->_currentLine];
        $this->_currentLine++;
        return $line;
    }
}

//Caretaker code
$service = new Service('http://giorgiosironi.blogspot.com');
$service->init();
echo $service->getLine();
echo $service->getLine();
echo $service->getLine();
$memento = $service->getState();
// now we can store Memento in the session
// let's simulate its handling
$mementoString = serialize($memento);
$newService = new Service();
$newService->setState(unserialize($mementoString));
echo $newService->getLine();
echo $newService->getLine();
echo $newService->getLine();

Zend Framework 1.8 Web Application Development review

This is my personal review of Zend Framework 1.8 Web Application Development, a book by Keith Pope (affiliate link).

First, a clarification on the book's name, which refers to Zend Framework 1.8: the book work flow is based on the 1.8 version but the Api of Zend Framework is now stable, and thus even if now the latest release is 1.10, the book is not outdated at all. Zend Framework 1 would have been the wrong title as the 1.5 and 1.8 (which saw the introduction of Zend_Tool) releases were the real turning points.

After having read its 300+ pages, my general impression of this book is good. A problem often faced by frameworks is the relearning of php basic features: how to pass a parameter, add a page, generate an url, etc. which were very quickly solved by the author in the first chapters. This book starts from the ground up and adds the functionalities that cover 80% of the use cases, harnessing the power of Zend Framework. Zend Framework has a large Api, and knowing the right methods to call is fundamental.
The author does not skip panoramics of what happens under the hood of Zend Framework, which is very useful for debugging purposes and for deepen the understanding of other infrastructure such as the ajax requests helper (not included) and Zend_Test.
The whole point of the book is in developing a showcase Zend Framework application for an online store, complete with an electronic cart. As you know, I prefer hands-on approaches to programming and I think code is the best model for a software, way better than diagrams. I was happy to see that every version of the application (which progresses trough the different chapters) is present in the downloadable package.

Components treated

• Zend_Controller and Zend_View (plus Zend_Layout) in the management of the Mvc machine.
• Zend_Db for the implementation of model persistence resources.
• Zend_Tool for the creation of an application from scratch.
• extensive guide to Zend_Form, which is one of the most time-saving components and one of the most difficult to leverage because of its large Api and many functionalities. Decorators, filters and validators are widely used.
• Zend_Session, in a quite elaborate example. I think I would have simply just put domain objects in a Zend_Session_Namespace.
• Zend_Auth and Zend_Acl for authentication and authorization (management of identity and permissions).
• Zend_Cache for optimization of different parts of the application. This is not micro-optimization, and the author highlights simple solutions like avoiding multiple actions dispatching and activating Apc over complicate practices.
• Zend_Test for integration testing of the controllers.

Model
The sample application in this book implements a maybe overengineered database-agnostic model, which would thus be a good solution for decoupling and testing purposes. Though, ZF 1.x does not have the infrastructure to achieve a pure Domain Model without a great effort (currently work is being done on integrating Doctrine for simpler object-relational mapping.) As a result, the domain model has highly specific ResourceItem and Resource classes which sit in the middle between the Active Record and the Data Mapper patterns.
This model design is not "pure", but it gets the job done, maintaining the fat model and skinny controller paradigm. Though, I would not purport it as an example. Remember that the Model part of your Mvc application is up to you: the book is only about learning how to leverage Zend Framework, not about object-oriented programming. There is a great example of a model that does not use the database at all: if you are stuck with ActiveRecord, this way of thinking will open your mind.

If you want to dive into Zend Framework, this book will teach you a lesson on the most important components, saving you hours of digging the Api and the reference manual.

Practical Php Patterns: Mediator

This post is part of the Practical Php Pattern series.
 
The pattern of the day is the Mediator one. The intent of this pattern is encapsulating the interactions of a set of objects, preventing aggressive coupling from each of them towards the other ones. A Mediator acts as a central point of convergence between the Colleague objects.
In any domain there are many operations that do not fit on existent, modelled-from-reality objects, and if forced as methods of an already existent class would add a dependency towards a possibly unrelated collaborator. This approach would result in a web of highly interrelated Colleague objects and not in the Ravioli code we want to work with. The Colleague objects should be kept loosely coupled to avoid having to reference them as a whole any time one of them is needed from a Client.

The solution to this common problem is implementing the Mediator pattern. When an object's relationships and dependencies start to conflict with its business responsibility (the reason it was created in the first place), we should introduce a Mediator that coordinates the workflow between the coupled objects, freeing them from this form of coupling; dependencies can be established from the Colleagues towards the Mediator and/or from the Mediator towards the Colleagues. Both directions of those dependencies can be broken with an interface AbstractColleague or AbstractMediator, if necessary.
No object is an island, and each object in an application must cooperate with other parts of the graph to get its job done and addressing one concern. Since the interactions are a source of coupling, a Mediator is one of the most effective patterns in limiting it, although, if abused, it may render more difficult to write cohesive classes.
As a practical example, Services in Domain-Driven Design are Mediators between Entities. For a php-related example, Zend_Form decorating and filtering capabilities are actually the implementation of a simple Mediator between Zend_Form_Decorator and Zend_Filter instances. The same goes for validation using Zend_Validate objects. Making every filter referencing the next one would build a Chain of Responsibility which potential would be unused.

When a Mediator must listen to Colleagues events, it is often implemented as an Observer resulting in a blackboard object where some Colleagues write and other ones read. Events are pushed to the Mediator from a Colleague, before it delivers them to the others subscribed Colleagues. There is no knowledge of others Colleagues in anyone of them: this architecture is successfully used in the Dojo javascript library shipped with Zend Framework.
Another advantage of this pattern is the variation of the objects involved in the computation: this goal can be achived by configuring the Mediator differently, whereas instancing interrelated objects would be an noncohesive operation and the collaboration relationships would be scattered between different containers or factories.

Participants

• Colleague: focuses on its responsibility, communicating only with a Mediator or AbstractMediator.
• Mediator: coordinates the work of a set composed by several Colleagues (AbstractColleagues).
• AbstractMediator, AbstractColleague: optional interfaces that decouple from the actual implementation of these roles. There may be more than one AbstractColleague role.

The code sample implements a filtering process for a form input that resembles Zend_Form_Element's feature.

<?php
/**
 * AbstractColleague.
 */
interface Filter
{
    public function filter($value);
}

/**
 * Colleague. We decide in the implementation phase
 * that Colleagues should not know the next Colleague
 * in the chain, resorting to a Mediator to link them together.
 * This choice succesfully avoids a base abstract class
 * for Filters.
 * Remember that this is an example: it is not only
 * Chain of Responsibility that can be alternatively implemented
 * as a Mediator.
 */
class TrimFilter implements Filter
{
     public function filter($value)
     {
         return trim($value);
     }
}

/**
 * Colleague.
 */
class NullFilter implements Filter
{
     public function filter($value)
     {
         return $value ? $value : '';
     }
}

/**
 * Colleague.
 */
class HtmlEntitiesFilter implements Filter
{
     public function filter($value)
     {
         return htmlentities($value);
     }
}

/**
 * The Mediator. We avoid referencing it from ConcreteColleagues
 * and so the need for an interface. We leave the implementation
 * of a bidirectional channel for the Observer pattern's example.
 * This class responsibility is to store the value and coordinate
 * filters computation when they have to be applied to the value.
 * Filtering responsibilities are obviously a concern of
 * the Colleagues, which are Filter implementations.
 */
class InputElement
{
    protected $_filters;
    protected $_value;

    public function addFilter(Filter $filter)
    {
        $this->_filters[] = $filter;
        return $this;
    }

    public function setValue($value)
    {
        $this->_value = $this->_filter($value);
    }

    protected function _filter($value)
    {
        foreach ($this->_filters as $filter) {
            $value = $filter->filter($value);
        }
        return $value;
    }

    public function getValue()
    {
        return $this->_value;
    }
}

$input = new InputElement();
$input->addFilter(new NullFilter())
      ->addFilter(new TrimFilter())
      ->addFilter(new HtmlEntitiesFilter());
$input->setValue(' You should use the <h1>-<h6> tags for your headings.');
echo $input->getValue(), "\n";