Stop writing foreach() cycles

At least in php. How many for() or foreach() have you written today? I bet a lot. Php 5.3 has a solution that will reduce the average number of iteration structures you need to write: closures applied by plain old array functions.
There are some array functions which have already been supported at least from Php 4, and that take as an argument a callback whose formal parameters have to be one or two elements of the array. I'm talking about array_map(), array_reduce(), array_filter() and uasort() (or similar custom sorting function.) These functions abstract away a foreach() cycle by applying a particular computation to all the elements of an array.
Back in Php 4 and Php 5.2, specifying a callback was cumbersome: you had to define an external function and then passing its name as a string; or passing an array containing an object or the class name plus the method name in case of a public (possibly static) method.
In Php 5.3, callbacks may also be specified as anonymous functions, defined in the middle of other code. These closures are first class citizens, and are treated as you would treat a variable, by passing it around as a method parameter. While I am not a fan of mixing up object-oriented and functional programming, closures can be a time saver which capture very well the intent of low-level processing code, avoiding the foreach() noise.
If you bear with me for a moment, I will show you with working code how you can avoid writing most of your foreach() cycles.

<?php
// obviously only the prime numbers less than 20
$primeNumbers = array(2, 3, 5, 7, 11, 13, 17, 19);

// array_map() applies a function to every element of an array,
// returning the result
$square = function($number) {
    return $number * $number;
};
$squared = array_map($square, $primeNumbers);
echo "The squares of those prime numbers are: ",
     implode(', ', $squared), "\n";

// array_reduce() applies a function recursively to pair
// of elements, reducing the array to a single value.
// there is the native array_sum(), but the application of
// a custom function is the interesting part
$sum = function($a, $b) {
    return $a + $b;
};
$total = array_reduce($primeNumbers, $sum);
echo "The sum of those prime numbers is ", $total, ".\n";

// array_filter() produces an array containing
// the elements that satisfy the given predicate
$even = function($number) {
    return $number % 2 == 0;
};
$evenPrimes = array_filter($primeNumbers, $even);
echo "The even prime numbers are: ",
     implode(', ', $evenPrimes), ".\n";

// uasort() customize the sorting by value,
// maintaining the association with keys
// there is the native asort(), but again the customization
// of the function is more interesting
$compare = function($a, $b) {
    if ($a == $b) {
        return 0;
    }
    return ($a > $b) ? -1 : 1;
};
uasort($primeNumbers, $compare);
echo "The given numbers in descending order are: ",
     implode(', ', $primeNumbers), ".\n";

Testing protected members

This is a follow-up to Testing private members.

In the previous post, we discussed how to make sure that code in a private method is tested without accessing it directly, since that would imply the use of reflection and a violation of the class contract (its public methods or its interface.)
Following the same procedure, let's find out why we end up with protected methods in our classes.
For starters, we should indeed write a method only if really necessary, to keep production code as simple and short as possible. This requirement is usually made explicit in a [unit] test and thereby automatically checked. Of course only public methods are directly called, while protected and private ones are called by public methods we thoroughly test (otherwise they would be simply deleted as unused code).
Though, there is a way a protected method can be reached from client code that a private one cannot: inheritance. Availability through inheritance is in fact the trait that distinguish protected class members from private ones.
Thus, there are two possible situations:

• the method is tested because at least a public method calls it. Given that the method logic does not warrant an external class, this arrangement would be fine.
• the method is intended to be called by Client subclasses, so it is not tested.

The latter case is the tricky one and there are two solutions I propose:

• write a simple subclass in a test case's source file and instantiate that class to test the protected method;
• refactor the method with a public signature on a collaborator class that will be passed to the constructor of the Client, effectively favoring object composition over an inheritance-based Api.

The second solution is by far the best one if the protected method contains real logic.
Providing functionality with inheritance seems useful by a raw count of the lines of code saved, but it is really an hassle as an application grows. First, in most languages only one parent is allowed for a class, and this restriction leads to long hierarchies where one never knows where a method is defined and overridden. For example, in Java a JFrame is a Frame, which is a Window, which is a Container, which is a Component, which is an Object.
Second, it leaves too many responsibilities on a Client class, which cannot select what members to inherit and if it could, it would not be able to discern the dependencies between different methods. Too much inheritance bloats the Api of the subclasses (the aforementioned JFrame has more than 300 public methods, and only 30 are not inherited) and the parent classes' one if they try to accomodate uncohesive features.
Finally, inheritance forces a dependency on a concrete class. For example historically, Object-Relational Mappers implemented an Active Record pattern. Now they are moving to a less invasive approach based on Plain Old Java/Php/.Net objects, without forcing the end user to inherit from their abstract classes and allowing him to play with objects without referencing a database.
With a moderate use of inheritance, there are really few good reasons to test a protected method per se. I hope from now on you will take a look at composition and dependency injection, which are dynamic dependencies, instead of always setting up static links between classes such as the extends keyword.

Repositories in Doctrine

Jebb wrote to me asking information about the Repository pattern:

I read your blog at http://giorgiosironi.blogspot.com/and I found it very helpful. I develop applications using DDD and Zend Framework however for persistence ignorance, I use some basic Repository as interface for DAOs. May I ask you how to implement Doctrine ORM in the Repository?

As I explain in depth in my previous post about this pattern, a Repository is an illusion of an in-memory collection of all persistent ignorant entities of the same class. A Repository may be abstracted beyond an interface defining only the mandatory methods that make sense in the domain model.
How an Orm fits in this discussion? Very few repositories accomplish their tasks alone: most of them compose an underlying generic Data Mapper layer which is usually an Object-Relational Mapper which translates classes and objects in tables and rows. A generic Data Mapper such as Doctrine 2 provides all the possibly imaginable operations on the collection of entities, and the Repository abstraction decouples the other parts of the application from knowing anything about an Orm. This is the primary advantage of a Repository.

As you may have thought at the persistence ignorance reference, with Doctrine 1 there are no chances to implement a real Repository. Doctrine 1 does not provide persistence ignorance because it requires entity classes to extend a base abstract Active Record.
In Doctrine 2, repositories can be implemented as Plain Old Php Objects: simply injecting the EntityManager and other collaborators they may need in the constructor will suffice. A Factory can then encapsulate the new operators.
This is the standard Repository pattern: a Plain Old Php Object which aggregates whatever is needed to hide the persistence mechanism of objects.

Doctrine 2 also provides a facility to quickly implement Repositories, but the freedom of implementation of this approach is limited. The process is described in the Doctrine manual and it consists in:

• extending an abstract class, which has a protected member $this->_em you can execute queries with;
• annotating the entity class with the class name of the concrete repository class;
• obtaining an instance with $em->getRepository('EntityClassName'): the EntityManager will create the object and automatically inject itself in the Repository.

Some notes on this architecture:

• there is no entity-specific interface;
• you extend an abstract class, which may be a cling because you want only some find*() methods to be available. However testing won't be affected since a concrete Repository will always involve a persistence mechanism.
• it would be useful for having different repositories referencing each other via the EntityManager; but you may manually inject them instead, maintaining the original solution.
• Doctrine 2 default Repository class is instantiated instead if you do not specify a subclass; if you use a lot of repositories and inject some of them in other service classes, the default implementation will be very handy. This would be a pro for taking advantage of the Orm support instead of using POPOs.

Tuning your Ubuntu machine with command line-fu

Ubuntu is in my opinion the leading Linux distribution because of its extended support for peripherals and large software repositories. Though, the default installation profile can be a bit heavy since it is thought as a catch-all configuration for a general purpose system.
Thus many software developers have the need for lightening the system load and reducing the disk space occupied by the distribution, along with the Ram eaten by daemons. This guide also works with Debian boxes since it is based on the dpkg and apt packaging system and on standard unix tools.

Let's open the hood and start with an example of unrequired packages. If you search packages that match the name ttf*:

dpkg -l ttf* | grep ii

you'll see the list of font packages installed. This collection comprehends Japanese, Korean, Gothic fonts, and so on till Futurama Alien Alphabet. You may want to remove some of them if you do not understand the languages which are written with these fonts.
Note that sometimes you will be asked to remove a package with a seemingly important name, such as ubuntu-standard. These metapackages are simply empty debs that depend on a large set of normal packages; they are meant as a shortcut for installing all those packages in a single shot: the Ubuntu installer simply requests the installation of ubuntu-standard and the apt system works out the details.

Let's try a more radical way to find space to free on the distribution partition:

dpkg-query --show --showformat='${Package}\t${Installed-Size} ${Status}\n' \
    | grep -v deinstall | sort -k 2 -n \
    | awk '{printf "%.1f MB \t %s\n", $2/(1024), $1}' \
    | tail -n 20

When I say that Unix tools are beautiful, now you'll know why.
This commands chain will show you the 20 packages with highest installed size, so that you can remove them with sudo apt-get remove. For example, I removed ubuntu-docs (hundreds of megabytes) and the openoffice Australian thesaurus (I'm not very keen on searching "synonyms" in the other emisphere.)

You may want to install some very small tools to simplify the management of your system:

• bum is a tool to enable and disable services at the bootstrap (this is actually a graphical tool, if you want you can play with /etc/rc2.d/);
• deborphan is a command line utility which lists packages that no other .deb depends on. deborphan | grep lib shows the list of libraries that are not used and so can be removed.

Remember to execute these commands periodically:

• sudo apt-get autoremove removes packages installed to satisfy dependencies that are now useless. For instance if you install the vlc video player and it requires also 100 MB of Qt libraries, if you subsequently remove vlc the libraries are still there. autoremove will delete such packages: if you want to keep some of them, sudo apt-get install package-name will set package-name to manually installed, making you free to execute autoremove on the remaining packages.
• sudo apt-get clean deletes cached *.deb files which have been downloaded in the past. After a successful installation, there is no need to keep them around.

I hope these tips can help you shrink your distro impact on machine resources. I have a 2 GB root partition on my EeePC, and in my case it is very important to remove packages installed by default that do not have a real use.

Practical Php Patterns: Command

This post is the second one in the behavioral patterns part of the Practical Php Pattern series.

The behavioral pattern we will discuss today is the Command one, a mechanism for encapsulation of a generic operation.
If you are familiar with C or Php, you have probably already encountered Command as its procedural equivalent: the callback, which is usually implemented as a function pointer or a data structure such as a string or an array in php.
Command is an abstraction over a method call, which becomes a first-class object with all the benefits of object orientation over a set of routines: composition, inheritance and handling.
For example, the GoF book proposes to use Commands to store a chain of user actions and supporting undoing and redoing operations.
Note that php 5.3 functional programming capabilities (Closures) can be used as a native implementation of the Command pattern. Though, there is an advantage in type safety in using an abstract data type for every Command hierarchy.

In this pattern, the Invoker knows that a Command is passed to it, without dependencies on the actual ConcreteCommand implementation. The solved problem is the association of method calls by configuration: for instance ui controls like buttons and menus refer to a Command and assume their behavior by composing a generic ConcreteCommand instance.

Participants:

• Command: defines an abstraction over a method call.
• ConcreteCommand: implementation of an operation.
• Invoker: refers to Command instances as its available operations.

The code sample provides Validator components implemented as Command objects.

<?php
/**
 * The Command abstraction.
 * In this case the implementation must return a result,
 * sometimes it only has side effects.
 */
interface Validator
{
    /**
     * The method could have any parameters.
     * @param mixed
     * @return boolean
     */
    public function isValid($value);
}

/**
 * ConcreteCommand.
 */
class MoreThanZeroValidator implements Validator
{
    public function isValid($value)
    {
        return $value > 0;
    }
}

/**
 * ConcreteCommand.
 */
class EvenValidator implements Validator
{
    public function isValid($value)
    {
        return $value % 2 == 0;
    }
}

/**
 * The Invoker. An implementation could store more than one
 * Validator if needed.
 */
class ArrayProcessor
{
    protected $_rule;

    public function __construct (Validator $rule)
    {
        $this->_rule = $rule;
    }

    public function process(array $numbers)
    {
        foreach ($numbers as $n) {
            if ($this->_rule->IsValid($n)) {
                echo $n, "\n";
            }
        }
    }
}

// Client code
$processor = new ArrayProcessor(new EvenValidator());
$processor->process(array(1, 20, 18, 5, 0, 31, 42));

Some implementation notes for this pattern:

• some of the parameters for the method call can be provided at the construction of the ConcreteCommand, effectively currying the original method.
• A Command can be considered as a very simple Strategy, with only one method, and the emphasis put on the operation as an object you can pass around.
• ConcreteCommands also compose every resource they need to achieve their goal, primarily the Receiver of the action where they call method to execute a Command.
• Composite, Decorator and other patterns can be mixed with the Command one, obtaining multiple Commands, decorated Commands and so on.

Testing private members

Yesterday, Sebastian Bergmann, the author of PHPUnit, posted his response to a question he is asked frequently, How do I test private members of a class?
He summarizes the ways to access private fields and methods of an object from a PHPUnit test case, and he concludes that being able to test them does not mean it is a good thing. Today I am going to explain why testing private methods explicitly is considered a bad practice.

Why do we limit the scope of a method to private or protected to begin with? To be able to subsequently refactor the code and change the signature or the behavior of the method accordingly to new requirements or what we have learned about the problem domain.
The same reasoning is even more valid for field references, which cannot encapsulate a computation and are simply stored variables. I haven't created public fields in production code for years.
Thus the first reason is: a test that exercises a private member couples the private member to code that is external to the system under test. This means you cannot safely delete or modify a private method simply by modifying a class source file: you will have to update the test even if the external behavior of the SUT did not change.

Sometimes you find a private method which contains purposeful logic, and you may want to cover it with specific testing. If this is the case, I suggest to move such method in an extracted class whose instance is stored as a private reference. This way you'll be able to test both the extracted class and the original one, maybe even mocking out the private method in the latter's unit tests. The abstraction provided by the original class is left untouched.
This solution is useful particularly when testing a private method becomes simpler than testing a public one of the same class; it is the sign that another interface is needed between the class and its private member. You're establishing a contract anyway, by writing a test for it: so why do not make this contract explicit and give it citizenship with a class?
The Single Responsibility Principle is the most abstract and the most overlooked of all object-oriented techniques.

Now let's tackle the same problem in the contest of Test-Driven Development. Remember what TDD means: writing code only to satisfy a unit test, while not being allowed to write more code than the really mandatory lines to get a green bar.
How do you get to write a method if you're doing TDD? There are two possible cases:

• the method is written to make a test pass directly; only public methods can serve this purpose if you do not let test code tinker with class internals.
• The method is written as a consequence of internal refactoring; but if you're refactoring, there are tests that cover the interested functionalities, so the method is tested anyway via public methods.

Thus the original question is similar to How do I test an abstract class? Since an abstract class can be possibly created only by extracting a superclass from well-tested concrete classes, it should not be tested as well as private methods should not be tested, because they are exercised by definitions by subclasses and public entities. If they weren't exercised, we would have simply thrown them away as unreachable code. If they need more exercising code (maybe they are not in shape :), they should declare a public contract and reside on an external class.
The moral of this analysis is: Test-Driven Development once again saves the day and makes sure our code is covered; unit testing becomes really hard only if you do not write tests first.

Practical Php Patterns: Chain of Responsibility

This post starts the behavioral patterns part of the Practical Php Pattern series.

The first behavioral pattern of this part of the series is named Chain of Responsibility. Its intent is organizing a chain of objects to handle a request such as a method call.
When a ConcreteHandler does not know how to satisfy a request from a Client, or it is not designed at all to do so, it delegates to the next Handler in the chain, which it maintains a reference to.
This pattern is often used in conjunction with Composite, where some Leaf or Container object delegates an operation to their parent by default. As another example, localization is often handled with a Chain of Responsibility: when the German translation adapter does not find a term for a translation key, if falls back to the English language adapter or even to displaying the key itself.

The coupling is reduced to a minimum: the Client class does not know which concrete class handles the request; the chain is configured at the creation of the object graph, at runtime, and ConcreteHandlers do not know which object is their successor in the chain. The behavior is successfully distributed between the objects, where the nearest object in the chain has priority on stepping in and assuming the responsibility to satisfy the request.

Participants:

• Client: makes a request to an Handler.
• Handler abstraction: accepts a request and satisfies it in some way.
• ConcreteHandlers: accept a request, try to satisfy it and delegate to the next handler if not successful.

The code sample implements one of the most famous examples of a Chain of Responsibility: a multi-level cache.

<?php
/**
 * The Handler abstraction. Objects that want to be a part of the
 * ChainOfResponsibility must implement this interface directly or via
 * inheritance from an AbstractHandler.
 */
interface KeyValueStore
{
    /**
     * Obtain a value.
     * @param string $key
     * @return mixed
     */
    public function get($key);
}

/**
 * Basic no-op implementation which ConcreteHandlers not interested in
 * caching or in interfering with the retrieval inherit from.
 */
abstract class AbstractKeyValueStore implements KeyValueStore
{
    protected $_nextHandler;

    public function get($key)
    {
        return $this->_nextHandler->get($key);
    }
}

/**
 * Ideally the last ConcreteHandler in the chain. At least, if inserted in
 * a Chain it will be the last node to be called.
 */
class SlowStore implements KeyValueStore
{
    /**
     * This could be a somewhat slow store: a database or a flat file.
     */
    protected $_values;

    public function __construct(array $values = array())
    {
        $this->_values = $values;
    }

    public function get($key)
    {
        return $this->_values[$key];
    }
}

/**
 * A ConcreteHandler that handles the request for a key by looking for it in
 * its own cache. Forwards to the next handler in case of cache miss.
 */
class InMemoryKeyValueStore implements KeyValueStore
{
    protected $_nextHandler;
    protected $_cached = array();

    public function __construct(KeyValueStore $nextHandler)
    {
        $this->_nextHandler = $nextHandler;
    }

    protected function _load($key)
    {
        if (!isset($this->_cached[$key])) {
            $this->_cached[$key] = $this->_nextHandler->get($key);
        }
    }

    public function get($key)
    {
        $this->_load($key);
        return $this->_cached[$key];
    }
}

/**
 * A ConcreteHandler that delegates the request without trying to
 * understand it at all. It may be easier to use in the user interface
 * because it can specialize itself by defining methods that generates
 * html, or by addressing similar user interface concerns.
 * Some Clients see this object only as an instance of KeyValueStore
 * and do not care how it satisfy their requests, while other ones
 * may use it in its entirety (similar to a class-based adapter).
 * No client knows that a chain of Handlers exists.
 */
class FrontEnd extends AbstractKeyValueStore
{
    public function __construct(KeyValueStore $nextHandler)
    {
        $this->_nextHandler = $nextHandler;
    }

    public function getEscaped($key)
    {
        return htmlentities($this->get($key), ENT_NOQUOTES, 'UTF-8');
    }
}

// Client code
$store = new SlowStore(array('pd' => 'Philip K. Dick',
                             'ia' => 'Isaac Asimov',
                             'ac' => 'Arthur C. Clarke',
                             'hh' => 'Helmut Heißenbüttel'));
// in development, we skip cache and pass $store directly to FrontEnd
$cache = new InMemoryKeyValueStore($store);
$frontEnd = new FrontEnd($cache);

echo $frontEnd->get('ia'), "\n";
echo $frontEnd->getEscaped('hh'), "\n";

Some implementation notes:

• the Chain of Responsibility may already exist in the object graph, like in the Composite's case.
• moreover, the Handler abstraction may already exist or not. The best choice is a segregated Handler interface with only the handleRequest() operation; do not force a chain in a hierarchy only because the latter already exists.
• there is also the possibility of introducing an abstract class but since the request handling is an orthogonal concern, the concrete classes may already inherit from other classes.
• the Handler (or next Handler) is injected in the Client or in the previous Handler via the constructor or a setter.
• the request object is often a ValueObject and may be implemented as a Flyweight. In php, it can be a scalar type such as a string; note that in some languages a String is an immutable ValueObject.