How to avoid phase-of-the-Moon bugs

Wikipedia documents some definitions of particularly dangerous bugs, which are very hard to fix and when they manifest constitute a real problem in development.
Pay attention to these examples:

• Heisenbug: bug that changes its behavior when someone is trying to reproduce it. The very attempt to study it changes the conditions (or requires irreproducible conditions) so that the Heisenbug does not manifest anymore. The naming of this bug is based on Heisenberg's uncertainty principle.
• Phase of the Moon bug: bug that arises from a dependency on external conditions, usually time. In the linked original definition, a piece of software has a marginal dependency on the Moon phase. Surely software developers should not feverishly expect monthly events to determine if their application works (unless it's some werewolves-oriented project.)

These are two categories that are really dangerous and can bring development to an halt by forcing long debugging session to find the cause of the failure, which is "non-deterministic" (in the former case) or hidden (in the latter). They certainly look scary but I chose them as examples because these particular bugs can be avoided by employing some engineering practices, such as a good test suite.

The test suite for a project should be mainly composed by unit tests. While acceptance end-to-end tests constitute an important part of it, because they validate the application's fulfillment of its requirements, unit tests are usually more powerful even if they do not drive the design. Their potential is to quickly locate defects, by testing the contract of individual classes: the first step in exposing a bug, and particularly an Heisenbug, consists in locating it, and being able to reproduce it reliably every time that is needed, to check that the bug has been fixed. A well-written test suite provides an automatic way to check, at the push of a button, that all previously fixed bugs have not reappeared, so that there are no regressions.
The other advantage of unit tests is in the way they promote isolation. External dependencies are usually mocked out in the testing environment, so that boundary conditions can be reproduced at will. If you suspect that a piece of software may fail during a combination of unusual date and time, you only have to add a test case where you provide the set of conditions that you are scared about.
The isolation of components is not limited to external capricious dependencies, such as time and database state. Mutable global state can also be avoided just by maintaining a unit test suite, primarily because it makes the tests brittle and difficult to write since they may fail depending on their execution order. A unit test should be able to coherently fail or pass both when the single test method is run as when the whole suite is. If  you're facing global state clings in the testing environment, which may lead to Heisenbugs and similar issues in production, listening to the tests will tell you to change your design to accomodate a simpler testing procedure, and a overall better architecture.
I agree completely to the Google Testing blog's motto:

Debugging sucks. Testing rocks.

Asserting out of tests

In programming, assertions are statements that should always evaluate to true, being invariant assumptions in respect to the input data of a program. From the mathematical point of view assertions are tautologies for the implementation which they are sunk in.
For instance, you can code assertions which verify that the input data consist of strings, or that the result of a calculation is coherent with the program flow. A failed assertion usually marks a logical bug.
Unit tests are disseminated with assertions, given the advantage that xUnit assertions are contained in test cases and thus separated from production code. However there are other places where assertions are used; many compiler or interpreter checks in modern programming languages are implicit assertions that provide type safety or other automatic controls:

• As I said earlier, assert*() methods like assertEquals() and assertTrue() are provided by instances of test cases to allow specification of behavior. These assertions are treated in detail in the relative testing series article.
• The assert() function (sometimes implemented as a macro in languages like C) in production code is the fastest way to check the correct flow of the program. The php assert() function takes as an argument a php expression (which is simply code that evaluates to a boolean) encoded in a string; the encapsulation in a string variable allows for the assertions to be skipped where particular flags are set.
• Type hinting on function parameters is actually a masqueraded assert(). In php, the assertion code would be assert('$param instanceof MyClass');
• Database constraints are commonly declared in the form of assertion in Sql code. For example, the result of some queries should be invariant or the value entered in a column should match a restriction of the field domain.

Given the various assertions you can make in different parts of your application, you should be cautious in inserting them in production environments. While in unit tests assertions are fundamental (but you shouldn't exaggerate with their number to simplify maintenance), typically explicit assertions are disactivated in live deployments. This behavior is preferred to leave checks in place, to avoid exposing errors to the real user and to speed up code execution.
Also caution should be used with database constraints if you work with an object model and an Orm, as they may result in logic duplication.

Failed assertions should be managed someway. Php lets you declare an assertion handler, which I usually set to a small function that throws a special exception with a message containing the error generated by the assertion code. I see failed assertions as a very serious problem which may indicate a bug, while normal exceptions often are used to signal incorrect inputs or state conditions that cause an error.
Some assertions get in the way of tests too: when we encounter such assertions we should demand why they exist in the first place, if they should be disactivated in production and also in testing. A common example is the null check/type hinting:

<?php 
class MyClass
{
public function __construct(MyClass $param)
{
    ... 

While Java would allow the client code to pass null as the value of $param, php raises a catchable fatal error, which stops the constructor execution. This means that if we don't need some collaborators in testing a particular method, we are forced to subclass MyClass to override the constructor, or to create fake collaborators only to fill the parameters list. If these collaborators require not-null parameters in the constructor, the problem becomes recursive.
So I prefer not to make unnecessary assumptions on the input parameters of constructors:

<?php 
class MyClass
{
public function __construct(MyClass $param = null)
{
    ...

The same problem arises in a different form for scalar type hinting, because it is not available in the syntax but can be implemented by the programmer:

public function __construct($config)
{
    assert('is_string($config)');
    ... 
}

Just do not assume $config is a string if there is even the remote possibility that tests will exercise the class without a config variable. Only if $config is invariably needed for the class to work we should check its type and structure.
Of course, we should also test in some way that the class is correctly instantiated, but this part should be covered by integration tests, or unit tests for the factory or the container. Integration errors are easy to spot since calling a method on null is not allowed, and if the construction process is not complete the first access to the missing collaborator stops the execution of the entire suite.
Now that you know the power of assertions, try to take the best out of them as they are not substitutes for separate unit testing, but can be replaced by unit tests in many cases.

Failed attempt to apply Tdd response

This is a response to the post "Why my attempt to apply tdd failed?" on the whyjava blog. I am a test-infected developer and I want to give some support to whoever had the courage to change and adopt new methodologies.

In the post the author makes a list of points which summarize the difficulties encountered while trying out Test-Driven Development. I have some comments on this list, but before analyzing it in depth I remind you that the TDD mantra is red-green-refactor. Keep in mind it every single minute since it's easy to drift away without noticing it.

1. Time lost: this is a classic myth about TDD, but it's true that initially writing tests will require time that has to be subtracted from writing production code. But if you look at the overall picture, you will find out that tests will save you more time in the long run that the amount spent to write them. While debugging and maintaining an application having a safety net composed by good unit tests is an aid that I won't exchange even with source control. Tests also act as a documentation and this will save time for new developers who has to learn how the components works together. They also keep regression out of your project.
2. Boring to write test cases: while practicing TDD, a developer has to find his own balance within test the obvious and not test anything. Experience at this game will make you not test getters and setters.
3. Client support: the client does not want the developers to write tests. Besides the fact the working in this way the client is never going to know if the application works, I would find strange to tell my surgeon I don't want you to use that bistoury, I have my Miracle Blade at home.
4. Writing exhaustive test cases: "I think tdd will make sense when you are able to cover most of the scenarios in your test class". You're right, but since in TDD you write the test cases before any production code, and only the absolutely necessary production code to make those tests pass, there's only covered scenarios. If you find a feature which is not covered by a unit test it means it has been developed before them, which is not TDD anymore.
5. Patience: of course every technique has its learnign curve. TDD is no exception.
6. Poor examples: if you think books are teaching you by simplicistic situations, simply download an open source software which is developed with TDD.
7. Legacy code: it's hard to replace old (maybe procedural) code with a TDD approach, and this is a valid point. However, you should apply TDD for new features if the design allows this.

I hope to have inspired the reader to embrace Test-Driven Development, or to continue his journey in it. Becoming test-infected was like seeing the Light for me, and I think I'm becoming a better developer every day because testable code leads to decoupled and reusable code.

How to stop getting megabytes of text when dumping an object

Php has a useful var_dump() function that helps debugging and unit tests writing by providing a text dump of all the properties of an object, being them public, private or protected. However, with an object that has a reference to the whole object graph, like a factory or an object that uses an abstract factory, the dump will be recursive and so long that it runs forever (probably it lasts several megabytes). Here's how to avoid it.

In the browser
When you are using a browser, the Xdebug extension comes in your help by generating a html dump truncated at an arbitrary level of depth. The installation on a unix box is pretty simple:
@ pecl install xdebug
run as root or with sudo. Pecl is the dual repository system of Pear and contains C extensions instead of Php userland packages; it is provided along with the pear binary in most of the installation of php.
Then add the following lines to php.ini:

zend_extension="/usr/local/lib/php/extensions/no-debug-non-zts-20090626/xdebug.so"
xdebug.var_display_max_depth=2

Probably the first has been already placed from the installer. The second directive set the maximum depth of var_dump(), which is overloaded by the extension, to 2 levels.
There are other directives that influences the behavior of var_dump(), and more and more for activation of other xdebug features.
Make also sure that html_errors is on in php.ini, or ini_set() it.

On the command line
The command line environment is more complex to use, but it is the ideal for debugging, given its speed and automation capability. The var_dump() usage must be tweaked because the var_dump() overloading, as of Xdebug 2.0, works only with html_errors active, producing an html dump that is ugly to view as plain text.
Assuming you have done the same setup of the browser section above, here's a workaround method to use:

function dump($var)
{
ini_set('html_errors', 'On');
ob_start();
var_dump($var);
$dump = ob_get_contents();
ob_end_clean();
echo strip_tags(html_entity_decode($dump));
}

It will get the html output of var_dump() even in a Cli environment that has the html_errors directive not set, and clean it to display as plain text.
Here's what I get dumping a PersistentCollection of Doctrine 2 now:

[16:54:23][giorgio@Marty:~/svn/doctrine2/tests]$ phpunit Doctrine/Tests/ORM/Functional/DetachedEntityTest.php
PHPUnit 3.3.17 by Sebastian Bergmann.
object(Doctrine\ORM\PersistentCollection)[180]
private '_type' => null
private '_snapshot' =>
 array
   empty
private '_owner' => null
private '_association' => null
private '_keyField' => null
private '_em' => null
private '_backRefFieldName' => null
private '_typeClass' => null
private '_isDirty' => boolean true
protected '_initialized' => boolean true
protected '_elements' =>
 array
   0 =>
     object(Doctrine\Tests\Models\CMS\CmsPhonenumber)[181]
       ...
   1 =>
     object(Doctrine\Tests\Models\CMS\CmsPhonenumber)[182]

The '...' ellipsis are added by Xdebug instead of showing thousands of objects and properties, because the PersistentCollection is provided with a reference to EntityManager for lazy-loading itself.
With the old var_dump() instead:

[16:54:23][giorgio@Marty:~/svn/doctrine2/tests]$ phpunit Doctrine/Tests/ORM/Functional/DetachedEntityTest.php > test.txt

I redirect the output to a text file because it is impossible to navigate from a terminal, but you have to expect some minutes because it takes forever to generate a full dump of the EntityManager and all other objects involved. Without redirecting lines are sent on terminal for a unspecified long time, and are also duplicated because var_dump() is not so smart in detecting recursion and it let happen for a while before stopping to output the same objects over and over. It was very frustrating but now it is easy to view even the most complex and coupled objects.
I know that the more an object is coupled, the less it is well written; but even if it only depends on an interface (a parameter in the constructor), at runtime that interface could be implemented by a very heavy object. Decoupling limits this process, but object must communicate and so they have to keep references to other collaborators: the Publish/Subscribe paradigm is very loose coupled, but dumping a publisher will output it, the blackboard and all the subscribers on Earth.
Summing up, Xdebug can boost your php productivity, and has many other features that will make easy to see what happens under the hood of your php application. Profiling and tracing are easily provided with other xdebug.* directives. If you are a php developer, install it as soon as possible.