Monday, November 30, 2009

Asserting out of tests

In programming, assertions are statements that should always evaluate to true, being invariant assumptions in respect to the input data of a program. From the mathematical point of view assertions are tautologies for the implementation which they are sunk in.
For instance, you can code assertions which verify that the input data consist of strings, or that the result of a calculation is coherent with the program flow. A failed assertion usually marks a logical bug.
Unit tests are disseminated with assertions, given the advantage that xUnit assertions are contained in test cases and thus separated from production code. However there are other places where assertions are used; many compiler or interpreter checks in modern programming languages are implicit assertions that provide type safety or other automatic controls:
  • As I said earlier, assert*() methods like assertEquals() and assertTrue() are provided by instances of test cases to allow specification of behavior. These assertions are treated in detail in the relative testing series article.
  • The assert() function (sometimes implemented as a macro in languages like C) in production code is the fastest way to check the correct flow of the program. The php assert() function takes as an argument a php expression (which is simply code that evaluates to a boolean) encoded in a string; the encapsulation in a string variable allows for the assertions to be skipped where particular flags are set.
  • Type hinting on function parameters is actually a masqueraded assert(). In php, the assertion code would be assert('$param instanceof MyClass');
  • Database constraints are commonly declared in the form of assertion in Sql code. For example, the result of some queries should be invariant or the value entered in a column should match a restriction of the field domain.
Given the various assertions you can make in different parts of your application, you should be cautious in inserting them in production environments. While in unit tests assertions are fundamental (but you shouldn't exaggerate with their number to simplify maintenance), typically explicit assertions are disactivated in live deployments. This behavior is preferred to leave checks in place, to avoid exposing errors to the real user and to speed up code execution.
Also caution should be used with database constraints if you work with an object model and an Orm, as they may result in logic duplication.

Failed assertions should be managed someway. Php lets you declare an assertion handler, which I usually set to a small function that throws a special exception with a message containing the error generated by the assertion code. I see failed assertions as a very serious problem which may indicate a bug, while normal exceptions often are used to signal incorrect inputs or state conditions that cause an error.
Some assertions get in the way of tests too: when we encounter such assertions we should demand why they exist in the first place, if they should be disactivated in production and also in testing. A common example is the null check/type hinting:
class MyClass
public function __construct(MyClass $param)
While Java would allow the client code to pass null as the value of $param, php raises a catchable fatal error, which stops the constructor execution. This means that if we don't need some collaborators in testing a particular method, we are forced to subclass MyClass to override the constructor, or to create fake collaborators only to fill the parameters list. If these collaborators require not-null parameters in the constructor, the problem becomes recursive.
So I prefer not to make unnecessary assumptions on the input parameters of constructors:
class MyClass
public function __construct(MyClass $param = null)
The same problem arises in a different form for scalar type hinting, because it is not available in the syntax but can be implemented by the programmer:
public function __construct($config)
Just do not assume $config is a string if there is even the remote possibility that tests will exercise the class without a config variable. Only if $config is invariably needed for the class to work we should check its type and structure.
Of course, we should also test in some way that the class is correctly instantiated, but this part should be covered by integration tests, or unit tests for the factory or the container. Integration errors are easy to spot since calling a method on null is not allowed, and if the construction process is not complete the first access to the missing collaborator stops the execution of the entire suite.
Now that you know the power of assertions, try to take the best out of them as they are not substitutes for separate unit testing, but can be replaced by unit tests in many cases.


TheSorrow said...

Hmm, php is not that far from design by contract.
with assert you can verify your pre/post condition.

Is there a way to deactivate assert (like in java) ?

Giorgio said...

assert_options(ASSERT_ACTIVE, false);

Tom Van Herreweghe said...

Design by contract would be a nice addition to PHP. But for that, the PHPDoc should be parsed on run time. Or it should be checked before you deploy, like unit-tests.

I wonder if DBC will be available in the future.

Giorgio said...

"Design by contract" is a very overloaded term. How would you specify a contract in the docblocks?

Anonymous said...

hi all

I figured it would be a good idea to introduce myself to everyone!

Can't wait to start some good conversations!


Thanks again!

Anonymous said...


just registered and put on my todo list

hopefully this is just what im looking for looks like i have a lot to read.