Introducing NakedPhp 0.1

I dedicated a bit of my time to develop a port of the Naked Objects framework for Java, which I named NakedPhp. Essentially, the Naked Objects pattern is the automatic animation of an object-oriented Domain Model via the generation of the user interface and of the persistence layer, both commonly sources of duplication and kitchen sinks for business logic. Naked Objects is a radical reinterpretation of MVC, where the Model is at the center and the other layers are inferred from it and customized. You will call methods and move objects around in such an application.
This is NOT yet another Php framework: NakedPhp leverages Zend Framework and Doctrine for all the stuff you would expect a normal framework to do.

Previous related posts
Naked objects, DDD and the user interface
A look at technical question on Naked Objects
Naked objects in Php
Where is business logic?

NakedPhp applies the same pattern to Php applications, and I have tagged its 0.1 release today (this is an alpha release.) The Api is borrowed from the original Java Naked Objects framework. This is by no means a complete framework, but it's a good start since its basic CRUD functions are already working.
NakedPhp integrates Zend Framework 1.x for the user interface management and Doctrine 2 for the persistence layer. As for all Doctrine 2 applications, it requires Php 5.3, which is why I don't have an online demo right now.
Links
NakedPhp 0.1 on SourceForge
Bug tracker
Git repository (view it online):
git clone git://nakedphp.git.sourceforge.net/gitroot/nakedphp/nakedphp

A simple example application realized with NakedPhp is provided in the released package, which uses a bundled sqlite database. An application for NakedPhp is just a plain old Zend Framework application with a controller that inherits from NakedPhp\Mvc\Controller, and that inits the Entitymanagerfactory (related to Doctrine 2) and Nakedphp resources with the mandatory options like the path to the model classes's folder. NakedPhp is very liberal about what you do in your application: it only generates the first-step views and leaves you the layout for customization.
I will write related documentation about how to hook NakedPhp in an application in the next days which I will store along with the code in the Git repository on SourceForge.
The behavior of the generated interface (which is not scaffolding: it's intended for actual use and not for being modified) is driven by annotations on the model classes and by methods with special names. For example, properties available on Entity objects are inferred from getters and their modification is possible if correspondent setters are present. Other methods are called when available to determine automatic hiding of properties and methods (hideName(), hideFindAllCities()), validation of data and so on.

The sample application provides a basic workflow for an hypothetical Domain Model for shops, pubs, or similar places. You can create Cities and Places, modify their properties, save them in the database, then clear the session and retrieve them via service classes. This is a direct interface to the Domain Model, without translations.
Intructions are included in the release for the super-simple setup of the example application (essentially running phing build-example).

Let me know if you are interested in a framework that writes the user interface for you. I know that the Naked Objects pattern is not appropriate for all situations, but the goal is to simplify the presentation of the real Domain Model to the user for certain kinds of model-driven applications. It is also providing a prototyping interface for Domain-Driven Design in Php.

Repositories in Doctrine

Jebb wrote to me asking information about the Repository pattern:

I read your blog at http://giorgiosironi.blogspot.com/and I found it very helpful. I develop applications using DDD and Zend Framework however for persistence ignorance, I use some basic Repository as interface for DAOs. May I ask you how to implement Doctrine ORM in the Repository?

As I explain in depth in my previous post about this pattern, a Repository is an illusion of an in-memory collection of all persistent ignorant entities of the same class. A Repository may be abstracted beyond an interface defining only the mandatory methods that make sense in the domain model.
How an Orm fits in this discussion? Very few repositories accomplish their tasks alone: most of them compose an underlying generic Data Mapper layer which is usually an Object-Relational Mapper which translates classes and objects in tables and rows. A generic Data Mapper such as Doctrine 2 provides all the possibly imaginable operations on the collection of entities, and the Repository abstraction decouples the other parts of the application from knowing anything about an Orm. This is the primary advantage of a Repository.

As you may have thought at the persistence ignorance reference, with Doctrine 1 there are no chances to implement a real Repository. Doctrine 1 does not provide persistence ignorance because it requires entity classes to extend a base abstract Active Record.
In Doctrine 2, repositories can be implemented as Plain Old Php Objects: simply injecting the EntityManager and other collaborators they may need in the constructor will suffice. A Factory can then encapsulate the new operators.
This is the standard Repository pattern: a Plain Old Php Object which aggregates whatever is needed to hide the persistence mechanism of objects.

Doctrine 2 also provides a facility to quickly implement Repositories, but the freedom of implementation of this approach is limited. The process is described in the Doctrine manual and it consists in:

• extending an abstract class, which has a protected member $this->_em you can execute queries with;
• annotating the entity class with the class name of the concrete repository class;
• obtaining an instance with $em->getRepository('EntityClassName'): the EntityManager will create the object and automatically inject itself in the Repository.

Some notes on this architecture:

• there is no entity-specific interface;
• you extend an abstract class, which may be a cling because you want only some find*() methods to be available. However testing won't be affected since a concrete Repository will always involve a persistence mechanism.
• it would be useful for having different repositories referencing each other via the EntityManager; but you may manually inject them instead, maintaining the original solution.
• Doctrine 2 default Repository class is instantiated instead if you do not specify a subclass; if you use a lot of repositories and inject some of them in other service classes, the default implementation will be very handy. This would be a pro for taking advantage of the Orm support instead of using POPOs.

How an Orm works

Some readers have been confused by the terms I use often in reference to Object relational mappers, so I want to describe some concepts of Orms and make some definitions. Particularly I want to focus on how a real Orm works and lets you write classes that do not extend anything (Plain Old Php Objects or Plain Old <insert oo-language here> Objects).
The persistence-abstraction standard is Java Persistence Api, which was extracted from Hibernate, and I will refer to it in this post. Doctrine 2 is the Orm which ports the specification in the php world and will be the reference implementation of these concepts in the explanation that follows.

The primary classification of Domain Model classes consists in dividing them in two categories: Entities, which primary responsibility is to maintain the state of the application, and Services, which responsibility is to perform operations that involves more than one Entity, and to link to the outside of the domain model, breaking direct dependencies. This distinction leaves out Specifications, Value Objects, etc., which add richness to a model but are less crucial parts of it. Repositories and Factories are still a particular kind of Service.
I know that primary responsibility of a class sounds bad, since a class should have only one responsibility; though, there is a trade-off between responsibility and encapsulation and an Entity class should certainly hide the workings of many operations that involve only its private data.
Examples of Entity class are User, Group, Post, Forum, Section, and so on. Typical Service class names can be UserRepository, UserFactory, HttpManager, TwitterClient, MyMailer. You can often recognize entities from their serializability.
Imagining that you are going to take advantage of an Orm's features, once you have your Entity classes defined it's up to you to define their mapping to relational tables in a format that the Orm understands - xml, yaml, ini files, or simple annotations. The Orm will use this information not only to move objects back and forth from the database, but also to create and maintain your schema, thus without introducing duplication.
The mapping consists of metadata that describe what properties of an entity you want to store, and how. There are multiple ways to map objects to tables and an Orm should not just invent how to fit them in a database.
Java annotations are objects which provide compile-time checks, while in php they are only comments included in the docblock due to lack of native support. This also means that with Doctrine 2 there is no dependency from the Entity class file to the Orm source code.
This is the simplest Entity I can think of, a City class, complete with mapping for Doctrine 2:

<?php
/**
 * Naked Php is a framework that implements the Naked Objects pattern.
 * @copyright Copyright (C) 2009  Giorgio Sironi
 * @license http://www.gnu.org/licenses/lgpl-2.1.txt
 *
 * This library is free software; you can redistribute it and/or
 * modify it under the terms of the GNU Lesser General Public
 * License as published by the Free Software Foundation; either
 * version 2.1 of the License, or (at your option) any later version.
 *
 * @category   Example
 * @package    Example_Model
 */

/**
 * @Entity
 */
class Example_Model_City
{
    /**
     * @Id @Column(type="integer")
     * @GeneratedValue(strategy="AUTO")
     */
    private $_id;

    /**
     * @Column(type="string")
     */
    private $_name;

    public function __construct($name)
    {
        $this->setName($name);
    }

    /**
     * @return string   the name
     */
    public function getName()
    {
        return $this->_name;
    }

    public function setName($name)
    {
        $this->_name = $name;
    }

    public function __toString()
    {
        return (string) $this->_name;
    }
}

Private properties are accessed via reflection.

A JPA-compliant Orm presents a single point of access to its functionalities: the Entity Manager, which is a Facade class. You should now understand the meaning of its name.
The Entity Manager object usually has two important collaborators: the Identity Map and the Unit Of Work, plus the generated proxy classes which serve for many purposes:

• the Identity Map is - as the name suggests - a Map which maintains a reference to every object which has been actually reconstituted from the database, or that the Orm knows somehow (e.g. because it has been told to persist it explicitly).
  
• Proxies, whose classes are generated on the fly, substitute a regular object in the graph with a subclass instance capable of lazy loading itself if and only if needed. The methods of the Entity class are overridden to execute the loading procedure before dispatching the call to the original versions.
  
• The Unit Of Work calculates (or maintains) a diff between the object graph and the relational database; it commits everything at the end of a request, or session, or when the developers requires so.

The shift in the workflow is from the classic ActiveRecord::save() method to the EntityManager::flush() one. It is a developer's responsibility to maintain a correct object graph, but it is the Orm's one to reflect the changes to the relational database. The power of this approach resides in letting you work on an object graph as it were the (almost) only version you know of the Domain model.

Php technologies' grades

 Last week I was asked by a client:

You used Zend Framework for this small php application. From 1 to 10 [which is the grade framework in Italian secondary schools], how much is this technology sophisticated comparing to the other ones in the php world?

I answered:

8 or 9, I can't think about a more advanced php technology (and with a so much steep learning curve), unless I think about Doctrine 2.

Before Symfony and CodeIgniter developers bite me: given the occasion, I would say quite the same of applications built with your frameworks, since I'm making a comparison with the legacy code I had to deal with in the past.

Stimulated by the question, I decided to rank common technologies and practices I (and many developers) chose (and still choose) for php applications architecture. Note that these ranks describe the complexity and inherent power of the different approaches/technologies, but by no means low ranked solutions should be deprecated: they still get the job done when something more elaborated it's not necessary, and we are not in the mood of killing a fly with the Death Star.
Here is my evaluation:

• 1: Welcome, today is <?php echo date("Y-m-d"); ?>. 0 is the same but with <? instead of <?php.
  
• 2: Html page with embedded php code. Very useful in 1990s and still work sometimes because of its simplicity for temporary and corner-case pages.
  
• 3: Set of semi-static php scripts with no code reuse.
  
• 4: header.php and footer.php applications; this is the structure of the website my application partners with.
• 5: header/footer inclusion but with business logic reuse, for instance application that comprehend modules, classes and functions.
  
• 6: Procedural open-source frameworks and Cms, for instance Drupal 6. They are becoming not pretty to the eye, but they do the job.
• 7: Object-oriented applications, that rely for example on in-house frameworks.
  
• 8: Zend Framework 1.x applications: object-oriented, more or less testable, little duplication when done right. But the inherent singletons prevent them to rank higher. See you in 2.x...
  
• 9: Doctrine 2: Data Mapper for persistent-agnostic domain models.
  
• 10: No such technology has been produced in php at the moment, primarily because of the slowly real object-oriented paradigm adoption.

Or do you think there is already a 10 to assign?

Saturday question: mixing Repository and Active Record

Saturday is becoming the 'questions day of week', since it is not the first time that after a week of work some readers email me to carry on the discussion on design and testability, two topics that are stressed in my blog posts. :)
This week, Fedyashev wrote to me about mixing architectural patterns in a single application:

I really like these Active record and Repository patterns.
The drawback of Repository pattern is its cost(takes more time then
Active record). Benefit is higher abstraction which really helps on
complicated business logic.
The drawback of Active record is that lower testability(db interaction
is required) and harder in handling complicated domain logic.
Is it acceptable to take the best of these two patterns to be used in
the same application?
I was thinking about using Active record for simple CRUDs and Repository
for complicated domain objects.
The idea behind this intention is to keep cost of code lower but still
have a good code.
What would you recommend?

There are cases in which Active Record would be an acceptable pattern. Since the drawback of Active Record is little testability, the primary scenario for its application is when there is nothing to test. Some applications are data intensive and require only to move information back and forth from the database.
CRUD screens, as you suggest, often have little logic and can take advantage of active records. But we should evaluate case by case, since it is very easy for logic to leak into Active Record instances, and logic should be thoroughly tested.
For example, logic is present in managing validation of entities upon insertion and editing: a classical situation is searching for already existent nicks upon user registration. A Repository is capable of performing validation using external resources as they can be injected at construction or passed as a method parameter, while an Active Record probably not (and it will be more complex to test this validation).

Another problem I see in mixing up these patterns is the different libraries requirements. Typically, we want repositories to aggregate an instance of a lower-layer framework that encapsulates Sql queries or whatever storage we are using (Hibernate or Doctrine 2), while Active Records are subclasses of other frameworks abstract base classes (Zend_Db or Doctrine 1).
The paradoxical result is that implementing both patterns leads to use two different version of Doctrine at the same time, which I do not recommend for maintenance reasons and code clarity.
A solution would be keep the implementations in two separate BoundedContext, which are different domain models that can communicate, for instance using the same underlying relational database. Though, BoundedContext is a DDD term and suppose that you work with persistent-ignorant models in both contexts.

However, the real choice is not between Active Record and Repository but between Active Record and Data Mapper (persistence-ignorant domain model). It seems for instance that Doctrine 2 provides a default repository class you can tweak later, although it has default methods only for retrieving entities and not to insert them (I think the insertion can be managed with events). It's not really difficult to change your approach from:

$user = new User();
$user->nick = 'John Doe';
$user->save();

to:

$user = new User();
$user->nick = 'John Doe';
$em->save($user);

when what you gain is freedom from activating a mysql daemon to test the User class, without using Repositories. Repositories may come into play later, when and where you want a finer control on the bridge with the database.

Doctrine 2 and Zend Framework first date

This morning I have tried for the first time to use Doctrine 2 in a Zend Framework application. I used the latest release, 2.0.0 alpha3, for this experiment.
The chosen application is my recently born project NakedPhp, a port of the Naked Objects Java framework which generates the user interface and let the end user manipulate domain objects directly.
During this first run, I have not set up an application resource yet and I have just hardcoded a few configurations to bootstrap correctly Doctrine. I will publish a resource class (conforming to the Zend_Application_Resource_Resource interface) soon when I have it ready.

Doctrine\ORM\EntityManager is the Facade class which act as a portal towards the functionality of Doctrine 2, and it is the homologue of Hibernate EntityManager. Our code should interact mainly with this class.
Though, I have isolated the EntityManager behind an interface since I do not want infrastructure code to slip in NakedPhp for now. The code will obviously depend on Doctrine but it is good practice to have an interface I can mock out easily, as I don't need all the methods of the EntityManager and this way I just hide everything is not mandatory instead of introducing coupling to it.

Doctrine 2 is released in three packages: Common, Database Abstraction Layer and ORM. Instead of downloading three different packages I just grab them from the subversion repository:

svn export http://svn.doctrine-project.org/tags/2.0.0-ALPHA3/lib/

and move the Doctrine/ and vendor/ folders in my library/ directory along with Zend/. The vendor folder contains a small annotation parser.
It can be useful also to export other resources:

svn export http://svn.doctrine-project.org/tags/2.0.0-ALPHA3/bin/
svn export http://svn.doctrine-project.org/tags/2.0.0-ALPHA3/sandbox/

The bin/ folder contains the doctrine.php and doctrine command-line scripts (same thing), while the sandbox provides a working example of Doctrine 2.

Doctrine 2 prescribes that model classes (entities) and proxies should be autoloaded, so after moving doctrine.php in application/ I deleted the reference to Doctrine autoloader and added:

require_once __DIR__ . '/../application/bootstrap.php';

which is my bootstrap file, where:

• the library/ folder is added to the include_path
• the autoloader is set up to load Zend/ classes
• a Zend_Loader_Autoloader_Resource sets up autoloading for my model classes.
• my autoloader is set up to take care of \Doctrine and \NakedPhp namespaces.

In the future, I will add proxies autoloading setup to this file. Since you probably don't have your own autoloader for namespaced classes, you can simply use IsolatedClassLoader from Doctrine\Common.

It's time to code a cli-config.php file to use with doctrine.php; this file should define two variables (it is well-documented in the sandbox example). My final result is:

$classLoader = new \Doctrine\Common\IsolatedClassLoader('Proxies');
$classLoader->setBasePath(__DIR__ . '/../application/');
$classLoader->register();

$config = new \Doctrine\ORM\Configuration();
$config->setMetadataCacheImpl(new \Doctrine\Common\Cache\ArrayCache);
$config->setProxyDir(__DIR__ . '/Proxies');
$config->setProxyNamespace('Proxies');

$connectionOptions = array(
    'driver' => 'pdo_sqlite',
    'path' => '/var/www/nakedphp/sqlite/database.sqlite'
);

// These are required named variables (names can't change!)
$em = \Doctrine\ORM\EntityManager::create($connectionOptions, $config);

$globalArguments = array(
    'class-dir' => __DIR__ . '/../application/models'
);

Which is practically the cli-config.php file grabbed from the sandbox, but slightly edited:

• there were two instances of Doctrine\Common\IsolatedClassLoader, one for the entities and one for the proxies. I deleted the first one since entities autoloading is already taken care for in the bootstrap.
• I haven't used proxies for now, but the configuration is mandatory. The default namespace and folder are enough.
• I changed the path to the sqlite database. Sqlite was the fastest choice to get the application up and running, but remember that both the sqlite database file and its directory must be writable by apache and php.
• I changed also the argument class-dir to specify my entities folder.

Before starting to use the Doctrine 2 command-line interface, you will have to define annotations on your model classes. For instance, @Column and @OneToOne annotations. Since I have developed without even caring about the database until now, I had to add also an $_id private property. :)
I also submitted a patch to improve the errors generated by the schema tool in case of incorrect field names referenced on relations, which is what happened to me today.

Now, it's time to generate your schema:

php bin/doctrine schema-tool --re-create --config=bin/cli-config.php

If cli-config.php is in the directory where you issue this command, you can leave out the --config option.
Maybe after playing a bit with Doctrine 2, you will want to see what was inserted in the database:

php bin/doctrine run-sql --sql="SELECT * FROM Example_Model_Place" --config=bin/cli-config.php 

To obtain an EntityManager reference in a controller, you can set up a dumb resource that includes cli-config.php and return $em. I used a factory which was already available and add a method for retrieving the instance.

So I now have an hacked working instance of Doctrine 2 in my project. The next step will be writing an application resource to allow configuration to be specified following the standard, in application.ini. I will publish this resource in the next days.

The image at the top is the NakedPhp example application screen that says saving was successful. I implemented the storage of my persistence-agnostic in-memory object graph in less than an hour.

What's going on with php object-relational mappers

Every once in a while, in a post, I say:

The Domain Model should not depend on anything else; it is the core of an application. Classes should not extend or implement anything extraneous. I do not want User extends Doctrine_Record. I want User.

Sorry to stress you, but this is one of the points of DDD and the one that gives advantages even applied in other architectures. The persistence problem can be solved by generic object-relational mappers which will act as the bridge between entities and the database used for persistence. But where do we find a generic Orm?

In php, no real generic Orms existed until 2009, since Zend_Db, Doctrine 1, Propel etc. are all implementations of the Active Record (or similar data gateway) pattern, requiring for example all your User and Post classes to subclass a base record. The only way to obtain a persistence-agnostic model was manual implementation of all the mapper classes, which translate between database rows and object graphs. When you are managing more than a few different entity classes the problem become quickly intractable.
The Data Mapper pattern describes exactly a generic Orm, but it is an even more general concept in the sense that the mapping is not limited to a relational database like MySql or Sql Server. You can write a Data Mapper to store your objects in plain text files or document-oriented databases if you want.

In the last summer, I encountered two in-development solutions to solve the persistence agnosticism problem: Doctrine 2 and Zend_Entity. They are implementations of the Data Mapper pattern, with reference to the Jpa specification and a similar Api. I learned that the Java guys had implemented a real Data Mapper years ago: Hibernate. Jpa is only a specification extracted as a subset of Hibernate, and it is an additional layer abstraction that decouples your mapping code (annotations or xml files) from the particular Orm.
Anyway, I contributed to the lazy loading capabilities of Doctrine 2 with php code and to Zend_Entity with some small patches before its discontinuance. I am currently waiting for a stable version of Doctrine 2 to integrate it in NakedPhp, only because I am not worrying about persistence for now. It is the power of the Data Mapper approach that decouples my work from a specific storage such as a relational database.
Fast forward to today, and Doctrine 2 is in alpha for being thoroughly tested. Zend_Entity has been dropped instead, in favor of Doctrine 2 integration in the Zend Framework. It is not useful to maintain two different code bases, with the same Api transposed from Jpa, which do the same persistence-related dirty work and developed by the same people. It's just a waste of the contributor's time.

Thus, Doctrine 2 is going to become the first production-ready Orm for php and to be favored with seamless integration in both Zend Framework and Symfony. If you have not yet tried it, you may want to give it a shot.
If you feel like helping with the integration, which involves Zend_Tool components for generation and Zend_Application resources, join the zf-doctrine mailing list. The integration also comprehends Doctrine 1 since Doctrine 2 requires php 5.3 and its adoption by hosting companies will be gradual.
The adoption of the 2.x branch, when ready, would give your design the freedom from the database you want. Doctrine 2 is for php the greatest thing since sliced bread.

Doctrine 2 now has lazy loading

Lazy loading is the capability of performing a expensive operation on demand, only when it reveals necessary from a client request: in the Orm field, the expensive operation is the loading of an object graph part. In Doctrine 2 I made some architectural choices implementing the proxy approach, which substitute a subclassing object to every association which is not eager-loaded.

Disclaimer: there is not a stable Api for Doctrine 2. This is a design post about how I coded the lazy-loading features with the help of the lead developer Roman Borschel. The name of methods or classes can slightly change in the future.

*-to-one associations: dinamic proxies
The technical solution for a one-to-one or many-to-one association is to use a dynamic proxy, an object whose class is generated on the fly, subclassing the original one. I talked about this approach extensively in the previous post about lazy loading. However, I feel to make some precisations developed while applying in practice the theoretical approach discusses there:

• the example was about subclassing a Group class to Group_SomeOrmToolNameProxy, and inject a proxy object as the User property ($user->group). This class is generated only the first time the lazy loading capabilities is used with Group as a target entity, and it is saved in a temporary folder to be recycled in subsequent requests. This folder should be cleaned out when rolling out new code.
• if the dinamic proxy class is already present, there is no need to generate another one and an attempt to redefine it would raise a php fatal error.
• the proxy object should contain foreign keys of the source object ($user in the example), put there when fetched. The original entity class does not contain fields to store foreign keys as it is persistence-agnostic, so the more cohesive class to place them is the proxy one.
• I do not enter in the detail on how the proxy loads itself, but a reference of an Doctrine\ORM\Mapping\AssociationMapping subclass is passed to it in the constructor. This allows independent unit testing of the proxy behavior and of the effective hydration of data in a object (load() method on AssociationMapping).
  

*-to-many associations and the need for a Collection interface
While generating a proxy for a one-to-one or many-to-one association is mandatory to fulfill the same contract of a complete graph, is somewhat simple to satisfy the loading of collections of objects. In Doctrine 2, entities are required to implement collections in their fields with an instance of Doctrine\Common\Collections\Collection interface, and this is commonly done with instancing in the constructor Collections\ArrayCollection; when reconstituting an object from the database, a Doctrine\ORM\PersistentCollection instance is substituted in hydration and it has the mandatory field reference to the AssociationMapping object and the EntityManager to load itself when required to do so.
The Collection interface is not orm-dependent and it has been placed in the Common namespace to let the user build a real persistence-ignorant Domain Model.
A quick solution to implement lazy loading would be to fill the PersistentCollection instance with dynamic proxies. However, this proves to be slow as every object in the collection will issue a different query to the database for retrieving its internal data, and requires to join association tables even if the collection is not used.
Instead, the current implementation injects in the PersistentCollection (which obviously implements Collection) the needed collaborators, as no constructor is specified in the interface:

• the collaborators are always EntityManager and the AssociationMapping instance.
• there's no need to store foreign keys in the PersistentCollection since *-to-many relations do not use foreign keys from the source object. In our example, the Groups a User belongs to are joined with the primary key of User, and other collections will do the same.
• also here the unit testing of the PersistenceCollection trigger capabilities is separated from the loading itself, performed by an AssociationMapping instance. There are also functional tests to run the overall process of lazy loading in its entirety.

Final notes
Remember that lazy loading is a handy feature, but can be easily abused. Performance can suffer when the load is performed as many queries are issued as needed instead of few, eager queries which hydrates the part of graph you need to work on: this is the point of join() in the Doctrine\ORM\Query class. Enabling lazy loading probably will make your php script more chatty but save time when not all the objects are needed.
Doing an eager load can be impossible if the relations are bidirectional, like in the User-Group example: try to simplify your model removing one side of associations when not strictly needed.
I hope you will enjoy using Doctrine 2 and its lazy-loading feature and I think I've done good job of implementing it and explain the architectural issues I've encountered. Feel free to ask any question I have missed out.

How to stop getting megabytes of text when dumping an object

Php has a useful var_dump() function that helps debugging and unit tests writing by providing a text dump of all the properties of an object, being them public, private or protected. However, with an object that has a reference to the whole object graph, like a factory or an object that uses an abstract factory, the dump will be recursive and so long that it runs forever (probably it lasts several megabytes). Here's how to avoid it.

In the browser
When you are using a browser, the Xdebug extension comes in your help by generating a html dump truncated at an arbitrary level of depth. The installation on a unix box is pretty simple:
@ pecl install xdebug
run as root or with sudo. Pecl is the dual repository system of Pear and contains C extensions instead of Php userland packages; it is provided along with the pear binary in most of the installation of php.
Then add the following lines to php.ini:

zend_extension="/usr/local/lib/php/extensions/no-debug-non-zts-20090626/xdebug.so"
xdebug.var_display_max_depth=2

Probably the first has been already placed from the installer. The second directive set the maximum depth of var_dump(), which is overloaded by the extension, to 2 levels.
There are other directives that influences the behavior of var_dump(), and more and more for activation of other xdebug features.
Make also sure that html_errors is on in php.ini, or ini_set() it.

On the command line
The command line environment is more complex to use, but it is the ideal for debugging, given its speed and automation capability. The var_dump() usage must be tweaked because the var_dump() overloading, as of Xdebug 2.0, works only with html_errors active, producing an html dump that is ugly to view as plain text.
Assuming you have done the same setup of the browser section above, here's a workaround method to use:

function dump($var)
{
ini_set('html_errors', 'On');
ob_start();
var_dump($var);
$dump = ob_get_contents();
ob_end_clean();
echo strip_tags(html_entity_decode($dump));
}

It will get the html output of var_dump() even in a Cli environment that has the html_errors directive not set, and clean it to display as plain text.
Here's what I get dumping a PersistentCollection of Doctrine 2 now:

[16:54:23][giorgio@Marty:~/svn/doctrine2/tests]$ phpunit Doctrine/Tests/ORM/Functional/DetachedEntityTest.php
PHPUnit 3.3.17 by Sebastian Bergmann.
object(Doctrine\ORM\PersistentCollection)[180]
private '_type' => null
private '_snapshot' =>
 array
   empty
private '_owner' => null
private '_association' => null
private '_keyField' => null
private '_em' => null
private '_backRefFieldName' => null
private '_typeClass' => null
private '_isDirty' => boolean true
protected '_initialized' => boolean true
protected '_elements' =>
 array
   0 =>
     object(Doctrine\Tests\Models\CMS\CmsPhonenumber)[181]
       ...
   1 =>
     object(Doctrine\Tests\Models\CMS\CmsPhonenumber)[182]

The '...' ellipsis are added by Xdebug instead of showing thousands of objects and properties, because the PersistentCollection is provided with a reference to EntityManager for lazy-loading itself.
With the old var_dump() instead:

[16:54:23][giorgio@Marty:~/svn/doctrine2/tests]$ phpunit Doctrine/Tests/ORM/Functional/DetachedEntityTest.php > test.txt

I redirect the output to a text file because it is impossible to navigate from a terminal, but you have to expect some minutes because it takes forever to generate a full dump of the EntityManager and all other objects involved. Without redirecting lines are sent on terminal for a unspecified long time, and are also duplicated because var_dump() is not so smart in detecting recursion and it let happen for a while before stopping to output the same objects over and over. It was very frustrating but now it is easy to view even the most complex and coupled objects.
I know that the more an object is coupled, the less it is well written; but even if it only depends on an interface (a parameter in the constructor), at runtime that interface could be implemented by a very heavy object. Decoupling limits this process, but object must communicate and so they have to keep references to other collaborators: the Publish/Subscribe paradigm is very loose coupled, but dumping a publisher will output it, the blackboard and all the subscribers on Earth.
Summing up, Xdebug can boost your php productivity, and has many other features that will make easy to see what happens under the hood of your php application. Profiling and tracing are easily provided with other xdebug.* directives. If you are a php developer, install it as soon as possible.

Naked objects in Php

The validity of Naked Objects pattern, if implemented successfully, can add value to the php scene and help to answer the "Is Php ready for the enterprise?" question. Currently there is no framework to support this approach in the php world.

Php in the enterprise
The enterprise world has several architectural choices to build complex applications: Model-View-Controller frameworks are currently in vogue, especially in the php world, where Zend Framework, Symfony, Code Igniter, CakePhp and many others all implement this paradigm. But there are also examples in different languages: Ruby On Rails, Django for Python, Spring and Stripes for Java.
There is also a niche where developers are tired of bloated controllers and views that contains logic; they're also tired of dumping their own objects and get some megabytes of data because they have dependencies on all the instantiated objects of the chosen framework: here comes in NakedObjects.
As I wrote last week, the Naked Objects pattern let the framework take care of 3 of the 4 layers of an enterprise application: infrastructure, controllers and views are provided as generic or generated objects. The developer has only to write a Domain Model layer that follows some conventions.
Two frameworks that are named Naked Objects exist: one is written in Java and it is open source, while the other runs on the .NET platform; they are provided by the same people behind nakedobjects.org. Domain-Driven Design is really possible when using this tools.
I couldn't find a similar possibility for the average php programmer: it's a pity because php has the potential to introduce more and more cloud computing in the enterprise applications, let you use only a browser as a client.
There is also JMatter available, for Java.

Why wait?
So I decided to write a new one, a port of the Naked Objects for Java framework. It will have only a web interface anyway, since it is the best fit for php (no local GUI as it does not make sense).
Since I don't want to reinvent the wheel, it will incorporate Doctrine 2 (which I am contributing to) in the persistence part of the infrastructure layer and Zend Framework with its MVC implementation in the upper ones. Since the Naked Objects pattern has to be followed, the Views and Controllers will be provided from this framework, which I named NakedPhp. Obviously I chose an Open Source license, the bullet proof LGPL version 2; supporting DDD is also a key feature which I want to include.
Here are some links that points to the main resources activated on SourceForge to support the project:
Main page
Naked Php wiki
In-browser view of the Subversion repository
I will blog often about the architectural decisions of NakedPhp and its improvements, and when a alpha downloadable package will be available. Of course the repository is open to anyone who wants to take a look at the source code.
If you're tired to write controllers and views, consider the choice of a Naked Objects approach. And if you are a php developer and you want to participate, contact me!

Lazy loading of objects from database

What is lazy loading? A practice in object/relational mapping which simulate the presence of the whole object graph in memory. There is some techniques for produce this illusion that I explain here.

An example
Suppose we have the classic User and Group objects in a php application. The same process is valid for any language that uses a relational database as a backend, like the Java/Hibernate example, but I will include php code snippets.
Tipically User and Group has a many-to-many relation: every user can subscribe to many groups and every groups has a bunch of users as its members. This means that if all objects are loaded from the database you can navigate like this:

$user = find(42); // find the user with id == 42
echo $user->groups[3]->users[2]->groups[4]->name;

Having a potentially infinite navigability in an object graph is not a great practice, but sometimes many-to-many relationships are needed and the simplest approach is to incur in additional overhead (why there is overhead will be explained later) and provide this simple Api.
The main problem is that we cannot load all the object graph, because it will not fit in the memory of the server and it will take much time to build, depending on the size of the database. Nor we can load an arbitrary subset of it, because the user has the freedom to navigate groups and users to the depth he need: if he arrives at the edge of the graph will see null values/null pointers where objects should be.

Solution: Lazy loading
The Proxy pattern comes in our help:

A proxy, in its most general form, is a class functioning as an interface to something else. The proxy could interface to anything: a network connection, a large object in memory, a file, or some other resource that is expensive or impossible to duplicate.

In the first navigation the proxy will be a subclass of Group. The Data Mapper, without further instructions, will provide this type of object graph:

var_dump($user); // User
var_dump($user->groups[3]); // Group_SomeOrmToolNameProxy
var_dump($user->groups[3] instanceof Group); // true

As said previously, an Orm that provides lazy loading will produce a proxy class to substitute the original one. The code for this class is generated on the fly and will look like this:

class Group_SomeOrmToolNameProxy
{
    public function __construct(DataMapper $mapper, $identifier)
    {
        // saves as field references the arguments
    }

    private function _load()
    {
        $this->loader->load($this, $id);
    }

    public function sendMessageToAllUsers($text)
    {
        $this->_load();
        parent::sendMessageToAllUsers($text);
    }
}

The new class proxies to the original methods but calls _load() before, giving the object a usable state. Before a call to _load(), or a call to one of the proxied methods, the domain object has only identifiers field in its internal data structure.
Since it is a subclass of Group, it provides the same interface to the user, that will not even notice that it is not its clean, infrastructure-free Group class.

What does it mean?
It means that the first level objects are fully loaded, while second level ones are placeholders that contains only the information to load themselves. Only if you access them they will go to the database to fetch all their fields:

$user = $em->find(42); // a query on the user table
echo $user->groups[3]->name; // another query is executed on the groups and user_groups tables

We can complicate this pattern as far as we want:

• We can specify with join() commands on the query objects or 'join' options for the methods of the Data Mapper how far it should go with the initial loading. This way if we know that we need to access the second level of the graph starting from user, only one query is executed. Still if we go to the third level ($user->groups[3]->users[2]->role) without specifying that at the reconstitution of $user, additional queries will be sent to the database and performance will suffer.
• We can activate or disactivate lazy loading, or logging it to view where it is executed and impacts the performance

Hibernate for Java uses this approach to feature lazy loading of object properties and relations. Doctrine 1.x use a simpler approach because it is based on Active Record and the code is put in the models base class Doctrine_Record.

Today I contributed to the ORM\Proxy namespace of Doctrine 2, the component that generates the proxy classes and objects basing on metadata about the original classes, and non-invasive lazy loading will soon became a reality.

Becoming a Doctrine 2 committer

In the last weeks I started to contribute with code to the Doctrine project. Doctrine is an Orm and database abstraction layer for php: it is the default orm for Symfony but it's not coupled to it and I integrated it successfully with Zend Framework. It is a good tool for abstracting the database layer and manage the persistence of the Model part of a MVC paradigm, which to me Zf does not very well with its Zend_Db component.

After opening tickets and upload patches for Doctrine 1.x for some months, I took a glimpse of the potentiality of the 2.x branch (that currently is not even in alpha). The code is in the trunk of the Doctrine repository and borrows concepts heavily from Hibernate:
http://trac.doctrine-project.org/wiki/Doctrine2.0
While Doctrine Query Language, based on Hql, was already present in the currently stable 1.x branch, the new Doctrine infrastructure is based on the Unit Of Work pattern instead of the old Active Record one.
The Active Record pattern is very famous and is used for example in Rails. It is the most simple solution of object relational mapping, which defines a class for every table of the database. A row is represented by an object of that class, that is subclassing Doctrine_Record/ActiveRecord depending on the framework and language you are using.
The Unit Of Work pattern does not substitute by itself the Active Record one, but it's part of the solution of refusing to have the domain objects depending on the infrastructure, in this case the mapping classes. This is very clean from a design and testing point of view as the most complex logic of an application resides in the domain classes (the User and Phonenumber ones for instance), and Doctrine 2 let the developer concentrate on writing and testing these models without (almost) taking into account where they will be persisted; how to map these classes is specified in annotations or via other metadata.
If you have written a bit of code you will probably find very interesting these solution, that is already used in Hibernate for the joy of Java programmers.

Said that, I really want to help the project as I dream of doing Domain Driven Design in php and persistence ignorance is a key point of DDD. So I asked how to start contribution to the project and the last week I've written eight test case class for phpunit plus the example model classes that they use. I was helped by Roman Borschel, the lead developer for the 2.x branch, that teached me how to use annotations properly and fixed my faults in the model classes. Keep in mind that contributing to an open source project you use or will use is a win-win situation because it helps the project going forward and obtain feedback, while it helps you to learn to use the tool well. And I will certainly use Doctrine 2 because I think it along with Zend Framework will kick Rails in the ass.

This monday, I have succesfully closed ticket #2276 and I look forward to learn more and go further. If you are a php developer, come to #doctrine-dev on the irc server freenode.net, or stay tuned to try the alpha releases of Doctrine 2 that will come in the future. Having a real domain model, free of dependencies versus concrete orms, is a dream of the php world that will be realized.

10 Doctrine gotchas

Here is a list of tips for Doctrine usage in php projects, ranging from hydration to magic method. They are valid for the 1.x versions.

I am currently working on generation of Zend_Form instances based on Doctrine objects, that use model metadata to create form elements of the right type. Doctrine is very powerful, but while developing Otk_Form_Doctrine, I had some gotchas that I'd like to share with other php developers that use this orm.

1. findAll() and findOne*() does not hydrate the object. If you obtain a collection or a record with one of these methods, a subsequent access to a relation would raise a new query to the database. Hydration should be your repositories concern, and you have to decide what relation hydrate by default.
2. you can generate table classes, with proper options set in Doctrine facade calls. "generateTableClasses" is the option needed, see the documentation for examples.
3. fixture keys are unique throughout the whole set of files imported. This means that if you have a bunch of yaml files where you write fixtures, you have to make sure that the keys of the various records are all different, so don't use 1, 2, ... as keys, but meaningful names instead.
4. unlink() is not permanent until saved. From version 1.1 of Doctrine, until you call save() on a record, unlinked relationship would not be reflected to the database. A more consistent behaviour than the previous 1.0 version one, where unlink() will automatically save changes when called. For a deep model introspection, it is better to have a single point of saving as we have now.
5. Doctrine validation is turned off by default, so every save() might not saving all the data, resulting in truncated text fields...
6. ...but if you turn it on, remember that a form value of "" on a integer field will result in a non valid record and save() will throw an exception().
7. Doctrine many-to-many relations are bidirectional. So even if you don't define a side of a m-n relationship, it will be present in the record and will be listed by methods like toArray() when hydrated.
8. The methods getRelations() and getColumns() of Doctrine_Table can give you a lot of introspection of the model from the script point of view. In my case, I was able to generate form elements and subforms according to the business object structure, for instance a dojo DateTextBox with a sexy calendar to represent a timestamp model column.
9. __toString() is very useful in a Doctrine_Record subclass, putting together the identity fields of the object to represent it for various purposes like dumping or putting it in a list.
10. __call() is also very handy in a decorator for a Doctrine_Record or Doctrine_Table object, used for proxying to the underlying instance methods.

Hope this tips help you to follow the Doctrine... :)