When to write setters

I have set out, almost unconsciously, to use constructor injection by default in the last few years while writing object-oriented applications. With Dependency Injection as a given, constructor injection satisfy most of my requirements for building an object graph and dynamically configuring collaborators.

The spectrum

I see the statefulness of an object not as an absolute but over a spectrum.
At one end of the spectrum we have immutable objects: these objects acquire a configuration in their constructor and are effectively final (to employ a Java-specific term) for the rest of their lifecycles. Their fields are private and there's no way of modifying them outside of the constructor; their collaborators are only scalars or other immutable objects. In Java fields can even be final so that accidental reassignment is regarded as a compile-time error.
The physical state of an object may still change without its external behavior being affected, like in the case of caching. I still consider this kind of white-box-immutable objects as immutable.
Stateful objects instead may have a behavior that changes as a result of its own state (or of its collaborators). You call some Command methods on it, and the response of further calls to Query methods change. Hopefully the Commands encapsulate some domain logic to constrain the state transition to a valid and modelled one.
At the extreme end of the spectrum we find setters: methods which only mutate the value of one or more fields, possibly skipping any validation, domain modeling or consistency check. The setters considered here are public methods, because their limited-scope versions do not provide the same violations.
If you want to write procedural code, setters proliferate (still it's probably easier to just use public fields at that point). There are only a few valid use cases where I have found setters useful in object-oriented programming, and here is the (short) list.

Configuration which has default values

Classes may have a few configuration values that you are able to tune; especially when there is more than a few of these parameters, I find useful to separate hard dependencies in the constructor and setters that are able to override the default parameters when the object has already been constructed. If you forgot to call these methods, the object still has to work correctly.
Alternative solutions for this use case are of course constructors with default parameters, which I definitely prefer if there are not so many options to tune (1 or 2). You can also look into with Value Objects which produce a new instance upon reconfiguration, and model all of the configuration parameters as a single entity; or into a Builder if you want to invest in an additional class and its API.

Adding Observers

Observers (or listeners if you prefer) are collaborators which are notified of internal events happening inside an object that may interest them.
I treat the observers of an object as an append-only list data structure, with an empty list or array as the natural Null Object. The object is initialized without any observer, and a setter like addListener(...) has the more limited capability of adding an observer but not of removing or modifying an existing one.
The nature of the Observer pattern is investing on a common bus that many observers can be attached to, even if they come from different packages and libraries. Therefore I find it natural to support the dynamic wiring of other objects, even if they make the object more mutable can before. The needs of integration become more important than guaranteeing safety of construction in these scenarios.

Reconstitution

When objects are unserialized from a cold storage such as a stream of bytes or a JSON object, encapsulation is very likely going to be violated. Object-relational mappers have been doing this for ages by working directly on annotated private fields, with sometimes powerful results but also lots of dangers from storage and code being out of sync.
In the scenarios where you control the reconstitution of objects, such as rebuilding an object from a MongoDB document, it's often easier to provide an explicit API like a setState() method than to rely on the magic of a library which is going to bypass your public methods. To constrast the possible misuse, you can tag this method as @private (or package protected in Java), or make it very awkward to use outside of the persistence context by requiring a particular data structure to be passed in.

Conclusion

There are very few use cases for setters in real object-oriented programming; default to constructor injection and to immutable objects to avoid overcomplicating your design. Employ setters for non-mandatory, cross-cutting initializations so that your code does not have to bend over backwards to support these use cases while at the same time it can be robust to cowboy modification of internal state.

Book review: Java Puzzlers

Java Puzzlers is a nice book on the corner cases of the Java language, taken as an excuse to explain its inner workings. I have read it over the holidays and I found it a nice refresher over the perils of some of Java's features and types.

What you'll learn

A first example of feature under scrutiny is the casting of primitive types to each other (byte, char int), with the related overflows and unintended loss of precision. Getting to one of these conditions can result in an infinite loop, so it's definitely something you want to be able to debug in case it happens.
There are more esotheric corner cases in the book, such as the moment in which i == i returns false; or the problem of initializations of classes and objects not happening in the order you expect.

Method

I don't want to spoil the book, since it's based on trying to run the examples yourself and propose an explanation for the observed behavior. I'd say this scientific method is very effective in keeping the reader engaged, to the point that I finished it in a few days.
To run the puzzles, you work on self-contained .java files provided by the books website; you can actually compile and run them from the terminal as there are no external dependencies.
Incidentally, this isolation also means you're working on the language and in some cases on the standard library; the knowledge you acquire won't be lost after the next release of an MVC framework.
Again, work on the pc and not on a printed book only; in that case, you won't be able to experiment and you will be forced to read solutions instead of try and solving the puzzles yourself.

Argh, Java!

It's easy to fall into another pitfall: bashing the Java language and its libraries. The authors, however, make clear that the goal of the book is just to explain the corner cases of the platform so that programmers can be more productive. Java has been a tremendously successful platform all over the world, the strange behaviors shown here notwithstanding.
Thus instead of saying This language sucks you can actually think If this happens in my production code I will know what to do. The biggest lesson of the book is to keep the code simple, consistent and readable so that there are no ticking time bombs or hidden, open manholes in a dark and stormy night.

Some sample quotes

Clearly, the author of this program didn’t think about the order in which the initialization of the Cache class would take place. Unable to decide between eager and lazy initialization, the author tried to do both, resulting in a big mess. Use either eager initialization or lazy initialization, never both.

the lesson of Puzzle 6: If you can’t tell what a program does by looking at it, it probably doesn’t do what you want. Strive for clarity.

Whenever the libraries provide a method that does what you need, use it [EJ Item 30]. Generally speaking, the libraries provides high-quality solutions requiring a minimum of effort

Conclusions

Some of these pitfalls may be found through testing; some will only be found when the code is heavily exercised in a production environment; some may remain traps never triggered. Yet, having this knowledge will make you able to understand what's happening in those cases instead of just looking for an unpleasant workaround. This is also a recreational book for programmers, so if you're working with Java get to know it with an easy and fun read.

Book review: Java Concurrency In Practice

I just finished reading the monumental book Java Concurrency in Practice, the definitive guide to writing concurrent programs in Java from Brian Goetz at al. This books gives you lots of information in a single easy place to find, so I'll delve immediately into describing what can you learn from it.

A small distributed system

On modern processor architectures, multithreading and concurrency have in general become a small distributed system inside a motherboard, spanning the centimeters that separate the CPU cores and the RAM.
In fact, you can see many parallels between the two field: CPUs are different machines, and coordinating between them is relatively more costly than allowing independent executions. The L1, L2 and L3 caches near the CPU cores behave as replicas, showing tunable consistency models and forcing compilers to introduce synchronization where needed.
Moreover, partial failure is always round the corner as threads run independently. Forcing programmers to deal with possible failure is one of the few usages of checked exceptions that I find not only acceptable but also desirable. I tend not to like checked exceptions too much as they tend to be replicated in too many places in the code, creating coupling. Still, they make forgetting about a possible thread interruption harder and also push for isolating the concurrent code from the domain models it is using underneath, to avoid throws clause contaminations.

Relevant JVM topics

The book is ripe with Java Virtual Machine concurrency concepts, building a pattern language for talking about thread safety and performance (which are the goals we are pursuing with concurrent applications.) Java's model is based on multithreading and shared memory, where the virtual threads are mapped 1:1 over the OS threads:

• thread safety is based on confinement, atomicity, and visibility. These are not generic terms but are really concrete, explained with many code samples.
• Publication and synchronization makes threads communicate, and immutable objects help keeping the collaboration simple. Immutability is not just a conceptual suggestion, because the JVM actually behaves differently when final fields are in place.
• Every concept boils down to an explanation built over the underlying Java Memory Model, a specification that JVMs have to respect when implementing primitive operations.

Libraries

Basic concepts are necessary for understanding what's going on in your VM, but they are an insufficient level of abstraction for productive work. For this reason, the book explains the usage of several standard libraries:

• synchronized data structures and their higher performance. ConcurrentHashMap is a work of art as it implements lock striping to avoid coordination when accessing different buckets in the map.
• The Executor framework provides thread pools, futures, task cancellation and clean shutdown. Creating threads by hand is a beginner's solution.
• Middleware such as latches, semaphores, and barriers to coordinate threads and stop them from clashing with each other without having to manually write synchronized blocks all over the place.

Thus part of the book has an emphasis of using the best tools available in Java SE instead of reinventing the wheel with Object.wait() and Object.notifyAll(), which are still explained thoroughly in the advanced chapters. Reinventing the wheel can be an error-prone task that produces inferior results, and it should not be the only option just because it's the only approach you know.

Be aware that...

The book is updated to Java 6 (it's missing the Fork/Join framework for example), but fortunately this version contains much of what you need on the theory and basic libraries. You will still need to integrate this knowledge with Java 8 parallel streams.
It takes focus to get through this book, and I spent several dozen hours to read the 16 chapters.
The annotations (such as @GuardedBy) won't compile if you don't download a separate package; it's too bad they're not a standard, since the authors are luminaries of the Java concurrency field, experts from many JSR groups and Java programming language authors.
As always for very technical books, I suggest to read it on a pc, with your preferred IDE and JUnit open to write tests and experiment with what you are learning. You probably will need some review on the most difficult topics, just to hear them as explained from different people. Stack Overflow and many blog articles will be your friend as you look for examples of unsafe publication or of the Java Memory Model.

Conclusions

I'm a fan of getting to the bottom of how things do work (and don't). I would definitely recommend this book if you are executing your code in multiple threads, as sooner or later you will be bitten without even understanding what went wrong. Even if you're just writing a Servlet, that code could become a target for concurrency.
Moreover, as for distributed systems, in concurrency simple testing is not enough: problems can be hard to find and combinatorially difficult to reproduce. You need theory, code review, static analysis: this book is one of the tools that can help you avoiding pesky bugs and much wasted time.

Book review: Thinking in Java

I recently read the 1000-page tome Thinking in Java, written by Bruce Eckel, with the goal of getting my feet wet in the parts of the language that were still obscure to me. Here is my review of the book, containing its strong and weak points.

Basic topics

This is a book that touches on every basic aspect of the Java language, keeping an introductory level throughout but delving into deep usage of the Java standard libraries when treating a specific subject.
The basic concepts are well covered by the first 200 pages:

• primitive values, classes and objects, control structures, and operators.
• Access control: private, package, protected, public for classes, methods and fields.
• Constructors and garbage collection.
• Polymorphism and interfaces that enable it.

Most of the basic topics are oriented to programmers coming from a C procedural background, so they don't dwell on syntax but instead focus on the semantics and the JVM memory model.
Even if you come from a modern dynamic language, you will still find this first part useful to intimately understand how the language works. You will learn common idioms and patterns such as delegating to parent methods, getting a feel for Java code instead of trying to write Ruby code in a Java environment.

Wide coverage

The larger part of the book instead will be useful to cover new ground, if your knowledge is lacking in some areas or if you want a complete understanding of it. For example, I have a good knowledge of data structures such as ArrayList, HashMap and HashSet; still the 90 pages on the Java collections framework introduced me to structures such as the PriorityQueue whose usage is infrequent but can be very useful when you encounter a problem that calls for them.
Here is a full list of the specific topics treated in the book:

• Inner static and non-static classes.
• Exceptions management with try/catch/finally blocks.
• Generics and all their advanced cases including example such as class SelfBounded<T extends SelfBounded<T>>.  I thought I knew generics until I read this chapter.
• The peculiarities of arrays, still present in some parts of the language such as method calls with a variable number of arguments.
• The Java collections framework, much more than List, Set and Map; lots of different implementations and a conceptual map of all the interfaces and classes.
• Input/Output at the byte and text level, plus part of the java.nio evolution.
• Enumerated types.
• Reflection and annotations (definition and usage).

By the way, a few of the chapters can safely be skipped to make the book shorter. Drop the graphical user interfaces chapter as totally outdated, I don't even write user interfaces different without HTML anymore nowadays. probably the most popular GUI framework right now is the Android Platform rather than what's described here.
I also suggest to skip the concurrency and threading chapter, since such a small treatment cannot do justice to this topic. I would prefer another dedicated introduction and then go on with a more advanced book like Java Concurrency in Practice, which will tell you also what not to do instead of showing all the language features.
On this point, I find the writing of Bruce Eckel conservative, showing caution with advanced and obscure features rather than showing off with the risk of writing unmaintainable code down the line. The point is making you able to read complex Java code, not enabling you to write a mess more quickly.

Style

The book is quite lengthy, but lets you select a subset of the chapters pretty well if you need to dig into a particular topic. The text is driven by code samples, and to experimenting instead of reciting theory.
I suggest to read a digital version with your IDE ready: at least in my case, I found it easier to pick up concepts and get involved by writing my own examples. A 1000-page book would be pretty daunting if read on a device with no interaction, as it's the polar opposite of dense.
I created many test cases like this one, which lead me to verify the assumed behavior of Java libraries and features with my own hands:

Currency

The drawback of this book is its not being up-to-date with the current version of the platform, Java 8. The most recent version is the 4th edition, available on Amazon since 2006, which treats every feature up to Java 5.
You will have to piece together knowledge of Java 8 and the intermediate versions from other sources. I would have expected this book to at least be up-to-date with Java 7 due to its popularity.
However, due to Java's backward compatibility, what you read is still correct: I only found one code sample to have a compilation problem. I wonder how long would this book be if it was edited again to include Java 8: it could probably get to 1500 pages or more and implode under its own weight.

Conclusions

If you work with Java, Thinking in Java is a must-read, either to get a quick introduction to the basic features or to delve into one of the specific areas when you need it. You will probably never be surprised by reading Java syntax or idioms again. However, don't expect a complete coverage of such a huge world: this should be your first Java book, not the last.

Building an application with modern Java technologies

Sometimes Java gets a bad rap from Agile software developers, who suspect it to be a legacy technology on par of Cobol. I understand that being the most used language on the planet means there must be projects of any kind out there, including lots of legacy. That said, Java and the JVM are a huge and alive ecosystem producing value in all kind of domains from banking to Android applications and scientific software.

So I built a sample Game Of Life with what I believe are modern Java tecnologies:

• Java 8 with lambdas support
• Gradle for build automation and dependency resolution (substitutes both Ant and Maven)
• Jetty as an embedded server to respond to HTTP requests
• Jersey for building the RESTful web service calculating new generations of a plane, using JAX-RS
• Freemarker templating engine to build HTML
• Log4j 2 for logging, encapsulated behind the interface slf4j.

On the testing side of things:

• JUnit 4 for unit testing infrastructure
• Mockito for Test Doubles
• JUnit Quickcheck for property-based testing
• Selenium for browser-based testing

Here's the result:

The application is self-contained and can be compiled, tested and run without any IDE or previous machine setup except having a JDK 8 on your machine. See the project on Github for details.

Microservices are not Jars

The cult of the monolith

I've been building microservices for two years and my main complaint is that they're still not micro- enough. Here's a rebuke of Uncle Bob's recent post Microservices and Jars, which he apparently has written after forming an opinion based on an article in Martin Fowler's bliki:

One of my clients recently told me that they were investigating a micro-service-architecture. My first reaction was: "What's that?" So I did a little research and found Martin Fowler's and James Lewis' writeup on the topic.

 "I didn't even know what microservices were up until several days ago. Now I'm ready to pontificate about the topic."

So what is a micro-service? It's just a little stand-alone executable that communicates with other stand-alone executables through some kind of mailbox; like an http socket. Lots of people like to use REST as the message format between them.
Why is this desirable? Two words. Independent Deployability.

Let's ignore the REST as the message format terminology. Only two words? Independent deployability is nice, but I've seen cases where independence is total, and cases where an end-to-end test suite still needs to run including the production version of services A and B and the new version C' that we want to deploy to substitute C.
Other interesting properties of microservices such as scaling them independently come to mind. Or writing them in different languages. Or adapting to Conway's law by aligning teams with microservices for most of their work.

You can fire up your little MS and talk with it via REST. No other part of the system needs to be running. Nobody can change a source file in a different part of the system, and screw your little MS up. Your MS is immune to all the other code out there.
You can test your MS with simple REST commands; and you can mock out other MSs in the system with little dummy MSs that do just what your tests need them to do.
Moreover, you can control the deployment. You don't have to coordinate with some huge deployment effort, and merge deployment commands into nasty deployment scripts. You just fire up your little MS and make sure it keeps running.
You can use your own database. You can use your own webserver. You can use any language you like. You can use any framework you like.
Freedom! Freedom!

<sarcasm> tag needed.

But wait. Why is this better? Are the advantages I just listed absent from a normal Java, or Ruby, or .Net system?

Yes:

• existing databases tend to be attractors when new persistence requirements come up. So if I have MySQL up and running in my application and a job that would be a good fit for MongoDB comes up, I'm definitely not going to introduce MongoDB given the infrastructure setup time. I'll just go with the existing infrastructure and create some new tables, perpetuating the growth of the monolith.
• Web servers are often tied to languages. If I want to use Node.js it will listen on the port 80 by itself, while PHP is commonly used with Apache, and Java with Tomcat or Jetty. 
• JARs are a pretty JVM-specific packaging system. I'm definitely not going to put PHP code into JARs.
• Frameworks come from the language, and even inside the same language I can have multiple PHP applications where one has a custom user interface and one serves a Angular single-page application.

Also the ones not listed:

• It's easier to find out machines which contain bottlenecks and replace them, CPU and IO usage maps directly to applications.
• It's easier to get started working as a new developer because you need just a single microservice to run on your machine.
• It's easier to throw away one microservice and replace it with a new one doing the same job, but better written.

What about: Independent Deployability?
We have these things called jar files. Or Gems. Or DLLs. Or Shared Libraries. The reason we have these things is so we can have independent deployability.

Replacing single JARs or DLLs seems pretty dangerous to me where there are compile-time and binary dependencies in play. Since Uncle Bob has experience with that, I'm going to trust him to deploy safely this way.

Most people have forgotten this. Most people think that jar files are just convenient little folders that they can jam their classes into any way they see fit. They forget that a jar, or a DLL, or a Gem, or a shared library, is loaded and linked at runtime. Indeed, DLL stands for Dynamically Linked Library.
So if you design your jars well, you can make them just as independently deployable as a MS. Your team can be responsible for your jar file. Your team can deploy your DLL without massive coordination with other teams. Your team can test your GEM by writing unit tests and mocking out all the other Gems that it communicates with. You can write a jar in Java or Scala, or Clojure, or JRuby, or any other JVM compatible language. You can even use your own database and wesbserver if you like.

You can use any language you like as long as you run it on the JVM. Sure there must be people who work on other infrastructure or don't want to run their languages on a compatibly-yet-really-secondary platform? PHP applications? Ruby programmers?

If you'd like proof that jars can be independently deployable, just look at the plugins you use for your editor or IDE. They are deployed entirely independently of their host! And often these plugins are nothing more than simple jar files.
So what have you gained by taking your jar file and putting it behind a socket and communicating with REST?

SOAP is the last acronym where simple was used this way. Look, by generating a WSDL from your objects along with an XSD file that can be used to validate XML messages you can pass requests over HTTP with a Soap-Action header and regenerate Java (or other compatible languages) code from the WSDL...

One thing you lose is time. It takes time to communicate through a socket. It takes time to decode REST messages. And that time means you cannot use micro-services with the impunity of a jar. If I want two jars to get into a rapid chat with each other, I can. But I don't dare do that with a MS because the communication time will kill me.

Of course, chatty fine-grained interfaces are not a microservices trait. I prefer accept a Command, emit Events as an integration style. After all, microservices can become dangerous if integrated with purely synchronous calls so the kind of interfaces they expose to each other is necessarily different from the one of objects that work in the same process. This is a property of every distributed system, as we know from 1996.

On my laptop it takes 50ms to set up a socket connection, and then about 3us per byte transmitted through that connection. And that's all in a single process on a single machine. Imagine the cost when the connection is over the wire!

It takes more to write a file line by line rather than doing it in a single shot. However, if the file is 2GB long, I prefer the first solution in order to preserve memory. I'm just trading off time for another resource.
In the case of microservices, I'm trading off the latency of single interactions between different services for more important resources: programmer time, independent scalability, even time experienced by the end user. A front end asynchronously publishing events to a backend service feels faster to the user than a monolithic application where I respond to user requests and generate report lines in the same process or on the same machines.

Another thing you lose (and I hate to say this) is debuggability. You can't single step across a REST call, but you can single step across jar files. You can't follow a stack trace across a REST call. Exceptions get lost across a REST interface.

To me debuggability and introspection into an application improves when using microservices, because you will be full of all the HTTP logs of every service calling one another. You don't have to predispose logging cut points as they come for free with the HttpChannel objects. For a more business-oriented monitoring, take a look at Domain Events: we publish them from different applications in order to build reports based on data from different components.

After reading this you might think I'm totally against the whole notion of Micro-Services. But, of course, I'm not. I've built applications that way in the past, and I'll likely build them that way in the future. It's just that I don't want to see a big fad tearing through the industry leaving lots of broken systems in it's wake.

For most systems the independent deployability of jar files (or DLLS, or Gems, or Shared Libraries) is more than adequate. For most systems the cost of communicating over sockets using REST is quite restrictive; and will force uncomfortable trade-offs.

Paraphrasing Stroustrup, there are only two kinds of achitectures: the ones people complain about and the ones nobody uses. We are here proposing microservices because they have provided value in many systems that were once thought not to need them. As long as you have reporting needs you don't want to burden your front end with, or need to scale up in the number of users or programmer, you can consider microservices (and their cost).

My advice:

Don't leap into microservices just because it sounds cool. Segregate the system into jars using a plugin architecture first. If that's not sufficient, then consider introducing service boundaries at strategic points.

Please don't! The interaction between microservices are very different from the ones between objects inside a single application. Each call outside of the boundary is a potential failure mode that you should try to model as an asynchronous message that can be retried when delivery fails (the receiving microservice being down, slow or not reachable). Retrofitting microservices over an existing code base is a costly endeauvour and you should only embark on it if you have an adequate time and money budget, possibly bigger than the one necessary to build with microservices in the first place.

Java versus PHP

If you exclude C and its child C++, the most popular programming languages in the world are Java and PHP, which power most of the dynamic web. I have working experience with PHP and for academical purposes I am deepening my knowledge of Java, thus I'd like to point out similarities and key differences between these two languages. Every language has its pros and cons, so there's no absolute winner here.

History
Java was originally developed at Sun Microsystems in 1995, as part of the Java platform. Java applications are compiled to an intermediate bytecode which is run by a virtual machine. While originally intended for client software and in-browser applets (with the motto write once, run everywhere), Java is now a key infrastructure for many web applications.
PHP was born in the same year, and it was instead specifically created for the web, as a server-side scripting language to embed in html pages. It has evolved through 5 major versions to became one of the most popular web programming languages, thanks also to shared hosting services, which are particularly simple to set up for PHP applications and drive down their operational costs.

Typing
Java is built over static typing: variables must have a declared (possibly polymorphic) type. Thus, it is commonly judged as verbose, although the verbosity isn't necessarily linked to static typing.
PHP uses dynamic typing instead: variables assume the type of the value currently contained in them, and can change their type to satisfy implicit casts and conversions. This approach is prone to errors which would be detected by a compiler, but unit tests are a wonderful substitute for compile-time checks in dynamic languages.
Note that both languages have primitive types: Java because of the performance problems of wrapping integer and character variables in objects at the time of its creation in 1995, and PHP because it wasn't originally thought as an object-oriented language.

Object-oriented programming
PHP 5 borrowed its object-oriented paradigm from Java, which is the standard implementation of an object-oriented language today. After the release of PHP 5, a key point in the introduction of a serious object paradigm, PHP is evolving towards oop and has borrowed more from Java products: Doctrine 2 is an object-relational mapper inspired by Hibernate and JPA; phpDocumentor is built on the example of Javadoc; PHPUnit is one of the xUnit products, which derive from the original JUnit.

Execution model
PHP classes, functions and data structures, when they do not employ external infrastructure like caches or databases, are created in a script and they are garbage-collected at the end of the request. Java applications are instead kept in memory between requests, and the architecture of these two kinds of applications if fundamentally different - though, I would not say one model is superior to the other one. PHP pulls in execution only what it needs, and it pays back with the inability to run periodical tasks such as cron. Java applications can start multiple threads, but their management is much more complex, from the compiling phase to the deployment which includes servlets reloading.

Infrastructure and web
PHP is simple to deploy in its basic form (.php scripts), but this means that increasingly often the average developer has to use frameworks which builds standard infrastructure features over the simple PHP interpreter. Ironically, these framework are similar to Java ones; for example Zend Framework's controllers are the equivalent of servlets: classes with a standard constructor that extend a common base class and act on a request object to produce a response one.
Java has less native features built in the language, as it is not strictly oriented to the web, but it has them in frameworks which adhere to a standard, the servlet containers. PHP capabilities are hindered by the absence of standards.
For instance, the web.xml file in WAR packages, which represent a web application, define routes to map urls to servlets. Imagine if PHP had a standard for defining routes that Zend Framework, Symfony and their siblings respected: that seems science fiction.

How to TDD a database application

After the publishing of Failed attempt to apply Tdd response, some readers have asked me how to test a database application, as an example of code which is not suitable for xUnit frameworks.

Let's start with saying that database is not a special case for testing, as it's only a port of your application. Whenever the application interact with an external program I would say it presents a port, and this pattern is called Hexagonal Architecture.
Your effort should be in testing thoroughly the core of the application, building a solid object-oriented Domain Model piece after piece, by writing a test at the time an making it pass. The Domain Model should not have dependency on any infrastructure like database, http requests, twitter adapters, and so on: this refinement of the Domain Model pattern is called Domain-Driven Design and when applied produces easy testable code. The Domain Model would be tested by injecting fake adapters as there should be no logic in the database: this is object-oriented programming and in database object do not exist nor we can test it easily and automatically with Junit.
Though, this approach is very complicated and it is considered when the domain has a rich set of rules and behavior, while many applications have only CRUD capabilities.

Think of your application as existing only in Ram memory and strip out all the unnecessary code. The classes which remains are the core of the Domain Model. For instance if I had to manage the list of the users of a forum, I would write initially only a User class and a generic collection of User objects. User in my point of view is a POJO or POPO, which does not extend anything:
class User { ...

Insulating the database from view will hide this generic collection behind an abstraction, since it is not present at all in memory except for caching. Subsets of it can be reconstituted as needed. This abstraction is called a Repository or, at a low level, a DataMapper. Hibernate for Java and Doctrine 2 for php are example of a DataMapper pattern: they let you work on your objects and then take them and synchronize the database with the new data you have inserted: a change in a User's password, or new Users to be added. To polish the DataMapper Api, which is very generic, a UserRepository class can be created.
Even if you do not have a generic DataMapper, and work with mysql_query, PDO or JDBC queries, you can write a UserRepository which will act as the port for the database (or maybe a text file; since the repository decouples it, the storage mechanism can be anything from a memcache server to a group of monkeys which writes down on paper serializations of objects).
Depending on your architecture, your controllers or other domain classes will now have the UserRepository as a collaborator, and will talk to it and call its methods instead of accessing the database directly; this is a form of Dependency Injection. Obviously if there are other entities which are persisted, like Groups, Posts, etc. they should have their own repository class.

Of course the point of this discussion is how to test this code, since to write it we have to prepare a test before (it's called red green refactor and not code and then try to test). If we manage to write these tests first, the code will be automatically testable, and very decoupled and reusable as this is a characteristic of testable code.
What we need to write are not tests, but unit tests. If you want to check the entire path of data in the application, you can write integration tests which will exercise even the user interface, but Test-Driven Development prescribes to write unit tests: your test methods should have a dependency only on the system under test, and to the interface it uses.
Continuing our example, we need:

• unit tests for User, Group, Post classes (Domain Model entities);
• unit tests for controllers or other classes from the Domain Model which uses the adapters.
  
• unit tests for UserRepository and similar classes (adapters);
• unit tests for DataMapper or (infrastructure);
  

The point of unit testing is you can test singular components separately, and not the entire application as you would do with functional testing. Moreover, you must test each component separately or otherwise it would reveal too difficult to write classes to satisfy an integration test and the TDD benefits of adding a bit of design with every test method would be lost.
With this knowledge, we can say:

• unit tests for Domain Model entities must be written by the developer of this application. If the project is mostly a CRUD application and is data-intensive, these test cases will be very short.
  
• unit tests for controllers and everything that uses the adapters also must be written by the developer, mocking the adapters out. If you use the real adapters in testing, it will be integration testing and it will be heavy and brittle; you won't know if it's the controller or the adapter which does not work after a red test.
• unit tests for the adapters concrete classes: this is the only interaction with the database. Fortunately, we are unit testing for this classes so we can even use a new database for every test method as every class will present a few methods; compare this approach with testing every single feature of the application by using a real database.
• unit tests for infrastructure: these are included in the framework so we shouldn't worry.
  

Note that if you use a DataMapper which abstracts the particular database vendor as infrastructure, you can run the unit tests for the adapters by using a in-memory database like sqlite instead of the real database which will likely be very heavy to manage and recreate every time; this way, you de facto exclude database from your unit tests. Of course some integration tests will be necessary, but they are not part of TDD and of the design process.

I hope to have given you an idea of what unit tests means and how to deal with an application which uses a database for persistence, which is a very common scenario. Feel free to raise questions in the comments if something is not clear.

Do you want more? There's a book for .NET developers which explains DDD and database-independent testing.
The image at the top is a photograph of integrated circuits, which are real world components designed for testability and which are tested indipendently from the card where they will work.

Coding an interface

Interfaces are a contract that a class must respect: a set of methods with specific signatures that should be implemented. Many components become dependent on interfaces that abstract away the concrete classes: mistakes have to be maintained and inherited.

In object oriented languages, an interface I corresponds to a set of methods (without body) which acts as a specification for classes. A class C that implements I must provide a body for every method that I has. The objects of class C are not only instanceof C but also instanceof I.
Interfaces are present in languages such as Java and Php. Since there's no limit to the number of interfaces that a class can implement, multiple inheritance is obtained via interfaces.

Here is a checklist for writing a clean interface:

1. Keep methods number to a minimum. The complementary of an interface is an implementation, and every implementation is forced to provide code for the all the methods specified. If you are extracting an interface from a particular class, consider to only a cohesive set of public methods, which lead us to the next point. E.g. a Queue interface should have only two methods: put(...) and get(...). Specifiy a constructor or a length() method is not very purposeful.
   
2. Segregate interfaces. The interface segregation principle says that a class should not depend on interfaces that it does not use. The meaning is that when an interface contains many methods, probably there are two or more interfaces which it should be broken into. E.g. Php SplQueue and SplStack could be interfaces instead of concrete classes, with only the purposeful methods included and SplDoubleLinkedList implementing both. As they are now they expose all the Iterator methods, which have no sense on a real stack or queue.
   
3. No static methods. Interfaces provides a contract for polymorphism, and static methods are only global procedures under a nice name. Althoug some languages like php don't complain when encountering an interface definition which contains static methods, they should be avoided. E.g. resist the urge to put a static count() method on Queue to count the instantiated objects. Create a AbstractQueueFactory instead with a count() instance method.
   
4. Abstract the meaning. Particularly if you are extracting an interface, pay attention to the method names: they have to carry an abstracted meaning and not implementation details, so some of them needs to be renamed. E.g. the Spl Iterator interface has generic methods like current() and key(); it does not assume anything on the keys of the elements which the iteration is performed on: in subclasses they can be filenames, array offset, object property names and so on.
   
5. Do not tie to concrete types. Depending on an interface decouple a system from the implementation of a particular service, but if the interface depends on other concrete classes the advantage is lost; the only classes cited in an interface definition should be Value Objects (like java String or php Datetime) or Entities (the example User class, without injected services). E.g. an Authenticator interface with a method signature which contains a Zend_Db object is useless.
   

I hope you'll be writing small and cohesive interfaces now. Let me know what are your guidelines for coding interfaces in the comments.

Naked objects, DDD and the user interface

The SeparatedPresentation pattern is used to abstract the user interface from the domain model of an application. But it isn't more useful to enrich the user with a view of the domain itself?
The MVC pattern is all about separating the presentation of data. The major frameworks like Zend Framework and Rails are built to take advantage of an MVC stack, which means write your models, controllers and views.
Tipically we have a Post model with a PostController class that governs the operations that can be performed on the model and which views to show.
But there's more than MVC in the world.
What I'm talking about is Naked Objects pattern, that is currently implemented in the Java and .Net world by a framework with the same name. This pattern can be the next paradigm shift in development: in short, the V and C from MVC are completely generic and are built from the M part, the model classes. Also the infrastructure part, like database persistence of entities, are taken care by the framework that includes an Orm in its libraries.
Let's discuss the advantages of this approach:

• there's no mapping between concepts, as the user interface is automatically created on-the-fly or generated and the user gains insight on the model (and thus on the domain). A driver aware of the fact that there is an engine will use the accelerator at its best. This helps an Ubiquitous Language to take shape.
• there's no views and controllers to write, obviously.
• there's no infrastructure coupling, also obviously if we exclude the annotations or xml metadata needed to make Hibernate or another Orm work. ActiveRecord is a reminescence of the past.
• there's no logic in the controllers: they are automatically generate and delegates to the model. Fat model, skinny controller is achieved by force.
• the domain model will be built without caring much of the other parts because it is the only thing to write. This allows a DDD approach, and the only attention is to conform to the conventions of the framework.
• no more repeated validation on the ui: it is simply delegated to the model, where it should reside.
  
• if the generated user interface does not satisfy you, what remains to you is a powerful and complete domain model that can be used to write an MVC application (thus using naked objects pattern only to prototype).
  

There are also issues of giving a complete user interface that cannot be edited by hand: this is not scaffolding. In the next posts I will talk about my journey in learning to use Naked Objects framework for Java, which is distributed as open source software.