Hello world in a production environment

An Hello World is a simple program whose only job is to print "Hello, World" on some form of output. The goal of an Hello World is to explain to programmers the syntax of a new language, but also to check that the infrastructure for compiling, interpreting or running the code is set up correctly.

Hello world in testing

The same concept can be used in the context of writing automated tests when you set up the simplest possible unit test:
When running this test with JUnit, you are validating that the environment is able to:

• retrieve a dependency such as the JUnit JARs and their own transitive dependencies
• build your Java code by compiling the test and its imports
• executing the .class files with the correct classpath.

During a workshop that involves Test-Driven Development, I would typically require everyone with a programming environment to have this simple test set up on the day before; this practice avoids having to put together an environment under time pressure. Especially when trying out new languages or frameworks, getting to this starting point can easily waste a lot of otherwise productive time.

In production

This Hello World pattern may not be limited to a development environment, as apparently simple things can get you a lot of mileage when deployed in production. Here are some examples.
The first API I implement in every microservice accessible through HTTP is the /ping API. It returns a 200 OK response with a content type of text/plain, containing just the text pong. Thanks to this API I can set up the first acceptance test for the project, running it in multiple environments such as CI and staging, and getting it to production where I will be able to call this API with curl and check its correct deployment on the whole server fleet.
I once set up an HelloCommand instead, a simple binary able to write a single log line with a custom text to the centralized log server. By deploying this to all environments we were able to test what happens when the target log server is unreachable or slow, checking that these slowndowns are not propagated to log clients. We also could insert a local proxy, buffering logs before sending them through TCP, and manually check the whole path from generating a log to its final collection.
One of the next things I want to add to the infrastructure is a sample Hello World microservice itself, consisting of its own source code repository, testing and deployment pipeline and monitoring.

What do we get out of Hello World

In the Hello World microservice case, having a template to clone greatly lowers the marginal cost of a new microservice, because the new project can easily be duplicated from a minimal definition. Cloning another existing service is a dangerous operation (we all know the problems of copy-and-paste) as you have to distinguish between what is common infrastructure and what is service-specific code that should not be ported to its siblings. By observing the minimal working template, you will also be able to contain the duplication between services as boilerplate will be highly visible:

How is it possible that we need to have a 200-line build file for an Hello World service?!

What's highly visible can be refactored. Given however that your template is not a code generator but a sample service continuously deployed in production, there will be a tight refactoring feedback loop between making a change to deployment or monitoring infrastructure common to all services, and validating it into the production environment. What you definitely want to avoid is to try to reduce duplication between the builds of different services, only to find out that your extracted method break production deployments on the next day when you're on holiday; or optimize one particular project build but discovering that the improvements cannot be ported to other generic services.
Moreover, there is also knowledge sharing in play: explicitly tested and running Hello World can be picked up by the other team members very quickly, to create a new API, a feature, or even a service. With the concept of living documentation, we prefer executable specifications like unit tests and Gherkin scenarios over complex documents; in the same way, we should prefer living and tested software to be used as a template rather than technical documentation which could be outdated two weeks after it has been written.

References

The Ginger Cake pattern by Dan North is a version of Hello World that start from concrete, complex instances to be cloned. I am wary of leaving too much stuff lying around after having duplicated the cake, so I prefer to start from the simplest possible example. The benefits of this choice are higher if the number of instances to be created is large, so it takes some tuning to recognize recurring technical tasks.
Nat Pryce hypothesis on TDD on the system scale pushes for considering immediately monitoring and system management into the APIs it should provide. Hello World examples are one of the lightweight tools that can show you if underlying resources such as CPU, databases are available and if the applicaton as a whole is working correctly; at the same time, isolating you from the complexity of real features and the myriad of ways in which they can fail even on a robust application layer.
Walking skeletons are a tiny implementation of the system that performs a small end-to-end function; they should put together all key architectural components to validate they can be integrated and can run in the target environment. Here I'm arguing for samples tasks that can be helpful to develop similar instances, a much smaller scale including for example single HTTP APIs.

The class design checklist

Given the good reception of the TDD checklist, I've decided to put together a similar one with suggestions for the generic class and interface design. These entities are the basic artifacts of object-oriented programming, thus this checklist is used at a lower level than the TDD one.
The form, however, is the same - a list of questions which should be answered by a developer before committing a new code artifact. There are really no particular phases in which to apply these questions, though, like in the Red-Green-Refactor cycle, so I organized them by topic. Feel free to pose these questions to yourself or to a developer you're pairing with anytime you perceive a smell in the code.
Note that many static analysis tools would point out some of this issues, but I think it is better to tackle possible problems early on in the development cycle, at the very moment of the initial design. If you employ TDD, design is performed one step at the time thus leaving you free of aggressive refactoring on the newly introduced concepts: this process converges towards cohesive and loosely coupled classes, which are blueprints for a good object graph.

Naming

• Does the class (or interface) name describe what an instance of this class (or interface) does? Usually a name or an adjective plus a noun are good for a class, while an adjective is more appropriate for an interface. Some interfaces have names composed by one or more nouns.
• Does the name contain unnecessary implementation details? Interface and abstract classes should not contain any reference to a particular implementation, but you should analyze this issue in context. For instance, XmlParser is not correct if at least a possible parser implementations does not work with Xml, while for a family of Xml parsers that differ in performance is appropriate. In the same motif, class names should not contain private implementation details of the class which may change, only the class real special trait in respect to the other implementations (e.g. XmlParser, HtmlParser, YamlParser.)
• Is the fully qualified name of the class or interface correct? (also known as: is this artifact placed in the right package or namespace?) You can make a guess based on the number of dependencies that this new artifact introduces.
• Naming conventions and best practices are also valid for method names, parameters, local variables, inline comments.
• Is the name consistent with the rest of the object model? Is it part of the Ubiquitous Language? The more public is the named entity (in the ascending order private, protected, [package where applicable,]  public, published), the more important is to get a valid name immediately.
• Should the name be influenced by a standard Ubiquitous Language? This is usually true for design patterns implementation, where leveraging the role names communicates much about the code structure. Other examples of a standard Ubiquitous Language are framework and programming language conventions.

Structure

• How many levels of indentation are there in your code? Supposing that the first level is dedicated to methods, other two levels are acceptable, with one (thus two in total) being the norm. Extract Method will help breaking up the complexity in different, orthogonal methods.
  
• Have you inserted switch constructs, especially similar ones? This structure is usually a smell, which can be refactored with a State or Strategy pattern.
  
• If-elseif chains or even if-else constructs, when repeated, are equivalent to switch, with the latter being its two-fold substitute.
• Are there new operators mixed with business logic? This is a no-brainer.
• Are there any controversial constructs in the code, such as the static keyword, or goto?
• If the code artifact is a subclass, does it extend the right parent class? If it is an implementation, does it implement the right interface? Check the semantics and analyze the proposition An instance of [entity] is always an instance of [parent], too.

Length

• May a class want to implement only part of this interface? Segregate it in different pieces as much as possible.
• Is the class longer than the standard size for your project? The suggested length varies with the programming language and the particular application, but a long class may be the sign of an imminent Extract Class refactoring.
• Is the class size of the same order of magnitude of other similar implementations?
• How many characters are the most complicated lines long? You may want to introduce intermediate methods or data structures to keep such lines readable. A common rule of thumb is the 80 characters of the original text terminal.
• Method length is also a useful metric. The rule of thumb is a method should be fully visible in a single screen, but this doesn't mean that with a 30'' LCD monitor you should write longer methods. The original screen was 25 lines high, but you may want to extend it a bit for practical purposes and refactor later: extracting local method is one of the simplest refactoring and it has a very limited impact on the rest of the codebase.

I hope you already had many of this questions in your mind while coding. As always, feel free to add new insights in the comments, I would be glad to learn new practices from you.

The TDD checklist (Red-Green-Refactor in detail)

I have written up a checklist to use for unit-level Test-Driven Development, to make sure I do not skip steps while writing code, at a very low level of the development process. Ideally I will soon internalize this process to the point that I would recognize smells as soon as they show up the first time.
This checklist is also applicable to the outer cycle of Acceptance TDD, but the Green part becomes much longer and it comprehends writing other tests. Ignore this paragraph if this get you confused.

TDD is described by a basic red-green-refactor cycle, constantly repeatead to add new features or fix bugs. I do not want to descend too much in object-oriented design in this post as you may prefer different techniques than me, so I will insist on the best practices to apply as soon as possible in the development of tests and production code. The checklist is written in the form of questions we should ask ourselves while going through the different phases, and that are often overlooked for the perceived simplicity of this cycle.

Red
The development of every new feature should start with a failing test.

• Have you checked in the code in your remote or local repository? In case the code breaks, a revert is faster than a rewrite.
• Have you already written some production code? If so, comment it or (best) delete it to not be implicitly tied to an Api while writing the test.
• Have you chosen the right unit to expand? The modified class should be the one that remains more cohesive after the change, and often in new classes should be introduced instead of accomodating functionalites in existing ones.
• Does the test fail? If not, rewrite the test to expose the lack of functionality.
• Does a subset of the test already fail? Is so, you can remove the surplus part of the test, avoiding verbosity; it can come back in different test methods.
• Does the test prescribe overly specific assertions or expectations? If so, lessen the mock expectations by not checking method calls order or how many times a method is called; improve the assertions by substituting equality matches with matches over properties of the result object.
• Does the test name describe its intent? Make sure it is not tied to implementation details and works as low-level documentation.
• How much can you change in an hypothetical implementation without breaking the test (making it brittle)?
• Is the failure message expressive about what is broken? Make sure it describes where the failing functionality resides, highlighting the right location if it breaks in the future.
• Are magic numbers and strings expressed as constants? Is there repeated code? Test code refactoring is easy when done early and while a test fails, since in this paradigm it is more important to keep it failing then to keep it passing.

Green
Enough production code should be written to make the test pass.

• Does the production code make the test pass? (Plainly obvious)
• Does a subset of the production code make the test pass? If so, you can comment or (best) remove the unnecessary production code. Any more lines you write are untested lines you'll have to read and maintain in the future.
• Every other specific action will be taken in the Refactor phase.

Refactor
Improve the structure of the code to ease future changes and maintenance.

• Does repeated code exist in the current class?
• Is the name of the class under test appropriate?
• Do the public and protected method names describe their intent? Are they readable? Rename refactorings are between the most powerful ones.
• Does repeated code exist in different classes? Is there a missing domain concept? You can extract abstract classes or refactor towards composition. At this high-level the refactoring should be also applied to the unit tests, and there are many orthogonal techniques you can apply so I won't describe them all here.

Feel free to add insights and items on the list in the comments. I value very much feedback from other TDDers.

Automated refactoring without heavy IDEs

Unix programs are beautiful and universally compatible. You can chain them to accomplish incredible tasks. Let's do some automated refactoring such as renaming php classes and methods without IDEs, using only sed, find and grep, plus your version control system of choice. You do not have to edit many hundred files by hand, and these tools should be available in every Linux distribution and maybe also on Mac Os X.

Renaming classes
START
Commit your work and make sure that the output of svn status is empty.

svn update
svn commit -m "what I have done until now..."
svn status

Refactoring is not an exact science, so wrong commands can destroy a codebase. In case you forget a \ and all your <?php tags are replaced by a apt-get cow, you'll simply enter

svn revert -R .

to restore the original state of the working copy after the last commit.
I'm sure your version control system has similar commands.

STEP 1
Provided that you use autoloading, moving or renaming the file that contains the class definition is the first step of the operation.

mv Package1\OldClassName.php Package2\NewClassName.php

Usually directly hooked in the version control system:

svn move Package1\OldClassName.php Package2\NewClassName.php

or what your VCS provides you with.

STEP 2
The syntax of the sed command is analogue to vim's search&replace one:

sed -i -e 's/\<old_classname\>/New_ClassName/g' \
    `find folder1 folder2 -name *.php`

This command renames all OldClassName occurrences:

• in place, without creating new files (-i)
• using an expression defined on the command line and that follows (-e)
• where OldClassName is present as a single word (\< and \> modifiers), and not for example AbstractOldClassName
• substituting them with NewClassName
• in all the file and thorughout all the single lines (/g)

We should list after the command all files we want to edit, but we can simply use a find command which find all files that:

• is in folder1 or folder2, or in their subdirectories (you can insert as many folders as you want)
• have a name that matches *.php (essentially their extension has to be .php)

The backticks (`) simply indicate to substitute the enclosed command with its result. They are a powerful tool to use sub-commands (otherwise we would have been stuck with find -exec.)The \ let the shell know that the command continues on the new line.

When the php classes to rename do not take advantage of Php 5.3 new features, the renaming it's very simple, since OldClassName is always the fully qualified name of the class. From php 5.3 namespaces are available to structure classes in different packages without very long names, but the namespace separator unfortunately concides with the shell backslash. Thus to enter it you have to insert another backslash:

sed -i -e 's/\<old\\classname\>/New\\KlassName/g' `find folder1 folder2 -name *.php`
sed -i -e 's/\<classname\>/KlassName/g' `find folder1 folder2 -name *.php`

The first sed modifies the use statements or the direct references, which are the means 99% of classes are referred by. The second sed modifies the remaining references to the base class name embedded in php files and it may not be necessary if you're only moving a class around.
A problem that occurs is when you're moving a class from a namespace and their old sibling did not have use statement to import its name but now they have to, or the equivalent specular case when you're moving a class into a namespace and you want use statements to vanish. You'll have to resort to manual editing to fix the interested files.

Don't forget to repeat the two steps also with the test classes. The exact commands depend on your naming convention, but often the test cases mirror the production code hierarchy, so that OldClassNameTest should be replaced with NewClassNameTest. In xUnit frameworks, test cases are first order citizens (classes), so there's nothing different from the production code renaming process.

END
Check the results by looking for every occurrence of the old or new name in your working copy:

grep -r "OldClassName"

or grep -rl to show only the list of files where the pattern does occur.
svn diff and svn status shows you the current changeset (every modified line, added and deleted ones) and the list of modified files respectively.
After manual check, run your whole test suite. If some of your test fail, discover the errors (that's what tests are for) and repair the situation manually. If the working copy is compromised, revert to the original revision.

Renaming methods
Provided that you have not overlapping method names in unrelated classes, the START and the END phases are the same, and they are very good practices to adopt in refactoring.
Methods may have a limited impact on the codebase in respect to classes:

• to rename private methods, you can edit them directly in your editor of choice, opening the class file.
• For protected methods, you should check also subclasses, and when they are a handful it's often faster to direct edit the files instead of running sed.
• For public methods, the problem is analogue to renaming a class, but there are no file movements nor namespacing issues. Follow Step 1.

Of course both these refactorings break backward-compatibility, so make sure you're not modifying any published interface in a minor release of your application. They are also dangerous if uncontrolled, because regular expressions are an advanced tool that can quickly mangle all your code. Make sure you can return to the original copy of the code via version control at any time, and that there are many tests in place that cover the functionalities whose provider classes you want to refactor.
Refactoring shouldn't be hard. I hope this little guide helps you.

Learning how to refactor

Refactoring is the process of improving the design and the flow of existing, working code by applying common patterns, like extracting a superclass or a method, or even introducing new classes as well as deleting existing ones.
Probably if you are here you have already experienced the refactoring process, but I want to clarify the common iterative method I use, to get feedback and being helpful to developers who are naive in this practice.

This is the general process to learn how to refactor code, which wraps the basic refactoring cycle.
Step 1: get the book Refactoring: Improving the Design of Existing Code by Martin Fowler (a classic) or a refactoring catalogue on Wikipedia or Fowler's website. In the book or in a similar guide, there are two lists which are very boring if read sequentially: smells and refactorings. Smells are situations that arise in an architecture, while refactorings are the standard solutions to eliminate smells. It is annoying to keep something that smells in your office.
Since it is very boring to read the Refactoring book if you have even a small experience with the practice, the best way to extract all Fowler's knowledge is to apply it directly.
Step 2: For S in 'smells':

• Read about S; understand what is the problem behind a practice that you may have used without worries.
  
• Loof for S in one of your personal projects or where you have commit access and responsibility for the code base; get convinced that this smell should be eliminated. If you are not convinced, stop here this iteration and go to the next smell; your existing solution can be pragmatically correct in the context of your style or in your architecture. Note that there are no absolute reference points and refactorings often come in pairs: it is up to you to choose if refactor in a direction or in the opposite one (Extract Method or Inline Method?)
  
• Find an appropriate refactoring R; there are multiple solutions that can eliminate a smell. Be consistent in your choice in different places.
  
• Make sure there are unit tests for the code you're going to edit. No further action should be taken before you are sure functionality is preserved. This is the answer to the question "Why change something that works?"... Because it will still work, but much better.
  
• Apply R in small steps, running focused tests every time to ensure you have not break anything.
• Once the refactoring is complete, run the entire test suite to find out if anything is not working. Note that failures in points distant from refactored code constitute a smell too: they are a symptom of coupling.
  
• svn diff will calculate a picture of your modifications; ensure that debug statements or workarounds are not in place anymore.
• svn commit (or git equivalent commands) pushes your improvements to the repository. Using version control is also fundamental in case you get in a not recoverable state: svn revert -R . is the time machine button (no, Apple has nothing to do with it) to restore the original code.
  

The goal of learning various refactoring techniques is to easily see smells in the future, to improve the efficiency of the Refactor phase in the Red-Green-Refactor cycle. Your bricks (classes) are very malleable when fresh, but when they solidifiy it becomes harder to add further modifications: it is good to refactor as much as possible just as you have finished adding code for functional purposes.

Applying Domain-Driven Design and Patterns review

Applying Domain-Driven Design and Patterns: With Examples in C# and .NETis a book from Jimmy Nilsson which shows you how to begin applying such things as TDD, object relational mapping, and DDD to .NET projects...techniques that many developers think are the key to future software development. Although it is based on .NET technology and features C# examples, it is a valuable resource for DDD adopting in every object-oriented language and one of the few practical books on Domain-Driven Design, along with the original one from Eric Evans.

The main feature of this book is a full description of the process followed in development: previous solutions are presented along with improvements to the object-oriented design and the reasons that have lead to refactoring. For instance, persistence ignorance is not assumed as the basis of a decoupled model but it is the final point of a research path.
This is a panoramic of the topics which the author deals with in the 600+ pages book.

Part 1: Background
The first part of the book is a introduction on low-level techniques like TDD, refactoring, and application of patterns (for example, State pattern). Problem and tradeoffs are discusses regarding databases and their impedance mismatch with an object-oriented model, together with the importance of layering and modelling. Part 1 is important because understanding of the rest of the book strongly depends on these concepts: a responsible programmer should test first and refactor where he expects changes.

Part 2: Applying DDD
In part 2 a full application requirements are discussed along with part of their implementation. Since the title is applying DDD the focus is on building the right model for the example, leaving out infrastructure and security concerns for a moment.
The requirements list comprehends orders and customers management, concurrency conflict detection, validation rules on amount of moneys in orders and customers, atomic saving and validation with an external service. This list is well crafted as it contains the main problems encountered when using a rich Domain Model, so the order management application is a good example to see DDD applied at its best.

Part 3: Applying PoEaa
While part 2 is centered on a persistent-ignorant model, part 3 discuss the infrastructure needed for the example application to work (object/relational mapping in primis) and tradeoffs from the incorporation of database concerns in the pure domain model. PoEaa is Patterns of Enterprise Application Architecture,a famous book from Martin Fowler which presents the principal patterns used in orms and big applications.
PoEaa is full of code examples so it is not important for the author to attach naked code but the discussion on how to persist a rich model is still crucial.

Part 4: What's next?
This part introduces other techniques and topics which are not strictly related to modelling and infrastructure: Service-oriented architecture, Inversion of Control and Dependency Injection, Aspect-oriented programming, Model-View-Controller pattern, user interface development. These are all interesting topics which a smattering of can only help the average developer to enlighten himself and face different ideas and paradigms: TDD for instance needs Dependency Injection to work.

Part 5: Appendices
Last but not least, other styles are proposed in contrast to DDD, such as database-driven development and rich service layers. The same application requirements from part 3 are rediscussed using these new point of views, and the reader gets a comparison between various implementation of the Domain Model pattern.
The appendices contain also a list of the patterns referenced in the book, but the should consider other sources for more information.

Summing up, Applying DDD is a good panoramic on practicing Domain-Driven Design, but also on implementing patterns and using modern tools and approaches like Dependency Injection and Test-Driven Development. It can also be read along or after the original DDD book to improve the understanding of Factories, Repositories and other domain patterns.

Keep growth under control

Systems grows and also classes do, but we should respect the Single Responsibility Principle. Rule of thumb for noticing potentially God classes.

In a good designed oop application, every class should do one thing, and doing it well, paraphrasing the unix philosophy.
The Single Responsibility Principle (SRP) is one of the five SOLID principles that governs object oriented programming.

A class should have one, and only one, reason to change.

Real world
But in reality, when you open .java and .php files or whatever else, you often find yourself scrolling up and down the class code to fix disparate things. What can be done to keep classes navigable and simple?
I have my quick dirty approach, that uses unix shell; it takes advantage of the one-to-one correspondency of class and source file, and also of the fact that:
Measuring programming progress by lines of code is like measuring aircraft building progress by weight.
-- Bill Gates
In my opinion, lines of code are a metric, but not the way many thinks: the more lines you commit, the more bloat you introduce. If you absolutely need hundreds of lines of code, you should at least divide them in a bunch of methods, functions, files and classes to keep them manageable.
This cli interface is available on ubuntu and every other linux distribution, since this commands are tipically built-in.

$ wc `find library/Otk/ tests/ -name '*.php'`| sort -n

Obviously you can replace library/Otk and tests/ with folders you like, and .php extension with .java or .c one.
What this command outputs is the full list of source files ordered by line count:
3 6 66 tests/TestHelperMysql.php
3 6 67 tests/TestHelperSqlite.php
8 21 202 tests/stubs.php
273 766 7975 library/Otk/Image.php
274 641 8949 library/Otk/Controller/Scaffolding.php
281 787 9713 library/Otk/Form/Generator.php
10298 24649 315267 total

The output of wc is defined as:
linesCount wordsCount charsCount fileName
so you will notice on the last lines of the output the files that contains the highest number lines. All lines are counted: blanks and comments are included. You might want to use cloc or other more specialized (but not standard) programs to count only lines of real code, but as I said before this is a quick'n'dirty approach.

Where I should start refactor?
Now observe the last lines of this output: if the bigger classes in your project are the ones which you are frequently working on, and you struggle to move up and down in their source code, it can be a good sign of where refactoring is needed. I started from having some classes for forms and repository that had 500+ lines and now I'm down to the half of them for my biggest class, which is Otk_Form_Generator. This also has improved my testcases that went from a full integration test of the form class to the unit testing of form generator and collaborator classes.
Typical tdd worflow is Red - Green - Refactor, but when Refactor is executed after some time, where you should start? Here's a brutal indicator.