Wednesday, April 23, 2014

Integrated tests are not feeling well. Long live design.

Take me down to the big Rails city / where the tests are green and they take 10 minutes
I checked the date this afternoon:
$ date
Wed Apr 23 14:38:08 CEST 2014

and apparently it's 2014, but there's still a widely held belief (in some circles) that integrated (or end-to-end) tests should be favored over unit tests. A belief that Test-Driven Development does not have a beneficial influence on the quality of your tests and code.
So today I'm repeating a few things I have been writing about in the last years.
Don't be too proud of this technological terror you've constructed. The ability to INSERT a row is insignificant next to the power of Domain Models.

A few properties of unit tests

Unit tests target one or a few objects at a time, without accessing different resources than the CPU and memory of the current process. With respect to integrated tests, they are:
  • Easier to write: their setup phase takes a few lines where Test Doubles are injected in a constructor.
  • Faster: a 1000-test suite takes seconds to run.
  • Isolated: they cannot interfere with each other or with the global state of the process.
  • Repeatable: they always give the same result no matter the initial conditions.
  • Precise: they tell you which wire does not work instead that the car does not start.

Listen to your tests

How many asserts must a man write down / before you call him a man

That's not to say integrated tests are not useful: I have a talk coming up at phpDay in which I will explain how our Behat test suite work and the optimizations we made to keep it under 5 minutes. Some concerns where integration tests shine are making sure systems built by different teams work together, refactoring legacy code, and acceptance-level tests written in the customer's language.
However, the ratio of unit tests to integrated tests should be in the range of 10 to 1, or even 50 to 1. If there is a force at work in a project that pushes for more integrated tests than unit tests, you are falling into a vicious cycle where instead of writing:
assertEquals("1.00", new Money(100).toString());
you're writing, more often than not:
createSubscription();
assertEquals(
    "<span class="money">1.00</span>",
    findPriceTag(get("/subscription/" + id))
);
and promoting coupling between the Money, Subscription and PageTemplate objects.
The Listen to the tests principle tell us to take difficult-to-write integrated tests as a smell: a warning that we need to break the dependencies between objects to be able to reuse them, for example in isolated tests (lowering coupling); and move responsibilities around until objects respond to an interface with a small surface area (increasing cohesion).
The benefit of TDD is continuously applying these two forces in your codebase. Renouncing to it while favoring integrated tests is thinking you're able to do the same in your mind, for the rest of the life of your codebase. We test because we don't want to break features, such as being able to perform a payment; but we unit test because we don't want to impact non-functional concerns such as reuse and the ability to change.
Take me to the magic of the moment / on a glory night / where the objects of tomorrow dream away / in the wind of change

Resources

I'll stop short. Here's where you can know more about integrated and unit tests, explained by some of the best people in the field.
And finally the style of this post is inspired by Call me maybe: MongoDB.

3 comments:

Unknown said...

Unit testing vs integration testing, or unit testing in general are not related with tdd

Giorgio said...

Try reading the "Listen to your tests" paragraph to see the connection: TDD as a way to force unit tests feedback on design to be heard.

Anonymous said...

Integration tests certainly do not look like your example in the rails world.

And, from my experience (nearly 30 years sw engineering, very large scale applications), integration tests are the most valuable. They show us errors that unit tests simply cannot detect.

Last not least: 1000 unit test that run in seconds do not test very much. I'm talking from experience since I've had the 'pleasure' to fix such an app that had thousands of isolated unit tests but threw exceptions left and right when running in production.

ShareThis