Automated refactoring without heavy IDEs

Unix programs are beautiful and universally compatible. You can chain them to accomplish incredible tasks. Let's do some automated refactoring such as renaming php classes and methods without IDEs, using only sed, find and grep, plus your version control system of choice. You do not have to edit many hundred files by hand, and these tools should be available in every Linux distribution and maybe also on Mac Os X.

Renaming classes
START
Commit your work and make sure that the output of svn status is empty.

svn update
svn commit -m "what I have done until now..."
svn status

Refactoring is not an exact science, so wrong commands can destroy a codebase. In case you forget a \ and all your <?php tags are replaced by a apt-get cow, you'll simply enter

svn revert -R .

to restore the original state of the working copy after the last commit.
I'm sure your version control system has similar commands.

STEP 1
Provided that you use autoloading, moving or renaming the file that contains the class definition is the first step of the operation.

mv Package1\OldClassName.php Package2\NewClassName.php

Usually directly hooked in the version control system:

svn move Package1\OldClassName.php Package2\NewClassName.php

or what your VCS provides you with.

STEP 2
The syntax of the sed command is analogue to vim's search&replace one:

sed -i -e 's/\<old_classname\>/New_ClassName/g' \
    `find folder1 folder2 -name *.php`

This command renames all OldClassName occurrences:

• in place, without creating new files (-i)
• using an expression defined on the command line and that follows (-e)
• where OldClassName is present as a single word (\< and \> modifiers), and not for example AbstractOldClassName
• substituting them with NewClassName
• in all the file and thorughout all the single lines (/g)

We should list after the command all files we want to edit, but we can simply use a find command which find all files that:

• is in folder1 or folder2, or in their subdirectories (you can insert as many folders as you want)
• have a name that matches *.php (essentially their extension has to be .php)

The backticks (`) simply indicate to substitute the enclosed command with its result. They are a powerful tool to use sub-commands (otherwise we would have been stuck with find -exec.)The \ let the shell know that the command continues on the new line.

When the php classes to rename do not take advantage of Php 5.3 new features, the renaming it's very simple, since OldClassName is always the fully qualified name of the class. From php 5.3 namespaces are available to structure classes in different packages without very long names, but the namespace separator unfortunately concides with the shell backslash. Thus to enter it you have to insert another backslash:

sed -i -e 's/\<old\\classname\>/New\\KlassName/g' `find folder1 folder2 -name *.php`
sed -i -e 's/\<classname\>/KlassName/g' `find folder1 folder2 -name *.php`

The first sed modifies the use statements or the direct references, which are the means 99% of classes are referred by. The second sed modifies the remaining references to the base class name embedded in php files and it may not be necessary if you're only moving a class around.
A problem that occurs is when you're moving a class from a namespace and their old sibling did not have use statement to import its name but now they have to, or the equivalent specular case when you're moving a class into a namespace and you want use statements to vanish. You'll have to resort to manual editing to fix the interested files.

Don't forget to repeat the two steps also with the test classes. The exact commands depend on your naming convention, but often the test cases mirror the production code hierarchy, so that OldClassNameTest should be replaced with NewClassNameTest. In xUnit frameworks, test cases are first order citizens (classes), so there's nothing different from the production code renaming process.

END
Check the results by looking for every occurrence of the old or new name in your working copy:

grep -r "OldClassName"

or grep -rl to show only the list of files where the pattern does occur.
svn diff and svn status shows you the current changeset (every modified line, added and deleted ones) and the list of modified files respectively.
After manual check, run your whole test suite. If some of your test fail, discover the errors (that's what tests are for) and repair the situation manually. If the working copy is compromised, revert to the original revision.

Renaming methods
Provided that you have not overlapping method names in unrelated classes, the START and the END phases are the same, and they are very good practices to adopt in refactoring.
Methods may have a limited impact on the codebase in respect to classes:

• to rename private methods, you can edit them directly in your editor of choice, opening the class file.
• For protected methods, you should check also subclasses, and when they are a handful it's often faster to direct edit the files instead of running sed.
• For public methods, the problem is analogue to renaming a class, but there are no file movements nor namespacing issues. Follow Step 1.

Of course both these refactorings break backward-compatibility, so make sure you're not modifying any published interface in a minor release of your application. They are also dangerous if uncontrolled, because regular expressions are an advanced tool that can quickly mangle all your code. Make sure you can return to the original copy of the code via version control at any time, and that there are many tests in place that cover the functionalities whose provider classes you want to refactor.
Refactoring shouldn't be hard. I hope this little guide helps you.

5 reasons to be happy in a terminal

I am not talking about an airport terminal, but about one of the terminal emulators which are provided by modern window managers, like gnome-terminal for Gnome and the similar Konsole for Kde, along with the minimal xterm. These are all unix applications but equivalent applications exist for other platforms like Windows, although their integration with the underlying operating system and with specific programs can be tricky.
Why should you, a software developer/engineer, want to pass most of your time in a dumb terminal instead of in a powerful and costly IDE? I have five reasons to convince you.

• instant access to unix programs. GUIs facilitate the job of naive users but the real power resides in the command line tools which perform the real work; moreover, cli programs can be chained in infinite ways using their universal plain text interface.
• the classic 80x25 terminal has short lines and a short number of lines: too much logic in a line stands out because the line wraps on the subsequent one. Too long methods are spotted as well, because they don't fit in a single or double screen and require multiple scrolling.
• transparent remoting with ssh. The same can be said for VNC, but it can be very slow and it's not always supported while many servers have a ssh daemon. It is so fast I did not notice the latency between my local machine and other boxes in my Lan, so I have given them different colored prompts to easily distinguish between environments.
  
• uninterrupted flow; not using the mouse makes you move very quickly in the cli environment, once you know what to write and how to leverage the text- based tools.
  
• every executed command is registered for possible future repetition and modification. Try record a procedure of 90 control panel clicks instead.

As a side note, thanks to history, it's also very simple to calculate statistics on which are the most popular commands you type on your development box:

[12:29:32][giorgio@Indy:~]$ history | awk '{print $2;}' | sort | uniq -c |
> sort -nr | head -n 10
   4924 vim
   1326 svn
    879 nakedphpunit    // it's an alias for phpunit --bootstrap=...
    616 sudo
    438 cd
    266 ls
    238 osstest_sqlite
    207 phing
    135 ./scripts/regenerate
    127 grep

Of course most of them were only typed the first time and then recalled. From these data you can infer that I use the command line interface a lot, and I've never been more productive. This statistic is a typical leverage of command line tools in a construct that took me less than a minute to write and that I can repeat whenever I want in a few seconds.

Sooner or later, the time comes when a developer feels constrained by his graphical interfaces and resorts to use the command line directly. If he avoids the command line, probably it's because he does not know how to work with it. Don't be so proud like this developer and take some time to learn: the cli will repay you soon.

Practical testing in php part 9: command line options

This is the ninth and last part of the php testing series. If you liked it, you should subscribe to the feed to check out similar articles in the future.

Optimizing a test suite by adding test methods and test cases can be useful to improve the quality of your application code. Yet, every optimization starts with a profiling phase, that tells you where there is a need for test cases and where there is already a good coverage.
Code coverage is defined as the ratio of lines of code exercised by the unit tests to the overall number of lines; the same ratio can be calculated using code blocks as the unit of measure.
Phpunit provides code coverage reports generation via command line switches: one of them is --coverage-html $directory which places a human-readable html report in the $directory specified. There are other formats available, such as Xml, created for the purpose of interpreting a report with a third party application.
Please note that phpunit code coverage features require the xdebug extension to work.

Another useful switch is the --configuration $file one. $file should be an xml configuration file that tells phpunit what files have to be considered as containing test cases. This is very handy to compose a suite and can substitute the famous and hard to mantain AllTests.php files.
Here is a simple example for a configuration file:

<phpunit>
    <testsuite name="Ossigeno Test Suite">
        <directory suffix="Test.php">tests/</directory>
        <directory suffix="Test.php">application/modules</directory>
    </testsuite>
</phpunit>

Running phpunit with this switch, instead of specifying a particular file, will force the runner to consider all php files which name ends in "Test.php" in the directories tests/ and application/modules/. While running a single test case gives as output a line of dots, running all these tests in sequence will result in multiple lines and in a list of all failed tests (although you can require the list of skipped and incomplete tests by using the --verbose switch) in the overall list generated according to the configuration.

Along with the --configuration option, I strongly suggest to use the --bootstrap $phpScript directive. Your test cases probably need a global bootstrap phase for autoloading and setting up the include_path or other options. In some old versions of phpunit, you had to include a require_once() call at the start of each test script to make sure it was executed before the test. Now you can simply tell phpunit to run a file of your choice before starting with the test phase.

Running an entire suite is a good practice to discover if your changes or refactorings have broken some functionalities. However, it's an overkill if you have to do it very often, like in a short feedback cycle for TDD: supposing you have more than one test case for your SUT, it can be useful to select all those tests and leaving out the rest of the suite.
This is the case when the @group annotation is handy. You can mark with the @group $name annotation the docblock of test case classes, and also add multiple lines if you feel the contained tests can be useful in more than one scenario. Then the --group $group command line switch excludes test cases which do not belong to $group from being run.

So we can finally give an example of running a test suite:

phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml

for instance we can restrict the selected tests to the NakedPhp_Form package ones:

phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml --group=NakedPhp_Form

or requiring a code coverage report to see where we need to add test code:

phpunit --bootstrap tests/TestHelper.php --configuration=tests/configuration.xml --coverage-html directory/

I hope these tips will be useful to you for utilizing phpunit at its best. It is a very well-crafted tool that you can take advantage of for TDD purposes, and also for functional and integration tests. Although the name suggests unit testing as a goal, you should certainly include in your test suite some functional tests, which exercise a feature provided of more than one object, and integration tests, which covers the wiring of your object and verify that your application works on an end-to-end basis.

Bonus tip: using --no-globals-backup and --no-static-backup can speed up your tests execution by avoiding unuseful isolation of tests. If your application has no global state they will work correctly anyway.
If you liked this testing series, you should subscribe to the feed to be informed of new articles on software development and php.