One way of keeping your test suites running fast is by organising them in a way that allows you to run the right tests at the right time. This might be running the faster, isolated tests to give you instant feedback in your TDD loop, or it might be running the most critical acceptance tests before you commit a changeset.

PHPUnit offers a number of ways to organise your test suite, but the docs are a little light on commentary, these are my thoughts.

Using the filesystem

The first and easiest way to get started is to use the filesystem. If you give PHPUnit a directory as an argument, it will scan that directory (recursively) for *Test.php files and then execute them. This means we can start to split our test suite up, just by putting files in different places.

For the main app's test suite, I have two top level PHPUnit directories.

The first is tests/unit and this is for isolated, fast, unit tests. These tests tend not to interact with any external systems, quite often not interacting with other classes, functions or components at all. They take just a few milliseconds each to run and there are quite a lot of them.

The second directory is called tests/integration and this directory gets everything else. I try not to get too caught up in naming the types of tests I write, but depending on your way of thinking, this directory includes unit, functional, integration, integrated, system and acceptance tests. The tests in here exercise larger parts of the code and usually interact with third party systems like databases or HTTP APIs. In a lot of cases, I use fakes to speed up these tests, but they're still slow, taking a couple of minutes to run the entire directory.

Under these directories, I tend to follow the file structure of the production code as close as possible, though that quite often doesn't apply to the acceptance tests that I run out of tests/integration, which tend to be something like tests/integration/src/Childcare/<module>/NameOfFeature.php. If I were starting green field today, I might have a separate top level directory for some of those, but I don't lose sleep over it. Either way, having the tests with a decent hierachy allows for flexibility in running just the tests for a section/module/component as needed.

Failing fast

Given this simple split of faster tests and slower tests, I am able to run the faster tests first, followed by the slow tests. In theory, this should mean I'm getting the feedback I need quicker. To faciliate this, I use a simple Makefile as a task runner.

.PHONY: check

    vendor/bin/phpunit tests/unit
    vendor/bin/phpunit tests/integration

Using groups

Even given this separation, I still find I need some further organisation. Some of those tests in tests/integration are really slow. I have a bunch of code that interacts with the Facebook API and to make sure my code keeps in tune with the API responses, I really want to have some tests that actually hit the API. It turns out that creating test user accounts is quite slow and this slows my tests down significantly, to the point where I start to get annoyed when I'm running tests before a merge. Rather than creating another top level directory to house the dozen or so tests that create Facebook users, I decided to use PHPUnit's group feature to allow me to exclude these tests as I please.

/** * @test * @group facebook */ public function some_expensive_test() {}

Again, using our Makefile as a task runner, I make a recipe that runs the tests without those facebook tests.

.PHONY: check check-quick

    vendor/bin/phpunit tests/unit
    vendor/bin/phpunit --stop-on-failure --exclude-group=facebook tests/integration

I have a few other groups excluded and I use this run regularly throughout the day, only running the full suite occasionally. Our continuos integration server always runs the full suite when I push code, so I can carry on developing carefree with regards to our Facebook integration. Because I use this for rapid feedback, I also use the --stop-on-failure switch. This can sometimes be annoying if you've broken code in several places, but more times than not the first failing test allows me to identify a smaller set of tests to run, make fixes and run again.

Using @small, @medium and @large

One alternative to separating faster and slower tests in to directories is to use the special @small, @medium and @large annotations. These are aliases for @group small etc, but also have special meaning if you install the phpunit/php-invoker package. With this package installed, if you run PHPUnit with --enforce-time-limit, PHPUnit will mark these tests as risky if they do not execute within a set of time thresholds.

Using test suites

Another way to organise the PHPUnit tests is to use test suites. This isn't something I currently utilise, but they are quite flexible and can mimic the behaviour I achieve using the Makefiles.

<phpunit bootstrap="src/autoload.php">
    <testsuite name="all">

The first thing to note is that when using a test suite, PHPUnit will run the tests in the order specified. This means with the example above, we get that failing fast scenario as our faster tests/unit tests get run before the slow tests/integration tests.

You can also get fairly specific about the files you want to include or even exclude, so depending on the way you set your files and directories up, you can achieve something like I did above with the facebook group.

    <testsuite name="quick">

I tend not to use test suites as I find the combination of directories and groups to be enough.

I plan to follow up on this post with a more detailed look at which tests to run at what time, drop your email in the box below to be notified when that one lands.

Faster Tests in PHP

  1. Avoiding latency with Fakes
  2. Organising Test Suites