Why do we write automated tests?
- Specify the correct behaviour of the system
- Improve code quality
- Fewer reported defects
- Make checking code faster
- Tell us when we’ve broken something
- Tell us when our work is done
- Allow others to check our code
- Encourage modular design
- Keep behaviour constant during refactoring
What do we test?
- The law of diminishing returns applies here
- Testing everything is infeasible. Don’t be unrealistic.
- 70% code coverage is actually pretty decent for most codebases.
- First, test the common stuff.
- Then test the critical stuff.
- Next, test the common exception-case stuff.
- Add other tests as appropriate.
When do we write an automated test?
- First :)
- Use a unit test to provide a framework for writing your code.
- If you find yourself running up an entire application more than once or twice to test a behaviour, wrap it in a unit test and use that test to invoke that behaviour directly.
- When you receive a bug report, write a test to reproduce the bug.
Some principles for automated tests
-
Test code is first-class code.
- Tests should be small and simple but treated as just as important as the code that actually performs the task at hand.
-
Each and every test must be able to be run in isolation
- Tests should the environment up for themselves and clean up afterwards
- Use your ClassInitialize, ClassCleanup, TestInitialize and TestCleanup attributes if you’re in MSTest-land, and the equivalents for NUnit, XUnit etc.
- Tests should never rely on being executed in any particular order (that’s part of the meaning of “unit”)
-
Tests should not rely overmuch on their environment
- Don’t depend on files’ being anywhere in particular on the filesystem. Use dynamically-derived, temporary paths if necessary.
- Don’t hard-code paths.
-
If a class depends on another class that depends on another class that you can’t easily instantiate in your unit test, this suggests that your classes need refactoring.
- Writing tests should be easy. If your classes make it hard, fix your classes first.
- If fixing your class structures is difficult, consider writing a pinning test to enable refactoring, then fixing your class coupling, then removing that pinning test.
-
Tests should be cheap to write.
- If writing a test is difficult, this suggests that the purpose of the code is unclear. Attempt to clarify the code’s purpose first.
-
Don’t worry about exception-handling.
- If an unexpected exception is thrown, the test fails. Don’t bother catching it and manually asserting failure.
- Don’t allow for variations in your output unless you absolutely have to.
- If there are going to be different outputs, ideally there should be different tests.
-
Tests should be numerous and cheap to maintain.
- Each test should test one behaviour.
- It’s much better to have lots of small tests that check individual functionality rather than fewer, complex tests that test many things.
- When a test breaks, we want to know exactly where the problem is, not just that there’s a problem somewhere in a call stack seven classes deep.
-
Tests should be disposable
- When the code it tests is gone, the test should be dropped on the floor.
- If it’s a simple, obvious test, it will be simple and obvious to identify when this should happen.
-
Tests need not be efficient.
- Efficiency helps but correctness is key.
- Optimise only when necessary.