Why do we write automated tests?
- Specify the correct behaviour of the system
- Improve code quality
- Fewer reported defects
- Make checking code faster
- Tell us when we’ve broken something
- Tell us when our work is done
- Allow others to check our code
- Encourage modular design
- Keep behaviour constant during refactoring
What do we test?
- The law of diminishing returns applies here
- Testing everything is infeasible. Don’t be unrealistic.
- 70% code coverage is actually pretty decent for most codebases.
- First, test the common stuff.
- Then test the critical stuff.
- Next, test the common exception-case stuff.
- Add other tests as appropriate.
When do we write an automated test?
- First :)
- Use a unit test to provide a framework for writing your code.
- If you find yourself running up an entire application more than once or twice to test a behaviour, wrap it in a unit test and use that test to invoke that behaviour directly.
- When you receive a bug report, write a test to reproduce the bug.
Some principles for automated tests
Test code is first-class code.
- Tests should be small and simple but treated as just as important as the code that actually performs the task at hand.
Each and every test must be able to be run in isolation
- Tests should the environment up for themselves and clean up afterwards
- Use your ClassInitialize, ClassCleanup, TestInitialize and TestCleanup attributes if you’re in MSTest-land, and the equivalents for NUnit, XUnit etc.
- Tests should never rely on being executed in any particular order (that’s part of the meaning of “unit”)
Tests should not rely overmuch on their environment
- Don’t depend on files’ being anywhere in particular on the filesystem. Use dynamically-derived, temporary paths if necessary.
- Don’t hard-code paths.
If a class depends on another class that depends on another class that you can’t easily instantiate in your unit test, this suggests that your classes need refactoring.
- Writing tests should be easy. If your classes make it hard, fix your classes first.
- If fixing your class structures is difficult, consider writing a pinning test to enable refactoring, then fixing your class coupling, then removing that pinning test.
Tests should be cheap to write.
- If writing a test is difficult, this suggests that the purpose of the code is unclear. Attempt to clarify the code’s purpose first.
Don’t worry about exception-handling.
- If an unexpected exception is thrown, the test fails. Don’t bother catching it and manually asserting failure.
- Don’t allow for variations in your output unless you absolutely have to.
- If there are going to be different outputs, ideally there should be different tests.
Tests should be numerous and cheap to maintain.
- Each test should test one behaviour.
- It’s much better to have lots of small tests that check individual functionality rather than fewer, complex tests that test many things.
- When a test breaks, we want to know exactly where the problem is, not just that there’s a problem somewhere in a call stack seven classes deep.
Tests should be disposable
- When the code it tests is gone, the test should be dropped on the floor.
- If it’s a simple, obvious test, it will be simple and obvious to identify when this should happen.
Tests need not be efficient.
- Efficiency helps but correctness is key.
- Optimise only when necessary.