Unit tests (and indeed all automated tests) are meant to be a safety net and status indicator for the project. They tell us when we have a problem, or if something is behaving differently than it did before. Sometimes, tests can be written that are inconsistent, erratic, or just downright flakey. I am of the opinion that these kinds of tests are not trustworthy, and as such, should NOT be part of a build.

Sometimes, tests are written with inherent problems that cause them to be unreliable:

• race conditions
• incomplete test data
• relying on external data
• date/time calculations dependent on the current date/time
• many other factors

If any of these kinds of issues are causing the tests to be inconsistent (that is pass and/or fail when run multiple times with no code changes), then these tests can’t be relied upon to give us the safety we require in a test suite. I recommend that teams immediately identify these types of tests and either exclude them or outright fix them so they can be trusted to give the right feedback.

Writing good tests is not just something we’re born with, it takes practice and skill. Skills like these are very important and very much required for developers to be successful. There are a lot of good books that we can read to acquire these skills, here are a few links which may help.

xUnit Test Patterns

http://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/ref=sr_1_1

Agile Testing

http://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468/ref=sr_1_1

Testing Extreme Programming

http://www.amazon.com/Testing-Extreme-Programming-Lisa-Crispin/dp/0321113551/ref=sr_1_6