How do you know when testing is complete? More testers might answer. "When I run out of time! but there are other factors the Test Manager can use to make this decision. The Test Manager must be able to report, with some degree of confidence, that the application will perform as expected in production and whether the quality goals defined at the start of the project have been met.

The Test Manager may use a set of test metrics, including Mean Time Between Failure or the percantage of coverage achieved by the executed tests, to determine whether the application is ready for production. Other factors, such as the number of open defets and their severity levels, must also be taken into consideration. Finally, the risk associated with moving the application into production, as well as the risk of not moving forward, must be taken into consideration.

General Concerns

There are three general concerns testers have in performing tests:
* Software is not in a testable mode for this test level.

The previous testing levels will not have been completed adequately to remove most of the defects and the necessary functions will not have been installed, or correctly insalled in the software (i.e., performing integration testing when some units contain defects). Thus testing will become bogged down in identifying problems that should have been identified earlier.

* There is inadequate time and resources.

Because of delays in development or failure to adequately budget sufficient time and resources for testing, the testers will not have the time or resources necessary to effectively test the software. In many IT organizations, management relies on testing to assure that the software is ready for production prior to being placed in production. When adequate time or resources are unavailable, management may still rely on the testers when they are unable to perform their test as expected.

* Significant problems will not be uncovered during testing.

Unless testing is adequately planned and executed according to that plan, the problems that can cause serious operational difficulties may not be uncovered. This can happen because testers at this step spend too much time uncovering defects rather than evaluating the operational performance of the application software.

When is Testing complete?