 |
 |
 |
Software Testing Techniques
Introduction
Because of the fallibility of its human designers and its own abstract, complex nature, software development must be accompanied by quality assurance activities. It is not unusual for developers to spend 40% of the total project time on testing. For life-critical software (e.g. flight control, reactor monitoring), testing can cost 3 to 5 times as much as all other activities combined. The destructive nature of testing requires that the developer discard preconceived notions of the correctness of his/her developed software.
Software Testing Fundamentals
Testing objectives include
1.Testing is a process of executing a program with the intent of finding an error.
2.A good test case is one that has a high probability of finding an as yet undiscovered error.
3.A successful test is one that uncovers an as yet undiscovered error.
Testing should systematically uncover different classes of errors in a minimum amount of time and with a minimum amount of effort. A secondary benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications. The data collected through testing can also provide an indication of the software's reliability and quality. But, testing cannot show the absence of defect -- it can only show that software defects are present.
White Box Testing
White box testing is a test case design method that uses the control structure of the procedural design to derive test cases. Test cases can be derived that
1.guarantee that all independent paths within a module have been exercised at least once,
2.exercise all logical decisions on their true and false sides,
3.execute all loops at their boundaries and within their operational bounds, and
4.exercise internal data structures to ensure their validity.
The Nature of Software Defects
Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed. General processing tends to be well understood while special case processing tends to be prone to errors.
We often believe that a logical path is not likely to be executed when it may be executed on a regular basis. Our unconscious assumptions about control flow and data lead to design errors that can only be detected by path testing. Typographical errors are random.
Introduction
Because of the fallibility of its human designers and its own abstract, complex nature, software development must be accompanied by quality assurance activities. It is not unusual for developers to spend 40% of the total project time on testing. For life-critical software (e.g. flight control, reactor monitoring), testing can cost 3 to 5 times as much as all other activities combined. The destructive nature of testing requires that the developer discard preconceived notions of the correctness of his/her developed software.
Software Testing Fundamentals
Testing objectives include
1.Testing is a process of executing a program with the intent of finding an error.
2.A good test case is one that has a high probability of finding an as yet undiscovered error.
3.A successful test is one that uncovers an as yet undiscovered error.
Testing should systematically uncover different classes of errors in a minimum amount of time and with a minimum amount of effort. A secondary benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications. The data collected through testing can also provide an indication of the software's reliability and quality. But, testing cannot show the absence of defect -- it can only show that software defects are present.
White Box Testing
White box testing is a test case design method that uses the control structure of the procedural design to derive test cases. Test cases can be derived that
1.guarantee that all independent paths within a module have been exercised at least once,
2.exercise all logical decisions on their true and false sides,
3.execute all loops at their boundaries and within their operational bounds, and
4.exercise internal data structures to ensure their validity.
The Nature of Software Defects
Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed. General processing tends to be well understood while special case processing tends to be prone to errors.
We often believe that a logical path is not likely to be executed when it may be executed on a regular basis. Our unconscious assumptions about control flow and data lead to design errors that can only be detected by path testing. Typographical errors are random.
Basis Path Testing
This method enables the designer to derive a logical complexity measure of a procedural design and use it as a guide for defining a basis set of execution paths. Test cases that exercise the basis set are guaranteed to execute every statement in the program at least once during testing.
Flow Graphs
Flow graphs can be used to represent control flow in a program and can help in the derivation of the basis set. Each flow graph node represents one or more procedural statements. The edges between nodes represent flow of control. An edge must terminate at a node, even if the node does not represent any useful procedural statements. A region in a flow graph is an area bounded by edges and nodes. Each node that contains a condition is called a predicate node. Cyclomatic complexity is a metric that provides a quantitative measure of the logical complexity of a program. It defines the number of independent paths in the basis set and thus provides an upper bound for the number of tests that must be performed.
The Basis Set
An independent path is any path through a program that introduces at least one new set of processing statements (must move along at least one new edge in the path). The basis set is not unique. Any number of different basis sets can be derived for a given procedural design. Cyclomatic complexity, V(G), for a flow graph G is equal to
1.The number of regions in the flow graph.
2.V(G) = E - N + 2 where E is the number of edges and N is the number of nodes.
3.V(G) = P + 1 where P is the number of predicate nodes.
Deriving Test Cases
1.From the design or source code, derive a flow graph.
2.Determine the cyclomatic complexity of this flow graph.
oEven without a flow graph, V(G) can be determined by counting the number of conditional statements in the code.
3.Determine a basis set of linearly independent paths.
oPredicate nodes are useful for determining the necessary paths.
4.Prepare test cases that will force execution of each path in the basis set.
oEach test case is executed and compared to the expected results.
Automating Basis Set Derivation
The derivation of the flow graph and the set of basis paths is amenable to automation. A software tool to do this can be developed using a data structure called a graph matrix. A graph matrix is a square matrix whose size is equivalent to the number of nodes in the flow graph. Each row and column correspond to a particular node and the matrix corresponds to the connections (edges) between nodes. By adding a link weight to each matrix entry, more information about the control flow can be captured. In its simplest form, the link weight is 1 if an edge exists and 0 if it does not. But other types of link weights can be represented:
•the probability that an edge will be executed,
•the processing time expended during link traversal,
•the memory required during link traversal, or
•the resources required during link traversal.
Graph theory algorithms can be applied to these graph matrices to help in the analysis necessary to produce the basis set.
This method enables the designer to derive a logical complexity measure of a procedural design and use it as a guide for defining a basis set of execution paths. Test cases that exercise the basis set are guaranteed to execute every statement in the program at least once during testing.
Flow Graphs
Flow graphs can be used to represent control flow in a program and can help in the derivation of the basis set. Each flow graph node represents one or more procedural statements. The edges between nodes represent flow of control. An edge must terminate at a node, even if the node does not represent any useful procedural statements. A region in a flow graph is an area bounded by edges and nodes. Each node that contains a condition is called a predicate node. Cyclomatic complexity is a metric that provides a quantitative measure of the logical complexity of a program. It defines the number of independent paths in the basis set and thus provides an upper bound for the number of tests that must be performed.
The Basis Set
An independent path is any path through a program that introduces at least one new set of processing statements (must move along at least one new edge in the path). The basis set is not unique. Any number of different basis sets can be derived for a given procedural design. Cyclomatic complexity, V(G), for a flow graph G is equal to
1.The number of regions in the flow graph.
2.V(G) = E - N + 2 where E is the number of edges and N is the number of nodes.
3.V(G) = P + 1 where P is the number of predicate nodes.
Deriving Test Cases
1.From the design or source code, derive a flow graph.
2.Determine the cyclomatic complexity of this flow graph.
oEven without a flow graph, V(G) can be determined by counting the number of conditional statements in the code.
3.Determine a basis set of linearly independent paths.
oPredicate nodes are useful for determining the necessary paths.
4.Prepare test cases that will force execution of each path in the basis set.
oEach test case is executed and compared to the expected results.
Automating Basis Set Derivation
The derivation of the flow graph and the set of basis paths is amenable to automation. A software tool to do this can be developed using a data structure called a graph matrix. A graph matrix is a square matrix whose size is equivalent to the number of nodes in the flow graph. Each row and column correspond to a particular node and the matrix corresponds to the connections (edges) between nodes. By adding a link weight to each matrix entry, more information about the control flow can be captured. In its simplest form, the link weight is 1 if an edge exists and 0 if it does not. But other types of link weights can be represented:
•the probability that an edge will be executed,
•the processing time expended during link traversal,
•the memory required during link traversal, or
•the resources required during link traversal.
Graph theory algorithms can be applied to these graph matrices to help in the analysis necessary to produce the basis set.
Software Testing Techniques