 |
 |
 |
Test Effort Estimation Using Use Case Points
This artical presents a new approach to the estimation of software testing efforts based on Use Case Points [UCP] as a fundamental project estimation measure. From preliminary applications on our web-based projects, we conjecture that this could in fact be more reliable than FP (Functional Point). The caveat here is that the Vmodel must be in use and use case generation must start becoming available right at the requirements gathering phase. The acceptance test plan is then prepared with the use cases from the requirement documents as input. Further work could provide a more exact relationship between the two.
Introduction
Probably the most crucial difference between the manufacturing industry and the software industry is that the former is able to stick to schedules most of the time. The reason why software development schedules are so unpredictable is not because workers in this industry are lazy or incompetent. To estimate the time make a product from scratch, and in many cases, without prior experience of the technology is no mean feat. However, conventional estimation techniques address only the development effort that goes into it.
It is known that a use case to test case mapping is possible. This means that the UCP figure for development can be indirectly used to provide a figure for the number of test cases. Using organizational test execution time metrics it is now possible to arrive at a figure for the total test effort. This is a viable and systematic approach towards test effort estimation and it makes a leap in providing more realistic figures. This means that the cost of testing can now be factored into projects. The other advantage is that test engineering gets treated as a process and not simply as another lifecycle phase.
This artical presents a new approach to the estimation of software testing efforts based on Use Case Points [UCP] as a fundamental project estimation measure. From preliminary applications on our web-based projects, we conjecture that this could in fact be more reliable than FP (Functional Point). The caveat here is that the Vmodel must be in use and use case generation must start becoming available right at the requirements gathering phase. The acceptance test plan is then prepared with the use cases from the requirement documents as input. Further work could provide a more exact relationship between the two.
Introduction
Probably the most crucial difference between the manufacturing industry and the software industry is that the former is able to stick to schedules most of the time. The reason why software development schedules are so unpredictable is not because workers in this industry are lazy or incompetent. To estimate the time make a product from scratch, and in many cases, without prior experience of the technology is no mean feat. However, conventional estimation techniques address only the development effort that goes into it.
It is known that a use case to test case mapping is possible. This means that the UCP figure for development can be indirectly used to provide a figure for the number of test cases. Using organizational test execution time metrics it is now possible to arrive at a figure for the total test effort. This is a viable and systematic approach towards test effort estimation and it makes a leap in providing more realistic figures. This means that the cost of testing can now be factored into projects. The other advantage is that test engineering gets treated as a process and not simply as another lifecycle phase.
Software Project Estimation
According to Rubin, the stage-wise effort distribution on software projects is as shown in the pie chart below.
According to Rubin, the stage-wise effort distribution on software projects is as shown in the pie chart below.
Software Test Engineering
Test Engineering covers a large gamut of activities to ensure that the final product achieves some quality goal. These activities must be planned well in advance to ensure that these objectives are met. Plans are based on estimations.
In the early years, the Waterfall model has been applied to software development. This model looks upon test engineering as merely a stage in the entire development lifecycle. When techniques evolved over the years for estimating development time and effort, the concept of estimating test-engineering time was overlooked completely. Test engineering is is seldom planned for in most organizations and as a result, products enter the market insufficiently tested. Negative customer reactions and damage to the corporate image is the natural consequence.
Test Engineering covers a large gamut of activities to ensure that the final product achieves some quality goal. These activities must be planned well in advance to ensure that these objectives are met. Plans are based on estimations.
In the early years, the Waterfall model has been applied to software development. This model looks upon test engineering as merely a stage in the entire development lifecycle. When techniques evolved over the years for estimating development time and effort, the concept of estimating test-engineering time was overlooked completely. Test engineering is is seldom planned for in most organizations and as a result, products enter the market insufficiently tested. Negative customer reactions and damage to the corporate image is the natural consequence.
Estimation is basically a four-step approach:
1. Estimate the size of the development product. This is either in LOC [Lines of Code] or FP [Function Points]. The concept of using UCP [Use Case Points] is still in its
infancy.
2. Estimate the effort in person-months or person-hours.
3. Estimate the schedule in calendar months.
4. Estimate the cost in currency.
Conventional Approach to Test Effort Estimation
Test engineering managers use many different methods to estimate and schedule their test engineering efforts. Different organizations use different methods depending on the type of projects, the inherent risks in the project, the technologies involved etc. Most of the time, test effort estimations are clubbed with the development estimates and no separate figures are available.
1. Estimate the size of the development product. This is either in LOC [Lines of Code] or FP [Function Points]. The concept of using UCP [Use Case Points] is still in its
infancy.
2. Estimate the effort in person-months or person-hours.
3. Estimate the schedule in calendar months.
4. Estimate the cost in currency.
Conventional Approach to Test Effort Estimation
Test engineering managers use many different methods to estimate and schedule their test engineering efforts. Different organizations use different methods depending on the type of projects, the inherent risks in the project, the technologies involved etc. Most of the time, test effort estimations are clubbed with the development estimates and no separate figures are available.
Here is a description of some conventional methods in use:
1. Ad-hoc method
The test efforts are not based on any definitive timeframe. The efforts continue until some pre-decided timeline set by managerial or marketing personnel is reached. Alternatively, it is done until the budgeted finances run out.
This is a practice prevalent in extremely immature organizations and has error margins of over 100% at times.
2. Percentage of development time The fundamental premise here is that test engineering efforts are dependent on the
development time / effort. First, development effort is estimated using some techniques such as LOC or Function Points.
The next step is using some heuristic to peg a value next to it. This varies widely and is usually based on previous experiences.This method is not defendable since it is not based on any scientific principles or techniques. Schedule overruns could range from 50 – 75% of estimated time. This method is also by far the most used.
3. From the Function Point estimates Capers Jones estimates that the number of test cases can be determined by the function points estimate for the corresponding effort. The formula is Number of Test Cases = (Function Points) 1.2
The actual effort in person-hours is then calculated with a conversion factor obtained from previous project data. The disadvantage of using FP is that they require detailed requirements in advance. Another issue is that modern object-oriented systems are designed with Use Cases in mind and this technique is incompatible with them.
1. Ad-hoc method
The test efforts are not based on any definitive timeframe. The efforts continue until some pre-decided timeline set by managerial or marketing personnel is reached. Alternatively, it is done until the budgeted finances run out.
This is a practice prevalent in extremely immature organizations and has error margins of over 100% at times.
2. Percentage of development time The fundamental premise here is that test engineering efforts are dependent on the
development time / effort. First, development effort is estimated using some techniques such as LOC or Function Points.
The next step is using some heuristic to peg a value next to it. This varies widely and is usually based on previous experiences.This method is not defendable since it is not based on any scientific principles or techniques. Schedule overruns could range from 50 – 75% of estimated time. This method is also by far the most used.
3. From the Function Point estimates Capers Jones estimates that the number of test cases can be determined by the function points estimate for the corresponding effort. The formula is Number of Test Cases = (Function Points) 1.2
The actual effort in person-hours is then calculated with a conversion factor obtained from previous project data. The disadvantage of using FP is that they require detailed requirements in advance. Another issue is that modern object-oriented systems are designed with Use Cases in mind and this technique is incompatible with them.
Use Cases
A use case captures a contract between the stakeholders of a system about its behavior. The use case describes the system’s behavior under various conditions as it responds to a request from one of the stakeholders, called the primary actor. The primary actor initiates an interaction with the system to accomplish some goal. The system responds, protecting the interests of all the stakeholders. Different sequences of behavior, or scenarios, can unfold, depending on the particular requests made and conditions
surrounding the requests. The use case collects together those different scenarios.
Mapping Use Cases to Test Cases
Use cases in their most primitive forms are basically representative of what the user wants from a system. The advantages of Use Cases are that they start becoming available early on in the project lifecycle.
A use case captures a contract between the stakeholders of a system about its behavior. The use case describes the system’s behavior under various conditions as it responds to a request from one of the stakeholders, called the primary actor. The primary actor initiates an interaction with the system to accomplish some goal. The system responds, protecting the interests of all the stakeholders. Different sequences of behavior, or scenarios, can unfold, depending on the particular requests made and conditions
surrounding the requests. The use case collects together those different scenarios.
Mapping Use Cases to Test Cases
Use cases in their most primitive forms are basically representative of what the user wants from a system. The advantages of Use Cases are that they start becoming available early on in the project lifecycle.
UCP Approach to Estimation
Estimation using UCP [Use Case Points] is rapidly gaining a faithful following. The approach for estimation using UCP only needs slight modification in order to be useful to estimate test efforts.
1. Determine the number of actors in the system. This will give us the UAW – the unadjusted actor weights. Actors are external to the system and interface with it. Examples are end-users, other programs, data stores etc. Actors come in three types: simple, average and complex. Actor classification for test effort estimation differs from that of development estimation. End users are simple actors. In the context of testing, end-user actions can be captured easily using automated tool scripts. Average actors interact with the system through some protocols etc. or they could be Data stores. They qualify as average since the results of test case runs would need to be verified manually by running SQL statements on the store etc. Complex users are separate systems that interact with the SUT through an API. The test cases for these users can only be written at the unit level and involves a significant amount of internal system behavioral knowledge.
Estimation using UCP [Use Case Points] is rapidly gaining a faithful following. The approach for estimation using UCP only needs slight modification in order to be useful to estimate test efforts.
1. Determine the number of actors in the system. This will give us the UAW – the unadjusted actor weights. Actors are external to the system and interface with it. Examples are end-users, other programs, data stores etc. Actors come in three types: simple, average and complex. Actor classification for test effort estimation differs from that of development estimation. End users are simple actors. In the context of testing, end-user actions can be captured easily using automated tool scripts. Average actors interact with the system through some protocols etc. or they could be Data stores. They qualify as average since the results of test case runs would need to be verified manually by running SQL statements on the store etc. Complex users are separate systems that interact with the SUT through an API. The test cases for these users can only be written at the unit level and involves a significant amount of internal system behavioral knowledge.
Estimation - Use Case Points