 |
 |
 |
Volume Testing belongs to the group of non-functional tests, which are often misunderstood and/or used interchangeably. Volume testing refers to testing a software application for a certain data volume. This volume can in generic terms be the database size or it could also be the size of an interface file that is the subject of volume testing. For example, if you want to volume test your application with a specific database size, you will explode your database to that size and then test the application's performance on it.
Another example could be when there is a requirement for your application to interact with an interface file (could be any file such as .dat, .xml); this interaction could be reading and/or writing on to/from the file. You will create a sample file of the size you want and then test the application's functionality with that file to check performance.
Another example could be when there is a requirement for your application to interact with an interface file (could be any file such as .dat, .xml); this interaction could be reading and/or writing on to/from the file. You will create a sample file of the size you want and then test the application's functionality with that file to check performance.
What is Volume Testing?
At first glance it might appear that volume testing should be fairly high up on the V model, especially if we take the above definition. Certainly it comes under non-functional requirements testing and Performance testing.
Volume testing can (and should) be used in component testing. This would be closely correlated with the code, so might at this level be connected to Dynamic Analysis.
Volume testing will also be undertaken (normally) as part of the User Acceptance test. Stress testing is closely related, as it seeks to find out how the software will behave beyond its specified limits.
At first glance it might appear that volume testing should be fairly high up on the V model, especially if we take the above definition. Certainly it comes under non-functional requirements testing and Performance testing.
Volume testing can (and should) be used in component testing. This would be closely correlated with the code, so might at this level be connected to Dynamic Analysis.
Volume testing will also be undertaken (normally) as part of the User Acceptance test. Stress testing is closely related, as it seeks to find out how the software will behave beyond its specified limits.
Why to Use Volume Testing?
1. To find faults
2. Give credible information about the state of the component, on which business decisions can be taken.
As regards faults that should be found through volume testing are those, where the behaviour of the software deviates from that expected for a specified volume of data. Thus a bank system will be tested for faults at much larger volumes of data, than that of small retailer software. A fault which is only manifested on a table with a million records will be of no concern to the retail software, but will be picked up by the bank testers.
Credible information about how the software will behave is essential. During the dot com boom many websites went live without knowing, what the effect would be if the back end database grew exponentially. Of course many suffered crashes as a result.
Why test at the component level? This is because we can then see how the code behaves, and confirm that the component will not be a bottle neck and slow down the whole system. (Alternatively, use too many system resources.)
For example, a window often used window object is populated with data, by calling a database object which runs a complex SQL query. Supposing the component is tested against tables with only 4-5 records. Of course it will return within seconds. Everything seems fine. It is then integrated with the window, and system tested. Again everything seems ok. It is only when the application is in User Acceptance (or even gone live) and it is tested against 100,000 records, is it discovered that, the SQL was not properly optimized and the tables not indexed. Thus it should have been tested at the component level.
Some methodologies such as the RUP suggest early testing. Therefore in the early phases, (inception and elaboration), volume testing might take place to confirm that the central core architecture is the one to proceed with.
1. To find faults
2. Give credible information about the state of the component, on which business decisions can be taken.
As regards faults that should be found through volume testing are those, where the behaviour of the software deviates from that expected for a specified volume of data. Thus a bank system will be tested for faults at much larger volumes of data, than that of small retailer software. A fault which is only manifested on a table with a million records will be of no concern to the retail software, but will be picked up by the bank testers.
Credible information about how the software will behave is essential. During the dot com boom many websites went live without knowing, what the effect would be if the back end database grew exponentially. Of course many suffered crashes as a result.
Why test at the component level? This is because we can then see how the code behaves, and confirm that the component will not be a bottle neck and slow down the whole system. (Alternatively, use too many system resources.)
For example, a window often used window object is populated with data, by calling a database object which runs a complex SQL query. Supposing the component is tested against tables with only 4-5 records. Of course it will return within seconds. Everything seems fine. It is then integrated with the window, and system tested. Again everything seems ok. It is only when the application is in User Acceptance (or even gone live) and it is tested against 100,000 records, is it discovered that, the SQL was not properly optimized and the tables not indexed. Thus it should have been tested at the component level.
Some methodologies such as the RUP suggest early testing. Therefore in the early phases, (inception and elaboration), volume testing might take place to confirm that the central core architecture is the one to proceed with.
Who to Use Volume Testing?
As volume testing can be introduced almost anywhere, from component testing to user acceptance testing, the range of people who can undertake it is similarly large.
Developers through to customers and end-users can do it. The testing may be outsourced to a testing lab, which specialises in performance testing.
As volume testing can be introduced almost anywhere, from component testing to user acceptance testing, the range of people who can undertake it is similarly large.
Developers through to customers and end-users can do it. The testing may be outsourced to a testing lab, which specialises in performance testing.
How to Use Volume Testing?
Volume testing needs two things. Firstly clear expected outcomes of how the software is to behave for a given level of data. Secondly, data, and lots of it.
The expected behaviour at various levels, should be in the specification documentation. Ideally this will say something like "the customers details will be returned returned on the screen within 3 seconds, from a database with 1 million customer records." This gives the tester a benchmark to base a test case on.
The second requirement for data, needs either real life data, or simulated data. Usually, real life data will come in the form of a customer database, that has had private information, such as names and account numbers scrambled. Alternatively records can be created from scratch using automated tools or by adding rules directly on to the database, with SQL.
As with all testing, proper records must be kept showing the inputs, outputs other information, to aid potential debugging and audit purposes.
Volume testing needs two things. Firstly clear expected outcomes of how the software is to behave for a given level of data. Secondly, data, and lots of it.
The expected behaviour at various levels, should be in the specification documentation. Ideally this will say something like "the customers details will be returned returned on the screen within 3 seconds, from a database with 1 million customer records." This gives the tester a benchmark to base a test case on.
The second requirement for data, needs either real life data, or simulated data. Usually, real life data will come in the form of a customer database, that has had private information, such as names and account numbers scrambled. Alternatively records can be created from scratch using automated tools or by adding rules directly on to the database, with SQL.
As with all testing, proper records must be kept showing the inputs, outputs other information, to aid potential debugging and audit purposes.
Volume Testing - Techniques
| ■ | Load Testing |
| ■ | Stress Testing |
| ■ | Domain Testing |
| ■ | Exploratory Testing |
| ■ | Recovery Testing |
| ■ | User Acceptance Testing |
| ■ | Alpha Testing |
| ■ | Beta Testing |
| ■ | Unit Testing |
| ■ | Static & Dynamic Analysis Testing |
| ■ | Functional Testing |
| ■ | Ad-hoc Testing |
Related Articles: