 |
 |
 |
USABILITY TESTING
This testing is also called as ‘Testing for User-Friendliness’. This testing is done if User Interface of the application stands an important consideration and needs to be specific for the specific type of user. Usability testing is the process of working with end-users directly and indirectly to assess how the user perceives a software package and how they interact with it. This process will uncover areas of difficulty for users as well as areas of strength.
The goal of usability testing should be to limit and remove difficulties for users and to leverage areas of strength for maximum usability. This testing should ideally involve direct user feedback, indirect feedback (observed behavior), and when possible computer supported feedback. Computer supported feedback is often (if not always) left out of this process. Computer supported feedback can be as simple as a timer on a dialog to monitor how long it takes users to use the dialog and counters to determine how often certain conditions occur (ie. error messages, help messages, etc). Often, this involves trivial modifications to existing software, but can result in tremendous return on investment. Ultimately, usability testing should result in changes to the delivered product in line with the discoveries made regarding usability. These changes should be directly related to real-world usability by average users. As much as possible, documentation should be written supporting changes so that in the future, similar situations can be handled with ease.
HISTORY OF USABILITY TESTING
A Xerox Palo Alto Research Center (PARC) employee wrote that PARC used extensive usability testing in creating the Xerox Star, introduced in 1981. Only about 25,000 were sold, leading many to consider the Xerox Star a commercial failure.
The Google Book Search preview, of the Inside Intuit book, says (page 22, 1984), "... in the first instance of the Usability Testing that later became standard industry practice, LeFevre recruited people off the streets... and timed their Kwik-Chek (Quicken) usage with a stopwatch. After every test... programmers worked to improve the program.") Scott Cook, Intuit co-founder, said, "... we did usability testing in 1984, five years before anyone else... there's a very big difference between doing it and having marketing people doing it as part of their... design... a very big difference between doing it and having it be the core of what engineers focus on. Cook may not have known of the PARC work, but it sounds more like he knew it only related to marketing design, as opposed to engineering and re-engineering decisions based on direct user input. In any event, at the time of this writing Google seems to have no Usability Testing projects between the PARC work and Quicken, but many after Quicken became a top commercial seller.
GOALS OF USABILITY TESTING
Usability testing is a black-box testing technique. The aim is to observe people using the product to discover errors and areas of improvement. Usability testing generally involves measuring how well test subjects respond in four areas: efficiency, accuracy, recall, and emotional response. The results of the first test can be treated as a baseline or control measurement; all subsequent tests can then be compared to the baseline to indicate improvement.
* Performance -- How much time, and how many steps, are required for people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)
* Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)
* Recall -- How much does the person remember afterwards or after periods of non-use?
* Emotional response -- How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a friend?
This testing is also called as ‘Testing for User-Friendliness’. This testing is done if User Interface of the application stands an important consideration and needs to be specific for the specific type of user. Usability testing is the process of working with end-users directly and indirectly to assess how the user perceives a software package and how they interact with it. This process will uncover areas of difficulty for users as well as areas of strength.
The goal of usability testing should be to limit and remove difficulties for users and to leverage areas of strength for maximum usability. This testing should ideally involve direct user feedback, indirect feedback (observed behavior), and when possible computer supported feedback. Computer supported feedback is often (if not always) left out of this process. Computer supported feedback can be as simple as a timer on a dialog to monitor how long it takes users to use the dialog and counters to determine how often certain conditions occur (ie. error messages, help messages, etc). Often, this involves trivial modifications to existing software, but can result in tremendous return on investment. Ultimately, usability testing should result in changes to the delivered product in line with the discoveries made regarding usability. These changes should be directly related to real-world usability by average users. As much as possible, documentation should be written supporting changes so that in the future, similar situations can be handled with ease.
HISTORY OF USABILITY TESTING
A Xerox Palo Alto Research Center (PARC) employee wrote that PARC used extensive usability testing in creating the Xerox Star, introduced in 1981. Only about 25,000 were sold, leading many to consider the Xerox Star a commercial failure.
The Google Book Search preview, of the Inside Intuit book, says (page 22, 1984), "... in the first instance of the Usability Testing that later became standard industry practice, LeFevre recruited people off the streets... and timed their Kwik-Chek (Quicken) usage with a stopwatch. After every test... programmers worked to improve the program.") Scott Cook, Intuit co-founder, said, "... we did usability testing in 1984, five years before anyone else... there's a very big difference between doing it and having marketing people doing it as part of their... design... a very big difference between doing it and having it be the core of what engineers focus on. Cook may not have known of the PARC work, but it sounds more like he knew it only related to marketing design, as opposed to engineering and re-engineering decisions based on direct user input. In any event, at the time of this writing Google seems to have no Usability Testing projects between the PARC work and Quicken, but many after Quicken became a top commercial seller.
GOALS OF USABILITY TESTING
Usability testing is a black-box testing technique. The aim is to observe people using the product to discover errors and areas of improvement. Usability testing generally involves measuring how well test subjects respond in four areas: efficiency, accuracy, recall, and emotional response. The results of the first test can be treated as a baseline or control measurement; all subsequent tests can then be compared to the baseline to indicate improvement.
* Performance -- How much time, and how many steps, are required for people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)
* Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)
* Recall -- How much does the person remember afterwards or after periods of non-use?
* Emotional response -- How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a friend?
WHAT USABILITY TESTING IS NOT
Simply gathering opinions on an object or document is market research rather than usability testing. Usability testing usually involves systematic observation under controlled conditions to determine how well people can use the product. Rather than showing users a rough draft and asking, "Do you understand this?", usability testing involves watching people trying to use something for its intended purpose. For example, when testing instructions for assembling a toy, the test subjects should be given the instructions and a box of parts. Instruction phrasing, illustration quality, and the toy's design all affect the assembly process.
METHODS
Setting up a usability test involves carefully creating a scenario, or realistic situation, wherein the person performs a list of tasks using the product being tested while observers watch and take notes. Several other test instruments such as scripted instructions, paper prototypes, and pre- and post-test questionnaires are also used to gather feedback on the product being tested. For example, to test the attachment function of an e-mail program, a scenario would describe a situation where a person needs to send an e-mail attachment, and ask him or her to undertake this task. The aim is to observe how people function in a realistic manner, so that developers can see problem areas, and what people like. Techniques popularly used to gather data during a usability test include think aloud protocol and eye tracking.
Simply gathering opinions on an object or document is market research rather than usability testing. Usability testing usually involves systematic observation under controlled conditions to determine how well people can use the product. Rather than showing users a rough draft and asking, "Do you understand this?", usability testing involves watching people trying to use something for its intended purpose. For example, when testing instructions for assembling a toy, the test subjects should be given the instructions and a box of parts. Instruction phrasing, illustration quality, and the toy's design all affect the assembly process.
METHODS
Setting up a usability test involves carefully creating a scenario, or realistic situation, wherein the person performs a list of tasks using the product being tested while observers watch and take notes. Several other test instruments such as scripted instructions, paper prototypes, and pre- and post-test questionnaires are also used to gather feedback on the product being tested. For example, to test the attachment function of an e-mail program, a scenario would describe a situation where a person needs to send an e-mail attachment, and ask him or her to undertake this task. The aim is to observe how people function in a realistic manner, so that developers can see problem areas, and what people like. Techniques popularly used to gather data during a usability test include think aloud protocol and eye tracking.
In Usability Testing approach, representative users work on typical tasks using the system (or the prototype) and the evaluators use the results to see how the user interface supports the users to do their tasks. We are all somewhat familiar with the range of methods that can be used to usability test our products or even early designs. But there may be more methods than you’ve thought about. How many of the following methods are you familiar with?
* Interviews/Observations: One-on-one sessions with users. At the Interview end of the spectrum, ask them questions about what they do. At the Observation end of the spectrum, watch what they really do. It’s often possible to conduct both types of session in the same on-site visit.
* Focus Groups: Often used in marketing well before there is any kind of prototype or product to test, a facilitated meeting with multiple attendees from the target user group.
* Interviews/Observations: One-on-one sessions with users. At the Interview end of the spectrum, ask them questions about what they do. At the Observation end of the spectrum, watch what they really do. It’s often possible to conduct both types of session in the same on-site visit.
* Focus Groups: Often used in marketing well before there is any kind of prototype or product to test, a facilitated meeting with multiple attendees from the target user group.
* Group Review or Walk-Through: A facilitator presents planned workflow to multiple attendees, who present comments on it.
* Heuristic Review: Using a predefined set of standards, a professional usability expert reviews someone else’s product or product design and presents a marked checklist back to the designer.
* Walk-Around Review: Copies of the design/prototype/wireframe are tacked to the walls, and colleagues are invited to comment. (Post-It® Notes are good for this) It also works well when users are around for some other purpose, and this is the only way you can get their attention.
* Do-it-Yourself Walk-Through: Set up a usability test situation, but without invited users. Make mock-ups of artifacts, but make the scenarios realistic. Walk through the work yourself.
* Paper Prototype Test: Use realistic scenarios but a fake product. If possible, have a colleague "play" the insides of the product or software.
* Prototype Test: A step up from a paper prototype, this one uses some type of animated prototype with realistic scenarios.
* Formal Usability Test: Using a stable product, an animated prototype, or even a paper prototype, test a reasonably large number of subjects against a controlled variety of scenarios. See How Many Subjects Do I Need for a Statistically Valid Survey by Daryle Gardner-Bonneau for pointers on how to decide when you have enough subjects.)
* Controlled Experiment: A comparison of two products, with careful statistical balancing, etc. This may be the hardest method to do "in the real world" but it’s the one you need to publish the results.
* Questionnaires: formal questionnaire, matching questionnaire (sometimes these are similar to a card-sorting exercise), phone questionnaire. Each of these formats has pluses and minuses, with questionnaire design being a field of its own.
* Heuristic Review: Using a predefined set of standards, a professional usability expert reviews someone else’s product or product design and presents a marked checklist back to the designer.
* Walk-Around Review: Copies of the design/prototype/wireframe are tacked to the walls, and colleagues are invited to comment. (Post-It® Notes are good for this) It also works well when users are around for some other purpose, and this is the only way you can get their attention.
* Do-it-Yourself Walk-Through: Set up a usability test situation, but without invited users. Make mock-ups of artifacts, but make the scenarios realistic. Walk through the work yourself.
* Paper Prototype Test: Use realistic scenarios but a fake product. If possible, have a colleague "play" the insides of the product or software.
* Prototype Test: A step up from a paper prototype, this one uses some type of animated prototype with realistic scenarios.
* Formal Usability Test: Using a stable product, an animated prototype, or even a paper prototype, test a reasonably large number of subjects against a controlled variety of scenarios. See How Many Subjects Do I Need for a Statistically Valid Survey by Daryle Gardner-Bonneau for pointers on how to decide when you have enough subjects.)
* Controlled Experiment: A comparison of two products, with careful statistical balancing, etc. This may be the hardest method to do "in the real world" but it’s the one you need to publish the results.
* Questionnaires: formal questionnaire, matching questionnaire (sometimes these are similar to a card-sorting exercise), phone questionnaire. Each of these formats has pluses and minuses, with questionnaire design being a field of its own.
Usability Testing
Related Articles: