7. How to Test
Last Updated: 4/21/25
Types of usability tests
Each type of usability test is best suited to answer a particular type of question. Articulating your testing question and knowing which test best responds to that question is key to your test’s success. Although some distinctions below are artificial it is helpful to familiarize yourself with the various tests before creating your own.
Some tips before you start:
- Know and use the tools you have access to. For instance, Google Analytics has a 'In Page Analytics' feature, which provides a click map. If your site already uses Google Analytics, you may not need a separate program to create a click map. Similarly, if you need a survey of your users, and your agency already uses a survey tool, there is no need to use a new tool.
- Keep all your usability documentation together and in one place. Because this is an iterative process, you will want to use your initial tests to inform any subsequent changes you make.
Exploratory or Formative Usability Testing
Scenario: In the initial phases of development you want to judge users reactions to the design.
What/Why: Exploratory usability tests are done early in the design phase, often using paper prototypes*. This type of test requires a lot of interaction between the moderators and the testers. It can help surface
- Users expectations of the site
- Users ability to distinguish between elements of each page
- User value of the functions the site presents
These tests typically yield information about user needs and enhancement ideas for the process and next iteration. Obtaining early testing results enables you to begin the design process fully informed with the exact needs of your users, saving your time, money and work throughout the process.
Types of Questions/Prompts:
- Testers think aloud while looking at screenshots/prototype of the website.
- How would a tester conduct realistic tasks?
- To gauge user’s understanding of the site, what assumptions do they have about the purpose/function of the site?
- Does anything confuse the tester?
- Additional features or enhancements- what do testers like/don’t like?
*Paper prototyping is a baseline prototype of the interface is given to the users, and they are able to change and make edits on the spot. It is often used in early stages because they are easy and cheap to make. http://usabilitygeek.com/paper-prototyping-as-a-usability-testing-technique
Assessment Test
Scenario: You chose a design for your website and you've just implemented your first prototype, you want to know if it's working as planned.
What/Why: These tests are done early on or midway through your design after your early concepts are implemented. The assessment test helps you evaluate the effectiveness of your website. You may want to do several rounds as you make changes.
Types of Questions/Prompts:
- Ask your user to complete a specific relevant task like complete a triage survey.
- Where would you go to make a donation?
In this part of the test, the moderator should try to stay out of the way and not interrupt the test taker.
Validation Test or Verification
Scenario: You are about to launch your new website!
What/Why: Validation tests ensure your website meets certain standards. Set benchmarks for how long tasks should take and evaluate your users against these benchmarks. This quantitative data is measured and can help identify any problem areas.
Types of Questions/Prompts:
- Create an account
- Complete pre-screening eligibility survey
- Navigate to a specific page
- Benchmark: 5 minutes
A/B Testing or Comparison Testing
Scenario: You have a few options in design layout or navigation.
What/Why: When deciding on a unique, limited site decision (such as the color of a navigation element, or the wording of part of your site) A/B testing can be very instructive. This is also helpful in site redesigns when users can test your current site and compare it with a new version of the site. You can compare both sites through user feedback including strengths and weaknesses as well as navigation error rates and time to ultimately create the strongest site. This establishes the option that provides the better user experience and which enables users to complete tasks most efficiently.
Types of Questions/Prompts:
- Users complete concrete realistic tasks.
- Observe users engaging with each task measuring time it takes to complete each task
- Which option does the user prefer?
Heuristic Evaluation
Scenario: You want to compare your website's interface against a set of widely accepted principles with a few people to assist.
What/Why: Heuristic evaluation involves a small set of evaluators examining an interface and judging its compliance with usability principles or "heuristics". Use a heuristic evaluation form to help you identify key problems (form attached). According to Nielsen, just 5 evaluators can detect 85% of errors with this method!
Types of Questions/Prompts:
- Create realistic tasks users can accomplish on your website
- Use the attached heuristic evaluation form to document each tester’s experience.
*Nielsen, Jakob. "10 Usability Heuristics for User Interface Design." 10 Heuristics for User Interface Design: Article by Jakob Nielsen. Nielsen Norman Group, 1 Jan. 1995. Web. 19 Dec. 2014. <http://www.nngroup.com/articles/ten-usability-heuristics>.
Self Test and Self Audit
Scenario: You want to test your site's usability, but you have limited time and resources.
What/Why: A self-audit can help you find usability problems. By looking at your site objectively, using the Heuristic Evaluation form to find potential problems, writing questions and testing yourself you can easily surface issues that impact your site. See tech tips in rectangular boxes throughout this guide for additional resources.
Types of Questions/Prompts:
- How does the site look on mobile devices?
- Is it easy to find content on my site?
- How does it look to people with visual impairments?
- Is it screen reader accessible?
- How long does it take to complete x or y-task?