8. Who to Test
Last Updated: 5/06/25
The idea of finding individual participants to test your site can be overwhelming and a stumbling block to routine user testing. However you do not need hundreds of testers to obtain good information.
The Nielson Norman group indicates that 5 users can uncover 85% of the major usability issues, and 15 users can find 100%.*
How many testers will I need?
The same studies suggesting small testing groups also stresses the need to conduct several rounds of testing on your site.
If you are testing at several stages (i.e.: beginning and during your design process, and prior to launch) consider testing with multiple groups at each point.
With these small and agile groups, it is important to define your audience and find representative users. Most sites have several user types, such as clients, advocates, community members, and the press. Your user testing should reflect each of the user types accessing your site. If you do not know who is using your site, look at your analytics, or engage users from groups you want accessing your content.
*Nielsen, Jakob. "Nielsen Norman Group." Why You Only Need to Test with 5 Users. Nielsen Norman Group, 19 Mar. 2000. Web. 19 Dec. 2014. http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users
Mediated vs. Unmediated Testing
Below are some of the differences between mediated and unmediated user testing.
Mediated
- Staff person walks the tester(s)through a series of tests and follows a script
- Script describes the test, handles user questions, and concludes the test
- Moderator says and acts exactly the same to avoid bias
Unmediated
- Less structured allowing the user to act naturally with minimal input
- You may choose to record a user's screen or monitor their clicks to evaluate how they use the site
- You can follow-up with a debrief or focus group with a moderator
In Person Testing - Website Usability Testing Guide
In-person testing
In person testing allows you to see and interact with users in real time, with minimal barriers. If you choose to test your site with in-person testers (where you monitor and your testers share a physical space) there are a few options:
- Hallway testing: Engaging 5-6 randomly selected people to determine if there are issues so large untrained users cannot navigate through them. Because of this, users should not be familiar with your site.
- Focus groups: Engaging 5-6 participants to participate in a more formalized setting to conduct testing. Typically these users have more familiarity with your site, and you may conduct different focus groups with different users groups.
Where to Conduct the Test
In-person user testing can be conducted anywhere comfortable to a user. Many agencies establish formal focus group sessions held in an office or a conference room. If you choose a formal setting consider factors such as the room configuration (i.e.:a classroom style, or circle/horseshoe). Just remember in-person testing does not have to be formal- you can engage clients in a waiting room with a staff person and a laptop, so long as you have a script.
Remote Testing - Website Usability Testing Guide
Remote testing
Remote testing allows you to conduct usability tests with the participants utilizing their own computers through online usability programs. Although these can be customized, overall the tests are between 15 and 30 minutes and contain 3-5 tasks per test. They can be moderated or unmoderated, depending on your needs and the platform.
Benefits of Remote Testing:
- Eliminates the need for physical space
- Encourages a larger more diverse group to engage
- Generally less-expensive than in-person testing
- Test can stay open longer.