Usability Testing a Website Redesign: How Many Users Is Enough?

A redesign usually gets reviewed by the people who commissioned it. The owner checks the homepage, a partner reads the services copy, someone tries the contact form on a phone. Everyone agrees it looks better. What nobody has done is watch a prospective client, with no stake in the project, try to find a price range or book a call.

Usability testing fills that gap, and the first question it raises is how many people to test with. The usual answer is five. This article looks at where that number comes from, what two later studies found when they tested it, and how to turn the evidence into a testing plan that fits a business website redesign.

---

Why a redesign needs testing with outsiders

A redesign team knows too much. The people who wrote the service descriptions know what each one means, and the people who chose the navigation labels know where everything lives. That knowledge makes it hard for them to notice a label that misleads or a page that buries the answer a visitor came for.

• Internal review checks taste and accuracy. It tells you whether the site says the right things and looks the way the business wants.
• Usability testing checks behavior. It tells you whether someone outside the business can complete a real task, such as comparing two services or sending an inquiry.
• The cost of skipping it arrives after launch. Problems that a handful of sessions would have surfaced show up instead as fewer inquiries, with no clear sign of the cause.

What usability testing is

In a moderated usability test, a researcher watches a participant try to complete specific tasks and asks them to think aloud as they go. The UK government’s Service Manual describes the method this way and notes that it can be used on prototypes, on a service already built, and on an existing service to learn about its problems. For a redesign, that means you can test the current site, the new design, or both.

The output is a list of problems: places where people hesitated, misread, took a wrong turn, or gave up. It is qualitative evidence. It shows what goes wrong and why, not how often it goes wrong across all visitors.

---

Where the five-user number comes from

The number traces back to a 1993 paper by Jakob Nielsen and Thomas K. Landauer, then at Bellcore, presented at the INTERCHI ‘93 conference in Amsterdam. Across 11 studies, they found that the discovery of usability problems, as more test users or expert evaluators were added, was well described by a simple mathematical model.

How the model works

The model treats each test session as an independent chance to catch each problem. If one average user reveals a certain share of the problems in a design, the second user reveals some new ones and repeats some the first user already hit. Each additional session adds less that is new.

The key parameter is that single-user share. In a widely cited 2000 article, Why You Only Need to Test with 5 Users, Nielsen reported a typical value of 31%, averaged across the projects he and Landauer studied. Plugged into the model, five users at that rate would uncover roughly 85% of the problems in a design.

The original paper made a cost-based argument as well. For a “medium” example project, Nielsen and Landauer estimated that 16 evaluations would be worth their cost, with the best benefit-to-cost ratio at four. The five-user figure is a point of diminishing returns, not a ceiling on what testing can find.

The part that usually gets dropped

Nielsen’s 2000 article doesn’t argue for a single test with five people. It argues for spending the same budget on several small tests. His example: if you can afford 15 participants, run three studies of five each, fixing the design between rounds. The second round checks whether the fixes worked, catches new problems the fixes introduced, and reaches deeper issues that surface-level problems had hidden in the first round.

He also set a condition on the number. The formula, he wrote, holds only for comparable users who will use the site in similar ways. For a site serving clearly different groups, he recommended three to four users from each group when testing two groups, and three from each when testing three or more.

---

What later studies found when they tested the rule

The five-user guideline drew empirical challenges almost immediately. Two are especially useful for anyone planning a website test.

Five users on real shopping sites

In a 2001 paper at the CHI conference, Jared Spool and Will Schroeder reported observing 49 users carry out the same task on four production websites. Three sites sold music, videos, and DVDs; the fourth sold electronic gadgets. Each participant was asked to describe an item they wanted and buy it on the site.

Tracking how quickly new problems appeared, the authors concluded that all four sites would need considerably more than five users to find 85% of their problems. Their explanation was that the earlier models ignored the effects of the specific site, the investigators running the tests, and their techniques, and that the complexity of the task must also change how many problems turn up. A short task on a simple interface behaves differently from an open-ended purchase on a large catalog site.

The variation hidden inside the average

Laura Faulkner, at the University of Texas at Austin, approached the question statistically in a study published in Behavior Research Methods, Instruments, & Computers in August 2003. She tested 60 users, who together surfaced 45 problems, and then drew 100 random samples at each of several group sizes to see what smaller studies would have found.

• The average held up. Across 100 random groups of five, the average share of problems found was 85%, in line with the earlier predictions.
• The range did not. Individual groups of five found anywhere from 55% to 99% of the problems.
• More users narrowed the risk. With 10 users, no group found fewer than 80% of the problems. With 20, no group found fewer than 95%.

Faulkner’s conclusion was that relying on any one set of five users risks missing nearly half of the problems, even though five users perform well on average.

“Five users tells you where to start. It doesn’t tell you when you’re done.”
— Windware Studio Practice Note

What the disagreement does and doesn’t mean

These studies aren’t as contradictory as they first appear. All three agree that each added participant finds fewer new problems than the one before. They disagree on how fast that happens, and the answer depends on the site, the tasks, the participants, and the share of problems a single user tends to reveal.

That 31% figure is an average across past projects, not a property of your website. A complex site with many paths, or a task that varies a lot between people, will tend to reveal less per user, which means five users find less. The evidence supports small rounds as an efficient way to find problems. It doesn’t support treating one round of five as proof that a redesign is ready.

---

A testing plan for a website redesign

The UK Government Digital Service turns the same tradeoff into a working rule. Its Service Manual says usability testing usually needs between four and eight participants per round, and that if you need more participants to get clear findings, you should do more rounds rather than one big round, so the team can adjust after each one. It recommends at least one round of research every two weeks during active development.

Repeated rounds work for a reason that has little to do with statistics. Picture proofreading a contract: the first read catches the obvious errors, and only after those are corrected does the second read reveal that a clause is in the wrong place. Usability testing behaves the same way when surface problems hide structural ones.

Round 1: Test the current site

• Recruit four to six people who match your likely clients.
• Give them the tasks that matter most: understand what you offer, judge whether you fit their situation, and get in touch.
• Record where they struggle. This becomes your list of problems the redesign must solve, and a baseline to compare against.

Round 2: Test the redesign as a prototype

• Test before development is complete, while changes are still cheap.
• Use the same core tasks so the results can be compared with Round 1.
• Expect new problems as well as fixed ones. A new layout can introduce confusion the old one didn’t have.

Round 3: Test the build before launch

• Run the same tasks on the working site, on the devices your visitors use.
• Confirm that problems found in Round 2 are actually fixed, not just believed fixed.
• Check the contact form end to end, including the confirmation message.

Designing the tasks

The GOV.UK guidance on moderated testing sets out what makes a good task: it gives the participant a clear goal, it is relevant and believable to them, it is challenging enough to uncover issues, and it doesn’t hint at the answer. “Find the Services page” fails that last test. “You need help with your company’s year-end accounts; find out whether this firm can help and what it would cost” passes it.

Checklist: before each round

• Participants are actual or likely clients, not staff, friends, or the project team.
• Tasks describe a goal in the visitor’s words, not in your navigation labels.
• If your site serves distinct audiences, each group has at least three participants.
• Disabled visitors are included where your audience is broad, as the Service Manual advises.
• Sessions are scheduled with time between them, and time afterward for the team to review findings together.
• You’ve agreed in advance what will change between this round and the next.

Question you need answered	Suitable method	Typical scale
Where do visitors get stuck, and why?	Moderated usability testing	4–8 participants per round, repeated
Did the redesign fix the problems we found?	Another round with the same tasks	4–8 participants
Is the new site measurably easier across many users?	Usability benchmarking	30–60 participants, per GOV.UK guidance
Did the redesign change inquiry rates?	Analytics or A/B testing	Hundreds of participants, per GOV.UK guidance

The limit of small-sample testing

Usability testing with a handful of people finds problems; it doesn’t measure them. Five sessions can show that the pricing section confuses people, but not what share of visitors it confuses or whether the fix raises inquiries. Those questions need larger samples, such as the 30 to 60 participants GOV.UK suggests for usability benchmarking, or the hundreds it says surveys and A/B tests require. A service business rarely needs that scale before launch. It does need to be clear about which kind of claim its testing can support.

---

Practical takeaways for your next website

1. Treat five as a round size, not a total. The research behind the number argues for several small tests with fixes between them, not one test before launch.
2. Start with the site you have. Testing the current site shows which problems the redesign must solve and gives you a baseline for later rounds.
3. Count your audiences. If you serve distinct groups, such as individual clients and business buyers, test each group rather than averaging them together.
4. Write tasks as real goals. Ask participants to accomplish something a client would actually try, without naming the page or link that gets them there.
5. Be honest about what you learned. Small rounds reveal problems and their causes. Claims about conversion or ease of use across all visitors need larger, quantitative evidence.

---

Ready to Discuss Your Website?

If you are planning a new website, a service redesign, or custom development for your business, we’re here to help you build usability testing into the project so the redesign is checked by the people it’s meant to serve.

• Explore Our Services: Web Design · Web Development · Website Redesign
• Start a Conversation: Tell us about your project via our Contact Form or email us directly at [email protected].