How many people do I talk to for good design research?
8. One-on-one, 60mins interviews. Keep moving after. Didn’t expect such a clear cut answer to what may seem such a subjective inquiry, eh?
When setting up a design sprint and conducting the initial research, clients often ask me “how many people can we interview?”. And I say, for one sprint with one set of goals and hypothesis, 8 one-on-one, 60mins interviews are enough.
And clients usually come back with:
“No! We need more. 8 is NOT comprehensive at all.”
“Ok, fine let’s do 8 one-on-ones but let’s add some focus groups so we can talk to more people!”
“Sure, in other industries, but not mine!”
They want to ensure that we talk to as many people as possible. Reduce bias. Really listen to the community. Incorporate voices that are often not heard. They are also want to ensure that the research findings hold credibility — they are looking to use this research to get buy-in internally and externally.
So a bigger number feels important, feels intuitive, feels mandatory.
And you may be thinking the same thing.
I get it. I really do!
So why do I still champion 8ish in-depth interviews?
Let me get meta. Here’s some research on this question about the research process:
Abbie Griffin and John Hauser, scholars from MIT and University of Chicago, set out to answer “how many customers need be interviewed to find customers needs and pain-points?”
Their conclusion?
One-on-one interviews > focus groups. One-on-ones are juicier, more nuanced insights + more cost-effective when assessed by resources needed to get to desired insights
20–30 interviews are necessary to get 95% of the customer needs
8–12 interviews get to 80% of the needs
Professor Steve Eppinger at MIT shared this research with me during his class on product design and development. And I have since collected a lot of evidence to support this.
So in the spirit of design thinking — where we want to move a bit fast, value doing over perfecting, work in sprints or cycles — 8ish 1:1 60mins interviews per sprint are enough!
But what about the voices we haven’t heard from? How is this equitable?
Okay. You’re right. 8 is not enough.
8 is not enough to ensure we have heard from voices of a million or more population group.
16 isn’t enough either.
100 isn’t enough either.
So “we haven’t heard from everyone” is a different goal than what design research is setting out to do.
Building relationships with consumers and partners, doing community engagement are continuous cultural practices. Design research be a piece of it via helping you better understand the people but it’s not there whole enchilada.
Can we evolve design practices to be more and more equitable and inclusive? Yes! We must!
Can we substitute design research or design sprints for community engagement? No!
Design research ≠ community engagement.
So interview any 8 people and call it day?
Well, not really.
Who you interview matters.
And that’s where we bring in a systems thinking and systems knowledge. Like many designers, I believe that those at the margins of our existing systems have the most experience with them. And you bring in a racial, gender, sexuality, ability, socio-income, adverse childhood events lens to it.
I often suggest to my clients to focus on “who is the existing system actively pushing out?”.
You’ll hear designers talk about extreme users aka those with unique perspectives on the problem.
Basically, use the 80-20 rule aka who are the 20% of the people who will get us to 80% of the insights.
And how we conduct the interviews matters. Yeah, the interview protocols and guides matter!
When preparing for design research, the north star is: understand humans and their behaviors.
At this point, I’ll often hear: “but we need to understand them as [insert user description] and ask direct questions on their interest in [interest product name] so we can justify doing this project”.
User interviews DON’T get us closer to humanity of others.
PERIOD.
Instead of seeing these as user interviews, see them as expert interviews with humans!
People are experts in their life — and we (as designers) are learners, with a beginner’s mindset. Designers are experts on the process. And our clients are experts in the subject matter, the service, or the technology.
Let the interviewee talk about their life. Focus on that during the interview. Later, when synthesizing, as experts in process and product, we can then deduce what it means from a product/program/service design perspective.
Another protocol tip? Have a live notetaker for every interview. That way when you debrief, there are multiple perspectives available! Pro-tip: clients as notetakers leads to enthusiastic buy-in of the insights!
So 8 well-run interviews will get me to the outcome?
Well, not exactly.
The design process is about iteration.
It’s sprints aka cycles of doing, building, learning. 8 are enough for one sprint. And then when you learn more, you redefine your hypothesis and conduct more interviews. Maybe you’ll get more focused in terms of your problem statement or population and conduct more interviews focused on that.
Think in sprints and cycles rather than one and done, linear process.
All of that to say...
To do human-centered design research well:
Intentionally set out some goals and hypothesis
Define the populations you want to focus on. Think those on the margins!
Build protocols and guides. Think: human > users!
Conduct 8 (ok, fine, max 12), 1:1 60min well-run interviews, with a notetaker
Have multiple people in the room and analyzing the data. I like to involve my clients in this step
And then you move: define problem statement, ideate or find a new hypothesis