Pick UX Study Participants with Inclusion, Exclusion, Diversity Criteria
Define behavioral inclusion criteria, exclude bias sources like pros, and use a recruitment matrix for diversity to ensure external validity and avoid misrecruits costing time, incentives, and bad decisions.
Costs of Misrecruits Undermine Study Validity
Poor participant selection destroys external validity, where results must represent real-world use. Misrecruits fall into poor-fit candidates (lacking experience, like students for expert roles), professional testers (over-familiar with studies, not typical users), and bad actors (lying or using AI on screeners). Spotting misrecruits mid-session requires paying incentives anyway, wasting researcher time and delaying replacements. Worse, undetected misrecruits pollute data, leading to misguided product decisions like wrong features or misunderstood needs. Prioritize screening to protect insights.
Behavioral and Attitudinal Criteria Trump Demographics
Demographics like age or income fail as behavior proxies—e.g., wealthy men born 1947-1949 could yield Ozzy Osbourne or King Charles III, with mismatched motivations. Instead, target past behaviors shaping mental models (strongest future predictor), like recent international travelers for a travel app, not aspirants. Add attitudes—what users value or prefer—for engaged, honest feedback.
Three Criteria Types Plus Recruitment Matrix for Representative Samples
Inclusion criteria specify must-haves tied to research: good-fit (e.g., nature lovers with smartphones) vs. best-fit (birders using phones outdoors for a birding app). Exclusion criteria block noise-makers like UX pros or developers who expert-review instead of user-test. Diversity criteria balance representation (e.g., tech-savviness, incomes, urban/rural) without skews—mix economy travelers, not just first-class for an airline app.
Build a recruitment matrix with behavioral/attitudinal segments as rows and diversity as columns for flexible quotas. Example for 8 bird-watching app users:
| Segment | Goal | Under 40 | 40+ | Urban | Rural/Suburban |
|---|---|---|---|---|---|
| Interested in birding | 3 | ||||
| Hobbyist Birders | 3 | ||||
| Experienced Birders | 2 | ||||
| TOTALS | 8 | 4 | 4 | 4 | 4 |
One participant fills multiple cells (e.g., suburban experienced birder 40+). Use as balancing tool: after filling urban quota, prioritize gaps. This mirrors real users, reduces bias, and maximizes insight value.