Open vs closed card sorting: which should you use?
The two main types of card sort answer different questions. Open card sorting is for discovery — you learn how users would group and name your content. Closed card sorting is for validation — you check whether a structure you already have makes sense to users. Pick the wrong one and you'll either confirm categories you should have questioned, or drown in messy data you didn't need. Here's how to choose, with examples.
Open card sorting
Participants group the cards and create and name their own categories.
- Answers: "How would users naturally organize this, and what would they call it?"
- Best for: early-stage IA design, discovering category labels in users' language, exploring an unfamiliar domain.
- Trade-off: messier data — everyone names things differently ("Billing," "Payments," "Money"), so analysis leans on the similarity matrix to find agreement beneath the label variation, plus a standardization step to count labels fairly.
Example. You're building a help center from scratch and have no idea what the top-level categories should be. You give users 25 support topics and let them group and name freely. Ten of them independently create a group they call "Account & Billing." You've just discovered both a category and its label — something a closed sort could never have told you, because you'd never have written that category yourself.
Closed card sorting
Participants sort cards into your predefined categories (they can't create new ones).
- Answers: "Does my proposed structure hold up? Do items land where I expect?"
- Best for: validating a candidate IA, testing whether specific items fit specific categories, comparing item placement across audiences.
- Trade-off: you learn nothing about better categories you didn't include — a closed sort can only confirm or challenge what you gave it.
Example. You've drafted five help-center categories and want to confirm them before building. You run a closed sort: 18 of 20 users put "Reset password" under "Account," which validates that placement. But "Data privacy" splits — 55% "Account," 45% "Legal." The closed sort flags the ambiguity, and you make a deliberate call (primary in Account, cross-link from Legal).
Hybrid card sorting
Some studies allow predefined categories plus the option to add new ones — a middle ground that validates your structure while still surfacing gaps.
- Answers: "Does my structure work, and is it missing anything?"
- Best for: when you have a draft IA but suspect it's incomplete.
- Trade-off: slightly messier than closed (people add groups), but far more informative when your draft has blind spots.
Example. Your five categories mostly work, but a cluster of users keep creating a new group called "Developers" for API keys, integrations, and webhooks — items you'd scattered across other categories. The hybrid sort just told you to add a sixth category you'd have missed.
Which to use, by scenario
| Your situation | Use |
|---|---|
| Brand-new IA, no structure yet | Open |
| You have a draft structure to check | Closed |
| You have a draft but suspect it's missing categories | Hybrid |
| You need category labels in users' words | Open |
| You're validating item placement in known categories | Closed |
| You're deciding between two structures | Closed (one per structure) or a tree test |
Open vs closed at a glance
| Open | Closed | |
|---|---|---|
| Who names the groups | Participants | You (predefined) |
| Primary purpose | Discovery | Validation |
| Best stage | Early | Later |
| Data cleanliness | Messier (needs standardization) | Cleaner |
| Reveals new categories | ✅ | ❌ |
| Reveals users' labels | ✅ | ❌ |
| Confirms your structure | ⚠️ Indirectly | ✅ |
The typical workflow: open, then closed
The two aren't rivals — they're a sequence.
- Open sort to discover natural groupings and label language.
- Draft an IA from the card sort analysis, using the discovered categories and names.
- Closed sort (or a tree test) to validate the draft with a fresh set of participants.
Open then closed is the classic one-two punch: discover, then confirm. Running only one of them is fine for narrow goals, but new structures benefit from both.
How many participants for each?
Both types stabilize around 15–20 participants per audience. Open sorts sometimes benefit from a few more, because label variety means you need enough responses for the standardized label counts to be meaningful. Closed sorts can be slightly leaner since the categories are fixed.
Common mistakes
- Using a closed sort when you needed discovery — you'll only ever confirm the categories you already imagined, and miss the one users actually want.
- Using an open sort when you just needed validation — you'll create unnecessary analysis work sorting through label variation for a question a closed sort would have answered cleanly.
- Skipping label standardization on open sorts — un-merged synonyms fragment your counts and hide the real consensus.
- Never validating — whichever type you run, confirm the resulting IA with a tree test before you build.
Run open, closed, or hybrid free
ResearchRocket supports open and closed card sorting with the same built-in analysis (similarity matrix + suggested clusters), so you can discover and validate in one tool, then tree-test the result. Try the free card sorting tool.
FAQ
What's the difference between open and closed card sorting? In an open sort, participants create and name their own groups (discovery); in a closed sort, they sort into your predefined categories (validation). Hybrid sits between — predefined categories that participants can add to.
When should I use an open card sort? Early in IA design, when you want to learn how users naturally group content and what labels they'd choose. It's the only type that reveals categories and names you didn't already have.
When should I use a closed card sort? When you already have a proposed structure and want to validate that items land in the categories you expect, or to compare placement across audiences.
What is a hybrid card sort? One where participants use your predefined categories but can also add their own — validating your structure while surfacing categories you missed.
Should I run open and closed sorts together? Ideally in sequence: open first to discover, then closed (or a tree test) to validate. For narrow goals, one type alone is fine.
How many participants do I need for each? Around 15–20 per audience for both; open sorts sometimes benefit from a few more so standardized label counts are meaningful.
Run open, closed, and hybrid card sorts free on ResearchRocket →
Related: How to run a card sort · Card sort analysis guide · Free card sorting tool