The default structure for a concept-testing survey is a single concept description or image, followed by a rating scale -- "how appealing is this, 1 to 10." The number that comes back is nearly impossible to act on in isolation. A 7 out of 10 could mean this concept is strong, or it could mean everything this audience rates lands around a 7 regardless of merit, because there's nothing to compare it to and no sense of the respondent's rating baseline.
Isolated ratings don't tell you which concept wins
The question a product or marketing team actually needs answered is comparative: of the concepts under consideration, which one resonates most, and why. A survey that shows concepts one at a time with an isolated rating scale can't answer that directly -- it produces a set of averages that have to be compared across separate response sets, which introduces noise from different respondents rating on different personal scales.
Structuring a concept test that produces a real comparison
- Show multiple concepts to the same respondent, not one concept per respondent. Having each person react to two or three concepts and directly rank or choose between them removes the cross-respondent scale problem entirely -- you're comparing what the same person thought of concept A versus concept B, not comparing different people's independent 1-10 scores.
- Force a relative choice, not just parallel ratings. "Which of these two appeals to you more?" produces a cleaner signal than "rate each of these separately," because it removes the ambiguity of two concepts both getting rated a 7 with no way to tell which one actually wins.
- Branch the follow-up questions by which concept won for that respondent. Once someone has picked a preferred concept, the more useful follow-up is specific to that concept -- what appealed about it, what would make it better -- rather than a generic open-ended question asked regardless of which concept they preferred.
- Control for position bias with an A/B split rather than assuming order doesn't matter. Run two variants of the same test -- concept order reversed between them -- as an actual A/B test, and compare win rates across both variants rather than one variant's results alone. That catches a concept winning mainly because it was shown first, which a single fixed-order test can't detect on its own.
What the output should look like
The result worth having isn't a list of average ratings per concept -- it's a clear win rate (concept A preferred by 61% of respondents who saw both A and B) plus the qualitative reasoning captured from the branch-specific follow-ups, tied to the concept that actually won for each respondent. That combination -- a hard number plus the "why" specific to the winning concept -- is what a product team can actually build a decision on.
An isolated rating tells you how one concept landed. A forced comparison tells you which concept to build.
Building this without a developer
Presenting multiple concepts to the same respondent, forcing a relative choice, and branching follow-up questions by which concept won is buildable visually using the logic engine's branching. Testing two order variants against each other for position bias uses the same A/B testing feature built into the platform, rather than a manual split you'd have to set up yourself. The work is loading your specific concepts and follow-up questions, not building the comparison or testing logic from scratch.