Synthetic Versus Human Research Is the Wrong Question
Every conversation about synthetic research eventually arrives at the same question. ...
Read ArticleArticles & Insights
Tag: Research Bias
Research Bias
Most poor research decisions are not caused by bad data. They are caused by bad data that looks fine. Survivorship bias hides the failed companies that never made it into the dataset. Selection bias means your survey respondents are the ones who chose to respond. Framing effects and question-order effects mean small wording changes flip the answer. Social desirability bias means your respondents tell you what they think you want to hear. None of these are exotic. All of them are everywhere.
The articles in this collection cover the biases that matter most in market research practice. Each one is a practical guide: what the bias is, how to spot it in your own data, why it tends to show up in synthetic research as well as traditional methods, and how to design studies that minimise its impact. These are the pieces to read before designing a study that will inform a real decision.
What you'll find
- Survivorship Bias: Why Your Data Only Shows the Winners — and how to systematically include the failures.
- Selection Bias: Why Your Survey Respondents Don't Represent Your Market — and the structural fixes that work.
- Question Order Effects: How Survey Structure Changes Your Data — randomisation patterns and ordering rules.
- The Framing Effect: How You Ask Changes What People Choose — and why pricing studies are particularly vulnerable.
Design a study that controls for bias at fish.dog
The Framing Effect: How You Ask Changes What People Choose
In the early 1980s, psychologists Amos Tversky and Daniel Kahneman presented people with a scenario:...
Read Article
Question Order Effects: How Survey Structure Changes Your Data
In 1948, researchers asked Americans two questions:...
Read Article
Selection Bias: Why Your Survey Respondents Don't Represent Your Market
Your survey says 85% of customers love your new feature. Your sales data says nobody's using it. What went wrong? ...
Read Article
Survivorship Bias: Why Your Data Only Shows the Winners
Survivorship bias is the logical error of focusing on things that made it past some selection process while ignoring the...
Read ArticleFrequently Asked Questions
What is the most common type of research bias?
Selection bias is probably the most pervasive: in any survey where participation is voluntary, the people who choose to respond differ systematically from the people who do not, and the difference is rarely accounted for. The selection bias article in this collection covers structural fixes including stratified sampling and weighted analysis.
What is survivorship bias?
Survivorship bias is when your dataset includes only the entities that survived some selection process, hiding the entities that did not. The classic example is studying "successful CEOs" without studying the equally hard-working CEOs whose companies failed. Without including the failures, every "success factor" is suspect.
How does framing affect research results?
Framing effects are when the wording or context of a choice systematically shifts which option people pick — even when the underlying options are mathematically identical. A famous example: people choose "surgery with a 90% survival rate" much more often than "surgery with a 10% mortality rate." Framing matters enormously in pricing research and concept testing.
Does synthetic research have the same biases as traditional research?
Synthetic research mitigates some biases (no participation bias, no social desirability bias, no question fatigue) but inherits others (any bias baked into the persona-construction data carries through to the panel). The articles in this collection note which biases are mitigated and which are inherited for each category.