Tested against people betting real money.
A survey records what people say. A prediction market records what people are willing to bet on, and every market ends in a fact. FishDog puts the same questions traders are betting on to its synthetic population, and keeps score.
Real money is the counterparty
Traders lose money when they are wrong, so a market price carries information a survey answer does not. It is a demanding benchmark to sit next to.
Every market settles
The event happens or it does not. There is no arguing about methodology after the fact, which makes it a signal-versus-outcome test.
The answer is not in the past
The tests run on events that postdate the model’s training, so the population cannot recall the answer. It has to reason its way there.
ContextReal-money markets have a long record as forecasters: the Iowa Electronic Markets, an academic market, beat contemporaneous polls 74 percent of the time across 964 polls (Berg, Nelson, and Rietz, 2008). FishDog uses that literature as context, not as its own evidence.
One market, start to finish
Drag through the life of one market, or jump to a moment, to see how it becomes a test.
Illustrative market · Will the Federal Reserve cut rates at its next meeting?
- Market price for yes
- Panel share betting yes
A reliability profile before the event settles
Because every market is scored on the same 18 axes, a new question can be compared with earlier questions of the same shape. That gives a reliability profile while there is still time to act. Pick a question to see its profile.
Stronger
Technical and expertise-driven events are where the population is strongest. Questions like this one resemble many earlier markets that settled.
What this benchmark is, and what it is not
A benchmark, not training data
Prediction markets are what the population is checked against. The markets themselves are not what it is built from. The population comes from census data and its own news sources.
A pattern, not a single call
One market is one event, and a single event cannot be back-tested. The value is in the accumulation: many markets, scored on the same axes, showing where the population holds and where it does not.
One layer of four
Prediction markets sit alongside the census construction floor, Gallup and Pew comparisons, and the University of Michigan consumer-sentiment series. The Synthetic Populations paper sets out all four.
Weak spots stated up front
Low-information markets, where there is little public signal to reason from, are the weakest area, and FishDog says so. When the map shows a gap, FishDog adds signal and the gap is tracked until it closes. A profile that admits its weak spots is one a reviewer can use.