Product Release

Polymarket events as a calibration source

FishDog adds a Polymarket integration that imports prediction-market events as calibration targets, runs recruited panels against them, and compares the panel's predictions to Polymarket's live pricing — falsifiable validation for synthetic personas.

4 March 2026

Phillip Gales

By Phillip Gales

Feature
FishDog's Michigan Consumer Sentiment Proxy page, showing the live ICS, ICC, and ICE composite indices computed from a 100-persona panel.

Key Takeaways

  • Polymarket events can now be imported into FishDog as calibration targets, with suitability filters that remove very low-liquidity markets.
  • A recruited panel answers the imported question; the resulting distribution is compared against Polymarket's mid-market price at run time.
  • Validation metrics (calibration error, Brier-score components) are computed and persisted to a calibration store, so repeated runs build a track record.
  • Useful for research-validity work, for selecting panels by demographic composition, and for internal benchmarking of the persona model.
  • Follow-up on 18th April broadened the import selection and added pagination to the events gallery.

Synthetic personas are useful only insofar as their predictions track the real world. Polymarket — a real-money prediction market — is one of the cleanest external sources we've found for grading them.

The new Polymarket integration imports markets into FishDog as calibration targets. Pose a market's question to a recruited panel, collect their predictions, and compare the panel's distribution to Polymarket's live pricing. The result is a falsifiable test of how well a FishDog panel predicts real-world outcomes for a given category of question.

What's new

  • Polymarket event importer. Browse Polymarket events directly in FishDog, filter by category and resolution date, and import the ones that fit your validation work. Suitability rules filter out very low-liquidity or thinly-traded markets — those that aren't a useful calibration target.

  • Calibration runs. A recruited panel answers the imported question; the panel's distribution is compared against Polymarket's mid-market price at the time of the run. Validation metrics (calibration error, Brier-score components) are computed and persisted.

  • Calibration store. Repeated runs against the same market over time build a track record. Useful for showing that a panel's predictions on, say, US elections are calibrated within a known error band — or aren't.

When to use this

  • Research-validity work. When a customer wants evidence that FishDog panels predict realistically on a given category, this is the cleanest external grading available.

  • Panel selection. Different panels predict differently. Calibration runs help identify which panel composition (demographic mix, recruitment filter set) tracks markets in your category most reliably.

  • Internal benchmarking. We use the calibrated track record ourselves when refining the persona model.

Also in this release

A small follow-up shipped on 18th April broadened the import selection (more event categories, looser default filters) and added pagination to the events gallery, so importing from a larger Polymarket session takes one trip rather than several.

Full reference is in the API docs.

---

Synthetic personas are useful only insofar as their predictions track the real world.
A falsifiable test of how well a FishDog panel predicts real-world outcomes for a given category of question.
Useful for showing that a panel's predictions on, say, US elections are calibrated within a known error band — or aren't.

Frequently Asked Questions

What is Polymarket and why use it?

Polymarket is a real-money prediction market where users buy and sell shares in the outcomes of specific events. Mid-market prices on those shares give a continuously updated, money-backed probability estimate. That makes Polymarket one of the cleanest external sources for grading whether a FishDog panel's predictions track the real world.

How does the calibration comparison work?

An imported Polymarket event becomes a question put to a recruited panel. Each persona produces a probability estimate; the panel's distribution is compared to Polymarket's mid-market price at the time of the run. Calibration error and Brier-score components are computed and persisted.

Which markets can I import?

All Polymarket events are visible in the importer, filtered by category and resolution date. Suitability rules remove very low-liquidity or thinly-traded markets — those that wouldn't be useful calibration targets. The 18th April follow-up broadened the default selection and added gallery pagination for larger import sessions.

What are calibration runs useful for?

Three things: (1) research-validity work — demonstrating to a customer that FishDog panels predict realistically in their category; (2) panel selection — identifying which demographic compositions track markets best; (3) internal model benchmarking when refining the persona model.

Can I run repeated calibration runs over time?

Yes. Calibration results are persisted to a calibration store, so repeated runs against the same market build a track record. This is the right shape for showing that a panel's predictions are calibrated within a known error band over time, rather than just at one moment.

Release Tags

Data SourcesProduct Release

About the Author

Phillip Gales

Phillip Gales

Phillip is a serial tech entrepreneur that specializes in applying AI and machine learning solutions to antiquated and heavy industries. He has been a senior leader or founder at a number of successful startups.

Phillip holds an MBA from Harvard Business School, an MEng from the University of Cambridge, and is a Y-Combinator alum

LinkedIn

More Releases

Zeitgeist now visualises the mood

The Zeitgeist mood survey now ships with proper visualisations — a Mood Balance index, a Mood Compass scatter plot, and a Recovery vs Strain breakdown. Public landing-page chart endpoint included.

Read Release