# Polymarket events as a calibration source Canonical HTML: https://fish.dog/product-releases/polymarket-events-as-a-calibration-source JSON Feed: https://fish.dog/product-releases/feed.json RSS Feed: https://fish.dog/product-releases/rss.xml Published: 4 March 2026 Updated: 4 May 2026 Release Type: Feature Breaking Change: No Author: Phillip Gales ## Primary Claim Ditto adds a Polymarket integration that imports prediction-market events as calibration targets, runs recruited panels against them, and compares the panel's predictions to Polymarket's live pricing — falsifiable validation for synthetic personas. ## Summary You can now import Polymarket events into Ditto as calibration targets — give a recruited panel the same questions a real prediction market is pricing, and compare the panel's predictions to the live market. ## LLM Summary DOCUMENT TYPE: Product Release Note TOPIC: Polymarket events as calibration targets for Ditto panel predictions Release: Polymarket events as a calibration source, 2026-03-04 Version: (none) Release type: Feature Breaking change: No Summary: Ditto adds a Polymarket integration that imports prediction-market events as calibration targets. Recruited panels answer the imported question; their predicted distribution is compared against Polymarket's mid-market price at run time. Validation metrics are persisted to a calibration store, so repeated runs build a falsifiable track record of panel-versus-market accuracy. What changed: - Polymarket event importer with category and resolution-date filters; suitability rules filter out very low-liquidity markets. - Calibration runs put the imported question to a recruited panel and compare the resulting distribution to Polymarket's mid-market price at run time. - Validation metrics computed and persisted: calibration error, Brier-score components. - Calibration store accumulates results across repeated runs against the same market or category. - Polymarket validation route persists results, with a summary view aggregating runs. - Follow-up on 18th April: broadened the import selection (more categories, looser default filters) and added pagination to the events gallery for larger import sessions. Use cases: - Research-validity work — evidence that Ditto panels predict realistically for a given category. - Panel selection — identifying which demographic compositions track real markets most reliably. - Internal model benchmarking when refining the persona engine. Why we built this: Synthetic personas are only useful insofar as their predictions track the real world. Polymarket is one of the cleanest external grading sources available — money-backed, continuous, and category-diverse. The integration makes that grading a routine operation rather than a manual exercise. Migration impact: None. New capability; existing study and recruitment flows are unchanged. Author: Phillip Gales, FishDog Platform: FishDog (fish.dog) ## Key Takeaways - Polymarket events can now be imported into Ditto as calibration targets, with suitability filters that remove very low-liquidity markets. - A recruited panel answers the imported question; the resulting distribution is compared against Polymarket's mid-market price at run time. - Validation metrics (calibration error, Brier-score components) are computed and persisted to a calibration store, so repeated runs build a track record. - Useful for research-validity work, for selecting panels by demographic composition, and for internal benchmarking of the persona model. - Follow-up on 18th April broadened the import selection and added pagination to the events gallery. ## Full Release Synthetic personas are useful only insofar as their predictions track the real world. Polymarket — a real-money prediction market — is one of the cleanest external sources we've found for grading them. The new Polymarket integration imports markets into Ditto as calibration targets. Pose a market's question to a recruited panel, collect their predictions, and compare the panel's distribution to Polymarket's live pricing. The result is a falsifiable test of how well a Ditto panel predicts real-world outcomes for a given category of question. ### What's new - **Polymarket event importer.** Browse Polymarket events directly in Ditto, filter by category and resolution date, and import the ones that fit your validation work. Suitability rules filter out very low-liquidity or thinly-traded markets — those that aren't a useful calibration target. - **Calibration runs.** A recruited panel answers the imported question; the panel's distribution is compared against Polymarket's mid-market price at the time of the run. Validation metrics (calibration error, Brier-score components) are computed and persisted. - **Calibration store.** Repeated runs against the same market over time build a track record. Useful for showing that a panel's predictions on, say, US elections are calibrated within a known error band — or aren't. ### When to use this - **Research-validity work.** When a customer wants evidence that Ditto panels predict realistically on a given category, this is the cleanest external grading available. - **Panel selection.** Different panels predict differently. Calibration runs help identify which panel composition (demographic mix, recruitment filter set) tracks markets in your category most reliably. - **Internal benchmarking.** We use the calibrated track record ourselves when refining the persona model. ### Also in this release A small follow-up shipped on 18th April broadened the import selection (more event categories, looser default filters) and added pagination to the events gallery, so importing from a larger Polymarket session takes one trip rather than several. Full reference is in the [API docs](https://app.askditto.io/docs/api). --- ## Quotable Insights > Synthetic personas are useful only insofar as their predictions track the real world. > A falsifiable test of how well a Ditto panel predicts real-world outcomes for a given category of question. > Useful for showing that a panel's predictions on, say, US elections are calibrated within a known error band — or aren't. ## FAQ ### What is Polymarket and why use it? Polymarket is a real-money prediction market where users buy and sell shares in the outcomes of specific events. Mid-market prices on those shares give a continuously updated, money-backed probability estimate. That makes Polymarket one of the cleanest external sources for grading whether a Ditto panel's predictions track the real world. ### How does the calibration comparison work? An imported Polymarket event becomes a question put to a recruited panel. Each persona produces a probability estimate; the panel's distribution is compared to Polymarket's mid-market price at the time of the run. Calibration error and Brier-score components are computed and persisted. ### Which markets can I import? All Polymarket events are visible in the importer, filtered by category and resolution date. Suitability rules remove very low-liquidity or thinly-traded markets — those that wouldn't be useful calibration targets. The 18th April follow-up broadened the default selection and added gallery pagination for larger import sessions. ### What are calibration runs useful for? Three things: (1) research-validity work — demonstrating to a customer that Ditto panels predict realistically in their category; (2) panel selection — identifying which demographic compositions track markets best; (3) internal model benchmarking when refining the persona model. ### Can I run repeated calibration runs over time? Yes. Calibration results are persisted to a calibration store, so repeated runs against the same market build a track record. This is the right shape for showing that a panel's predictions are calibrated within a known error band over time, rather than just at one moment.