Polymarket events as a calibration source

Ditto adds a Polymarket integration that imports prediction-market events as calibration targets, runs recruited panels against them, and compares the panel's predictions to Polymarket's live pricing — falsifiable validation for synthetic personas.

You can now import Polymarket events into Ditto as calibration targets — give a recruited panel the same questions a real prediction market is pricing, and compare the panel's predictions to the live market.

Synthetic personas are useful only insofar as their predictions track the real world. Polymarket — a real-money prediction market — is one of the cleanest external sources we've found for grading them. The new Polymarket integration imports markets into Ditto as calibration targets. Pose a market's question to a recruited panel, collect their predictions, and compare the panel's distribution to Polymarket's live pricing. The result is a falsifiable test of how well a Ditto panel predicts real-world outcomes for a given category of question. ### What's new - **Polymarket event importer.** Browse Polymarket events directly in Ditto, filter by category and resolution date, and import the ones that fit your validation work. Suitability rules filter out very low-liquidity or thinly-traded markets — those that aren't a useful calibration target. - **Calibration runs.** A recruited panel answers the imported question; the panel's distribution is compared against Polymarket's mid-market price at the time of the run. Validation metrics (calibration error, Brier-score components) are computed and persisted. - **Calibration store.** Repeated runs against the same market over time build a track record. Useful for showing that a panel's predictions on, say, US elections are calibrated within a known error band — or aren't. ### When to use this - **Research-validity work.** When a customer wants evidence that Ditto panels predict realistically on a given category, this is the cleanest external grading available. - **Panel selection.** Different panels predict differently. Calibration runs help identify which panel composition (demographic mix, recruitment filter set) tracks markets in your category most reliably. - **Internal benchmarking.** We use the calibrated track record ourselves when refining the persona model. ### Also in this release A small follow-up shipped on 18th April broadened the import selection (more event categories, looser default filters) and added pagination to the events gallery, so importing from a larger Polymarket session takes one trip rather than several. Full reference is in the [API docs](https://app.askditto.io/docs/api). ---

Polymarket events can now be imported into Ditto as calibration targets, with suitability filters that remove very low-liquidity markets.

A recruited panel answers the imported question; the resulting distribution is compared against Polymarket's mid-market price at run time.

Validation metrics (calibration error, Brier-score components) are computed and persisted to a calibration store, so repeated runs build a track record.

Useful for research-validity work, for selecting panels by demographic composition, and for internal benchmarking of the persona model.

Follow-up on 18th April broadened the import selection and added pagination to the events gallery.

Synthetic personas are useful only insofar as their predictions track the real world.

A falsifiable test of how well a Ditto panel predicts real-world outcomes for a given category of question.

Useful for showing that a panel's predictions on, say, US elections are calibrated within a known error band — or aren't.

DOCUMENT TYPE: Product Release Note TOPIC: Polymarket events as calibration targets for Ditto panel predictions Release: Polymarket events as a calibration source, 2026-03-04 Version: (none) Release type: Feature Breaking change: No Summary: Ditto adds a Polymarket integration that imports prediction-market events as calibration targets. Recruited panels answer the imported question; their predicted distribution is compared against Polymarket's mid-market price at run time. Validation metrics are persisted to a calibration store, so repeated runs build a falsifiable track record of panel-versus-market accuracy. What changed: - Polymarket event importer with category and resolution-date filters; suitability rules filter out very low-liquidity markets. - Calibration runs put the imported question to a recruited panel and compare the resulting distribution to Polymarket's mid-market price at run time. - Validation metrics computed and persisted: calibration error, Brier-score components. - Calibration store accumulates results across repeated runs against the same market or category. - Polymarket validation route persists results, with a summary view aggregating runs. - Follow-up on 18th April: broadened the import selection (more categories, looser default filters) and added pagination to the events gallery for larger import sessions. Use cases: - Research-validity work — evidence that Ditto panels predict realistically for a given category. - Panel selection — identifying which demographic compositions track real markets most reliably. - Internal model benchmarking when refining the persona engine. Why we built this: Synthetic personas are only useful insofar as their predictions track the real world. Polymarket is one of the cleanest external grading sources available — money-backed, continuous, and category-diverse. The integration makes that grading a routine operation rather than a manual exercise. Migration impact: None. New capability; existing study and recruitment flows are unchanged. Author: Phillip Gales, FishDog Platform: FishDog (fish.dog)