Research question
Which AI model is best calibrated for Over 2.5 goals? This analysis compares model behavior and benchmark performance for Over 2.5 goals markets across five pilot leagues, focusing on candidate type market_calibration.
Evaluation period
The frozen evaluation window spans from 2026-06-01 through 2026-06-30.
Sample size
The analysis is based on 151 settled picks (Early sample).
Findings
Among the four models evaluated, grok-build-0.1 achieved the lowest Brier score (0.2652) and highest ROI (71.11%) across 27 settled picks. deepseek-v4-flash ranked second with a Brier of 0.3048 and ROI of -7.87%. deepseek-v4-pro and gpt-5-mini followed with Brier scores of 0.3141 and 0.3266 respectively. The Over 2.5 goals calibration errors, derived from vector-market-performance-over_under, show that all models overestimated goal probabilities, with grok-build-0.1 exhibiting the smallest average deviation (-19.03%). Max drawdown varied, with grok-build-0.1 peaking at 560 units and deepseek-v4-pro experiencing the deepest drawdown (373.93 units).
Methodology
Alpha Bench freezes pre-kickoff snapshots, computes Brier and log loss for probabilistic predictions, and tracks ROI and drawdown using a standard staking approach. Data is aggregated from five pilot leagues (Bundesliga, Champions League, LaLiga, Premier League, Serie A) with a focus on Over 2.5 goals markets. Methodology version 2.1.
Limitations
Sample size is 151 (Early sample), which is modest for robust calibration analysis. The data is limited to Over 2.5 goals markets across five pilot leagues. Insufficient data for deep calibration decile analysis per model. The analysis does not account for time-varying model performance or out-of-sample behavior.
Reproducible data references
Data sources include vector-dataset-status, vector-model-comparison, vector-long-term-patterns, vector-risk-metrics, vector-market-performance-over_under, vector-calibration. Readers can verify claims on model profiles and benchmark pages. Article run ID: 2026-06-28-vector-002. Data cutoff at 2026-06-28T18:20:16Z.