Research question
The question asks which AI model demonstrates the best calibration for the "both teams to score" (BTTS) market. This matters because calibration—how well predicted probabilities match actual outcomes—directly affects expected value and risk assessment. Overconfident models may overstate win probabilities, leading to poor bankroll allocation, while underconfident models may miss profitable edges. In this analysis, we focus on the market_calibration candidate type, comparing four models across the BTTS market using Brier score, calibration error, and return on investment (ROI) metrics.
This question is particularly relevant for model evaluation because BTTS markets are popular but present unique calibration challenges due to the binary nature of the event and varying scoring patterns across leagues. By isolating this market, we can assess whether models exhibit systematic biases that might be obscured in aggregated win/draw/loss predictions.
Evaluation period
The evaluation covers bets settled between 2026-06-01 and 2026-06-30, a one-month window. This period includes pilot leagues such as Bundesliga, Champions League, LaLiga, Premier League, and Serie A (as listed in 2026-06-01 through 2026-06-30). Only pre-kickoff snapshots of each model's predictions are included; no in-play adjustments or post-hoc modifications are considered. The window excludes any picks from leagues outside the pilot set and bets that were voided or unsettled.
The relatively short evaluation window limits the generalizability of results to other periods or leagues. Seasonal effects, fixture congestion, and different competition dynamics could alter model performance. As a research snapshot, this provides preliminary insights but should not be extrapolated broadly.
Sample size
The analysis is supported by a total of 69 settled picks across all models, labeled as an "Early sample". The dataset is flagged as not insufficient (false), meaning minimum sample thresholds are met for public content. However, 69 picks shared across four models means individual model samples vary widely: deepseek-v4-pro has 64 picks, while grok-build-0.1 has only 13 picks. Such small per-model samples introduce high variance and limit the statistical power of comparisons.
To strengthen confidence in these findings, larger sample sizes—ideally several hundred picks per model—would be needed. The current data is particularly thin for grok-build-0.1 and gpt-5-mini, making their metrics more susceptible to noise.
Model comparison at a glance
Below is the ranking of all four models on Brier score, ROI, and win rate for the BTTS market.
| Model | Settled picks | Brier score | ROI % | Win rate % | |-------|--------------|-------------|-------|------------| | grok-build-0.1 | 13 | 0.2611 | -35.6 | +38.5 | | deepseek-v4-pro | 64 | 0.3089 | +4.3 | +53.1 | | deepseek-v4-flash | 51 | 0.3115 | +12.9 | +54.9 | | gpt-5-mini | 69 | 0.3149 | -9.6 | +50.7 |
The spread in Brier scores is relatively narrow (from 0.2611 to 0.3149), but ROI differences are stark: deepseek-v4-flash shows the highest ROI at +12.9%, while grok-build-0.1 posts a negative -35.6%. It is notable that grok-build-0.1 achieves the best Brier score yet worst ROI, suggesting that its probability predictions are accurate but its betting strategy—or the odds available—may not convert into profit. Deepseek-v4-pro, despite the weakest Brier (0.3089), yields positive ROI (+4.3%), indicating that calibration alone does not determine profitability.
Calibration and probability accuracy
Calibration error, measured by model-level metrics, reveals a consistent overconfidence bias across all models. Each model's calibration_error is negative: -17.7 percentage points for deepseek-v4-flash, -30.4 for deepseek-v4-pro, -13.1 for gpt-5-mini, and -27.6 for grok-build-0.1. This indicates that models systematically overestimate the probability of both teams scoring; actual BTTS outcomes occur less frequently than predicted.
The aggregated calibration data across 69 picks confirms that miscalibration persists across probability ranges. Among the models, gpt-5-mini has the smallest calibration error (-13.1%), while deepseek-v4-pro shows the largest overconfidence (-30.4%). This pattern suggests that even models with better overall accuracy still tend to overestimate BTTS frequency, which has direct implications for expected value calculations.
Risk, drawdown, and bankroll behavior
Equity trajectories vary dramatically. deepseek-v4-flash achieved a peak equity of 1094.5 units but experienced a maximum drawdown of 391.96 units (+35.8% of peak). In contrast, deepseek-v4-pro reached a higher peak (1123.48 units) but suffered a much deeper drawdown of 923.09 units (+82.2%), illustrating the volatility associated with higher-risk picks. grok-build-0.1, despite its small sample, shows current equity of 1273.4 units versus a peak of 1303.4 units, with a drawdown of 560 units (+51.4%). gpt-5-mini has a more modest drawdown of 373.93 units but also lower peak equity.
The risk-return trade-off is pronounced: models with high drawdown (deepseek-v4-pro, grok-build-0.1) do not consistently deliver superior returns. deepseek-v4-flash offers a more balanced profile with reasonable ROI and moderate drawdown, while gpt-5-mini exhibits the most conservative equity behavior but negative ROI.
Model-by-model notes
grok-build-0.1 leads on Brier score (0.2611) but suffers from a negative ROI of -35.6% over only 13 picks. Its win rate (+38.5%) is the lowest among models, and its large calibration error (-27.6%) indicates significant overestimation of BTTS probability. The small sample makes these numbers highly provisional.
deepseek-v4-pro has the poorest Brier (0.3089) but a positive ROI (+4.3%), driven by a win rate of +53.1% on 64 picks. Its calibration error (-30.4%) is the largest, suggesting that even with inaccurate probabilities, favorable odds or selective betting allowed profitability. The maximum drawdown of 923.09 units (+82.2%) highlights substantial risk.
deepseek-v4-flash achieves the highest ROI (+12.9%) with a win rate of +54.9% on 51 picks. Its Brier (0.3115) is middle-of-the-pack, and its calibration error (-17.7%) is moderate. The drawdown of 391.96 units (+35.8%) is the most contained, indicating a steadier equity curve.
gpt-5-mini has the worst Brier (0.3149) and slightly negative ROI (-9.6%). Its win rate is +50.7% on 69 picks, with the smallest calibration error (-13.1%) among the set. Its drawdown (373.93 units, +36.8%) is moderate, but the negative ROI suggests that calibration improvements did not translate into profits.
What this means for readers
This benchmark reveals that no single model dominates both calibration and profitability for BTTS. Deepseek-v4-flash offers the best risk-adjusted returns in this window, but its calibration error suggests its probability outputs should be adjusted. The results do not prove that any model will remain top-performing over time or across leagues. Readers should consult model profiles (e.g., deepseek-v4-flash, grok-build-0.1) and the methodology page to understand data sources and evaluation criteria. For further context on calibration, see our prior research on model calibration and the profitability paradox. No betting advice is offered; these are descriptive findings from a limited sample.
Methodology
Alpha Bench freezes pre-kickoff model snapshots for all picks, computes Brier score, log loss, and ROI from settled bets. Calibration error is calculated as the difference between expected and actual win rates across probability buckets. Drawdown is measured from peak equity to trough. All metrics use the 2.1 methodology. The data cutoff is 2026-06-29T16:00:02Z. For further details, see our methodology page and metric definitions.
Limitations
The analysis is constrained by a small overall sample (69 picks), particularly for two models with under 30 picks. Results are specific to the BTTS market within pilot European leagues; they may not transfer to other markets or competitions. The evaluation period covers only one month, missing seasonal variation. Additionally, closing line value (CLV) data was not available for these models, limiting the ability to separate skill from luck. The findings should be viewed as preliminary—larger, multi-period samples are needed for robust conclusions.
Reproducible data references
This article uses the following Alpha Bench data sources: 2026-06-29-vector-001, data cutoff 2026-06-29T16:00:02Z. Verified context entries: vector-dataset-status, vector-model-comparison, vector-calibration, vector-risk-metrics, vector-market-performance-btts, vector-long-term-patterns, vector-version-changes. All model metrics are sourced from the models[] array in the package, including fields such as brier, roi_percent, win_rate, settled_picks, calibration_error, max_drawdown, max_drawdown_pct, peak_equity, current_equity. For per-model daily metrics, see historical_metrics (e.g., model_daily_metrics:grok-build-0-1:2026-06-29). Readers can verify claims on model profiles linked above.