Calibrated Prediction Across 16 Sports, and the Honest Wall of the Closing Line

Most sports-betting projects fail not because the model is bad but because they attacked a sport whose single-game outcome is mostly variance, or mistook the irreducible noise floor for a modeling failure and overfit into it. sports-AI starts from the opposite discipline: before spending a GPU-hour on a sport, score how learnable it is. The framework decomposes each outcome into a learnable component — true strength, matchup, situation — plus irreducible noise (luck, bounces, refs, single-game variance), and estimates the noise floor: the best log-loss an omniscient model could reach. A sport where the favorite wins 80–95% of the time (heavy tennis or boxing mismatches) is highly learnable; one where the best team wins a single game only ~60% of the time (MLB, hockey) is fighting variance no matter how good the model.

Crucially, learnability is not profitability. A high-noise sport can still be profitable if the market prices it even worse than the model does; a perfectly learnable sport is worthless to trade if the market already prices it to the floor. So each sport is scored on six axes — single-game predictability, data granularity, roster stability, structural signal strength, market inefficiency, and sample size — and the two that dominate the composite are edge headroom and predictability. The honest questions a sport spec must answer are not 'can we predict it?' but 'what is the gap between the market and the noise floor, and where is the market softest?' That is where any money actually is.

Every model is trained the only way that does not lie: expanding-window walk-forward with strictly point-in-time features, so each prediction sees only rows before its fold's training cutoff, and per-fold isotonic calibration fit on the training tail so the model never calibrates on its own future. The baseline layer is Elo/Glicko team-strength feeding a gradient-boosted classifier. On top sits a 'sharp' granular layer that ingests what professional shops actually use — Statcast pitcher-matchup blends for MLB, opponent-adjusted efficiency and player-availability impact for basketball, surface-specific Elo plus serve/return and fatigue for tennis, QB-level EPA and CPOE for the NFL, and an xG-driven Dixon–Coles model for soccer.

The sharp layer works: every one of nine sharp models beats its game-level baseline out-of-sample. UFC improves from 0.6841 to 0.6654 log-loss; the NFL QB-EPA model closes roughly 18% of the gap to the market; MLB's Statcast blend cuts calibration error 63%; soccer's xG Dixon–Coles lands at the academic frontier (RPS 0.202 versus market 0.194). Tennis reports the largest headline jump (0.6146→0.5607 log-loss, AUC 0.722→0.783), but the leakage audit is explicit that this one is not clean: the claimed sharp-over-baseline delta (+0.006 AUC) is smaller than a 0.036 AUC within-tournament same-date leak, so the audit judged the tennis sharp gain not robust — it is cited here as flagged, not as a headline. Beyond these, a model-family suite was implemented and measured — 170 families across 16 sports, 82 of them historically scoreable — with a leak-free logistic stack per sport. The families cluster tightly because they re-express one team-strength signal, so stacking beats the best single family only marginally.

Across 16 sports the leak-free models span 0.572 AUC (golf, audited leak-free) to 0.866 (boxing), with tennis at 0.719, the college sports near 0.75–0.76, and the high-noise sports — MLB (0.574), UFC (0.585), esports (0.582) — sitting near the coin-flip floor by nature, not by failure. What is deploy-grade is calibration: expected calibration error is at or below 0.03 on almost every sport (boxing 0.0073, MLB 0.0281, NBA 0.0264), and calibration, not accuracy, is what drives staking and closing-line value.

The confident subset is the honest home of high accuracy. Restricting to games where the model is most sure (|p−0.5| ≥ 0.40), tennis is 96.4% accurate on 33% of all matches and boxing 94.5% on 31% — genuinely tradeable slices of extreme mismatches. College basketball and football cross 90% on 21–25% of games. High-variance sports keep a >90%-ish edge too, but coverage collapses below 3% because the model is correctly, rarely that sure. No single-game team sport reaches 0.90 full-sample AUC, and that is the point: a pre-game model claiming >0.90 on single games is measuring leakage, not skill.

Then the wall. For the nine sports carrying a closing price, the de-vigged liquid close's own out-of-sample AUC is the empirical ceiling, and no pre-game model clears it — calibrated Elo sits 0.015 to 0.11 AUC below the close. Betting the model's pick flat into the vig-inclusive close loses $254,583 on $8,692,100 staked (−2.9%); only college football shows a positive point estimate (+4.1%), and its Monte-Carlo confidence interval straddles zero. This is market efficiency, not a bug: the close already aggregates everyone's sharp models, ours included.

The program treats its own results as guilty until cleared. An adversarial leakage audit reran every claimed out-of-sample gain against real code and real data. It caught golf's committed 0.9824 AUC as a coupled within-tournament leak: the pairwise backbone expands each stroke-play event into head-to-head matchups whose winner is decided by final finish, and per-row Elo/form updates let a golfer's later same-tournament rows read state that already encodes that tournament's outcome — 99.5% of golf rows peeked at a same-date sibling. Deferring both updates until the date advances collapses the AUC to 0.592, with a shuffled-label placebo at 0.5006 confirming no residual signal. The same class of bug inflated the tennis sharp model, whose merged-frame headline of 0.876 corrects to a leak-free 0.723 (on its own walk-forward frame the leaky 0.767 falls to 0.731, a 0.036 AUC within-tournament effect). Both were re-reported at their leak-free numbers.

The audit also confirmed what was real: tennis pre-game 0.865 is invariant to the same-date fix (a legitimate skill-gap signal from a mixed ATP/WTA field), boxing 0.866 is genuine, and the in-game state model's >0.90 late-game AUC survives GroupKFold game-integrity checks — the high numbers come from genuinely near-decided states, not leakage. Most decisively, an edge-hunt built the strongest possible advocate — ensemble plus residual plus soft segments plus fractional Kelly plus an EV filter, 504 configurations — and found that in-sample ROI predicts out-of-sample ROI at r = −0.015 (p = 0.77). Selecting a strategy because it was profitable in-sample carries zero information about its future profit. This is the single most important discipline in the repo, and the reason the backtests are designed to disprove edges rather than showcase them.

If pre-game is a wall, the natural next question is whether betting during the game helps. It does not, against a sharp line. The trained in-game win-probability model posts 0.5203 log-loss against the live Vegas line's 0.4825 — Vegas is the sharper live read, so betting the model's divergence from it is betting on the model's own error. Redoing the backtest the literature's way (market-calibrated WP, an EV filter, and fractional Kelly instead of a flat stake that a late-game odds-to-1.0 dynamic guarantees loses) removes the staking artifact but keeps the verdict: still −EV against the sharp line, now cleanly because the model is worse, not because the stake size was broken. Against a lagging soft book the strategy can look profitable — but a dumb price-copier with no model captures the same or more, so it is latency arbitrage, not model alpha.

The one mechanism that even shows a positive number is making, not taking: quote a bid and ask around calibrated fair value, collect the spread, and use the model as an adverse-selection filter. In an isolated simulation over ~168,926 game-orientations across 7 sports, the base case earns +1.17¢/contract net (3.00¢ gross spread minus 1.83¢ adverse selection), with break-even toxicity moving from 0.24 to 0.36 once retail's documented YES-overbet is included, and the model's low ECE (2.90pp) protects — not manufactures — the spread. But that number does not survive integration: the repo's combined 'three sleeves as one book' portfolio, measured on real data, finds none of the sleeves is +EV at realistic execution. The maker sleeve is −0.16¢/contract once a realistic 30% queue-win-rate is modeled — the earlier +0.37¢ was a perfect-queue artifact and was killed — the predictive family stack loses to the de-vigged close on every book (cross-sport ROI −14.9%), and the in-play WP trails the venue's own price with a CI including zero. Making is plausible in the isolated sim but negative under realistic queue mechanics, and the whole question stays gated on live order flow.

The program's honesty is its product. It establishes, with reproducible walk-forward numbers, adversarial audits, and cost-inclusive backtests, exactly where a sports-betting edge is and is not: it is not in out-predicting a liquid closing price, which is proven impossible on the data available; it is at best conditionally in making (harvesting a behavioral retail surplus) and in genuinely soft or lagging venues the program cannot yet price — and even making goes negative once a realistic queue-win-rate is modeled. The combined 'many models as one book' portfolio confirms the structure without inventing the edge — cross-sport return correlation of ~0.08 cuts book volatility 22%, real Simons-style risk management that cannot manufacture an edge that is not there.

The red-team assessment of the model-family thesis is unusually candid about its own limits. Diversification reduces variance, not edge, so a portfolio of individually-zero-edge signals is a precise estimate of zero that loses the spread. Families keyed off the same rating are far more correlated than they look — at a plausible ρ̄ = 0.7, eight families behave like fewer than 1.5 independent bets, and tail correlation goes toward 1 exactly when the shared rating breaks. Limited independent history means even a real 2% soft-market edge is unprovable on achievable sample sizes. The gating blocker is not modeling but the forward closing-line-value and live-execution pipeline: until families are validated on weeks of live Kalshi/Polymarket order flow, queue-win-rate, and settlement, the correct capital allocation to forecasting families is zero, and the honest expected ceiling is a small making book plus a few un-provable niches — not a scalable Renaissance portfolio. Stating that plainly, rather than shipping an in-sample cherry, is the contribution.