Sports Provider
- Canonical MLB schedule, scores, participants, moneyline and totals markets, and latest prices.
- Primary live source for dashboard links and recommendation artifacts.
Baseball
How it Works
The Baseball app is a model-ops dashboard over canonical MLB facts, sportsbook prices, Retrosheet/FanGraphs feature artifacts, and a Python predictor. This page answers the concrete questions behind a pick: what data was loaded, which features matter, what the latest evaluation has taught us, and what proves the pipeline is healthy.
Primary factsSports Provider
Historical corpus30k games
Latest model AUC0.642
Serving gateQuality checks
Sources
What data is loaded?
Baseball uses live canonical facts for today and offline artifacts for historical context. The model should only be trusted when the source and artifact coverage are visible.
Retrosheet
- 2010-2022 event files, rosters, and game logs.
- 390 team-season event files and about 30,054 historical games in the local inventory.
Baseball Savant / Statcast
- 2015-2022 pitch-level caches.
- Rolling batter and pitcher aggregates such as exit velocity, hard-hit rate, barrel rate, xwOBA, fastball velocity, whiff rate, and chase rate.
FanGraphs and Chadwick IDs
- MLBAM-to-Retrosheet identity maps for probable starters and feature lookup.
- FanGraphs current-season artifacts are optional but quality-critical when available.
Odds feeds
- The Odds API for live h2h prices through Sports Provider.
- Kaggle MLB Vegas closing lines for historical evaluation; true openers are still a missing input.
S3 model artifacts
- `models/game_model.pkl`, `warehouse/games_final.parquet`, and `warehouse/player_stats.parquet` are required for hosted scoring.
- Park factors, weather, pitch-mix, game-script, and lineup-state files are tracked as optional artifacts.
Pipeline
From schedule to recommendation
Each run moves through the same steps, whether it is triggered by cron, a local smoke command, or a manual production refresh.
1. Canonical schedule and prices
Sports Provider is the preferred live feed. It normalizes MLB Stats schedule data and source-labeled sportsbook, Kalshi, and Polymarket prices into canonical games, participants, markets, and price snapshots. Projections prefer a fresh sportsbook quote, then use a fresh executable Kalshi or Polymarket ask when necessary; MLB Stats remains the schedule fallback.
2. Game selection and identity
The runner asks for a date window, keeps scheduled games with usable h2h prices, maps MLB teams to Retrosheet team codes, and carries probable pitcher MLBAM IDs into the Retrosheet/Chadwick/FanGraphs identity join.
3. Historical feature warehouse
The offline pipeline exports Retrosheet play-by-play into `plays.jsonl`, rolls it into `games_final.parquet` and `player_stats.parquet`, then enriches with lineups, starters, bullpen usage, Statcast, park, pitch-mix, and odds artifacts.
4. Cloud model sync
Hosted runs sync the required model bundle from S3 under `baseball/model-artifacts/current`: the pickled game model, `games_final.parquet`, and `player_stats.parquet`. Optional artifacts are counted separately.
5. Prediction and policy
The Python predictor returns home/away win probability, projected runs, calibration method, and defaulting flags. Moneylines use the calibrated win probability with the normal Kelly policy. Totals use a Poisson distribution centered on projected runs to emit over/under/push probabilities. Qualifying moneylines and totals persist capped hypothetical Kelly stakes in one paper ledger for outcome, ROI, and CLV tracking.
6. Serving artifact
The run writes a normalized projection artifact to `baseball/projections/latest.json` with moneyline and totals projections, tracking decisions, canonical event/market links, model metadata, feature snapshot refs, and a quality gate summary.
Features
Which features matter most?
The model is a Retrosheet LightGBM/Poisson blend. Its best signals come from team form, starter quality, lineup strength, bullpen fatigue, park context, and Statcast/pitch-type deltas.
Team form and run environment
Rolling runs scored/allowed, win rate, streaks, rest days, pythagorean luck, park run factor, elevation, roof/dome flags, and weather when available.
Starting pitchers
Starter ERA/WHIP/K9/BB9/HR9, recent form, batters faced, fastball velocity, whiff/chase rates, pitch-mix usage, and pitch-type value features.
Lineup strength
Lineup BA/OBP/SLG/K%, platoon aggregates, bench usage, and Statcast hitting deltas such as exit velocity, hard-hit rate, barrel rate, and xwOBA.
Bullpen fatigue
Three-day workload, seven-day run prevention, closer/setup/bulk/long-relief usage, appearance counts, and recent rest proxies.
Market and policy fields
Live American odds are converted to no-vig probabilities. They do not make a pick by themselves; they define edge, expected value per dollar, and stake size.
Data availability
Starter-ID coverage, optional artifact coverage, calibration fallback, clipped probabilities, and defaulted feature pressure are first-class serving fields.
Learnings
What have we learned?
The parser and feature pipeline are useful, but the betting question is still model-vs-market. These are the concrete results worth carrying forward.
The parser is the strongest part
The imported Retrosheet parser reports 100% parse success on 2.69M plays across 390 event files. Score validation covers 30,054 games at 97.0%; the documented misses are missing expected game-log entries, not score mismatches.
The model has signal, but not enough proof
The imported model-run manifest records AUC 0.642, Brier 0.2308, and market correlation 0.494. Older clean 2022 holdout notes show AUC about 0.599 and Brier about 0.241, so fresh evaluation matters after each feature-schema change.
Statcast and starter shape matter
Project notes repeatedly call out lineup exit-velocity gaps plus starter fastball velocity, whiff rate, and chase rate as top feature-importance signals once Statcast and pitch-type features are joined.
Pitch-type features helped in a frozen test
A 2022 overlapping-sample snapshot improved AUC from 0.598 to 0.615 and Brier from 0.2417 to 0.2383 after pitch-type matchup features were added.
Betting edge is still unproven
The first rolling-origin totals benchmark produced positive threshold ROI over 13,859 games from 2015–2021, but the model trailed the closing market on Brier score and log loss. True opener ingestion, strict nested early stopping, and live CLV remain required before real-money promotion.
Bullpen fatigue is not decorative
The pipeline now treats recent relief workload and role-based bullpen usage as material features, but the docs still flag better days-rest and workload validation as high priority.
Health
How do we know the pipeline is working?
A healthy run is traceable from source facts to feature artifacts, model output, policy decisions, and dashboard status.
Stage dependency checks
`pnpm --filter baseball retrosheet:pipeline:dry-run` verifies the offline Retrosheet stage graph before running expensive parser/model work.
Parser and model tests
The app exposes root pnpm wrappers for Rust fmt, clippy, workspace tests, Python tests, dry-run pipeline checks, and Sports Provider game-fact exports.
Live quality gate
Declared early projections warn instead of failing and remain information-only. During schedule gaps they may use critical form snapshots up to seven days old; missing or older critical inputs, unexplained starter gaps, thin artifact coverage, saturation, and collapsed forecasts still fail publication.
Model-vs-market gate
`pnpm --filter baseball model-vs-market:gate` compares offline model probabilities against Sports Provider closing no-vig price history and refuses to run when required artifacts are missing.
Manifest coverage
The current local feature/model manifests record source environment, feature schema, row count, source versions, model version, metrics, canonical game coverage, and canonical market coverage.
Serving does not recompute
The dashboard and recommendations API read normalized projection artifacts. Page load does not rebuild Python predictions, which keeps UI behavior traceable to one generated run.
Sanity
What keeps the output honest
These checks are designed to make weak data visible before it becomes a confident-looking recommendation.
Starter identity coverage
Probable pitcher MLBAM IDs are resolved to Retrosheet IDs before feature lookup. Before starters are named, the board publishes an explicitly labeled schedule-only projection with default starter features and disables staking; unexplained identity gaps still fail the quality gate.
Information-stage labels
Every game identifies whether it uses schedule-only inputs, probable starters with neutral lineup features, or both confirmed lineups. Projected rosters can still be shown for context, but the model does not treat them as confirmed inputs. Early stages are useful for watching the forecast evolve, but only the confirmed-lineup stage can produce a stake.
Optional artifact coverage
The run reports how many optional artifacts are available. Missing park, weather, pitch-mix, or lineup-state artifacts do not crash the run, but they reduce confidence and show up in the UI.
Calibration guardrails
If isotonic calibration returns a saturated 0% or 100% probability, the predictor falls back to Platt calibration or the raw model probability before serving a result.
Probability bounds
Served live probabilities are clipped to configurable bounds, defaulting to 2% and 98%, so recommendations never display artificial certainty.
Freshness status
The dashboard degrades when the projection artifact is older than 36 hours, when Sports Provider falls back, or when the model-quality gate fails.
Market discipline
The board is intentionally model-vs-market. A pick appears because model probability clears the no-vig market by policy, not because the model probability looks high by itself.
Artifact Inputs
Required
- models/game_model.pkl
- warehouse/games_final.parquet
- warehouse/player_stats.parquet
Optional but quality-critical
- warehouse/fangraphs/player_identity_map.parquet
- data/external/park_factors.csv
- data/external/park_orientation.csv
- data/features/park_weather.csv
- warehouse/pitch_mix_batter_features.parquet
- warehouse/pitch_mix_pitcher_features.parquet
- warehouse/team_game_script_features.parquet
- warehouse/team_lineup_state_features.parquet
Current Limits
- The next strategic test is still model-vs-closing-market log loss. MLB markets are liquid, so the market may beat the model.
- If Sports Provider has no scheduled games with h2h prices, the runner refuses to publish an empty fresh projection.
- Schedule-only and probable-starter forecasts run outside the confirmed-lineup model’s training stage, so treat them as directional context rather than calibrated betting probabilities. The board labels them and forces their stake to zero.
- Live lineups, weather, and some pregame features can still default when upstream artifacts are missing or stale. Pitch mix now refreshes daily and renders as unavailable rather than displaying neutral defaults as observed usage.
- True opener data is still missing, so historical CLV and ROI claims should be treated as research signals rather than production proof.
- Every positive moneyline or totals stake is a paper recommendation. The UI and Telegram track the same hypothetical bets and settlement results; no bet tickets are created and no sportsbook execution is authorized.