Baseball · Data

Data lineage

Architecture reference for how sources, jobs, artifacts, and model heads connect. This is generated from the feature registry and cron configuration; it is not a live health dashboard.

Architecture reference

Source-to-model paths

Use Current healthfor today's coverage, faults, and raw values. This map answers where data comes from and which job moves it.

Upstream sources
11
Owned jobs
19
Automated jobs
16
Feature families covered
31
Data lineage view

This inventory is generated from the feature registry and checked against configured Baseball and Sports Provider schedules. A new family, source reference, or changed cron cannot silently disappear from this view.

Live serving

Live game, roster and venue context

Canonical schedule facts become stage-aware predictor rows; projected lineups remain context-only until both lineups are confirmed.

Sources consulted
  • MLB Stats APISchedules, teams, venues, probable pitchers, lineups, scores, current player pitching and staff context.
  • FanGraphs scoreboardPregame lineup enrichment and FanGraphs game probabilities attached to canonical MLB events.
  • Sports Provider canonical storeCanonical MLB games, participants, lineups, venue conditions, markets and immutable price snapshots normalized from upstream providers.
Triggered by
  • MLB Stats schedule and scoresEvery 15 minutes · Sports Provider/api/v1/admin/cron/baseball-mlb-stats
  • FanGraphs MLB enrichmentFour times daily at 14:15, 17:15, 20:15, and 23:15 UTC · Sports Provider/api/v1/admin/cron/baseball-fangraphs
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Normalize games, identities, probable pitchers, lineups and venues
  2. Read the Pacific-day slate from Sports Provider
  3. Derive schedule-only / probable-starter / confirmed-lineup stage
  4. Build symmetric game-context columns and enforce input contracts
2 materialized artifacts
  • Sports Provider canonical Game and Participant records
  • Baseball projection input rows
Feeds
  • Moneyline head
  • Totals head
6 feature families · 8 ML / 9 totals columns
FamilyArtifactBuild → serveTrainingInferenceMLTotals
Lineup Rateswarehouse/player_stats.parquetlineup_features.pypredict_game.pyRequired if enabledRequired from confirmed lineup04
Lineup Statewarehouse/games_final.parquetadd_pitcher_lineup_features.pypredict_game.pyRequired if enabledNot selected00
Playoff Leveragein-processplayoff_leverage_features.pypredict_game.pyRequired if enabledRequired from schedule33
Starter Handednessdata/external/biofile0.csvteam_handedness_features.pymaterialize_starter_handedness.pyRequired if enabledRequired from probable starters21
Travel Circadianin-processcontext-experiments/compact_signal_features.pypredict_game.pyRequired if enabledRequired from schedule20
Venue Splitin-processfeature_symmetry.build_venue_split_featurespredict_game.pyRequired if enabledRequired from schedule11
Live serving

Live market prices

Immutable moneyline and total observations are normalized before the projection run derives decision-time prices, movement, EV and settlement keys.

Sources consulted
  • The Odds APISportsbook moneyline, run-line and total price snapshots.
  • SportsGameOddsSportsbook MLB moneyline, run-line and total price snapshots.
  • PolymarketFull-game MLB moneyline and line-aware total order books.
  • KalshiMLB moneyline and full-game total-threshold markets.
  • Sports Provider canonical storeCanonical MLB games, participants, lineups, venue conditions, markets and immutable price snapshots normalized from upstream providers.
Triggered by
  • The Odds API MLB oddsHourly at minute 25 · Sports Provider/api/v1/admin/cron/baseball-odds
  • SportsGameOdds MLB oddsHourly at minute 30 · Sports Provider/api/v1/admin/cron/baseball-sportsgameodds
  • Baseball Polymarket pricesEvery 15 minutes · Sports Provider/api/v1/admin/cron/baseball-polymarket
  • Baseball Kalshi pricesEvery 15 minutes, staggered from tennis Kalshi polling · Sports Provider/api/v1/admin/cron/baseball-kalshi
  • Baseball closing linesEvery 15 minutes · Sports Provider/api/v1/admin/cron/baseball-closing
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Match provider events and selections to canonical MLB games
  2. Append immutable PriceSnapshot observations
  3. Derive open, decision and close checkpoints
  4. Select line-aware totals surfaces and calculate decision EV
2 materialized artifacts
  • Sports Provider Market and PriceSnapshot records
  • Projection and BetDecision artifacts
Feeds
  • Moneyline head
  • Totals head
  • Decision policy

Operational input; not owned by a feature family.

Live serving

Current MLB operational snapshots

One bounded daily job refreshes identity, completed-game, starter, bullpen, manager and umpire inputs, validates them, then reprojects.

Sources consulted
  • MLB Stats APISchedules, teams, venues, probable pitchers, lineups, scores, current player pitching and staff context.
  • Sports Provider canonical storeCanonical MLB games, participants, lineups, venue conditions, markets and immutable price snapshots normalized from upstream providers.
  • Baseball model artifact storeValidated, versioned feature snapshots, model configuration, estimator and projection history consumed by the hosted predictor.
Triggered by
  • Refresh critical feature snapshotsDaily April–October at 11:30 UTC · Baseball/api/admin/cron/refresh-critical-features
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Refresh upcoming player identity mappings
  2. Build current completed-game, starter, bullpen, manager and umpire snapshots
  3. Apply row-grain, coverage, null-rate, range and freshness contracts
  4. Publish validated artifacts and invoke the predictor
5 materialized artifacts
  • warehouse/current_game_facts.json
  • warehouse/mlb_current_pitcher_stats.parquet
  • warehouse/mlb_current_bullpen.parquet
  • warehouse/mlb_current_managers.parquet
  • warehouse/mlb_umpire_assignments.parquet
Feeds
  • Moneyline head
  • Totals head
10 feature families · 34 ML / 29 totals columns
FamilyArtifactBuild → serveTrainingInferenceMLTotals
Player Recent Workloadwarehouse/pitching-observationsscripts/lib/player_workload_features.pyplayer_workload_features.player_workload_snapshotRequired if enabledNot selected00
Official Pitching Rateswarehouse/mlb_current_pitcher_stats.parquetscripts/lib/pitching_rate_features.pypitching_rate_features.official_serving_ratesRequired if enabledNot selected00
Bullpen Fatiguewarehouse/games_final.parquetadd_bullpen_features.pyfetch_mlb_current_bullpen.pyRequired if enabledRequired from schedule62
Bullpen Roleswarehouse/games_final.parquetadd_bullpen_features.pyfetch_mlb_current_bullpen.pyRequired if enabledRequired from schedule83
Official Bullpenwarehouse/mlb_current_bullpen_stats.parquetscripts/features/build_official_bullpen_matrix.pyofficial_bullpen_features.official_bullpen_snapshotRequired if enabledNot selected00
Manager Recordwarehouse/manager_stats.parquetbuild_manager_stats.pycurrent_manager_features.pyRequired if enabledRequired from schedule32
Manager Scriptwarehouse/games_final.parquetadd_bullpen_features.pyfetch_mlb_current_bullpen.pyRequired if enabledNot selected00
Starter Rollingwarehouse/player_stats.parquetbuild_player_stats.pypredict_game.pyRequired if enabledRequired from probable starters78
Team Recordwarehouse/games_final.parquettrain_game_model.build_team_statspredict_game.pyRequired if enabledRequired from schedule1011
Umpire Formoutputs/factors/umpire_year_counts.parquetbuild_park_umpire_factors.pyfetch_mlb_umpire_assignments.pyRequired if enabledRequired from schedule03
Training + serving

MLB injured-list availability

Official MLB IL transactions become daily team availability state; retroactive effective dates are retained for audit but never backdated into model information.

Sources consulted
  • MLB Stats APISchedules, teams, venues, probable pitchers, lineups, scores, current player pitching and staff context.
  • Baseball model artifact storeValidated, versioned feature snapshots, model configuration, estimator and projection history consumed by the hosted predictor.
Triggered by
  • Refresh MLB injured-list stateDaily April–October at 11:00 UTC · GitHub Actions.github/workflows/baseball-injury-refresh.yml
  • Offline model buildExplicit reviewed model build · Baseballpipeline.py / context-experiments
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Fetch and semantically deduplicate IL placements, activations and transfers
  2. Apply changes only after their public transaction date
  3. Materialize one known-state row per team and calendar date
  4. Attach prior-season PA/IP burden and preserve missing state as unknown
2 materialized artifacts
  • warehouse/mlb_injury_transactions.parquet
  • warehouse/team_injury_state_features.parquet
Feeds
  • Moneyline head
  • Totals head
1 feature families · 0 ML / 0 totals columns
FamilyArtifactBuild → serveTrainingInferenceMLTotals
Injury Availabilitywarehouse/team_injury_state_features.parquetfetch_mlb_injuries.pypredict_game.pyRequired if enabledNot selected00
Live serving

FanGraphs current-season form

The prior completed date's FanGraphs batting/pitching and MLB Stats fielding facts are identity-joined, merged as a new as-of partition and promoted only after content validation.

Sources consulted
  • FanGraphs leaderboardsPrior-season and season-to-date batting/pitching form plus projection snapshots; fielding date windows come from MLB Stats.
  • MLB Stats APISchedules, teams, venues, probable pitchers, lineups, scores, current player pitching and staff context.
  • Baseball model artifact storeValidated, versioned feature snapshots, model configuration, estimator and projection history consumed by the hosted predictor.
Triggered by
  • Refresh FanGraphs as-of formDaily April–October at 13:30 UTC · Baseball/api/admin/cron/refresh-fangraphs-asof
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Fetch date-windowed FanGraphs batting/pitching and MLB Stats fielding through the target date
  2. Normalize and join FanGraphs/MLBAM IDs to Retrosheet IDs
  3. Convert position-relative error prevention to the versioned defensive run proxy
  4. Merge one player/date partition into the leakage-safe history
  5. Validate, advance current last, and trigger reprojection
1 materialized artifacts
  • warehouse/fangraphs_player_form_asof.parquet
Feeds
  • Moneyline head
  • Totals head
2 feature families · 8 ML / 8 totals columns
FamilyArtifactBuild → serveTrainingInferenceMLTotals
Fangraphs Current Formwarehouse/fangraphs_player_form_asof.parquetbuild_fangraphs_asof_form.pyfetch_fangraphs_current.pyRequired if enabledRequired from confirmed lineup55
Lineup Warwarehouse/games_with_war.parquetwar/build_war_features.pypredict_game.pyRequired if enabledRequired from confirmed lineup33
Training + serving

FanGraphs projections and prior form

Scheduled leader and projection captures preserve dated player forecasts while full-season leaders feed prior-form features at model-build time.

Sources consulted
  • FanGraphs leaderboardsPrior-season and season-to-date batting/pitching form plus projection snapshots; fielding date windows come from MLB Stats.
  • Baseball model artifact storeValidated, versioned feature snapshots, model configuration, estimator and projection history consumed by the hosted predictor.
Triggered by
  • Refresh FanGraphs projections and leadersDaily April–October at 13:30 UTC · GitHub Actions.github/workflows/baseball-fangraphs-asof-refresh.yml
  • Offline model buildExplicit reviewed model build · Baseballpipeline.py / context-experiments
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Capture Steamer/Depth Charts projections and season leaderboards
  2. Normalize identities and retain dated snapshots
  3. Build prior-season and optional projection feature indexes
  4. Publish reviewed artifacts for training and serving
3 materialized artifacts
  • warehouse/fangraphs_player_form.parquet
  • warehouse/fangraphs/projections_asof.parquet
  • warehouse/serving/fangraphs_projections.parquet
Feeds
  • Moneyline head
  • Totals head
2 feature families · 10 ML / 10 totals columns
FamilyArtifactBuild → serveTrainingInferenceMLTotals
Fangraphs Prior Formwarehouse/fangraphs_player_form.parquetbuild_fangraphs_form_features.pypredict_game.pyRequired if enabledRequired from confirmed lineup1010
Fangraphs Projectionswarehouse/fangraphs/projections_asof.parquetfangraphs_projection_utils.pypredict_game.pyRequired if enabledNot selected00
Training + serving

Current Statcast and pitch mix

Historical Statcast builds training windows; the scheduled current refresh materializes compact serving snapshots for upcoming starters and lineups.

Sources consulted
  • Baseball Savant / StatcastPitch, contact-quality, batter and pitcher rolling windows plus pitch-mix profiles.
  • Baseball model artifact storeValidated, versioned feature snapshots, model configuration, estimator and projection history consumed by the hosted predictor.
Triggered by
  • Refresh current Statcast snapshotsDaily April–October at 12:15 UTC · GitHub Actions.github/workflows/baseball-statcast-current-refresh.yml
  • Offline model buildExplicit reviewed model build · Baseballpipeline.py / context-experiments
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Fetch pitch and batted-ball observations
  2. Build leakage-safe batter, pitcher and pitch-mix windows
  3. Materialize current player snapshots with coverage dates
  4. Resolve lineup/starter profiles and derive matchup interactions
4 materialized artifacts
  • warehouse/serving/statcast_batter_features.parquet
  • warehouse/serving/statcast_pitcher_features.parquet
  • warehouse/serving/pitch_mix_batter_features.parquet
  • warehouse/serving/pitch_mix_pitcher_features.parquet
Feeds
  • Moneyline head
  • Totals head
6 feature families · 14 ML / 13 totals columns
FamilyArtifactBuild → serveTrainingInferenceMLTotals
Lineup Warwarehouse/games_with_war.parquetwar/build_war_features.pypredict_game.pyRequired if enabledRequired from confirmed lineup33
Pitch Mix Interactionsin-processscripts/features/context_features.pypredict_game.pyRequired if enabledRequired from confirmed lineup53
Pitch Mix Matchup Xwarwarehouse/pitch_mix_batter_features.parquetpitch_mix_utils.pybuild_pitch_mix_serving_snapshot.pyRequired if enabledNot selected00
Pitch Mix Usagewarehouse/pitch_mix_pitcher_features.parquetpitch_mix_utils.pybuild_pitch_mix_serving_snapshot.pyRequired if enabledRequired from probable starters62
Statcast Batterswarehouse/statcast_batter_features.parquetbuild_statcast_features.pyfetch_current_statcast_features.pyRequired if enabledRequired from confirmed lineup03
Statcast Pitcherswarehouse/statcast_pitcher_features.parquetbuild_statcast_features.pyfetch_current_statcast_features.pyRequired if enabledRequired from probable starters02
Training + serving

Weather, elevation and park orientation

Archived pregame forecasts form the training contract; live canonical venue conditions override the same fields at projection time without using postgame observations.

Sources consulted
  • Open-MeteoVenue weather, wind, humidity, precipitation and elevation observations or archived forecasts.
  • Curated baseball reference filesReviewed park orientation, park-factor, player-identity and related static mappings shipped with model artifacts.
  • Sports Provider canonical storeCanonical MLB games, participants, lineups, venue conditions, markets and immutable price snapshots normalized from upstream providers.
Triggered by
  • Open-Meteo venue importOperator/provider-fetch request · Sports Providerprovider-fetch baseball:open-meteo:sport-event-ingestion
  • Offline model buildExplicit reviewed model build · Baseballpipeline.py / context-experiments
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Join parks to coordinates, roof/elevation and center-field azimuth
  2. Capture archived day-ahead weather for historical games
  3. Attach normalized live venue conditions to canonical games
  4. Resolve wind components and park-adjusted context at inference
3 materialized artifacts
  • data/features/park_weather_archived_forecast.csv
  • data/external/park_orientation.csv
  • Sports Provider Game.venueConditions
Feeds
  • Moneyline head
  • Totals head
2 feature families · 0 ML / 1 totals columns
FamilyArtifactBuild → serveTrainingInferenceMLTotals
Historical Weatherdata/features/park_weather_archived_forecast.csvfetch_open_meteo_historical_forecasts.pypredict_game.pyRequired if enabledRequired from schedule01
Park Orientationdata/external/park_orientation.csvfetch_park_orientation.pypredict_game.pyRequired if enabledNot selected00
Live serving

Point-in-time MLB park geometry

Monthly official venue snapshots preserve field-sector distances without copying current dimensions backward into historical games; the artifact remains research-only until enough honest history exists.

Sources consulted
  • MLB Stats APISchedules, teams, venues, probable pitchers, lineups, scores, current player pitching and staff context.
  • Baseball model artifact storeValidated, versioned feature snapshots, model configuration, estimator and projection history consumed by the hosted predictor.
Triggered by
  • Capture MLB park geometryMonthly at 11:45 UTC plus manual dispatch after dimension changes · GitHub Actions.github/workflows/baseball-park-geometry-refresh.yml
  • Publish reviewed model artifactsAfter model promotion and trust-manifest review · Baseballpnpm --filter baseball model-artifacts:publish
Processing
  1. Fetch all active MLB teams with hydrated venue location and fieldInfo
  2. Validate 30 teams and mandatory foul-line/center distances
  3. Append one immutable team snapshot without interpolating missing sectors
  4. Publish the cumulative history for future prior-only park experiments
1 materialized artifacts
  • warehouse/mlb_park_geometry_snapshots.parquet
Feeds
  • Moneyline head
  • Totals head

Operational input; not owned by a feature family.

Training + serving

Reviewed model publication and scoring

Training selects separate win and run contracts; publication binds estimator bytes to reviewable configuration and a trusted digest before the hosted predictor can load them.

Sources consulted
  • Baseball model artifact storeValidated, versioned feature snapshots, model configuration, estimator and projection history consumed by the hosted predictor.
Triggered by
  • Offline model buildExplicit reviewed model build · Baseballpipeline.py / context-experiments
  • Publish reviewed model artifactsAfter model promotion and trust-manifest review · Baseballpnpm --filter baseball model-artifacts:publish
  • Project upcoming gamesHourly at :45 and after material refreshes · Baseball/api/admin/cron/project-upcoming
Processing
  1. Run chronological selection, calibration and market gates
  2. Review and promote a candidate model pair
  3. Publish estimator, JSON sidecar and trusted SHA-256 manifest
  4. Score moneyline and discrete totals heads, then apply decision policy
4 materialized artifacts
  • models/game_model.pkl
  • models/game_model.json
  • pipeline/trusted-model-manifest.json
  • Projection and BetDecision history
Feeds
  • Moneyline head
  • Totals head
  • Decision policy

Operational input; not owned by a feature family.