- Data Engineering
- Machine Learning / AI
- Data Science
- Automation
MLB Intelligence Engine
A front-office analytics platform on 150 seasons of data and 8.4M Statcast pitches.
Ingests 150 seasons of historical stats and 8.4 million Statcast pitches into a PostgreSQL star-schema warehouse, then runs four models to find undervalued players, call pitcher-vs-batter matchups and predict live games — with a SHAP explanation on every prediction.
- Python 3.11
- pandas
- NumPy
- pybaseball
- Statcast
- Lahman DB
- MLB Stats API
- PostgreSQL 16
- SQLAlchemy
- scikit-learn
- XGBoost
- CatBoost
01
Project overview
- What it does
- A front-office style analytics tool that answers three questions: which players are undervalued relative to their salary, who has the edge in a given pitcher-vs-batter matchup, and who will win a game that is being played right now.
- Why I built it
- To build a complete data platform end to end — ingestion, warehouse modeling, modeling and an explainable interface — on a dataset large and messy enough to make the engineering matter.
- What it solves
- Baseball data is spread across sources with different grains, IDs and refresh cycles. Useful decisions need them joined, modeled and kept current.
- Who it's for
- Baseball operations analysts, scouts and fans who want model-backed answers they can interrogate rather than black-box scores.
02
The problem
Historical stats (Lahman), pitch-level tracking (Statcast) and live game data (MLB Stats API) each describe the game at a different grain and update on a different schedule. Answering front-office questions means reconciling player identities across sources, modeling 8.4 million pitch events efficiently, and keeping everything fresh without manual work — then presenting predictions in a way a decision-maker can trust.
03
The solution
An automated ETL layer pulls each source with pybaseball and the MLB Stats API, normalizes it with pandas, and loads it into a PostgreSQL 16 star schema through SQLAlchemy. Four models sit on top of the warehouse — built with scikit-learn, XGBoost and CatBoost — and every prediction ships with a SHAP explanation. A Streamlit and Plotly interface exposes the three core questions, and an OpenAI-powered assistant answers ad-hoc questions with SQL that is validated by sqlglot before it ever runs. The whole stack is containerized with Docker Compose, driven by a Typer CLI and PowerShell automation, refreshes daily, and is tested with pytest.
04
Architecture & workflow
- STEP 01
Sources
Lahman DB, Statcast, MLB Stats API
- pybaseball
- STEP 02
Ingestion
Daily automated pulls
- Typer CLI
- PowerShell
- STEP 03
Transform
Clean, reconcile IDs, conform grains
- pandas
- NumPy
- STEP 04
Warehouse
Star schema: facts + dimensions
- PostgreSQL 16
- SQLAlchemy
- STEP 05
Models
4 models + SHAP explanations
- XGBoost
- CatBoost
- scikit-learn
- STEP 06
App
Dashboards + validated text-to-SQL
- Streamlit
- OpenAI
- sqlglot
05
Technology stack
Languages
- Python 3.11
Data Engineering
- pandas
- NumPy
- pybaseball
- Statcast
- Lahman DB
- MLB Stats API
Databases
- PostgreSQL 16
- SQLAlchemy
Machine Learning
- scikit-learn
- XGBoost
- CatBoost
- SHAP
AI / LLM
- OpenAI API
- sqlglot
Visualization
- Streamlit
- Plotly
Cloud & Infrastructure
- Docker
- Docker Compose
- PowerShell
- Typer CLI
Testing
- pytest
06
Project demonstration
07
Key features
Undervalued player finder
Compares modeled player value against salary to surface market inefficiencies.
Matchup edge
Pitcher-vs-batter predictions built on pitch-level Statcast history.
Live win probability
Predicts the winner of games in progress using live MLB Stats API data.
Explained predictions
A SHAP breakdown accompanies every prediction, so the 'why' is always visible.
Guarded SQL assistant
Natural-language questions become SQL, validated with sqlglot before execution.
Self-refreshing data
Daily automated refresh keeps the warehouse current without manual steps.