Randomness, Inference & Simulation
Monte Carlo seasons, bootstrap intervals, permutation tests, chi-square and Bayesian updating — how to tell signal from luck.
The hardest question in sports analytics isn't computing a number — it's deciding whether the number means anything. Six tutorials on exactly that: simulate a season to see how much of a standings table is luck, bootstrap a confidence interval instead of pretending you have one, test a difference by shuffling, and update a belief the Bayesian way.
Everything is built from scratch with numpy — no scipy black boxes — so when a p-value comes out, you watched it being made.
What you'll be able to do
- Simulate seasons with Monte Carlo and read the spread honestly
- Attach uncertainty to any stat with the bootstrap
- Test claims with permutation and chi-square tests you wrote yourself
- Update estimates as evidence arrives with Beta-Binomial updating
The bundle contains each part's finished script, the shared helpers, and the sample data they read, with a README listing the run order - the whole course works offline.
The syllabus
Pace: Strictly in order - each part leans on the one before. Six sittings.
0 of 6 parts marked complete on this device.
Every part of this course is marked complete on this device. The capstone below is the victory lap - and the quiz will tell you if anything slipped.
- Monte Carlo: How Much Does Luck Move a Season's Record?Before trusting any standings gap, see how much a coin-flip league produces.
- Bootstrap a Confidence Interval: Is Home-Court Advantage Real or Luck?Error bars from resampling - no formula memorization required.
- Is the Difference Real? A Permutation TestThe significance test you can explain to anyone: shuffle and count.
- Crosstabs: Build a Contingency Table to Test Home AdvantageCategories cross-tabulated - the table every count-based test starts from.
- The Chi-Square Test from Scratch: Is an NBA Floor Left-Right Symmetric?Is the imbalance real? The classic test, built by hand.
- Bayesian Updating: Learn a Win Probability Game by GameBeliefs that update as evidence arrives - shooting percentages done properly.
Check yourself
Five questions, graded on the page, nothing recorded anywhere - pick an answer and the explanation appears.
1. A permutation test answers the question…
Shuffle the labels, recompute the gap, repeat. The fraction of shuffles beating the observed gap is the p-value - that's the whole test.
2. A bootstrap confidence interval comes from…
The bootstrap treats your sample as a stand-in for the population and reads the spread of the statistic across resamples.
3. A chi-square test compares…
It sums (observed − expected)² / expected over the cells of a count table. Means and medians belong to other tests.
4. A Beta(a, b) prior, after observing k makes in n attempts, becomes…
Add successes to the first parameter, failures to the second. That conjugate update is the entire mechanism of the tutorial.
5. The lasting lesson of Monte Carlo season simulations is…
Even a league of known, fixed team strengths produces big standings spread from coin-flip variance alone. Some of every table is luck.
Where next: Machine Learning from Scratch.