Machine Learning from Scratch
Nine models and methods — gradient descent to decision trees — every one implemented in pure numpy on real NBA data.
No scikit-learn. Nine tutorials that implement the core of applied machine learning — gradient descent, ridge and logistic regression, train/test discipline, cross-validation, ROC curves, k-NN, decision trees, k-means — from first principles, in numpy, against real basketball data.
The order is a genuine curriculum: each step uses machinery the previous one built. By the end you will have written, not imported, an entire introductory ML course — and you will know exactly what every library call you make afterwards is doing.
What you'll be able to do
- Implement gradient descent and watch it land on the closed-form answer
- Build logistic regression and evaluate it like a professional: splits, k-fold, ROC/AUC
- Write k-NN, a decision tree and k-means clustering from bare numpy
- Explain regularization by having implemented it
The bundle contains each part's finished script, the shared helpers, and the sample data they read, with a README listing the run order - the whole course works offline.
The syllabus
Pace: Strictly sequential through part 6; the last three are siblings you can reorder. Give it three focused weeks.
0 of 9 parts marked complete on this device.
Every part of this course is marked complete on this device. The capstone below is the victory lap - and the quiz will tell you if anything slipped.
- Gradient Descent From Scratch: How Models Actually LearnThe engine under everything here: follow the gradient downhill.
- Ridge Regression From Scratch: Taming Collinear FeaturesRegularization as a dial you implemented, not a checkbox you ticked.
- Logistic Regression From Scratch: Predicting a Win, Not a NumberFrom predicting numbers to predicting wins - the sigmoid changes the game.
- Train/Test Split and the Confusion Matrix: Grading a Classifier HonestlyThe discipline: never grade a model on the data that taught it.
- K-Fold Cross-Validation: A Score You Can TrustOne split can lie; five folds lie less.
- ROC Curves and AUC From ScratchThresholds are choices; ROC shows every one of them at once.
- k-Nearest Neighbors From ScratchNo training at all - prediction by neighborhood vote.
- A Decision Tree From ScratchInterpretable splits, built greedily - see exactly why it predicts.
- K-Means Clustering From Scratch: Find Team ArchetypesNo labels needed: let the data cluster itself.
Check yourself
Five questions, graded on the page, nothing recorded anywhere - pick an answer and the explanation appears.
1. Gradient descent updates parameters by…
The gradient points uphill in loss; stepping against it, scaled by a learning rate, is the whole algorithm.
2. Ridge regression adds what to the loss?
Ridge is the L2 penalty - squared weights. Absolute weights would be lasso (L1).
3. Logistic regression outputs…
The linear score is squashed through the sigmoid into (0, 1); the class label is just a threshold you choose afterwards.
4. Why does test-set discipline exist?
A model is partly a memory of its training data. Only unseen data measures what it learned rather than what it memorized.
5. An AUC of 0.5 means the model…
AUC is a ranking measure: 0.5 is chance-level ranking. Accuracy at some threshold is a different (and threshold-dependent) number.
Where next: Randomness, Inference & Simulation.