Linear Discriminant Analysis

Step-by-step LDA tutorial covering scatter matrices, eigenvectors, and Python code.

Principal Component Analysis in 3 Simple Steps

Standardization, covariance matrices, eigendecomposition, and projection with Python.

Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning — Part I

Generalization performance, train-test splits, bias, variance, and evaluation basics.

Creating Confidence Intervals for Machine Learning Classifiers

Bootstrap and normal approximation methods for classifier confidence intervals.

Training an XGBoost Classifier Using Cloud GPUs

Practical guide to GPU-accelerated XGBoost training without local hardware.

Naive Bayes and Text Classification

From-scratch Naive Bayes implementation applied to text classification in Python.

Machine Learning — Fundamentals & Tutorials

In-depth tutorials on machine learning fundamentals — model evaluation, dimensionality reduction, classification, and Python implementations with scikit-learn and NumPy. Many of these complement my Python Machine Learning and Machine Learning with PyTorch and Scikit-Learn books.

2023

Jan 15 Training an XGBoost Classifier Using Cloud GPUs Without Worrying About Infrastructure Imagine you want to quickly train a few machine learning or deep learning models on the cloud but don't want to deal with cloud infrastructure. This short...

2022

Jul 24 A Short Chronology Of Deep Learning For Tabular Data Occasionally, I share research papers proposing new deep learning approaches for tabular data on social media, which is typically an excellent discussion...
Apr 25 Creating Confidence Intervals for Machine Learning Classifiers Developing good predictive models hinges upon accurate performance evaluation and comparisons. However, when evaluating machine learning models, we...

2018

Nov 10 Model evaluation, model selection, and algorithm selection in machine learning Part 4 of the model evaluation series explaining statistical tests, algorithm comparisons, corrected resampled tests, and nested cross-validation.

2016

Oct 2 Model evaluation, model selection, and algorithm selection in machine learning Part 3 of the model evaluation series covering hyperparameter tuning, model selection, validation sets, k-fold cross-validation, and nested workflows.
Aug 13 Model evaluation, model selection, and algorithm selection in machine learning Part 2 of the model evaluation series explaining bootstrap methods, holdout validation, resampling variance, uncertainty estimates, and model stability.
Jun 11 Model evaluation, model selection, and algorithm selection in machine learning Part 1 of a practical model evaluation series covering generalization performance, train-test splits, bias, variance, and supervised learning workflow basics.

2015

Mar 24 Single-Layer Neural Networks and Gradient Descent History and fundamentals of single-layer neural networks and gradient descent, with Python implementations of the perceptron and ADALINE for classification.
Jan 27 Principal Component Analysis Step-by-step PCA tutorial that explains standardization, covariance matrices, eigendecomposition, explained variance, and projection with Python code.
Jan 11 Implementing a Weighted Majority Rule Ensemble Classifier Here, I want to present a simple and conservative approach of implementing a weighted majority rule ensemble classifier in scikit-learn that yielded...

2014

Sep 14 Kernel tricks and nonlinear dimensionality reduction via RBF kernel PCA Tutorial on kernel methods and nonlinear dimensionality reduction with RBF kernel PCA, including the kernel trick and a Python implementation.
Aug 25 Predictive modeling, supervised machine learning, and pattern classification When I was working on my next pattern classification application, I realized that it might be worthwhile to take a step back and look at the big picture of...
Aug 3 Linear Discriminant Analysis Step-by-step Linear Discriminant Analysis tutorial covering scatter matrices, eigenvectors, class separation, dimensionality reduction, and Python code.
Jun 19 Kernel density estimation via the Parzen-Rosenblatt window method The Parzen-window method (also known as Parzen-Rosenblatt window method) is a widely used non-parametric approach to estimate a probability density function...
Apr 13 Implementing a Principal Component Analysis (PCA) Implementing Principal Component Analysis from scratch in Python: scatter matrices, eigenvectors, variance explained, and comparison with scikit-learn PCA.