Blog and Notes

RSS Feed

Subscribe via Email

Browse by topic: Large Language Models PyTorch Machine Learning

2026

Sep 2, 2026 Quick Model Note OpenAI Astra and Looped Transformers A short note on OpenAI Astra, recurrent depth, looped transformers, Nanbeige 4.2, and the Mixture-of-Recursions paper.
Aug 30, 2026 Quick Video Note Reasoning Models From Scratch: Code Setup A short video on how conventional LLMs relate to reasoning models and agents, along with Python and PyTorch setup using uv.
Aug 29, 2026 Quick Book Note Two Live Book Club Q&As on September 3 Two free live book club Q&A sessions on September 3 about Build a Reasoning Model From Scratch and related reader questions.
Aug 26, 2026 Quick Model Note GLM-5.3-Flash Architecture Notes Architecture note on GLM-5.3-Flash, formerly Ox Alpha, covering its KDA and MLA/DSA hybrid attention, sparse MoE backbone, and four-stream mHC residual path.
Aug 22, 2026 How Claude Watermarks AI-Generated Text A 48-minute video walkthrough of token sampling, watermark detection, and removal A 48-minute video walkthrough of token sampling, watermark detection, and removal
Aug 15, 2026 Building an AI Text Detector From Scratch An End-to-End Project With Dataset Construction, Model Training, Local Deployment, and RLVR An End-to-End Project With Dataset Construction, Model Training, Local Deployment, and RLVR
Aug 15, 2026 Quick Model Note How Claude's Text Watermarking Works Short illustration of how Claude's text watermarking is supposed to work based on Anthropic's released materials.
Aug 12, 2026 Quick Book Note Build a Reasoning Model From Scratch Is Now on Amazon Short note on the Amazon availability of Build a Reasoning Model From Scratch and a warning about counterfeit black-and-white copies sold through Amazon India.
Aug 11, 2026 Quick Model Note Muse Glimmer 30B Architecture Notes Short architecture note on Meta Muse Glimmer 30B, including gated local and global GQA, KV-cache efficiency, and release-time benchmark comparisons.
Aug 7, 2026 Quick Project Note LLMs From Scratch Reaches 100,000 GitHub Stars Short note celebrating the LLMs-from-scratch repository passing 100,000 GitHub stars and summarizing its learning materials.
Jul 28, 2026 Quick Model Note Kimi K3 Architecture Notes Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.
Jul 26, 2026 Quick Model Note A Few Notable Open-Weight Models This Week Short note on the architectures of six new open-weight models, including Nanbeige 4.2, Laguna S 2.1, Motif-3-Beta, Solar Open 2, Antares 1B, and BTL-3.
Jul 25, 2026 Quick Book Note Correction for Listing 6.5 in Build a Reasoning Model From Scratch Short correction note for the random seed in Listing 6.5 on page 198 of Build a Reasoning Model From Scratch.
Jul 18, 2026 Controlling Reasoning Effort in LLMs How LLMs Learn Low-, Medium-, and High-Effort Reasoning Modes How LLMs Learn Low-, Medium-, and High-Effort Reasoning Modes
Jul 16, 2026 Quick Model Note Inkling: A New Open-Weight 975B MoE with a Few Surprises Architecture and benchmark notes on Thinking Machines Lab's 975B Inkling MoE, including short convolutions, relative-position bias, training, and effort control.
Jul 12, 2026 Quick Blog Note 200,000 Subscribers Short note celebrating Ahead of AI reaching 200,000 subscribers.
Jul 9, 2026 Quick Model Note GPT 5.6 Has 72 Possible Configurations. What's A Good Default? Short note on how GPT 5.6 model and effort choices map onto training-time and inference-time scaling, producing 72 configurations.
Jun 30, 2026 Quick Book Note Build a Reasoning Model From Scratch Is Out Short note announcing the release of Build a Reasoning Model From Scratch and linking the publisher and Amazon pages.
Jun 29, 2026 Quick Article Note Using Local Coding Agents Short note linking a new article on setting up local coding agents with open-weight models.
Jun 27, 2026 Using Local Coding Agents Using Open-Weight Models in Local Coding Harnesses as an Alternative to Claude Code and Codex Subscriptions Using Open-Weight Models in Local Coding Harnesses as an Alternative to Claude Code and Codex Subscriptions
Jun 26, 2026 Quick Benchmark Note Local Open-Weight LLMs in Coding Harnesses Short note on trying local open-weight LLMs across Qwen-Code, Codex, and Claude Code harnesses.
Jun 18, 2026 Quick Model Note GLM-5.2 and IndexShare for Long-Context Sparse Attention How GLM-5.2 uses IndexShare to reuse sparse-attention token selections across layers and reduce long-context indexer computation.
Jun 17, 2026 Quick Model Note VibeThinker-3B and the Strength of Post-Training A closer look at VibeThinker-3B, a Qwen2.5-Coder-based model whose reported reasoning gains come from a detailed post-training pipeline.
Jun 12, 2026 Quick Model Note North Mini Code and Agentic Coding Benchmarks Architecture and benchmark notes for North Mini Code, Cohere's 30B-A3B MoE trained for repository, terminal, and code-generation tasks.
Jun 6, 2026 LLM Research Papers: The 2026 List (January to May) A curated roundup of notable LLM research papers that came out this year
Jun 4, 2026 Quick Architecture Note Nemotron 3 Ultra and Latent MoE Scaling Architecture notes on Nemotron 3 Ultra, including its 108-layer hybrid stack, Latent MoE scaling, NVFP4 recipe, MTP, and inference results.
May 27, 2026 Quick Paper Note MiniMax M2 Technical Report Notes Technical notes on MiniMax M2, including full attention, fine-grained MoE routing, agent training data, speed rewards, and self-evolution.
May 23, 2026 Quick Architecture Note DeepSeek Sparse Attention From Scratch Short note on a DeepSeek Sparse Attention from-scratch implementation added to the LLMs-from-scratch repository.
May 16, 2026 Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention From Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs From Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs
May 14, 2026 Quick Architecture Note Implementing LLM Architectures From Scratch Short note linking a talk on implementing LLM architectures from scratch and comparing new open-weight model implementations against references.
Apr 18, 2026 My Workflow for Understanding LLM Architectures A learning-oriented workflow for understanding new open-weight model releases A learning-oriented workflow for understanding new open-weight model releases
Apr 4, 2026 Components of A Coding Agent How coding agents use tools, memory, and repo context to make LLMs work better in practice How coding agents use tools, memory, and repo context to make LLMs work better in practice
Apr 2, 2026 Quick Benchmark Note Gemma 4 Architecture and Benchmark Notes Architecture and benchmark notes for Gemma 4 31B and 26B-A4B, including hybrid attention, long-context changes, and evaluation caveats.
Mar 26, 2026 Quick Architecture Note LLM Architecture Gallery Diff Tool Compare two LLM architectures side by side across attention, decoder type, layer recipe, model scale, context length, and KV-cache use.
Mar 22, 2026 A Visual Guide to Attention Variants in Modern LLMs From MHA and GQA to MLA, sparse attention, and hybrid architectures From MHA and GQA to MLA, sparse attention, and hybrid architectures
Mar 14, 2026 Inside the LLM Architecture Gallery Browse 93 LLM architecture diagrams with model fact sheets, source links, concept explainers, and a side-by-side comparison tool.
Mar 12, 2026 Quick Model Note Nemotron 3 Super Throughput Notes Nemotron 3 Super combines Mamba-2, Latent MoE, sparse GQA, and shared-weight MTP in a throughput-oriented 120B-A12B model.
Feb 25, 2026 A Dream of Spring for Open-Weight LLMs: 10 Architectures from Jan-Feb 2026 A Round Up And Comparison of 10 Open-Weight LLM Releases in Spring 2026
Feb 1, 2026 State of AI 2026 with Sebastian Raschka, Nathan Lambert, and Lex Fridman I recently sat down with Lex Fridman and Nathan Lambert for a comprehensive 4.5 h interview to discuss the current state of progress of AI, and what the...
Jan 24, 2026 Categories of Inference-Time Scaling for Improved LLM Reasoning And an Overview of Recent Inference-Scaling Papers (Including Recursive Language Models) Inference scaling has become one of the most effective ways to improve answer quality and accuracy in deployed LLMs. The idea is straightforward. If we are...

2025

Dec 30, 2025 The State Of LLMs 2025: Progress, Problems, and Predictions A 2025 review of large language models, from DeepSeek R1 and RLVR to inference-time scaling, benchmarks, architectures, and predictions for 2026.
Dec 30, 2025 LLM Research Papers: The 2025 List (July to December) A curated list of LLM research papers from July–December 2025, organized by reasoning models, inference-time scaling, architectures, training efficiency...
Dec 7, 2025 From Random Forests to RLVR: A Short History of ML/AI Hello Worlds Two years ago, I posted a list of Hello World examples for machine learning and AI on social. Here, the Hello World means beginner-friendly examples to...
Dec 2, 2025 From DeepSeek V3 to V3.2: Architecture, Sparse Attention, and RL Updates Understanding How DeepSeek's Flagship Open-Weight Models Evolved Similar to DeepSeek V3, the team released their new flagship model over a major US holiday weekend. Given DeepSeek V3.2's really good performance (on GPT-5...
Nov 11, 2025 Recommendations for Getting the Most Out of a Technical Book This short article compiles a few notes I previously shared when readers ask how to get the most out of my building large language model from scratch books...
Nov 3, 2025 Beyond Standard LLMs Linear Attention Hybrids, Text Diffusion, Code World Models, and Small Recursive Transformers After I shared my Big LLM Architecture Comparison a few months ago, which focused on the main transformer-based LLMs, I received a lot of questions with...
Oct 28, 2025 DGX Spark and Mac Mini for Local PyTorch Development First Impressions and Benchmarks The DGX Spark for local LLM inferencing and fine-tuning was a pretty popular discussion topic recently. I got to play with one myself, primarily working...
Oct 4, 2025 Understanding the 4 Main Approaches to LLM Evaluation (From Scratch) Multiple-Choice Benchmarks, Verifiers, Leaderboards, and LLM Judges with Code Examples Multiple-Choice Benchmarks, Verifiers, Leaderboards, and LLM Judges with Code Examples
Sep 6, 2025 Understanding and Implementing Qwen3 From Scratch A Detailed Look at One of the Leading Open-Source LLMs Previously, I compared the most notable open-weight architectures of 2025 in The Big LLM Architecture Comparison. Then, I zoomed in and discussed the...
Aug 9, 2025 From GPT-2 to gpt-oss: Analyzing the Architectural Advances And How They Stack Up Against Qwen3 OpenAI just released their new open-weight LLMs this week: gpt-oss-120b and gpt-oss-20b, their first open-weight models since GPT-2 in 2019. And yes, thanks...
Jul 19, 2025 The Big LLM Architecture Comparison From DeepSeek V3 to GLM-5: A Look At Modern LLM Architecture Design It has been seven years since the original GPT architecture was developed. At first glance, looking back at GPT-2 (2019) and forward to DeepSeek-V3 and...
Jul 1, 2025 LLM Research Papers: The 2025 List (January to June) A topic-organized collection of 200+ LLM research papers from 2025 The latest in LLM research with a hand-curated, topic-organized list of over 200 research papers from 2025.
Jun 17, 2025 Understanding and Coding the KV Cache in LLMs from Scratch KV caches are one of the most critical techniques for efficient inference in LLMs in production. KV caches are an important component for compute-efficient...
May 9, 2025 Coding LLMs from the Ground Up: A Complete Course Why build an LLM from scratch? It's probably the best and most efficient way to learn how LLMs really work. Plus, many readers have told me they had a lot...
Apr 18, 2025 The State of Reinforcement Learning for LLM Reasoning Understanding GRPO and New Insights from Reasoning Model Papers A lot has happened this month, especially with the releases of new flagship models like GPT-4.5 and Llama 4. But you might have noticed that reactions to...
Mar 29, 2025 First Look at Reasoning From Scratch: Chapter 1 As you know, I've been writing a lot lately about the latest research on reasoning in LLMs. Before my next research-focused blog post, I wanted to offer...
Mar 8, 2025 Inference-Time Compute Scaling Methods to Improve Reasoning Models Part 1: Inference-Time Compute Scaling Methods This article explores recent research advancements in reasoning-optimized LLMs, with a particular focus on inference-time compute scaling that have emerged...
Feb 5, 2025 Understanding Reasoning LLMs Methods and Strategies for Building and Refining Reasoning Models Overview of four ways to build reasoning-capable LLMs, including inference-time scaling, supervised finetuning, reinforcement learning, and search-based methods.
Jan 23, 2025 Noteworthy LLM Research Papers of 2024 —12 influential AI papers from January to December 2024 This article covers 12 influential AI research papers of 2024, ranging from mixture-of-experts models to new LLM scaling laws for precision.
Jan 17, 2025 Implementing A Byte Pair Encoding (BPE) Tokenizer From Scratch Implements byte pair encoding (BPE) tokenization from scratch: tokenizer training, GPT-style merge rules, and step-by-step Python examples.

2024

Dec 29, 2024 LLM Research Papers: The 2024 List I want to share my running bookmark list of many fascinating (mostly LLM-related) papers I stumbled upon in 2024. It's just a list, but maybe it will come...
Nov 3, 2024 Understanding Multimodal LLMs An Introduction to the Main Techniques and Latest Models There has been a lot of new research on the multimodal LLM front, including the latest Llama 3.2 vision models, which employ diverse architectural...
Sep 21, 2024 Building A GPT-Style LLM Classifier From Scratch Finetuning a GPT Model for Spam Classification This article shows you how to transform pretrained large language models (LLMs) into strong text classifiers. But why focus on classification? First...
Sep 1, 2024 Building LLMs from the Ground Up: A 3-hour Coding Workshop Three-hour coding workshop that builds the core pieces of a GPT-style large language model from the ground up for developers who want implementation intuition.
Aug 17, 2024 New LLM Pre-training and Post-training Paradigms -- A Look at How Moderns LLMs Are Trained There are hundreds of LLM papers each month proposing new techniques and approaches. However, one of the best ways to see what actually works well in...
Jul 20, 2024 Instruction Pretraining LLMs -- The Latest Research in Instruction Finetuning This article covers a new, cost-effective method for generating data for instruction finetuning LLMs; instruction finetuning from scratch; pretraining LLMs...
Jun 2, 2024 LLM Research Insights: Instruction Masking and New LoRA Finetuning Experiments? Discussing the Latest Model Releases and AI Research in May 2024 This article covers three new papers related to instruction finetuning and parameter-efficient finetuning with LoRA in large language models (LLMs). I work...
Jun 2, 2024 Developing an LLM: Building, Training, Finetuning A Deep Dive into the Lifecycle of LLM Development This is an overview of the LLM development process. This one-hour talk focuses on the essential three stages of developing an LLM: coding the architecture...
May 12, 2024 How Good Are the Latest Open LLMs? And Is DPO Better Than PPO? Discussing the Latest Model Releases and AI Research in April 2024 What a month! We had four major open LLM releases: Mixtral, Meta AI's Llama 3, Microsoft's Phi-3, and Apple's OpenELM. In my new article, I review and...
Apr 20, 2024 Using and Finetuning Pretrained Transformers Guide to using and finetuning pretrained transformers, comparing feature extraction, prompt-based use, full finetuning, and parameter-efficient LLM adaptation.
Mar 31, 2024 Tips for LLM Pretraining and Evaluating Reward Models Research Papers in March 2024 It's another month in AI research, and it's hard to pick favorites. This month, I am going over a paper that discusses strategies for the continued...
Mar 3, 2024 Research Papers in February 2024 — A LoRA Successor, Small Finetuned LLMs Vs Generalist LLMs, and Transparent LLM Research Once again, this has been an exciting month in AI research. This month, I'm covering two new openly available LLMs, insights into small finetuned LLMs, and...
Feb 18, 2024 Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch Technical tutorial on LoRA and DoRA for parameter-efficient finetuning, with from-scratch PyTorch code and intuition for weight-decomposed low-rank adaptation.

2023

Sep 15, 2023 Optimizing LLMs From a Dataset Perspective Practical guide to improving LLM finetuning with better instruction datasets, covering data curation, prompt-output pairs, synthetic data, and experiment ideas.
Aug 10, 2023 The NeurIPS 2023 LLM Efficiency Challenge Starter Guide Large language models (LLMs) offer one of the most interesting opportunities for developing more efficient training methods. A few weeks ago, the NeurIPS...
Jul 1, 2023 Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch Peak memory consumption is a common bottleneck when training deep learning models such as vision transformers and LLMs. This article provides a series of...
Jun 14, 2023 Finetuning Falcon LLMs More Efficiently With LoRA and Adapters Finetuning allows us to adapt pretrained LLMs in a cost-efficient manner. But which method should we use? This article compares different...
May 11, 2023 Accelerating Large Language Models with Mixed-Precision Techniques Training and using large language models (LLMs) is expensive due to their large compute requirements and memory footprints. This article will explore how...
Apr 26, 2023 Parameter-Efficient LLM Finetuning With Low-Rank Adaptation (LoRA) Pretrained large language models are often referred to as foundation models for a good reason: they perform well on various tasks, and we can use them as a...
Apr 12, 2023 Understanding Parameter-Efficient Finetuning of Large Language Models: From Prefix Tuning to LLaMA-Adapters In the rapidly evolving field of artificial intelligence, utilizing large language models in an efficient and effective manner has become increasingly...
Mar 28, 2023 Finetuning Large Language Models On A Single GPU Using Gradient Accumulation Previously, I shared an article using multi-GPU training strategies to speed up the finetuning of large language models. Several of these strategies include...
Mar 23, 2023 Keeping Up With AI Research And News When it comes to productivity workflows, there are a lot of things I'd love to share. However, the one topic many people ask me about is how I keep up with...
Feb 23, 2023 Some Techniques To Make Your PyTorch Models Train (Much) Faster This blog post outlines techniques for improving the training performance of your PyTorch model without compromising its accuracy. To do so, we will wrap a...
Feb 9, 2023 Understanding and Coding Self-Attention, Multi-Head Attention, Causal Attention, and Cross-Attention in LLMs A deep-dive implementation of self-attention, multi-head attention, causal attention, and cross-attention — the mechanisms behind modern transformer LLMs.
Feb 9, 2023 Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch Step-by-step tutorial explaining scaled dot-product self-attention for large language models, with Python code that builds the mechanism from scratch.
Feb 7, 2023 Understanding Large Language Models -- A Transformative Reading List Curated reading list for understanding large language models, from attention and transformers to BERT, GPT, scaling laws, instruction tuning, and RLHF.
Feb 1, 2023 What Are the Different Approaches for Detecting Content Generated by LLMs Such As ChatGPT? And How Do They Work and Differ? Since the release of the AI Classifier by OpenAI made big waves yesterday, I wanted to share a few details about the different approaches for detecting...
Jan 29, 2023 Comparing Different Automatic Image Augmentation Methods in PyTorch Data augmentation is a key tool in reducing overfitting, whether it's for images or text. This article compares three Auto Image Data Augmentation...
Jan 16, 2023 Curated Resources and Trustworthy Experts: The Key Ingredients for Finding Accurate Answers to Technical Questions in the Future Conversational chat bots such as ChatGPT probably will not be able replace traditional search engines and expert knowledge anytime soon. With the vast...
Jan 15, 2023 Training an XGBoost Classifier Using Cloud GPUs Without Worrying About Infrastructure Imagine you want to quickly train a few machine learning or deep learning models on the cloud but don't want to deal with cloud infrastructure. This short...
Jan 5, 2023 Open Source Highlights 2022 for Machine Learning & AI Recently, I shared the top 10 papers that I read in 2022. As a follow-up, I am compiling a list of my favorite 10 open-source releases that I discovered...
Jan 3, 2023 Influential Machine Learning Papers Of 2022 Every day brings something new and exciting to the world of machine learning and AI, from the latest developments and breakthroughs in the field to emerging...

2022

Oct 15, 2022 Ahead Of AI, And What's Next? About monthly machine learning musings, and other things I am currently workin on ...
Jul 24, 2022 A Short Chronology Of Deep Learning For Tabular Data Occasionally, I share research papers proposing new deep learning approaches for tabular data on social media, which is typically an excellent discussion...
Jul 5, 2022 No, We Don't Have to Choose Batch Sizes As Powers Of 2 Regarding neural network training, I think we are all guilty of doing this: we choose our batch sizes as powers of 2, that is, 64, 128, 256, 512, 1024, and...
Jun 30, 2022 Sharing Deep Learning Research Models with Lightning Part 2: Leveraging the Cloud Deploys a Super Resolution GAN (SRGAN) app to the cloud with Lightning.ai — covers containerization, model serving, and production deployment steps.
Jun 17, 2022 Sharing Deep Learning Research Models with Lightning Part 1: Building A Super Resolution App Build a Super Resolution GAN app with Lightning.ai: interactive UI, model serving, and a modern workflow for sharing deep learning research.
Jun 12, 2022 Taking Datasets, DataLoaders, and PyTorch’s New DataPipes for a Spin The PyTorch team recently announced TorchData, a prototype library focused on implementing composable and reusable data loading utilities for PyTorch. In...
May 18, 2022 Running PyTorch on the M1 GPU Today, PyTorch officially introduced GPU support for Apple's ARM M1 chips. This is an exciting day for Mac users out there, so I spent a few minutes trying...
Apr 25, 2022 Creating Confidence Intervals for Machine Learning Classifiers Developing good predictive models hinges upon accurate performance evaluation and comparisons. However, when evaluating machine learning models, we...
Apr 4, 2022 Losses Learned -- Optimizing Negative Log-Likelihood and Cross-Entropy in PyTorch (Part 1) The cross-entropy loss is our go-to loss for training deep learning-based classifiers. In this article, I am giving you a quick tour of how we usually...
Mar 24, 2022 TorchMetrics -- How do we use it, and what's the difference between .update() and .forward()? TorchMetrics is a really nice and convenient library that lets us compute the performance of models in an iterative fashion. It's designed with PyTorch (and...
Feb 25, 2022 Machine Learning with PyTorch and Scikit-Learn -- The *new* Python Machine Learning Book Machine Learning with PyTorch and Scikit-Learn has been a long time in the making, and I am excited to finally get to talk about the release of my new book...

2021

Dec 29, 2021 Introduction to Machine Learning -- Video Lectures about Python Basics, Tree-based Methods, Model Evaluation, and Feature Selection About half a year ago, I organized all my deep learning-related videos in a handy blog post to have everything in one place. Since many people liked this...
Jul 9, 2021 Introduction to Deep Learning -- 170 Video Lectures from Adaptive Linear Neurons to Zero-shot Classification with Transformers I just sat down this morning and organized all deep learning related videos I recorded in 2021. I am sure this will be a useful reference for my future...
Feb 11, 2021 Datasets for Machine Learning and Deep Learning -- Some of the Best Places to Explore With the semester being in full swing, I recently shared this set of dataset repositories with my deep learning class. However, I thought that beyond using...
Jan 21, 2021 Book Review: Deep Learning With PyTorch -- A Practical Deep Learning Guide With a Computer Vision Focus and an Interesting Structure After its release in August 2020, Deep Learning with PyTorch has been sitting on my shelf before I finally got a chance to read it during this winter break...
Jan 3, 2021 How I Keep My Projects Organized Since I started my undergraduate studies in 2008, I have been obsessed with productivity tips, notetaking solutions, and todo-list management. Over the...

2020

Sep 27, 2020 Scientific Computing in Python: Introduction to NumPy and Matplotlib -- Including Video Tutorials Beginner's guide to NumPy and Matplotlib for scientific computing: arrays, indexing, broadcasting, linear algebra, and plotting with Python code examples.
Aug 26, 2020 Interpretable Machine Learning -- Book Review and Thoughts about Linear and Logistic Regression as Interpretable Models Review of Christoph Molnar's Interpretable Machine Learning, plus practical notes on GAMs, LIME, and SHAP for explaining black-box model predictions.
Aug 5, 2020 Chapter 1: Introduction to Machine Learning and Deep Learning The first chapter (draft) of the Introduction to Deep Learning book, which is a book based on my lecture notes and slides.
Jan 6, 2020 Book Review: Architects of Intelligence by Martin Ford A brief review of Martin Ford's book that features interviews with 23 of the most well-known and brightest minds working on AI.

2019

Dec 12, 2019 What's New in the 3rd Edition A brief summary of what's new in the 3rd edition of Python Machine Learning.
May 24, 2019 My First Year at UW-Madison and a Gallery of Awesome Student Projects Not too long ago, in the Summer of 2018, I was super excited to join the Department of Statistics at the University of Wisconsin-Madison after obtaining my...

2018

Nov 10, 2018 Model evaluation, model selection, and algorithm selection in machine learning Part IV - Comparing the performance of machine learning models and algorithms using statistical tests and nested cross-validation Part 4 of the model evaluation series explaining statistical tests, algorithm comparisons, corrected resampled tests, and nested cross-validation.
Aug 2, 2018 Generating Gender-Neutral Face Images with Semi-Adversarial Neural Networks to Enhance Privacy I thought that it would be nice to have short and concise summaries of recent projects handy, to share them with a more general audience, including...

2016

Oct 2, 2016 Model evaluation, model selection, and algorithm selection in machine learning Part III - Cross-validation and hyperparameter tuning Part 3 of the model evaluation series covering hyperparameter tuning, model selection, validation sets, k-fold cross-validation, and nested workflows.
Aug 13, 2016 Model evaluation, model selection, and algorithm selection in machine learning Part II - Bootstrapping and uncertainties Part 2 of the model evaluation series explaining bootstrap methods, holdout validation, resampling variance, uncertainty estimates, and model stability.
Jun 11, 2016 Model evaluation, model selection, and algorithm selection in machine learning Part I - The basics Part 1 of a practical model evaluation series covering generalization performance, train-test splits, bias, variance, and supervised learning workflow basics.

2015

Sep 24, 2015 Writing 'Python Machine Learning' – A Reflection on a Journey It's been about time. I am happy to announce that "Python Machine Learning" was finally released today! Sure, I could just send an email around to all the...
Aug 24, 2015 Python, Machine Learning, and Language Wars – A Highly Subjective Point of View This has really been quite a journey for me lately. And regarding the frequently asked question “Why did you choose Python for Machine Learning?” I guess it...
Mar 24, 2015 Single-Layer Neural Networks and Gradient Descent History and fundamentals of single-layer neural networks and gradient descent, with Python implementations of the perceptron and ADALINE for classification.
Jan 27, 2015 Principal Component Analysis in 3 Simple Steps Step-by-step PCA tutorial that explains standardization, covariance matrices, eigendecomposition, explained variance, and projection with Python code.
Jan 11, 2015 Implementing a Weighted Majority Rule Ensemble Classifier in scikit-learn Here, I want to present a simple and conservative approach of implementing a weighted majority rule ensemble classifier in scikit-learn that yielded...

2014

Dec 5, 2014 MusicMood – A Machine Learning Model for Classifying Music by Mood Based on Song Lyrics Music mood classification project: predicting a song's emotional valence from lyrics with NLP, feature engineering, and scikit-learn in Python.
Nov 28, 2014 Turn Your Twitter Timeline into a Word Cloud – using Python Last week, I posted some visualizations in context of Happy Rock Song data mining project, and some people were curious about how I created the word clouds...
Oct 4, 2014 Naive Bayes and Text Classification – Introduction and Theory Naive Bayes classifiers, a family of classifiers that are based on the popular Bayes’ probability theorem, are known for creating simple yet well performing...
Sep 14, 2014 Kernel tricks and nonlinear dimensionality reduction via RBF kernel PCA Tutorial on kernel methods and nonlinear dimensionality reduction with RBF kernel PCA, including the kernel trick and a Python implementation.
Aug 25, 2014 Predictive modeling, supervised machine learning, and pattern classification — the big picture When I was working on my next pattern classification application, I realized that it might be worthwhile to take a step back and look at the big picture of...
Aug 3, 2014 Linear Discriminant Analysis – Bit by Bit Step-by-step Linear Discriminant Analysis tutorial covering scatter matrices, eigenvectors, class separation, dimensionality reduction, and Python code.
Jul 18, 2014 Dixon's Q test for outlier identification – A questionable practice I recently faced the impossible task to identify outliers in a dataset with very, very small sample sizes and Dixon's Q test caught my attention. Honestly...
Jul 11, 2014 About Feature Scaling and Normalization – and the effect of standardization for machine learning algorithms I received a couple of questions in response to my previous article (Entry point: Data) where people asked me why I used Z-score standardization as feature...
Jun 27, 2014 Entry Point Data – Using Python's sci-packages to prepare data for Machine Learning tasks and other data analyses Overview of Python tools for data entry and pattern recognition workflows — file I/O, CSV handling, argparse, and command-line scripting best practices.
Jun 26, 2014 Molecular docking, estimating free energies of binding, and AutoDock's semi-empirical force field Discussions and questions about methods, approaches, and tools for estimating (relative) binding free energies of protein-ligand complexes are quite...
Jun 20, 2014 An introduction to parallel programming using Python's multiprocessing module – using Python's multiprocessing module Parallel programming in Python with multiprocessing: bypassing the GIL, spawning processes, using Pool.map, and speeding up CPU-bound workloads.
Jun 19, 2014 Kernel density estimation via the Parzen-Rosenblatt window method – explained using Python The Parzen-window method (also known as Parzen-Rosenblatt window method) is a widely used non-parametric approach to estimate a probability density function...
Jun 19, 2014 Numeric matrix manipulation – The cheat sheet for MATLAB, Python NumPy, R, and Julia At its core, this article is about a simple cheat sheet for basic operations on numeric matrices, which can be very useful if you working and experimenting...
Jun 1, 2014 The key differences between Python 2.7.x and Python 3.x with examples Many beginning Python users are wondering with which version of Python they should start. My answer to this question is usually something along the lines...
May 28, 2014 5 simple steps for converting Markdown documents into HTML and adding Python syntax highlighting 5-step tutorial to add code syntax highlighting to Markdown blog posts using Python-Markdown, Pygments, and custom CSS stylesheets.
May 20, 2014 Creating a table of contents with internal links in IPython Notebooks and Markdown documents Many people have asked me how I create the table of contents with internal links for my IPython Notebooks and Markdown documents on GitHub. Well, no...
May 12, 2014 A Beginner's Guide to Python's Namespaces, Scope Resolution, and the LEGB Rule A short tutorial about Python's namespaces and the scope resolution for variable names using the LEGB-rule with little quiz-like exercises.
Apr 21, 2014 Diving deep into Python – the not-so-obvious language parts Some while ago, I started to collect some of the not-so-obvious things I encountered when I was coding in Python. I thought that it was worthwhile sharing...
Apr 13, 2014 Implementing a Principal Component Analysis (PCA) – in Python, step by step Implementing Principal Component Analysis from scratch in Python: scatter matrices, eigenvectors, variance explained, and comparison with scikit-learn PCA.
Mar 13, 2014 Installing Scientific Packages for Python3 on MacOS 10.9 Mavericks I just went through some pain (again) when I wanted to install some of Python's scientific libraries on my second Mac. I summarized the setup and...
Mar 7, 2014 A thorough guide to SQLite database operations in Python Complete SQLite guide for Python: creating databases, running queries, bulk inserts, pandas integration, and tips for working with large datasets.
Feb 23, 2014 Using OpenEye software for substructure alignments and best-matching low-energy conformer overlays This is a quickguide showing how to use OpenEye software command line tools to align target molecules to a query based on substructure matches and how to...

2013

Dec 14, 2013 Unit testing in Python – Why we want to make it a habit Let’s be honest, code testing is everything but a joyful task. However, a good unit testing framework makes this process as smooth as possible. Eventually...
Dec 8, 2013 A short tutorial for decent heat maps in R I received many questions from people who want to quickly visualize their data via heat maps - ideally as quickly as possible. This is the major issue of...
Nov 3, 2013 SQLite – Working with large data sets in Python effectively My new project confronted me with the task of screening a massive set of large data files in text format with billions of entries each. I will have to...