Hi, I’m Sebastian, and I work as an LLM Research Engineer. Most people know me from my books on machine learning and large language models (LLMs).

Most of the LLM content I share is motivated by something I am personally curious about. I read the paper and look at the config or reference implementation. Then, I often I build a smaller version in PyTorch. This is often a fairly manual process, and I like it that way since it’s a good learning experience. Working code doesn’t lie, and it usually requires details that are easy to miss in a diagram.

Lately, this has meant spending most of my time on LLM architectures and some (small-scale) training. I am super interested in inference and evaluation as well, though.

Before moving to industry full time, I was a professor in the Department of Statistics at the University of Wisconsin-Madison. I taught statistics, machine learning, and deep learning there and did research on deep neural networks. Later, I worked as a senior engineer at Lightning AI. My publications page lists my academic work, and LinkedIn has my current CV details.

Where to start

For a bottom-up introduction to LLMs, Build a Large Language Model (From Scratch) builds a GPT-style model and then walks through pre-training and fine-tuning it in PyTorch.

Build a Reasoning Model (From Scratch), which is essentially a sequel to the one above, begins with a pre-trained model and covers inference-time scaling, reinforcement learning, and installation. (My other books are listed on the books page.)

For various short- and long-form articles covering LLMs, deep learning, and more, see the blog and notes archive or the Machine Learning FAQ. The course pages collect lectures and code from current and past courses. I also maintain a Start Here page and a shorter list of favorite (longer-form) articles for readers who would rather browse by topic.

Elsewhere

Most of my open-source code is on GitHub. Google Scholar has the citation records for my papers. I publish longer articles on my Substack blog. Recorded talks and courses are on YouTube, while shorter updates are on X.