News 2024
Nov 21, 2024
I recently had the pleasure of joining Neil to discuss a wide range of topics on AI Stories. It was a fun chat, and our conversation spanned lots of (hopefully interesting) topics:
- Life in both academia and industry
- LLM training and cloud computing
- The ins and outs of implementing and building LLMs from scratch
- Transitioning from GPT architectures to Llama 3.2
- Long-context LLMs vs Retrieval-Augmented Generation (RAG)
- How multimodal LLMs work
Sep 16, 2024
After 1.5 years of hard work, "Build A Large Language Model (From Scratch)" is finally published! Print and ebook copies are available on Manning's website. And the book is also available on Amazon.
July 31, 2024
II'm excited for my first PyTorch conference! Can't wait to meet the community and chat about the latest in AI and LLM developments!
May 17, 2024
I'll be giving a 1-hour ACM Tech Talk on June on "Understanding the LLM Development Cycle: Building, Training, and Finetuning". This talk will guide you through the key stages of developing large language models (LLMs), from initial coding to deployment. This talk is virtual, and you can register for free here.
May 15, 2024
Last week, I sat down with Hugo Bowne-Anderson from the Vanishing Gradient to record a podcast all about LLMs. We ended up covering the entire LLM lifecycle, what type of skills you need to work with them, what type of resources and hardware, prompt engineering vs finetuning vs RAG, and more! (Plus, I gave a 30 min live demo finetuning an LLM for classification). You can find a link to the podcast here and a YouTube video version here.
Mar 25, 2024
I somehow made it to GitHub's top-1 spot on the Trending Developers list. I've been coding on GitHub quite consistently for about 12 years, but I honestly never expected to find myself up there! As a passionate coder and open-source developer, this is perhaps the nicest compliment for me so far!
Mar 19, 2024
I had the pleasure of sitting down with Jon Krohns on the
SuperDataScience podcast
for what turned into a roughly 2-hour mega interview on LLMs. We covered pretty much everything!
- Developing efficient open-source codebases for finetuning and pretraining LLMs
- A developer-friendly codebase for experimenting with LLM research ideas
- Accelerating PyTorch code with Fabric
- Scaling research: Running multi-GPU and multi-node AI experiments with Lightning Studios
- LLM architecture deep dives: OLMo and Gemma LLMs
- From 70B to 3B; and generalized vs specialized LLMs
- LoRA vs DORA: Efficient LLM finetuning
- Writing books about LLMs
Feb 8, 2024
I'll be giving a 3.5 hour deep learning workshop at PyCon 2024 in May. This tutorial is aimed at Python
programmers new to PyTorch and deep learning. However, even more experienced
deep learning practitioners and PyTorch users may be exposed to new concepts
and ideas when exploring other open source libraries to extend PyTorch.
It's my first PyCon, and I'm very excited!
All News: 2024 ● 2023 ● 2022 ● 2021 ● 2020 ● 2019