Developing an LLM: Building, Training, Finetuning
A Deep Dive into the Lifecycle of LLM Development
If your weekend plans include catching up on AI developments and understanding Large Language Models (LLMs), I’ve prepared a 1-hour presentation on the development cycle of LLMs, covering everything from architectural implementation to the finetuning stages.
The presentation also includes and overview and discussion of the different ways LLMs are evaluated, along with the caveats of each method.
Below, you’ll find a table of contents with links to specific segments of the video, allowing you to jump directly to topics of interest:
- Using Large Language Models
- Stages of Developing an LLM
- Dataset Considerations
- Multi-Word Output Generation
- Tokenization Explained
- Pretraining Datasets
- Architecture of LLMs
- Pretraining Techniques
- Finetuning for Classification
- Instruction Finetuning
- Preference Finetuning
- Evaluating Large Language Models
- Rules of Thumb for Pretraining and Finetuning
It’s a slight departure from my usual text-based content, but if you find this format useful and informative, I might occasionally create and share more of them in the future.
Happy viewing!