Correction for Listing 6.5 in Build a Reasoning Model From Scratch
Thanks everyone for all the kind words and feedback. Super happy that you are enjoying Build a Reasoning Model (From Scratch)!
Unfortunately, there’s a small typo in Listing 6.5 on page 198 (see the video below). The line torch.manual_seed(0) should be torch.manual_seed(5).
This is a tiny change, but it’s needed to reproduce the generated response in Listing 6.5 and the corresponding log-probability outputs later in Chapter 6. If you use 0, the code is of course still correct but the numeric results may be a bit different, which could be confusing. Anyway, it should be a small 1 character change :).
This will be fixed in the next printing.
I am sorry about the oversight, and I hope this note saves you some debugging time.
Source: lightly edited website version of my Substack note.
Read Next
Inkling: A New Open-Weight 975B MoE with a Few Surprises
Short note on Thinking Machines Lab's 975B Inkling open-weight model, its benchmark profile, sparse MoE design, short convolutions, embedding RMSNorm, and
200,000 Subscribers
Short note celebrating Ahead of AI reaching 200,000 subscribers.
GPT 5.6 Has 72 Possible Configurations. What's A Good Default?
Short note on how GPT 5.6 model and effort choices map onto training-time and inference-time scaling, producing 72 configurations.