A personal, implementation-oriented record of Stanford CS336: Language Modeling from Scratch.
This repository has two equal parts:
- Feishu knowledge base: the primary lecture notes, explanations, derivations, supplements, and errata.
- Assignments: implementations, experiments, tests, and progress records. Assignment work is intentionally marked by status; unfinished work is not presented as completed.
- Open the Feishu notes index to read the original knowledge-base documents.
- Read the editorial review to understand the current structure and the cleanup rules.
- Check assignment progress before treating any implementation or result as final.
| Lectures | Focus | Practical output |
|---|---|---|
| 1-4 | Tokenization, Transformer architecture, attention alternatives, and MoE | Assignment 1 foundations |
| 5-8 | GPU performance, kernels, communication, and parallel training | Benchmarks and distributed-training notes |
| 9-11 | Scaling laws, initialization, optimization, and training recipes | Scaling experiments and analysis |
| 12-14 | Evaluation and data construction | Evaluation reports and data-pipeline notes |
| 15-16 | SFT, RLHF, RLVR, and reasoning models | Post-training reading notes |
| 17 | Multimodal models | Vision-language system notes |
- The Feishu knowledge base is the source of truth for lecture explanations.
- GitHub provides navigation, version history, implementation code, experiment records, and carefully selected mirrors or links.
- Main lectures, supplementary explanations, assignment work, and errata remain visibly separate.
- Claims about modern models, benchmarks, hardware, and legal or licensing matters must include a source or a clear scope qualifier.
- No private tokens, credentials, course PDFs, copied slides, or unlicensed third-party assets belong in this repository.
This is an independent study repository. Stanford CS336, the official lecture materials, and assignment specifications belong to their respective authors and Stanford University. The official course repository is linked for reference; this repository is not an official Stanford release.
Original code is released under MIT. Original prose and diagrams are released under CC BY-NC-SA 4.0. Third-party materials remain under their original licenses.