LLMs from Scratch

This repository contains the code for developing, pretraining, and finetuning a GPT-like LLM and is the official code repository for the book Build a Large Language Model (From Scratch).

课时

1. Appendix A: Introduction to PyTorch
2. LLMs from Scratch
3. Appendix A: Introduction to PyTorch
4. Appendix C: Exercise Solutions
5. Appendix D: Adding Bells and Whistles to the Training Loop
6. Appendix E: Parameter-efficient Finetuning with LoRA
7. Recommendations for Getting the Most Out of a Technical Book
8. Chapter 2: Working with Text Data
9. Chapter 2: Working with Text Data
10. Chapter 2: Working with Text Data
11. Chapter 2: Working with Text Data
12. Byte Pair Encoding (BPE) Tokenizer From Scratch
13. SimpleTokenizerV3
14. Chapter 2: Working with Text Data
15. Chapter 3: Coding Attention Mechanisms
16. More Efficient Multi-Head Attention Implementations
17. Understanding PyTorch Buffers
18. Chapter 3: Coding Attention Mechanisms
19. Chapter 4: Implementing a GPT Model from Scratch To Generate Text
20. Chapter 4: Implementing a GPT Model from Scratch To Generate Text
21. Bonus Material: KV Cache

开始学习 → Appendix A: Introduction to PyTorch