# Week 10: Quiz (10 questions, 8/10 to pass)

Answer on your own first, then check the answer key. Each question notes where the
answer lives, so you can re-read the exact section or cell you missed.

## Questions

1. **(MCQ)** What is Ng's selection order, and where does fine-tuning sit? (see Concepts §"the last lever")
   - A) Fine-tune → RAG → prompt → agentic
   - B) Prompt (with eval) → RAG → agentic → fine-tune last
   - C) RAG → fine-tune → prompt → agentic
   - D) Fine-tune first, because weights always beat context

2. **(MCQ)** For a LoRA module of shape `(in=1536, out=1536)` with rank `r=16`, how many trainable parameters does LoRA add to that one module? (see Concepts §Worked example 1)
   - A) 1536
   - B) 49,152
   - C) 700,416
   - D) 1.54 billion

3. **(Short answer)** In the LoRA worked example, the total trainable parameters come to ~0.70M of ~1.54B, about 0.05%. Why does that tiny fraction matter for the budget? (see Concepts §Worked example 1)

4. **(MCQ)** In `notebooks/01-sft-dataset-and-lora-training.ipynb`, what is the assistant turn of each extraction example? (see notebook cell 6)
   - A) A one-word category
   - B) The ground-truth JSON of `bol["fields"]`
   - C) An empty string
   - D) The system prompt

5. **(Short answer)** Why does the knowledge base say "a few hundred clean, diverse, on-task examples beat tens of thousands of noisy ones"? (see Concepts §"SFT: imitation learning")

6. **(MCQ)** What distinguishes DPO from classic RLHF? (see Concepts §"SFT → DPO → RLVR")
   - A) DPO needs a separately trained reward model
   - B) DPO folds the preference signal directly into a supervised objective on `(prompt, chosen, rejected)` pairs
   - C) DPO requires a verifiable reward like a unit test
   - D) DPO freezes the entire model and trains nothing

7. **(MCQ)** In `notebooks/02-dpo-and-before-after-evals.ipynb`, what is the "forgetting check"? (see notebook cell 6)
   - A) A held-out set of 5 generic questions the model should already know, graded before and after
   - B) A test that the adapter still fits in VRAM
   - C) A check that the loss curve decreased
   - D) A comparison of two different base models

8. **(Short answer)** Compute the net gain: before = {extraction 0.60, triage 0.72, generic 0.95}; after = {extraction 0.88, triage 0.91, generic 0.93}. What is `net = extraction_delta + triage_delta − forgetting_cost`, and does it deploy? (see Concepts §Worked example 2)

9. **(MCQ)** Your fine-tune raises the new-task eval but drops the generic check from 0.95 to 0.80. What is the correct action per the "How it breaks" section? (see Concepts §"How it breaks")
   - A) Ship it, the specialist gain is what matters
   - B) Don't deploy; reduce rank/alpha or add a fraction of general data back
   - C) Add more epochs
   - D) Ignore the generic check

10. **(MCQ)** Why must facts that change (e.g., last quarter's prices) *not* be stored in weights? (see Concepts §"When fine-tuning pays")
    - A) Weights cannot hold numbers
    - B) Retrieved knowledge is wrong for volatile facts, retrieve, don't memorize
    - C) Fine-tuning always deletes old facts
    - D) Facts are too small to matter

## Answer key

1. **B)** The order is managed-API prototype → prompt/context with an eval → RAG → agentic → fine-tune last. Fine-tuning is the final, most expensive, hardest-to-undo lever.

2. **B)** LoRA adds `r × (in + out) = 16 × (1536 + 1536) = 49,152` per module. That is one attention projection; the full `q/k/v/o` set is `4 × 49,152 = 196,608`.

3. **Because LoRA trains almost nothing, the adapter is a few MB and the fine-tune needs a fraction of full-training memory.** QLoRA puts the base in 4-bit, so a 1.5B (or 7 to 8B) run fits a free Colab T4 or a 16 GB laptop instead of a datacenter.

4. **B)** The assistant turn is `json.dumps(bol["fields"], sort_keys=True)`, the ground-truth structured fields the model learns to imitate (the label).

5. **A small model internalizes whatever pattern dominates the data.** If most examples are off-target or contradictory, that is exactly what you teach. Coverage of *behavior* (diversity, edge cases) matters more than row count.

6. **B)** DPO folds the preference signal directly into a supervised-style objective on `(prompt, chosen, rejected)` pairs, simpler and more stable than RLHF's reward model + PPO loop. (RLVR, not DPO, uses a verifiable reward.)

7. **A)** The forgetting check is the 5 generic questions (`GENERIC_QA`) graded before and after, it catches the adapter un-teaching general competence.

8. **net = (0.88−0.60) + (0.91−0.72) − (0.95−0.93) = 0.28 + 0.19 − 0.02 = +0.45.** Positive, so the specialist gains earned the tiny generic regression → deploy. (If the forgetting cost exceeded the gains, the answer would be don't deploy.)

9. **B)** A generic drop that large means catastrophic forgetting. Don't ship; lower the learning rate, add a small fraction of general data back, or reduce rank/alpha, never add epochs.

10. **B)** Weights are the wrong place for facts that change; the model would already be wrong next quarter. New knowledge is RAG's job, retrieve it, and use fine-tuning for *behavior* (style, format, vocabulary).