Week 05: Quiz (10 questions, 8/10 to pass)

Answer from the concepts and the notebook code. Each question notes where to find it.

1. (MCQ) An LLM "answering" a question is best described as: (a) retrieving the closest fact from a knowledge base, (b) sampling the next token from a learned distribution, repeatedly, (c) executing a reasoning plan stored between calls, (d) looking up the prompt in a cache. (see Concepts §"Tokenization is the front door")

2. (Short answer) In 01-tokenization-lab.ipynb, why does enc.encode("ZoroLogistics") return 4 tokens while enc.encode("shipment") returns 1? (see notebook cell 6)

3. (MCQ) The causal mask exists so that: (a) attention runs faster, (b) a token cannot attend to future positions, (c) embeddings stay normalized, (d) the softmax doesn't overflow. (see Concepts §"Self-attention")

4. (Short answer) Write the formula estimate_cost uses in notebook 1, and compute the per-call cost for 57 input tokens and 60 output tokens at $1.00/$3.00 per Mtok (to the nearest $0.000001). (see notebook cell 10)

5. (MCQ) In the notebook's chars/token table, a bill of lading tokenizes at about: (a) 5.3 chars/token, (b) 3.0 chars/token, (c) 4.0 chars/token, (d) 1.0 chars/token. (see notebook cell 8 / Concepts table)

6. (Short answer) In 02-embeddings-and-attention-lab.ipynb, what is the correct top-1 commodity for the query "temperature-sensitive medical cargo that must stay cold", and why does cosine similarity rank it first? (see notebook cells 2 & 6)

7. (MCQ) The KV cache turns generation from O(n²) to O(n) per step, at the cost of: (a) more compute, (b) linearly growing memory with sequence length, (c) losing the causal mask, (d) lower temperature. (see Concepts §"The KV cache")

8. (Short answer) For a 7B-class model (32 layers, 32 heads, head_dim 128) at 4096 context and FP16, compute the KV-cache size in GB using 2 × layers × heads × head_dim × seq_len × bytes_per_param. (see Concepts Worked example 2)

9. (MCQ) Which knowledge-acquisition mechanism changes the model's weights? (a) zero-shot ICL, (b) RAG retrieval, (c) fine-tuning, (d) prompt caching. (see Concepts table)

10. (Short answer) In notebook 1 cell 12, why is the daily cost over 100,000 notes computed from the average note token count rather than the first note's count? (see notebook cells 11 to 12)

Answer key

1. (b): the model samples the next token from a learned distribution, repeatedly; "answering" is that loop run until EOS/max_tokens.
2. BPE builds its vocabulary by merging frequent character pairs, so the common word "shipment" became a single token while the rare brand name "ZoroLogistics" was never merged and splits into subword pieces (Z/oro/Log/istics).
3. (b): the causal (lower-triangular) mask zeroes out future positions so a token attends only to itself and earlier tokens, which is also what makes incremental generation possible.
4. (input_tokens/1e6)·price_in + (output_tokens/1e6)·price_out → 57/1e6·1.00 + 60/1e6·3.00 = 0.000057 + 0.000180 = $0.000237.
5. (b): the BoL is ~3.0 chars/token (364 chars, 123 tokens) because its IDs, codes, and numeric fields split into subwords.
6. pharmaceuticals: "pharmaceuticals and medical supplies needing cold-chain handling" is semantically closest to "temperature-sensitive medical cargo," so its embedding vector is most aligned (highest cosine) with the query vector.
7. (b): caching past K/V makes each step O(n) but the cache grows linearly (2·L·H·hd·seq·bpp), which is the dominant memory cost of long-context serving.
8. 2 × 32 × 32 × 128 × 4096 × 2 bytes = 2,147,483,648 bytes ≈ 2.15 GB.
9. (c): fine-tuning changes the weights; ICL and RAG leave them untouched, and caching is a serving optimization.
10. The daily figure must represent the typical note, not the first one; the first note is longer than average (41 tokens vs. ~30), so using it alone would overstate the daily cost.