Week 23: Quiz (10 questions, 8/10 to pass)

Answer each, then check the answer key. Each question ends with a pointer to the Concepts section (§) or notebook cell it tests.

1. (MCQ) Why are the old MLflow Registry "Staging/Production" stages deprecated in favor of aliases? (see Concepts §aliases)

   • A) Aliases are faster to query
   • B) Aliases are mutable, so you can hold several named references (Champion, Challenger, @prod) and promote by re-pointing a pointer
   • C) Stages never worked with XGBoost
   • D) Aliases require a model signature

2. (Short answer) In 01-mlflow-feature-engineering-training.ipynb, the carrier_daily_stats table declares timeseries_columns=["metric_date"] and the lookup sets timestamp_lookup_key="planned_departure". What would leak into training if you removed the timestamp key and joined the daily table directly on carrier_id? (see Concepts §feature engineering; notebook 01 cell 9)

3. (MCQ) Which AI Search search mode fuses vector similarity with keyword (BM25) matching via Reciprocal Rank Fusion? (see Concepts §AI Search)

   • A) ANN
   • B) semantic
   • C) hybrid
   • D) direct upload

4. (MCQ) AI functions like ai_classify and ai_extract require which runtime environment? (see Concepts §AI functions)

   • A) Any classic all-purpose cluster
   • B) Serverless compute and DBR 18.2+
   • C) A dedicated GPU cluster only
   • D) Pro SQL warehouse with Photon

5. (Short answer) Why does the notebook compute groundedness as the fraction of the answer's tokens that appear in the retrieved context, rather than just printing "the answer looks good"? (see Concepts §worked RAG example; notebook 02 cell 13)

6. (MCQ) In the champion/challenger comparison, the challenger is trained after dropping the daily_* columns. What does a positive champion_MAE < challenger_MAE (i.e., a positive "point-in-time gain") tell you? (see Concepts §worked example; notebook 01 final cell)

   • A) The time-series feature is hurting the model
   • B) The time-series feature adds predictive value beyond static features
   • C) The model is overfitting
   • D) The training set is too small

7. (MCQ) You want a low-traffic ETA endpoint that costs $0 while idle. Which serving setting is correct? (see Concepts §Serving)

   • A) min_provisioned_throughput > 0
   • B) scale_to_zero_enabled: true
   • C) provisioned throughput
   • D) an external-model endpoint

8. (Short answer) What is the difference between ai_mask (an AI function) and a Unity Catalog column mask? (see Concepts §AI functions; How it breaks)

9. (MCQ) Genie Agents cannot exceed the caller's grants because… (see Concepts §Genie)

   • A) Genie only runs pre-written SQL
   • B) Genie's SQL runs as the viewer against tables they can already read
   • C) Genie disables all SQL functions
   • D) Genie routes through an external model

10. (MCQ) In 02-vector-search-rag.ipynb, the Delta Sync index uses embedding_vector_column="embedding" (self-managed). What is the managed alternative? (see Concepts §AI Search; notebook 02 cell 5)

• A) Provide embedding_source_column + an embedding_model_endpoint_name and let the index embed
• B) Upload vectors one by one with upsert
• C) There is no managed alternative
• D) Store embeddings in a CSV

Answer key

1. B. Aliases are mutable named references, so you can hold Champion, Challenger, and @prod simultaneously and promote by re-pointing one alias, stages forced a single linear state.

2. Future carrier statistics would leak. Joining carrier_daily_stats on carrier_id alone pulls the carrier's latest stats (possibly after the shipment departed); the timestamp key makes the join AS OF planned_departure, so only history before departure is used.

3. C: hybrid. Hybrid combines ANN vector search with BM25 keyword search via Reciprocal Rank Fusion.

4. B: serverless compute and DBR 18.2+. AI functions will not run on Pro/Classic warehouses.

5. Groundedness must be measured, not asserted. Token overlap is a cheap, reproducible proxy that checks whether the answer is built from retrieved text rather than the model's prior, an answer that sounds right but invents a clause would score low.

6. B. The champion (with the AS-OF time-series feature) beats the static-only challenger, so the point-in-time feature carries predictive signal; the gain is that measured margin.

7. B: scale_to_zero_enabled: true (with min_provisioned_throughput: 0) lets the endpoint scale to zero replicas while idle, costing nothing until a request cold-starts it.

8. ai_mask is a content transform: it rewrites free text once (redacting named PII entities) as a batch operation. A column mask is a deterministic governance policy enforced at query time that controls who sees a column's value, not a text rewrite.

9. B. Genie's generated SQL executes as the viewer with the viewer's Unity Catalog grants, so it cannot read tables or see unmasked values the caller cannot.

10. A. Managed embeddings let Databricks compute vectors from a text column using an embedding_model_endpoint_name; self-managed means you pre-computed and stored the embedding column yourself.