# Week 23: Exercises & Checklist

## Graded exercises

1. **Easy**: Run `notebooks/01-mlflow-feature-engineering-training.ipynb` to completion. It prints the champion and challenger test MAE and the final `@prod`/`@challenger` version mapping. Record all four in the Week 23 tracker sheet.

2. **Standard**: In `notebooks/02-vector-search-rag.ipynb`, add a fifth policy document to the inline corpus, re-embed it, re-sync the AI Search index, and confirm a matching question now retrieves the new document in the top results. Note the new groundedness score next to the old one.

3. **Stretch**: In `01-mlflow-feature-engineering-training.ipynb`, train a variant that drops the time-series `FeatureLookup` (static features only). Quantify the test-MAE degradation and write one sentence on what the point-in-time feature is worth, the measured value of preventing leakage.

4. **Portfolio**: Advance the **ZoroLogistics point-in-time ETA model** milestone: serve the `@prod` model on a Model Serving endpoint, call it with a sample feature vector, and record the prediction plus latency for your portfolio.

## Hints

1. **Easy**: Run notebook 01 to the final cell; record the two MAEs, the
   `@prod`/`@challenger` mapping, and the `point-in-time gain`. If autologging emits
   warnings, they are safe, the metrics still land on the run.
2. **Standard**: Add your fifth doc to the inline `docs` list *before* the
   `createDataFrame` write, re-run embed → index (`pipeline_type="TRIGGERED"` needs a
   sync) → query, and compare the new groundedness score to the old one.
3. **Stretch**: Reuse the challenger path in notebook 01 but drop the *entire*
   point-in-time lookup from `create_training_set`; the test-MAE delta against the
   champion is the measured value of preventing leakage.
4. **Portfolio**: Serve `@prod` with a scale-to-zero endpoint, call it with
   `dataframe_records` (the same feature dict the model signature expects), and record
   the prediction plus cold/warm latency.

## Checklist (mirrors manifest.json + Excel tracker)

- [ ] Mon: MLflow + feature tables: log a baseline; build point-in-time features.
- [ ] Tue: Train the ETA model (XGBoost or PyTorch); register it in MLflow; compare challenger.
- [ ] Wed: Create the serving endpoint + AI Gateway rate limit; call from Python.
- [ ] Thu: Vector Search: index policy docs; build the RAG chain with ai_query.
- [ ] Fri: Use case: Genie space on the gold tables; verify its SQL; publish it.
- [ ] Sat: Take the Week 23 quiz (quiz.md), pass with 8/10; record the score in Notes.
- [ ] Milestone: Update the Excel tracker; commit the ML assets.