Week 03: Exercises & Checklist

Graded exercises

1. Easy: Run notebooks/01-eta-regression-baseline.ipynb to completion. It prints a final comparison of MAE/RMSE for the mean/median baseline and three regressors. Record the best test MAE in the Week 3 tracker sheet.

2. Standard: Add a fourth regressor (for example HistGradientBoostingRegressor) using the same time-aware split and features. Report its MAE/RMSE in the same comparison table and state, in one line, whether it beats the baseline and by how much (the delta).

3. Stretch: Replace the single validation split with TimeSeriesSplit from scikit-learn and cross-validate your best model, reporting mean ± standard deviation of MAE across the folds. Add a markdown cell explaining why a time-ordered cross-validation estimate is fairer than one random split.

4. Portfolio: Advance the ETA prediction (ML → DL) with model cards milestone (see curriculum/projects/README.md): commit the Week 3 model card and an experiments.json log recording every run's model, split dates, seed, and metrics, the raw material the Week 4 neural model must beat.

Hints

1. Easy: Record the baseline and each model's MAE/RMSE from the printed comparison table; the "best" row is the one with the lowest test MAE, and it may not be a model at all.
2. Standard: Add HistGradientBoostingRegressor to the same models dict and keep the identical Pipeline([("prep", prep), ("model", ...)]) wrapper so the only thing that changes is the estimator.
3. Stretch: TimeSeriesSplit produces ordered train/test index pairs; iterate them with your pipeline and average the fold MAEs, and note that you still never shuffle inside a fold.
4. Portfolio: In experiments.json, store the split as the two cut dates (CUT1, CUT2) rather than a percentage, so the log is reproducible, a "80/20" string is not enough for a stranger to rebuild your split.

Checklist (mirrors manifest.json + Excel tracker)

• [ ] Mon: Study supervised learning and the eval mindset: pick the metric before the model.
• [ ] Tue: Run the ETA regression baseline; log MAE/RMSE for 3 models in a comparison table.
• [ ] Wed: Implement a time-aware split and cross-validation; compare models fairly.
• [ ] Thu: On-time classification: precision/recall/F1, threshold tradeoff, confusion matrix.
• [ ] Fri: Use case: write the Week 3 model card (metric, split, top error source) and commit it.
• [ ] Sat: Take the Week 3 quiz (quiz.md), pass with 8/10; record the score in Notes.
• [ ] Milestone: Update the Excel tracker; log all experiments to a JSON experiment log.