# Regularization Tuning Campaign

**Status**: Active

**Parent Master**: `765a36b0700b3a20d552f48b8ca2b75636aa3e69`

**Theme**: Systematic exploration of regularization hyperparameters to improve validation bpb.

## Experiments

### embedding-wd-001 (CANCELLED)
- **Hypothesis**: Increase embedding weight decay from 0.0005 to 0.001 to test whether stronger regularization on token embeddings improves validation bpb.
- **Change**: Embedding optimizer group `weight_decay = 0.001` (was 0.0005)
- **Result**: Job cancelled due to hanging at step 02348 (97.6%) during final evaluation phase.
- **Conclusion**: Hypothesis remains untested due to infrastructure issues.

## Notes

- The embedding weight decay experiment encountered a job hang during evaluation, not during training.
- This suggests the issue may be related to the evaluation phase or infrastructure rather than the training configuration itself.
- Consider retrying this experiment or investigating the evaluation phase bottleneck.