W

Warmup

• Category: Math & training
• What it actually means: An initial training phase in which the learning rate rises from a smaller value toward the main schedule's target value.
• Why it matters: Early gradients and optimizer statistics can be unstable, especially in large-batch or transformer training, so abrupt full-size updates may damage optimization.
• In practice: Define warmup in steps or processed tokens, log the realized curve, and tune it with the batch, optimizer, and total training budget held visible.
• Common confusion: Warmup is not required for every model and does not make an otherwise unsuitable learning rate safe.
• Related terms: Learning Rate Schedule, Learning Rate, Batch Size, AdamW
• Sources: Accurate, Large Minibatch SGD

Weight

• Category: Math & training
• What people say: A learned number inside a model.
• What it actually means: A trainable coefficient in a model transformation. Weights are usually organized into tensors, and optimization adjusts them to reduce the training objective.
• Common confusion: Not every parameter is called a weight; biases, embeddings, and normalization scales are parameters too.
• Related terms: Parameter, Tensor, Optimizer

Weight Decay

• Category: Math & training
• What people say: Regularization that shrinks weights during optimization.
• What it actually means: An update rule that reduces selected parameter magnitudes over training, often by multiplying weights by a shrinkage factor separate from the gradient update.
• Why it matters: It can improve generalization, but the useful coefficient and excluded parameter groups depend on model, optimizer, schedule, and data.
• Common confusion: Decoupled weight decay is equivalent to an L2 loss penalty for some simple optimizers, but not generally for adaptive optimizers such as Adam.
• Related terms: AdamW, Overfitting, Optimizer

Worktree

• Category: AI-native development
• What it actually means: In Git, a working directory attached to a repository and branch or commit, with shared object storage but its own checked-out files and index.
• Why it matters: Separate worktrees let people and agents work concurrently without constantly switching or overwriting one checkout.
• In practice: Give each coding agent a named feature branch and exact worktree path, then review and integrate patches through normal Git history.
• Common confusion: A worktree isolates checked-out files, not every process, port, cache, database, or secret on the machine.
• Learn it: Workbench for Real Repositories
• Sources: git-worktree documentation
• Related terms: Coding Agent, Patch, Scope Contract, Handoff