O

Observability

• Category: AI-native development
• What it actually means: The ability to understand an AI system's behavior from recorded inputs, outputs, state transitions, tool calls, timings, costs, errors, and evaluation signals.
• Why it matters: AI failures often span model, retrieval, tools, and orchestration. You need correlated evidence to locate the failing boundary.
• In practice: Record a trace ID across retrieval, model calls, tool execution, approvals, and final scoring while applying redaction and access controls.
• Common confusion: Logging collects events. Observability makes those events structured and connected enough to answer operational questions.
• Learn it: Agent Observability Platforms
• Related terms: Trace, Evaluation (Eval), Agent State, Time to First Token (TTFT)

Optimizer

• Category: Math & training
• What people say: The algorithm that updates weights.
• What it actually means: An algorithm that transforms gradients into parameter updates. Plain stochastic gradient descent is a simple baseline; momentum, Adam, and other optimizers change the update using history or adaptive scaling. Each choice has different memory, stability, and tuning behavior.
• Common confusion: The optimizer consumes gradients; backpropagation computes them.
• Related terms: Adam (Optimizer), AdamW, Gradient, Learning Rate

Orchestration

• Category: Agents & tools
• What it actually means: The control logic that sequences, branches, delegates, retries, pauses, resumes, and terminates work across model and tool steps.
• Why it matters: Reliable agent behavior depends on explicit workflow decisions outside the model, especially when tasks have dependencies or consequential side effects.
• In practice: Encode stable steps as a workflow or state machine, expose bounded decisions to the model, and persist transitions before external writes.
• Common confusion: Orchestration is not synonymous with autonomy or multi-agent systems; one agent can be orchestrated through a deterministic workflow.
• Related terms: Agent Harness, Planning, Delegation, Durable Execution
• Sources: Building Effective Agents

Overfitting

• Category: Math & training
• What people say: The model memorized the training data.
• What it actually means: A generalization gap in which performance on training data is substantially better than performance on representative unseen data. Memorization can contribute, but the operational symptom is poor generalization.
• In practice: Compare training and held-out metrics, inspect subgroup failures, and test changes such as data quality, regularization, early stopping, or model capacity.
• Related terms: Underfitting, Dropout, Weight Decay, Eval Set