E

Early Fusion

• Category: Multimodal systems
• What it actually means: Combining raw or low-level representations from several modalities before most task-specific modeling occurs.
• Why it matters: Early interaction can expose fine-grained cross-modal relationships, but it also requires compatible representations and careful handling of alignment and missing inputs.
• In practice: Convert each modality into a declared token or feature representation, preserve source and position markers, fuse them before the shared backbone, and compare against single-modality and late-fusion baselines.
• Common confusion: Early fusion describes where streams are combined in the architecture. It does not guarantee that the model learns useful alignment between them.
• Learn it: Chameleon Early-Fusion Tokens
• Related terms: Late Fusion, Multimodal Fusion, Modality Alignment, Token
• Sources: Chameleon: Mixed-Modal Early-Fusion Foundation Models; Multimodal Machine Learning: A Survey and Taxonomy

Eigenvalue

• Category: Math & training
• What people say: A matrix property used in PCA.
• What it actually means: A scalar that describes how a linear transformation scales a corresponding nonzero eigenvector without changing its direction. In covariance-matrix PCA, larger eigenvalues correspond to directions with more variance.
• Related terms: Tensor, Feature, Latent Space

Embedding

• Category: Data & representations
• What people say: A vector that represents meaning.
• What it actually means: A learned mapping from discrete items (words, images, users) to dense vectors in continuous space, where similar items end up close together
• Common confusion: Similarity depends on the model, training objective, and metric. Distance in one embedding space does not carry over to another.
• Why it's called that: The items are placed, or embedded, in a geometric representation space.
• Learn it: Embeddings
• Related terms: Cosine Similarity, Semantic Search, Vector Database

Encoder

• Category: Models & inference
• What people say: The input side of a model.
• What it actually means: A component that transforms input into a representation. A transformer encoder commonly uses non-causal self-attention, subject to any masks, so each position can incorporate context from across the input.
• Common confusion: Encoder-only models can produce outputs through task heads even though they are not typically used for autoregressive text generation.
• Related terms: Decoder, Transformer, Embedding

Epoch

• Category: Math & training
• What people say: One pass through the training data.
• What it actually means: One traversal of the defined training dataset. In distributed or sampled training, the exact implementation of an epoch depends on the data loader and sampling policy.
• Common confusion: More epochs do not guarantee better generalization; evaluate on held-out data.
• Related terms: Batch Size, Overfitting, Eval Set

Error Budget

• Category: Reliability & operations
• What it actually means: The amount of unsuccessful service allowed by a service-level objective over its measurement window before the objective is exhausted.
• Why it matters: It gives reliability and product work a shared decision boundary: teams can spend remaining budget on change while slowing risk when user-visible failure consumes it.
• In practice: Derive the budget from the SLO, track burn by cause and user segment, define release actions before exhaustion, and avoid resetting the accounting after an incident.
• Common confusion: An error budget is not a quota for causing incidents. It is an operating policy derived from a user-facing reliability target.
• Related terms: Service Level Objective (SLO), Service Level Indicator (SLI), Availability, Incident Response
• Sources: Google SRE Workbook: Error Budget Policy

Eval Set

• Category: Evaluation & safety
• Aliases: Evaluation set
• What it actually means: A versioned collection of inputs, expected properties, scoring rules, and metadata used to measure an AI system against a defined capability or risk.
• Why it matters: A repeatable set turns vague quality claims into comparable evidence and catches regressions after prompts, models, tools, or retrieval change.
• In practice: Keep representative support questions, adversarial instructions, expected citations, and failure labels in a reviewed dataset that is separate from development examples.
• Common confusion: A development eval guides iteration, a final held-out test estimates performance after choices are fixed, and a standardized benchmark supports comparison under a shared protocol. Repeated tuning against any held-out set leaks test information and inflates results.
• Learn it: Eval-Driven Agent Development
• Related terms: Evaluation (Eval), Regression Test, LLM-as-a-Judge, Verification Gate

Evaluation (Eval)

• Category: Evaluation & safety
• Aliases: Eval
• What it actually means: A defined process for measuring model or system behavior on representative tasks using explicit success criteria, data, scorers, and review procedures.
• Why it matters: You cannot improve reliability if success is only a subjective impression from a few demos.
• In practice: Run the same customer-support scenarios before and after changing retrieval, score correctness and citation support, and inspect failures by category.
• Common confusion: A benchmark score is one evaluation result, not a complete account of production quality.
• Learn it: LLM Evaluation
• Related terms: Eval Set, LLM-as-a-Judge, Cost per Successful Task, Regression Test

Exact Match (EM)

• Category: Evaluation & safety
• What it actually means: A metric that counts an output as correct only when its normalized representation exactly equals an accepted reference answer.
• Why it matters: It is deterministic and easy to audit for tasks with one canonical answer, but it exposes no partial credit.
• In practice: Define normalization and all accepted references before evaluation, then pair exact match with task-specific checks when several outputs can be valid.
• Common confusion: A low exact-match score can reflect harmless formatting differences, while a matching string can still be unsupported or unsafe in context.
• Related terms: ROUGE, Eval Set, Structured Output, Pass@k
• Sources: SQuAD

Expert Parallelism

• Category: Infrastructure & serving
• What it actually means: Distributing mixture-of-experts subnetworks across devices and routing each token's activations to the devices that host its selected experts.
• Why it matters: Sparse experts increase model capacity without executing every expert for every token, but routing introduces communication, load-balance, and placement constraints.
• In practice: Measure token distribution by expert, provision communication bandwidth, cap or route overflow deliberately, and test quality when traffic produces uneven expert demand.
• Common confusion: Expert parallelism partitions experts selected by a router. Tensor parallelism partitions the tensor operations inside layers.
• Learn it: Mixture of Experts
• Related terms: MoE (Mixture of Experts), Tensor Parallelism, Pipeline Parallelism, Model Serving
• Sources: GShard