P

Paged KV Cache

• Category: Infrastructure & serving
• What it actually means: A KV-cache memory manager that stores attention state in fixed-size blocks and maps logical sequence positions to physical blocks instead of requiring one contiguous allocation per sequence.
• Why it matters: Variable sequence lengths create fragmentation and unpredictable growth, so block-based allocation can improve usable memory and enable flexible sharing.
• In practice: Select block size from workload measurements, track allocation and eviction, isolate state between requests, and test cancellation and prefix sharing under memory pressure.
• Common confusion: Paged KV cache manages runtime attention-state memory. It does not move model parameters to disk or extend the model's trained context limit.
• Learn it: vLLM Serving Internals
• Related terms: KV Cache, Prefix Caching, Context Window, Model Serving
• Sources: Efficient Memory Management for Large Language Model Serving with PagedAttention

Parameter

• Category: Models & inference
• What people say: A number used to describe model size.
• What it actually means: A value learned during training, commonly a weight, bias, embedding element, or normalization parameter. Parameter count is one measure of model capacity, but it does not directly determine quality, memory, or serving cost.
• Common confusion: Memory per parameter depends on numerical format, quantization metadata, sharding, optimizer state, activations, and runtime overhead.
• Related terms: Weight, MoE (Mixture of Experts), Quantization

Pass@k

• Category: Evaluation & safety
• What it actually means: Across a task set, the fraction of tasks for which at least one of k sampled candidates passes a defined correctness test.
• Why it matters: It measures the value of sampling several attempts for tasks such as code generation where an automatic verifier can check each candidate.
• In practice: Generate candidates independently under a fixed configuration, run the same isolated tests on each, and report k with the sampling and estimator details.
• Common confusion: Pass@k is not single-attempt accuracy, and a higher score can reflect a larger attempt budget rather than a better first answer.
• Related terms: Coding Agent, Regression Test, Eval Set, Test Oracle
• Sources: Evaluating Large Language Models Trained on Code

Patch

• Category: AI-native development
• What it actually means: A reviewable representation of changes to one or more files, usually expressed as additions and deletions against a known base revision.
• Why it matters: A patch gives people and agents a narrow artifact to inspect, test, apply, or reject without accepting an entire working directory.
• In practice: Ask a coding agent to return a unified diff, then verify that it touches only allowed files and applies cleanly to the expected commit.
• Common confusion: A patch captures file changes, not the reasoning, test evidence, or approval needed to ship them.
• Learn it: Workbench for Real Repositories
• Related terms: Coding Agent, Worktree, Scope Contract, Regression Test

Patch Embedding

• Category: Multimodal systems
• What it actually means: A learned projection that converts an image patch into a fixed-width vector used as one element of a transformer input sequence.
• Why it matters: It creates the interface between a spatial image grid and a sequence model, with patch size controlling token count and retained local detail.
• In practice: Record patch and image dimensions, handle padding or resizing explicitly, add position information, and measure how resolution changes affect both accuracy and token cost.
• Common confusion: A patch embedding is the vector representation of a patch, not a semantic object detector or a guarantee that patch boundaries match visual entities.
• Learn it: Vision Transformer Patch Tokens
• Related terms: Vision Transformer (ViT), Image Token, Embedding, Token
• Sources: An Image is Worth 16x16 Words

Perplexity

• Category: Models & inference
• What people say: How surprised a language model is by a dataset.
• What it actually means: The exponentiated average negative log-likelihood under a stated tokenization and logarithm convention. Lower values mean the model assigned higher probability to the evaluated sequence.
• Common confusion: Perplexity is not comparable across different tokenizers or evaluation setups and does not directly measure factuality or usefulness.
• Related terms: Cross-Entropy, Token, Evaluation (Eval)

Pipeline Parallelism

• Category: Infrastructure & serving
• What it actually means: Partitioning sequential groups of model layers across devices and moving microbatches or requests through those stages as a pipeline.
• Why it matters: It lets models exceed one device's memory, but stage imbalance, pipeline bubbles, activation transfers, and failure coordination affect usable performance.
• In practice: Balance stage cost, choose a microbatch schedule, measure idle time and interconnect traffic, and keep model and checkpoint partition metadata versioned.
• Common confusion: Pipeline parallelism divides layers by depth. Tensor parallelism divides tensor operations within a layer.
• Learn it: Scaling and Distributed Training
• Related terms: Tensor Parallelism, Expert Parallelism, Batch Size, Model Serving
• Sources: GPipe

Planning

• Category: Agents & tools
• What it actually means: Constructing, selecting, or revising a sequence of actions and dependencies intended to move from the current state to a goal.
• Why it matters: Explicit plans make assumptions and ordering visible before an agent commits to expensive or irreversible actions.
• In practice: Ask for a short dependency-aware plan, validate it against available tools and permissions, then re-plan when observations invalidate an assumption.
• Common confusion: A generated plan is a proposal, not proof that the steps are feasible, sufficient, or safe.
• Related terms: Agent State, ReAct, Orchestration, Verification Gate
• Sources: LLM+P

Postmortem

• Category: Reliability & operations
• What it actually means: A durable incident record that explains impact, detection, response, contributing conditions, recovery, and owned follow-up actions without assigning blame as a substitute for analysis.
• Why it matters: A resolved outage still has value as evidence. Capturing system conditions and decisions turns one event into improvements that reduce recurrence and response time.
• In practice: Build the timeline from traces and logs, distinguish triggering events from contributing conditions, assign dated actions, and review whether each action changed the relevant control.
• Common confusion: A postmortem is not a meeting transcript or a search for one person's mistake. It should produce testable system improvements.
• Related terms: Incident Response, Regression Test, Audit Log, Observability
• Sources: Google SRE: Postmortem Culture

Precision & Recall

• Category: Evaluation & safety
• What people say: Two metrics for classification or retrieval quality.
• What it actually means: Precision asks how many flagged items were correct; recall asks how many relevant items were found. When you change the decision threshold for one fixed scoring model, improving recall often lowers precision and vice versa. A better model can improve both. F1 is their harmonic mean.
• Common confusion: The right threshold and metric depend on the cost of each error and the prevalence of the target class.
• Related terms: Eval Set, Semantic Search, Guardrails

Prefill

• Category: Infrastructure & serving
• Aliases: Prefill Phase
• What it actually means: The initial inference stage that processes all supplied input tokens to produce their representations and the attention state required for subsequent autoregressive generation.
• Why it matters: Prompt shape, queueing, and cache reuse affect prefill cost, and prefill competes differently for compute than decode, so it strongly influences startup latency and serving schedules.
• In practice: Record prompt tokens and prefill latency, separate queue time from execution time, compare cached and uncached prefixes, and test long prompts beside active decode traffic.
• Common confusion: Prefill is the runtime prompt-processing stage, not the first generated token itself. The first token appears only after prefill and any queueing complete.
• Learn it: Disaggregated Prefill and Decode
• Related terms: Decode Phase, KV Cache, Time to First Token (TTFT), Chunked Prefill
• Sources: Sarathi-Serve; DistServe

Prefix Caching

• Category: Infrastructure & serving
• What it actually means: Reusing KV-cache blocks produced for an identical eligible token prefix across requests so the serving runtime can skip repeated prefix computation.
• Why it matters: Shared system instructions, templates, or documents can consume substantial prefill work, but reuse only helps when token sequences and cache eligibility match.
• In practice: Place stable tokens before request-specific content, include model and tokenizer versions in cache identity, isolate tenant-sensitive state, monitor hit rate, and treat eviction as normal.
• Common confusion: Prefix caching reuses runtime attention state for exact token prefixes. Prompt caching is a broader provider or application contract, while semantic caching reuses a prior result for a similar request.
• Learn it: Inference Optimization
• Related terms: Prompt Cache, Semantic Cache, KV Cache, Paged KV Cache
• Sources: SGLang

Progressive Disclosure

• Category: AI-native development
• What it actually means: Supplying a person or model with the minimum useful context first, then revealing deeper detail when the task or evidence requires it.
• Why it matters: It limits context noise and cost while keeping authoritative detail available on demand.
• In practice: Give a coding agent repository rules and a map first; load full implementation files only after it identifies the relevant module.
• Common confusion: Progressive disclosure is staged access to detail, not deliberate withholding of information required for a decision.
• Learn it: Workbench for Real Repositories
• Related terms: Context Engineering, Repository Map, Token Budget, Handoff

Prompt Cache

• Category: Prompting & context
• What it actually means: Reuse of provider-side or application-side computation for an identical or eligible prompt prefix so repeated inference avoids some preprocessing work.
• Why it matters: Stable instructions and large shared documents can become cheaper or faster across repeated calls when the provider's cache contract is satisfied.
• In practice: Place stable policy text before request-specific content, monitor cache-hit metadata, and treat misses as normal because eligibility and lifetime vary by provider.
• Common confusion: A prompt cache is a provider or application reuse contract and may use prefix caching internally. Prefix caching specifically reuses eligible exact-token KV state, while semantic caching reuses a prior result for a sufficiently similar request.
• Learn it: Prompt Caching
• Related terms: Semantic Cache, Prefix Caching, KV Cache, Time to First Token (TTFT)

Prompt Engineering

• Category: Prompting & context
• What people say: Wording instructions so a model follows the task.
• What it actually means: Designing model-facing instructions, examples, constraints, and output requirements to improve behavior on a defined task.
• Common confusion: Prompt wording cannot compensate for missing evidence, unsafe permissions, poor tool contracts, or absent evaluation.
• Learn it: Prompt Engineering
• Related terms: Context Engineering, Few-Shot, System Prompt, Structured Output

Prompt Injection

• Category: Evaluation & safety
• What people say: An adversarial instruction that redirects a model.
• What it actually means: An attack or failure mode in which untrusted content influences a model to disregard intended instructions, expose data, misuse tools, or take actions outside the user's goal. The content can arrive directly from a user or indirectly through retrieved pages, files, messages, or tool output.
• Why it matters: Models process instructions and data through the same language channel, so input filtering alone cannot reliably separate every malicious instruction from legitimate content.
• In practice: Treat external content as untrusted, isolate it from authority-bearing instructions, minimize tool permissions, require approval for consequential writes, and verify outputs and actions.
• Common confusion: Prompt injection is not technically the same mechanism as SQL injection, and a stronger system prompt is not a complete defense.
• Learn it: Prompt Injection Defense
• Sources: OWASP prompt injection guidance
• Related terms: Least Privilege, Sandbox, Approval Gate, Tool Contract

Prompt Sensitivity

• Category: Prompting & context
• What it actually means: Variation in model output or measured performance caused by changes to prompt wording, order, formatting, or examples that preserve the intended task.
• Why it matters: A system that succeeds under one convenient phrasing may be unreliable for real users or misleading in evaluation.
• In practice: Create semantically equivalent prompt variants, measure variance by case, and keep variants in regression tests instead of optimizing one prompt against one eval set.
• Common confusion: Sensitivity is not always a prompt defect; it can reveal ambiguity, weak model robustness, unstable decoding, or an inadequate scoring rule.
• Related terms: Prompt Engineering, Eval Set, Regression Test, Few-Shot
• Sources: ProSA

Provenance Attestation

• Category: Security & governance
• What it actually means: Authenticated, machine-readable metadata that binds an artifact to claims about how, where, when, and from which inputs it was produced.
• Why it matters: It lets automated policy and reviewers verify supply-chain claims instead of trusting an unsigned build note.
• In practice: Generate an attestation in the build system, bind it to artifact digests, sign it with a controlled identity, and verify it before release.
• Common confusion: A signature identifies the attester and protects integrity; it does not prove that every claim inside the attestation is true.
• Related terms: Data Provenance, Reproducible Build, Audit Log, Verification Gate
• Sources: SLSA Software Attestations

Purpose Limitation

• Category: Security & governance
• What it actually means: For personal data, collecting and using it only for specified, explicit purposes unless a new use has an appropriate compatible or authorized basis.
• Why it matters: Data that was acceptable for one workflow can create privacy and governance risk when silently reused for model training, evaluation, personalization, or unrelated analytics.
• In practice: Record the purpose with each dataset, check new pipelines against it before access, separate incompatible uses, and require a documented decision when the purpose changes.
• Common confusion: Purpose limitation governs why data is used. Data minimization governs how much data that purpose actually requires.
• Related terms: Data Minimization, Data Classification, AI Risk Assessment, Audit Log
• Sources: General Data Protection Regulation, Article 5(1)(b)