B

Backpressure

• Category: AI-native development
• What it actually means: A flow-control mechanism that slows or rejects upstream work when a downstream component cannot process it safely at the current rate.
• Why it matters: Without backpressure, queued agent runs, tool calls, or streamed events can exhaust memory, exceed rate limits, and amplify retries.
• In practice: When the evaluator queue reaches its limit, pause new agent jobs or return a retryable response instead of accepting unbounded work.
• Common confusion: Backpressure protects capacity before failure. A circuit breaker stops calls after failures show a dependency is unhealthy.
• Related terms: Rate Limit, Retry with Backoff, Circuit Breaker

Backpropagation

• Category: Math & training
• What people say: How neural networks learn.
• What it actually means: An efficient application of the chain rule that propagates derivatives from a scalar loss backward through a computation graph. It computes gradients; an optimizer uses those gradients to update parameters.
• Common confusion: Backpropagation calculates gradients. It does not choose the update rule or learning rate.
• Why it's called that: Derivative information moves backward from the loss toward earlier operations.
• Learn it: Backpropagation from Scratch
• Related terms: Autograd, Gradient, Optimizer

Batch Size

• Category: Math & training
• What people say: How many examples are processed at once.
• What it actually means: The number of examples whose losses contribute to one gradient estimate before an optimizer update. Larger batches can improve hardware utilization and reduce gradient noise, but they require more memory and may need different learning-rate or scheduling choices.
• Common confusion: There is no universal batch-size range or rule that says every batch increase should produce the same learning-rate increase.
• Related terms: Learning Rate, Gradient, Optimizer

Benchmark Contamination

• Category: Evaluation & safety
• What it actually means: Overlap or information leakage between evaluation examples and data used to pretrain, tune, prompt, select, or otherwise improve the evaluated system.
• Why it matters: Contamination can make a benchmark score reflect prior exposure rather than the ability to generalize to unseen tasks.
• In practice: Track dataset provenance, search training sources for exact and near duplicates, hold back private test cases, and refresh public evals with newly authored examples.
• Common confusion: Contamination is broader than exact copying. Paraphrases, answer keys, benchmark metadata, and repeated prompt tuning can also leak evaluation information.
• Related terms: Data Leakage, Data Deduplication, Eval Set, Exact Match (EM)
• Sources: Investigating Data Contamination in Modern Benchmarks for Large Language Models

BM25

• Category: Retrieval & generation
• What it actually means: A lexical ranking function that scores a document from query-term matches while accounting for term rarity, repeated occurrences, and document length.
• Why it matters: It is a strong exact-term retrieval baseline and complements dense retrieval for identifiers, rare words, and domain-specific phrases.
• In practice: Retrieve candidates with BM25 and dense search, combine their ranks, then evaluate the merged results before adding a more expensive reranker.
• Common confusion: BM25 does not understand semantic similarity directly, and its score has no universal meaning across different queries or index configurations.
• Related terms: Hybrid Retrieval, Dense Retrieval, Reranker, RAG (Retrieval-Augmented Generation)
• Sources: The Probabilistic Relevance Framework: BM25 and Beyond

Byte Pair Encoding (BPE)

• Category: Data & representations
• What it actually means: A subword-tokenization method that repeatedly merges frequent adjacent units to construct a fixed vocabulary from training text.
• Why it matters: It balances vocabulary size with the ability to represent rare or unseen words as smaller units.
• In practice: Train the tokenizer only on approved corpus splits, version its merge rules with the model, and inspect how it segments code, multilingual text, and whitespace.
• Common confusion: BPE is one tokenizer family, not a universal description of how every model creates tokens.
• Related terms: Tokenization, Vocabulary, Token, Embedding
• Sources: Neural Machine Translation of Rare Words with Subword Units